Validity and reliability of the Maslach Burnout Inventory-Student Survey in Sri Lanka

In the context of ever increasing educational expectations and demands having negative repercussions on mental wellbeing of student populations, exploration of the problem of burnout has become a timely need across the globe. However, defining burnout as a clinical entity has been a controversial issue throughout its course. Yet, the most widely used definition is the three-dimensional concept of burnout that was described by Maslach, Jackson, and Leiter [1].

The virtual use of the Maslach Burnout Inventory (MBI) at the budding stages of burnout research has led to the artefactual notion that burnout was exclusively found among the human services professionals [2]. The introduction of the Maslach Burnout Inventory-General Survey (MBI-GS) has paved the way to expand the horizons of burnout research outside the human services, as its dimensions are defined more generally and do not refer to working with recipients [3].

The concept of student burnout has been in the limelight with the introduction of the Maslach Burnout Inventory-Student Survey (MBI-SS) by Schaufeli et al. [2]. Though students are not employed in a work setting, the core structured and obligatory activities they are involved in, such as attending classes and finishing assignments, are targeted at the ultimate objective of passing examinations [4]. Hence, from a psychological perspective, their coercive core activities can be considered as ‘work’ [5].

In accordance with the original definition of burnout, Schaufeli et al. [2] have defined student burnout as, “a three-dimensional syndrome that is characterised by feelings of exhaustion due to the demands of studying, a cynical attitude of withdrawal and detachment, and reduced professional efficacy regarding academic requirements”. According to that definition [2], Exhaustion (EX) can be defined as feelings of strain, particularly chronic fatigue resulting from overtaxing work. Cynicism (CY) is manifested in an indifferent or a distal attitude toward work in general, a loss of interest in one’s work and not seeing it as meaningful. Reduced Professional Efficacy (PE) refers to diminished feelings of competence as well as less successful achievements and to lack of accomplishment in one’s work.

Though various study instruments have been used to assess burnout among student populations, the first reported literature pertaining to the invention of a specific measure to assess student burnout is the invention of MBI-SS by Schaufeli et al. [2]. Since then, MBI-SS has been cited as the most widely used research instrument to assess burnout in different student populations across the globe [6–8]. MBI-SS is one of the latest additions to the family of inventories of MBI. This inventory is a modified form of the MBI-GS. The MBI-SS, which is a self-administered questionnaire, consists of 16 items representing the three dimensions of student burnout. The three-factor conceptualisation of the MBI-SS has been confirmed in different student populations in different countries [2, 5, 9–12]. However, hitherto, there is no published literature pertaining to the validity of MBI-SS in the South Asian context.

Though not widely used as the MBI-SS, the School Burnout Inventory, which consists of three dimensions, developed by Salmela-Aro et al. [13], the two-factor Oldenburg Burnout Inventory student version developed by Campos et al. [14] and the Copenhagen Burnout Inventory-Student version developed by Campos et al. [15] have been used to assess the concept of student burnout.

Even though a plethora of research have been conducted among different student populations pertaining to burnout across the globe, the published literature on the topic in the South Asian context is scanty. In Sri Lanka, the period of general education comprises all grades from grade one to thirteen in the school system and the collegiate cycle in the education system consists of grade twelve and grade thirteen. At the end of the collegiate cycle, grade thirteen students sit for the General Certificate of Examination (GCE) Advanced Level, which is the national level selection examination for state university admissions. Studies conducted on assessing mental health issues among Sri Lankan collegiate cycle students reveal that the prevalence of mental health problems such as depression and anxiety are high and further evidence suggests that symptoms are mainly attributable to examination induced stress [16]. In addition, the findings of a national survey revealed that nearly one in five adolescents in schools appear to have clinically relevant mental health problems [17] and approximately one third of adolescents had indicated that they felt pressurized due to the parents’ and teachers’ expectations of higher academic performance [18]. Against the backdrop of high prevalence of mental health problems in students, exploring the concept of student burnout is extremely important and a timely, as burnout directly assesses the psychological well-being in relation to academic endeavours. However, owing to the absence of a validated instrument to assess burnout in the Sri Lankan context, this important research area is not widely explored.

In this background, the present study was designed to assess the construct validity and reliability of the MBI-SS among collegiate cycle students and to explore the applicability of the three-factor model of the MBI-SS in the Sri Lankan context.

The concept of student burnout has been explored across different student populations representing varying educational contexts [2, 5, 9–12]. However, the novelty of the concept and the absence of a proper assessment tool have hindered the exploration of the concept in many of the South Asian countries including Sri Lanka.

The present study was designed with the objective of validating the Sinhala version of the MBI-SS among Sri Lankan collegiate cycle students. Hence, a cross sectional design deemed appropriate for this purpose. Review of literature in relation to student burnout has demonstrated that the concept of burnout shows heterogeneity across different educational contexts. Hence, it is pertinent to select a specific student population to whom a common educational context is applicable. Taking this issue into consideration, the students in the collegiate cycle, who were studying in grade thirteen, were selected to minimise the heterogeneity in relation to their academic endeavours. In addition, the study sample was selected to represent both male and female students studying in all four subject streams. Three Sinhala medium government schools were selected considering the logistic feasibility to conduct clinical interviews by the Consultant Psychiatrist, the ease of accruing a relatively large number of students on a given date of data collection and having satisfactory infrastructure facilities in the schools to arrange suitable places for data collection and conducting clinical interviews. The study sample was similar to the national statistics related to the sex distribution and the sample distribution pattern with regard to subject streams was not very different from that of the country profile, with Arts, Science and Commerce being the main subject streams and a relatively small percentage of students studying in the Technology subject stream. Furthermore, as mentioned above, the sample size of the study deemed adequate to conduct a validation study [22].

The forward-backward translation method, which is a widely accepted method for cross-cultural adaptation of study instruments [19–21], was employed in the translation of the study instrument. During the process, particular emphasis was given to ensure semantic equivalence, conceptual equivalence and normative equivalence. This was achieved by conducting this process in conjunction with language experts and the technical experts. Furthermore, a multi-disciplinary panel of experts representing many important fields related to student burnout has assessed the judgmental validity of the questionnaire. Expect for the item 13, all the other items were found to have high median rating scores. Item 13 (“I just want to get my work done and not be bothered”) was identified as ambivalent. Though it is an item that reflects a negative attitude, it could also be interpreted as a positive attitude by those who would like to successfully complete academic endeavors without making them to bother their lives. Similarly, this item was identified as having poor psychometric properties by some other researchers largely owing to its ambivalent nature [2, 5, 25, 28]. The studies that have used 15-item MBI-SS, have also omitted this specific item from the study instrument [11, 29]. However, the reasons for the omission have not been stated.

Multi-trait scaling analysis was used to assess the hypothesised scale structure of the MBI-SS as the primary step in analysing whether the set of items in MBI-SS can be appropriately combined into summated rating scales [23]. In multi-trait scaling analysis, except for the item 13, item-convergent validity and item-discriminant validity were confirmed for other 15 items in the MBI-SS. The ambivalent nature of the item 13 may have resulted in not having satisfactory item-convergent and item-discriminant validity in multi-trait scaling analysis.

CFA is considered as a viable method of assessing the construct validity of study instruments. CFA necessitates a strong priori theory underlying the measurement model before analysing data [30]. Additionally, CFA is often used in data analysis to examine the expected level of causal connections between variables [31]. Since the tri-dimensional structure of the MBI-SS has been widely established in literature, CFA was employed to assess the construct validity of the MBI-SS in the present study. Since there is no consensus as to what category of model fit indices are to be used in assessing the model fit, a combination of absolute fit indices, relative fit indices, and parsimony fit indices were used in the present study for that purpose [32].

The results of the CFA revealed that the three-factor model fitted the data set better than the one-factor and the two-factor models. This finding is congruent with the findings of several other researchers who have tested the CFA of one-factor model and the three-factor model [1]. Analysis revealed that, the values for all absolute fit indices, relative fit indices, and parsimony fit indices of three-factor model were better than those of one-factor and two-factor models. However, among the absolute fit indices, only SRMR had reached the stipulated cut-off value (< 0.05).

Since the fit indices of the three-factor model showed room for further improvement, specification search for the three-factor model was carried out. To overcome the psychometric limits of the MBI, several procedures have been highlighted in the literature. These methods include, allowing correlated error terms, allowing items to load on more than one factor, eliminating items, and increasing or decreasing the number of factors [33]. In the present study, the specification search was carried out considering the psychometric properties evaluated for the questionnaire items in previous validity assessment methods. In relation to that, since item 13 had received low median rating scores in assessing the judgmental validity and since the item had unsatisfactory results at multi-trait scaling analysis, a modified three-factor model with item 13 deleted was tested in CFA. Furthermore, the suggestions for modifications in relation to adding correlated error terms offered by LISREL were considered in the specification search.

The modified three-factor model with item 13 deleted proved to be a better model fit to the data in comparison to the original three-factor model. Additionally, the modified three-factor models with addition of correlated error terms had revealed superior fit to data in comparison with the original three-factor model. The possible reasons behind this phenomenon are, the existence of random measurement errors or unmeasured variables underlying the items. However, this improvement in model fit is at the expense of lack of generalisability of the findings. According to MacCallum et al. [24], the specification search process is inherently susceptible to capitalisation on chance, owing to the potential role of idiosyncratic characteristics of the sample influencing the particular modifications.

Even though the modified three-factor models with addition of correlated error terms had revealed superior fit to data in comparison with the original three-factor model, they had not shown substantial improvement in the model fit when compared with the modified three-factor model with item 13 deleted. Considering the fact that no model fits real-world phenomena exactly and the problems encountered with addition of correlated error terms to the model, the three-factor model with item 13 deleted was considered as an acceptable model, which fits the data. This conclusion is substantiated with having a combination of fit indices representing all the three categories, which reached desired threshold values (RMSEA = 0.080, SRMR = 0.0470, CFI = 0.978, NNFI = 0.973, PGFI = 0.630, PNFI = 0.798). This finding is consistent with other studies conducted to assess the validity of the MBI-SS [2, 5, 34]. Even though, few other MBI-SS validation studies had revealed that 15-item MBI-SS showed acceptable fit to data, they have not specified which item had been deleted from the original 16-item version of the MBI-SS [35, 36].

Assessment of reliability of the Sinhala version of the MBI-SS was done by assessing the internal consistency and the test-retest reliability. Internal consistency revealed that, all three subscales of burnout were having high Cronbach’s α coefficient values. This finding is consistent with findings of other studies conducted across the globe, involving different language versions of the MBI-SS. The internal consistency was revealed as high in Portuguese version [2, 11, 29], Spanish version [2], Dutch version [2], Chinese version [5], Turkish version [34–36], and Persian version [37] of the MBI-SS.

The present study revealed that the test-retest correlation coefficients were high for each of the three subscales of the Sinhala version of the MBI-SS. The study conducted by Kutsal and Bilge [34] revealed very high coefficient values for all three subscales in a sample of Turkish high school students after a gap of three weeks.

Given that the results generated from factor analysis are often sample specific [38], generalisability of the present study findings to other populations should be done with caution, considering the variations in educational and cultural contexts. Furthermore, selecting the three Sinhala medium government schools considering the logistic feasibility is another study limitation, which affects the generalisability of the study findings. Even though the total sample size was adequate to conduct factor analysis, it is important to note the relatively small number of study participants in different subject streams, which highlights the need for future studies involving multi-group analyses.

The present study confirms the three dimensional structure of the student burnout concept. The Sinhala version of the 15-item MBI-SS is a valid and a reliable instrument to assess the burnout status among collegiate cycle students in Sri Lanka.

The Sinhala version of the 15-item MBI-SS, due to its brevity, relative ease of administration, and sound psychometric properties, could be used as an effective screening tool for the assessment of burnout at the school level. It will allow identification of the affected students at early stages, which is important in effective secondary prevention, and identification of vulnerable students, which is imperative for the primary prevention.

Moreover, given that the three-factor structure of the MBI-SS has been established in the Sri Lankan context strengthening the evidence base for the relevance and the applicability of the concept of student burnout in the South Asian context, future research could be conducted involving different South Asian student populations.