Study design and setting
This school-based, cross-sectional validation study was conducted in the Kurunegala district, North Western province, Sri Lanka. The study was conducted from May 2014 to April 2015 in three Sinhala medium government schools in the Kurunegala district. All these three schools have students studying in all four collegiate cycle subject streams, viz., Science, Arts, Commerce, and Technology.
Participants
Three classes each were selected from the three selected schools and this selection represented both male and female students studying in all four subject streams. The total number of students participated in the study was 194 and the response rate was 100.0%. The majority of the participants were females (n = 107, 55.2%). The mean age of the sample was 18.3 years (SD = 0.43 years). The number of students in the Science, Arts, Commerce, and the Technology streams were 78 (40.2%), 60 (30.9%), 41 (21.2%), and 15 (7.7%) respectively.
Measures
In the 16-item MBI-SS, which is a self-administered questionnaire, five items are targeted at identifying EX, five items are targeted at identifying CY, and six items are targeted at identifying PE. A seven-point rating scale is used to assess the frequency in which the respondents experience feelings related to each dimension and this rating scale ranges from 0 (never) to 6 (every day). According to the scores of each dimension, the high scores on EX and CY and low scores on PE are indicative of burnout.
The forward-backward translation method was used to translate the 16-item MBI-SS to Sinhala. This forward-backward translation method is a widely accepted method for cross-cultural adaptation of study instruments [19,20,21]. The method included, forward translation, backward translation, and pre-testing and cognitive interviewing. Two bilingual translators, who are fluent in Sinhala and English, independently translated the questionnaire into Sinhala while ensuring semantic equivalence, conceptual equivalence, and normative equivalence. To produce a synthesis of the two forward translations, an independent reviewer, who is fluent in both languages, reviewed both translations together with the original English version. Any discrepancies and ambiguities between the translated versions and any deficiencies compared to the original English version were resolved by consensus. The synthesised forward translated version was agreed upon for the backward translation. Two sworn language translators, who were totally blind to the original English version of the MBI-SS, independently translated the synthesised forward translation of MBI-SS back into English, without referring to the original version. Pre-testing of the synthesised forward translation of the MBI-SS was conducted among a sample of 25 grade thirteen students who were studying in schools outside the study setting. This sample consitsted of both male and female students studying in all four Advanced Level subject streams.
Face, content, and the consensual validity were assessed in order to appraise the judgemental validity of the questionnaire. A multi-disciplinary panel of experts representing the fields of psychiatry, psychology, public health, teaching, student counseling, and medical education assessed the consensual validity of the MBI-SS Sinhala version. The expert panel assessed each item of the questionnaire on its relevance in assessing burnout among grade thirteen students, appropriateness of the wording used, and acceptability in the local context for assessing burnout among grade thirteen students by using a rating scale of 0 to10, in which 0 being strong disagreement and 10 being strong agreement. In addition to rating of each item, the panelists were asked to make additional remarks related to the phrasing of items. Except for the item 13 stating, “I just want to get my work done and not be bothered”, all other items had a median score more than 7 for all the aspects. Based on the compiled rating scores and the comments, it was decided to include all 16 items in the synthesised forward translation of MBI-SS with suggested modifications, to be considered for confirmatory factor analysis (CFA).
Procedure
Ethical approval for this study was obtained from the Ethics Review Committee of the Faculty of Medicine and Allied Sciences, Rajarata University of Sri Lanka (Reference no: ERC/2014/057). Administrative clearance for the study was obtained from the Provincial Director of Education, North Western province, and the principals of the selected three schools. Data collection was done according to the logistic convenience of the schools to minimise the disturbance to the routine academic and other endeavours. Prior to data collection, informed written consent was obtained from all the participants and each participant was given the Sinhala version of MBI-SS to be filled independently. Confidentiality of the data collected and the anonymity of the participants were maintained. To assess the test-retest reliability of the study instrument, two weeks after the initial date of data collection, the same questionnaire was re-administered to students in a grade thirteen class who were included in the initial data collection.
Data analysis
Multi-trait scaling analysis and CFA were carried out on the scores obtained from the study participants to assess the construct validity of the MBI-SS. In relation to the scores of the data set, as low scores on PE subscale are indicative of burnout, reversed PE (rPE) scores were used for further statistical analysis.
Prior to performing statistical analyses, the suitability of the data set was assessed for any violations of assumptions demanded by the analytical techniques and the dataset did not violate the assumptions related to the level of measurement, related pairs, independence of observations, normality (using histograms and standardised skewness and kurtosis values), linearity (using bivariate scatter plots), outliers, and multicollinearity. Since the sample size was 194 and there were 16 observed variables, the ratio of observations to variables was approximately 12.1:1; hence, the sample size was adequate to conduct the analysis [22].
Multi-trait scaling analysis
Multi-trait scaling analysis was conducted using the SPSS version 17.0. Item-scale correlations were analysed and item-convergent and item-discriminant validity were assessed. In assessing item-convergent validity, a stringent criterion of correlation of 0.40 or greater between an item and its own subscale was considered as a success for assessing [23]. Items which correlated significantly higher (more than 1.96 standard errors) with its own subscale than with the other two subscales were considered as scaling successes in assessing item-discriminant validity.
CFA
In relation to factorial validity, as the three-factor model of MBI-SS is well established and substantiated by numerous research findings, CFA was employed to assess the extent to which underlying three-factor model was replicated in the observed data using the analytic software Linear Structural Relations (LISREL) version 9.1. The structure of the MBI-SS was evaluated based on a variety of fit indices, including absolute fit indices, relative fit indices and parsimony fit indices. Satorra-Bentler scaled chi-square test, Root Mean Square Error of Approximation (RMSEA), Goodness-of-Fit Index (GFI), Adjusted Goodness-of-Fit Index (AGFI), and Standardised Root Mean Square Residual (SRMR) were used as the absolute fit indices. Comparative Fit Index (CFI) and Non-Normed Fit Index (NNFI) were used as the relative fit indices, while Parsimony Goodness-of-Fit Index (PGFI) and Parsimonious Normed Fit Index (PNFI) were used as the parsimony fit indices.
The analysis was conducted in two steps. In the first step, following models were assessed.
-
a)
One-factor model: All 16 items of MBI-SS were loaded on to one latent factor.
-
b)
Two-factor model: Items measuring EX (five items) and measuring CY (five items) were loaded on to a single latent factor and items measuring rPE (six items) were loaded on to a different latent factor.
-
c)
Three-factor model: Items measuring EX, CY, and rPE were loaded on to three separate latent factors.
In the second step, specification search for the three-factor model was carried out considering the psychometric properties evaluated for the items in previous validity assessment methods, changes made to the three-factor model in the previous studies, and also the suggestions for modifications offered by LISREL analysis. In this step, modified three models of the original three-factor model were compared with each other.
-
a)
Model 1: Owing to the complexity of covariance structure models and correlational data, it is likely that model modifications would substantially improve the fit of the model to the data [24]. Hence, six correlated error terms were added to the three-factor model as per the suggestions for modifications offered by LISREL analysis.
-
b)
Model 2: Previous studies regarding the factorial validity of MBI-SS have removed the item 13, as it was found to be ambivalent and thus unsound [2, 25]. In appraising consensual validity of the items of MBI-SS, this item received low median rating scores by the multi-disciplinary panel of experts. Furthermore, item 13 did not yield a scaling success at item-discriminant validity in multi-trait scaling analysis. Hence, it was decided to delete this item from MBI-SS and the modified model was evaluated in CFA.
-
c)
Model 3: Though the model evaluated in the previous step yielded improvement in several fit indices, it was decided to incorporate six correlated error terms as per the suggestions for modifications offered by LISREL analysis.
Assessment of reliability
In order to assess the reliability or the consistency of information gathered by the Sinhala version of MBI-SS, two methods, viz., internal consistency and test-retest reliability were employed.
Test-retest reliability was assessed by administering the Sinhala version of MBI-SS after a gap of two weeks in a sub-sample of participants enrolled in the study.