Skip to main content
Download PDF

Validation of the Norwegian versions of the Implementation Leadership Scale (ILS) and Multifactor Leadership Questionnaire (MLQ) in a mental health care setting

BMC Psychology volume 10, Article number: 25 (2022) Cite this article

Abstract

Background

The implementation of evidence-based practices (EBPs) is of crucial importance in health care institutions and requires effective management from leaders. However, there is a lack of assessment tools sufficient to evaluate the degree to which the employees´ rate how well their leaders are at implementing EBPs. This emphasises the need for validated and widely used scales relevant for EBPs.

Methods

The current study evaluated the psychometric properties of the Multifactor Leadership Questionnaire (MLQ) and Implementation Leadership Scale (ILS) in a Norwegian mental health care setting.

Results

Results from confirmatory factor analyses indicate that the MLQ and ILS are valid instruments for measuring general and implementation leadership. The scales demonstrate good convergent validity. In addition, attitudes towards EBPs did not seem to be associated with the ILS and MLQ, further supporting the applicability of the instruments.

Conclusions

The two scales demonstrate good psychometric properties in a Norwegian mental health care setting, which suggests that the MLQ and ILS are valid and reliable tools for measuring leadership in an implementation setting. More research is greatly needed to disentangle the link between perceived leadership and objective measures of successful implementation of EBPs.

Trial registration NSD 690,133, NSD 60,059/3/OOS.

Peer Review reports

Background

Effective leadership has been identified as an important factor associated with the successful implementation of evidence-based practices (EBPs) in mental health services [1,2,3]. The focus on implementing EBPs in the healthcare system started in the 1990’s [4]. Over the years, it has subsequently been shown that successfully implementing EBPs may lead to better and more effective health care services, with lower costs, higher job satisfaction among clinicians, and higher patient satisfaction [5,6,7]. Leaders are an important part of any implementation process, as they can influence the organisational climate at the workplace, cooperation between team members, and employee’s attitudes towards the EBP [8,9,10,11]. With the growing interest in the role that leadership plays in effective EBP implementation, there is a need for establishing reliable and valid measures to assess leadership behaviours that relate to successful implementation.

In the quest to discern the leader’s role in the implementation of EBPs, there was initially a focus on identifying general leadership behaviours associated with different implementation outcomes [11, 12], using leadership concepts such as the Full-Range Leadership (FRL; [13]) theory. FRL is one of the most widely used leadership theories. It describes different leadership behaviours such as transformational leadership, in which leaders motivate and encourage employees; transactional leadership, where the leader rewards and punishes employees based on performance; and non-leadership, where the leader has a more “hands off” approach and avoids making decisions [14, 15]. Several studies have shown a positive relation between transformational leadership and different implementation outcomes, such as employees’ attitudes towards EBPs [16], motivation [17], turnover intention [18], burnout [19], and overall improved performance at all levels of the workplace [20,21,22].

The Multifactor Leadership Questionnaire (MLQ) is based on the FLR leadership theory and is likely the most frequently used scale to measure leadership [23]. The scale has been psychometrically validated several times, showing acceptable scores (ranging from α = 0.78 to 0.94; [21,22,23,24]). The MLQ has also previously been validated in a Norwegian sample [24], where each subscale showed adequate psychometric properties (α = 0.62—0.84). However, the validations differ as a result of researchers altering the original factor structure by either combing or excluding certain factors or items [25, 27]. The original structure consists of nine subscales, where idealised influence was separated into behaviours and attributed charisma [31]. However, several researchers treat idealised influence as one factor [23, 25, 27], and the scale has accordingly been broadly used as an eight-factor scale, consisting of idealised influence, inspirational motivation, intellectual stimulation, individualised consideration, contingent reward, management-by-exception (active), management-by-exception (passive) and laissez-faire. However, although most use the eight- or nine-factor models [25, 28, 29], some have rearranged the subscales into categories different from what was originally proposed (i.e., two subscales measuring non-leadership; [26, 30]). These alterations could cause misunderstandings when using the MLQ in relation to implementation outcomes. The creators of the scale have recommended individually analysing each of the eight subscales of the MLQ, with the exception of the transformational leadership subscales, which can be combined [31]. The current study therefore considers the MLQ as consisting of eight subscales: four subscales measuring transformational leadership, three subscales measuring transactional leadership and one measuring non-leadership.

While interest in the relationship between these more general leadership concepts and successful EBP implementation was increasing, research on strategic leadership behaviours related to implementation was lacking. Building on the growing evidence base that general leadership behaviours relate to effective EBP implementation, scientists have turned their interest towards specific leadership behaviours that may be more proximally related to successful EBP implementation. Newer research has established that such specific, or strategic, as often termed, leadership behaviours may provide additional explanatory value in the investigation into how leadership relates to key implementation outcomes [8, 32, 33].

The focus on identifying strategic leadership behaviours for implementation coincides with a more general call within the implementation research field to develop simple, brief and psychometrically sound implementation measures [34, 35]. In 2014, Aarons, Ehrhart and Farahnak developed the Implementation Leadership Scale (ILS), drawing from a broad base of theory and research on implementation, leadership, and organisational climate [11]. Results from studies investigating the effect of strategic leadership behaviours (i.e., implementation leadership) have revealed that these promote organisational change [12]. This is consistent with findings which report that employee-ratings on the ILS correlate with factors considered to be important during the implementation of EBPs and their sustainment [36].

The ILS was initially developed in the U.S., and has been validated several times [37,38,39]. In investigations using both employee-ratings and leader self-ratings, the ILS has shown excellent psychometric properties in multiple sectors [11, 39,40,41,42]. The ILS contains four subscales, including proactive leadership, knowledgeable leadership, supportive leadership and perseverant leadership, and the suggested four-factor structure has been confirmed in all studies [5, 11, 37, 39]. Analysis of reliability has found internal consistency to be excellent (Cronbach’s α ranging from 0.92—0.98) [11, 39, 41]. Convergent validity has been investigated by correlating the ILS with the MLQ, finding moderate to high correlations (Pearson’s correlation ranging from 0.63 to 0.75) between the two leadership concepts [11]. Discriminant validity has been established by correlating the ILS to theoretically unrelated implementation concepts, such as the Evidence-based Practice Attitude Scale (EBPAS), finding zero to low correlations (Pearson’s correlations ranging from 0.05 to 0.4) [11]. Only two studies have investigated the psychometric properties of the ILS outside the U.S., doing so in China and Greece [5, 43]. Employee ratings have previously been used when investigating the psychometric properties in a U.S. context [11], and a similar investigation in a Norwegian context would provide further evidence for the relevance of the concept of implementation leadership.

Based on the abovementioned, it is clear that successful implementation of EBPs relies on effective leadership. To accurately and reliably measure elements of effective leadership important for EBP implementation, we need valid measures. The purpose of this study is to examine the psychometric properties of the Norwegian version of the MLQ and ILS. First, the factor structure and internal consistencies of the two scales will be explored. Secondly, we will examine the convergent and divergent validity of the MLQ and ILS. Based on previous findings regarding the ILS, we expect to find support for a four-factor model and high internal consistency for the total scale and all subscales. In addition, we anticipate that the ILS will have moderate to high correlations with the MLQ and subsequently low correlations with the EBPAS. In accordance with other studies, we expect to find support for an eight-factor model [44, 45], as well as similar results regarding convergent and divergent validity as hypothesised above.

Method

Procedure

The study took place as part of a national implementation of evidence-based treatment for post-traumatic stress disorder (PTSD) in Norwegian specialised mental health care clinics for adults (N = 25) and youth (N = 22) [46]. The data was collected in the context of the utilisation of the Leadership and Organisational Change for Implementation (LOCI) as an implementation strategy [46, 47]. Local health trusts were contacted via e-mail with an invitation to participate in the implementation project and research study. Participating clinics were included in the hybrid type II project [48] based on motivation and availability. Data was collected from leaders and therapists working at the local health trusts between 2018 and 2020. Participation was voluntary and informed consent was attained from all participants in the study.

Participants

Participants were 804 therapists working at child or adult mental health clinics in Norway. The final sample size was 795 after removing missing data. Close to half (46.2%) of the participants were psychologists; 75 percent were females, and the average age was 43.9 years. Participants rated their leaders (N = 47) by filling out questionnaires regarding leadership, work climate and other measures relevant for the intervention. The average age for leaders was 49.7, and 55 percent had a background in psychology (Table 1).

Table 1 Participant demographics
Full size table

Measures

Implementation Leadership Scale (ILS)

ILS is a 12-item measure addressing leadership support for the usage of EBP [11]. It covers four different implementation leadership dimensions: Proactive Leadership describes the degree to which the leader anticipates and addresses implementation challenges; Knowledgeable Leadership refers to the degree to which a leader has a deep understanding of EBP and implementation issues; Supportive Leadership measures the degree of the leader’s support of followers’ adoption and use of EBP; and Perseverant Leadership refers to the degree to which the leader is consistent, unwavering, and responsive to EBP implementation. It is scored from 0 (not at all) to 4 (to a very great extent). The total ILS score is created by computing the mean of the four subscales. ILS was translated to Norwegian by an independent research group at the Regional Center for Children and Adolescent Mental Health (RBUP). The third and fourth author completed an additional back-translation. Both the initial translation and back-translation were done in close collaboration with the developers of the scale. There were only small differences between the two translations, and minor adjustments were made to align the translations.

Multifactor Leadership Questionnaire (MLQ)

The MLQ is a 36-item questionnaire, measuring transformational and transactional leadership, as well as non-leadership [49]. Transformational leadership consists of four subscales. Inspirational Motivation measures how positive and motivated the leader is about the future, which may influence the employees’ feelings of motivation. Idealised Influence focuses on the leaders’ attributes, like perceived power, values and ideals, and underlines a collective sense of these mission and values [25]. Intellectual Stimulation refers to whether the leader introduces new methods of viewing issues and seeks different perspectives. Lastly, Individualised Consideration measures how well the leader considers individual needs and helps the employees develop their strengths.

Transactional leadership consists of three subscales. Contingent Reward is a leadership behaviour that focuses on clear, defined requirements, and rewards desired outcomes through economical or emotional advantages. In both Management-by-exception active and passive, the leader provides corrective action when they notice behaviours that deviate from the norm. In the active subscale, the leader actively monitors actions and intervenes before the deviations start occurring, while in the passive, the leader waits until the deviations have occurred [50]. Lastly, non-leadership is measured by Laissez-faire, which assesses the absence of leadership. The MLQ is scored from 0 (Not at all) to 4 (Frequently, if not always). A Norwegian translation was used in the current study [24].

Evidence-based Practice Attitude Scale (EBPAS)

The EBPAS is a 15-item scale measuring mental health providers’ attitudes toward the adoption of evidence-based practices [51]. The scale has four subscales: Appeal, which describes the intuitive appeal of the practice; Openness, referring to openness to new practices; Requirement, the likelihood of adopting the EBP if required to do so; and Divergence, which refers to the new practice’s perceived divergence from usual practice. The EBPAS is scored on a 5-point Likert scale ranging from 0 (Not at all) to 4 (To a very great extent). The overall score is calculated by reversing the scores on the divergent subscale, and then averaging the items from all scales. A Norwegian translation was used in the current study [52]. The scale has shown adequate psychometric properties in a Norwegian sample of therapists working in mental health care services (α = 0.86; (56), α = 0.81; [53], and similar properties were found in the current study (15-items; α = 0.87, CI (95% bootstrapping based on 1000 samples) = 0.853– 0.884).

Statistical analyses

Internal consistency

Internal consistency was assessed by examining Cronbach’s alpha for all subscales and the total scale (Table 2).

Table 2 Summary statistics for the ILS total scale, subscales, and scale items and the MLQ subscales and scale items
Full size table

Confirmatory factor analysis

Confirmatory factor analysis was conducted using MPlus 8.3 [54]. Fit indices (standardised root mean square residual = SRMR, root mean-square error of approximation = RMSEA, comparative fit index = CFI and Tucker-Lewis index = TLI) were used as indicators of validity. We used recommended cut-offs that indicate a good fit for validation: CFI & TLI ≥ 0.90, SRMR < 0.08, RMSEA < 0.08 [55, 56]. The CFA was conducted using the weighted least square mean and variance estimation (WLSMV), which is ideal for categorical data. As clinicians working in the same clinic had the same leader, we controlled for the multilevel, nested data structure.

Convergent and discriminant validity

Discriminant and convergent validity were subsequently calculated by correlation analyses. To assess convergent validity, the MLQ and ILS were compared to each other. Divergent validity was measured by viewing the correlations between both aforementioned scales to the EBPAS subscale scores. This was done by using employee-ratings (clinicians working with patients) in specialised Norwegian mental health care services. The analyses were conducted using IBM SPSS Statistics for Windows, V.26 [57] and Rstudio [58].

Results

Reliability

The internal consistency of the ILS total scale (α = 0.96) and the four factors were excellent (ranging from α = 0.93–0.97; see Table 2). The internal consistency of Transformational Leadership was also excellent (α = 0.96). Individualised Consideration (a = 0.86), Intellectual Stimulation (a = 0.91), Inspirational Motivation (a = 0.84), Idealised Influence (a = 0.92), Contingent Reward (a = 0.86), Management-by-Exception Active (a = 0.89), Management-by-Exception Passive (a = 0.85), and Laissez-faire (a = 0.88) showed acceptable internal consistency.

Confirmatory factor analysis

Implementation leadership scale

As each subscale of the ILS is considered an indicator of overall implementation leadership, a higher order model was considered. The higher order CFA model demonstrated excellent fit (χ2 (48) = 112.575, p < 0.001; CFI = 0.999, TLI = 0.999; RMSEA = 0.043; SRMR = 0.010). Standardised factor loadings are displayed in Fig. 1, and all factor loadings were significant (p’s < 0.001). Nested data was controlled for by using the function cluster = and type = complex in Mplus and grouped by leaders (see Fig. 1).

Fig. 1
figure 1

Standardised factor loadings for the ILS. The parameters are presented as standardised path coefficients. The circular shapes represent the factors, or subscales. The square boxes below represent the items. Arrows pointing from the factors to the items are the first-order factor loadings. Arrows from the latent construct (ILS) to the subscales are the second-order factor loadings. Pers = Perseverant, Supp = Supportive, Knowl = Knowledgeable, Pro = Proactive

Full size image

Multifactor Leadership Questionnaire

The eight-factor model also showed excellent fit (χ2 (566) = 1891.317, p < 0.001; CFI = 0.968, TLI = 0.964; RMSEA = 0.056; SRMR = 0.050). Nested data was controlled for in the same way as aforementioned. (See Fig. 2). As the previous Norwegian version of the MLQ had been validated as a three-factor structure [24], this was also done in the current study. Results revealed an unacceptable model fit (χ2 (591) = 16,024.869, p < 0.001; CFI = 0.559, TLI = 0.530; RMSEA = 0182; SRMR = 0.182) according to the recommended cut-off indices. This suggests that the 8-factor version validated in the current study is more suitable.

Fig. 2
figure 2

Standardised factor loadings for the MLQ—The parameters are presented as standardised path coefficients. The circular shapes represent the factors, or subscales. The square boxes below represent the items. Arrows pointing from the factors to the items are the factor loadings. mlqlf = Laissez-faire, mlqmbep = Management-by-exception passive, mlqmbea = Management-by-exception active, mlqcr = Contingent Reward, mlqii = Idealised Influence, mlqim = Inspirational Motivation, mlqis = Intellectual Stimulation, mlqic = Individualised Consideration

Full size image

Convergent and discriminant validity

The ILS subscales had moderate to high correlations with the score of the MLQ Transformational Leadership subscale and the Contingent Reward subscale, which is consistent with previous findings [11]. Correlations ranged from 0.49 to 0.57, as shown in Fig. 3. As predicted, the ILS and MLQ both had low correlations with the EBPAS subscales, with correlations ranging from -0.17 to 0.23 (see Fig. 3).

Fig. 3
figure 3

Correlation matrix including the ILS, MLQ and EBPAS. White squares indicate p > .05. The bar on the far right indicates the correlation between the subscales. Blue colors indicate a positive correlation, while orange colors indicate a negative correlation

Full size image

Discussion

The aim of the current study was to investigate the psychometric properties of the Multifactor Leadership Questionnaire and Implementation Leadership Scale in a Norwegian clinical mental health care setting. Both general leadership [11] and implementation leadership [11] have been shown to be important for facilitating successful implementation. Results from the confirmatory factor analyses showed an excellent fit, supporting an eight-factor model for the MLQ and a four-factor model for the ILS. Regarding the ILS, the internal consistency was excellent for the total scale and for subscales. Results from the correlations indicate strong support for both convergent validity in regard to MLQ transformational leadership and discriminant validity in regard to EBPAS. Consistent with previous studies [11], the moderate to high correlations with the MLQ transformational leadership indicate that similar constructs are measured in both of the two scales, but not so similar that they would be considered identical. This suggests that the two scales can be used together to get an overall picture of behaviours important for effective implementation of EBPs. This is in line with theories suggesting that general and strategic leadership are different concepts, both influencing the implementation of EBPs [11]. Furthermore, the low correlations between the EBPAS and the ILS, and the EBPAS and the MLQ, support the hypothesis that they theoretically measure different constructs, which is in line with previous studies [11].

Research on the MLQ has varied in that researchers occasionally combine different subscales into one factor [27, 59], instead of assessing the subscales individually as done in the current study and according to recommendations from the developers of the scale [31]. Due to the acceptable values regarding internal consistency and expected results regarding convergent and discriminant validity, the findings indicate that the original eight-factor structure can be maintained. Furthermore, results indicate that each of the subscales constituting the scale measure different constructs within general leadership.

With measures such as the ILS and MLQ, we are able to assertively measure leadership behaviours assumed necessary to successfully implement changes that consequently improve patients’ well-being. Moreover, the validation of the MLQ is novel in regard to the factor structure. The previous validation in a Norwegian sample found a three-factor structure consisting of several subscales in each factor [24], while we found support for each subscale as individual factors. Previous research has suggested that the inconsistent findings regarding the MLQ subscales may occur due to heterogeneous samples of leaders from different cultural and professional backgrounds [28, 31]. The current sample is a quite homogenous group, consisting of Norwegian mental health service practitioners, mostly women aged 44. This supports previous findings regarding the individual factors of the MLQ [28]. Furthermore, findings from the current study add to the literature regarding the leadership and implementation of EBPs by including results from a Nordic sample. Overall, investigations into the psychometric properties of scales frequently used for measuring key implementation concepts, such as in this study, lays the foundation for gaining valid knowledge on the complex process of successfully implementing EBPs, which subsequently have large clinical implications as these factors influence patient outcomes [19].

The current study had a large sample size, spread across 43 clinics throughout the country. This variety is a clear strength, as it increases generalisability. A limitation of the current study was that data was only collected from participants in mental health clinics. We encourage future studies to investigate the psychometric properties of these scales in other sectors, and between different professions within these sectors. In addition, it would be interesting to compare results from different time points to examine the test–retest reliability. This could not be done in the current study as we expect the LOCI intervention to alter scores on the ILS and MLQ. Although results suggest that high scores on the transformational leadership and contingent reward subscales are positively correlated to factors of implementation leadership, it has not yet been established whether these leaders actually have been successful at implementing EBPs.

Conclusion

As research has established leadership as an important factor for successful implementation, there is a need for efficient measures that assess both general and strategic leadership behaviours. The current study demonstrates that the Norwegian versions of the Multifactor Leadership Questionnaire and Implementation Leadership Scale are valid and reliable instruments for measuring general leadership and leadership in the context of EBP implementation, respectively. Results show that the subscales of transformational leadership and contingent reward correlate with all subscales of the ILS. Divergent validity analysis shows that the EBPAS is a theoretically different construct compared to the MLQ and ILS. More research is needed to further understand how different leadership behaviours, both general and strategic, relate to the successful implementation of EBPs.

Availability of data and materials

The dataset used in the current study is available from the corresponding author upon reasonable request.

Abbreviations

EBP:

Evidence-based practice

ILS:

Implementation Leadership Scale

MLQ:

Multifactor Leadership Questionnaire

EBPAS:

Evidence-based Practice Attitudes Scale

PTSD:

Post-traumatic stress disorder

References

  1. Birken S, Lee S-Y, Weiner B. Uncovering middle managers’ role in healthcare innovation implementation. Vol. 7. 2012. 28 p.

  2. Edbrooke-Childs J, Calderon A, McDonnell M, Hirvonen H, Deighton J, Wolpert M. A qualitative exploration of the role of leadership in service transformation in child and adolescent mental health services. Child Adolesc Ment Health. 2018;

  3. Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q. 2004;82(4):581–629.

    PubMed  PubMed Central  Google Scholar 

  4. Bhargava K, Bhargava D. Evidence Based Health Care: a scientific approach to health care. Sultan Qaboos Univ Med J. 2007;7(2):105–7.

    PubMed  PubMed Central  Google Scholar 

  5. Mandrou E, Tsounis A, Sarafis P. Validity and reliability of the Greek version of Implementation Leadership Scale (ILS). BMC Psychol. 2020;8(1):49.

    PubMed  PubMed Central  Google Scholar 

  6. Melnyk BM, Fineout-Overholt E, Gallagher-Ford L, Kaplan L. The state of evidence-based practice in US nurses: critical implications for nurse leaders and educators. J Nurs Adm. 2012;42(9):410–7.

    PubMed  Google Scholar 

  7. Bodenheimer T, Sinsky C. From triple to quadruple aim: care of the patient requires care of the provider. Ann Fam Med. 2014;12(6):573–6.

    PubMed  PubMed Central  Google Scholar 

  8. Hong Y, Liao H, Hu J, Jiang K. Missing link in the service profit chain: a meta-analytic review of the antecedents, consequences, and moderators of service climate. J Appl Psychol. 2013;98(2):237–67.

    PubMed  Google Scholar 

  9. Scott SG, Bruce RA. Determinants of innovative behavior: a path model of individual innovation in the workplace. Acad Manage J. 1994;37(3):580–607.

    Google Scholar 

  10. Gumusluoglu L, Ilsev A. Transformational leadership, creativity, and organizational innovation. J Bus Res. 2009;62(4):461–73.

    Google Scholar 

  11. Aarons GA, Ehrhart MG, Farahnak LR. The Implementation Leadership Scale (ILS): development of a brief measure of unit level implementation leadership. Implement Sci IS. 2014;9(1):45.

    PubMed  Google Scholar 

  12. Hong Y, Liao H, Hu J, Jiang K. Missing link in the service profit chain: a meta-analytic review of the antecedents, consequences, and moderators of service climate. J Appl Psychol. 2013;98:237–67.

    PubMed  Google Scholar 

  13. Burns JM. Leadership. New York: Harper and Row; 1978.

    Google Scholar 

  14. Bass BM. From transactional to transformational leadership: Learning to share the vision. Organ Dyn. 1990;18(3):19–31.

    Google Scholar 

  15. Bass BM, Avolio BJ, Jung DI, Berson Y. Predicting unit performance by assessing transformational and transactional leadership. J Appl Psychol. 2003;88(2):207.

    PubMed  Google Scholar 

  16. Farahnak LR, Ehrhart MG, Torres EM, Aarons GA. The influence of transformational leadership and leader attitudes on subordinate attitudes and implementation success. J Leadersh Organ Stud. 2020;27(1):98–111.

    Google Scholar 

  17. Fernet C, Trépanier S-G, Austin S, Gagné M, Forest J. Transformational leadership and optimal functioning at work: on the mediating role of employees’ perceived job characteristics and motivation. Work Stress. 2015;29(1):11–31.

    Google Scholar 

  18. Bycio P, Hackett RD, Allen JS. Further assessments of Bass’s (1985) conceptualization of transactional and transformational leadership. J Appl Psychol. 1995;80(4):468–78.

    Google Scholar 

  19. Corrigan PW, Diwan S, Campion J, Rashid F. Transformational leadership and the mental health team. Adm Policy Ment Health Ment Health Serv Res. 2002;30(2):97–108.

    Google Scholar 

  20. Zohar D. Modifying supervisory practices to improve subunit safety: a leadership-based intervention model. J Appl Psychol. 20020206;87(1):156.

  21. Hater JJ, Bass BM. Superiors’ evaluations and subordinates’ perceptions of transformational and transactional leadership. J Appl Psychol. 1988;73(4):695–702.

    Google Scholar 

  22. Howell JM, Avolio BJ. Transformational leadership, transactional leadership, locus of control, and support for innovation: key predictors of consolidated-business-unit performance. J Appl Psychol. 1993;78(6):891–902.

    Google Scholar 

  23. Toor S-R, Ofori G. Ethical leadership: examiningthe relationships with full range leadership model, employee outcomes, and organizational culture. J Bus Ethics. 2009;90(4):533.

    Google Scholar 

  24. Hetland H, Sandal G. Transformational leadership in Norway: Outcomes and personality correlates. Eur J Work Organ Psychol. 2003;12(2):147–70.

    Google Scholar 

  25. Tejeda MJ, Scandura TA, Pillai R. The MLQ revisited: psychometric properties and recommendations. Leadersh Q. 2001;12(1):31–52.

    Google Scholar 

  26. Moreno-Casado H, Leo FM, López-Gajardo MA, García-Calvo T, Cuevas R, Sánchez-Oliva D. Adaptation and validation of the MLQ-5X Leadership Scale to the Spanish educational context. An Psicol. 2021;37(2):311–22.

    Google Scholar 

  27. Kanste O, Miettunen J, Kyngäs H. Psychometric properties of the multifactor leadership questionnaire among nurses. J Adv Nurs. 2007;57(2):201–12.

    PubMed  Google Scholar 

  28. Antonakis J, Avolio BJ, Sivasubramaniam N. Context and leadership: an examination of the nine-factor full-range leadership theory using the Multifactor Leadership Questionnaire. Leadersh Q. 2003;14(3):261–95.

    Google Scholar 

  29. Muenjohn N, Armstrong P. Evaluating the structural validity of the Multifactor Leadership Questionnaire (MLQ), capturing the leadership factors of transformational-transactional leadership. Contemp Manag Res. 2008;4:3–14.

    Google Scholar 

  30. Serrat O. Leading solutions. The Chicago School of Professional Psychology, USA: Springer; 2021. p. 353.

    Google Scholar 

  31. Bass, Bernard M., Riggio, Ronald E. Transformational leadership. 2006.

  32. Kelloway EK, Mullen J, Francis L. Divergent effects of transformational and passive leadership on employee safety. J Occup Health Psychol. 2006;11(1):76–86.

    PubMed  Google Scholar 

  33. Powell BJ, Mandell DS, Hadley TR, Rubin RM, Evans AC, Hurford MO, et al. Are general and strategic measures of organizational context and leadership associated with knowledge and attitudes toward evidence-based practices in public behavioral health settings? A cross-sectional observational study. Implement Sci. 2017;12(1):64.

    PubMed  PubMed Central  Google Scholar 

  34. Martinez RG, Lewis CC, Weiner BJ. Instrumentation issues in implementation science. Implement Sci. 2014;9(1):118.

    PubMed  PubMed Central  Google Scholar 

  35. Proctor EK, Landsverk J, Aarons G, Chambers D, Glisson C, Mittman B. Implementation research in mental health services: an emerging science with conceptual, methodological, and training challenges. Adm Policy Ment Health. 2009;36(1):24–34.

    PubMed  Google Scholar 

  36. Aarons GA, Ehrhart MG, Torres EM, Finn NK, Beidas RS. The Humble leader: association of discrepancies in leader and follower ratings of implementation leadership with organizational climate in mental health. Psychiatr Serv. 2016;68(2):115–22.

    PubMed  PubMed Central  Google Scholar 

  37. Finn NK, Torres EM, Ehrhart MG, Roesch SC, Aarons GA. Cross-validation of the Implementation Leadership Scale (ILS) in child welfare service organizations. Child Maltreat. 2016;21(3):250–5.

    PubMed  Google Scholar 

  38. Lyon AR, Cook CR, Brown EC, Locke J, Davis C, Ehrhart M, et al. Assessing organizational implementation context in the education sector: confirmatory factor analysis of measures of implementation leadership, climate, and citizenship. Implement Sci. 2018;13(1):5.

    PubMed  PubMed Central  Google Scholar 

  39. Torres EM, Ehrhart MG, Beidas RS, Farahnak LR, Finn NK, Aarons GA. Validation of the Implementation Leadership Scale (ILS) with supervisors’ self-ratings. Community Ment Health J. 2018;54(1):49–53.

    PubMed  Google Scholar 

  40. Aarons GA, Ehrhart MG, Farahnak LR, Finn N. Implementation leadership: confirmatory factor analysis and supervisor-clinician discrepancy in ratings on the Implementation Leadership Scale (ILS). Implement Sci. 2015;10(1):A70.

    PubMed Central  Google Scholar 

  41. Finn NK, Torres EM, Ehrhart MG, Roesch SC, Aarons GA. Cross-validation of the implementation leadership scale (ILS) in child welfare service organizations. Child Maltreat [Internet]. 2016;21. Available from: https://doi.org/10.1177/1077559516638768

  42. Lyon AR, Cook CR, Brown EC, Locke J, Davis C, Ehrhart M. Assessing organizational implementation context in the education sector: confirmatory factor analysis of measures of implementation leadership, climate, and citizenship. Implement Sci [Internet]. 2018;13. Available from: https://doi.org/10.1186/s13012-017-0705-6

  43. Hu J, Hu J. Translation and Validation of the Implementation Leadership Scale in Chinese Nursing Context. In STTI; 2019 [cited 2021 Feb 4]. Available from: https://stti.confex.com/stti/congrs19/webprogram/Paper94783.html

  44. Hemsworth D, Muterera J, Baregheh A. Examining basss transformational leadership in public sector executives: a psychometric properties review. J Appl Bus Res. 2013;29:853–62.

    Google Scholar 

  45. Xu L, Wubbena Z, Stewart T. Measurement invariance of second-order factor model of the Multifactor Leadership Questionnaire (MLQ) across K-12 principal gender. J Educ Adm. 2016;54(6):727–48.

    Google Scholar 

  46. Egeland KM, Skar A-MS, Endsjø M, Laukvik EH, Bækkelund H, Babaii A, et al. Testing the leadership and organizational change for implementation (LOCI) intervention in Norwegian mental health clinics: a stepped-wedge cluster randomized design study protocol. 2019;14(1):28.

  47. Aarons GA, Ehrhart MG, Farahnak LR, Hurlburt MS. Leadership and organizational change for implementation (LOCI): a randomized mixed method pilot study of a leadership and organization development intervention for evidence-based practice implementation. Implement Sci [Internet]. 2015;10. Available from: https://doi.org/10.1186/s13012-014-0192-y

  48. Curran GM, Bauer M, Mittman B, Pyne JM, Stetler C. Effectiveness-implementation hybrid designs. Med Care. 2012;50(3):217–26.

    PubMed  PubMed Central  Google Scholar 

  49. Bass BM, Avolio BJ. MLQ: Multifactor Leadership Questionnaire. Redwood City: Mind Garden; 1995.

  50. Puni A, Mohammed I, Asamoah E. Transformational leadership and job satisfaction: the moderating effect of contingent reward. Leadersh Organ Dev J. 2018;39(4):522–37.

    Google Scholar 

  51. Aarons GA. Mental health provider attitudes toward adoption of evidence-based practice: the evidence-based practice attitude scale (EBPAS). Ment Health Serv Res. 2004;6(2):61–74.

    PubMed  PubMed Central  Google Scholar 

  52. Rye M, Torres EM, Friborg O, Skre I, Aarons GA. The evidence-based practice Attitude Scale-36 (EBPAS-36): a brief and pragmatic measure of attitudes to evidence-based practice validated in US and Norwegian samples. Implement Sci. 2017;12(1):44.

    PubMed  PubMed Central  Google Scholar 

  53. Egeland K, Ruud T, Ogden T, Lindstrøm J, Heiervang K. Psychometric properties of the Norwegian version of the Evidence-Based Practice Attitude Scale (EBPAS): To measure implementation readiness. Health Res Policy Syst. 2016;14.

  54. Muthén LK, Muthén BO. Mplus User’s Guide [Internet]. Mplus User’s Guide (Eigth ed.). 1998 [cited 2021 May 6]. Available from: https://www.statmodel.com/html_ug.shtml

  55. Hair JF, Black, William C., Babin BJ, Anderson RE. Multivariate Data Analysis [Internet]. 7th ed. Pearson; 2013 [cited 2021 May 6]. Available from: https://www.pearson.com/uk/educators/higher-education-educators/program/Hair-Multivariate-Data-Analysis-Pearson-New-International-Edition-7th-Edition/PGM1052102.html

  56. Kline RB. Principles and practice of structural equation modeling, 2nd ed. New York, NY, US: Guilford Press; 2005. xviii, 366 p. (Principles and practice of structural equation modeling, 2nd ed).

  57. IBM Corp. IBM SPSS Statistics for Windows. Version 27.0. 2020.

  58. RStudio Team. RStudio: Integrated Development Environment for R [Internet]. Boston, MA: RStudio, PBC.; 2020. Available from: http://www.rstudio.com/

  59. Edwards G, Schyns B, Gill R, Higgs M. The MLQ factor structure in a UK context. Leadersh Organ Dev J. 2012;33(4):369–82.

    Google Scholar 

Download references

Acknowledgements

Not applicable.

Funding

The current study is part of a national implementation of EBPs for PTSD in Norwegian specialised mental health clinics for children and adults. The implementation project was funded by the Norwegian Ministry of Health and Care Services. The funders had no part in any aspect of the project, neither the study design, data collection, analysis, nor in writing the manuscript.

Author information

Authors and Affiliations

  1. Norwegian Centre for Violence and Traumatic Stress Studies (NKVTS), Gullhaugveien 1-3, 0484, Oslo, Norway

    Nora Braathu, Erlend Høen Laukvik, Karina M. Egeland & Ane-Marthe Solheim Skar

Authors
  1. Nora Braathu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Erlend Høen Laukvik
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Karina M. Egeland
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ane-Marthe Solheim Skar
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

KME and AMSS were principal investigators and contributed to the translation of the measurement instruments. NB developed the manuscript outline, conducted the data analysis, and drafted the manuscript. EHL contributed to the background and discussion section of the manuscript. All authors were involved in editing and reviewing the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Nora Braathu.

Ethics declarations

Ethics approval and consent to participate

The current study is approved by the Norwegian Centre for Research Data (NSD). All participants consented to participate in the study. All parts of the study were conducted in compliance with the Helsinki Declaration.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Braathu, N., Laukvik, E.H., Egeland, K.M. et al. Validation of the Norwegian versions of the Implementation Leadership Scale (ILS) and Multifactor Leadership Questionnaire (MLQ) in a mental health care setting. BMC Psychol 10, 25 (2022). https://doi.org/10.1186/s40359-022-00725-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s40359-022-00725-8

Keywords

  • Evidence-based practice
  • Mental health
  • Implementation climate
  • Implementation strategies
  • PTSD
  • Transformational leadership
  • Implementation leadership

BMC Psychology

ISSN: 2050-7283

Contact us