Study design
We used a quasi-experimental study design with repeated measures whereby all eligible participants received a literature-based condition (brochure about mentalizing) and a one-day mentalizing skills training in a predefined order. The literature-based condition was implemented to estimate possible repeated measurement and expectational effects. The training took place at three different locations in Germany (Heidelberg, Cologne, and Berlin). Participants self-selected to the training location based on its proximity to their workplace.
After T0, all participants received the brochure and were asked to read it within a time frame of two weeks prior to the next assessment (T1) which was presented two days before the training. We expected that during a couple of working days participants would have the opportunity to implement new skills and experience a change of the working relationship. Therefore, T2 was presented to all participants seven days after the training. A follow-up assessment (T3) took place 28 days after the training. The time frame was chosen to specify the effects and to assess possible long-term effects found in previous brief mentalizing skills trainings [34].
Data collection was carried out online via SoSci Survey with an average of approximately 90 min completion time across participants and at each time point. The survey included self-report questionnaires and instructions for five-minute speech samples about difficult interpersonal situations at work at all time points. We also assessed satisfaction with both the training and the literature-based conditions. Audio files with participants’ speech samples were sent to the study center electronically after completing the online survey.
Participants were blind to the study’s hypotheses. It was impossible to blind trainers to experimental hypotheses. Raters of observer-based mentalizing were blind to time point.
The ethical approval for this study was obtained from Ethical Committee, Heidelberg University, Medical Faculty (No. S-309/2018 approved May 22, 2018). All participants gave their informed consent to participate in the study.
Sample characteristics
Data was collected from January 2019 to March 2019. The study was coordinated by the Institute for Psychosocial Prevention, University Hospital Heidelberg. Participants were recruited from the nationwide network of the National Centre for Early Prevention (Nationales Zentrum Frühe Hilfen, NZFH), using websites, flyers, and Twitter and were asked to contact the study center when interested in study participation. We included professionals currently working as home visitors in ECI, including family midwifes or pediatric nurses, as well as lay volunteers. Certain ECI programs in Germany provide service by volunteers (e.g., within the service “Welcome” visits after birth) and 1.4% of young families receive visits by volunteers [2] which is why they were included in this study. Home visitors who usually reported typically seeing families only once without a follow-up were excluded from study participation. In a phone screening, inclusion and exclusion criteria were assessed and eligible home visitors were invited for study participation if they provided verbal and written informed consent.
Study participants
Figure 1 displays the participant flow. Among the 200 interested parties, 124 were reached for a phone screen and assessed for eligibility. Twenty-five individuals were excluded from study participation; in most cases because they worked in administrative functions or did not work in the field of ECI. Of the 74 individuals that consented, n = 73 participated in T0. Twenty-five individuals were initially put on a waiting list, out of whom 14 were included after other participants cancelled. The remaining 11 individuals needed to be excluded due to training locations’ capacities that limited the number of participants.
Two (2.74%) participants were men and 94.5% were professionals in ECI trained as a midwife or a pediatric nurse. Two of the four volunteers were social workers. Mean age of participants was 50.82 (SD = 8.01; range 29 to 66). Participants reported an average of M = 16.71 years (SD = 9.14, range 2–31) of previous experience working in ECI and 66% reported having participated in a workshop on communication or working relationship in the past.
Interventions
Mentalizing skills training
The manualized one-day (8-h) training in mentalizing skills for ECI home visitors was developed with Anthony Bateman based on the existing MBT skills training for mental health professionals [36] and the adaptive mentalization-based integrative treatment concept, which is a team-based approach to address the needs of complex clients [11]. The objective of the training was to improve the quality of future working relationships by strengthening mentalizing skills.
The training was delivered in a group setting and combined didactic teaching, guided reflections, experiential and simulation-based training, and video clips. The schedule was divided into eight sections focused on certain contents and tought competences, one sections to start off and close with, as well as several comfort breaks and a lunch break. The eight sections were: (1) Starting with an exercise: How do you know who I am?; (2) What is the meaning of mentalizing?; (3) Mentalizing stance and mentalizing communication; (4) Mentalizing problems; (5) Mentalizing of challenging working relationships in ECI; (6) Balancing one’s own mentalizing; (7) Strengthening parental mentalizing; and (8) Repair ruptures in working relationships. The last section included an overall reflection on the acquired skills and giving feedback.
Video clips on interpersonally difficult situations were shown to engage participants in mentalizing and demonstrate core theoretical concepts. Core theoretical concepts (e.g., defining mentalizing and stress-related mentalizing problems) and related skills were taught and practiced in group exercises and real- and role plays in small groups, including how to engage in self- and other-focused reflections, maintain and communicate a mentalizing stance in stressful situations, and how to repair ruptures in the relationship. Additionally, specific MBT techniques (e.g., stop, stand, and rewind) and supplementary tools (e.g., checklist for mentalization-based work involving for example questions on the practitioners perception of the family during the last visit in black/white or multifaceted) were taught and practiced in role plays. Home visitors’ experiences with all real- and role plays, and their perceived learning outcome were afterwards discussed within the larger group. To maximize the effect on interactions within the home visiting context, participants were encouraged to individual reflections on their working relationships and the discussion of personally relevant situations that affected the working relationship with families in the group.
The training was conducted by two female trainers who were trained in MBT (AKG and SH). Throughout the training, the trainers modelled a mentalizing stance in discussions about home visitors’ personal experiences and by pointing out moments where they struggled to do so, both in their communications with participants and with each other. Fidelity was ensured by following a training manual [37] that provided specific guidelines for each of the trainings’ sections as well as supporting training materials. In addition, the training was video recorded and the implemented training elements were checked against the training manual by the two trainers. All elements were implemented according to the training manual.
Literature-based intervention
The 10-page brochure covered the same core theoretical concepts in the day-long training along with illustrative examples and case descriptions relevant to ECI. One item included in T1 (‘knowledge about mentalization’) was used as a measure of whether the brochure was read as an estimate of fidelity.
Measures
Working alliance inventory (WAI)
The WAI assesses the quality of a therapeutic relationship [38]. It has robust reliability [39] and good validity [40]. The WAI is comprised of 16 items scored on a 5-point Likert scale (rarely to always); higher scores signify a better working alliance with the client, according to the practitioner. The item wording was adapted to fit the context of ECI. Cronbach’s α was 0.91.
Therapists’ work involvement scales (TWIS)
The TWIS [41, 42] assess the quality of therapeutic engagement in the working relationship utilizing two scales: healing involvement (TWIS heal) and stressful involvement (TWIS stress). The scales obtained good validity and satisfactory reliability [41]. The TWIS contains 52 items which score on a 6-point Likert scale (none/not at all/never to many/very/very often) or a 4-point Likert scale (never to very often). We adopted the wording to the context of ECI. Cronbach’s α was 0.80 (TWIS heal) and 0.83 (TWIS stress).
Interpersonal reactivity index (IRI)
Two scales of the IRI [43] were used to assess empathy: change of perspective (cognitive dimension of empathy) and personal distress in emotionally difficult situations (emotional dimension of empathy) (4 items each, scoring on a 5-point Likert scale ranging from does not describe me well to describes me very well). The IRI has very good reliability and validity [43]. Change of perspective had an α of 0.84 and personal distress had an α of 0.79.
Generalized self-efficacy scale (GSES)
The GSES [44] reliably and validly assesses perceived self-efficacy with 10 items scored on a 4-point Likert scale (not true to completely true) with higher scores indicating more self-efficacy. Cronbach’s α was 0.83.
We utilized a range of measures for the different components of mentalizing due to the complexity of the construct [25] as well as the lack of previous training studies in this specific population. Self-report measures assessed attributional complexity, metacognition, and reflective functioning. The five-minute speech sample was coded using an observer-based measure to assess mind-mindedness while discussing a difficult interpersonal situation.
Attributional complexity scale-short (ACS)
Interest in mentalization was assessed with the ACS [45]. The questionnaire has been validated and contains seven items. In this study, it scored on a 5-point Likert scale ranging from completely wrong (1) to completely correct (5). Higher scores signify higher interest in mentalizing. Cronbach’s α was 0.81.
Metacognition self-assessment scale (MSAS)
The MSAS’ mastery scale assesses metacognition in problem solving (i.e., strategies which individuals use to exploit their knowledge of themselves and of others to solve psychological and interpersonal problems) and has sufficient validity and reliability [46]. The scale consists of five items which score on a 5-point Likert scale (almost never to almost always) with higher scores signifying more metacognition. This study is the first to use MSAS in a German sample; thus, it was translated via back-and-forth translation (English to German). The German MSAS had not been validated before. A Cronbach’s α of 0.71 in this study points towards a satisfying internal consistency.
Reflective functioning questionnaire (RFQ-8)
The RFQ-8 assesses quality of mentalizing with eight items on two scales, certainty (RFQ-c) and uncertainty of mental states (RFQ-u) [47]. Despite its low to satisfactory reliability and validity [47, 48], the RFQ-8 is the gold standard of self-reported mentalizing capacity. The items score on a 7-point Likert scale (completely disagree to completely agree). High or low scores on both scales indicate low mentalizing ability. Cronbach’s α was 0.84 (RFQ-c) and 0.58 (RFQ-u). Thus, RFQ-u was excluded from data analysis.
Five-minutes speech samples-mind-mindedness (FMSS-MM)
FMSS-MM [49] measures observer-rated mind-mindedness (MM) in speech samples. Five-minutes speech samples have been successfully used to assess quality/degree of mentalizing in parents’ narratives [24] using the MM coding manual [23]. FMSS-MM applies an adapted version of the MM coding manual to five-minute speech samples of home visitors’ narratives about difficult interpersonal situations. Two five-minute monologues per participant were collected within two tasks. The first task consisted of the presentation of a vignette which describes a typical difficult interpersonal situation in ECI (task 1). The second task prompts the participant to discuss a personally difficult interpersonal situation encountered while working as home visitors (task 2). After reading (task 1) respective after having described the personally difficult interpersonal situation (task 2), participants were asked to speak at least 5 min, answering four questions that pull for mentalizing (e.g., “How do you understand the family?”).
The coding of the speech samples counted the number of mind-minded words in the transcript using three scales: MM-other is the relative frequency (to the total words of the transcript) of mind-minded words related to another person (e.g., the mother was maybe afraid that I would take her child away); while MM-self is the relative frequency of mind-minded words related to the self (e.g., I felt a huge responsibility in this situation). MM-nk (MM not-knowing) is the relative frequency (to the total of mind-minded words) of words that reflect a not-knowing stance related to mind-minded comments (e.g., the mother was maybe afraid that I would take her child away). Higher values in MM-other and MM-self indicate more MM and higher values in MM-nk indicate a greater not-knowing stance. In sum, six scores were used for analyses: MM-other, MM-self, and MM-nk for task 1 (vignette) and task 2 (personally relevant situation).
Three raters (two authors of the study (AKG, SH) and an undergraduate psychologist) recalibrated the method by independent ratings of pilot data until reliability was deemed sufficient (ICC ≥ 0.82). To ensure blinding, transcripts were re-coded by a research assistant. FMSS-MM ratings were completed at the end of the study. ICCs were calculated for 50 of the 229 transcripts (21.83%) and ranged from MM-nk (0.72) to MM-self (0.87).
Knowledge about mentalization
A test was constructed to assesses knowledge about mentalization (e.g., demonstration of understanding the concept of mentalization) by utilizing 13 items in multiple choice format. Correctly answered items were summed into a sum score. At T1, the correctly answered question “knowledge about the basic theory of mentalization” was used to indicate whether the brochure had been read. The test had an α of 0.54 and thus it was excluded from further analysis.
Satisfaction with brochure and training
Satisfaction with the brochure and the training was assessed via two items, “general satisfaction” and “learning growths”, from the German Client Satisfaction Questionnaire (CSQ-8; [50]). The items scored on a 4-point Likert scale where higher scores indicate more satisfaction respective learning growths.
Statistical analysis
A priori power analysis for the primary outcome was conducted assuming an average small to medium effect of f = 0.2 based on previous studies showing effects on attitudes working with challenging clients [32, 34]. An alpha of α = 0.05 and a power of β = 0.85 resulted in a sample size of N = 63 considering four measurement points and an assumed ICC of 0.2. Because of the exploratory nature of the study with regards to secondary outcomes, type I errors were not controlled for [51]. We corrected for multiple tests on the primary outcome scales (WAI, TWIS heal, TWIS stress) utilizing Bonferroni-Holm procedure.
To answer the question of whether or not the training had an effect on home visitor mentalizing and related outcomes, several multilevel models (MLM) were conducted with primary and secondary outcomes (level 1) nested within participants (level 2). A nesting of participants within training centers (level 3) was considered but dropped from the final model for lack of variance. Every model consisted of categorical time (T0, T1, T2 and T3) at level 1 as the main predictor, as well as age, additional training experience (dummy coded as 0 = none, 1 = some), and job experience in years as covariates on level 2, and a random effect for person to account for the nested data structure. Q-Q plots and sphericity plots were utilized to examine if the assumptions of MLM were met and no significant deviations were found.
Since the assessments were conducted online, forced-choice format was used to achieve data without missing values. The data was scanned for multivariate outliers utilizing Mahalanobis distance and no significant outliers were found.
Regression analyses with each outcome variable utilized as a criterion predicted by the rest of the outcome variables that were used as predictors were done to estimate multivariate collinearity between outcome measures [52]. Drop-out analyses comparing participants who dropped out during the study with completers regarding sample characteristics and outcome measures were performed with Mann–Whitney-U and t-Tests.
All statistical analyses were performed with RStudio version 1.2.5033 [53] using R version 3.6.3 [54].