Skip to main content

Short-term effects of the “Together at School” intervention program on children’s socio-emotional skills: a cluster randomized controlled trial

Abstract

Background

Together at School is a universal intervention program designed to promote socio-emotional skills among primary-school children. It is based on a whole school approach, and implemented in school classes by teachers. The aim of the present study is to examine the short-term effects of the intervention program in improving socio-emotional skills and reducing psychological problems among boys and girls. We also examine whether these effects depend on grade level (Grades 1 to 3) and intervention dosage.

Methods

This cluster randomized controlled trial design included 79 Finnish primary schools (40 intervention and 39 control) with 3 704 children. The outcome measures were the Strengths and Difficulties Questionnaire (SDQ) and the Multisource Assessment of Social Competence Scale (MASCS) with teachers as raters. The intervention dosage was indicated by the frequencies six central tools were used by the teachers. The data was collected at baseline and 6 months later. Intervention effects were analyzed using multilevel modeling.

Results

When analyzed across all grades no intervention effect was observed in improving children’s socio-emotional skills or in reducing their psychological problems at 6-month follow-up. Among third (compared to first) graders the intervention decreased psychological problems. Stratified analyses by gender showed that this effect was significant only among boys and that among them the intervention also improved third graders’ cooperation skills. Among girls the intervention effects were not moderated by grade. Implementing the intervention with intended intensity (i.e. a high enough dosage) had a significant positive effect on cooperation skills. When analyzed separately among genders, this effect was significant only in girls.

Conclusions

These first, short-term results of the Together at School intervention program did not show any main effects on children’s socio-emotional skills or psychological problems. This lack of effects may be due to the relatively short follow-up period given the universal, whole school-based approach of the program. The results suggest that the grade level where the intervention is started might be a factor in the program’s effectiveness. Moreover, the results also suggest that for this type of intervention program to be effective, it needs to be delivered with a high enough dosage.

Trial registration

ClinicalTrials.gov identifier: NCT02178332; Date of registration: 03-April-2014.

Peer Review reports

Background

Epidemiological research shows that behavioral, emotional and social difficulties often start at early age with 5–15 % of children and 20–25 % of youth suffering from some mental health problem [15]. These difficulties have negative effects on children’s’ quality of life in general and increase the risk of various psychological, physical, and socioeconomic problems, as well as substance abuse and delinquency later in life [6, 7]. Despite the availability, growing use of, and advances in treatments for mental health problems services [8, 9], many children suffering from such problems will not seek or receive treatment, or terminate it prematurely, fail to respond to it, or continue to have difficulties despite treatment [8]. Thus, there is a need for alternative intervention approaches that could reach children and adolescents with mental health problems more widely as well as provide a means for the prevention of such problems.

There is growing evidence of the benefits of prevention and promotion aimed toward reducing the risk of mental health problems and increasing psychological well-being at an early stage and age [8, 10]. Current approaches to prevention include universal interventions, which are targeted to whole child populations regardless of their health or risk status [8, 11, 12]. In schools, practically the whole population of children and adolescents can be reached which makes school a natural environment for universal interventions. Furthermore, the school environments provide stability with an existing school curricula, structures, agreed policies, and resources, which are all essential for well planned, systematic and long-term mental health interventions [1317]. School-based, universal socio-emotional learning (SEL) programs have been shown to have significant positive effects on children’s socio-emotional skills: according to their meta-analysis of 213 studies, Durlak et al. [14] reported a mean effect size of 0.57 (Hedges’ g) for socio-emotional skills, while somewhat smaller effects for other outcomes including social behaviors, conduct problems, emotional distress, academic performance, and attitudes.

Although the importance of prevention has been acknowledged within educational and public policies, there is still much to be done concerning governmental structures and a shared commitment among the respective stakeholders [15, 18]. In Finland, the Ministry of Health and Social Affairs recognized the need for a program promoting children’s socio-emotional skills and mental health in schools and, in 2003, initiated the development of a school-based intervention program. This process resulted in the Together at School intervention program, which is a carefully developed program combining effective components from other school-based programs as well as unique elements developed to fit the Finnish school system and primary-school curriculum [19]. The Together at School intervention was developed in close cooperation with school staff and tested in every-day school work across several years. The aim of the program is to promote children’s socio-emotional skills in a whole school context. The intervention program consists of manualized tools and methods, training of the intervention elements, and school visits by the instructors [20]. The intervention is carried out in classrooms by teachers who are seen as the primary agents of the children’s SEL process. In order to support the SEL process of the children in line with the whole school approach, the intervention also aims to provide similar experiences of SEL to school staff with the help of the principal. Teacher-parent collaboration is also supported.

Earlier research suggests that school based interventions, especially those promoting broader developmental domains enhancing socio-emotional skills, should be started early with the youngest children [16]. In line with this, the Together at School program has focused on the first school years, with the first school year, when the child arrives in a new educational environment, being considered especially important for the training of social relations and emotions. In the present Randomized Controlled Trial (RCT) the Together at School intervention was administered also at the second and third grades, in order to examine whether the program is equally efficient when administered at different grades, and for children of different ages (in Finland first graders are seven, third graders 9 years old).

Concerning intervention implementation, the question whether and to what extent the intervention dosage is related to its effectiveness, is important. Accordingly, the present study analyses the amount that the intervention methods and tools are used in real life school work situations. It has been pointed out that there is a gap in research regarding how the implementation variables interact with the intervention program and affect implementation effectiveness and student outcomes [21]. Moreover, dosage effects have been somewhat underreported, even if implementation quality is considered to be important for both intervention success and as one of the possible explanations for the absence of positive intervention results [22, 23]. Available research suggests that intervention dosage is related to intervention effectiveness and that a higher dosage potentially leads to more positive student outcomes [15, 21].

Aims

The aims of the study were, first, to examine the short-term effects of the Together at School intervention program, a universal, whole school-based program targeted at improving primary-school children’s socio-emotional skills and reducing psychological problems.

Second, the study examined whether the intervention effects vary depending on the grade (Grades 1–3) the children are in when the intervention program is started. In the view of earlier research our hypothesis is that the intervention is likely to be more effective among younger children, i.e. when started already in the beginning of the child’s school path.

Third, we addressed the question regarding how the intervention dosage is related to intervention effectiveness and our hypothesis is that the intervention would be effective more likely when implemented with the intended intensity.

Finally, while the Together at School intervention is intended to be used among both boys and girls, we were also interested to see whether there are any gender differences regarding the aforementioned study questions. We know from previous literature that boys and girls differ significantly in emotional and social skills and psychiatric problems at elementary school years [2426]. Thus, in addition to presenting results for the total sample as the primary analysis, we also present data separately for boys and girls.

Methods

The context of the present study

Finland is an egalitarian country with a rather high standard of living and relatively small socio-economic differences. It is compulsory to attend school in Finland from the age of seven (Grade 1) until the age of 15 (Grade 9). The school system is financed and organized by local municipalities and regulated by the Ministry of Education and Culture, and only a very small minority of Finnish children attend private schools. To examine the effectiveness of the Together at School intervention program, a cluster RCT was organized. The RCT was conducted in the whole of Finland including schools from different parts of the country. Data was collected at baseline, 6 monthsFootnote 1 after baseline, and will also be collected 18 months after baseline from the same participants (children and their parents, teachers and the principals). The present study is part of this RCT and focuses on the primary child outcomes (socio-emotional skills and psychological problems) assessed by the teachers at baseline (T0) and 6-month follow-up (T1).

Prior to the RCT, the intervention program went through an excessive development process of several years, during which a group of teachers, principals, and healthcare professionals tested, modified and adopted the intervention methods and tools in close collaboration with three development schools. Moreover, the intervention program was piloted in four schools in four different towns. Analyses of the pilot study indicated that the intervention program was feasible, perceived beneficial and suitable in different school settings [27].

Ethics approval and funding

The study protocol was approved by the Ethics Committee of the National Institute for Health and Welfare in Helsinki, Finland (27.9.2012) and the trial is registered in the ClinicalTrials.gov registry (NCT02178332). The trial was funded by the Finnish Ministry of Education and Culture, the National Institute for Health and Welfare and the town of Ylöjärvi.

Recruitment procedure

All Finnish primary schools were invited to participate in the study on the condition that the school had a minimum of two teachers, who agreed to participate for the whole study period of two school years, and who were teaching the first, second or third grades. Of the 109 schools that were willing to participate, 23 were excluded from the study as they were considered non-eligible due to the risk of contamination or excessive training costs. The eligible 86 schools were randomized into either intervention or control groups. After the randomization, seven schools declined their participation due to various reasons (e.g. school economic situation or personnel shortage) resulting in 79 (40 intervention and 39 control) schools in the study. The participant flow is outlined in Fig. 1 and the recruitment process and randomization are reported more in detail in the study protocol [20].

Fig. 1
figure 1

Flow chart of participants. aThere were 2 intervention and 6 control group classes where the teacher did not report any data valid for the present study and were thus excluded, leaving 134 intervention and 108 control group classes for the analyses. bAll in all there were 2036 (out of 2090) children in the intervention and 1668 (out fo 1754) in the control group, for whom the teacher reported valid data (outcomes) either at baseline or follow up

All parents of the participating classes received an information letter regarding the intervention program and aims of the study. The parents were informed about the voluntary nature of the participation in the data collection and a consent form for data collection was included in the information letter. The teachers and principals consented by agreement [20].

The proportion of children with parental consent for data collection was higher in the intervention group (n = 2176, 86.9 %) compared to the control group (n = 1776, 77.3 %) (Fig. 1). Reasons for participant loss (children without parental consent) were gathered from teachers of ten selected schools with the lowest consent percentages. According to these data, the most common reasons for nonconsent were: difficulties in school/teacher-parent communication, cultural and language challenges, and parental stress especially in large and economically-strained families.

The Together at School intervention program

The Together at School intervention program employed methods and tools within three areas in order to guarantee the whole school approach. All the methods and tools are designed to be integrated into the normal school curriculum. The first set of methods, carried out in class by the teachers, are designed for the children: Circle time, Do-It-Myself lesson, Do-It-Together lesson, and teacher-child individual discussions. Circle time is a 15 min session consisting of guided greetings (e.g. eye contact, friendly touch), children taking turns in telling others about something important to them, and playing – the aim is to practice children’s communication and emotional skills and enhance classroom climate. The Do-It-Myself lesson is a 10–40 min weekly lesson aimed at practicing children’s skills of independent work: concentrating, focusing on one’s own task and problem solving. In the Do-It-Together lesson children work in small groups to practice cooperation skills. At the beginning of the lesson, children are given a vision of successful teamwork. When needed, help and encouragement are provided by the teacher. Children learn to present their own point of view, listen to others’, take turns, and negotiate. Individual teacher-child discussions (twice a year) where the teacher has a role more as a listener are aimed at creating a good and confidential relationship between the teacher and the child.

The second set of methods and tools, carried out by the principal and the staff, are designed to improve the school work environment (Planning of Collaborative Time, Staff Meeting, Service Station, and Toolkit Session). For example, a Toolkit session (45 min, once or twice a year) held by a staff member offers the teaching staff a possibility to share know-how based on their own interests and expertise, aiming at enhancing occupational know-how among the teaching staff. The third set of methods, the teacher-parent methods, carried out by the teachers are aimed at improving and maintaining a good relationship between the home and school and enhance teacher-parent collaboration. The methods include materials for meeting the parents individually (allowing the parents to express their thoughts freely and give information about their child) and for organizing the Parents’ Evening (aimed to activate teacher-parent interaction and provide support to the parents and the teacher in their child rearing work). For a more comprehensive set of descriptions of the contents and purposes of the methods and tools, see additional file in the study protocol [20].

The intervention group teachers received program training before starting the implementation of the intervention. Six instructors with a degree in pedagogics (trained teachers) were responsible for the intervention program training. The program training consisted of theory and practice of the intervention methods and tools (e.g. lessons, exercises, group discussions) and school visits by the instructors. As part of the training teachers received a 258-page Together at School manual where all the intervention methods and tools are described in detail. The training extended over 10 months and included four modules which proceeded in four waves [20]. After each training module the teachers started to use the methods and tools in their own classes individually.

The control group teachers and headmasters received two 3-hour lessons given by the psychologists and child psychiatrists of the research group. In November 2013 topics were children’s mental health in general, emotions and development of emotional and behavioral regulation. In March 2014 the topics were teachers’ well-being and professional development and how to establish good relationship and to cope with challenging situations with children and their parents. Lectures were offered in four central locations in Finland and they were videotaped to be available for those control group teachers and headmasters who could not attend the meeting. After the intervention study (the RCT) the control group teachers will receive the Together at School manual.

Measure of intervention dosage

Teachers completed detailed intervention protocols in order to keep a log of the tools and methods they had carried out in their classes [20]. The protocols were used to monitor the implementation process and measure the implementation fidelity, and based on these protocols intervention dosages were calculated. There were four classroom and two teacher-parent methods and tools, six in total, five of which were used in the autumn term 2013 and five in the spring term 2014 (see Fig. 2). The school environment/school staff methods were not included in the measure of dosage in the present study. To calculate the dosage, the intervention tools and methods were all rated first on a scale from 0 to 3 depending on how frequently the teachers had used them in their class during the term so that the maximum value (3) was given when a method was used with the frequency/extent that was specified in the intervention protocol (codes/ratings for the methods are given in Fig. 2). The maximum score for the dosage was 15 (5 x 3) for each term. If dosage was not available for one term due to a missing protocol (19 classes), the dosage of the other term was used as a replacement; two classes with no available protocols were coded to the sample mean dosage value. For the analyses, a mean score of the two terms was calculated and this mean dosage score was then divided into two groups reflecting whether or not the intervention was delivered with the intended intensity (as indicated by the protocol). The dosage groups were named as “intervention below the intended intensity” (0–12.0 points; 78 %) and “intervention as intended” (12.1–15 points; 22 %).

Fig. 2
figure 2

Intervention methods and tools and the frequencies they were used by the teachers during the school terms. For each method and frequency the rating that was used in the calculation of the intervention dosage is given in the parenthesis. aOnly in the spring term 2014. bOnly in the autumn term 2013

Measures of outcome

Children’s socio-emotional skills and psychological problems were measured using electronic questionnaires filled in by the teachers at T0 and T1. The Strengths and Difficulties Questionnaire (SDQ) and the Multisource Assessment of Social Competence Scale (MASCS) were used as the primary outcome measures. The SDQ is widely used and has good psychometric properties [2830]. Also the Finnish version of the SDQ has been shown to have good psychometric characteristics [3133]. The MASCS measures social competence and it has been designed to fit the Finnish elementary school context [34]. It is partly based on the School Social Behavior Scale (SSBS) [35] and has been validated in Finland [34]. The MASCS includes four subscales (impulsivity, disruptiveness, cooperation, and empathy) of which the two prosocial subscales, cooperation (range 5–20) and empathy (range 3–12), along with the prosocial behavior subscale (range 0–10) of the SDQ, were used to measure children’s socio-emotional skills in the present study. Children’s psychological problems were measured with the SDQ subscales for conduct disorder, hyperactivity, peer relations, and emotional problems, which together formed the SDQ psychological problems measure (SDQ total; range 0–40) [2830] used in the analyses.

Statistical methods

All analyses were made first for the total sample, and then separately for boys and girls. Intervention and control group differences in demographic characteristics at T0 were analyzed using chi-square test. Due to the clustered nature of the data the analyses of change between T0 and T1 in the outcome measures (i.e. the intervention effectiveness) were conducted using multilevel modeling with MLwiN Version 2.32 [36]. In clustered data, observations are non-independent, which means that, for example, the responses of the children attending one school class (sharing the same classroom, classmates and the teacher etc.) are more likely to be similar compared with children from a different class. The non-independence within classes might be even more pronounced in the present study, as we used outcome data of the children reported by the (within-class shared) teacher. If this non-independence is not taken into account in the modelling, then there is a possibility of inaccurate standard errors [37].

In the multilevel models, variance was estimated for each dependent variable at four levels: time, children, classes and schools. Also intraclass correlations (ICC), i.e. the proportions of variance each level explains of the total variance, were calculated as indicators of variation among children, classes and schools. While the ICC values at the child level were higher than the class level, they (and corresponding variances) were significant also at the class level indicating that children who share the same classroom were more alike compared to children from other classes. At the school level, the ICC values were low for each dependent variable and the variances were non-significant. Due to this, the school level was excluded from the successive analyses. Thus, a three-level model was fitted to represent change over time and differences between children and classes.

Multilevel models for change over time in socio-emotional skills and psychological problems were made separately for each of the four outcome variable: cooperation (MASCS), empathy (MASCS), prosocial behavior (SDQ) and psychological problems (SDQ). The distributions of the SDQ prosocial behavior and psychological problems scales were skewed, but as the residuals were quite normally distributed no transformation was made to keep the interpretation of the results as clear as possible. The intervention (intervention vs. control), time (T1 vs. T0) and grade (2nd, 3rd vs. 1st) were entered as independent variables. The intervention effect was presented with the Intervention x T1 interaction term (the interaction between group status and time), which can be interpreted as the difference between intervention and control group average change in the outcome measure from time T0 to T1. To examine whether intervention effects were different depending on the grade, the second-order interaction terms Intervention x grade x T1 were introduced to the model. The last set of analyses assessed whether the intervention effects varied depending on the intervention dosage (below/with intended intensity vs. control) using the resulting two interaction terms between intervention dosage and time (intervention below intended intensity/as intended x T1).

Sample characteristics

As a whole, 242 classes participated in the trial from 79 primary schools (40 intervention and 39 control). The present study sample (n = 3704) consisted of all those children who were rated by the teacher either at T0 or T1 on any of the four outcome measures and had parental consent for the teacher assessments. The mean age of the children was 8.1 years (SD = 0.85). As shown in Table 1, there were no major differences in the baseline demographic characteristics between the intervention and control group children or their families, although the proportions of second and third graders were different between the study groups.

Table 1 Child demographics by group status at baseline (T0)

Results

Descriptive statistics of outcome variables

Descriptive statistics of the outcome variables are given in Table 2. In general, boys had lower scores in socio-emotional skills and higher scores in psychological problems compared to girls. Preliminary comparisons between T0 and T1 scores indicated that there was an overall trend showing a raise in socio-emotional skills and a decrease in psychological problems. Frequencies of cases in the borderline/abnormal category of the psychological problems score (SDQ total) at T0 and T1 are presented in Additional file 1.

Table 2 Children’s socio-emotional skills and psychological problems at baseline (T0) and 6 months (T1) by group status, means

Intervention effects

Parameter estimates from the multilevel models for intervention effects on children’s socio-emotional skills and psychological problems are presented in Tables 3, 4 and 5. Coefficients for the intervention variable represent the differences between the intervention and control groups at T0. The intervention and control groups did not differ significantly regarding the outcome variables at T0, except for the higher levels of SDQ psychological problems among intervention group boys.

Table 3 Intervention effect on school children’s socio-emotional skills and psychological problems, total sample. Regression estimates from multilevel models: intervention effect (Model A, term Intervention x T1) and intervention effect moderated by school grade (Model B, terms Intervention x 2nd/3rd grade x T1, 1st grade as the reference)
Table 4 Intervention effect on school children’s socio-emotional skills and psychological problems among boys. Regression estimates from multilevel models: intervention effect (Model A, term Intervention x T1) and intervention effect moderated by school grade (Model B, terms Intervention x 2nd/3rd grade x T1, 1st grade as the reference)
Table 5 Intervention effect on school children’s socio-emotional skills and psychological problems among girls. Regression estimates from multilevel models: intervention effect (Model A, term Intervention x T1) and intervention effect moderated by school grade (Model B, terms Intervention x 2nd/3rd grade x T1, 1st grade as the reference)

The first set of models (Models A, Tables 3, 4 and 5) addressed the question of an intervention effect on the outcome variables across all grades by studying the Intervention x T1 interaction terms. The positive values of these interaction terms indicate that the average change from T0 to T1 corresponds to a larger increase in the outcome variable in the intervention group compared to the control group; similarly negative values indicate a relatively larger decrease in the outcome in the intervention group. Thus, for the intervention to be effective the Intervention x T1 interaction terms need to be positive (and significant) on the socio-emotional skills outcomes and negative on the psychological problems outcome. Inspection of these terms in Models A (Tables 3, 4 and 5) indicated no significant intervention effects.

In the second set of models (Models B, Tables 3, 4 and 5), the moderating role of grade on the intervention effect was examined using interaction terms between intervention, grade and time. These models indicated differences in intervention effects on SDQ psychological problems between third and first graders (the reference group) as marked by the significant Intervention x 3rd grade x T1 interaction term (Table 3). Stratified analyses by gender indicated further, that this interaction term was prevalent only among boys and that among them also the corresponding interaction term relating to MASCS cooperation skills between third and first graders was significant (Table 4). To interpret these interaction terms, separate models were specified for each grade level among boys. The results from these models showed that the intervention was effective in reducing psychological problems among third grade boys (regression estimate −0.994, p = 0.025), while among first grade boys the effect was close to zero and non-significant (0.294, p = 0.364). Regarding cooperation skills, the intervention had a marginally significant positive effect on increasing them among third grade boys (0.528, p = 0.078), whereas for the first graders the effect was slightly negative, but again not significantly different from zero (-0.328, p = 0.234). There were no other significant Intervention x grade x T1 interaction terms among boys or girls, indicating no other intervention effects moderated by grade. This was also tested between the third and second grades in additional models (not shown).

The last set of analyses examined the moderating role of the intervention dosage on the intervention effects (Table 6). As indicated by the non-significant dosage x T1 interaction terms, intervention implemented below the intended intensity level was not effective, which means that changes in the outcome measures in this group were not significantly different from the changes that took place in the control group. However, among girls the group who received the intervention as intended showed a significant increase in MASCS cooperation skills (interaction term estimate 0.586, p = 0.018) and a marginally significant increase in SDQ prosocial behavior (0.404, p = 0.053) compared to the control group girls. Similar results were observed for the total sample. Among boys, the intervention effects for the group who received the intervention as intended were in the expected direction, but did not reach the level of statistical significance.

Table 6 Intervention effect on school children’s socio-emotional skills and psychological problems moderated by intervention dosage, regression estimates from multilevel models. Intervention effect moderated by levels of intervention dosage (intensity) with control group as the reference: terms Intervention below/intended intensity x T1

Discussion

The Together at School program was designed for primary-school children in order to promote socio-emotional skills and prevent psychological problems in a whole school context. The findings reported here represent the first results concerning the short-term effectiveness of this universal school-based program.

In their meta-analysis of school-based universal SEL intervention programs Durlak et al. [14] reported significant effects of these programs in increasing socio-emotional skills and also in reducing conduct problems and emotional distress, although to a lesser degree. As a whole, we found no similar intervention effects of the Together at School program in improving primary school children’s socio-emotional skills or in reducing their psychological problems 6 months from the baseline. The lack of main effects in our study may be due to the short follow-up period. It is well known that behavioral changes may require a relatively long learning period and/or that they may appear only later on [38]. Similarly, it takes time and energy on the part of the teachers and principals to take in, process and implement a new method in the school curriculum, which might also explain the lack of intervention effects at this point. Indeed, the idea behind this universal whole school intervention program is to produce mental health effects in the longer term by incorporating the program into the teachers’ and school staff’s continuous daily work practices, eventually becoming an integral part of the school curriculum and children’s school environment. Thus, the intervention is likely to need a longer time to display the positive effects it was planned for, and our future task will be to evaluate the program’s effectiveness after a longer time period at the forthcoming 18-month follow-up point.

On the other hand, positive intervention effects have been reported already after relatively short intervention periods, for example in a classroom-based intervention (Incredible Years) for preschool children [39]. However, comparing the studies is difficult, because the settings, targeted children (all vs. selected most problematic), and the age groups, have been quite different. Especially, in a universal intervention, like the Together at School, where the whole group is targeted, and also the targeted behaviors are at a reasonably good level to begin with, any improvements are likely to be smaller and/or require more time to develop, whereas targeting only higher risk children would produce more dramatic intervention effects [16]. Were we to analyze only those children with the lowest levels of socio-emotional skills or highest levels of psychological problems at baseline, we might have been able to observe stronger effects already at this point.

While not showing intervention effects across all grades, our analyses indicated that the Together at School program was effective among third graders in reducing their psychological problems. Stratified analyses showed that this effect was significant among boys only, and that among them the intervention seems to be effective also in improving third graders’ cooperation skills, although regarding this latter effect the results were only suggestive. This result was somewhat unexpected, as previous reviews had recommended starting school interventions, particularly those aiming to develop generic social and emotional skills, early with young children [16]. Since there should not be large qualitative developmental differences between first and third grade boys (i.e. between 7 and 9 year old boys), our result is likely to be related either to the contents of the intervention itself or the school system, or both. The latter might be more relevant here, since in the Finnish school system there is a “leap” between the second and third grades as the curriculum becomes more academically oriented from the third grade onwards. Moreover, more complex socio-emotional skills are required from the children as they move from first to second, and then to the third grade. It might be that for boys these changes are more challenging, thus leaving more room for a SEL intervention to have positive effects among them in the third grade. In line with this, one possible explanation for the absence of similar findings among girls might be their higher levels of socio-emotional skills and lower levels of psychological problems already at baseline.

The question whether and to what extent factors relating to intervention implementation play a role in intervention effectiveness has been recently brought up in the literature [15]. In the present study we addressed this question by analyzing the modifying role of intervention dosage, i.e. the amount of intervention methods and tools used by the teachers in their classes, on the intervention effectiveness. We expected the intervention to be effective more likely when administered with a high enough dosage. In line with this, the results indicated that when the intervention was carried out with the intended intensity, intervention effects were found on cooperation skills. The stratified analyses indicated further, that this effect on cooperation skills was significant only in girls, and that among them the intervention had also a marginally significant effect on prosocial behavior when delivered with the intended intensity. No effects were observed when the intervention was implemented below the intended intensity level. These results are in line with some earlier studies suggesting that a high enough dosage is needed for the intended intervention effects to occur [16, 21]. This may be the case especially in the universal and whole-school approaches, where the quality of school environment is considered to be an integral part of positive child development [16]. An important issue in our future studies will be to develop a finer grained analysis of which methods and tools of the Together at School intervention program are the most relevant for positive intervention effects, and at which level of dosage. While the results of the present study represent only the first short-term findings, they suggest however, at face value, that intervention efforts should be carried out with fidelity and commitment, while there seems to be no point in using these methods only to a low or moderate degree. This finding, especially if it prevails in the longer follow-up analyses, would also be important when making decisions at the policy level–in other words, it would only be worthwhile investing in the program if the required resources for its proper implementation are allocated.

Strengths and limitations

The strengths of this study are found in the fact that the Together at School intervention program has been carefully developed and tested for several years in a real world school context. Moreover, the program has been adapted specifically for the national school system and culture. We also consider it important that the program is based on the whole school approach and that the methods are integrated as an imminent part of the curriculum, aimed toward instigating profound long-term changes in the practices and ethos of the school.

The strengths of the present study were also found in the large sample size and randomized-controlled study design. The proportion of children with parental consent to participate in the data collection was relatively high (82.3 %). According to teacher reports from ten selected schools with the lowest consent percentages, the reasons for nonconsent related usually to difficulties in school/teacher-parent communication, cultural/language challenges, or parental economic stress. This might be an indication of selective non-response, and as such might have had some influence on the results (possibly lowering the observed effects). The outcome measures that were used have been validated in the Finnish context.

A limitation of the present paper is that we used only the teacher ratings of the children: the fact that the teachers both delivered the intervention and rated the children could have led to some bias, which should be kept in mind when interpreting the results. The decision to use only teacher rating data at this point was made as the parent rating data included more missing information and would have led to a considerable reduction in the number of cases in our analyses.

Contamination poses a possible risk in RCTs diminishing any observable effects of a potentially effective intervention. To avoid this we conducted randomization at the school level. It has also been indicated that contamination is less likely when the intervention method itself is rather complex and/or aims at behavior changes [40]. On the other hand our control group was not a “pure” no-treatment group, but was given lectures on the same themes that the intervention was targeting. In addition, the lecturers reported (from informal discussions with the lecturers) that during the lectures the control group teachers shared actively with each other their experiences of supporting children’s wellbeing and social and emotional skills, indicating that some of them already used some kind of methods comparable to the intervention and also that they were highly motivated in the topics related to supporting child’s socio-emotional development (also evident by their being participants of the study, albeit in the control group). These characteristics of the control condition could have made it less optimal for the intervention effects to be observed.

The follow-up period was short, being in practice between 4 and 6 months. Nevertheless, the results are in line with the prior feasibility study [27] providing further reassurance that the intervention program can be considered as safe in that there were no negative effects in the studied outcomes. As the recent study by Stallard et al. [41] have pointed out, it is important to keep in mind that interventions can also be potentially harmful. That some of our analyses were not specified as primary or planned analyses in the study protocol [20] is a limitation. This holds for both the separate analyses for boys and girls as well as the analyses of the moderating role of dosage on the intervention effectiveness.

In addition to the studied modifiers of the intervention effect (grade level and intervention dosage), also other factors regarding program implementation and teacher behavior may have been contributing to the program effects. For example, those teachers implementing the intervention with the intended intensity might differ in other relevant aspects (e.g. motivational, personality characteristics, etc.) from their colleagues who implemented the intervention below the intended level, and this induces the possibility that the effects in the outcomes are not totally related to the intervention itself. Therefore, more detailed analyses are needed regarding whether the interplay between intervention-related variables and the aforementioned teacher characteristics, as well as the children’s background characteristics, e.g. prior socio-emotional skills and psychological problem levels, has an influence on the intervention program effect. These factors remain as important study questions for further studies.

Conclusions

The present study reported the short-term results of the Together at School intervention program, a universal school intervention on children’s socio-emotional skills delivered by teachers under real world conditions and integrated to normal classroom education and the school curriculum in a whole school context. No main intervention effects were observed after a 6 month intervention period. Those in the third grade, especially the boys, seemed to benefit from the program, indicating that the grade level where the intervention program is implemented might be a factor in the program’s effectiveness. The results also indicate that for this type of universal intervention program to be effective, it is important that the intervention is delivered with a high enough dosage. These and other modifiers of the potential effectiveness of the Together at School intervention program as well as its long-term effectiveness will be addressed in up-coming follow-up studies.

Notes

  1. Due to practical and organizational reasons the baseline phase of the study spanned over a period of two months. Consequently the follow-up times varied in practice between 4 and 6 months. For the sake of clarity we refer to this 4–6 month measurement point as “6 months” in the text.

Abbreviations

ICC:

intraclass correlation

MASCS:

multisource assessment of social competence scale

RCT:

randomized controlled trial

SD:

standard deviation

SDQ:

strengths and difficulties questionnaire

SEL:

socio-emotional learning

SSBS:

school social behavior scale

References

  1. Almqvist F, Kumpulainen K, Ikäheimo K, Linna S, Henttonen I, Huikko E, et al. Behavioural and emotional symptoms in 8–9-year-old children. Eur Child Adolesc Psychiatry. 1999;8(4):S7–S16.

    Article  Google Scholar 

  2. Costello EJ, Egger H, Angold A. 10-year research update review: the epidemiology of child and adolescent psychiatric disorders: I. Methods and public health burden. J Am Acad Child Adolesc Psychiatry. 2005;44(10):972–86.

    Article  PubMed  Google Scholar 

  3. Costello EJ, Foley DL, Angold A. 10-year research update review: the epidemiology of child and adolescent psychiatric disorders: II. Developmental epidemiology. J Am Acad Child Adolesc Psychiatry. 2006;45(1):8–25.

    Article  PubMed  Google Scholar 

  4. Kessler RC, Avenevoli S, Merikangas KR. Mood disorders in children and adolescents: an epidemiologic perspective. Biol Psychiatry. 2001;49(12):1002–14.

    Article  PubMed  Google Scholar 

  5. Merikangas KR, Nakamura EF, Kessler RC. Epidemiology of mental disorders in children and adolescents. Dialogues Clin Neurosci. 2009;11(1):7–20.

    PubMed  PubMed Central  Google Scholar 

  6. Hyland L. The Incredible Years Teacher Classroom Management Programme in Ireland: A process evaluation and observational assessment of teacher-pupil outcomes [dissertation]. Maynooth: National University of Ireland; 2014. http://eprints.maynoothuniversity.ie/4998/1/Lynda_Hyland_PhD.pdf.

  7. Tremblay RE, Nagin DS, Seguin JR, Zoccolillo M, Zelazo PD, Boivin M, et al. Physical aggression during early childhood: trajectories and predictors. Pediatrics. 2004;114(1):e43–50.

    Article  PubMed  PubMed Central  Google Scholar 

  8. Farrell LJ, Barrett PM. Prevention of childhood emotional disorders: Reducing the burden of suffering associated with anxiety and depression. Child Adolesc Mental Health. 2007;12(2):58–65.

    Article  Google Scholar 

  9. Sourander A, Niemelä S, Santalahti P, Helenius H, Piha J. Changes in psychiatric problems and service use among 8-year-old children: a 16-year population-based time-trend study. J Am Acad Child Adolesc Psychiatry. 2008;47(3):317–27.

    Article  PubMed  Google Scholar 

  10. Stockings EA, Degenhardt L, Dobbins T, Lee YY, Erskine HE, Whiteford HA, et al. Preventing depression and anxiety in young people: a review of the joint efficacy of universal, selective and indicated prevention. Psychol Med. 2016;46(01):11–26.

    Article  PubMed  Google Scholar 

  11. Bierman KL, Coie JD, Dodge KA, Greenberg MT, Lochman JE, McMahon RJ, et al. The effects of a multiyear universal social–emotional learning program: The role of student and school characteristics. J Consult Clin Psychol. 2010;78(2):156–68.

    Article  Google Scholar 

  12. Greenberg MT, Domitrovich C, Bumbarger B. The prevention of mental disorders in school-aged children: Current state of the field. Prev Treat. 2001;4(1):1a.

    Google Scholar 

  13. Elias MJ, Zins JE, Graczyk PA, Weissberg RP. Implementation, sustainability, and scaling up of social-emotional and academic innovations in public schools. Sch Psychol Rev. 2003;32(3):303–19.

    Google Scholar 

  14. Durlak J, Weissberg R, Dymnicki A, Taylor R, Schellinger K. The Impact of Enhancing Students’ Social and Emotional Learning: A Meta-Analysis of School-Based Universal Interventions Social and Emotional Learning. Child Dev. 2011;82(1):405–32.

    Article  PubMed  Google Scholar 

  15. Jones SM, Bouffard SM. Social and Emotional Learning in Schools: From Programs to Strategies. Soc Policy Rep. 2012;26(4):1–22.

    Google Scholar 

  16. Weare K, Nind M. Mental health promotion and problem prevention in schools: what does the evidence say? Health Promot Int. 2011;26 Suppl 1:i29–69.

    Article  PubMed  Google Scholar 

  17. Wells J, Barlow J, Stewart-Brown S. A systematic review of universal approaches to mental health promotion in schools. Health Educ. 2003;103(4):197–220.

    Article  Google Scholar 

  18. Aber L, Brown JL, Jones SM, Berg J, Torrente C. School-based strategies to prevent violence, trauma, and psychopathology: The challenges of going to scale. Dev Psychopathol. 2011;23(02):411–21.

    Article  PubMed  Google Scholar 

  19. Kampman M, Solantaus T, Karlsson L, Marttunen M. Lasten ja nuorten sosioemotionaalisen kehityksen tukeminen ja masennuksen ennaltaehkäisy koulussa [Supporting the socio-emotional development of children and adults and preventing depression in schools]. In: Moring J, Martins A, Partanen A, Bergman V, Nordling E, Nevalainen V, editors. Kansallinen mielenterveys- ja päihdesuunnitelma 2009–2015: Toimeenpanosta käytäntöön vuonna 2010 [National plan for mental health and substance abuse work 2009–2015: From Implementation to Practice in 2010]. Report 006/2011. Helsinki: National Institute for Health and Welfare (THL); 2011. p. 81–9.

    Google Scholar 

  20. Björklund K, Liski A, Samposalo H, Lindblom J, Hella J, Huhtinen H, et al. “Together at school”-a school-based intervention program to promote socio-emotional skills and mental health in children: study protocol for a cluster randomized trial. BMC Public Health. 2014;14:1042.

    Article  PubMed  PubMed Central  Google Scholar 

  21. Reyes MR, Brackett MA, Rivers SE, Elbertson NA, Salovey P. The interaction effects of program training, dosage, and implementation quality on targeted student outcomes for the RULER approach to social and emotional learning. School Psych Rev. 2012;41(1):82–99.

    Google Scholar 

  22. Charlebois P, Brendgen M, Vitaro F, Normandeau S, Boudreau J. Examining dosage effects on prevention outcomes: Results from a multi-modal longitudinal preventive intervention for young disruptive boys. J Sch Psychol. 2004;42(3):201–20.

    Article  Google Scholar 

  23. Rosenblatt JL, Elias MJ. Dosage effects of a preventive social-emotional learning intervention on achievement loss associated with middle school transition. J Prim Prev. 2008;29(6):535–55.

    Article  PubMed  Google Scholar 

  24. Almqvist F, Puura K, Kumpulainen K, Tuompo-Johansson E, Henttonen I, Huikko E, et al. Psychiatric disorders in 8–9-year-old children based on a diagnostic interview with the parents. Eur Child Adolesc Psychiatry. 1999;8(4):S17–28.

    Article  Google Scholar 

  25. Junttila N. Social competence and loneliness during the school years. Issues in assessment, interrelations and intergenerational transmission [dissertation]. Turku, Finland: University of Turku; 2010. http://www.doria.fi/handle/10024/59283.

  26. Paananen R, Santalahti P, Merikukka M, Rämö A, Wahlbeck K, Gissler M. Socioeconomic and regional aspects in the use of specialized psychiatric care - a Finnish nationwide follow-up study. Eur J Public Health. 2013;23(3):372–7.

    Article  PubMed  Google Scholar 

  27. Appelqvist-Schmidlechner K, Liski A, Kampman M, Together at School work group: Solantaus T, Santalahti P, Anttila N, Björklund K, et al. Yhteispeli: Arviointitutkimus menetelmien turvallisuudesta, soveltuvuudesta ja koetusta hyödystä [Evaluation study on the safety, feasibility, and perceived benefits of the “Together at School” intervention programme]. Discussion paper 9/2015. Helsinki: National Institute for Health and Welfare (THL); 2015.

    Google Scholar 

  28. Goodman R. The Strengths and Difficulties Questionnaire: a research note. J Child Psychol Psychiatry. 1997;38(5):581–6.

    Article  PubMed  Google Scholar 

  29. Goodman R, Scott S. Comparing the Strengths and Difficulties Questionnaire and the Child Behavior Checklist: is small beautiful? J Abnorm Child Psychol. 1999;27(1):17–24.

    Article  PubMed  Google Scholar 

  30. Goodman R. Psychometric properties of the strengths and difficulties questionnaire. J Am Acad Child Adolesc Psychiatry. 2001;40(11):1337–45.

    Article  PubMed  Google Scholar 

  31. Koskelainen M, Sourander A, Kaljonen A. The Strengths and Difficulties Questionnaire among Finnish school-aged children and adolescents. Eur Child Adolesc Psychiatry. 2000;9(4):277–84.

    Article  PubMed  Google Scholar 

  32. Borg A, Kaukonen P, Joukamaa M, Tamminen T. Finnish norms for young children on the Strengths and Difficulties Questionnaire. Nord J Psychiatry. 2014;68(7):433–42.

    Article  PubMed  Google Scholar 

  33. Borg A, Kaukonen P, Salmelin R, Joukamaa M, Tamminen T. Reliability of the Strengths and Difficulties Questionnaire among Finnish 4-9-year-old children. Nord J Psychiatry. 2012;66(6):403–13.

    Article  PubMed  Google Scholar 

  34. Junttila N, Voeten M, Kaukiainen A, Vauras M. Multisource assessment of children’s social competence. Educ Psychol Meas. 2006;66(5):874–95.

    Article  Google Scholar 

  35. Merrell KW, Gimpel G. Social skills of children and adolescents: Conceptualization, assessment, treatment. Mahwah: Lawrence Erlbaum; 1998.

    Google Scholar 

  36. Rasbash J, Charlton C, Browne WJ, Healy M, Cameron B. MLwiN Version 2.1. Bristol: Centre for multilevel modelling, University of Bristol; 2009.

    Google Scholar 

  37. Raudenbush SW, Bryk AS. Hierarchical linear models: applications and data analysis methods. 2nd ed. Newbury Park: Sage; 2002.

    Google Scholar 

  38. Jones SM, Brown JL, Hoglund WL, Aber JL. A school-randomized clinical trial of an integrated social–emotional learning and literacy intervention: Impacts after 1 school year. J Consult Clin Psychol. 2010;78(6):829.

    Article  PubMed  Google Scholar 

  39. Baker-Henningham H, Scott S, Jones K, Walker S. Reducing child conduct problems and promoting social skills in a middle-income country: cluster randomised controlled trial. Br J Psychiatry. 2012;201:101–8.

    Article  PubMed  PubMed Central  Google Scholar 

  40. Keogh-Brown MR, Bachmann MO, Shepstone L, Hewitt C, Howe A, Ramsay CR, et al. Contamination in trials of educational interventions. Health Technol Assess. 2007;11(43):iii. ix-107.

    Article  PubMed  Google Scholar 

  41. Stallard P, Sayal K, Phillips R, Taylor JA, Spears M, Anderson R, et al. Classroom based cognitive behavioural therapy in reducing symptoms of depression in high risk adolescents: pragmatic cluster randomised controlled trial. BMJ. 2012;345:e6058.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

The research team would like to thank all the members of the Tampere Unit for Computer-Human Interaction (TAUCHI) at the School of Information Sciences at the University of Tampere for their valuable contribution and collaboration regarding the assessments and the electronic data collection procedure used in the Together at School study. Furthermore, the research team would also thank all the participating schools, their principals and teachers, the children and their parents and all the assistants who participated in the data collection.

Funding

The main funding of this project came from the Finnish Ministry of Education and Culture.

Availability of data and materials

Data sets on which the analyses and conclusions of the present study rely are available from the authors.

Authors’ contributions

All the authors contributed to the content of this paper. More specifically, TO and PA were in charge of the content of the intervention program; AL and PS were in charge of the study design; AL, RS, OK and HLK were in charge of the statistical expertise; HLK did the statistical analyses; KB, JL, RLP and PS were in charge of the psychological and child psychiatric expertise; AL, HS, KB, HK, NH, TO and HLK participated in the data collection; HS, AL and PA were in charge of the data collection procedure; EH and PS were in charge of the epidemiological expertise; EH, RLP and RS provided expertise on randomized controlled trials; KB, JL, RLP and PS were in charge of the measures; NH was in charge of the administration of project; and KB and OK were in charge of the writing and the manuscript as a whole. All authors contributed to the refinement of the study protocol, and have read and approved the final manuscript. KB, OK and PS finalized the manuscript.

Competing interests

The authors declare that they have no competing interests.

Consent for publication

Not applicable.

Ethics approval and consent to participate

The study protocol was approved by the Ethics Committee of the National Institute for Health and Welfare in Helsinki, Finland (27.9.2012). All parents of the participating classes received an information letter regarding the intervention program and aims of the study. The parents were informed about the voluntary nature of the participation in the data collection and a consent form for data collection was included in the information letter. The teachers and principals consented by agreement.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Olli Kiviruusu.

Additional information

Olli Kiviruusu and Katja Björklund are joint first authors

Additional files

Additional file 1:

Frequencies of psychological problems (SDQ total) by gender, group status and grade at baseline (T0) and 6 months (T1). (DOC 100 kb)

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kiviruusu, O., Björklund, K., Koskinen, HL. et al. Short-term effects of the “Together at School” intervention program on children’s socio-emotional skills: a cluster randomized controlled trial. BMC Psychol 4, 27 (2016). https://doi.org/10.1186/s40359-016-0133-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s40359-016-0133-4

Keywords