Open Access
Open Peer Review

This article has Open Peer Review reports available.

How does Open Peer Review work?

Psychometric properties of self-sufficiency assessment tools in adolescents in vocational education

  • Rienke Bannink1Email author,
  • Suzanne Broeren1,
  • Jurriën Heydelberg2,
  • Els van’t Klooster3 and
  • Hein Raat1
BMC Psychology20153:33

https://doi.org/10.1186/s40359-015-0091-2

Received: 20 April 2015

Accepted: 17 September 2015

Published: 25 September 2015

Abstract

Background

Self-sufficiency is the realisation of an acceptable level of functioning either by the person him/herself or through the adequate organisation of help from informal or formal care providers. Assessment of self-sufficiency for determining an individual’s functional strengths and areas for improvement is increasingly being applied among adolescents in vocational education, a group considered vulnerable with high school dropout rates and often characterised by an accumulation of problems. This study examined the psychometric properties of two instruments, i.e. a self-report questionnaire assessing self-sufficiency and the Self-Sufficiency Matrix for professionals (SSM-D) conducted among adolescents in vocational education.

Methods

The self-report questionnaire used to assess self-sufficiency was completed by 581 adolescents. Professionals completed the SSM-D for 224 of the 581 adolescents. Furthermore, constructs related to the domains of self-sufficiency were assessed with self-report questionnaires and information about school absenteeism was monitored via the school registration system.

Results

For both self-report and professional-report ratings, the internal consistency was satisfactory (Cronbach’α > 0.70) and various minor to strong correlations were found between the domains of self-sufficiency and related constructs. For most of the domains, there was little or no agreement between professionals and adolescents.

Conclusions

Both the self-report questionnaire assessing self-sufficiency and the SSM-D applied in this study seem to possess adequate psychometric properties. The results indicated that adolescents and professionals provide different views of adolescents’ self-sufficiency, which merits further study. In the meantime, we recommend assessment of adolescents’ self-sufficiency by using both the self-report questionnaire and the SSM-D to get a comprehensive measure of adolescents’ self-sufficiency.

Trial registration

Netherlands Trial Register: NTR3545; 30 July 2012.

Keywords

Psychometrics Reliability Validity Self-sufficiency Adolescents Self-sufficiency matrix

Background

Mental health problems are highly prevalent in adolescents, and risk behaviours, such as substance abuse and truancy, are often acquired during adolescence [1]. These problems and behaviours can negatively affect the functioning of adolescents in different life domains [2]. Furthermore, mental health problems and risk behaviours often do not occur in isolation in adolescents, but are associated with each other and accumulate [39]. The co-occurrence of mental health problems and risk behaviours, and the influence that these problems and behaviours have on the functioning of adolescents in various life domains, suggests that professionals should preferably address problems and risk behaviours in multiple life domains simultaneously. However, to date most intervention programmes and assessment tools take a single-problem/risk-behaviour/life-domain approach instead of an integrated approach [10].

A Self-Sufficiency Matrix (SSM) is an instrument that has adopted such an integrated approach [11, 12]. The basis for the SSM was developed in the 1990s in the United States. It is a standardised tool for measuring self-sufficiency. Self-sufficiency is defined as the realisation of an acceptable level of functioning either by the person him/herself or through the adequate organisation of help from informal or formal care providers [2]. A standardised tool to measure economic self-sufficiency was first developed by Pearce et al. [13]. This economic self-sufficiency measure was then extended to include a number of domains, resulting in the first published version of a multidimensional SSM in 2004 [14]. Different versions of the SSM are currently being used in different settings. The SSM can be used by professionals as a screening tool during consultations for determining functional strengths and areas for improvement in, for example, vulnerable adolescents. It expresses functioning in terms of levels of self-sufficiency in several domains (e.g. mental health and social network) [2]. The SSM is a screening or assessment tool that is often used also to measure outcomes of intervention programmes in populations experiencing multiple interlinked problems.

Although the SSM is applied in the United States [11, 12] and is quickly gaining popularity in other countries as well [15], to the best of our knowledge there is only one study available that examines the psychometric properties of the SSM. Fassaert et al. [2] showed that an adapted 11-domain version of the SSM (SSM-D), based on Utah and Arizona versions of the SSM, is a reliable instrument for assessment by professionals of the self-sufficiency of adolescents (>18 years) with severe and complex psychiatric problems. As the SSM is also increasingly used among other populations, such as adolescents in vocational education (≥15 years), further evaluation of the psychometric properties of the SSM among other populations is needed. This study focuses on these adolescents in senior vocational education, a group that is considered vulnerable. In the Netherlands, 75 % of school dropouts occur in senior vocational education [16]. Furthermore, many adolescents in vocational education experience problems, such as debts and substance abuse, and these problems often accumulate [3, 4, 17].

So far, the SSM is only available for professionals to complete during consultations. However, previous research has shown low correlations between different informants (e.g. adolescents and professionals) when assessing problems, and that a valuable unique contribution can be made by different informants [1822]. Hence, assessment of self-sufficiency by means of a questionnaire for adolescents alongside a proxy rating by a professional could give a more comprehensive measure of adolescents’ self-sufficiency. Therefore, this study employed assessment of self-sufficiency by means of a questionnaire for adolescents in addition to assessment of self-sufficiency by a proxy rating provided by professionals.

The purpose of this study was to assess the psychometric properties of a self-report questionnaire assessing self-sufficiency and the SSM-D in a group of vulnerable adolescents (i.e. in vocational education). This study investigated: (1) internal consistency of both instruments assessing self-sufficiency (i.e. self-report questionnaire and SSM-D), and (2) correlations between adolescents’ and professionals’ ratings in domains of self-sufficiency and related constructs (concurrent validity). Additionally, we examined the degree of agreement between adolescent and professional ratings in the domains of self-sufficiency.

Since there are some conceptual differences between the domains of self-sufficiency and the related constructs that were used to assess concurrent validity (e.g. finances and debts), minor to strong correlations are expected depending on the level of overlap between the constructs under study. In line with previous studies on adolescents’ psychopathology that measured agreement between informants [18, 19, 21, 22], we hypothesise that the degree of agreement between adolescents and professionals in the domains of self-sufficiency will be fair at most. Low levels of agreement between adolescents and professionals could indicate that these informants cannot be substituted for one another because they provide unique information [18].

Methods

Data collection

This study used data obtained from enrolments in the Your Health study, a cluster randomised controlled trial (Trial registration: www.trialregister.nl; Netherlands Trial Register: NTR 3545; 30 July 2012). A total of 44 first-year classes of students in vocational education in the Rotterdam region of the Netherlands participated. School classes (clusters) were randomly assigned to the Your Health or the control condition. The intervention study itself is described in detail elsewhere [23]. A few weeks prior to the start of the study, all adolescents and parents received information about the study. Parents were asked passive written informed consent. If parents did not want their child to participate, and their child was not yet 18 years old, they could object to the child’s participation. During a classroom session, adolescents who were present in class were asked to provide active written informed consent before they completed a set of questionnaires. The set of questionnaires included the self-report questionnaire assessing self-sufficiency and questionnaires assessing the related constructs. After the questionnaires had been administered, school classes were randomly assigned to the Your Health or the control condition. Adolescents in the intervention group were invited to attend a preventive health consultation with the school nurse. During this consultation, the nurse used the SSM-D and rated the self-sufficiency of the adolescent.

Of the 830 adolescents who received information about the study, 584 (70.4 %) were present at the time of assessment, provided written informed consent and participated; 280 in the Your Health group and 304 in the control group. The main reason for non-participation was absence at the time of the assessment. The questionnaire used to assess self-sufficiency was completed by 581 of the 584 (99.5 %) participating adolescents. Of the 280 adolescents who were invited to attend a consultation, 224 (80.0 %) attended (see Fig. 1).
Fig. 1

Flow chart of the adolescent’s participation

Ethics statement

The Medical Ethical Committee of Erasmus MC has reviewed the research proposal for this study and declared that this study does not fall within the ambit of the Medical Research Involving Human Subjects Act (also known by its Dutch abbreviation “WMO”) and, therefore, does not require further approval of an ethics review board. The Medical Ethical Committee had no objection against the execution of this research proposal (MEC-2012-367).

Measurements

Assessment of self-sufficiency by professionals

The Dutch version of the SSM (SSM-D) was used to assess an individual’s level of self-sufficiency in 11 life domains: finances, day-time activities, housing, domestic relations, mental health, physical health, addiction, activities daily life, social network, community participation, and judicial [24, 25]. Each of the domains was measured by a single item and the level of self-sufficiency was rated on a 5-point scale: 1 = ‘acute problem’, 2 = ‘not self-sufficient’, 3 = ‘barely self-sufficient’, 4 = ‘adequately self-sufficient’, and 5 = ‘completely self-sufficient’. Indicators that specify each level of self-sufficiency were defined for each domain. Together, these indicators form a matrix of domains and levels of self-sufficiency [2, 24]. For an example of the indicators of an SSM-D domain (i.e. finances), see Table 1. Prior to the consultations, nurses were trained to work with the SSM-D.
Table 1

Example of an indicator in the Dutch Self-Sufficiency Matrix: Finances

Rating

Label

SSM-D description

1

Acute problem

No income. High, increasing debts.

2

Not self-sufficient

Insufficient income and/or spontaneous or inappropriate spending. Increasing debts.

3

Barely self-sufficient

Can meet basic needs with income and/or appropriate spending. If there are debts, they are at least stable and/or controlled by a third party.

4

Adequately self-sufficient

Meets basic needs without receiving social security benefits. Manages possible debts without assistance and they are decreasing.

5

Completely self-sufficient

Income is ample, well managed. Has the ability to save with income.

Note: Copyright 2012 by GGD Amsterdam. Reprinted with permission

Assessment of self-sufficiency by adolescents

A self-report questionnaire assessing self-sufficiency was developed based on the 11-domain version of the SSM-D. Each domain name was translated into simple language, and a short description was provided describing the content of each domain in simple language. Simple language was used because some adolescents may have relatively poor reading skills. Subsequently, based on group discussions and consensus between professionals, language adjustments were made and the response scale of the professional version was simplified. Professionals indicated that the word ‘self-sufficiency’, used in the response scale of the SSM-D, would be too difficult for adolescents to understand. Therefore, the response scale was replaced in a simple 5-point Likert scale: 1 = ‘no problems’, 2 = ‘few problems’, 3 = ‘not few/not many problems’, 4 = ‘many problems’, and 5 = ‘very many problems’. Furthermore, a smiley was displayed with each response option to support adolescents with poor reading skills. Finally, a pilot was conducted among the target group (i.e. adolescents in vocational education) to examine whether the language and the response scale used were clear, and whether the instrument was usable in this group. No further adjustments were needed based on this pilot.

Our self-report questionnaire differs in some respects from the SSM-D. First, the self-report questionnaire provides a short description for each domain, but it does not define any indicators specifying each level of self-sufficiency as is the case in the SSM-D. Second, the self-report questionnaire has a different 5-point response scale (with smileys) than the SSM-D. For an example of a domain of the self-report questionnaire (i.e. finances), see Table 2.
Table 2

Example of a self-sufficiency domain in the self-report questionnaire: Finances

Description

Rating

    

Did you experience problems getting by financially over the past six months?

No problems ☺ ☺

Few problems ☺

Not few/not many problems

Many problems

Very many problems

Demographics

Demographic characteristics included the age, gender, country of birth of the adolescent and both parents, and whether or not the adolescent already was a parent him/herself. Ethnicity was classified as Dutch or non-Dutch, in accordance with the definitions used by Statistics Netherlands [26].

Related constructs

Debts, homelessness, alcohol consumption, soft drug use and delinquency were assessed by items based on existing instruments previously developed by Municipal Public Health Services and health institutes in the Netherlands [27]. To reduce respondent burden, only a number of related construct were assessed. No data was obtained on 3 domains of self-sufficiency (i.e. domestic relations, activities daily life, and social network).

Debts

Debts were assessed on an ordinal scale by the following items: (1) do you have debts? (yes/no/don’t know), and (2) approximately how high is the sum of all your debts? (less than 50 euros – more than 2,500 euros).

Homelessness

Homelessness was assessed by the item: “Have you been homeless in the past three months? This means that you had no perspective, for at least one night per month, of a permanent place to sleep.” (yes/no).

Alcohol and soft drugs

Alcohol consumption was covered by the following two items: (1) how often have you drunk five or more alcoholic drinks on a single occasion over the past four weeks? (never – nine or more times), and (2) how often have you been drunk or tipsy over the last four weeks? (never – 20 or more times). Soft drug use was assessed by how often the adolescent had used soft drugs over the previous four weeks (never – 20 or more times).

Delinquency

Delinquency was assessed by the item: “In the past 12 months, have you been questioned at a police station because you were accused of doing something that was not permitted?” (never – 6 or more times).

Mental health status

Mental health status was assessed by the Mental Health Inventory (MHI-5) [28]. The MHI-5 includes five questions referring to both positive and negative aspects of mental health. All questions contain six possible response categories, scored between 1 and 6. The total score is transformed into a variable range of 0–100, with a score of 100 representing optimal mental health (current study α = 0.69).

Depressive symptoms

Symptoms of depression were assessed by the Center for Epidemiologic Studies Depression Scale (CES-D) [29]. The CES-D consists of 20 items. The frequency of symptoms is rated on a 4-point scale ranging from 0–3. Items scores are summed (range from 0–60), with higher scores indicating higher levels of depressive symptoms (current study α = 0.89).

Health-related quality of life

Health-related quality of life was assessed by the Short Form-12 Health Survey (SF-12). The SF-12 consists of 12 items, with variable response categories across the items. The scores are summarised into two components, corresponding to mental and physical health-related quality of life, with scores ranging from 0 (worst possible health state) to 100 (best possible health state) (current study α = 0.72).

School absenteeism

In the school registration system every hour of absence was registered either as permitted (i.e. because of illness or another valid reason) or not permitted (i.e. without notification or valid reason). Absenteeism was defined as the number of hours adolescents were absent (permitted or not permitted) in a 2-month period around the administration of the questionnaire.

Statistical analyses

Internal consistency was assessed by Cronbach’s alpha, for which a value of ≥0.70 was considered adequate [30]. To determine concurrent validity, ratings for eight domains of both instruments (i.e. self-report questionnaire and SSM-D) assessing self-sufficiency were compared to ratings for related constructs. Concurrent validity was assessed by calculating the rank biserial, polychoric, or polyserial correlation between each domain and related constructs. Rank biserial correlation (rrb) is used to determine the correlation between an ordinal and dichotomous variable. Polychoric correlation (rpc) is used to determine the correlation between two ordinal variables, and polyserial correlation (rps) is used to determine the correlation between a continuous and an ordinal variable [31]. Furthermore, concurrent validity was assessed by calculating Pearson correlations (r) between the total score on SSM-D (which ranges from 11–55) and related constructs, and between the total score on the self-report questionnaire (which ranges from 11–55) and related constructs. The criteria for judging the size of the correlation coefficient suggested by Cohen were applied: correlations <0.30 are considered minor, correlations between 0.3 – 0.49 are considered medium, and ≥0.5 are considered strong [32].

The degree of agreement between professionals and adolescents in each of the domains was determined with weighted kappa with linear weights. Weighted kappa is a measurement of agreement for categorical data with an ordinal level [33]. Linear weighting is used when the difference between each category has the same importance. According to Altman’s guidelines [34], K is poor when it has a value of ≤0.20, fair when it is between 0.21–0.40, moderate when it is between 0.41–0.60, and good when it is ≥0.60.

Statistical analyses were performed using SPSS version 21. Polyserial, polychoric, and rank biserial correlations were calculated in SAS version 9.3. Additionally, polyserial correlations between each domain of self-sufficiency and the total score on SSM-D, and between each domain and the total score on the self-report questionnaire, were assessed (see Addtional file 1).

Results

Adolescents’ characteristics

The self-report questionnaire assessing self-sufficiency was completed by 581 adolescents. The average age of these adolescents was 18.3 years (SD = 2.60); 39.0 % were male, 28.1 % were of Dutch ethnicity, and 10.6 % were parents (Table 3). Professionals completed the SSM-D for 224 of these adolescents. The average age of this subsample of adolescents was 18.3 years (SD = 3.59); 41.7 % were male, 25.3 % were of Dutch ethnicity, and 12.2 % were parents.
Table 3

Demographic characteristics of the study population

 

Self-sufficiency

 

Completed questionnaires - Adolescents

Completed SSM-D - Professionals

 

Total group

Intervention group

 

Number (n)

581

280

224

Mean age; years (SD)

18.27 (2.60)

18.46 (2.65)

18.26 (2.59)

Gender of adolescent (male, %)

39.0

43.0

41.7

Ethnicity (Dutch, %)

28.1

24.9

25.3

Being a parent (yes, %)

10.6

13.4

12.2

Self-sufficiency

A score of “not to barely self-sufficient” can be seen as a level of self-sufficiency that can be improved. The domains in which the professionals deemed the highest percentages of adolescents as being “not to barely self-sufficient” were community participation (36.7 %), domestic relations (15.8 %) and social network (14.5 %) (Table 4). The domains in which the adolescents themselves deemed the highest percentage of adolescents as being “not to barely self-sufficient” were different, namely, finances (23.3 %), domestic relations (17.4 %) and mental health (16.7 %).
Table 4

Professionals’ and adolescents’ ratings of self-sufficiency (n = 224)

 

Not to barely self-sufficienta

Acute problem

Not self-sufficient

Barely self-sufficient

Adequately self-sufficient

Completely self-sufficient

 

%

%

%

%

%

%

Professionals’ ratings (n = 224)

1–3

1

2

3

4

5

Finances

12.9

0.9

4.9

7.1

42.4

44.6

Day-time activities (n = 1 missing)

1.3

0.0

0.0

1.3

85.2

13.5

Housing (n = 3 missing)

9.0

0.0

1.4

7.7

26.7

64.3

Domestic relations (n = 2 missing)

15.8

0.5

3.2

12.2

25.7

58.6

Mental health (n = 1 missing)

8.5

0.0

0.4

8.1

22.4

69.1

Physical health

4.5

0.0

0.4

4.0

29.0

66.5

Addiction

3.6

0.0

0.0

3.6

34.8

61.6

Activities daily life (n = 2 missing)

4.5

0.0

0.0

4.5

29.3

66.2

Social network (n = 3 missing)

14.5

0.9

1.8

11.8

43.4

42.1

Community participation (n = 3 missing)

36.7

0.5

12.7

23.5

47.1

16.3

Judicial (n = 1 missing)

12.6

0.0

5.4

7.2

18.4

69.1

 

Not to barely self-sufficienta

Very many problems

Many problems

Not few/ not many problems

Few problems

No problems

 

%

%

%

%

%

%

Adolescents’ rating (n = 224 b )

1–3

1

2

3

4

5

Finances (n = 1 missing)

23.3

4.9

5.4

13.0

23.3

53.4

Day-time activities (n = 2 missing)

8.1

0.5

1.4

6.3

19.8

72.1

Housing (n = 1 missing)

12.1

2.2

2.2

7.6

6.3

81.6

Domestic relations

17.4

2.7

4.0

10.7

17.9

64.7

Mental health (n = 3 missing)

16.7

3.2

4.1

9.5

17.2

66.1

Physical health (n = 2 missing)

11.3

1.8

2.3

7.2

18.9

69.8

Addiction (n = 5 missing)

7.8

0.9

1.4

5.5

9.1

83.1

Activities daily life

6.3

0.4

0.9

4.9

10.3

83.5

Social network (n = 1 missing)

8.1

0.9

0.9

6.3

12.6

79.4

Community participation (n = 3 missing)

8.1

0.9

2.3

5.0

14.9

76.9

Judicial

5.4

0.4

0.9

4.0

7.1

87.5

aA rating of ≤ 3 is considered as not to barely self-sufficient

bIn this table, only ratings for adolescents for whom a professional rating was available are displayed (n = 224)

Internal consistency

Internal consistency was adequate. The Cronbach’s alpha of the self-report questionnaire was 0.84 and of the SSM-D 0.71.

Concurrent validity

Various minor to strong correlations were found between domains and related constructs (Table 5). All significant correlations were in the hypothesised direction. Correlations between professionals’ ratings of self-sufficiency in the different domains and related constructs varied from no correlation to strong correlations. The strongest correlations were found between the domains of finances and debts (rpc = −0.66), the domains addiction and soft drug use (rpc = −0.53), the domains addiction and alcohol consumption (drunk or tipsy) (rpc = −0.41), and the domains judicial and delinquency (rpc=-0.41).
Table 5

Concurrent validity: correlations between professionals’ and adolescents’ ratings of self-sufficiency and related constructs

Self-sufficiency

Related constructs

Correlation

 
  

With professionals’ self-sufficiency rating (n = 224)

With adolescents’ self-sufficiency rating (n = 581)

Total self-sufficiency score

Mental health-related quality of life (SF-12)a

0.21b

0.46b

 

Physical health-related quality of life (SF-12)a

0.12b,f

0.28b

Finances

Debts

−0.66c

−0.74c

Day-time activities

Not-permitted school absenteeism

−0.26d

−0.17d

 

Permitted school absenteeism

0.01d,f

−0.11d

Housing

Homelessness

−0.41e,f

−0.39e

Mental health

Mental health status (MHI-5)g

0.30d

0.60d

 

Depressive symptoms (CES-D)

−0.33c

−0.59d

 

Mental health-related quality of life (SF-12)a

0.29d

0.54d

Physical health

Physical health-related quality of life (SF-12)a

0.10d,f

0.33d

 

Permitted school absenteeism

−0.08d,f

−0.13d

Addiction

Alcoholic drinks: 5 or more on 1 occasion

−0.30c

−0.39c

 

Alcohol: drunk or tipsy

−0.41c

−0.53c

 

Soft drug use

−0.53c

−0.53c

Community participation

Not-permitted school absenteeism

−0.20d

0.03d,f

 

Permitted school absenteeism

−0.11d,f

−0.04d,f

Judicial

Delinquency

−0.41c

−0.58c

aA higher score indicates a better quality of life

bPearson correlation

cPolychoric correlation

dPolyserial correlation

eRank biseral correlation

fNon-significant correlations; all other correlations were significant at p <0.05

gA higher score indicates less mental health problems

Correlations between adolescents’ ratings of self-sufficiency in the different domains and related constructs also varied from no to strong correlations. Comparable with correlations between professionals’ ratings and related constructs, the strongest correlations between adolescents’ ratings and related constructs were found between the domains of finances and debts (rpc = −0.74), the domains addiction and soft drug use (rpc = −0.53), the domains addiction and alcohol consumption (drunk or tipsy) (rp = −0.53), and the domains judicial and delinquency (rpc=-0.53). In addition, strong correlations were found between the domains of mental health and mental health status (rps = 0.60), depressive symptoms (rps = −0.59) and mental health-related quality of life (rps = −0.54). Furthermore, a strong correlation was found between the total score on the self-report questionnaire and domain mental health-related quality of life (r = 0.46).

Degree of agreement between professionals and adolescents

The degree of agreement between professionals and adolescents varied from no agreement to fair agreement (Table 6). The degree of agreement, in accordance with Altman’s guidelines [34], was fair in four domains: finances (k = 0.22), housing (k = 0.28), domestic relations (k = 0.21), and judicial (k = 0.21). The degree of agreement was poor for the domains Day-time activities (k = 0.07), mental health (k = 0.15), physical health (k = 0.17), and addiction (k = 0.18). No agreement (all ps > 0.05) was found in the three remaining domains (activities daily life, social network, and community participation).
Table 6

Degree of agreement between professionals’ and adolescents’ ratings of self-sufficiency (n = 224)

Self-sufficiency

Weighted kappa

Finances

0.22

Day-time activities

0.07

Housing

0.28

Domestic relations

0.21

Mental health

0.15

Physical health

0.17

Addiction

0.18

Activities daily life

0.004a

Social network

0.01a

Community participation

−0.003a

Judicial

0.21

aNon-significant correlations; all other correlations were significant at p <0.01

Discussion

Both the self-report questionnaire assessing self-sufficiency and the SSM-D applied in this study seem to possess adequate psychometric properties in a group of vulnerable adolescents. The internal consistency was satisfactory. For most of the domains, there was also poor or fair agreement between professionals and adolescents.

More specifically, various minor to strong correlations were found between the domains of both the self-report questionnaire assessing self-sufficiency and SSM-D, on the one hand, and the related constructs, on the other. This is in line with our hypothesis as conceptual differences existed between the domains and the related constructs that were measured. Furthermore, different raters were used and this is reflected by the higher correlations that were found between the domains of the self-report questionnaire assessing self-sufficiency and the adolescent-reported related constructs than between the domains of the SSM-D and the adolescent-reported related constructs.

A low degree of agreement between informants is in line with previous research. For example, a small mean correlation (r = 0.22) between subjects and other informants was found when using questionnaires to measure adolescents’ psychopathology [18]. Furthermore, ‘needs’, as measured with an assessment instrument using areas of life related to the self-sufficiency domains (i.e. Camberwell Assessment of Need), are often assessed differently by professionals and clients [20].

There are several factors that could have decreased the degree of agreement between adolescents and professionals. First, the self-report questionnaire has, as opposed to the SSM-D, no indicators that specify each level of self-sufficiency, which may have contributed to a lower degree of agreement. A second potential explanation is that the subjective norms of professionals and adolescents differ from each other [19]. For example, professionals may have judged an adolescent as not being self-sufficient in the “day-time activities” domain because the adolescent in question had been truanting during the past week, while the adolescent him/herself might not see this as a problem if the truanting only happens occasionally. Third, it could be that adolescents are only partially aware of their problems [21], such as having “bad” friends or being addicted, whereas professionals may be able to assess these problems better. Fourth, professionals cannot observe all aspects of the life of an adolescent and they depend on what the adolescent tells them [19].

The low degree of agreement between adolescents and professionals indicates that both informants can provide different information on the self-sufficiency of adolescents [18]. Since there is no golden standard against which to validate measures of adolescent functioning in the various domains, it is essential to use the contributions of different informants to get a more complete picture of the problems adolescents are dealing with [18]. Having the adolescent complete the self-report questionnaire assessing self-sufficiency prior to the consultation with the professional could encourage professionals to also pay attention to adolescents’ views on their self-sufficiency and help them to determine which crucial aspect (s) to focus their discussions on [35, 36]. Furthermore, previous research has shown that completing a questionnaire on topics that are relevant to the consultation familiarises the adolescent with the topics the professional will bring up and better enables the adolescent to actively participate in the consultation [37].

Nevertheless, we recommend further improvement of the self-report questionnaire assessing self-sufficiency with respect to user-friendliness. In contrast to the SSM-D, which was completed by professionals, the self-report questionnaire has no indicators that specify each level of self-sufficiency (from 1 to 5) per domain yet, but only gives a short description of the content of the domain. In order to make the self-report questionnaire more user-friendly and each level of self-sufficiency easier to interpret, it is desirable to add indicators that specify each level of self-sufficiency per domain. Preferably, these indicators should correspond with the indicators available for professionals.

A strength of the study is the high response rate among a vulnerable population. A high percentage of the adolescents in our sample suffer from depressive symptoms and often engage in behaviours that negatively impact their health, such as substance abuse [23]. Given the fact that the level of self-sufficiency in the different domains was relatively high in this study, the results seem to suggest that adolescents in vocational education are often – still – able to deal with problems they encounter in daily life. Furthermore, about 11 % of the adolescents were already parents. Recently, additional self-sufficiency domains on parenting have been developed. Since about 11 % of the adolescents were already parents, it would be of interest to include these domains in future research on this population. However, the present study also has its limitations. The study relied on adolescents in vocational education. Therefore, the psychometric properties of both the self-report questionnaire assessing self-sufficiency and the SSM-D remain to be established in other settings and populations. Furthermore, the temporal stability of both instruments could not be examined. The concurrent validity could also not be examined for three domains (i.e. domestic relations, activities daily life, and social network) because no related constructs were measured. Moreover, conceptual differences existed between the other self-sufficiency domains and related constructs. A comparison of ratings for the self-sufficiency domains with ratings for an instrument that is as closely related to the domains as possible (e.g. Camberwell Assessment of Need) [20] would have strengthened this study.

Conclusion

In conclusion, the results of this study show that both the self-report questionnaire assessing self-sufficiency and the SSM-D seem to possess adequate psychometric properties. Future research is necessary to investigate whether the results presented here can be replicated in different settings and populations as well as to investigate additional psychometric properties. We recommend using the adolescent-report questionnaire assessing self-sufficiency and the SSM-D concurrently to get a more complete picture of adolescent self-sufficiency. Both instruments express functioning in terms of levels of self-sufficiency in several domains, and can be considered for screening, monitoring or evaluation purposes. The instruments can be used during consultations with a professional to determine the functional strengths and areas for improvement. Furthermore, both instruments can be used to increase transparency in the decision-making processes in healthcare systems [38]. A great advantage of the self-report questionnaire assessing self-sufficiency and the SSM-D is that both versions can be completed in a short time, are freely available and can be used in a group of vulnerable adolescents.

Abbreviations

SSM: 

Self-sufficiency matrix

SSM-D: 

Dutch version of the Self-sufficiency matrix

NTR: 

Netherlands trial register

MHI-5: 

Mental health inventory

CES-D: 

Center for epidemiologic studies depression scale

SF-12: 

Short form-12 health survey

rrb

Rank biserial correlation

rps

Polyserial correlation

rpc

Polychoric correlation

r: 

Pearson correlation

k: 

kappa

Declarations

Acknowledgements

The authors thank the adolescents and the schools for their participation in this study.

Funding

This work was supported by the Netherlands Organization for Health Research and Development (ZonMw) (grant number 156512005). The publication of this study was supported by a grant of the Netherlands Organization for Scientific Research (NWO).

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Authors’ Affiliations

(1)
Department of Public Health, Erasmus University Medical Center Rotterdam
(2)
Municipality of Rotterdam
(3)
Public Health Care for Youth

References

  1. van Dorsselaer, S, de Looze, M, Vermeulen-Smit, E, de Roos, S, Verdurmen, J, ter Bogt, T, & Vollebergh, W (2009). Gezondheid, welzijn en opvoeding van jongeren in Nederland. Utrecht: Trimbos-instituut, Universiteit Utrecht, Sociaal en cultureel planbureau.Google Scholar
  2. Fassaert, T, Lauriks, S, van de Weerd, S, Theunissen, J, Kikkert, M, Dekker, J, Buster, M, & de Wit, M (2014). Psychometric properties of the Dutch version of the Self-Sufficiency Matrix (SSM-D). Community Ment Health J, 50(5), 583–590.PubMedGoogle Scholar
  3. Wetenschappelijke Raad voor het Regeringsbeleid. Vertrouwen in de school. Over de uitval van ’overbelaste’ jongeren [Confidence in school. About the dropout of ‘overburdened’ adolescents]. http://www.wrr.nl/fileadmin/nl/publicaties/PDF-Rapporten/Vertrouwen_in_de_school.pdf.
  4. Vogel, I, van de Looij-Jansen, PM, Mieloo, CL, Burdorf, A, & de Waart, F (2012). Risky music-listening behaviors and associated health-risk behaviors. Pediatrics, 129(6), 1097–1103.PubMedGoogle Scholar
  5. Busch, V, Van Stel, HF, Schrijvers, AJ, & de Leeuw, JR (2013). Clustering of health-related behaviors, health outcomes and demographics in Dutch adolescents: a cross-sectional study. BMC Public Health, 13, 1118.PubMedPubMed CentralGoogle Scholar
  6. Hoeve, M, Jurrius, K, van der Zouwen, M, Vergeer, M, Voogt, M, & Stams, GJ (2011). Problemic debts and criminal behaviour of adolescents and young adults. Amsterdam: Kohnstamm Instituut.Google Scholar
  7. Vaughn, MG, Maynard, BR, Salas-Wright, CP, Perron, BE, & Abdon, A (2013). Prevalence and correlates of truancy in the US: Results from a national sample. J Adolescence, 36(4), 767–776.Google Scholar
  8. Henry, KL, Knight, KE, & Thornberry, TP (2012). School disengagement as a predictor of dropout, delinquency, and problem substance use during adolescence and early adulthood. J Youth Adolescence, 41(2), 156–166.Google Scholar
  9. Kuntsche, E, Rehm, J, & Gmel, G (2004). Characteristics of binge drinkers in Europe. Soc Sci Med, 59(1), 113–127.PubMedGoogle Scholar
  10. Jackson, CA, Henderson, M, Frank, JW, & Haw, SJ (2012). An overview of prevention of multiple risk behaviour in adolescence and young adulthood. J Public Health-Uk, 34, I31–I40.Google Scholar
  11. Senteio, C, Marshall, KJ, Ritzen, EK, & Grant, J (2009). Preventing homelessness: an examination of the transition resource action center. J Prev Community, 37(2), 100–111.Google Scholar
  12. Hickert, AO, & Taylor, MJ (2011). Supportive housing for addicted, incarcerated homeless adults. J Soc Serv Res, 37(2), 136–151.Google Scholar
  13. The Self-sufficiency standard website. Dec 2011. http://www.selfsufficiencystandard.org/standard.html.
  14. The Results Oriented Management and Accountability (ROMA) website. Dec 2011. http://www.roma1.org/.
  15. Lifelong Learning Programme. Voices of youth. Sweden - Belgium - Netherlands. http://voicesofyouth.eu/.
  16. Dutch Ministry of Education Culture and Science. (2011). Bijlage VSV-brief 2011 [Appendix school dropout circular 2011]. The Hague: OCW.Google Scholar
  17. GGD Rotterdam-Rijnmond. Onderzoek naar gezondheid en leefstijl van ROC deelnemers [Research on senior vocational students’ health and lifestyle]. http://www.rotterdam.nl/COS/publicaties/MOR/012%20Jeugdmonitor%20Rijnmond%20Onderzoek%20naar%20gezondheid%20en%20leefstijl%20ROC%20deelnemers.pdf
  18. Achenbach, TM, McConaughy, SH, & Howell, CT (1987). Child/adolescent behavioral and emotional problems: implications of cross-informant correlations for situational specificity. Psychol Bull, 101(2), 213–232.PubMedGoogle Scholar
  19. Mieloo, C, Raat, H, van Oort, F, Bevaart, F, Vogel, I, Donker, M, & Jansen, W (2012). Validity and reliability of the strengths and difficulties questionnaire in 5–6 year olds: differences by gender or by parental education? PLoS One, 7, e36805.PubMedPubMed CentralGoogle Scholar
  20. Slade, M, Phelan, M, Thornicroft, G, & Parkman, S (1996). The Camberwell Assessment of Need (CAN): comparison of assessments by staff and patients of the needs of the severely mentally ill. Soc Psychiatry Psychiatr Epidemiol, 31(3–4), 109–113.PubMedGoogle Scholar
  21. van Widenfelt, BM, Goedhart, AW, Treffers, PD, & Goodman, R (2003). Dutch version of the Strengths and Difficulties Questionnaire (SDQ). Eur Child Adolesc Psychiatry, 12(6), 281–289.PubMedGoogle Scholar
  22. Muris, P, Meesters, C, & van den Berg, F (2003). The Strengths and Difficulties Questionnaire (SDQ)--further evidence for its reliability and validity in a community sample of Dutch children and adolescents. Eur Child Adolesc Psychiatry, 12(1), 1–8.PubMedGoogle Scholar
  23. Bannink, R, Broeren, S, Heydelberg, J, Van’t Klooster, E, Van Baar, C, & Raat, H (2014). Your Health, an intervention at senior vocational schools to promote adolescents’ health and health behaviors. Health Educ Res, 29(5), 773–785.PubMedGoogle Scholar
  24. Lauriks, S, Buster, MCA, de Wit, MAS, van de Weerd, S, Tigchelaar, G, & Fassaert, T (2010). Zelfredzaamheid-Matrix [Self-Sufficiency Matrix]. Amsterdam: GGD Amsterdam.Google Scholar
  25. GGD Amsterdam. Self-Sufficiency Matrix. http://www.selfsufficiencymatrix.org/zrm-int.aspx
  26. Centraal Bureau voor de Statistiek. Allochtoon [Mirgrant]. http://www.cbs.nl/nl-NL/menu/methoden/begrippen/default.htm?ConceptID=37.
  27. Monitor gezondheid. Lokale en nationale monitor gezondheid [Local and national health monitor]. http://www.monitorgezondheid.nl.
  28. Hoeymans, N, Garssen, AA, Westert, GP, & Verhaak, PF (2004). Measuring mental health of the Dutch population: a comparison of the GHQ-12 and the MHI-5. Health Qual Life Outcomes, 2, 23.PubMedPubMed CentralGoogle Scholar
  29. Radloff, LS (1977). The CES-D Scale: a self-report depression scale for research in the general population. Appl psych meas, 1, 385–401.Google Scholar
  30. Nunnally, JC (1994). Psychometric theory. New york: McGraw-Hill.Google Scholar
  31. Tabachnick, B. G., & Fidell, L. S. (2007). Using multivariate statistics. Boston: Pearson Education.Google Scholar
  32. Cohen, J (1998). Statistical power analysis for the behavioral sciences. New Jersey: Lawrence Erlbaum Associates.Google Scholar
  33. Fleiss, JL, Levin, B, & Paik, MC (2003). Statistical methods for rates and proportions (3rd ed.). Hoboken, New Jersey: John Wiley & Sons.Google Scholar
  34. Altman, D. G. (1991). Practical statistics for medical research. London: Chapman and Hall.Google Scholar
  35. Golsäter, M, Sidenvall, B, Lingfors, H, & Enskar, K (2011). Adolescents’ and school nurses’ perceptions of using a health and lifestyle tool in health dialogues. J Clin Nurs, 20(17–18), 2573–2583.PubMedGoogle Scholar
  36. Bannink, R, Broeren, S, Joosten-van Zwanenburg, E, van As, E, van de Looij-Jansen, P, & Raat, H (2014). Use and appreciation of a Web-based, tailored intervention (E-health4Uth) combined with counseling to promote adolescents’ health in preventive youth health care: survey and log-file analysis. JMIR Res Protoc, 3(1), e3.PubMedPubMed CentralGoogle Scholar
  37. Golsäter, M, Sidenvall, B, Lingfors, H, & Enskar, K (2010). Pupuls’ perspectives on preventive health dialogues. Br J Sch Nurs, 5(1), 26–33.Google Scholar
  38. Lauriks, S, de Wit, MAS, Buster, MCA, Fassaert, TJL, van Wifferen, R, & Klazinga, NS (2014). The use of the Dutch Self-Sufficiency Matrix (SSM-D) to inform allocation decisions to public mental health care for homeless people. Community Ment Health J, 50(7), 870–878.PubMedGoogle Scholar

Copyright

© Bannink et al. 2015

Advertisement