Development and psychometric evaluation of the CanSmart questionnaire to measure chronic disease self-management tasks

Abstract

Background

Psychometrically sound measures of chronic disease self-management tasks are needed to improve identification of patient needs and to tailor self-management programs. This study aimed to develop and conduct a preliminary psychometric analysis of the CanSMART questionnaire among a diverse, multimorbid Canadian population.

Methods

The data were drawn from a cross-sectional online survey to examine self-management needs and support preferences. Participants were 306 Canadian adults with one or more physical and/or emotional chronic conditions. The questionnaire on frequency of self-management tasks was developed with substantial patient partner input. We conducted Exploratory Factor Analysis (EFA) and Confirmatory Factor Analysis (CFA) of the 11 self-management tasks comprising the scale in two randomly selected subsamples, followed by Rasch analysis. Associations between patient characteristics and the self-management task subscales and individual items were explored.

Results

The factor analyses identified two self-management task subscales that were labelled Coping tasks (6 items) and Physical tasks (3 items), with Cronbach’s alpha of 0.70 and 0.67, respectively. Rasch analysis suggested that participants had difficulty discriminating between response options “mostly” and “always”. In analyses of independent associations with patient characteristics, both Coping and Physical tasks were associated with reporting more than one chronic disease and employment disability. The Coping tasks subscale was associated with female sex. Two items, on medication use and monitoring biological parameters, did not load on either scale. Both were associated with specific diagnoses.

Conclusions

In this preliminary analysis, two self-management tasks subscales exhibit good psychometric properties. Two items that did not load on either scale may represent additional dimensions of self-management. This work provides the basis for further scale development and use in research and clinical practice.

Introduction

The prevalence of chronic disease in Canada continues to grow rapidly, with over half of all Canadian adults already living with at least one chronic disease and 1 in 4 with multiple chronic diseases (multimorbidity) [1, 2]. These individuals often require ongoing, complex care and treatment and will experience effects on their physical and emotional quality of life [3, 4, 5]. The Chronic Care Model, involving pro-active healthcare providers and patients who are equipped to interact effectively with their healthcare team [6], changes the way chronic disease care is organized and delivered, leading to improved patient outcomes [7]. Self-management, a key component of the Chronic Care Model [6], refers to those behaviors necessary to promote one’s health and manage the physical, emotional, and social effects of an illness or illnesses [8]. People skilled at self-management understand their conditions and treatments and are actively involved in their care. They help create and follow their care plans, and monitor their conditions. Although self-management behaviors may vary by chronic disease, there may be core behaviors across diseases that are related in part to individual characteristics such as socio-economic status [9]. There is also generally good consensus from several major initiatives that there are common elements in chronic disease self-management, even across different chronic diseases [10, 11, 12, 13].

Whereas there are disease specific measures of self-management tasks (e.g., Diabetes Self-Management Questionnaire [14]), given the increasing prevalence of multimorbidity, valid and reliable measures applicable across chronic diseases to better understand all the self-management undertaken by patients are needed. Such measures would help to guide self-management support programs and tailor them to individual needs. Two previous studies have proposed categories of self-management tasks, but have not validated them. Schulman-Green [15] undertook a synthesis of 101 qualitative studies that described processes of self-management in chronic disease. These were summarized across three categories: (1) focusing on illness needs (including medication management, health behaviors, keeping appointments); (2) activating resources (including family, community, and healthcare resources) and living with a chronic illness (including coping skills such as processing emotions; and (3) adjusting to, and integrating illness into daily life. The second initiative, the Dutch Patient Assessment of Self-management Tasks (PAST) questionnaire [16], created utilizing expert input, included four categories of activities: (1) medical management, (2) communication with healthcare providers, (3) coping with the consequences of having a chronic illness, and (4) making lifestyle changes.

We sought to develop a psychometrically sound questionnaire on self-management task frequency that could be used in the Canadian context across chronic diseases and those with multimorbidity, inspired by the PAST, but using feedback from Canadian patients and self-management researchers and clinicians (CanSMART: Canadian Self-Management Research Team). The first version of the CanSMART self-management task questionnaire was used in an online survey of Canadian adults with chronic diseases [9]. In this paper, we report on the psychometric properties of this questionnaire.

Methods

Questionnaire development

Beginning in April 2015 CanSMART expert members collaborated to design an online survey to capture the self-management tasks, needs and support preferences of Canadians with a chronic illness. The CanSMART team was composed of 15 individuals from across Canada, including chronic disease researchers, clinical psychologists, family physicians, nurses, rehabilitation science practitioners, epidemiologists, and patient research partners. These stakeholders had come together as part of the Canadian Institutes of Health Research’s Strategy for Patient Oriented Research. Four successive versions of the survey design were circulated electronically for comments to the CanSMART team. Feedback was incorporated in turn, with each survey version discussed and further revised during team meetings until agreement among team members was reached.

Questions on self-management tasks were inspired by the structure of the PAST [16]. The PAST was developed in the Netherlands and asks respondents to indicate how often each of 19 listed self-management tasks are completed and how often support is needed with the tasks. Patient research partners first reviewed the wording and content of PAST items to assess their relevance to Canadians with chronic conditions. Additional input from other team members led to further changes: coping with pain and limitations was separated into 2 questions; 5 individual lifestyle questions on diet, exercise and health habits were collapsed into a single question; and new items were added to address asking for or needing help with household chores, self-care (e.g., bathing, dressing), and employment issues. The PAST items on interactions with healthcare providers (an important aspect of self-management) were excluded from the CanSMART questionnaire because corresponding questions from the Patient Assessment of Chronic Illness Care (PACIC) [17] were included in the larger survey [18]. Frequency of tasks was reported on a 4-point ordinal response scale: 1 (never), 2 (sometimes), 3 (mostly), or 4 (always), with a “not applicable” response option [16]. The final CanSMART self-management tasks questionnaire comprised 12 items (Additional file 1: Appendix 1, see also Table 2 but note that item on employment is not listed in Table). The team reviewed all materials with an eye to using plain language.

Recruitment of study sample

A cross-sectional survey was launched in English and French in August 2015 across Canada [9]. We recruited a convenience sample by asking CanSMART team members and representatives of national and regional patient and disease-specific advocacy groups to circulate the survey link via email, newsletters, social media and relevant websites. Advocacy groups were identified by team members and through web searches. In total, 30 advocacy groups were contacted to share study information and the survey link. Interested individuals aged 18 and over, with at least one chronic physical and/or emotional condition were invited to participate. The invitation specified, “To be in this study, you must be 18 years of age, living in Canada, and have one or more chronic physical and/or emotional illnesses”. Age, sex, level of education, employment status, province of residence and type of chronic conditions were included in the brief demographics section appearing at the end of the survey. The online survey did not require any identifying information, and no incentive or compensation was provided. The welcome page informed participants that the survey was voluntary, anonymous, and that completion of the survey was interpreted as informed consent. The study protocol was approved by the Research Ethics Committee of St. Mary’s Hospital Center, Montreal. The survey was closed in February 2016.

Psychometric analysis

We based the scale development on standard psychometric methodology [19], applied to the ordinal items on self-management task frequency (Additional file 1: Appendix A). The item #12 on managing work-related limitations was excluded, because 57% of the sample had “missing” responses. The study sample was restricted to the subset of patients who had completed the remaining 11 items. The first step in our psychometric analysis was to identify valid subscales by using both Exploratory Factor Analysis (EFA) and Confirmatory Factor Analysis (CFA). Ideally, we would need to work with two large independently collected data sets, and perform scale development on one, using EFA, and validation on the other, using CFA. However, in clinical settings such an approach may not be possible, due to the inherent difficulties of human data collection. Using guidelines recommended by Anderson [20], we therefore randomly split the sample into two equal size subsamples called ‘exploratory’ and ‘confirmatory’. We applied EFA to the former and CFA with maximum likelihood (ML) to the latter [21, 22]. Pearson’s Chi-square and T-tests were used to compare patient characteristics and frequency of self-management tasks in the two subsamples. From the EFA we determined the dimensionality of the factor space, which we refer to as ‘p’; the dimensionality was determined by parallel analysis [22, 23]. To interpret the dimensions found, we listed the ‘important’ items for each dimension, i.e., those with factor loadings (after ‘varimax’) larger than 0.4 [23].

The ‘important’ items for each dimension were grouped to represent one structural equation model (SEM). To take into account variables which we considered meaningful but do not appear ‘important’, we defined informally a number of alternative models, i.e. we consider several ways to assign the non-important variables to any dimension, including removal. We then evaluated these models by CFA. In CFA, several statistics are available to assess the fit of a model. Following Kline [24], we relied on the following four: Chi-Square, Root Mean Square Error of Approximation (RMSEA), Bentler’s Comparative Fit Index (CFI), and the Standardized Root Mean Square Residual (SRMR). Values of Chi-Square close to zero indicate an acceptable model fit; in practice, it is usually required that RMSEA ≤ 0.10, CFI ≥ 0.90, and SRMR ≤ 0.10. Recent references suggest stricter criteria: RMSEA ≤ 0.06, CFI ≥ 0.95, and SRMR ≤ 0.08), but these stricter criteria are only valid for larger sample sizes than ours (n > 500) [25]. In large sample sizes (> 300), since CFA requires (approximate) multivariate normality of the items, ordinal items such as ours might be analyzed by replacing the Pearson correlation matrix with the polychoric correlation matrix [25]; this was not appropriate given our smaller sample size.

The second step in this psychometric analysis was to perform Rasch analysis [26] on the selected model. Likelihood Ratio (LR) tests [27] were performed to determine whether the Partial Credit Model (PCM) [28] or the Rating Scale Model (RSM) [29] would be used. According to Rasch analysis, threshold points on a response scale should be correctly ordered for each item; e.g. the respondents would consider endorsing 'always' to represent greater need for the latent trait rather than 'mostly'. In contrast, ‘disordered thresholds’ occur when respondents have difficulty consistently discriminating between response categories. The Category Characteristic Curve (CCC) [30], which displays the probability of a respondent endorsing a particular response category based on their level of need for the item (intensity), was obtained to examine the performance of the response scale. A disorder threshold was detected in the CCC plot when the line of a corresponding answer crossed other lines (answer) at the same latent trait value; if an item had a disordered threshold, the CCC informed response re-categorisation. To determine item fit, ‘infit’ and ‘outfit’ statistics were calculated, whereby values between 0.7 and 1.3 are acceptable [31]. Finally, unidimensionality was assessed by the Martin-Löf Test [32].

Once validity and ‘scoring properties’ were confirmed, we computed a subscale score for each dimension by simply averaging the important items for the dimension. Internal consistency was assessed using Cronbach’s alpha for each subscale; acceptable reliability can be achieved with a Cronbach Alpha of 0.7 but a threshold of 0.6 is also considered acceptable for short scales (< 10 items) [33].

Finally, we studied the variation of the subscales (computed after Rasch analysis) across subgroups defined by patient characteristics (including diseases). To determine whether these associations were independent we conducted multivariable linear regression analyses [34]. All the analyses were performed with SAS version 9.4 and R. [27] (eRm package).

Results

Study sample

The online questionnaire was activated by 353 potential respondents. Of these, 19 entered no information and 28 did not complete the section on self-management tasks. Table 1 presents selected characteristics of the study sample. A large number of the remaining 306 participants did not complete the sociodemographic section of the survey (n = 45). Participants were mostly aged 45 or older (73%), female (78%), and well educated (74% had some college education), spoke English at home (90%), and were born in Canada (87%). Participants reported a mean of 2.8 (SD = 1.6) chronic diseases, with the most common ones being arthritis (62%) and emotional problems (58%).

Table 1 Patient Characteristics in the 2 Study Samples (n = 306)

Psychometric results

Factor analysis

From the comparison of two subsamples used for EFA and CFA, respectively, we found that only age differed significantly (p-value < 0.05) across samples (Table 1). NA (not applicable) responses to the 11 items used in the scale development process were infrequent (mean number of NA was 6 per item, range: 1–19), and were assigned to the response ‘no need’. Only 2 items revealed some departure from normality: “Taking medication daily” (skewness = − 2.2, kurtosis = 3.4) and “Asking for or needing help with self-care” (kurtosis = 2.9): these values are slightly out of the acceptable range (− 2 to + 2 for both) [35].

In the EFA, 2 dimensions were enough to explain the total variance (76% was explained by the first dimension and 23% by the second), see Additional file 1: Appendix 2. The factor loadings (for each item are presented in Table 2; the ‘important’ factor loadings (> 0.4) are in bold font. Three items (Manage discomfort, Take medication daily, Check things such as blood pressure or blood sugar) had factor loadings below threshold for both dimensions, which led us to consider 5 alternative models (corresponding to different ways of dropping items or assigning them to dimensions).

Table 2 Exploratory factor analysis on 'exploratory' sample (n = 153)

CFA was performed on the confirmatory sample to test these five models. The results of CFA are summarized in Table 3, which shows the standardized coefficients of the equations expressing factor estimates as linear combinations of items. According to the Chi-square test, none of the models fits the data well (p-value < 0.05), but other indices suggested acceptable fits. Only Model 1 showed an acceptable fit for all other indices. We therefore retained Model 1; the two subscales were labeled as: a) Coping tasks (Managing discomfort/pain, Managing fatigue, Making lifestyle changes (e.g. diet), Emotional coping, Avoiding/limiting activities, Dealing with unexpected or new life problems), and b) Physical tasks (Dealing with physical limitations, Asking for or needing help with household chores, Asking for or needing help with self-care).

Table 3 Confirmatory Factor Analysis with the 'Confirmatory' Sample (best models) (n = 153)

Rasch analysis

The PCM fit the data better than did the RSM for the Coping tasks (LR = 18, df = 5, p-value < 0.001), but not for the Physical tasks (LR = 3.15, df = 2, p = 0.207). Despite this, we report the results for the PCM, since the findings were the same as the RSM (PCM can handle items with different scaling).

All items show a potential ‘disorder’ in the region from ‘Mostly’ to ‘Always’. Also, Additional file 1: Appendix 3a and 3c show that the line corresponding to the answer ‘Sometimes’ crosses the other two lines at the same latent trait value. This inspection suggested recoding ‘Mostly’ and ‘Always’ as the same response. Indeed, the new CCC graphs did not suggest ‘disorder’ (see Additional file 1: Appendix 3b and 3d), except for the item “Asking for or needing help with self-care” (Physical Tasks subscale); in this case the CCC plot suggest grouping ‘Sometimes’ with ‘Mostly and Always’.

After these changes, the mean square fit statistics for all items were acceptable, ‘infit’ and ‘outfit’ ranging from 0.69 to 1.00 (details not shown). The Martin–Löf Test for unidimensionality for Coping tasks is consistent with one dimension (p-value = 0.769); this test was not performed for the 3 items Physical tasks subscale since it requires at least 4 items.

Cronbach’s alpha

The Cronbach’s alphas for Model 1 were 0.70 for Coping tasks, and 0.67 for Physical tasks.

Associations with patient characteristics

Table 4 shows the results of the regression of each subscale on patient characteristics. Note that the self-management tasks subscales were computed using the scoring suggested by the Rasch analysis (see Table 5): all items except one used a 3-level response scale: never, sometimes, mostly/always. One item (asking for or needing help with self-care) used the binary scoring suggested by the Rasch analysis: never and sometimes/mostly/always. Mean frequencies of both Coping and Physical tasks were significantly higher among participants on employment disability, and among those with musculoskeletal conditions other than arthritis. Also, the mean frequency of Coping tasks was significantly higher among female participants, those with emotional conditions, asthma, and no heart disease. The mean frequency of Physical tasks was significantly higher among participants with arthritis, and neurological conditions. The regression analyses presented in Table 4 that includes the number of diseases and not the individual diseases (not shown) revealed similar significant results for all variables except for the Physical subscale, whereby the effect of age was significant (older patient have more tasks); the effect of the number of diseases was significant, having 1 more disease is associated with a 3% increase in both scores. Of the two medical management items that did not load on either subscale, both were significantly associated with employment disability (not shown in the Tables). The medication task frequency item was significantly higher among participants who reported emotional problems; the self-monitoring of biological parameters was associated significantly with a diagnosis of diabetes and hypertension.

Table 4 Association between patient characteristics and self-management tasks subscales* (coping and physical), results of multivariable linear regression models (n = 246**)

Table 5 Self-management tasks frequency by subscales

We also conducted a sensitivity analysis, not presented here, in which all items were coded using the same 3-level response scale. The results were almost identical to those reported in Table 4. On average the beta coefficient varied by ± 5%, with the exception of age 65–74 years (20% variation). None of these variations led to changes in the statistical significance, with the exception of musculoskeletal diseases which became statistically non-significant (beta: 0.18 [− 0.02; 0.38]).

Table 5 shows the final self-management task subscales with response options and scoring based on the Rasch analysis.

Discussion

This study sought to advance chronic disease self-management research through the collaborative development with patient partners of a measure to assess the type and frequency of self-management tasks and to conduct a preliminary analysis of its psychometric properties.

Using accepted guidelines on scale development, we randomly separated the sample into two subsamples of equal size and used one subsample for EFA and the other for CFA, a major step towards valid scale development. Our psychometric analysis revealed two self-management task subscales: one assessing frequency of Coping tasks (Managing discomfort/pain, Managing fatigue, Making lifestyle changes (e.g. diet), Emotional coping, Avoiding/limiting activities, Dealing with unexpected or new life problems), and the second assessing Physical tasks (Dealing with physical limitations, Asking for or needing help with household chores, Asking for or needing help with self-care). The Rasch analyses suggested appropriate response scales for each item. Two additional tasks (medication use and self-monitoring of biological parameters) did not load on either subscale and may represent additional dimensions of self-management.

There has been little prior quantitative research on describing the tasks that people with chronic diseases do—the work of self-management. The subscales developed in the CanSMART study do not map exactly onto the categories identified in the qualitative synthesis of Schulman–Green [15], although they include many of the processes described. Our Coping self-management tasks subscale captures several aspects of the third category (adjusting, integrating) along with health behaviors, which could also be conceptualized as coping activities. Our Physical subscale captures aspects of the second category dealing with mobilizing help for daily activities. Illness needs were represented in the CanSMART survey by items on taking medication and on self-monitoring. Neither of these items loaded onto our two subscales. In the case of taking medication, this task is very frequent among most people with chronic illnesses and represents an important dimension of self-management. In the case of self-monitoring, the need for this task and the type of monitoring required appeared to be quite disease-dependent, being performed more frequently for hypertension (blood pressure monitoring) and diabetes (blood sugar monitoring). These two illnesses have established measurement tools and treatment benchmarks—which give patients discrete targets to aim for, and can make self-monitoring more straightforward. It should be noted that the item wording may have biased respondents to reporting these two types of self-monitoring rather than others (e.g., self-monitoring of depression symptoms). Similar to our results, Schulman-Green [15] also reported that comorbidities increased the type and complexity of self-management tasks performed.

Van Houtum et al. [36] used a theory-based approach in developing the PAST scales of self-management tasks and related support needs. In contrast, in CanSMART, we asked patients to review items and adapted them as needed to ensure that they were understandable and relevant. The PAST scales distinguished four types of self-management tasks: medical management (including medication management and self-monitoring); communication with healthcare providers; coping with the consequences of disease; and making lifestyle changes. Again, our Coping self-management tasks subscale overlaps substantially with the PAST coping scale. Lifestyle changes were covered in a single question in CanSMART. As noted above, medical management did not emerge as a distinct dimension. Communication with healthcare providers was assessed in a separate scale, the PACIC [17]. This is an important aspect of self-management and should be considered for inclusion in future iterations for the CanSMART questionnaire. We separated coping with pain and physical limitations into two different items, and in fact these items loaded into two different factors: pain with coping tasks, and physical limitations with physical tasks.

One CanSMART question considered important by our patient partners addressed employment issues such as absenteeism; however, this item could not be included in the psychometric analysis because of missing data for a significant proportion of participants. Nevertheless, our secondary analysis suggested that this item correlated in particular with the Coping task subscale.

The validity of the task subscales and items is supported by their associations with patient characteristics For example, the Coping tasks subscale was associated with employment disability and certain diagnoses (emotional, asthma, and other musculoskeletal). Women reported higher overall coping task frequency than men. The Physical self-management tasks subscale was associated with employment disability, arthritis, neurological, and other musculoskeletal diseases. The medication task frequency item was significantly higher among participants with employment disability, and those who reported emotional problems; the self-monitoring of biological parameters was associated significantly with a diagnosis of diabetes and hypertension, both diseases commonly requiring patient self-monitoring [37].

The Coping and Physical subscales, when finalized, have multiple applications in research and practice. For example, they could be used to target interventions. Our previous research using these scales found that they could be used to identify vulnerable self-managers, with high frequency of self-management tasks but low self-efficacy in performing them [9].

Limitations

The CFA was performed on a sample of 153 participants. In view of our limited sample size, we did not perform more complex analysis, such as maximum likelihood with robust standard error and weighted least square with polychoric correlation. It would be desirable to repeat the CFA on a larger sample size (at least 300 is recommended [38]). As regards the Rasch analysis, participants had difficulty discriminating between categories “mostly” and “always”, suggesting that these categories should be merged in future studies to calculate the subscale scores. Only one item needed a different re-coding (collapsing “sometimes”, “mostly” and “always”). This difference might limit the practical use of the scale. However, our sensitivity analyses using uniform three category coding revealed almost identical results suggesting that universal application of the 3-level response set would be acceptable.

Finally, the sample is one of convenience and results may not generalize to all Canadian adults with chronic diseases. In particular, the sample is more highly educated than the general population. Future studies might want to consider quota sampling to ensure enough participants are of lower education status and are still employed to further assess the psychometric properties of the scale.

Conclusion

Our study is one of the first to attempt to measure quantitatively the tasks of chronic disease self-management. As such, it provides a preliminary basis on which further scale development and psychometric analysis could be conducted. Our study confirms the existence of a distinct coping dimension of self-management, in keeping with previous research. Our results also suggest a second distinct dimension that addresses mobilization of family and community resources to assist with physical self-care. Questions on mobilization of medical professionals should be added to future versions. Among the other items measured, our results suggest that medication management is an almost universal task. On the other hand, self-monitoring appears to be disease specific.