Gain +1 or Avoid −1: Validation of the German Regulatory Focus Questionnaire (RFQ)

Background The Regulatory Focus Questionnaire (RFQ) assesses regulatory promotion and prevention focus, which represent orientations towards gains or losses. The main objective of this study was to examine the psychometric properties of the newly translated German version. Methods A sample of 1024 participants answered the questionnaire and several related instruments. We used an online survey tool to collect this data. Data analysis was conducted using methods of exploratory and confirmatory factor analysis in SPSS and AMOS. Results The RFQ displayed acceptable reliability, while its correlations with other, related psychological constructs indicated good validity. Factor analysis showed good fit for a two-dimensional model. Tests of measurement invariance revealed clear evidence for metric invariance while scalar invariance remained uncertain. Differences in regulatory focus based on sociodemographic characteristics are reported and discussed. Conclusions Overall, the RFQ can be recommended for application in fields dealing with motivation and goal attainment in a broad sense.


Background
Regulatory focus theory (RFT) is a goal pursuit theory that categorizes individuals' thoughts and behaviors in terms of an orientation towards gains and losses [1][2][3][4].
The promotion system focuses on the attainment of a desired state whereas the prevention system centers on the avoidance of undesirable states. Accordingly, promotion-oriented individuals seek to make gains, seize opportunities, and take risks in order to advance in their pursuits towards ideals. In contrast, prevention-oriented individuals aim to minimize risks, maintain a given status quo, and remain vigilant against potential threats to oughts. These tendencies influence the processing and usage of information and decision making on many levels, and therefore play an important role in several fields of psychological research such as motivation, attitude, persuasion, and leadership, among others [5][6][7][8][9][10]. Furthermore, considering that specific regulatory focus states can be easily primed, applications of this theory are abundant and diverse [11]. The regulatory focus systems are rooted in specific neural components, as indicated by neural correlates that have been identified, including an activation of the amygdala, the anterior cingulate cortex, and the extrastriate cortex [12]. Additionally, promotion focus relates to an activation of the right prefrontal cortex while prevention focus correlates with an activation of the left prefrontal cortex [13].
Molden, Lee, and Higgins [14] argued that the regulatory focus system is orthogonal to the approach-avoidance system. Proposing a 2 × 2 model, they demonstrated how in approaching a positive end state, individuals can either approach gains (promotion) or approach non-losses (prevention). Similarly, in avoiding a negative end state, individuals can either avoid non-gains (promotion) or avoid losses (prevention). For example, one can strive for (or approaching) a positive end state of health by either exercising regularly to reap the benefits (gains, i.e. promotion) or by not smoking or drinking in order to not lose the health status one already possesses (non-losses, i.e. prevention). On the other hand, one can strive to avoid the negative end state of sickness with the exact same behavior, in a reframed setting: One exercises to avoid missing out on the positive results (non-gains, i.e. promotion) and avoids smoking and drinking in order not to experience the negative effects associated with those behaviors (losses, i.e. prevention). Although the regulatory focus system is theoretically orthogonal to the approachavoidance system, studies comparing regulatory focus measures with approach-avoidance measures have shown that these two constructs are in fact moderately correlated [15]. As noted by Haws, Dholakia, and Bearden [16], the two regulatory focus measures that are being used most frequently in psychological research are the Regulatory Focus Questionnaire (RFQ) by Higgins and colleagues [17] and the General Regulatory Focus Measure (GRFM) by Lockwood, Jordan, and Kunda [9]. Summerville and Roese [15] found stronger associations with Behavioral Inhibition/ Approach Scales (BIS/BAS; Carver & White, 1994) for the GRFM than they did for the RFQ and stronger correlations for promotion than for prevention focus.
There have been a number of attempts at establishing a German measure of regulatory focus: Several translations of the RFQ have been employed by researchers in the past (e.g., [18][19][20][21]). However, none of the conducted studies reported detailed psychometric propertiesespecially the factor structure was never discussed. As this is a very central step in investigating the validity and therefore the theoretical soundness as well as the practical applicability of a scale, it should not be skipped. The GRFM has also been translated and applied before [22], but there is again no discussion of factorial structure. Finally, Fellner, Holler, Kirchler, and Schabmann [23] created a new scale that seeks to address the short-comings of the RFQ and the GRFM, but does only achieve mediocre factorial validity.

Aims of the study
The present study seeks to validate the newly translated German version of the RFQ. Specifically, it aims to a) investigate psychometric properties including item characteristics and reliability, b) confirm the two-factorial structure proposed by Higgins and colleagues [17], c) examine validity towards related psychological constructs, and d) analyze measurement invariance as well as differences in promotion and prevention focus based on sociodemographic variables.
Considering the moderate correlations found between promotion focus and behavioral approach, and between prevention focus and behavioral inhibition, similar correlations are expected in the present study. Furthermore, regulatory focus (mostly promotion focus) plays an important role in predicting work-related outcomes [24]. Two other constructs that significantly predict workrelated outcomes are core self-evaluations and the Big Five, and therefore, the relationships between the RFQ and the subscales of the Core Self-Evaluation Scale (CSES; [25]) and the Big Five Inventory-10 (BFI-10; [26]) were examined. Correlations of regulatory focus and the Big Five have been shown by previous research, such as openness [27,28]. Individuals with high promotion focus look for opportunities and seek to maximize gains, this implies a necessity for openness to new experiences. In this line of argument, we also expect a moderate association of promotion focus with extraversion. In contrast, individuals with prevention focus, want to maintain vigilance and avoid losses, therefore a positive association with conscientiousness and neuroticism is expected. As regulatory focus relates to self-regulation high correlations with the CSES, which contains among others self-efficacy and self-esteem, are also expected. Hazlett, Molden, and Sackett (2011) have shown that promotionoriented individuals tend to be optimistic, whereas prevention-oriented individuals favor pessimism. For this reason, the revised Life-Orientation-Test (LOT-R; [29]) was utilized in the present study. Lastly, based on the relationship between regulatory focus and optimism/pessimism, corresponding associations with health outcomes are expected as well [30,31]. Furthermore, regulatory focus orientation has been shown to be an important regulator of responses to health messages [32].

Participants and procedures
The study sample was acquired between December 2015 and February 2016 utilizing the online survey tool SoSci-Survey [33],after the design was met with approval by the ethics commission of the University of Applied Sciences Magdeburg-Stendal (AZ-3973-51). Participants were recruited for the study by means of social networks and bulletin boards. After receiving an introduction with regard to the general purpose of the study, participants gave their informed consent.
The total number of participants who started the survey by giving consent was N = 1173, of which n = 282 (24%) aborted the survey before answering all questions. Participants, who aborted the survey after providing their sociodemographic information but before completing any of the presented questionnaires (n = 149 [13%]) differed significantly from those who completed additional questionnaires in two of the six sociodemographic variables.
Participants that aborted the survey were more likely to report a male gender (χ 2 (2) = 12.39, p = .002) and lower education (χ 2 (6) = 14.58, p = .024). Age (U = 68,923.00, p = .740), employment status (χ 2 (5) = 3.59, p = .610), family status (χ 2 (5) = 0.70, p = .983), and monthly net household income (χ 2 (8) = 6.23, p = .622) did not differ between participants who continued with the survey and those who did not. Due to the design of the online survey, either participants answered all items of a given scale or none at all; there was no missing data. Individuals who were too young to take part in the study (under the age of 18 years) were excluded. Thus, the used sample consisted of n = 1024. Participants who were included in the analysis had a mean age of around 30 years (M = 29.38; SD = 10.79) with a range from 18 to 70 years. Detailed sample characteristics are presented in Table 1.

Regulatory focus questionnaire (RFQ)
The Regulatory Focus Questionnaire [17] comprises eleven items assessing the regulatory focus orientation of an individual. It includes six items for promotion focus and five items for prevention focus. Options for answering the items range from 1 -"never or seldom"/"never true"/ "certainly false" to 5 -"very often"/"very often true"/"certainly true". Seven of the items (1,2,4,6,8,9,11) need to be reverse-scored before calculating the respective mean scale scores. Higgins and colleagues [17] reported internal consistency as α = .73 for the promotion subscale and as α = .80 for the prevention subscale, and their intercorrelation was r = .21. It should be noted that values lower than α = .70 have been found before for the promotion subscale [34]. For the present study, two professional translators converted the original English version items into German independent of one another. After reaching a consensus on a single translation the items were translated back into English by two native speakers and compared with the original. Both language versions are displayed in Table 3. Scale characteristics for the German version are reported in the results section.

Behavioral inhibition/approach system scale (BIS/BAS)
The German BIS/BAS [35] was used to measure approach and avoidance motivation. It consists of 20 items, which are split among four subscales (BIS, BAS-Drive, BAS-Fun Seeking, BAS-Reward Responsiveness), and four filler items. Scale values are calculated by averaging item scores after reverse-scoring two of them. Strobel and colleagues [35] reported the internal consistency of the BIS scale as α = .78, and the BAS scale as α = .81.

Core self-evaluations scale (CSES)
The CSES ( [36]) includes facets of self-esteem, locus of control, neuroticism, and self-efficacy in a 12 item scale.
For the German version, evidence by Zenger and colleagues [25] suggested a two-factor interpretation. The, two scale scores are obtained by averaging the positivelyand the negatively worded items, respectively. Internal consistency was reported as between α = .81 and .86.
Big five Inventory-10 (BFI-10) The Big Five personality dimensions (Openness, Conscientiousness, Extraversion, Agreeableness, and Neuroticism) were measured using the BFI-10 [26]. Every subscale consists of two items, one of which has to be reverse-scored before calculating the mean. Rammstedt and John [26] reported test-retest-reliability as r tt = .78 for Openness, r tt = .83 for Conscientiousness, r tt = .66 for Agreeableness, r tt = .87 for Extraversion, and r tt = .71 for Neuroticism.

Life orientation test -Revised (LOT-R)
The LOT-R [29] uses ten items to measure optimism and pessimism, four of which are filler items. A twofactor interpretation has been found to be preferable for the German version [37]. Scale scores for the two subscales are computed by adding individual item scores. Cronbach's α was reported as α = .70 for the optimism scale and as α = .74 for the pessimism scale in the German general population [38].

Patient health Questionnaire-4 (PHQ-4)
The PHQ-4 [39] is an ultra-short screening instrument for symptoms of depression and anxiety using four items. Participants indicate to what extent they suffered from specific symptoms during the last two weeks. Summing up the items yields a scale score, measuring psychological distress. Löwe and colleagues [39] reported an internal consistency of α = .82 for the scale.

Somatic symptom Scale-8 (SSS-8)
The SSS-8 measures experienced somatic stress using eight items [40]. Adding all items provides a total score. The internal consistency of the scale was reported as α = .81 by Gierk and colleagues [40].

Subjective health status
From the EuroQol-5D [41], the visual analogue scale (VAS) was utilized to measure participants' current subjective health status. It ranges from (0) "worst imaginable health status" to (100) "best imaginable health status".

Statistical analyses
Statistical operations were conducted using IBM SPSS Statistics 23 and AMOS 23. Pearson product-moment correlation coefficients were employed for reporting correlations. An α level of .05 was used for tests of significance unless otherwise noted. Properties of scales and items, such as means, standard deviations, itemdifficulty indices as well as item-total correlations, were determined for the RFQ. Additionally, item and scale distributions were tested for normality by calculating skewness and kurtosis. The assumptions of sphericity and sampling adequacy were examined.
For the exploratory (EFA) and the confirmatory factor analyses (CFA), two randomly split subsamples of roughly the same size were used (n EFA = 510; n CFA = 514). The subsamples did not differ significantly in terms of age, gender, and RFQ item scores. For the EFA, principal component analysis (PCA), the minimum average partial (MAP; [42]) test, and parallel analysis (PA; [43]) were used. In the MAP test, the average squared partial correlations serve as indicators to determine the ideal number of factors. In PA, eigenvalues of randomly generated correlation matrices based on the original raw data (same number of variables and cases) are tested for significant differences from the empirically found ones. O'Connor [44] supplies SPSS syntaxes for these operations. Covariance matrices and the maximum likelihood method were used for the CFA. The following model fit indices were used for the CFA with commonly agreed upon cut-off values [45][46][47][48]. The χ 2 -statistic and the minimum discrepancy divided by degrees of freedom (CMIN/DF) were used. Ideally, the former should be non-significant, although that rarely happens with larger sample sizes [49],while the CMIN/ DF should be lower than five. The comparative fit index (CFI) and the Tucker-Lewis index (TLI) should be greater than .95, while a CFI/TLI that is greater than .90 can still be acceptable. The standardized root mean square residual (SRMR) should be lower than .08, although ideally lower than .06. Similar values are used for the root mean square error of approximation (RMSEA) and its 90% confidence interval. The Bayesian Information Criterion (BIC) is a comparative measure of fit and is utilized for comparisons between several models which do not necessarily have to be nested [50,51]. Smaller BIC values indicate better fit. Raftery [51] reported guidelines for interpreting differences in BIC between models, suggesting that a margin of 10 between model BICs is the equivalent of a significant difference at the p = .001 level, given a sample size of at least 30. We use the BIC to compare between the two alternative models reported in the present study.
A multiple-group factor analysis was used to test for measurement invariance in a two-step process. Firstly, the unconstrained, configural model was compared with the metric model, which constrains unstandardized item loadings to be equal across groups. Secondly, the metric model and the scalar model, which constrains both, unstandardized item loadings and item intercepts, across groups, were compared. As per previous research, the differences in CFI and gamma hat (GH; [52]) were used as indicators for invariance along with the differences in the χ 2 -statistic [53,54]. A deviation of more than .01 in CFI or GH should be considered a sign of violations of measurement invariance.
Finally, differences in promotion and prevention focus across sociodemographic groups were tested for significance, for groups with at least 20 members. Normal distribution and equality of variances could be confirmed. In order to avoid an accumulation of α error probability, a significance level of .01 was utilized for the ANOVAs. Furthermore, Tukey's HSD was used to compare individual groups for significant differences. Reported effect sizes are interpreted using Cohen's d and η 2 , including 95% and 90% confidence intervals, respectively [55].

Item characteristics and reliability
Item and scale characteristics are reported in Table 2. Skewness and kurtosis are well within the norms of having an absolute value of less than 1 for skewness and less than 3 for kurtosis [56]. Thus, a normal distribution can be assumed for all items and scales. Means and itemdifficulty indices suggested that participants tended to answer items in the direction of the trait in question. Corrected item-total correlations ranged from .24 to .65. Usually, a value of .3 is considered a cut-off point for this coefficient [57]. The internal consistency of the subscales was ω = .78, 95%-CI = [.76; .80] for prevention focus, which is a good, and ω = .61, 95%-CI = [.58; .65] for promotion focus, which is mediocre and questionable [58]. After removing Item 3 the reliability of the promotion subscale did not worsen significantly, remaining at ω = .61, 95%-CI = [.57; .65]. This in conjunction with the poor item characteristics for Item 3 suggest its exclusion from the scale.

Factor structure
The PCA of the first subsample (n = 510) using a Varimax rotation reduced the eleven items of the RFQ to two components: A prevention factor with an eigenvalue of 2.75 (explaining 25% of total variance) as well as a promotion factor with an eigenvalue of 2.18 (explaining an additional 20% of total variance). Similarly, the scree plot also indicated a distinct decline of explained variance after two factors. The intercorrelation of the extracted factors was r = .13. As reported in Table 3, factor loadings showed strong associations between all items and their respective factor. With the exception of Item 3, which loaded on its factor with .46, all items exhibited loadings of .60 and higher. The MAP test showed that the lowest average partial correlations between items could be found when assuming two factors. Likewise, the PA indicated that eigenvalues of factors one and two were larger than what could be expected with random data sets of the same number of variables and cases with a 95% margin of error. Thus, all methods of EFA suggested a two-factor solution ( Table 4). The EFA clearly suggested a two-factor model. This model was subsequently tested in the CFA using the second subsample (n = 514). Model fit indices for models including and excluding Item 3 are reported in Table 5.
The fit for the model including Item 3 was barely acceptable in terms of CFI and TLI, while showing good fit via SRMR and RMSEA. The exclusion of Item 3 led to sizable improvements across all fit indices. Furthermore, BIC clearly indicated that the model excluding Item 3 fit the data better than the original model. Loadings ranged between .54 and .74 for the prevention factor and between .41 and .52 for the promotion factor, except for Item 3, which loaded very weakly on its factor with .29. After removal of Item 3, the promotion factor loadings improved slightly to between .42 and .54. The correlation of the latent factors was r = .12 with and r = .15 without Item 3.
The analysis of measurement invariance revealed clear evidence for metric invariance across males and females as well as across age groups, as neither the χ 2 -statistic nor the CFI or the GH indicated statistically significant differences (or just barely significant differences in the case of the χ 2 -test for age groups). Scalar invariance  The random data represents the upper limit of the 95% confidence interval of the eigenvalue distribution of 1000 random data sets however could not be confirmed unequivocally. The differences in the GH index for both comparisons were not larger than .01, however both the CFI and the χ 2 -test indicated significant differences between models (Table 6).

Validity
The RFQ Promotion and the RFQ Prevention scales were correlated with the conceptually related scales mentioned in the Introduction in order to examine the construct validity of the RFQ. These scales include: a behavioral-motivational scale (BIS/BAS), a core selfevaluation questionnaire (CSES), a personality scale (BFI-10), an instrument measuring optimism and pessimism (LOT), as well as three short questionnaires assessing somatic and mental health-related constructs (PHQ-4, SSS-8, Health VAS). Correlation coefficients are presented in Table 7.

Differences based on socio-demographic variables
Means and standard deviations of all compared groups are presented in Table 1. Women were found to be significantly more prevention-oriented than men, t(1015) = 3.

Discussion
The aim of the present study was to translate the RFQ into German, to test its psychometric properties, and examine aspects of validation. All items showed good psychometric properties with the exception of Item 3, which displayed a poor correlation with the total scale score. Additionally, factor loadings in EFA as well as in CFA were good for all items except Item 3. Higgins and colleagues [17] had found a similarly small factor loading of .37 for Item 3 on the promotion factor. Therefore, despite the original intention of making the German version of the scale as comparable as possible to the English version, Item 3 had to be excluded from the scale.
Reliability coefficients were good for the prevention scale and questionable for the promotion scale, even with the exclusion of Item 3. Previous research suggests that the application of translated questionnaires in different countries or cultures can lead to a decline in reliability, especially when reverse-scored items are used [59,60]. This could explain the present findings with regard to the promotion scale and Item 3. To put these findings in perspective, it is important to note that even with a reliability as low as .60, strong correlations of up to r = .77 towards other constructs are possible, as evidenced by the high correlations of the promotion scale with the CSES and the LOT.
Fit indices for the two-factor model including Item 3 indicated acceptable fit. However, there was a decided improvement of the fit between data and model, when Item 3 was removed from the promotion scale. Weak factorial (or metric) measurement invariance could be shown for gender as well as age groups. Although strong factorial (or scalar) measurement invariance was indicated for both groups by the acceptable deviation in GH, this evidence is ambiguous because of the larger than .01 difference in CFI between models. Measurement invariance would suggest that participants across groups respond to the given items in a comparable manner with regard to the latent construct. Thus, it is important to unambiguously confirm or reject scalar invariance of the measure in a more representative sample of the general population. We suspect this potential deviation from invariance to be founded in the wording of the original English items, for which there was never an analysis of measurement invariance.
Largely, we could find the expected pattern of correlations for the promotion subscale, but only in part for the prevention subscale. Overall, the promotion scale had moderate to strong associations with most of the employed questionnaires, suggesting good convergent validity. Correlations for the prevention scale were however much lower than they were for the promotion scale. This is in line with previous research consistently showing higher correlations for promotion than for prevention focus [61,62]. We suspect that these low associations might be due to the prevention focus scale's focus on a person's childhood experiences as opposed to current personality traits. This is also a crucial limitation to not just the German version of the scale but the RFQ in general. As predicted, promotion focus correlated positively with behavioral activation and negatively with behavioral inhibition, while prevention focus correlated negatively with behavioral activation and positively with behavioral inhibition. This replicates the findings of Summerville and Roese [15], who found very similar correlations. Promotion focus was shown to be a very good predictor for evaluations of self-esteem and capabilities, as evidenced by the correlation with the CSES. Furthermore, promotion focus displayed relations with all dimensions of the BFInot just openness and extraversion -, while prevention focus was only moderately associated with Conscientiousness, Extraversion, and Agreeableness. In keeping with Hazlett and colleagues [34], promotion focus correlated negatively with pessimism and positively with optimism, while prevention focus did not show the expected associations.
Finally, promotion as well as prevention focus were associated with health-related outcomes, although the (weak negative) correlation of prevention focus was not expected. Several differences in regulatory focus based on sociodemographic variables became apparent. Firstly, females were found to be significantly more prevention focused then men. Secondly, individuals with a lower monthly net household income exhibited higher prevention focus than those with higher incomes. Finally, unemployment was related to lower promotion focus, while students/ apprentices showed higher prevention focus than those working full time. The differences in regulatory focus across employment status groups correspond to a moderately large effect. This is a very interesting finding and may warrant further investigation. Regulatory focus could therefore play a role in developments leading to unemployment or unemployment could lead to a decline of promotion focus.
Regulatory focus is an important construct in personality and social psychology and is highly relevant towards important domains such as work-related outcomes. Therefore, the RFQ can be recommended for application in all fields dealing with motivation and goal attainment processes.

Limitations
When comparing the sample with population averages obtained from the Federal Statistical Office of Germany [63], it became clear that representativeness can not be assumed. The present study sample was relatively young. In addition, women are over-and men underrepresented. Furthermore, the sample was more educated than expected in the general population, with more than 80 % reporting a university entrance qualification, compared to roughly 30 % in the general population. Household net income was reported as lower than the population average, which could also be because of the high number of singles and young people in the sample. Lastly, study participants were more likely to be students or apprentices, and less likely to be working, unemployed, staying at home, or retired. Therefore, the RFQ should be examined with a more representative sample in further studies in order to establish norm values.
Strong measurement invariance could not be shown unambiguously. There is clear evidence for metric invariance for gender and age groups but full scalar invariance could not be demonstrated beyond a doubt. Therefore, the comparisons between sociodemographic groups should be interpreted with caution. Further analysis in representative samples is recommended.
In terms of convergent validity, it became clear that especially the prevention focus subscale warrants further investigation, as it showed weak to moderate correlations across the board, despite good psychometric properties, such as high reliabilityespecially when taken in context of the high correlations the promotion subscale achieved in spite of its low reliability.
Finally, the present study is entirely based on crosssectional self-reports. Therefore, we can not make any predictions with regard to behavior apart from the association with other personality measures.

Conclusion
Overall, the RFQ is a measure of regulatory focus that shows acceptable reliability and good validity towards related psychological constructs. Factor structure and fit between data and theoretical model were very good. Therefore, the German RFQ can recommended for use in research of regulatory focus and practical applications.