What motivates you right now? Development of a measure of momentary-chronic regulatory focus

Abstract

Regulatory focus is a motivational construct that describes humans’ motivational orientation during goal pursuit. It is conceptualized as a chronic, trait-like, as well as a momentary, state-like orientation. Whereas there is a large number of measures to capture chronic regulatory focus, measures for its momentary assessment are only just emerging. This paper presents the development and validation of a measure of Momentary–Chronic Regulatory Focus. Our development incorporates the distinction between self-guide and reference-point definitions of regulatory focus. Ideals and ought striving are the promotion and prevention dimension in the self-guide system; gain and non-loss regulatory focus are the respective dimensions within the reference-point system. Three-survey-based studies test the structure, psychometric properties, and validity of the measure in its version to assess chronic regulatory focus (two samples of working participants, N = 389, N = 672; one student sample [time 1, N = 105; time 2, n = 91]). In two further studies, an experience sampling study with students (N = 84, k = 1649) and a daily-diary study with working individuals (N = 129, k = 1766), the measure was applied to assess momentary regulatory focus. Multilevel analyses test the momentary measure’s factorial structure, provide support for its sensitivity to capture within-person fluctuations, and provide evidence for concurrent construct validity.

Introduction

Since its development, regulatory focus theory (Higgins, 1997, 1998) has received substantial interest from various disciplines in social science. Building on the hedonic principle of approaching pleasure and avoiding pain, regulatory focus theory proposes that there are two orientations of goal striving: The promotion focus is driven by the need for growth and development; it is concerned with reaching one’s ideal self. The prevention focus is driven by the need for safety and security; it is concerned with fulfilling duties, responsibility, and obligations. Regulatory focus (RF) is conceptualized both as a chronic, trait-like orientation and as a momentary, state-like orientation, the latter reflecting responses to situational stimuli and requirements (Higgins, 1997, 1998).

A substantial body of research demonstrates that RF has implications for a wide breadth of phenomena. Experimental research that focuses on a momentarily elicited RF suggests that RF influences goal attainment strategies, decision making, emotions, perception, and other cognitive and behavioral outcomes (e.g., Crowe & Higgins, 1997; Freitas & Higgins, 2002), while field studies that look at the effect of chronic RF demonstrate its impact on a wide range of outcomes, for example, health behavior and consumer choices (Joireman et al., 2012; Kees et al., 2010; Keller, 2006). Studies in the domain of work and organizational psychology have shown that chronic RF affects different facets of organizational performance, work behaviors, and experiences (Johnson et al., 2015; Lanaj et al., 2012; Neubert et al., 2008).

While the interest in RF has been growing, organizational behavior research and other fields of applied psychology have also seen an upsurge of interest in dynamic, short-term processes that take place within people (Fisher & To, 2012). Using daily-diary or experience sampling methodology, the study of within-person variability takes into account that an individual’s level of motivation, affect, or performance is unlikely to be the same at any time of the day or the same across days, weeks, or months (Beal et al., 2005). Identifying the antecedents and consequences of such within-person dynamics has become a growing field of research.

Regulatory focus theory assumes the existence of momentary fluctuations of RF based on situational cues (Higgins, 1997, 1998), and fluctuations are implied in experimental work that activates RF by situational stimuli and demands. Considering that RF affects a wide range of phenomena of relevance for the functioning of organizations—e.g., creativity (e.g., Sacramento et al., 2013), citizenship, or safety behaviors (Lanaj et al., 2012)—and beyond work (Joireman et al., 2012; Keller, 2006), it is important to gain a better understanding of its momentary fluctuations and their antecedents. Being able to track peoples’ momentary RF allows revealing which characteristics of the work environment will trigger which momentary RF, and whether this is the RF that is most beneficial for the current work demand.

Within-person research requires, however, measures that are suitable for within-person research designs, e.g., being sensitive to momentary fluctuations (Hektner et al., 2007). To date, there is a wide range of RF measures suitable for the assessment of dispositions (for an overview, see Lanaj et al., 2012), or for context-specific purposes (e.g., work, Wallace et al., 2009). To the best of our knowledge, there are very few studies that employed state-like measures of RF. Those measures are mostly based on individual items taken from chronic measures of RF, adapted to the workplace (Dong et al., 2015; Koopmann et al., 2016; Lin & Johnson, 2015). However, because of their focus on the workplace, they are confined to be used at the workplace, thus of limited use for research beyond work.

In the light of the fact that there “are very few validated multi-item scales for use in ESM” (Fisher & To, 2012, p. 870), the present research seeks to develop a measure of RF tailored specifically for the use in experience sampling methodology (ESM) and daily-diary studies. The measures will go beyond the recently used measures in three respects. First, we develop a measure that is context free and can thus be applied across a wide range of settings. This is of use for any field of applied psychology interested in identifying factors that shape RF and that seek to track how it translates into relevant outcomes. A context free measure particularly facilitates research that studies how experiences in one domain shape motivation and outcomes in a different domain. For example, research that seeks to unravel how RF at home is affected by events at work needs to measure RF in both domains (for an example on affect, see Harris & Daniels, 2005).

Secondly, in alignment with the requirements of many designs using experience sampling or daily-diary methodologies, we aimed at developing parallel measures of momentary, state-like RF and chronic, trait-like RF. Research trying to understand momentary variations of a given variable often includes the effect of the chronic level of this variable; furthermore, hypotheses on the combined effect of trait and state can be of research interest in its own right (e.g., Fay & Sonnentag, 2012). This is of particular relevance for the study of Regulatory Fit Theory (Higgins, 2000), which holds that alignment (or “fit”) of chronic RF and momentary RF results in motivational and other benefits.

Third, we develop a more fine-grained measure of RF. It has been noted that broad operationalizations of constructs may lead to overlooking specific phenomena. Momentary research benefits from the inclusion of measures with narrower bandwidth: emotion research, for example, even looks at discrete emotions (Brief & Weiss, 2002).

Requirements of measures for within-person research

When developing measures for within-person research, the specific needs of this research methodology have to be taken into account (Bolger et al., 2003; Fisher & To, 2012; Hektner et al., 2007). First, there are issues about the length of items and scales. The item text should be short and simple, and the number of items per scale has to be kept to a minimum (Fisher & To, 2012; Ohly et al., 2010). This helps to avoid participant fatigue which is otherwise likely to result from repeated assessments.

Second, measures for within-person research need to be easily adjustable to the specific time frame required by study design, as this time frame can vary considerably. Some research questions focus on fluctuations that take place within minutes; others examine fluctuations that take place within hours, days, or weeks (Bolger et al., 2003; Hektner et al., 2007; Ohly et al., 2010). Therefore, a new measure of RF should ideally be applicable to a wide range of different time frames, including the assessment of chronic RF. This approach has, for example, been implemented in the Positive and Negative Affect Schedule (Watson et al., 1988).

Third, momentary research benefits from the inclusion of measures with narrower bandwidth because it has been noted that broad operationalizations of constructs may lead to overlooking specific phenomena (see, for example, research on discrete emotions as in Brief & Weiss, 2002). Momentary research oftentimes employs single items (Ohly et al., 2010).

Factorial structure of RF: distinguishing ideals, oughts, gain, and non-loss RF

Existing measures of RF are two-dimensional, distinguishing the promotion from the prevention focus (Higgins et al., 2001; Lockwood et al., 2002; Neubert et al., 2008; Wallace et al., 2009). The present research goes beyond this. Building on RF theory (Higgins, 1997), we aimed at developing two sets of RF measures, each comprising a measure of promotion and prevention focus that map onto the self-guide and the reference-point system of RF. Regulatory focus theory describes each focus as comprising multiple aspects, having several and distinct inputs and consequences (Higgins, 1997). Among the different inputs, two types of regulatory systems are distinguished: The self-guide system describes the personally relevant goals that the self is striving for. The reference-point system encompasses the nature of the hedonic goal (or reference point) pursued (Summerville & Roese, 2008). More specifically, within the self-guide system, Higgins suggested that individuals adopt a promotion focus when they strive for their ideal-self and for growth and nurturance and that they adopt a prevention focus when they strive for their ought-self and for security and safety. The reference-point system of desired end states defines RF such that a focus on achieving gains implies a promotion focus while a focus on achieving non-losses implies a prevention focus. Distinction between self-guide and reference point system maps well onto two experimental procedures typically employed to activate momentary RF (priming procedure, Higgins et al., 1994; goal framing procedure, Liberman et al., 2001). Accordingly, Higgins stated that “the concept of regulatory focus is broader than just socialization of strong promotion-focus ideals or prevention-focus oughts” (Higgins, 1997, p. 1282).

Drawing on Higgins’ conceptualization of RF, we develop measures for two regulatory systems, called self-guide RF and reference-point RF (Summerville & Roese, 2008). This approach enables us to develop dimensions with narrow bandwidth. Each system comprises two foci: a measure of ideals RF (promotion) and oughts RF (prevention) for the self-guide system and gain RF (promotion) and non-loss RF (prevention) for the reference-point system.

The hypothesized measurement models are displayed in Fig. 1. Based on RF theory, we test two sets of models. First, according to RF theory, promotion and prevention focus are two distinct constructs that should map onto two separable factors. We test this by specifying a measurement model with two latent factors; this is done for each system (self-guide and reference-point system) separately (see Fig. 1, model Ia and Ib). We expect small to medium positive interrelations between the respective promotion and prevention dimensions. This relationship results from both dimensions capturing motivation, i.e., a general level of goal striving. The positive relationship between promotion and prevention focus is well-established. Previous studies show substantial variations in effect sizes (e.g., r = .52, Neubert et al., 2008; Φ = .30 to .48, Haws et al., 2010); meta-analyses yielded smaller but significant positive associations (e.g., ρ = .11; Lanaj et al., 2012).

Fig. 1

Measurement models of the Momentary-Chronic RF scales (models III and V depicted without residuals)

In order to provide a more rigorous test of the notion held by RF theory that promotion and prevention are distinct factors, each RF system’s two-factor model (Fig. 1, model Ia and model Ib) is compared with a more parsimonious one-factor model (model IIa and IIb, not depicted), in which all items within each RF system load on the same latent variable.

The second set of measurement models tests how the two RF systems can be integrated. On the one hand, again based on RF theory, the four dimensions (ideals, gain, oughts, non-loss) should be represented as a hierarchical model because each of the two RF systems captures aspects of promotion and prevention. The proposed hierarchical model is depicted in model III (Fig. 1) as a second-order factor model with two correlated second-order factors for promotion and prevention focus. The second-order factors should determine the first-order dimensions ideals and gain (for promotion) and oughts and non-loss (for prevention).

On the other hand, it may well be that the self-guide and the reference point system operate independently, because it has been argued that they represent two unique systems “rather than a single phenomenon” (Summerville & Roese, 2008, p. 253). This idea implies that there should be no significant relationships across systems, i.e., no association between promotion self-guide (ideals) and promotion reference-point (gain) or between prevention self-guide (oughts) and prevention reference-point (non-loss). This theoretical assumption is tested with model IV (not depicted in Fig. 1) in which the self-guide and the reference-point RF are two unrelated systems. Model IV corresponds to the simultaneous test of models Ia and Ib, in which only the respective promotion and prevention factors of each system are allowed to correlate (i.e., ideals and oughts RF, and gain and non-loss RF, respectively), but in which there are no relationships across systems.

Finally, we test a correlated four-factor model (Fig. 1, model V). This model takes two sources of covariation among the four dimensions of RF into account. First, it reflects RF theory which argues that there should be covariance between each of the two promotion and prevention factors, respectively (see model III). Second, it takes into account that each of the four dimensions captures motivation, a general level of goal pursuit (as implied in models Ia and Ib, see Lanaj et al., 2012, for meta-analytically derived evidence). Thus, the four-factor model also includes links between the promotion and prevention factors, within and across the RF systems.

Overview of scale development and validation approach

We followed a standard procedure recommended for measurement development (Hinkin, 1998), incorporating recommendations for momentary measures (Bolger et al., 2003; Hektner et al., 2007; Ohly et al., 2010). We first developed a measure of chronic RF. We compiled a pool of items which was exposed to exploratory factor analyses in a pilot study (N = 220) and then shortened. Study 1 (N = 389) tested the model structure with confirmatory factor analysis (CFA), computed internal consistency, inspected factor loadings, and shortened the measure further. Study 2 (N = 672) re-tested the factorial structure of the shortened measure (CFA), computed internal consistencies, and inspected factor loadings. Study 3 (N = 105, time 1; n = 91, time 2) inspected the test-retest reliability and provided evidence of construct validity in terms of convergent and discriminant validity. To test the momentary version, we conducted a field study with experience sampling methodology (study 4: N = 84, k = 1649) and a daily-diary study (study 5: N = 129, k = 1766). The different types of designs of the momentary studies allow testing the robustness of the proposed structural model across experience sampling versus daily diary. Thus, study 4 and study 5 re-tested the factorial structure of the momentary measure with multilevel-CFA. Studies 4 and 5 also tested the psychometric properties of the momentary version, the sensitivity of the measure to capture within-person variability (1-ICC), and the strength of association between chronic and momentary assessment. Study 5 provides evidence for the construct validity of the momentary measure in terms of concurrent validity. To ensure the wide applicability of the measure, we drew on student (studies 3 and 4) as well as adult populations (studies 1, 2, 5). There was no overlap between any of the samples. All confirmatory factor and multilevel analyses were performed with Mplus 7.11 (Muthén & Muthén, 1998–2013) using the MLR estimator for latent analyses. Descriptive data and correlation tables were computed with SPSS 23.

In studies 1, 2, 4, and 5, incident of missing data was very low (maximum of 1%). In order to retain as much data as possible, we did not eliminate any events. Instead, we worked with the default procedure of Mplus, which is a Full Information Maximum Likelihood (FIML) estimation. An exception to this are study 4 analyses that include trait or demographic data. Here, we excluded three participants that had not returned the questionnaire that assessed the relevant characteristics. Furthermore, in study 3, n = 14 individuals failed to participate at wave 2. They were not included in the estimation of the test-retest reliability or convergent validity.

To summarize, evaluation takes place in terms of factorial validity (CFA, multilevel-CFA), internal consistency (Cronbach’s α and multi-level reliability), test-retest-reliability, convergent and discriminant validity, within-person variability, momentary-chronic-linkage, and concurrent validity.

Methods and results for the development of a chronic measure of RF

Item development and pilot test

We screened existing measures of RF (e.g., Higgins et al., 2001; Lockwood et al., 2002; Neubert et al., 2008; Wallace et al., 2009) for items that could be modified such that they would be aligned with the goals of the new measure (i.e., usable for momentary assessments, mapping unambiguously either the reference-point or self-guide RF, context free, adaptable to various time frames). Some items implied contextual restrictions (e.g., “My major goal in school right now is to achieve my academic ambitions”; Lockwood et al., 2002), while others were not suitable for momentary assessments (“Did you get on your parents’ nerves often when you were growing up?”; Higgins et al., 2001). We translated the most suitable items into German and revised them such that they were in accordance with the goals described above.

To enable the adaptation of the measure to different time frames, we formulated the items such that the time frame (i.e., “right now,” “today,” “in general”) was implemented through the instruction without altering the stem or the items per se (e.g., as in PANAS, Watson et al., 1988).

The instruction for the chronic version was as follows: (Instruction) “Please rate how well each of the following statements applies to you in general. A rating of 1 signifies that a statement doesn’t apply to you in the least, whereas a rating of 7 signifies that it describes you perfectly. Use the ratings in between to denote more precisely intermediate degrees of correspondence. (Stem) My thoughts and actions are mainly directed toward:

• pursuing my hopes

• fulfilling my responsibilities …”

The initial pool of 30 items (ideals and oughts RF operationalized with seven items each, gain and non-loss RF with eight items each) was administered to 220 psychology undergraduates. An initial test of whether the items for each factor would result in a unidimensional solution (based on exploratory factor analyses) led to the elimination and rewording of items. This resulted in a set of 20 items altogether (five for each factor) which we tested in two studies.

Study 1: Item reduction and initial test of factorial structure

Sample and procedure

In order to run a more rigorous test of the factorial structure via confirmatory factor analysis (CFA) and for further item reduction, the measure was applied to a mixed convenience sample of working individuals (N = 389) using an online survey. The sample had a high level of school education (82.3% higher education entrance qualification, 9.8% 5 years of secondary school education, < 1% no school graduation). The majority was employed full-time (69.7%).

Results

We tested the measurement models Ia and Ib depicted in Fig. 1. All fit indices are summarized in Table 1. Results of the CFA for the self-guide model with each of the 10 items loading on its corresponding factor were just acceptable (Browne & Cudeck, 1993; Hu & Bentler, 1995), with the exception of the CFI. The fit of the reference-point model was acceptable. To obtain a more parsimonious measure, we removed one item from each factor based on factor loading or content overlap with another item (item wording of removed items obtainable from first author; full item wording of final set of items is presented in Additional file 1). The resulting measurement model with four items per dimension had a good fit for the self-guide model, as well as for the reference-point model. Items loaded significantly on their respective factor, and all internal consistencies (Cronbach’s alpha) exceeded α = .80 (see Table 2). The latent correlation of the self-guide measures (ideals/oughts) was Φ = − .01 (p = .863); for the reference-point measures (gain/non-loss), it was Φ = .44 (p < .001). Means (and standard deviations) of the four measures are as follows: ideals = 4.90 (SD = 1.02), oughts = 4.96 (SD = 1.09), gain = 5.53 (SD = .97), and non-loss = 4.85 (SD = 1.32).

Table 1 Confirmatory factor analysis of the Momentary–Chronic RF scales

Table 2 Loadings for confirmatory factor analyses (standardized path coefficients) and internal consistencies by study

To test whether the data could be modeled more parsimoniously, we calculated the model fit of one-factor models for each regulatory system. The fit indices of the one-factor “self-guide” model fell far below acceptable levels; the one-factor “reference-point” model resulted in a comparably poor fit (models IIa and IIb in Table 1).

Study 2: Replication of the factorial structure and psychometric properties

Sample and procedure

We sought to replicate the factorial structure Ia and Ib in a larger sample of working individuals. Data were collected among students enrolled at a distance learning university in an undergraduate program of Psychology. Using an online survey, we obtained N = 963 completed questionnaires. Excluding careless responders (e.g., on the basis of response time, n = 63) and focusing on individuals who were currently employed resulted in a sample of n = 672. Descriptive data are presented in Table 3.

Table 3 Descriptive statistics and intercorrelations of chronic RF and demographic variables (studies 2 and 3)

Results

The two factor model of self-guide RF led to a good fit; the fit of the two-factor model of reference-point RF was also acceptable (see Table 1). Each item loaded significantly on its factor; Cronbach’s alphas exceeded α = .84 (see Table 2). The latent correlation of the self-guide measures was Φ = .01 (p = .802), and of the reference-point measures Φ = .18 (p < .001). Replicating the findings of study 1, the respective one-factor models did not fit the data well (models IIa and IIb in Table 1).

Intercorrelations among the four RF dimensions were positive or non-significant; the highest correlations emerged across regulatory systems between ideals and gain and oughts and non-loss. Correlations with age and gender were small or non-significant (see Table 3, bottom triangle).

Integration of the two RF systems

Given the positive correlations of dimensions across RF systems (Table 3), we tested whether and how the two systems can be integrated. To this end, we compared three alternative models (models III, IV, and V) described earlier. Results are displayed in Table 1. When testing model III, the second-order factor model with two correlated second-order promotion and prevention focus factors, we encountered an identification problem. We reduced the complexity of the model by constraining the latent residuals of the first-order dimensions (ideals and gain RF) to be equal. We proceeded likewise with the two first-order dimensions of the second-order factor prevention focus. Model fit was good. For model IV, which assumes two unrelated systems of RF, the CFI and SRMR exceeded the conventional standards. The structure of model V, a correlated four-factor model, also fit the data well. Thus, overall, results of the factor analyses indicate that the two systems of RF are not unrelated; instead, the data can be well modeled as a second-order structure with a promotion and a prevention factor.

Study 3: Validation of the chronic RF measure through test-retest reliability and convergent and discriminant validity

Sample and procedure

Psychology undergraduates filled in two online surveys to which they responded with a time lag of at least 24 h. The first survey included the new RF items, other measures (not reported here), and demographic data (N = 105, time 1). The second survey presented again the RF items as well as other existing measures of RF (more details follow below; n = 91, time 2). The internal consistencies were good (Cronbach’s α for time 1 and time 2, respectively: ideals = .85/.88; oughts = .79/.84; gain = .83/.77; non-loss = .91/.94).

Test-retest reliability

We calculated the test-retest reliabilities, restricting the time lag between measures to 8 days maximum (M = 2.55 days; n = 83). Test-retest reliabilities were high r_{t1 − t2} = .72 (ideals), r_{t1 − t2} = .85 (oughts), r_{t1 − t2} = .73 (gain), r_{t1 − t2} = .83 (non-loss), p < .01, and they were comparable to the retest reliability of other RF measures (r_{t1 − t2} = .62 to r_{t1 − t2} = .75, five-week time lag, Haws et al., 2010).

Convergent and discriminant validity

Convergent and discriminant validity was explored with widely used measures of RF (Higgins et al., 2001; Lockwood et al., 2002; Wallace et al., 2009). We inspected the correlations between (a) the new ideals and gain RF and existing promotion focus measures and (b) the new oughts and non-loss RF and existing prevention focus measures to obtain information on convergent validity. We expected large- as well as medium-sized effect sizes for three reasons. First, existing measures mix items referring to the self-guide and the reference point; second, some of the measures are context specific (e.g., Lockwood’s measure relates to the academic context; Wallace’s measure to the domain of work). Information on discriminant validity is collected by inspection of the correlations between (c) the new ideals and gain RF and existing prevention focus measures and (d) the new oughts and non-loss RF and existing promotion focus measures.

Data were collected in the time 2 survey. Results are displayed in Table 4. Of the 12 correlation coefficients representing convergent validity, nine were of medium (> .30) or large (> .50) effect size (n = 91, p < .01; Cohen, 1988). The average convergent correlation (based on r-to-z transformation) was .43. The average of all discriminant correlation coefficients (absolute values) was .16, which corresponds well with the meta-analytically derived relationship between promotion and prevention foci of ρ = .11 (Lanaj et al., 2012).

Table 4 Means, standard deviations, correlations, and internal consistencies of the new and existing RF measures (study 3)

Even though this data speaks in general for the convergent and discriminant validity, a closer look reveals some inconsistencies. First, the two convergent coefficients for oughts RF and non-loss RF showed little convergence with Higgins’ prevention focus measure. Lack of convergence of the Higgins’ measure with other RF measures has been reported elsewhere. For example, in two independent samples, Haws et al. (2010, p. 971) report zero or negative convergent correlations of Higgins’ prevention focus with Lockwood’s prevention measure (r = − .14, ns; r = − .18, p < .01). Similarly, a daily-diary study which applied Higgins’ RF to predict daily goal striving showed that Higgins’ chronic prevention focus did not predict daily prevention goal focus (Eddington et al., 2012). These different findings together suggest that Higgins’ prevention measure captures an element of prevention focus not included in the other measures.

Second, the discriminant relationships of oughts RF and non-loss RF with the Wallace promotion focus measure were positive and significant. However, the convergent relationships with the Wallace prevention focus measure are substantial and significantly higher than the discriminant links with promotion focus (p < .05, one-tailed test). We attribute the association of oughts and non-loss with a promotion focus measure to the slight tendency that some people are in general more “motivated” than others, i.e., that they have a higher level of goal striving. This also reflects in small but substantial relationships between promotion and prevention measures (Lanaj et al., 2012) and may also account for the positive association between gain RF and the Wallace prevention measure.

Summary of chronic RF

We applied the self-guide and reference-point RF measures to assess the chronic RF in three samples (N = 1166). Internal consistencies were good across all samples and dimensions; the test-retest reliability was appropriate. Confirmatory factor analyses demonstrate that we developed four dimensions that can be most parsimoniously modeled as two promotion and two prevention factors. Construct validity in terms of convergent and discriminant validity was satisfactory.

Methods and results for the administration of the new RF items as a momentary measure

In a next step, the items were applied with an instruction that should permit the assessment of the momentary RF. The following two field studies are dedicated to (a) testing the factorial structure of the new RF items in its momentary version, (b) assessing the level of within-person variability, (c) testing their relationship with chronic RF, (d) and providing evidence for their construct validity.

Study 4: Experience sampling method study

Sample and procedure

We performed a study using the ESM. Eighty-four psychology students carried a handheld computer with them for five consecutive days. They were compensated with course credits. The participants provided data for k = 1649 measurement points.

Study participants were requested to first complete a paper-pencil questionnaire assessing their chronic RF and demographic data. All but three participants returned the questionnaire, resulting in a sample size of n = 81 for analyses on chronic RF. Each participant was individually briefed by a research assistant on how to use the handheld computer. The handheld computer prompted participants to fill in a survey four times each day. The instruction for the momentary version was identical with the chronic instruction, except from the time referent: “… in general” was replaced by “… at this very moment”. The item stem remained unchanged and was followed by the items as presented in Table 2 (see Additional file 1 for full item text and instruction). We applied the same 7-point response format as before.

In order to test the concurrent validity of the momentary measure, we further obtained information on the type of activity participants were pursuing at the moment of measurement. Based on a pilot study, we distinguished several types of activities that capture what students do during a typical week: attending university-related tasks (attending classes, preparing for exams, completing coursework assignment); other, non-university obligations (working in a part time job, performing household chores); recovery and idling (including internet surfing, reading, watching TV, eating, napping, body care); pursuing active, goal-oriented leisure time activities (sport regularly performed, pursuing one’s hobbies); social activity (spending time with friends, family, or partner); and traveling (e.g., to work, university). Participants could tick “other activity” in case none of the previous categories applied.

Factorial structure of the momentary RF

Because of the nested nature of the data—repeated momentary measurements nested in people—we tested the measurement models for the ESM assessments with multilevel confirmatory factor analyses (e.g., Roesch et al., 2010). The measurement model of self-guide RF with two factors (Model Ia) resulted in a good fit (Table 1); the same is true of the reference-point measurement model (model Ib). All factor loadings were significant. Multi-level internal consistencies were calculated following the procedures described by Geldhof et al. (2014). At the within-person level as well as at the between-person level, ω was ≥ .90 (Table 2). The fit indices of the respective one-factor models fell below acceptable levels (models IIa and IIb; Table 1). Descriptive statistics and intercorrelations of the momentary RF dimensions are presented in Table 5 (bottom triangle).

Table 5 Descriptive statistics and intercorrelations of momentary RF and demographic variables (studies 4 and 5)

Corresponding to the chronic measure, we tested models III to V which integrate the two regulatory systems. However, because multilevel factor analyses are very complex, in the present sample the number of free parameters exceeded the number of clusters (i.e., persons). We therefore simplified the models by restricting the factor loadings on each first-order dimension to be equal (within the same dimension); for model III, we also constrained the latent residuals of the first-order factors to be equal (within the same second-order factor). Simplification then permitted model estimation. Model III, the second-order factor model, had a good model fit with only the SRMR-between exceeding conventional standards (Table 1). Model V, the less parsimonious correlated four-factor model, resulted in a slightly better model fit. In contrast, for model IV, which assumes that both systems are unrelated, CFI and SRMR fell below acceptable standards.

Intraindividual variability

A key test of the measure for its suitability as a momentary measure of RF is its level of intraindividual variability across measurement points. Overall, intraindividual variability (i.e., 1-ICC; n = 84, k = 1649) was substantial (ideals = .600; gain = .794; oughts = .759; non-loss = .603). This suggests that the measure is sufficiently sensitive to capture momentary changes (Binnewies et al., 2009).

Relationship of momentary RF with chronic RF

Momentary RF should be positively associated with the chronic level of RF (Zuckerman, 1983). Since state and trait assessments tap into the same construct, they can be used for validation purposes (Hektner et al., 2007). We used the chronic assessment of RF completed prior to the ESM-study and the ESM-based data (n = 81, k = 1588) to estimate the relationship between the chronic RF and its respective momentary counterpart. In a multilevel structural equation model (SEM), we included the chronic RF at the between-level and modeled its latent correlation with the between-level momentary RF. We conducted the analyses separately for each dimension (i.e., ideals, oughts, gain, non-loss RF). Model fit indices of the four models were good (all CFI ≥ .97, RMSEA ≤ .05, SRMR within < .01, SRMR between ≤ .08). The latent relationships between the momentary and chronic RF dimensions were substantial (ideals: Φ = .61, gain¹: Φ = .57, oughts: Φ = .54, non-loss: Φ = .55, all p < .001). These relationships fall well within the range of state-trait relationships observed for other individual difference variables (Augustine & Larsen, 2012).

Study 5: Daily-diary study

Sample and procedure

We further performed a daily-diary study with adults, specifically, employed individuals. The study purpose was twofold: First, we sought to test the factor structure of momentary RF with working individuals (study 4: students). Second, we sought to test the intraindividual variability of the measure with a time frame that would relate to a part of the day (i.e., morning or afternoon). This timeframe is frequently used in daily-diary studies. Given the longer time frame that the assessment of RF would relate to in this study, we expected the intraindividual variability to be lower than in the ESM study (which asked for the RF “at that moment”).

Data were obtained from 129 employees from different organizations (e.g., clerical staff from statutory health insurance, employment agencies, public services such as water and electricity). Participation was voluntary and compensated with 30 Euros. All participants had a minimum of 5 years of secondary school education. Data were collected across eight working days with two daily assessments of RF. On average, participation lasted for 7.1 days (range 3–12 days).

Demographic data were collected with a paper-pencil questionnaire (see Table 5); the questionnaire also included the chronic RF. A handheld computer prompted participants twice each day (late morning and afternoon) to fill in the momentary RF items. The instruction included the following time referent for the morning and afternoon assessment, respectively: “The following statements describe what could motivate and drive you at work this morning / this afternoon. Please rate how well each of the following statements applies to you this morning / this afternoon. My thoughts and actions this morning / this afternoon are mainly directed toward …”. We retained the 7-point answering format.

Deleting measurements that did not follow the study protocol (e.g., morning assessment that was filled in in the afternoon) resulted in k = 1766 usable measurement points in total, 920 of which were morning assessments and 846 were afternoon assessments.

Factorial structure of momentary RF

Measurement models of the daily-diary data were again tested with multilevel CFA. To reduce the complexity of the analyses, the time point of measurement—morning or afternoon—was not taken into account. The measurement models of the self-guide and of the reference-point RF measures (Models Ia and Ib) resulted again in a good fit (Table 1), with exception of the SRMR-between of the reference-point model. All factor loadings were significant, and multi-level internal consistency ω ranged between .78 and .86 at the within-person level and between .93 and .98 at the between-person level (Table 2). The latent correlation of the self-guide measures was Φ = .04 (ns) and .11 (p < .001) at the within- and between-level, respectively, and Φ = .29 and .46 (p < .001) at the within- and between-level of the reference-point measures. Tests of the respective one-factor models indicated again inacceptable model fit (models IIa and IIb, Table 1). Descriptive statistics and intercorrelations of the momentary RF dimensions are presented in Table 5 (top triangle).

We tested models III to V using the same strategies for model simplification as in study 4. Again, for model IV, CFI and SRMR fell below acceptable standards. Model V, the correlated four-factor model, resulted in a good model fit (Table 1). When estimating model III, we encountered matrix identification problems, which could be resolved by setting the correlation between oughts RF and gain RF free. Fit indices were then acceptable and comparable to the four-factor model with the exception of the SRMR-between, which exceeded conventional standards. Thus, for the daily-diary data, the correlated four-factor model showed a better fit.

Intraindividual variability

Intraindividual variability (i.e., 1-ICC) in the morning and afternoon, respectively, was .330 and .292 for ideals, .392 and .339 for gain, .432 and .388 for oughts, and .251 and .223 for non-loss RF. This variability is substantial and, as expected, considerably lower than the variability of the ESM Study.

Replication: relationship of momentary RF with chronic RF

To test the linkage between the momentary RF and its respective chronic counterpart again, we applied the same approach as in study 4 (see the “Study 4” and “Relationship of momentary RF with chronic RF” sections). We used the chronic assessment of RF completed prior to the daily-diary study and the daily-diary data (n = 129, k = 1766). In a multilevel SEM, we included the chronic RF at the between-level and modeled its latent correlation with the between-level momentary RF. Model fit indices of the four models were good (all CFI ≥ .95, RMSEA ≤ .05, SRMR within ≤ .04, SRMR between ≤ .06). The latent relationships between the momentary and chronic RF dimensions were somewhat smaller than in study 4 but substantial (ideals: Φ = .45, gain: Φ = .60, oughts: Φ = .54, non-loss: Φ = .43, all p < .001).

Concurrent validity of RF

To test the concurrent validity of the momentary RF assessments, we tested whether the level of momentary RF is systematically associated with the type of activities individuals pursue at the moment of measurement. This assumption is based on two related literatures. First, RF theory holds that situational cues affect momentary RF (Higgins, 1997). The day-to-day activities that people pursue differ in the types of cues and goals they expose people to, and the particular goals pursued should activate different types of motivational orientations through enhancing accessibility. Second, the goal framing manipulation used in RF experiments has been successfully used to manipulate participants’ momentary RF through setting participants a promotion (or prevention) goal and making them pursue this goal (e.g., Liberman et al., 2001). Thus, momentary RF should be a function of the activity currently pursued, because activities differ in the nature of associated goals.

This notion could be tested within study 4. Participants indicated for each moment of data collection which activity they were currently pursuing (see the section “Sample and procedure,” study 4). Ideals RF should be higher in situations that relate to the pursuit of goals that imply personal growth and that are self-set (e.g., active leisure activities, university-related tasks); oughts RF should be higher during the pursuit of externally set goals, and in attempts to meet obligations (e.g., other obligations such as part time job, household chores) and low during recovery and idling; gain RF should be elevated in activities that permit achieving positive end results (active leisure activities, university related tasks), and low during recovery and idling; and non-loss should be high in situations where something of significance “can go wrong” (e.g., university, other obligations).

We performed multi-level regression analyses to test this. Six dummy variables (1 = activity pursued; 0 = activity not pursued) code the six activities that the participants pursued at the time of momentary assessment. The effect of each activity was tested in a separate multi-level regression analysis. This approach tests whether, for example, the level of momentary ideal striving is significantly higher when involved in university-related tasks in comparison to all other activities. To account for the increased risk of type I error due to repeated analyses, we applied a Bonferroni-correction and report only effects with p < .008. Chronic RF, age (both uncentered), and gender were included as level 2 predictors. Results are displayed in Table 6. Chronic RF was always a significant positive predictor of the level of the respective momentary RF. With regard to the momentary predictors (i.e., activities), the patterns of results indicate support for the notion that momentary RF differs as a function of activity. Ideals RF is significantly higher when involved in active leisure (e.g., pursuing hobbies, involved in sports, cultural activities), or university-related tasks, and significantly lower when pursuing other obligations or during recovery and idling. Gain RF follows a similar pattern but is not lower when pursuing obligations; instead, gain is lower during social activities. Oughts RF is highest when engaged in university-related tasks and during other obligations (part-time job, household chores), and low during recovery and idling, active leisure, and social activities. Non-loss has some similarities with this pattern but it also differs from oughts RF such that it is not affected by active leisure or other obligations. Overall, the pattern of effects is aligned with the generic assumption that activities differ in the types of goals pursued (i.e., growth, obligations) which should in turn be associated with specific levels of RF.

Table 6 Multilevel results for models predicting ideals, gain, oughts, and non-loss from type of activity (study 4)

General discussion

Motivated by the increasing interest in organizational behavior research and other fields of applied psychology in dynamic, within-person processes and the likewise continuous interest in RF (Higgins, 1997, 1998), the present paper describes the development of a measure suitable to capture momentary RF. Because within-person study designs such as daily-diary or experience sampling studies often require trait-measures too, our approach included also the development of a parallel measure of chronic RF. We conducted five studies to test the measure. Three questionnaire-based studies tested the chronic RF, one ESM study and one daily-diary study assessed momentary RF. Together, the studies support the existence of four distinct RF dimensions and consistently indicate a good internal consistency of each dimension. The momentary measure shows sufficient within-person variability, adequate relationship with chronic RF, and concurrent validity.

The exact magnitude of within-person variability differed by the specific time referent used for RF assessments and the location of assessment. As expected, within-person variability was comparatively higher in study 4 than in study 5. In study 4, participants reported their RF “at that very moment”. A truly momentary assessment is more likely to catch more extreme peaks of RF, increasing the measured variability. Furthermore, in study 4, situational stimuli differed from one assessment to the next (RF assessed during social activities, when attending university-related tasks, working in a part-time job, etc.), and participants responded at different times of the day. In contrast, in study 5, participants reported their RF for a longer time frame, e.g., “during the morning”, mostly in the same context (i.e., at work), and mostly at a similar time each day. Differences in variability between studies 4 and 5 are therefore the result of timeframe and variability of external stimuli (Higgins, 1997, 1998) and, thus, reflect meaningful differences in the phenomenon assessed. This finding also provides evidence for the sensitivity of the measure.

The measure of momentary and chronic RF contributes to existing measures in several respects: First, it is a measure that is suitable to perform within-person research which can be applied across a wide range of settings, and is easily adaptable to different time frames. It is not confined to the use at the workplace (Koopmann et al., 2016; Lin & Johnson, 2015) and is thus not only suitable to study spill-over effects (e.g., work-nonwork) but may be useful to researchers from a wide range of applied researches. Researchers interested in the “natural” antecedents of momentary RF, and into the consequences of these fluctuations in a stream of behavioral episodes, might find this measure useful (Beal et al., 2005).

Second, the momentary measure is supplemented with a parallel chronic measure of RF. This will be particularly useful for the study of research questions around the notion of regulatory fit (Higgins, 2000). A parallel measure of state and trait should permit a more reliable and valid assessment of the level of “fit” experienced at a given point in time. It needs to be pointed out, however, that the new prevention measures did not converge with Higgins’ prevention measure (Regulatory Focus Questionnaire, RFQ, see Higgins et al., 2001). This lack of convergence has already been reported for other RF-measures, too. Taking the nature of the RFQ into account may help to shed some light on this. The RFQ assesses the “subjective history of success with promotion-related eagerness (promotion pride) … [and the] subjective history of success with prevention-related vigilance (prevention pride)” (Higgins et al., 2001, p. 15). Lack of convergence of RFQ prevention means that individuals who are driven by rules, responsibilities, and obligations (oughts), or keen on avoiding losses or mistakes (non-loss), have not necessarily had a past history of success in prevention-related vigilance (RFQ). Research that focuses on chronic RF should carefully consider their choice of chronic measure because it seems that the RFQ-prevention covers an aspect of RF not included in other measures; including RFQ may help to cover all aspects of RF.

Third, our goal to develop measures with narrower bandwidth permitted to take the distinction between self-guide RF and reference-point RF into account. This distinction is implied by RF theory (Higgins, 1997) and also resembles the distinction between two experimental procedures typically employed to activate momentary RF (priming procedure, Higgins et al., 1994; goal framing procedure, Liberman et al., 2001). The factor analyses indicate that four dimensions of RF can be separated; results on the higher-order structure, however, are so far not fully conclusive. The fit of the second-order structure was almost as good as the less parsimonious correlated four-factor structure; only in the last study, there was a strong relationship across the two second-order constructs (i.e., between oughts and gain RF). It needs to be noted, however, that direct comparisons between the second-order structure and the correlated four-factor model must be made with caution. Both multilevel models required imposing constraints on the model; however, the complexity of the multilevel second-order factor model required more constraints than the correlated four factor model. This implies that the second-order model is always more likely to obtain a poorer model fit than the comparatively less constrained correlated factor model.

Measures with narrower bandwidth can further contribute to future research, in particular within-person studies (e.g., Brief & Weiss, 2002), because broad operationalizations of constructs involve the threat of overlooking specific phenomena. Thus, the present measure that separates the self-guide and reference-point system of RF will be helpful to detect more differentiated aspects and consequences of RF.

Limitations and future research

Future studies need to gather more evidence on the construct validity of the momentary RF, for example, in terms of its nomological network. The literature provides evidence for the nomological network of the chronic assessment of RF (e.g., Wallace et al., 2009); whether this nomological network likewise applies to the momentary RF needs to be investigated. However, so far, validated momentary measures of the respective nomological network variables that could be used for validation purposes are scarce (Fisher & To, 2012).

Depending upon the data collection protocol, future research might be interested in using even shorter scales than the four item measures presented here. Unfortunately, formal guidance on how to shorten ESM measures is scarce (Gabriel et al., 2018). A common approach is to select items with the highest factor loadings. Given that loadings were high across all studies (see Table 2), they are of little help for the present measure. We therefore suggest to include two items per factor that represent the respective construct best from a substantive point of view and that have good intra-individual variations. Items that fulfill the two criteria best are identified in Table 2 and in the Additional file 1.

Considering the increasing interest in the role of RF in numerous areas of applied psychology, this measure can become useful in conducting process-oriented research. For example, in organizational behavior, specifically leadership research, the RF is seen as one motivational orientation that links specific leadership variables with subordinates’ workplace behavior (Neubert et al., 2008). The momentary measure of RF permits to study which specific supervisor behaviors momentarily enhance promotion and prevention focus (Kark & van Dijk, 2007). Furthermore, if leaders seek to influence or shape the RF of their followers, studying momentary RF permits a systematic test of whether the act of influence will be likewise effective for each follower. Related to the notion of fit, capturing momentary RF permits scholars of organizational behavior, for example, to test whether the work environment elicits a RF appropriate for the task or work demand at hand (Lanaj et al., 2012). The momentary approach can also help to better understand individual events of behaviors, e.g., in the study of safety performance. Leadership behaviors shape subordinates’ RF (Neubert et al., 2008), and subordinates’ RF, in turn, affects their safety performance (Wallace & Chen, 2006). Capturing momentary RF will aid scholars of organizational behavior to identify situations that are particularly susceptible to accidents (e.g., situations of high overload; Zohar, 2000) and the specific supervisor behaviors that help to elicit the appropriate RF.

Above and beyond work, the study of momentary and chronic RF might be of interest to many disciplines, because the nature, antecedents, and consequences of the intraindividual variability of a phenomenon may differ in direction or magnitude from interindividual differences (Dalal & Hulin, 2008, p. 80). For example, proactivity research revealed that daily proactivity was associated with higher levels of daily cortisol output (intraindividual analysis), whereas trait proactivity was associated with lower levels of cortisol output (interindividual analysis) (Fay & Hüttges, 2017).

Conclusion

Together, results of five studies provide sufficient information on the validity of the instrument to justify its use. The key contribution of the present paper is the development and validation of the momentary measure. Considering the special requirements placed on momentary assessments, the new measure obtained good results. It is sensitive to capture intra-individual variation; the magnitude of intra-individual variation differs meaningfully by study design; results on concurrent validity are well in line with theoretical predictions; there is good convergence of the momentary measure with the chronic assessment. Overall, in our view, this justifies the employment of the measure in the field; however, at the same time, we acknowledge that our approach to validation could be expanded by, for example, applying the measure in diary studies with weekly designs or applying it to more specific domains, such as health behaviors.