The short version of the Metacognitive Prospective Memory Inventory (MPMI-s): factor structure, reliability, validity, and reference data

Abstract

Prospective memory, the ability to remember to execute an intended action at the appropriate moment in the future, is frequently assessed with standardized questionnaires. Prospective-memory abilities strongly depend on the different strategies people use to remember their intentions. In this study, we introduce the short version of the Metacognitive Prospective Memory Inventory (MPMI-s) that allows for a quick assessment of individual differences in self-reported prospective-memory abilities as well as in the use of mnemonic strategies (e.g., intention rehearsal) and external memory aids (e.g., calendars). Based on data from two waves of the GESIS longitudinal panel, we provide evidence that this novel questionnaire offers reliable and valid measures of prospective-memory abilities as well as of internal and external strategy use. As the panel sample is representative of the German population, we are further able to provide reference data that allow evaluating individual PM ability and strategy-use scores obtained with the MPMI-s.

Introduction

Prospective memory (PM) refers to the ability to remember an intention at the appropriate moment in the future (Cohen & Hicks, 2017). Typical everyday examples of PM tasks are remembering to take a cake out of the oven after 20 min, remembering to buy a birthday present for a significant other, or remembering to take prescription pills after breakfast. From these examples, it is obvious that PM failures can have negative personal consequences, from a burnt cake or missing present ruining a birthday party to severe health issues from forgetting to take one’s medicine. PM not only plays an important role in our daily lives but also in work environments (Dismukes, 2012) and for neurological and clinical disorders (cf. Raskin, 2018). PM deficits are associated with several disabling clinical disorders, such as (mild) cognitive impairments, schizophrenia, and Parkinson’s and Alzheimer’s disease (Costa, Carlesimo, & Caltagirone, 2012; Farina, Young, Tabet, & Rusted, 2013; Zhou et al., 2017). Therefore, brief screening tools for PM impairments are of crucial importance. To this end, we present a 22-item questionnaire that allows to not only reliably asses self-perceived PM abilities but also use of strategies that may influence PM abilities in daily life. We will show that the short version of our newly developed Metacognitive Prospective Memory Inventory (MPMI-s) has good psychometric properties. We will also test for its convergent and discriminant validity by investigating whether the MPMI-s scales are, as shown for previous PM ability questionnaires, moderately related to conscientiousness and only weakly to other personality variables (Uttl & Kibreab, 2011). We will further investigate to which extent the MPMI-s scales predict everyday planning behavior. Finally, we will provide reference data for the MPMI-s based on a large representative German sample.

The cognitive underpinnings of PM have been studied extensively within specifically devised laboratory tasks (see Cohen & Hicks, 2017, for a recent overview). Additionally, standardized objective (performance-based) tests are available to assess individual differences in PM abilities (Raskin, 2009; Wilson et al., 2005). However, the use of these tasks in the diagnosis of clinically relevant PM deficits can be hampered by practical limitations such as rather extensive administration times. Critically, such tasks include artificial experimenter-imposed intentions and thus performance on these tasks may have little relation to performance on self-imposed everyday PM tasks. Indeed, there are striking differences between the performance in laboratory/experimenter-generated and naturalistic/self-imposed PM settings (Arnold & Bayen, 2019; Bailey, Henry, Rendell, Phillips, & Kliegel, 2010; Schnitzspahn et al., 2016). Thus, although some have questioned the validity of self-reported everyday PM abilities based on its non-correlation with laboratory PM performance (Uttl & Kibreab, 2011), self-reports of PM—given that they can be reliably measured—can provide crucial insights beyond artificial laboratory PM tasks. Indeed, there is good evidence for the clinical relevance of self-perceived memory abilities, which predict conversion to dementia in older adults without objectively detectable memory deficits (Mitchell, Beaumont, Ferguson, Yadegarfar, & Stubbs, 2014).

A few questionnaires to assess perceived everyday PM abilities have been developed previously, such as the Prospective Memory Questionnaire (PMQ; Hannon, Adams, Harrington, FriesDias, & Gipson, 1995), the Comprehensive Assessment of Prospective Memory (CAPM; Chau, Lee, Fleming, Roche, & Shum, 2007), or the Prospective and Retrospective Memory Questionnaire (PRMQ; Crawford, Smith, Maylor, Della Sala, & Logie, 2003; Smith, Della Sala, Logie, & Maylor, 2000). However, the PMQ and the CAPM comprise 52 and 39 items, respectively. In many test situations, it is not feasible to use such extensive scales (Rammstedt & Beierlein, 2014; Stanton, Sinar, Balzer, & Smith, 2002). Indeed, the most frequently used PM questionnaire is the PRMQ, which comprises only eight PM ability items (complemented with eight retrospective-memory-ability items). This is also the only questionnaire for which normative data is available. However, the PRMQ, as well as the other PM questionnaires, exclusively rely on items that assess PM failures and are thus prone to acquiescence biases (Danner, Aichholzer, & Rammstedt, 2015; Danner & Rammstedt, 2016). Further, laboratory and everyday PM performances strongly depend on the use of mnemonic strategies, such as intention rehearsal, imagery, etc., as well as of external memory aids, such as calendars, to-do lists, etc. (e.g.,Gilbert, 2015; Penningroth & Scott, 2013; Shelton et al., 2016). Notably, this is particularly true for groups with PM impairments when performing naturalistic tasks, such as older adults who need to remember to make phone calls (Maylor, 1990) or patients with brain injuries who need to keep track of their activities (McDonald et al., 2011). Thus, we deem it important to assess the frequency with which people use such strategies alongside PM abilities because they (a) allow for a better interpretation of perceived PM abilities (e.g., is PM ability normal given the strategies used or are people already compensating deficits via extensive strategy use?) and because they (b) may identify means for interventions (e.g., could more external aids be used to avoid PM failures?).

To this end, we recently developed and tested a novel questionnaire specifically designed to assess perceived everyday PM abilities and PM-strategy use: the Metacognitive Prospective Memory Inventory (Rummel, Kuhlmann, & Danner: A questionnaire for the asssessment of perceived prospective memory abilities and strategy use: The metacognitive prospective memory inventory (MPMI), in preparation). The original 44-item version of the MPMI comprises three scales for the assessment of PM abilities and PM-strategy use in everyday life, with strategies being further differentiated into internal (i.e., cognitive) and external (i.e., memory-aid) strategies. For the present MPMI short version, we selected those eight items of each scale that (a) covered a wide range of everyday PM situations and strategies, (b) loaded highest on their respective scale and not on the other scales, and (c) were rated as most content valid by two PM experts. Also, half of the PM-ability-scale items are reverse-keyed such that item-wording effects—and, among them, acquiescence biases—can be controlled for (Billiet & McClendon, 2000; Weijters, Baumgartner, & Schillewaert, 2013). The MPMI-s was included in two GESIS panel waves allowing us not only to investigate its factor structure and reliability, but also to derive general population norms. Furthermore, we present evidence for the scale’s validity by replicating well-established relations to personality measures (i.e., conscientiousness; Cuttler & Graf, 2007; Uttl & Kibreab, 2011) and by additionally showing that perceived PM abilities relate to vacation planning in everyday life.

Methods

Participants

Participants were part of the GESIS panel, a probability-based, longitudinal, mixed mode access panel for the academic community featuring a sample representative of the German adult population (GESIS, 2018). The MPMI-s was administered in waves bc with N = 4069 participants and bf with N = 3857 (i.e., approx. 95% of the original bc-wave participants). In wave bc (bf), the sample featured 52% (52%) females, a mean age of M = 47.11, SD = 14.32 (M = 47.44, SD = 14.21), and an age range from 19 to 71 years. A total of 46% (46%) of the participants had a high school degree eligible for attending a university (German Allgemeine-/ Fachhochschulreife), 53% (53%) a high school degree not eligible for university entrance (German Realschul-/ Hauptschulabschluss), and 1% (1%) did not hold a school degree. All participants answered the MPMI-s alongside several other questionnaires.

Measures

Metacognitive Prospective Memory Inventory short version (MPMI-s)

The English item translations of the MPMI-s are presented in Table 1. The original German items are provided as Additional file 1. The MPMI-s consists of three scales with eight items each. Items were selected from the MPMI long version without modification. The Prospective Memory Ability (PMA) scale measures how people experience their PM abilities (e.g., “I am able to remind myself of phone calls I need to make, such as calling a friend on his or her birthday.”). The items of this scale are formulated in a way so that it is obvious that they refer to memory abilities and not memory strategies (i.e., remind oneself rather than being reminded by another person or a device). Half of the items referred to prospective remembering, the other half to prospective forgetting. So, for half of the items, higher scores indicate better PM abilities, for the other half, higher scores indicate worse PM abilities. The latter items should usually be reverse-coded so that higher PMA scores always reflect better PM abilities. Because we intended to control for wording effects in the present study, however, we did not reverse-code these items for the reported analysis but only for the reference data. The PMSi scale assesses the frequency with which people use internal PM strategies to better remember their intentions (e.g., “In the morning, I go through the day’s tasks in my head so that I don’t forget to remember something.”). Higher PMSi scores always indicate more frequent strategy use. The PMSe scale measures how frequently people use external PM strategies such as memory aids or preparatory actions, to better remember their intentions (e.g., “I write myself a to-do list to remind me of things that I still need to accomplish”). Again, higher PMSe scores indicate more frequent strategy use.¹ For all items, the same 5-point Likert response scale is used with categories being labeled 1 = rarely, 2 = rather rarely, 3 = sometimes, 4 = rather often, and 5 = often.

Table 1 Standardized factor loadings (CFA) of all items

Big-Five Personality Inventory short version (BFI-2S)

The German version of the BFI-2S (Rammstedt, Danner, Soto, & John, 2018; Soto & John, 2017) was used to assess the personality domains Extraversion, Agreeableness, Conscientiousness, Negative Emotionality (Neuroticism), and Open-Mindedness (Openness) with six items each. In the present sample, this version showed a good reliability for all scales, 0.65 ≥ α ≥ 0.79.

Vacation planning

As part of one wave, the GESIS panel participants were asked several questions about their next vacation. Four questions investigated which aspects of their upcoming vacation trips participants had already planned at the time of assessment. These questions always started with “Which things did you already plan for your holiday trip?” and then referred to one particular planning aspect (i.e., arrival, accommodation, food, and activities during vacation). Participants answered them using a simple yes-no response format.

Data collection

Data collection was completed as part of three different data collection waves of the GESIS longitudinal panel (GESIS, 2018). That is, the MPMI-s was assessed in waves bc (June to August 2014) and bf (December 2014 to February 2015), the BFI-2S in wave ec (June to August 2017), and vacation planning in wave bc (June to August 2014).

Results

The GESIS panel data is publically available. All analysis codes are provided as Additional file 2.

Factorial structure

Based on previous research (Rummel, Kuhlmann, & Danner: A questionnaire for the asssessment of perceived prospective memory abilities and strategy use: The metacognitive prospective memory inventory (MPMI), in preparation), we hypothesized that items should form three different factors, that is, a PM ability (PMA), an internal PM strategy use (PMSi), and an external PM strategy use (PMSe) factor, that should be positively but moderately correlated with each other. A confirmatory factor analysis for a three-factor solution fitted the MPMI-s data from both data collection waves well, RMSEA ≤ 0.066, CFI ≥ 0.945, SRMR ≤ 0.033. In this model, we also controlled for wording effects (including acquiescence) by specifying a second factor with positive loadings for both reversed and non-reversed items (Billiet & McClendon, 2000; Maydeu-Olivares & Coffman, 2006). Item-factor loadings and separate fit-indices for each factor for both assessments are presented in Table 1. As expected, PMA correlated moderately with PMSi, r = 0.17, and PMSe, r = 0.21; the correlation between PMSi and PMSe was higher, r = 0.69. Taken together, the questionnaire factor structure from the original MPMI was replicated with the short version.

Reliability

To assess the MPMI-s’ reliability within both assessment waves, we calculated McDonald’s Omega and Cronbach’s Alpha for each scale. Results, which are displayed in Table 2, indicate good reliability for all scales and measurement points (ω ≥ 0.76, α ≥ 0.70). Retest correlations between the data obtained in the first (wave bc) and the second (wave bf) assessment further indicated that the manifest test scores were relatively stable across the 6-month assessment interval (r ≥ 0.64, see Table 2).

Table 2 Reliability estimates for all MPMI-s scales

Validity

The BFI-2S domains were modeled as latent variables using exploratory structural equation models (Asparouhov & Muthen, 2009) including a random intercept as wording/acquiescence factor (Aichholzer, 2014; Danner et al., 2015). We assessed latent correlations between all three MPMI-s scales and the five personality domains of the BFI-2S. As evident from Table 3, due to the large sample size, almost all correlations between the MPMI-s scales and personality domains reached conventional levels of significance. Therefore, we decided to only interpret correlations of at least r = 0.10, that is, only correlations of at least small sizes according to Cohen’s conventions (Cohen, 1968). Replicating prior research, the highest correlation observed was the moderate correlation between PMA and conscientiousness (r = 0.41). Both strategy scales (PMSi and PMSe) were also weakly positively correlated with conscientiousness (r = 0.21 and r = 0.18), suggesting that conscientious people invest more cognitive as well as preparation effort to avoid forgetting of intentions. We further found small positive correlations with agreeableness for both PMA (r = 0.15) and PMSe (r = 0.10). Unexpectedly, we observed similarly weakly positive correlations between negative emotionality and the two strategy scales (r = 0.15 and r = 0.21) as well as between all three PM scales and open-mindedness (0.11 ≤ r ≤ 0.15). Finally, as expected, PMA was positively related to vacation planning (r = 0.15).

Table 3 Latent correlations between MPMI-s scales and external criteria

Age and gender differences

As evident from Table 3, there was some evidence for small age-related differences on the (manifest) PMA scale and for gender differences on all three scales. Tests of measurement invariance (e.g., Chen, 2007) were conducted for these demographic variables (see Table 4 for the results).

Table 4 Measurement invariance tests

For the PMA scale, measurement invariance tests comparing age groups (up to 30 years = 0; 31–40 years = 1; 41–50 years = 2; 51–60 years = 3; 61 years and older = 4) were ambiguous. That is, some tests favored scalar invariance but others only metric or configural invariance, suggesting that age groups should be compared only within structural equation models accounting for age-group differences in item loadings and item intercepts. The tests further suggest scalar invariance between women and men (males = 1; females = 2), implying that manifest PMA scores can be compared between women and men.

For the PMSi and the PMSe scales, measurement invariance test results for age groups again did not consistently favor a certain level of invariance, suggesting that age groups should be compared within structural equation models accounting for age-group differences in item loadings and item intercepts, only. Test results further indicated metric invariance between women and men, suggesting that differences between women and men should be investigated with structural equation models accounting for gender differences in item intercepts.

Reference data

Reference data separated by age and gender for all three scales are provided as Additional file 3. As the GESIS panel provides a sample that is representative of the German population, the reference data can be used as normative data for the interpretation of individual test scores. Because all PM scales showed some indication of age-group related variance and the PM strategy scales also varied with gender, we recommend using age-specific reference data for the interpretation of all test scores and gender-specific reference data for the interpretation of PM strategy scores.

Discussion

PM abilities are crucial for mastering our daily work and life activities (Dismukes, 2012) and PM deficits accompany several clinical disorders (Raskin, 2018). Therefore, the brief and reliable assessment of individual differences in PM is of interest for both psychology researchers and (clinical) practitioners. Furthermore, the assessment of PM strategies is important both for a better interpretation of reported PM abilities and for identifying means for PM improvements. The presented short version of the MPMI allows for a relatively quick and reliable assessment of internal and external PM-strategy use in addition to the assessment of self-reported PM abilities. A further advantage of our PM ability scale is that it allows to control for wording effects and particularly acquiescence bias (Weijters et al., 2013), because it features negatively and positively formulated items (i.e., items that refer to prospective remembering and items that refer to prospective forgetting). In the present investigation, we confirmed the three-factor structure of the original longer version. Despite their brevity, all scales of the short version showed good reliabilities and tests scores were sufficiently stable over 6 months.

Furthermore, we found some evidence for the short version’s validity: replicating prior research with the PRMQ, PM abilities correlated moderately with conscientiousness (Cuttler & Graf, 2007; Uttl & Kibreab, 2011) and, to a weaker extent, with agreeableness. Uttl and Kibreab (2011) reported a similar correlational pattern, but for objective PM measures, only. The positive correlation of our PM ability scale with agreeableness may be due to the fact that at least some of the everyday PM scenarios used in the newly developed items have an obvious social component to it (e.g., remembering to call a friend on their birthday, see Table 1). We further observed a weakly positive correlation between PM abilities and open-mindedness, which seems reasonable, as open-mindedness has been shown to relate to cognitive abilities (Ziegler, Danay, Heene, Asendorpf, & Bühner, 2012). The small but reliable correlation of PM abilities with the vacation-planning index further corroborates the scale’s construct and criterion validity in everyday life.

To our knowledge, this is the first inventory to include both internal and external strategies of PM. The two strategy factors support the previously made theoretical distinction between internal and external PM strategies (Maylor, 1990; Penningroth & Scott, 2013). As expected, both strategy factors are positively related to reported PM abilities. Their negligible correlations with the vacation planning measure are not surprising given that the vacation planning measure assessed completed plans (i.e., having booked an accommodation) rather than the strategic process of planning. Interestingly, more conscientious people seem to not only perceive their PM abilities as better but also seem to engage more strategies to prevent prospective forgetting. Furthermore, it appears that people scoring higher on negative emotionality generally use more PM strategies (internal or external), maybe to buffer their fears of forgetting to execute important intentions. Again, we observed weakly positive correlations between the external strategy scale and agreeableness as well as between both strategy scales and open-mindedness. As suggested for the PM ability scale, the former might be due to the social aspects of many everyday intentions and the latter to the well-known link between open-mindedness and general cognitive abilities. In sum, we found good evidence for convergent validity of all three scales of the novel PM questionnaire (i.e., small to moderate correlations between conscientiousness and all PM scales). Somewhat unexpectedly, several (weak) correlations with other personality factors were also observed, however, suggesting that reported PM abilities and strategy use are not completely independent of people’s levels of open-mindedness, agreeableness, and emotionality. Intuitively, these relationships make sense within the nomological network of the Big-5 personality factors. However, as these relations were not observed in previous (less well powered) studies (Uttl & Kibreab, 2011), they require further replication.

Although PM performance in objective performance-based PM laboratory tasks has been shown to decline with (old) age (Kliegel, Jager, & Phillips, 2008), neither perceived PM abilities nor PM strategy-use varied much with age in the present study. Although this finding may in part be caused by the rather low upper age limit of 71 in the present sample, it replicates previous results of no age-related differences in reported PM abilities obtained with the PRMQ (Crawford et al., 2003). For one, this can be explained by older adults rating their memory in comparison to their peers on such questionnaires (Rabbitt, Maylor, Mcinnes, Bent, & Moore, 1995). Further notable, it has been shown that age-related PM deficits also do not manifest themselves in naturalistic PM tasks that sometimes even produce age-related PM benefits (Schnitzspahn, Ihle, Henry, Rendell, & Kliegel, 2011). The absence of age-related differences in perceived PM abilities thus may be reflective of a factual absence of age-related declines in real-life PM tasks.

In line with previous research using the PRMQ (Crawford et al., 2003; Uttl & Kibreab, 2011), we did not observe gender differences in perceived PM abilities. However, we observed substantial gender-related differences in internal and external strategy use, indicating that women tend to use certain strategies (e.g., mental intention rehearsal, writing to-do lists) more frequently than men. This is in line with previous research that also reported a more frequent use of memory aids in women than in men (Uttl & Kibreab, 2011). As the gender-related strategy-use differences seem to be reliable, it is advisable to take them into account when interpreting individual differences in this domain and chose reference data accordingly.

Conclusion

Taken together, the short version of the MPMI has been proven a reliable and valid instrument for the investigation of PM abilities and PM-strategy use. We hope that the MPMI-s and the reference data from the GESIS longitudinal panel will be useful for researchers who are interested in investigating individual differences in perceived PM abilities or strategy use as well as practitioners, for example in work, clinical, or rehabilitation contexts, who are interested in the comprehensive yet quick assessment of perceived PM-ability and strategy-use deficits.