The Challenged Sense of Belonging Scale (CSBS)—a validation study in English, Arabic, and Farsi/Dari among refugees and asylum seekers in Germany

Abstract

This study introduces and investigates the validity of a brief scale measuring a challenged sense of belonging. The sense of belonging as well as challenges to this sense are important, albeit neglected aspects of social integration and of significance to migration and refugee studies as well as to virtually all other social science contexts. Assessing a challenged or eroded sense of belonging provides important insights into how individuals relate to their environment and whether they feel socially connected or disconnected from it. The construct goes beyond national or cultural identity, instead emphasizing the dynamic processes of emotional attachment. Reviewing the substantial theoretical literature on belonging, we identify four of its key elements: connection, participation, identification, and congruence. Drawing on existing measurement instruments, we propose a brief Challenged Sense of Belonging Scale (CSBS) that addresses each of the four elements and investigate its validity in a unique, multi-lingual random sample of 3783 adult refugees in Germany from various national and cultural backgrounds. We provide evidence for the scale’s validity separately for three main survey languages (English, Arabic, Farsi/Dari) using confirmatory factor analysis, a test of measurement invariance, item test and rest correlations, and correlation analysis to explore convergent validity. Our findings suggest that the scale is a suitable instrument for the assessment of a challenged sense of belonging in a heterogeneous population of refugees.

Introduction

The concept of “belonging” is used across various disciplines, including sociology (Probyn, 1996; Yuval-Davis, 2006), psychology (Baumeister & Leary, 1995; Hagerty, Lynch-Sauer, Patusky, Bouwsema, & Collier, 1992), and geography (Mackenzie, 2004; Madsen & van Naerssen, 2003). In each field, a range of modes of belonging as well as different objects of attachment have been analyzed and discussed, ranging from occupational identification to family ties to regional attachments.

The construct of belonging is of particular importance with regard to migrants and refugees since they have to cope with forced re-orientations to new societal and cultural environments. For these populations, an eroded foundation of belonging constitutes a major obstacle to social integration and psychological well-being (Fortier, 2000; Nibbs, 2014).

While several fields of research rely on the concept of belonging, the emotionally charged and situationally contingent sense of belonging that stems from successfully placing oneself in relation to one’s environment (Pfaff-Czarnecka, 2013) remains conceptually and empirically underspecified. We argue that a sense of belonging is an essential hallmark of social integration and participation, while a challenged or eroded sense of belonging is a notable threat to both. An obstacle to the empirical assessment of this dimension of experience is the lack of adequate psychometric scales. The existing scales (Sense of Belonging Instrument (Hagerty & Patusky, 1995), Social Connectedness Scale (Lee & Robbins, 1995), Global Belongingness Scale (Malone, Pillow, & Osman, 2012)) are, in their original forms, primarily designed to capture elements of personality or individuals’ fundamental modes of relating rather than a context-sensitive feeling or experience. Scales that measure related concepts in migration research, such as the Multigroup Ethnic & National Identity Measure (Maehler, Zabal, & Hanke, 2019) focus on ethnicity or national identity or, like the Cross-Cultural Loss Scale (Wang, Wei, Zhao, Chuang, & Li, 2015), discuss belonging in the context of national privileges—very specific instances and types of belonging. Additionally, existing scales are too extensive for general multi-purpose large-scale surveys.

This study seeks to fill this research gap by introducing and assessing a brief scale designed to measure a challenged sense of belonging. We first report our review of the literature on belonging and propose four essential elements of belonging. We then introduce the corresponding 4-item “Challenged Sense of Belonging Scale” that we developed based on existing measurement instruments. Next, we provide information on how the scale was translated into Arabic and Farsi/Dari (from English or German) and implemented into the IAB-BAMF-SOEP Survey of Refugees living in Germany, a unique large-scale, randomly sampled survey of refugees and asylum seekers in Germany, covering a wide range of national, cultural, and linguistic backgrounds. Finally, we assess the functioning and validity of the scale separately for the three language groups (English, Arabic, Farsi/Dari). Construct validity is investigated, first, by means of psychometric tests assessing the internal structure of the scale and, second, by means of testing associations with exogenous variables that the literature suggests may be linked to belonging.

Forced migration as a challenge to belonging

In migration studies, the concept of belonging reflects the mental and emotional challenges migrants face in their struggle for social and cultural (re-)orientation. Migration often entails rapid alterations in objects of identification, social ties, and attachments, and with this, a re-alignment of belonging toward novel circumstances of life (Vertovec, 2010). The deterioration of bonds with a familiar environment and social support networks along with removal from the material and immaterial context of one’s identity pose a substantial psychological risk (Nibbs, 2014).

For displaced individuals and refugees in particular, the sudden and often violent disruption of ordinary life circumstances can lead to a disconnection from long-standing objects of attachment. For refugees, social liminality as well as legal restrictions often inhibit participation (see Yuval-Davis, 2006) and pose a threat to social integration into the host society (Ager & Strang, 2008, p. 173). In many cases, immigrants and refugees struggle with emotional transnationality. They are frequently caught between two or multiple realms of belonging, feel “caught between-worlds,” or report concurrent experiences of being “here and there” (Gidwani & Sivaramakrishnan, 2003; Levitt & Glick-Schiller, 2004). Re-aligning and re-building a sense of belonging over time and becoming an active part of a new social and cultural environment are considered important hallmarks of social integration (Ager & Strang, 2008).

While challenges to belonging are obvious in the context of flight and migration, the deterioration of belonging is not limited to these contexts. It can also occur, e.g., through marginalization, downward social mobility, and domestic relocation. Therefore, it is important to develop a broadly applicable measure of (a challenged) sense of belonging.

Theoretical background

The concept of the sense of belonging

Theoretical work in psychology (e.g., Anant, 1966; Hagerty et al., 1992) portrays belonging as the satisfaction of an individual’s need to be personally involved with their environment and to feel part of a larger social entity—socially embedded. Thus, belonging is often seen as part of a healthy state of being and defined in positive terms as a central feature of a person’s well-being (Hagerty et al., 1992). When belonging is challenged or eroded, it has the potential to become a severe psychological stressor, threatening mental health (Nibbs, 2014). Beyond these basic premises, psychological research has operationalized belonging mainly from a personality and individual differences research perspective, framing the tendency to experience a sense of belonging (or lack thereof) as a trait or fundamental mode of relating (e.g., Hagerty & Patusky, 1995; Lee & Robbins, 1995; Malone et al., 2012). This emphasis on individuals’ psychological makeup determines these constructs’ applicability and, in particular, limits their pertinence to enquiries which are interested in the sense of belonging as context-dependent and time-variant.

Sociological studies of belonging have focused on processes of exclusion and inclusion, often in the form of legal membership, and on a general feeling of “being at home.” Both accounts tend to neglect the emotional dimensions of the sense of belonging. On the one hand, studies focusing on legal membership as markers of belonging (Krzyzanowski & Wodak, 2007; Sicakkan & Lithman, 2005) as well as on identity politics and place-related aspects of exclusion and inclusion (Castles & Davidson, 2000; Yuval-Davis, Anthias, & Kofman, 2005) hinge heavily on pre-defined categories of belonging to account for personally and culturally mediated emotional processes, or focus only on certain aspects of belonging such as ethnic identity (e.g., Maehler et al., 2019). References to “feelings of being at home,” on the other hand, have been criticized for romanticizing belonging (Antonsich, 2010; Yuval-Davis, 2006) and for mainly focusing on potential objects of belonging, instead of defining what these feelings of being “at home” actually entail.

A comprehensive concept of belonging that accounts for some of these limitations is provided by Pfaff-Czarnecka (2013), who defines belonging as “an emotionally charged, ever-dynamic social location” (p. 13). Based on this understanding, we conceive of a sense of belonging as an emotionally charged experience that is dynamic, constantly in flux, and frequently challenged rather than being stable and fixed. We also regard the sense of belonging as context-sensitive and primarily related to a social location, rather than to specific objects or “modes” of belonging. Building on Pfaff-Czarnecka’s (2013) definition and drawing on a broad theoretical literature, we emphasize four central elements of belonging (or lack thereof): (1) connectedness, (2) participation, (3) identification, and (4) congruence.

The connectedness element represents the feeling of having a place within a social system, a national space, or geographical space (Antonsich, 2010; Dixon & Durrheim, 2004; Pfaff-Czarnecka, 2013). Lacking this sense of place and being disconnected from social networks promotes feelings of loneliness and insecurity (Newcomb, 1990; Pieloch, McCullough, & Marks, 2016). Previous studies highlight the importance of connections to, for example, occupational or educational institutions (Gibson, Bejínez, Hidalgo, & Rolón, 2004; van Houtte & van Maele, 2012). Any socially defined structure can satisfy or frustrate what Kohut (1984) labeled “connectedness”—a central element of belonging.

The second defining element of SoB goes beyond having a place within an environment or system and focuses on a reciprocal feeling of acceptance and being valued within a system, arising from socially participating in this system. According to Baumeister and Leary (1995), humans need “frequent, non-aversive interactions” (p. 497) as well as people they trust (Pearce, 2008) to feel like they participate in the lives of others, to feel indispensable. A lack of participation can result in feelings of being left out and unrecognized, frustrating what Hagerty (1992, p. 173) labeled the “valued involvement” element of belonging.

Third, the identificational element of belonging essentially captures the feeling of personal “fit” or “non-fit” within one’s environment (Hagerty et al., 1992; Pfaff-Czarnecka, 2013). Identificational feelings arise when individuals’ roles within their environment resonate with their identity or the location they envisage for themselves within that environment (Antonsich, 2010; Probyn, 1996; Yuval-Davis, 2006). An individual’s identification with their place in their environment arises out of emotional acceptance of their involvement and their social integration (Orton, 2012). A strong “fit” or identification typically leads to feelings of safety and stability, which are essential for a sense of belonging; lack of identification, in turn, erodes this sense.

The final element of belonging, congruence, acknowledges the potential existence of multiple, overlapping sources of belonging that can come into conflict with each other. For Savage, Bagnall, and Longhurst (2004), congruence refers to the concurrence of an individual’s current place of residence and their life story. In our understanding, congruence indicates a non-conflicting, cohesive relationship between coexisting sources of belonging. This means that congruence is potentially challenged in situations of re-orientation toward new social and cultural environments or when multiple environments are in conflict. These situations can result in unsettling feelings of being torn between different realms of belonging or a far-reaching loss of belonging.

Following these theoretical considerations on the sense of belonging and challenges to this sense, we developed a standardized measure that captures a challenged sense of belonging and tested it in a population of refugees and asylum seekers.

Method

Scale development

We searched the literature for scales measuring “sense of belonging,” “belonging,” and “belongingness” to construct a brief Challenged Sense of Belonging Scale (CSBS) based on items from one or multiple existing scales. Specifically, we looked for items that correspond to the four previously defined elements of belonging: participation, connectedness, identification, and congruence. Our motivation for selecting items from existing scales was to draw on established wordings to improve our chances of constructing a valid and reliable scale with apt, answerable items.

Although most existing scales that address belonging beyond a national or ethnic group or beyond general notions of “feeling at home” were created for very specific contexts—mostly in educational institutions (e.g., Hoffman, Richmond, Morrow, & Salomone, 2002)—we identified two scales from the psychological literature that include some items that are relevant to our approach: the Social Connectedness Scale (SCS) and the Sense of Belonging Instrument (SOBI). Importantly, as addressed above, these scales were designed to capture a sense of belonging as a psychological trait or mode of relating, while we conceive of a sense of belonging as a situationally contingent experience that is subject to changes and challenges across contexts and time. In other words, these scales and many of their items address related but different constructs (e.g., the SOBI-items: “I could disappear for days and it wouldn’t matter to my family”, “I would describe myself as a misfit in most social situations”, “I feel like a piece of a jig-saw puzzle that doesn’t fit into the puzzle”; e.g., the SCS-item: “Even among my friends, there is no sense of brother/sisterhood”). We therefore selected questions from these scales that work as situationally sensitive items and not primarily as dispositional measures, especially when combined with other items (see Table 1). Our brief scale has the advantage that it can be included in large, multi-purpose surveys such as the one used in this study.

Table 1 Wording of items in all tested languages

The Social Connectedness Scale (SCS) was designed to capture the “connectedness” element of Kohut’s (1984) “belongingness need” (need for a sense of “being a part of” in avoidance of loneliness and alienation) (see also Lee & Robbins, 1995). Connectedness “allows people to maintain feelings of being ‘being human among humans’ and to identify with those who may be perceived as different from themselves” (Lee & Robbins, 1995, p. 233, paraphrasing Kohut, 1984, p. 200). Although the authors justify their scale by referring to an increased lack of societal belongingness, also among immigrants, the SCS was created in the context of personality and individual differences psychology. We nonetheless found items that reflect our participation and connectedness elements in the SCS. We changed the wording of the connectedness item from “disconnected from the world around me” to “disconnected from those around me” to reduce phrasing redundancy (our Identification and Congruence items aim at “worlds”), as well as to emphasize challenged social connectedness in this item.

The Sense of Belonging Instrument (SOBI) was created by Hagerty and Patusky (1995) using Hagerty and colleagues (1992) concept of belonging (outlined above). One of the two central elements of belonging captured by this scale is “fit” or, as we call it here, identification. While the SOBI, like the SCS, is designed to capture a psychological trait, with other items capturing a sense of being a misfit or an outsider in social situations, one item expresses lacking identification in a way that is both psychologically and sociologically applicable. Finally, our concept of belonging includes congruence. We did not find an item that reflects this element in the existing literature. Therefore, referencing transnational theory’s notion of being “torn between worlds” (Levitt & Glick-Schiller, 2004), we formulated the congruence item of the scale ourselves. This item was devised with the migration context in mind, but, as we argue, it is suitable for capturing a multitude of incongruences.

As noted above, the objective of our scale is to identify instances of challenged or lacking belonging with respect to a particular context. All of our scale items are negatively worded, ascertaining the extent to which the sense of belonging is challenged rather than the extent to which it is intact. We agree with the authors of the SCS that it is the “frustrations of belongingness” (Lee & Robbins, 1995, p. 235) that are of interest to research, especially research on marginalized or disadvantaged groups and groups vulnerable to mental health problems. Additionally, we worded all scale items consistently in the negative based on the consideration that reverse wording, rather than lowering response bias, could lead to erroneous response patterns owing to inattention and confusion (Van Sonderen, Sanderman, & Coyne, 2013).

Unlike the SCS and the SOBI, which employed a quasi-forced choice scale with an even number of items (Guy & Norvell, 1977), our scale was tested using the traditional five-point Likert scale (1, “strongly agree”, to 5, “strongly disagree”, positive order). With a five-point scale, respondents were able to express neither feeling challenged in their sense of belonging nor distinctly not feeling challenged in their sense of belonging. To include a midpoint is standard in the Socio-economic Panel, the panel study in which the CSBS was tested (see “Sample” section below) (Jacobsen, Klikar, & Schupp, 2017; Richter, Metzing, Weinhardt, & Schupp, 2013). The CSBS is conceived of as additive–that is, a total score is calculated for use in all analyses.

Translation procedure

The original English language items were translated into German by some of the authors through a forward-translation-based process involving independent translations, comparisons, and discussion. All three authors involved are German native speakers and native or fluent in English and culturally competent regarding German and major English-speaking communities as well as competent regarding the content of the items and sociological testing practices (Muniz et al., 2013). The translation of the German or English questionnaire into Arabic and Farsi/Dari was provided by professional Arabic or Farsi/Dari mother tongue translators in the translation agency that works with Kantar, the company that conducts the IAB-BAMF-SOEP Survey of Refugees. The translation process within the translation agency used a TRAPD-like procedure (Harkness, 2003): First, two translators separately produced a translation. Next, a third translator discussed the translations with the others and resolved any discrepancies. Finally, trained interviewers working with Kantar, who were native speakers, tested the translated questionnaire in the field and reported back to the translation agency in case of problems (Britzke & Schupp, 2019, p. 49; Jacobsen, 2018).

Sample

We tested the scale using the IAB-BAMF-SOEP Survey of Refugees (Goebel et al., 2019; IAB-BAMF-SOEP Survey of Refugees 2018, 2020). The IAB-BAMF-SOEP Survey of Refugees is a random sample of asylum seekers and refugees who migrated to Germany between 2013 and 2016 (Kühne, Jacobsen, & Kroh, 2019). It is a joint project of the German Socio-economic Panel (SOEP), the Research Centre of the Federal Office for Migration and Refugees (BAMF-FZ), and the Institute for Employment Research (IAB). The sampling frame is the German Central Register of Foreign Nationals (AZR), which comprises all individuals living in Germany for longer than 3 months without holding a German passport (Babka von Gostomski & Pupeter, 2008). The CSBS was implemented in the third wave of the panel containing 4376 individuals living in 3061 households (for more information on sampling and response rates, see Kühne et al., 2019).

The mode of the interviews was computer-assisted personal interviewing (CAPI). Due to the implementation of audio-recordings of all questionnaires within the CAPI setting, the interview mode was easy to switch to (A)CASI ((audio) computer-assisted self-interview) if required. At the beginning of each interview, respondents chose their preferred interview language—English (N = 341, 8.7%), Arabic (N = 2957, 75%), Farsi/Dari (N = 485, 12.4%), Pashto (N = 30, 0.8%), Urdu (N = 54, 1.4%), or Kurmanji (N = 44, 1.1%). All questions were then presented in German and the chosen language side-by-side (Jacobsen, 2018).

For the present study, we analyzed data from the 3783 third wave respondents who received a questionnaire including the CSBS and who responded in English, Standard Arabic (hereafter just “Arabic”), or Farsi/Dari. These language groups had a sufficient number of participants for separate analyses.

Beyond language, our sample is heterogeneous in various respects (see Table 2 and Additional File 1 for more detail). Most Arabic-speaking respondents are from Syria (73%), and most Farsi/Dari-speaking respondents are from Afghanistan (79%). English is linked to a more diverse range of countries of origin. Many English-speaking respondents are from Eritrea (32%), but the majority is distributed in small proportions across a vast array of countries. Across all three languages, most respondents have a medium educational background (Levels 2–4 of ISCED11, United Nations Educational, Scientific and Cultural Organization (UNESCO, 2012). The Arabic-speaking group had the highest proportion of respondents who completed tertiary education, with around 20% having attained tertiary qualifications. In terms of religion, our sample is also diverse. Most respondents report an Islamic denomination. However, those who answered the CSBS in English most often reported a Christian denomination. The sample is rather young, with an average age of around 33 years across all languages. Most respondents have only been in Germany for around 3 years. See Additional File 1 for the correlations between each of these indicators and the CSBS.

Table 2 Key characteristics of the sample across languages

Statistical analysis

To investigate the validity of the scale in several languages and to account for potential measurement errors introduced by our translation procedures (Harkness et al., 2010), we carried out all analyses separately for the three language groups (English, Arabic, and Farsi/Dari). Throughout our analyses, we treated the CSBS Likert scale item responses as metric.¹ All analyses were carried out using Stata 14.S.E.

In a first step, we inspected response distributions for each scale item. Next, we examined item nonresponse in each language group. Questions in migrant samples that are perceived as sensitive in subject matter or difficult to comprehend have been found to exhibit high item nonresponse (Pennell, Cibelli Hibben, Lyberg, Mohler, & Worku, 2017). We also analyzed patterns of nonresponse by means of logistic regression to decide whether imputation or weighting procedures were necessary for subsequent analyses. The second and third steps aimed at establishing the psychometric properties of the scale as potential evidence for validity based on the scale’s internal structure (American Educational Research Association, American Psychological Association, National Council on Measurement in Education (AERA, APA, & NCME), 2014). To test whether the CSBS showed the expected single-factor structure, we employed a confirmatory factor analysis for all language groups separately. This analysis was based on a Pearson correlation matrix and used maximum likelihood estimation methods. We defined sufficient fit as follows: a CFI of at least 0.90 (Bentler, 1990), RMSEA below 0.08 (Kline, 2010), and SRMR below 0.08 (Hu & Bentler, 1999).

Based on this confirmatory factor analysis, we tested for measurement invariance to assess whether the model estimates the same latent construct in all language groups. If this is the case, the means of latent indicators can be reliably compared across groups. We employed a bottom-up estimation approach for measurement invariance testing (Saris, Pirralha, & Zavala-Rojas, 2018). This means that we started with the least restrictive model (configural invariance, loadings, and intercepts are not set to be equal across groups), applying the following rules to determine further steps: In the case that this model shows an adequate fit, loadings are restricted to be equal across groups in order to test metric invariance. If this model fits the data well and the deterioration of the CFI does not exceed 0.01, intercepts are set to be equal as well in order to determine full scalar invariance (see Chen, 2007). If this model shows appropriate fit as well and the deterioration of the CFI is 0.01 or smaller, full measurement invariance is given.

Fourth, through cross-item correlations, item-test correlations (ITC), item-rest correlations (IRC), and Cronbach’s alpha as well as McDonald’s omega (e.g., Tang, Cui, & Babenko, 2014), we examined the internal consistency of the CSBS. We consider ITC and IRC above 0.50 indicative of a moderate association, and ITC and IRC above 0.80 indicative of a strong association. We treat Cronbach’s alpha higher than 0.70 as indicative of adequate and above 0.80 of good internal reliability (Ferguson, 2009; Kline, 2000). McDonald’s omega higher than 0.8 is considered good (e.g., Catalán, 2019).

In the final step, we examined convergent validity by testing associations between the CSBS scores and other variables that have been linked to belonging in the literature. Understanding how a new scale is associated with theoretically related measures is an important aspect of exploring different sources of evidence for scale validity (American Educational Research Association, American Psychological Association, National Council on Measurement in Education (AERA, APA, & NCME), 2014). Ideally, convergent validity is assessed by comparing the scale in question to an established measure of the same construct. However, scales measuring precisely the same construct that we intend to measure do not exist. This means that we were only able to investigate how this new scale relates to measures included in the survey questionnaire that are theoretically or empirically linked to the construct of belonging in a broader sense (please refer to Additional File 3 for descriptives of all variables used for assessing convergent validity).

To do so, we computed partial correlations, separately for each language group, between the CSBS total score and a number of survey items and scales we hypothesized to be related to a challenged sense of belonging, in particular measures of mental health, well-being, and social embeddedness as indicated by the literature reviewed above. First, we assessed the association between the CSBS total score and the Mental Component Summary Scale of the Short-Form Health Survey (SF-12, Ware, Kosinski, & Keller, 1996). Scores range from 0 to 100 (from mental health languishing to flourishing) and are mean normalized to the 2004 wave of the SOEP (see Tibubos & Kröger, 2020). Second, we assessed associations between the CSBS total score and a standard one-item measure of life satisfaction (Schimmack, Schupp, & Wagner, 2008). To assess social embeddedness, we correlated the CSBS with (a) the number of individuals with whom respondents share private thoughts and feelings (between none and five individuals), (b) the frequency of contact to people from the country of origin who are not relatives (1, “never”; 6, “daily”), (c) the frequency of contact to Germans (1, “never”; 6, “daily”), (d) the frequency of contact with Germans in one's friend group (1, “never”; 6, “daily”), all treated metrically. We predict positive associations between CSBS scores and these measures.

We controlled for respondents’ duration of stay in Germany under the assumption that regaining or re-aligning one’s sense of belonging takes time, as does building a social network, making friends, and gaining a feeling of stability and life satisfaction—i.e., time is likely to mediate the relations between CSBS and the assessed variables. We do not define specific cut-offs for Pearson’s r, since magnitudes are highly context-specific (Schober, Boer, & Schwarte, 2018), but discuss results more comparatively and interpret the meaning of the identified associations.

Results

Descriptive analyses

All four variables follow a similar distribution in all three languages (see Table 3 for means and SD, and Figures 1, 2, and 3 in the “Appendix” section for histograms). What stands out is that the highest value of the scale, indicating no challenge to one’s sense of belonging, is the modal value for all four items for all languages. The other four responses, indicating various degrees of challenged belonging, display an almost normal distribution.

Table 3 Respondents, mean, and standard deviation, Cronbach’s alpha, item-test correlations, and item-rest correlations for each factor over languages

The proportions of missing items vary between 2.7 and 14.4% (see Table 6 in the “Appendix” section), depending on the survey language. Respondents who answered the scale in English skipped more questions than those who responded in Arabic or Farsi/Dari across all items. While 92.8% of the Arabic-speaking respondents and 89.1% of the Farsi-speaking respondents answered all four scale items, only 78.0% of the English-speaking respondents did. Among English-speaking respondents, average item nonresponse across all four items was 13.2%, compared to only 3.5% among Arabic-speaking and 5.8% among Farsi-speaking respondents.

There does not seem to be a strong systematic pattern as to who answered all CSBS questions and who did not (please refer to Additional File 2 for the results of a logistic regression analysis on response behavior). There is some indication that respondents with higher levels of education were more likely to answer all scale questions and that those who did not report their legal status were less likely to answer all scale questions. However, the findings are not consistent across languages and the confidence intervals are large. Thus, for further analysis, we treated the missing values as “missing completely at random” (Little & Rubin, 1987), refraining from imputation or weighting adjustments.

Evidence for construct validity

To assess construct validity, we first employed a confirmatory factor analysis for all language groups separately. As indicated by the factor loadings (see Table 4), all items map onto the same latent construct. Additionally, fit-indices (CFI, RMSEA, SRMR) suggest that the proposed structure fits the data well.

Table 4 Confirmatory factor analysis

The test for measurement invariance indicates full scalar invariance as all steps of the bottom-up procedure show adequate fit (configural: CFI = 0.98, SRMR = 0.03; metric: CFI = 0.98, SRMR = 0.06; scalar: CFI = 0.97, SRMR = 0.06). With full scalar invariance supported, we assume that the latent construct is the same across language groups.

Evidence for internal reliability

Reliability was measured using pairwise Pearson correlations, Cronbach’s alpha, and McDonald’s omega. As shown in Table 3, Cronbach’s alphas and McDonald’s omegas for the different sub-samples are all adequate to good. Additionally, the item-test correlations (ITC) and the item-rest correlations (IRC) are all of a sufficient magnitude, indicating that the items are highly correlated with the overall score. Pairwise correlations shown in Table 7 in the “Appendix” section reveal that some items are only moderately correlated with each other. Nevertheless, all effects are strongly significant, and no correlation is smaller than 0.42.

Evidence for convergent validity

We calculated partial correlations controlling for duration of stay in Germany separately for all three language groups in order to assess the association between the CSBS score and measures linked to belonging as evidence for convergent validity (see Table 5). Across all languages, CSBS scores are positively associated with greater mental health, that is: less challenged sense of belonging is associated with greater mental health. Out of all associations tested, this is the strongest one, with lower and upper confidence interval bounds for Pearson’s r values of a minimum of 0.32 and a maximum of 0.55 across languages. The well-being measure, life satisfaction, is positively, albeit less strongly, associated with the CSBS total score. The Pearson’s r confidence intervals show that there is an association across languages: the weakest among Farsi-speaking respondents (0.11–0.29) and the highest among Arabic-speaking respondents (0.25–0.32). Regarding the social embeddedness variables, we find that the number of people with whom respondents can share private thoughts and feelings is modestly positively associated with CSBS among Arabic-speaking respondents (0.03–0.11) and modestly or not at all associated in Farsi-speaking respondents (− 0.05 to 0.14). Frequency of contact to Germans in general and to Germans in the respondents’ own friend groups, are both positively associated with the CSBS score with confidence interval bounds for Pearson’s r between 0.01 and 0.34 across languages. A higher frequency of contact to people from the country of origin who are not relatives is modestly or not at all associated with higher CSBS scores in Farsi-speaking respondents (− 0.08 to 0.10) and modestly negatively associated with CSBS scores among English-speaking respondents (− 0.20 to 0.03), indicating that a higher frequency of contact could actually be linked to a more challenged sense of belonging.

Table 5 Partial correlation coefficients—convergent validity

Discussion

The present study introduces and investigates a Brief Challenged Sense of Belonging Scale (CSBS) using a unique large-scale sample of 3783 asylum seekers and refugees in Germany. The results provide support for the CSBS as an applicable measure of situationally contingent challenged sense of belonging in the context of forced migration across languages for English, Arabic, and Farsi/Dari. After assessing the functionality and validity of the CSBS based on item response and nonresponse, descriptive statistics, as well as the internal structure of the scale (factor analyses, reliability measures), and its relationship to external variables (convergent validity), we find substantial evidence in favor of the validity of the CSBS across English, Arabic, and Farsi/Dari.

An inspection of the response distributions for the individual items of the CSBS revealed that the absence of a challenged sense of belonging is the modal value for all four items for all languages, while the various degrees of challenged sense of belonging follow an almost normal distribution. The dominance of responses indicating no challenged sense of belonging is somewhat surprising in light of the various potential threats to sense of belonging refugees are likely to encounter. This points to the possibility of enduring sources of belonging throughout flight and resettlement for a substantial proportion of this population and calls for further investigation. The relatively low missing values frequencies suggest that the scale is likely comprehensible overall and does not seem to pose a major problem regarding sensitivity in subject matter. Because the missing value frequencies for both Arabic and Farsi/Dari respondents are uniformly low, we attribute the higher missing values frequencies across items in English to the fact that English is likely not the mother tongue language for most respondents. The comprehension of this scale might be reduced for non-native speakers. However, the structural properties of the scale in English still indicate good comprehension of CSBS questions.

The confirmatory factor analysis suggests that all four items map onto a single latent construct in each language, as intended. We also found that the CSBS exhibits adequate to good reliability across language groups. The internal consistencies are acceptable to good and of very similar magnitude across all groups. Additionally, the item-test correlations and item-rest correlations showed that the individual items are highly associated with the overall scale score across groups.

Our exploration of the associations between belonging and concepts we predicted to be linked to belonging revealed significant associations between CSBS scores and several indicators of well-being and social embeddedness. The negative relationship between a challenged sense of belonging and overall mental health and life satisfaction across language groups is in line with the emphasis on the importance of a sense of belonging for well-being throughout the belonging literature (e.g., Hagerty et al., 1992; Nibbs, 2014). Moreover, the negative associations between a challenged sense of belonging and indicators of general social embeddedness and social embeddedness in Germany in particular (having people with whom one shares feelings, as well as contact with Germans and having Germans in one’s group of friends) are as expected. While some of these associations were of a small magnitude for some language groups, the results align with the emphasis on social embeddedness and connectedness throughout the belonging literature (see Baumeister & Leary, 1995). Interestingly, a challenged sense of belonging is not related to the frequency of contact with individuals from the country of origin, suggesting that contact with people from the host society may be more important for fending off challenges to one’s sense of belonging. The findings are robust with respect to the time since respondents arrived in Germany. Overall, these findings provide evidence for the validity of the CSBS based on associations with external variables (AERA, APA, & NCME, 2014). It should be noted that differences regarding significant effects in these associations between language groups could be due to sample size differences.

The results of the present study must be viewed in light of some limitations. First, while the IAB-BAMF-SOEP Survey of Refugees comes with the advantage of being a large, high-quality dataset, we did not have any broader influence on the content of the questionnaire. Our assessment of convergent validity is especially affected by this circumstance since we were only able to explore the relationship between a challenged belonging and the variables included in this survey. However, it is important to recognize that there are no perfect measures of convergent validity for a new scale measuring a construct that no other established scales have addressed. Qualitative evidence of validity would be of particular interest in this case (AERA, APA, & NCME, 2014; Hawkins, Elsworth, & Osborne, 2019). Second, although there are advantages to our sample being multi-lingual, it also poses some challenges: first, the employed languages are not necessarily respondents’ mother tongue languages. This is especially true for English and, to some extent, for Arabic, seeing as the scale items were presented in Standard Arabic and not in regional dialects, which are more commonly used among Arabic speakers. Third, although the sample is unique in that it consists of people from all over the world, it is selective at the same time in that it focuses on refugees and asylum seekers. This calls external validity into question and, therefore, the scale should additionally be assessed in other samples. Fourth, although the tests regarding item nonresponse did not reveal a systematic pattern, we still found some noise in the data. Thus, in future applications of this scale, item nonresponse should be checked for systematic patterns—especially if the survey language is not the mother tongue of respondents. Fifth, since we argue that belonging is fundamentally situational and dynamic, a longitudinal design would help to indicate whether the scale is genuinely situationally sensitive. Finally, we acknowledge that, in general, validation must be thought of as an ongoing process (Flake, Pek, & Hehman, 2017) and that future assessments of the CSBS are needed to accumulate evidence in favor of its validity.

Conclusion

In conclusion, our analyses and results provide strong initial evidence in favor of the basic functioning, internal reliability, and validity of the newly developed CSBS based on its statistical properties in a large and diverse sample. Despite some limitations, the scale represents a potentially valuable measure of interest to researchers in a variety of fields, including sociology, psychology, and health research. In its design and theoretical background, this study fills two gaps in the existing literature on belonging: First, it provides a brief, but rich and situationally sensitive, measure for a challenged sense of belonging that can readily be included in large-scale surveys. Second, it broadens our understanding of the sense of belonging construct in a forced migration context, offering an alternative to previous constructs like belongingness to nations. This broader scope is also what makes the CSBS potentially applicable beyond the migration context: the sense of belonging can come under threat in a range of adverse, destabilizing circumstances, for example, in unemployment or other instances of marginalization. Challenged belonging, as defined and operationalized here, is a valuable construct for a large variety of populations that should be subject to further testing.

Appendix

Fig. 1

Histograms showing the distribution of the items of the CSBS, English, own calculations

Fig. 2

Histograms showing the distribution of the items of the CSBS, Arabic, own calculations

Fig. 3

Histograms showing the distribution of the items of the CSBS, Farsi/Dari, own calculations

Table 6 Item nonresponse for factors and the whole construct

Table 7 Pairwise correlations between items