- Validation of Measurement Instruments
- Open Access
External validity of the benign and malicious envy scale with Japanese undergraduate student and non-student samples
Measurement Instruments for the Social Sciences volume 4, Article number: 5 (2022)
This study examined the validity of the Japanese version of the Benign and Malicious Envy Scale (BeMaS) with Japanese undergraduate student and non-student samples. Previous studies have identified two types of envy, benign and malicious, that motivate different types of behavior. However, the validity of the BeMaS, developed to measure two types of dispositional envy, has not been adequately confirmed in East Asian countries. Furthermore, it is unclear whether the two-factor structure of BeMaS is identical across various samples. Thus, in this study, we specified the Japanese words describing envy, namely, urayamashii or netamashii, suitable for the Japanese BeMaS. Additionally, we tested the validity of the scale’s two-factor model across undergraduate students and non-student samples. The questionnaire survey results showed that the validity of BeMaS’s two-factor structural model was confirmed in both samples and the goodness of fit was better for urayamashii than for netamashii. Moreover, measurement invariance across the two samples was established in configural and metric models.
Envy is a powerful emotion produced by an awareness of another person’s enjoyment of a desired possession or characteristic (Parrott & Smith, 1993; Smith & Kim, 2007). Recent studies (van de Ven, Zeelenberg, & Pieters, 2009) have identified two types of envy: benign and malicious. Both types are negative emotions that people experience when they recognize others’ superiority. Nevertheless, researchers have found that benign and malicious envy predict different motivations, cognitions, and behaviors (Crusius & Lange, 2014)—benign envy motivates constructive behavior, whereas malicious envy evokes hostile behavior. Lange and Crusius (2015) developed the Benign and Malicious Envy Scale (BeMaS) to measure these two types. They conducted surveys primarily among American and Indian participants and used the resultant data to confirm BeMaS’s two-factor structural model. However, there is inadequate evidence concerning the validity of BeMaS in other cultures, especially in East Asia.
Cultural differences in envy
Although envy is considered a universal emotion (Smith & Kim, 2007), some researchers have pointed out the differences between collectivistic and individualistic cultures. For example, Foster (1972) reported that members of a collectivistic society were likely to feel malicious envy and fear being envied by others. East Asian countries are predominantly collectivistic in their cultural orientation (Markus & Kitayama, 1991); thus, East Asians might primarily experience malicious envy. Similarly, Rodriguez-Mosquera et al. (2010) demonstrated that when individuals in a collectivistic culture (e.g., Japan) become the target of envy, they respond with their positive (e.g., increased self-confidence) and negative (e.g., fear of ill will) emotions simultaneously than those in an individualistic culture (e.g., the United States). This evidence suggests that a single-factor model (i.e., malicious envy) may better fit envy than a two-factor model in East Asian countries, such as Japan. Although some previous studies tested the validity of the BeMaS in Eastern Asia (De Zoysa, Kumar, & Amarasuriya, 2021 in Sri Lanka; Sawada & Fujii, 2016 in Japan; Xiang, Chao, & Ye, 2018 in China), little is known about the comparison of the factorial structure between one-factor and two-factor models (except for Sawada & Fujii, 2016). Therefore, in this study, we examined the factorial structure of the BeMaS in Japanese samples.
The difficulties in testing the external validity of BeMaS and need to validation
Two points should be considered in testing the external validity of the BeMaS. The first issue is the translation of envy. Envy has only one expression in the English language: “envy,” unlike other languages. For example, the word benign envy is expressed as benijden, and the word malicious envy is translated as afgunst in the Dutch language. The former and latter are fit for beneiden and missgönnen in German. Similarly, in some languages of East Asian countries, envy is not always described using a single term. For example, the Chinese use the terms jidu and xianmuin to express envy. In the Japanese language, both netamashii and urayamashii are used to put envy into words, but netamashii includes more “malicious” connotations, while urayamashii encompasses both “benign” and “malicious.” Such diversity of the translated words in envy prevents replication in testing the validity of BeMaS in other countries, especially in Japan. Although in many cases, envy is likely to be translated into netamashii, Japanese may not classify natamashii and urayamashii into conceptually different categories. Therefore, the translated terms that can fit BeMaS should be confirmed based on a statistical criterion.
Furthermore, sample bias is problematic. Most studies using the BeMaS employed undergraduates (De Zoysa, et al., 2021; Sawada & Fujii, 2016; Xiang, Chao, & Ye, 2018) or non-student populations (Bolló, Háger, Galvan & Orosz, 2020; Lange and Crusius, 2015; Navarro-Carrillo, Beltrán-Morillas, Valor-Segura & Expósito, 2018) as samples. This difference in the samples may generate different responses to BeMaS. For example, previous studies have found malicious envy predominant in workplace settings (Schaubroeck & Lam, 2004) and those with high social status or economic wealth (Hill, DelPriore, & Vaughan, 2011; Inoue, Hoogland, Takehashi, & Murata, 2015). Hence, to test the validity of the BeMaS, the measurement invariance of the BeMaS should be examined using both undergraduate student and non-student samples.
To examine measurement invariance, we conducted multi-group confirmatory factor analysis (MGCFA;Vandenberg & Lance, 2000). The MGCFA can control the measurement invariance generated by different response periods in the scale. We established the four models in MGCFA to reveal measurement invariance between the two samples (Vandenberg & Lance, 2000). The first configural model imposed the constraint that equivalent parameters exist across groups. The second metric model further involved the constraint of equivalent factor loadings for both groups. The third model is the scalar model, which adds constraints on the invariance of the variable item intercepts and factor loadings to the second metric model. The fourth model refers to the residual model, which is the one wherein the factor loadings, variable intercepts, and error variances are set to be equal across groups. These models should be tested so that the next model cannot be tested until the previous model has been validated because the model is characterized as nested (Vandenberg & Lance, 2000).
We conducted questionnaire surveys to:
(1) Compare the fitness of the models using the urayamashii version and a netamashii version of the Japanese version of the BeMaS
(2) Test the measurement invariance of the BeMaS between undergraduate student and non-student sample.
Ethical approval for this study was obtained from Toyo University.
This study consisted of 426 participants (192 women, 233 men, 1 unknown). The sample was split into two groups: 194 Japanese undergraduate students (73 women, 120 men, 1 unknown) with a mean age of 19.88 years (SD = 4.68), and 232 Japanese non-students (119 women, 113 men) with a mean age of 39.63 years (SD = 10.76), recruited from Marketing Applications, Inc. By participating in the survey, undergraduate participants received course credit, while the non-student participants received a reward as set by the research company.
Procedure and Measures
Participants responded to each of the 10 items of the BeMaS on a 6-point Likert scale ranging from 1 (strongly disagree) to 6 (strongly agree). Undergraduate students completed both the urayamashii and netamashii versions of the Japanese BeMaS; the order of presentation of these two scales was counterbalanced (urayamashii version: benign: α = .76, M = 3.66, SD = 0.93, malicious: α = .82, M = 2.52, SD = 1.09; netamashii version: benign: α = .72, M = 3.52, SD = 0.92, malicious: α = .81, M = 2.57, SD = 1.09). They completed the measures in the following order: one version of the BeMaS (urayamashii or netamashii), the Social Comparison Scale (Gibbons & Buunk, 1999; α = .63, M = 3.44, SD = 0.66), and the other version of the BeMaS (netamashii or urayamashii). In the non-student sample, respondents answered either the urayamashii or netamashii version (urayamashii version: benign: α = .86, M = 2.67, SD = 1.02, malicious: α = .90, M = 3.17, SD = 0.96; netamashii version: benign: α = .83, M = 2.89, SD = 0.97, malicious: α = .82, M = 3.27, SD = 0.91). While undergraduate students answered with paper and pencil, non-students responded to an online survey.
Following Lange and Crusius (2015), these surveys included the Social Comparison Scale (Gibbons & Buunk, 1999) to confirm the relationship between envy assessed by the BeMaS and the personal trait of social comparison. Moreover, since the term urayamashii is often used in the same sense as admiration without social comparison (van de Ven, Zeelenberg & Pieters, 2011), we also aimed to confirm whether urayamashii was an appropriate measure of envy with social comparison.
Back-translation: The back-translation of the BeMaS into Japanese was completed by a bilingual person. If the back-translation revealed differences between the original and Japanese versions of the BeMaS, we discussed them with the translator. Subsequently, we revised the Japanese version until they were fit for the original version.
First, confirmatory factor analyses (CFAs) were conducted for each version of the BeMaS, urayamashii, and netamashii. Subsequently, a multi-group confirmatory factor analysis (MGCFA) was performed (Vandenberg & Lance, 2000). Based on the argument of Chen (2007), we checked the CFI, RMSEA, and SRMR among these models to evaluate the validity of each model. Chen (2007) argued that, in the cases of small sample sizes (N < 300) and unequal sample sizes between groups, invariance will not be confirmed if the variation from one model to the next model is within the proposed range. The range is CFI >= -.005, and RMSEA ≧.010 in the configural, metric, and scalar models, and SRMR ≧.025 in the configural model, while it is ≧ .005 in the metric and scalar models. We explored measurement invariance based on Chen’s criteria. We used SPSS Amos 27 for the following analysis.
CFAs in single groups
Confirmatory factor analysis was conducted separately for both versions of the BeMaS (Table 1). The results revealed that a two-factor model was a better fit for the data in both versions (undergraduates–urayamashii: χ2 = 59.533, CFI =.956, RMSEA = .062, SRMR = .064, AIC = 121.533; undergraduates–netamashii: χ2 = 99.868, CFI = .884, RMSEA = .100, SRMR = .091, AIC = 161.868; nonstudents–urayamashii: χ2 = 79.150, CFI = .929, RMSEA = .108, SRMR = .097, AIC = 141.150; nonstudents–netamashii: χ2 = 85.854, CFI = .894, RMSEA = .115, SRMR = .066, AIC = 147.854) than the one-factor model (undergraduates–urayamashii: χ2 = 252.269, CFI = .624, RMSEA = .179, SRMR = .167, AIC = 312.269; undergraduates–netamashii: χ2 = 264.685, CFI = .595, RMSEA = .184, SRMR = .157, AIC = 324.685; nonstudents–urayamashii: χ2 = 275.931, CFI = .623, RMSEA = .246, SRMR = .191, AIC = 335.931; nonstudents–netamashii: χ2 = 146.736, CFI = .772, RMSEA = .166, SRMR = .091, AIC = 206.736; Table 2)Footnote 1. Moreover, the results of the goodness-of-fit analysis indicated that the two-factor model of the urayamashii version was a better fit for the data, which is consistent with the results of Lange and Crusius (2015). Thus, compared to netamashii, urayamashii was a better descriptor of envy in the Japanese translation of the BeMaS. As shown in Table 1, across undergraduate and non-student samples, a social comparison was positively related to benign and malicious envy with both the urayamashii and netamashii versions (undergraduates–urayamashii: benign: r = .36, p < .01, malicious: r = .33, p < .01; undergraduates–netamashii: benign: r = .40, p < .01, malicious: r = .37, p < .01; nonstudents–urayamashii: benign: r = .49, p < .01, malicious: r = .50, p < .01; nonstudents–netamashii: benign: r = .59, p < .01, malicious: r = .50, p < .01). Thus, the convergent validity of the Japanese version of the BeMaS was confirmed.
Confirmatory factor analysis confirmed the goodness of fit of the two-factor model of the urayamashii version. Table 3 shows the fitness of the four models of the MGCFA to examine the measurement invariance in the urayamashii two-factor model. Configural invariance (Model 1) fitted the data well (χ2 = 138.782, CFI = .942, RMSEA = .058, SRMR = .086, AIC = 262.782). The results of the metric invariance (Model 2) showed that the model fitted the data (χ2 = 155.667, CFI = .935, RMSEA = .058, SRMR = .096, AIC = 263.667). The comparison of CFI, RMSEA, and SRMR between Model 1 and Model 2 revealed that the difference in CFI of -.007, could not meet the criteria of Chen (2007), while RMSE and SRMR were within the range, confirming the metric invariance. However, scalar invariance in Model 3 (χ2 = 202.135, CFI = .905, RMSEA = .066, SRMR = .095, AIC = 290.135) did not support our prediction because CFI, RMSEA, and SRMR cannot be within the proposed range. Further, no invariance was observed in partial scalar invariance. Thus, only configural invariance and metric invariance of BeMaS were confirmed in this study.
In this study, we conducted surveys among two samples—undergraduate students and non-students, to test the external validity of the Japanese BeMaS. The goodness of fit of the structural models was compared between the two versions of the BeMaS, entailing two different translations of envy, namely urayamashii and netamashii. The results revealed that in both versions, the two-factor structural model of the BeMaS showed a better fit than the one-factor structural model. These results are consistent with previous studies (Lange & Crusius, 2015; Sawada & Fujii, 2016 in Japanese). Furthermore, the better goodness of fit of the two-factor model in the urayamashii version than in the netamashii version suggested that translation of the word envy from English to Japanese corresponded well in urayamashii. This finding would contribute to the confirmation of the external validity of BeMaS and the enhancement of cross-cultural comparisons.
Measurement invariance was established in the configural and metric models in the MGCFA, indicating that the two groups had the same structure of BeMaS and responded similarly to its items. However, the result of the scalar invariance model not being established indicates different responses to the items by the two groups. This result may be due to two reasons. First, students and non-students may face different circumstances in which different types of envy could be made salient. Previous studies have shown that malicious envy tends to be evoked in their work environment (Schaubroeck & Lam, 2004), and those with high social status and plentiful economic resources are likely to be the target of envy (Hill et al., 2011; Inoue et al., 2015). A non-student population tends to elicit more malicious envy than a student population. Conversely, students may be in an environment where benign envy is aroused more frequently, such as in situations of high perceived control, which are known to arouse benign envy (Inoue & Murata, 2014; van de Ven et al., 2009). These differences in the environment each population faces may differ in responses between a student and a non-student sample. Another reason is the different response procedures in a student and a non-student sample. For example, while the students completed the questionnaire in a classroom simultaneously with their peers, the non-students answered it individually on the web. Others in a classroom may suggest perceived social comparison and social desirability. In this data, the MGCFA incorporating socially desirable responding (SDR) modeling (e.g., Ziegler & Buehner, 2009) revealed that the measurement invariance was established only in the configural modelFootnote 2. Thus, consideration of SDR did not enhance the establishment of the measurement invariance. However, the fit of the configural model was improved by considering the SDR (CFI = .954, RMSEA = .059, and SRMR = .069). These results suggest that the effect of social desirability on the two-factor model of BeMaS should be considered, and are consistent with the finding that benign envy is more socially desirable than malicious envy (Cohen-Charash & Larson, 2017).
Another limitation of this study is the inequality of the gender ratio between undergraduate students and a non-student sample. The measurement invariance of gender in BeMaS showed little effect of gender ratio because of confirming scalar invariance of genderFootnote 3. However, the study’s results are not consistent with those of a previous study that found gender differences in malicious envy in Sri Lanka (De Zoysa et al., 2021). Therefore, we should revise the response and data collection procedure and conduct a replication study to confirm the robustness of our findings in the future.
The findings in this study reflecting that the responses to BeMaS were different between a student and a non-student sample imply that the quality of samples should be considered in the data collection of BeMaS, especially in a cross-cultural study. Kwiatkowska et al. (2020) reported the measurement invariance of the BeMaS in the United States, Germany, Russia, and Poland; however, only Germany’s data was obtained from a student sample, while the others were non-student samples. Similarly, Dinić and Branković (2021, study1) employed the Serbian and US sample to test the cross-cultural invariance of BeMaS, but the Serbian sample was students while the US sample was non-students. These differences in samples could confound the effect of the sample and culture on the response of BeMaS. The findings of this study can contribute to the development of a cross-cultural comparison of BeMaS.
Availability of data and materials
The data that support the findings of this study are available from the corresponding author upon reasonable request.
The interaction effects of the order of presentation of the two versions were not significant for the fit of the two-factor model (netamashii version: netamashii in advance: χ2(34) = 71.14, CFI = .81, AIC = 133.14, RMSEA = .11; urayamashii in advance: χ2(34) = 74.50, CFI = .91, AIC = 136.47, RMSEA = .11. urayamashii version: netamashii in advance: χ2(34) = 50.86, CFI = .95, AIC = 112.86, RMSEA = .07; urayamashii in advance: χ2(34) = 51.58, CFI = .93, AIC = 113.58, RMSEA = .07).
The MGCFA considering influence of social desirability resulted in CFI=.954, RMSEA=.059, and SRMR=.069 for the configural model (Model 1); CFI=.935, RMSEA=.065, and SRMR=.089 for the metric model (Model 2); and CFI=.913, RMSEA=.070, and SRMR=.080 for the scalar model (Model 3); CFI=.791, RMSEA=.102, and SRMR=.103 for the residual model (Model 4). Comparison of CFI, RMSEA, and SRMR between Model 1 and Model 2 showed that RMSEA and SRMR met Chen's (2007) criteria, but the difference in CFI (-.019) exceeded Chen's (2007) criteria. Therefore, we determined that only the congifural model was established.
The measurement invariance of BeMaS across gender was found to be CFI=.941, RMSEA=.057, and SRMR=.081 for the configural model (Model 1); CFI=.942, RMSEA=.054, and SRMR=.081 for the metric model (Model 2); and CFI=.940, RMSEA=.052, and SRMR=.081 for the scalar model (Model 3). Therefore, it can be concluded that the scalar model was satisfied according to the criteria of Chen (2007).However, in the residual model (Model 4), CFI=.929, RMSEA=.053, and SRMR=.081; therefore, the residual invariance was not supported.
the Benign and Malicious Envy Scale
Confirmatory factor analyses
Multi-group confirmatory factor analysis
Socially desirable responding
Bolló, H., Háger, D. R., Galvan, M., & Orosz, G. (2020). The role of subjective and objective social status in the generation of envy. Frontiers in psychology, 11, 3402.
Chen, F. F. (2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural equation modeling: a multidisciplinary journal, 14(3), 464-504.
Cohen-Charash, Y., & Larson, E. C. (2017). An emotion divided: Studying envy is better than studying “benign” and “malicious” envy. Current Directions in Psychological Science, 26, 174–183.
Crusius, J., & Lange, J. (2014). What catches the envious eye? Attentional biases within malicious and benign envy. Journal of Experimental Social Psychology, 55, 1–11.
de Zoysa, P., Kumar, S., & Amarasuriya, S. D. (2021). An Assessment of Envy in Undergraduates of a State University in Sri Lanka. Psychological Studies, 66, 191–199.
Dinić, B. M., & Branković, M. (2021). Cross-cultural comparison of the Benign and Malicious Envy Scale (BeMaS) across Serbian and US samples and further validation. European Journal of Psychological Assessment, 38, 49–60.
Foster, G. M., Apthorpe, R. J., Bernard, H. R., Bock, B., Brogger, J., Brown, J. K., … Whiting, B. B. (1972). The anatomy of envy: A Study in Symbolic Behavior [and Comments and Reply]. Current Anthropology, 13, 165–202.
Gibbons, F. X., & Buunk, B. P. (1999). Individual differences in social comparison: Development of a scale of social comparison orientation. Journal of Personality and Social Psychology, 76, 129–142.
Hill, S. E., DelPriore, D. J., & Vaughan, P. W. (2011). The cognitive consequences of envy: Attention, memory, and self-regulatory depletion. Journal of Personality and Social Psychology, 101, 653–666.
Inoue, Y., Hoogland, C. E., Takehashi, H., & Murata, K. (2015). Effects of resource divisibility and expectations of sharing on envy. Motivation and Emotion, 39, 961–972.
Inoue, Y., & Murata, K. (2014). The effect of attainability on envy. Shinrigaku kenkyu: The Japanese journal of psychology, 85, 1–8 (in Japanese).
Kwiatkowska, M. M., Rogoza, R., & Volkodav, T. (2020). Psychometric properties of the Benign and Malicious Envy Scale: Assessment of structure, reliability, and measurement invariance across the United States, Germany, Russia, and Poland. Current Psychology, 1–11.
Lange, J., & Crusius, J. (2015). Dispositional envy revisited: Unraveling the motivational dynamics of benign and malicious envy. Personality and Social Psychology Bulletin, 41, 284–294.
Markus, H. R., & Kitayama, S. (1991). Culture and the self: Implications for cognition, emotion, and motivation. Psychological Review, 98, 224–253.
Navarro-Carrillo, G., Beltrán-Morillas, A. M., Valor-Segura, I., & Expósito, F. (2018). The pernicious effects of malicious versus benign envy: perceived injustice, emotional hostility and counterproductive behaviors in the workplace. The Spanish journal of psychology, 21, 1–9.
Parrott, W. G., & Smith, R. H. (1993). Distinguishing the experiences of envy and jealousy. Journal of Personality and Social Psychology, 64, 906–920.
Rodriguez Mosquera, P. M., Parrott, W. G., & Hurtado de Mendoza, A. (2010). I fear your envy, I rejoice in your coveting: On the ambivalent experience of being envied by others. Journal of Personality and Social Psychology, 99, 842–854.
Sawada, M., & Fujii, T. (2016). Do envious people show better performance? Focusing on the function of benign envy as personality trait. Shinrigaku kenkyu: The Japanese journal of psychology, 87, 198–204 (in Japanese).
Schaubroeck, J., & Lam, S. S. K. (2004). Comparing lots before and after: Promotion rejectees’ invidious reactions to promotees. Organizational Behavior and Human Decision Processes, 94, 33–47.
Smith, R. H., & Kim, S. H. (2007). Comprehending envy. Psychological Bulletin, 133, 46–64.
van de Ven, N., Zeelenberg, M., & Pieters, R. (2009). Leveling up and down: The experiences of benign and malicious envy. Emotion, 9, 419–429.
van de Ven, N., Zeelenberg, M., & Pieters, R. (2011). Why envy outperforms admiration. Personality and Social Psychology Bulletin, 37, 784–795.
Vandenberg, R. J., & Lance, C. E. (2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3, 4–70.
Xiang, Y., Chao, X., & Ye, Y. (2018). Effect of gratitude on benign and malicious envy: The mediating role of social support. Frontiers in Psychiatry, 9, 139.
Ziegler, M., & Buehner, M. (2009). Modeling socially desirable responding and its effects. Educational and Psychological Measurement, 69, 548–565.
We sincerely thank Hirofumi Hashimoto from Osaka City University for his helpful comments on an earlier draft of this article.
The Japan Society for the Promotion of Science (JSPS) funded this study (Grant-in-Aid for Young Scientists [B], 26780351, and Grant-in-Aid for JSPS Research Fellow, 16J09133).
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Inoue, Y., Yokota, K. External validity of the benign and malicious envy scale with Japanese undergraduate student and non-student samples. Meas Instrum Soc Sci 4, 5 (2022). https://doi.org/10.1186/s42409-022-00034-1
- Benign envy
- Malicious envy
- Dispositional envy
- Measurement invariance