The multiple group IRT measurement invariance analysis of the Self-Compassion Scale in ten international samples
Abstract
The purpose of this study was to examine the measurement invariance of the Self-Compassion Scale by IRT differential test functioning in ten distinct populations (n = 13623 participants) from ten different countries: Australia (n = 517), China (n = 321), Czech Republic (n = 5081), Germany (n = 2510), Italy (n = 384), Portugal (n = 512), Slovakia (n = 1181), South Korea (n = 1813), Turkey (n = 471), and USA (n = 833). We assessed differential test functioning for the two SCS subscales, Self-compassionate responding and Self-uncompassionate responding separately, because previous bifactor and two-tier analyses of the scale showed the best fit with two separate general factors, and not for the overall score. Only 13 of the 45 comparisons for Self-compassionate responding and 13 of the 45 comparisons for Self-uncompassionate responding (analyses of every pair) demonstrated measurement invariance (no differential test functioning). Generally, our results revealed that the two subscales of Self-compassionate responding and Self-uncompassionate responding were not equivalent among all countries and groups. Therefore, it is impossible to compare overall scores across all countries. Two subscales of the Self-Compassion Scale (Self-compassionate responding and Self-uncompassionate responding) are valid and reliable instruments with substantial potential of use cross-culturally, but results reveal significant cross-cultural differences in the way these two constructs are measured by the subscales of the SCS. Future analyses of the meanings and connotations of this construct across the world are necessary to develop a scale which allows cross-cultural comparisons of various treatment outcomes related to self-compassion.
Keywords
References
Arimitsu, K. (2014). Development and validation of the Japanese version of the Self-Compassion Scale. The Japanese Journal of Psychology, 85(1), 50–59. https://doi.org/10.4992/jjpsy.85.50
Azizi, A., Mohammadkhani, P., Lotfi, S., & Bahramkhani, M. (2013). The validity and reliability of the Iranian version of the Self-Compassion Scale. Iranian Journal of Clinical Psychology, 2(3), 17-23.
Barnett, M. D., & Sharp, K. J. (2016). Maladaptive perfectionism, body image satisfaction, and disordered eating behaviors among US college women: The mediating role of self-compassion. Personality and Individual Differences, 99, 225-234. https://doi.org/10.1016/j.paid.2016.05.004
Behling, O., & Law, K. S. (2000). Translating questionnaires and other research instruments: Problems and solutions. Thousand Oaks: Sage Publications. https://doi.org/10.4135/9781412986373
Benda, J., & Reichová, A. (2016). Psychometrické charakteristiky české verze Self-Compassion Scale (SCS-CZ). Československá psychologie, 60(2), 120-136.
Birkett, M. A. (2014). Self-compassion and empathy across cultures: Comparison of young adults in China and the United States. International Journal of Research Studies in Psychology, 3(3), 25-34. https://doi.org/10.5861/ijrsp.2013.551
Brown, L., Bryant, C., Brown, V. M., Bei, B., & Judd, F. K. (2014). Self-Compassion weakens the association between hot flushes and night sweats and daily life functioning and depression. Maturitas. 78(4), 298-303. https://doi.org/10.1016/j.maturitas.2014.05.012
Castilho, P., & Pinto-Gouveia, J. (2011). Self-Compassion: validation of the Portuguese version of the Self-Compassion Scale and its relation with early negative experiences, social comparison and psychopathology. Psychologica, 54, 203-231. https://doi.org/10.14195/1647-8606_54_8
Chalmers, R. P. (2012). mirt: A multidimensional item response theory package for the R environment. Journal of Statistical Software, 48(6), 1-29. https://doi.org/10.18637/jss.v048.i06
Chalmers, R. P. (2015). Extended mixed-effects item response models with the MH-RM algorithm. Journal of Educational Measurement, 52, 200-222. https://doi.org/10.1111/jedm.12072
Chalmers, R. P, Counsell, A., & Flora, D. B. (2016). It might not make a big DIF: improved differential test functioning statistics that account for sampling variability. Educational and Psychological Measurement, 76(1), 114–140. https://doi.org/10.1177/0013164415584576
Chen, J., Yan, L., & Zhou, L. (2011). Reliability and validity of Chinese version of Self-compassion Scale. Chinese Journal of Clinical Psychology, 19(6), 734-736.
Conversano, C., Ciacchini, R., Orrù, G., Di Giuseppe, M., Gemignani, A., & Poli, A. (2020). Mindfulness, Compassion, and Self-Compassion Among Health Care Professionals: What's New? A Systematic Review. Frontiers in psychology, 11, 1683. https://doi.org/10.3389/fpsyg.2020.01683
Costa, J., Marôco, J., Pinto‐Gouveia, J., Ferreira, C., & Castilho, P. (2016). Validation of the psychometric properties of the Self-Compassion Scale. Testing the factorial validity and factorial invariance of the measure among borderline personality disorder, anxiety disorder, eating disorder and general populations. Clinical Psychology & Psychotherapy. 23(5), 460-468. https://doi.org/10.1002/cpp.1974
Cucina, J. & Byle, K. (2017). The bifactor model fits better than the higher-order model in more than 90% of comparisons for mental abilities test batteries. Journal of Intelligence, 5, 27. https://doi.org/10.3390/jintelligence5030027
Cunha, M., Xavier, A., & Castilho, P. (2016). Understanding self-compassion in adolescents: Validation study of the Self-Compassion Scale. Personality and Individual Differences, 93(9), 56–62. https://doi.org/10.1016/j.paid.2015.09.023
DeMars, C. E. (2011). An analytic comparison of effect sizes for differential item functioning. Applied Measurement in Education, 24, 189-209. https://doi.org/10.1080/08957347.2011.580255
DeMars, C. E. (2012). A comparison of limited-information and full-information methods in Mplus for estimating Item Response Theory parameters for non-normal populations. Structural Equation Modeling: A Multidisciplinary Journal, 19, 610-632. https://doi.org/10.1080/10705511.2012.713272
Deniz, M. E., Kesici, S., & Sumer, A. S. (2008): The validity and reliability of the Turkish version of the Self-Compassion Scale. Social Behavior and Personality, 36(9), 1151-1160. https://doi.org/10.2224/sbp.2008.36.9.1151
Dundas, I., Svendsen, J. L., Wiker, A. S., Granli, K. V., & Schanche, E. (2016). Self-compassion and depressive symptoms in a Norwegian student sample. Nordic Psychology, 68(1), 58-72. https://doi.org/10.1080/19012276.2015.1071203
Falconer, C.J., King, J.A., & Brewin, C.R. (2015). Demonstrating mood repair with a situation-based measure of self-compassion and self-criticism. Psychology and psychotherapy. 88(4), 351-65. https://doi.org/10.1111/papt.12056
Garcia-Campayo, J., Navarro-Gil, M., Andrés, E., Montero-Marin, J., López-Artal, L., & Demarzo, M. M. (2014). Validation of the Spanish versions of the long (26 items) and short (12 items) forms of the Self-Compassion Scale (SCS). Health and Quality of Life Outcomes, 12(1), 4. https://doi.org/10.1186/1477-7525-12-4
Gilbert, P., Clark, M., Hempel, S., Miles, J.N.V. & Irons, C. (2004). Criticising and reassuring oneself: An exploration of forms, styles and reasons in female students. British Journal of Clinical Psychology, 43, 31-50. https://doi.org/10.1348/014466504772812959
Gilbert, P., & Irons, C. (2008). Shame, self-criticism and self-compassion in adolescence. In NB Allen & LB Sheeber (Eds.), Adolescent Emotional Development and the Emergence of Depressive Disorders. (pp. 195-214). Cambridge University Press. https://doi.org/10.1017/CBO9780511551963.011
Greenberg, L. S. & Watson, J. (2005). Emotion-focused therapy of Depression. Washington, D. C.: American Psychological Association Press.
Halamová, J., Kanovský, M., & Pacúchová, M. (2017). Self-Compassion Scale: IRT psychometric analysis validation and factor structure – Slovak translation. Psychologica Belgica. 57(4), 190–209. https://doi.org/10.5334/pb.398
Halamová, J., Kanovský, M., Petrocchi, N., Moreira, H., López Angarita, A. et al. (2021). The factor structure of the Self-Compassion Scale in eleven distinct populations across the world. Measurement and Evaluation in Counseling and Development. 54(1), 1-23, https://doi.org/10.1080/07481756.2020.1735203
Heine, S. J., Lehman, D. R., Markus, H. R., & Kitayama, S. (1999). Is there a universal need for positive self-regard? Psychological Review, 106, 766- 794. https://doi.org/10.1037/0033-295X.106.4.766
Heshmati, R., & Pellerone, M. (2018). Relationship of dispositional mindfulness, self-destruction and self-criticism in academic performance of boy students. Mediterranean Journal of Clinical Psychology, 6(3). https://doi.org/10.6092/2282-1619/2018.6.1716
Hupfield, J. & Ruffieux, N. (2011). Validierung einer deutschen version der Self-Compassion Scale (SCS-D). Zeitschrift für Klinische Psychologie und Psychotherapie, 40 (2), 115–123. https://doi.org/10.1026/1616-3443/a000088
Hwang, S., Kim, G., Yang, J. W., & Yang, E. (2016). The moderating effects of age on the relationships of self-compassion, self-esteem, and mental health. Japanese Psychological Research. 58(2), 194-205. https://doi.org/10.1111/jpr.12109
Kankaraš, M., Vermunt, J. K., & Moors, G. (2011). Measurement equivalence of ordinal items: a comparison of factor analytic, item response theory, and latent class approaches. Sociological Methods & Research, 40(2), 279 – 310. https://doi.org/10.1177/0049124111405301
Kim, E. S., & Yoon, M. (2011). Testing measurement invariance: a comparison of multiple-group categorical CFA and IRT. Structural Equation Modeling: A Multidisciplinary Journal, 18(2), 212-228. https://doi.org/10.1080/10705511.2011.557337
Kitayama, S., & Karasawa, M. (1997). Implicit self-esteem in Japan: Name letters and birthday numbers. Personality and Social Psychology Bulletin, 23, 736-742. https://doi.org/10.1177/0146167297237006
Kotsou, I., & Leys, C. (2016). Self-Compassion Scale (SCS): Psychometric properties of The French zranslation and its relations with psychological well-being, affect and depression. PloS one, 11(4). https://doi.org/10.1371/journal.pone.0152880
Körner, A., Coroiu, A., Copeland, L., Gomez-Garibello, C., Albani, C., Zenger, M., & Brähler, E. (2015). The role of self-compassion in buffering symptoms of depression in the general population. PLoS ONE, 10(10). https://doi.org/10.1371/journal.pone.0136598
Khramtsova I.I., & Chuykova T.S. (2016). Mindfulness and self-compassion as predictors of humor styles in US and Russia. Sotsial'naia psikhologiia i obshchestvo = Social Psychology and Society, 7(2), 93—108. https://doi.org/10.17759/sps.2016070207
Kwan, V. S. Y., Kuang, L. L., & Hui, N. H. H. (2009). Identifying the sources of self-esteem: The mixed medley of benevolence, merit, and bias. Self and Identity, 8, 176-195. https://doi.org/10.1080/15298860802504874
Lee, W. K., & Lee, K. (2010). The validation study of the Korean version of Self-Compassion Scale with adult women in community [Korean]. Journal of Korean Neuropsychiatric Association, 49(2), 193-200.
López, A., Sanderman, R., Smink, A., Zhang, Y., van Sonderen, E., Ranchor, A., & Schroevers, M. J. (2015). A reconsideration of the Self-Compassion Scale’s total score: self-compassion versus self-criticism. PloS One, 10(7). https://doi.org/10.1371/journal.pone.0132940
Mansolf, M., & Reise, S. P. (2017). When and why the second-order and bifactor models are distinguishable. Intelligence, 61, 120-129. https://doi.org/10.1016/j.intell.2017.01.012
Mantzios, M., Wilson, J. C., & Giannou, K. (2013). Psychometric properties of the Greek versions of the self-compassion and mindful attention and awareness scales. Mindfulness, 6(1), 123-132. https://doi.org/10.1007/s12671-013-0237-3
Meade, A. W. & Lautenschlager, G. J. (2004). A comparison of item response theory and confirmatory factor analytic methodologies for establishing measurement equivalence/invariance, Organizational Research Methods, 7(4), 361-388. https://doi.org/10.1177/1094428104268027
Meares, R., Gerull, F., Stevenson, J., & Korner, A. (2011). Is self-disturbance the core of borderline personality disorder? An outcome study of borderline personality factors. Australian and New Zealand Journal of Psychiatry, 45, 214–222. https://doi.org/10.3109/00048674.2010.551280
Merlo, E. M., Stoian, A. P., Motofei, I. G., & Settineri, S. (2020). Clinical Psychological Figures in Healthcare Professionals: Resilience and Maladjustment as the “Cost of Care”. Frontiers in Psychology, 11. https://doi.org/10.3389/fpsyg.2020.607783
Montero-Marín, J., Gaete, J., Demarzo, M., Rodero, B., Lopez, L.C., & García-Campayo, J. (2016). Self-Criticism: A measure of uncompassionate behaviors toward the self, based on the negative components of the Self-Compassion Scale. Frontiers in Psychology. 30(7), 1281. https://doi.org/10.3389/fpsyg.2016.01281
Moreira, H., Carona, C., Silva, N., Nunes, J., & Canavarro, M. C. (2016). Exploring the link between maternal attachment-related anxiety and avoidance and mindful parenting: The mediating role of self-compassion. Psychology and Psychotherapy: Theory, Research and Practice, 89(4), 369-384. https://doi.org/10.1111/papt.12082
Moreira, H., Gouveia, M. J., Carona, C., Silva, N., & Canavarro, M. C. (2015). Maternal attachment and children's quality of life: The mediating role of self-compassion and parenting stress. Journal of Child and Family Studies, 24(8), 2332-2344. https://doi.org/10.1007/s10826-014-0036-z
Murray, G., Judd, F., Jackson, H., Fraser, C., Komiti, A., Hodgins, G., Pattison, P., Humphreys, J., & Robins, G. (2004). Rurality and mental health: the role of accessibility. Australian and New Zealand Journal of Psychiatry. 38(8), 629-634. https://doi.org/10.1111/j.1440-1614.2004.01426.x
Neff, K. D. (2003). The development and validation of a scale to measure self-compassion. Self and identity, 2(3), 223-250. https://doi.org/10.1080/15298860390209035
Neff, K. D. (2004). Self-compassion and psychological well-being. Constructivism in the Human Sciences, 9, 27-37. https://doi.org/10.1037/e633942013-240
Neff, K. D. (2016). The Self-Compassion Scale is a valid and theoretically coherent measure of self-compassion. Mindfulness, 7(1), 264-274. https://doi.org/10.1007/s12671-015-0479-3
Neff, K. D., & Dahm, K. A. (2014). Self-Compassion: What it is, what it does, and how it relates to mindfulness (pp. 121-140). In M. Robinson, B. Meier & B. Ostafin (Eds.) Mindfulness and Self-Regulation. New York: Springer. https://doi.org/10.1007/978-1-4939-2263-5_10
Neff, K. D., Pisitsungkagarn, K., & Hseih, Y. (2008). Self-compassion and self-construal in the United States, Thailand, and Taiwan. Journal of Cross-Cultural Psychology, 39, 267-285. https://doi.org/10.1177/0022022108314544
Neff, K. D. & Vonk, R. (2009). Self-compassion versus global self-esteem: Two different ways of relating to oneself. Journal of Personality, 77(1), 23-50. https://doi.org/10.1111/j.1467-6494.2008.00537.x
Neff, K. D., Whittakar, T. & Karl, A. (2017). Evaluating the factor structure of the Self-Compassion Scale in four distinct populations: Is the use of a total self-compassion score justified? Journal of Personality Assessment. 99, 596–607. https://doi.org/1080/00223891.2016.1269334
O’Connor, R. C. & Noyce, R. (2008). Personality and cognitive processes: Self-criticism and different types of rumination as predictors of suicidal ideation. Behaviour Research and Therapy, 46(3), 392–401. https://doi.org/10.1016/j.brat.2008.01.007
Petrocchi, N., Ottaviani, C., & Couyoumdjian, A. (2014). Dimensionality of self-compassion: translation and construct validation of the self-compassion scale in an Italian sample. Journal of Mental Health, 23(2), 72-77. https://doi.org/10.3109/09638237.2013.841869
Pisitsungkagarn, K., Taephant, N., & Attasaranya, P. (2014). Body image satisfaction and self-esteem in Thai female adolescents: the moderating role of self-compassion. International journal of adolescent medicine and health, 26(3), 333-338. https://doi.org/10.1515/ijamh-2013-0307.
Raju, N. S., Laffitte, L. J. & Byrne, B. B. (2002). Measurement equivalence: A comparison of methods based on confirmatory factor analysis and item response theory. Journal of Applied Psychology, 87(3), 517–529. https://doi.org/10.1037//0021-9010.87.3.517
Reise, S. P., Widaman, K. H. & Pugh, R.H. (1993). Confirmatory factor analysis and item response theory: two approaches for exploring measurement invariance, Psychological Bulletin, 114(3), 552-566. https://doi.org/10.1037/0033-2909.114.3.552
Ritter, P., Lorig, K., Laurent, D., & Matthews, K. (2004). Internet versus mailed questionnaires: a randomized comparison. Journal of Medical Internet Research, 6(3), e29. https://doi.org/10.2196/jmir.6.3.e29
Shahar, B., Carlin, E.R., Engle, D.E., Hegde, J., Szepsenwol, O., & Arkowitz, H. (2012). A pilot investigation of emotion-focused two-chair dialogue intervention for self-criticism. Clinical Psychology and Psychotherapy, 6, 496–507. https://doi.org/10.1002/cpp.762
Shahar, B., Szsepsenwol, O., Zilcha-Mano, S., Haim, N., Zamir, O., Levi-Yeshuvi, S., & Levit-Binnun, N. (2015). A wait‐list randomized controlled trial of loving‐kindness meditation programme for self‐criticism. Clinical psychology & psychotherapy, 22(4), 346-356. https://doi.org/10.1002/cpp.1893
Şenyuva, E., Kaya, H., Işik, B., & Bodur, G. (2014). Relationship between self-compassion and emotional intelligence in nursing students. International journal of nursing practice, 20(6), 588-596. https://doi.org/10.1111/ijn.12204
Terry, M. L., & Leary, M. R. (2011). Self-compassion, self-regulation, and health. Self and Identity, 10(3), 352–362. https://doi.org/10.1080/15298868.2011.558404
Tóth-Király, I., Bőthe, B., & Orosz, G. (2016). Exploratory structural equation modeling analysis of the Self-Compassion Scale. Mindfulness, 7, 1–12. https://doi.org/10.1007/s12671-016-0662-1
Vandenberg, R. J. & Lance, C. E. (2000). A review and synthesis of the measurement invariance literature: suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3(1), 4-70. https://doi.org/10.1177/109442810031002
Voruz, S. (2013). An examination of the psychometric properties of the Self-Compassion Scale – Short Form among a help-seeking clinical sample (Doctoral dissertation, Pacific University). Retrieved from: http://commons.pacificu.edu/spp/430
Warren, R., Smeets, E. & Neff, K. D. (2016). Self-criticism and self-compassion: Risk and resilience for psychopathology. Current Psychiatry, 15(12), 18-32.
Zeng, X., Wei, J., Oei, T. P., & Liu, X. (2016). The self-compassion scale is not validated in a Buddhist sample. Journal of religion and health, 55(6), 1996-2009. https://doi.org/10.1007/s10943-016-0205-z
DOI: https://doi.org/10.6092/2282-1619/mjcp-2682
Refbacks
- There are currently no refbacks.