This paper is in the following e-collection/theme issue:

Mobile Health (mhealth) (2939) Tools and Questionnaires in Human Factors Evaluation (51) Research Instruments, Questionnaires, and Tools (576)

Published on in Vol 24, No 11 (2022): November

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/38108, first published .
Psychometric Properties of the Chinese Warwick-Edinburgh Mental Well-being Scale in Medical Staff: Cross-sectional Study

Psychometric Properties of the Chinese Warwick-Edinburgh Mental Well-being Scale in Medical Staff: Cross-sectional Study

Psychometric Properties of the Chinese Warwick-Edinburgh Mental Well-being Scale in Medical Staff: Cross-sectional Study

Authors of this article:

Aishu Dong1 Author Orcid Image ;   Jing Huang2 Author Orcid Image ;   Shudan Lin2 Author Orcid Image ;   Jianing Zhu2 Author Orcid Image ;   Haitao Zhou1 Author Orcid Image ;   Qianqian Jin3 Author Orcid Image ;   Wei Zhao1 Author Orcid Image ;   Lianlian Zhu2 Author Orcid Image ;   Wenjian Guo4 Author Orcid Image
Article Authors Cited by (5) Tweetations (2) Metrics

    Original Paper

    1Cardiology Department, Second Affiliated Hospital of Wenzhou Medical University, Wenzhou, China

    2Wenzhou Medical University, Wenzhou, China

    3Emergency Department, Second Affiliated Hospital of Wenzhou Medical University, Wenzhou, China

    4Oncology Department, Second Affiliated Hospital of Wenzhou Medical University, Whenzhou, China

    *these authors contributed equally

    Corresponding Author:

    Wenjian Guo, PhD

    Oncology Department

    Second Affiliated Hospital of Wenzhou Medical University

    College West Road 109, 0577

    Whenzhou, 325000

    China

    Phone: 86 13957729220

    Email: guowenjian@wmu.edu.cn


    Background: Worldwide, mental well-being is a critical issue for public health, especially among medical staff; it affects professionalism, efficiency, quality of care delivery, and overall quality of life. Nevertheless, assessing mental well-being is a complex problem.

    Objective: This study aimed to evaluate the psychometric properties of the Chinese-language version of the 14-item Warwick-Edinburgh Mental Well-being Scale (WEMWBS) in medical staff recruited mainly from 6 hospitals in China and provide a reliable measurement of positive mental well-being.

    Methods: A cross-sectional online survey was conducted of medical staff from 15 provinces in China from May 15 to July 15, 2020. Confirmatory factor analysis (CFA) was conducted to test the structure of the Chinese WEMWBS. The Spearman correlations of the Chinese WEMWBS with the 5-item World Health Organization Well-Being Index (WHO-5) were used to evaluate convergent validity. The Cronbach α and split-half reliability (λ) represented internal consistency. A graded response model was adopted for an item response theory (IRT) analysis. We report discrimination, difficulty, item characteristic curves (ICCs), and item information curves (IICs). ICCs and IICs were used to estimate reliability and validity based on the IRT analysis.

    Results: A total of 572 participants from 15 provinces in China finished the Chinese WEMWBS. The CFA showed that the 1D model was satisfactory and internal consistency reliability was excellent, with α=.965 and λ=0.947, while the item-scale correlation coefficients ranged from r=0.727 to r=0.900. The correlation coefficient between the Chinese WEMWBS and the WHO-5 was significant, at r=0.746. The average variance extraction value was 0.656, and the composite reliability value was 0.964, with good aggregation validity. The discrimination of the Chinese WEMWBS items ranged from 2.026 to 5.098. The ICCs illustrated that the orders of the category thresholds for the 14 items were satisfactory.

    Conclusions: The Chinese WEMWBS showed good psychometric properties and can measure well-being in medical staff.

    J Med Internet Res 2022;24(11):e38108

    doi:10.2196/38108

    Keywords

    psychometric property; Chinese Warwick-Edinburgh Mental Well-being Scale; classical test theory; well-being; item response theory; medical staff; China 


    Background

    Mental well-being is a public health concern worldwide; adequate mental well-being is associated with better health-related quality of life and longer life expectancy [1]. In recent years, the mental well-being of employees in several occupations has gained substantial attention [2-6]. A meta-analysis revealed that numerous health care workers had various psychological problems [7]. It is well known that medical staff experience many work-related stresses (eg, prolonged and irregular working hours, night shifts, high-intensity work, emotional exhaustion, chronicity of care, and moral conflicts), which may negatively influence their mental well-being, causing depression, anxiety, sleeping disorders, and other problems. Impaired mental well-being can affect health care providers’ professionalism, quality of care delivery, efficiency, and overall quality of life [8,9].

    Moreover, it has been reported that the overall mental health status of Chinese medical staff is unfavorable [10,11]. This finding suggests that the mental well-being of medical staff is critically important to public health [12,13]. For this reason, it is crucial to measure the mental health status of medical staff and identify work-related risk factors to protect their well-being [14].

    The Warwick-Edinburgh Mental Well-being Scale (WEMWBS) is a relatively new, short, acceptable scale that has been translated into several languages [15-18]. It has demonstrated excellent reliability, good validity, and internal consistency [19]. Studies of public mental health have confirmed the WEMWBS’s ability to offer rigor in psychological evaluations [20]; it focuses on protective and promoting factors that can provide a rational basis for the orientation of policy makers formulating interventions [21].

    Previous studies have reported the psychological performance of the Chinese-language version of the WEMWBS in clinical and nonclinical settings in China, but all have had limitations [15,22]. Research by Liu et al [23] appears to be the earliest psychometric analysis of the Chinese WEMWBS; however, 2 issues need addressing. First, their paper was written in Chinese, making it burdensome to read for non–Chinese-speaking investigators and impeding comparisons of China with other countries. Second, the age of the study participants ranged from 60 to 97 years, resulting in information and selection bias. The generalizability of the findings from Dong et al [22] is problematic, because the 191 patients with chronic heart failure in that study came from 1 hospital in a Chinese city. A study by Fung [24] and an earlier study by Dong et al [15] were limited because all respondents were university students recruited from either a single university or a single hospital nursing internship program in a Chinese city; this could have caused pervasive information and selection bias in these studies’ assessment of the psychometric properties of the WEMWBS. A study by Waqas et al [25] explored the reliability and validity of the WEMWBS in Pakistan; Taggart et al [26] investigated the WEMWBS in a targeted sample of minority ethnic groups living in the UK who self-identified as Chinese or Pakistani by background. Additionally, no previous investigation has combined a graded response model (GRM), item response theory (IRT), and classical test theory (CTT) to evaluate the psychometric properties of the WEMWBS. It is necessary to find a comprehensive method and a better representative sample that covers participants from southern and northern areas to assess the performance of the Chinese WEMWBS.

    Objective of the Study

    We administered the Chinese WEMWBS to medical staff to evaluate their psychological characteristics and explore and popularize this questionnaire on mental well-being, which is suitable for Chinese national conditions. We aim to provide theoretical support for improving the mental well-being of medical staff.


    Data Collection

    From May 15 to July 15, 2020, purposeful sampling was conducted to recruit 572 medical staff online, mainly from 6 hospitals in mainland China (the First Affiliated Hospital of Wenzhou Medical University, the Second Affiliated Hospital of Wenzhou Medical University, the Second Hospital of Dalian Medical University, the Second Affiliated Hospital of Zhongguo Medical University, Lishui People’s Hospital, and Chenzhou Third People’s Hospital).

    Ethics Approval

    All participants provided informed consent before participation, and the Medical Ethics Committee of the Second Affiliated Hospital of Wenzhou Medical University approved the study (LCKY2019-288).

    Instruments

    Data were collected via a self-administered online questionnaire. The first section collected sociodemographic characteristics, including age, marital status, gender, body weight (in kilograms), height (in meters), professional status, and education level. The second section examined lifestyle habits, including working hours, night shifts per week, smoking history, drinking history, consumption of vegetables and fruit, physical exercise, and self-reported personality. The third section examined mental well-being using the WEMWBS and self-perceived quality of life (QoL). The WEMWBS is a 14-item sequential scale that measures 3 aspects of mental well-being: positive psychological function, emotion, and interpersonal relationship satisfaction. All items were scored on a 5-point Likert scale, including 1 (never), 2 (occasionally), 3 (yes), 4 (often), and 5 (always). The total score ranged from 14 to 70, with higher scores representing stronger subjective well-being. The third section of the questionnaire used the 36-Item Short Form Health Survey, Version 2 (SF-36 v2) to assess self-perceived QoL. The SF-36 v2 is a 36-item structured scale that comprehensively summarizes respondents’ QoL across 8 dimensions: physical functioning (10 items), role-physical (4 items), bodily pain (2 items), general health (5 items), vitality (4 items), social functioning (2 items), role-emotional (3 items), and mental health (5 items). The physical component summary and the mental component summary are 2 subscales of the 8 dimensions. In addition to the 8 dimensions listed above, the SF-36 v2 includes another health condition, reported health transition, which measures overall changes in health status over the past year.

    Statistical Analysis

    We used EpiData (version 3.1; EpiData Association) for double entry and data management. Data collection and analysis were carried out using SPSS (version 27.0; IBM Corp) and R (version 4.1.1; R Foundation for Statistical Computing). Means and SDs were calculated for continuous data and frequencies and percentages for categorical data.

    Dimensionality Test

    Principal component analysis of the Chinese WEMWBS was used to independently identify a 1D hypothesis; this analysis indicates good quality (ie, statistical power) of the 1D structure of the model when the first eigenvalue is more than 50% of the total variation.

    Ceiling Effect and Floor Effect

    A ceiling or floor effect is present when subjects receive the scale’s highest or lowest score. Measurement scales with ceiling or floor effects may have questionable validity, reliability, and reactivity. The significance level should be 20%.

    Item Analysis

    Item analysis determines effectiveness and the ability to discriminate the entire scale. The process used is to sum the scores of the items for each participant, divide them into high-score and low-score groups (with 27% and 73% quantiles as the boundaries), and finally use a 2-tailed t test to identify differences between the groups. If there is a difference, the scale item is appropriately designed; otherwise, it indicates that the item has a questionable ability to discriminate between respondents, meaning that the item should be deleted or rearranged.

    Reliability Analysis: Internal Consistency of the Scale

    We used the Cronbach α and split-half reliability (λ) to represent internal consistency reliability. The former indicates the homogeneity of each item in the scale; we considered α=.7 as the threshold above which the scale showed desired reliability. The latter measures consistency between the 2 halves of these items, divided according to the precedence and the odd-even sequence of the serial number. Generally, a correlation coefficient of r≥0.70 is considered acceptable.

    Test-Retest Reliability

    The test-retest reliability of the WEMWBS scale was estimated within a 2-week interval by comparing 2 sets of scores using the intraclass correlation coefficient.

    Construct Validity

    Confirmatory factor analysis (CFA) of item responses was implemented using the weighted least-squares method to test the structural equation modeling of the hypothesized unidimensionality of the WEMWBS. Statistical analysis of correlations was performed using SAS (version 9.4; SAS Institute Inc), assuming no relationship between the residuals. A stepwise strategy was then used to add the matrix elements with the highest dependencies until sufficient fit statistics were achieved.

    The predicted levels of the goodness-of-fit index and adjusted goodness-of-fit index based on degrees of freedom correction were >0.9 and >0.8, respectively.

    A root mean square error of approximation (RMSEA) below the accepted level of 0.06 [27] indicates only a tiny number of unintended deviations. A chi-square statistic with P<.05 indicates a considerable amount of actual covariance between measurements that the model cannot explain [28]. Nevertheless, large sample sizes may exaggerate this and are therefore unsuitable [29].

    Compatible Validity

    This parameter refers to the extent to which the scores of the new scale are relevant to the scores of another scale with the same content and known validity. If the compatibility coefficient is high, the 2 scales measure the same content, and the new scale is equally effective. Based on the range of these 2 scales, we hypothesized a strong correlation between the WEMWBS and the 5-item World Health Organization Well-Being Index (WHO-5) scale for capturing mental well-being, with a coefficient above r=0.7.

    Convergence Validity

    Convergence validity refers to the similarity of measurement results when different algorithmic methods are grouped to determine the same feature. The evaluation indices usually include composite reliability (CR), factor loading, and average variance extracted (AVE), where AVE greater than 0.5 and CR greater than 0.7 indicate that the aggregation validity is acceptable.

    IRT Analysis

    IRT, also known as latent trait theory, is a modern psychometric theory proposed to compensate for the limitations of CTT. According to an exploratory factor analysis of CTT, the Chinese WEMWBS is a 1D scale. Therefore, in this study, the responses of the 572 participants to the WEMWBS on a 5-point Likert-type scale were interpreted with the Samejima GRM [30]. These parameters, including a discrimination parameter (referred to as a), a difficulty parameter (referred to as b), item characteristic curves (ICCs), and item information curves (IICs), were administered to implement filtering entry. The discrimination parameter evaluates the strength of the relationship between each item and the scale; the difficulty parameter identifies an item in the potential continuum of the structure that best distinguishes each item. Each item has 5 levels; we used level 1 as a reference and set the remaining 4 levels as difficulty levels. The difficulty level parameter was calculated between 1 and 2, 2 and 3, 3 and 4, and 4 and 5, denoted as thresholds: ≥2, ≥3, ≥4, and 5.

    When the discrimination parameter is <0.4 or >3 and the difficulty parameter range exceeds –3 to 3, the item should be considered for deletion. The model simulates ICCs for each option for the 14 items. The first and fifth ICCs change unvaryingly, and the second, third, and fourth ICCs are typically distributed, which can be considered ideal. The more ideal the ICC distribution, the more considerable the corresponding project information. Moreover, a larger item information function results in greater accuracy. Item screening was then carried out. When an item did not meet the requirement for 3 or more parameters, it was considered for deletion based on professional knowledge and expert opinion. These calculations were performed using Stata/MP (version 14.0 for Mac; StataCorp LP).


    Descriptive Statistics of the Scale

    The total sample of 572 medical staff had a mean score for the Chinese WEMWBS of 38.47 (95% CI 37.45-39.61; SD 13.23; skewness 0.449; kurtosis –0.486) and a median score of 37, indicating a latent skewed trait distribution (Figure 1). An independent-sample t test showed no difference between the total WEMWBS score and gender (t1=–1.477; P=.14). A Pearson correlation analysis did not indicate any significant relationship between the score for mental well-being and age; therefore, further validation analyses did not include participant age.

    ‎
    Figure 1. Histogram showing the scores of medical staff (N=572) on the Chinese-language version of the WEMWBS. The mean score was 38.47 (SD 13.227). WEMWBS: Warwick-Edinburgh Mental Well-being Scale.
    View this figure

    Item Analysis

    As shown in Table 1, the values for specific items were significantly different in the high-score and low-score groups (P<.001), meaning that all 14 items could differentiate the 2 groups well, and that none should be discarded. The correlation coefficient between each item and the total score of the instrument ranged from r=0.727 to r=0.900. As seen in Table 2, none of the items reached a rate of 20%, suggesting that there were no ceiling or floor effects.

    Table 1. Item analysis (discrimination analysis) and item-scale correlation of the Chinese version of the Warwick-Edinburgh Mental Well-being Scale.
    ItemLow-score group (n=171), mean (SD) scoreHigh-score group (n=164), mean (SD) scoret (decision value)P valueItem-scale correlation, r
    11.78 (0.68)3.91 (0.98)–23.125<.0010.778
    21.56 (0.51)3.66 (0.99)–24.248<.0010.823
    31.81 (0.53)4.24 (0.81)–32.428<.0010.831
    41.77 (0.65)4.05 (0.94)–25.866<.0010.727
    51.88 (0.56)4.43 (0.67)–37.806<.0010.859
    61.75 (0.46)3.89 (0.87)–27.92<.0010.862
    71.75 (0.43)3.84 (0.90)–26.892<.0010.871
    81.76 (0.45)4.16 (0.78)–34.36<.0010.900
    91.70 (0.48)3.73 (0.96)–24.182<.0010.842
    101.82 (0.53)4.12 (0.79)–31.044<.0010.870
    111.72 (0.48)3.70 (1.00)–22.947<.0010.819
    121.69 (0.48)3.77 (0.89)–26.601<.0010.817
    131.73 (0.50)3.96 (0.91)–27.705<.0010.789
    141.77 (0.46)4.04 (0.81)–31.21<.001 0.879
    Table 2. Floor effect, ceiling effect, and item-scale correlation of Chinese-language version of the Warwick-Edinburgh Mental Well-being Scale. The floor and ceiling effects were defined as the lowest (1 point) and highest (5 points) scores, respectively (N=572).
    ItemSubjects with floor effect, n (%)Subjects with ceiling effect, n (%)Item-scale correlationa, r
    169 (12.1)67 (11.7)0.764
    2100 (17.5)44 (7.7)0.809
    345 (7.9)86 (15)0.823
    479 (13.8)82 (14.3)0.697
    534 (5.9)92 (16.1)0.837
    647 (8.2)49 (8.6)0.867
    751 (8.9)43 (7.5)0.885
    845 (7.9)69 (12.1)0.904
    963 (11)40 (7)0.835
    1045 (7.9)64 (11.2)0.865
    1163 (11)45 (7.9)0.832
    1266 (11.5)44 (7.7)0.825
    1367 (11.7)58 (10.1)0.768
    1447 (8.2)52 (9.1)0.875

    aCorrelations were deemed significant at the P<.01 (ie, 2 significant figures) level.

    Reliability Analysis

    Internal consistency reliability was good (Cronbach α=.965). The corrected item-total correlation values of the items were all greater than 0.5, indicating a good correlation between items and reliability (Table 3). Two weeks after completing the questionnaire, 35 subjects completed it again; the test-retest reliability was measured at 0.810, indicating that the scale had good stability. The split-half reliability of the scale was λ=0.947 according to the first half and the second half of the serial number, while the value was λ=0.970 according to the odd-even status of the serial number.

    Table 3. Cronbach reliability analysis of Chinese-language version of the Warwick-Edinburgh Mental Well-being Scale.
    ItemAverage score after deleting each itemScaled variance after deleting termsCorrected item-total correlationSquared multiple correlationCronbach α if item deleted
    135.73151.7660.7390.642.964
    235.97151.6540.7920.691.963
    335.52149.8330.7990.711.963
    435.60152.1740.6760.510.966
    535.43149.2690.8320.743.962
    635.75151.8300.8390.769.962
    735.84151.7220.8500.791.962
    835.63148.9710.8810.822.961
    935.87152.4970.8170.716.963
    1035.61150.1370.8460.774.962
    1135.87152.9460.7900.677.963
    1235.83152.4080.7860.664.963
    1335.71151.6720.7520.608.964
    1435.75150.6360.8580.756.962

    Construct Validity: Exploratory Factor Analysis

    A Kaiser-Meyer-Olkin value of 0.963 for the 14 items and a value of 7844.584 for the Bartlett sphericity test (P<.001) demonstrated that the data obtained were suitable for factor analysis. A principal component factor analysis was used with varimax rotation to evaluate construct validity. Table 4 shows factor loadings for the 14 items, which ranged from 0.714 for item 4 to 0.903 for item 8.

    Table 4. Validity analysis result of Chinese-language version of the Warwick-Edinburgh Mental Well-being Scale.
    ItemFactor loadingCommon degree (common factor variance)
    10.7720.597
    20.8210.675
    30.8280.685
    40.7140.510
    50.8560.733
    60.8680.753
    70.8770.769
    80.9030.816
    90.8480.719
    100.8730.762
    110.8230.678
    120.8180.669
    130.7840.615
    140.8810.776

    CFA Results

    An analysis of mean average precision (MAP) showed that the WEMWBS had a 1D structure. The minor average squared partial correlation was 0.02221, and the most negligible average fourth-power partial correlation was 0.00100. According to the revised MAP test [31], the number of factors was 1.

    We conducted a CFA test of the hypothetical single-factor structure of the Chinese WEMWBS and measured the goodness-of-fit of the single confirmatory factor model. Assuming that there was no correlation between the residuals, the initial model fit poorly. The χ2/df was 8.437; the comparative fitting index (CFI) was 0.927; the RMSEA was 0.114; for the normed fit index (NFI), delta 1 was 0.918; for the relative fit index (RFI), rho 1 was 0.903; for the incremental fit index (IFI), delta 2 was 0.927; for and the Tacker-Lewis index (TLI), rho was 2.914.

    Compatible Validity

    There was a significant positive correlation between the Chinese WEMWBS and the WHO-5, with a correlation coefficient of 0.746 (95% CI 0.722-0.794; P<.01).

    Combination Reliability and Convergent Validity

    A CFA showed that the AVE value was 0.674 (ie, greater than 0.5). The CR value was 0.966 (ie, greater than 0.7), suggesting that the sample had good convergence validity.

    IRT Analysis

    Table 5 shows the results of the GRM analysis. The discrimination difference indices of the items ranged from 2.026 to 5.098, which demonstrates that the Chinese WEMWBS scores of low-score individuals differed from high-score individuals, corresponding to latent trait sensitivity. The item difficulty of thresholds ≥2, ≥3, ≥4, and 5 ranged from 1.06 to 1.73, 0 to 0.23, 0.56 to 1.06, and 1.12 to 1.66, respectively.

    Table 5. Results of the graded response model analysis of the Chinese-language version of the Warwick-Edinburgh Mental Well-being Scale.
    ItemCoefficient95% CISEzP>z
    1

    		Discrimination difference2.5262.200-2.8530.16715.160<.001

    		Item difficulty


    		≥21.3941.578-1.2100.09414.84<.001


    		≥30.0810.039-0.2010.0611.32.19


    		≥40.8060.669-0.9430.07011.54<.001


    		51.4551.270-1.6400.09415.42<.001
    2

    		Discrimination difference3.0102.621-3.4000.19915.140<.001

    		Item difficulty


    		≥21.0641.212-0.9160.07514.09<.001


    		≥30.2320.117-0.3480.0593.94<.001


    		≥41.0580.914-1.2030.07414.38<.001


    		51.6561.463-1.8500.09916.78<.001
    3

    		Discrimination difference3.0242.641-3.4070.19515.490<.001

    		Item difficulty


    		≥21.6191.814-1.4240.10016.25<.001


    		≥30.1190.233-0.0040.0582.03.04


    		≥40.5590.439-0.6790.0619.13<.001


    		51.2081.053-1.3640.07915.22<.001
    4

    		Discrimination difference2.0261.756-2.2970.13814.670<.001

    		Item difficulty


    		≥21.4221.625-1.2190.10313.74<.001


    		≥30.1150.245-0.0160.0671.72.09


    		≥40.6430.501-0.7850.0738.87<.001


    		51.4051.207-1.6020.10113.19<.001
    5

    		Discrimination difference3.4953.051-3.9390.22715.420<.001

    		Item difficulty


    		≥21.7261.928-1.5250.10316.79<.001


    		≥30.2200.331-0.1090.0573.88<.001


    		≥40.4690.356-0.5830.0588.10<.001


    		51.1170.974-1.2610.07315.26<.001
    6

    		Discrimination difference4.1103.573-4.6460.27415.020<.001

    		Item difficulty


    		≥21.4801.650-1.3100.08717.06<.001


    		≥30.0010.108-0.1060.0550.01.99


    		≥40.8730.748-0.9970.06313.79<.001


    		51.4591.296-1.6230.08417.45<.001
    7

    		Discrimination difference4.2583.680-4.8350.29514.450<.001

    		Item difficulty


    		≥21.4221.586-1.2580.08417.02<.001


    		≥30.1490.042-0.2550.0542.74.006


    		≥40.8870.762-1.0110.06313.99<.001


    		51.5321.364-1.7010.08617.82<.001
    8

    		Discrimination difference5.0984.400-5.7960.35614.310<.001

    		Item difficulty


    		≥21.4591.622-1.2970.08317.58<.001


    		≥30.0600.164-0.0430.0531.14.25


    		≥40.6820.571-0.7930.05712.00<.001


    		51.2351.094-1.3750.07217.25<.001
    9

    		Discrimination difference3.5713.111-4.0320.23515.200<.001

    		Item difficulty


    		≥21.3341.496-1.1730.08316.17<.001


    		≥30.1130.004-0.2230.0562.02.04


    		≥40.9670.833-1.1000.06814.19<.001


    		51.6381.455-1.8220.09417.47<.001
    10

    		Discrimination difference3.9423.439-4.4460.25715.360<.001

    		Item difficulty


    		≥21.4981.672-1.3230.08916.18<.001


    		≥30.1170.224-0.0090.0552.12.03


    		≥40.6550.539-0.7710.05911.05<.001


    		51.3201.167-1.4730.07816.91<.001
    11

    		Discrimination difference3.1712.762-3.5800.20915.200<.001

    		Item difficulty


    		≥21.3741.544-1.2040.08715.85<.001


    		≥30.1340.021-0.2470.0582.33.02


    		≥41.0110.872-1.1510.07114.17<.001


    		51.5971.411-1.7830.09516.79<.001
    12

    		Discrimination difference2.9622.583-3.3410.19315.320<.001

    		Item difficulty


    		≥21.3731.545-1.2000.08815.59<.001


    		≥30.1180.003-0.2340.0592.01.04


    		≥40.9170.781-1.0530.06913.20<.001


    		51.6391.445-1.8330.09916.56<.001
    13

    		Discrimination difference2.5392.215-2.8640.16615.330<.001

    		Item difficulty


    		≥21.4271.613-1.2410.09515.02<.001


    		≥30.0420.078-0.1620.0610.69.49


    		≥40.7130.579-0.8460.06810.48<.001


    		51.5541.361-1.7470.09915.77<.001
    14

    		Discrimination difference3.9253.411-4.4400.26214.960<.001

    		Item difficulty


    		≥21.4931.666-1.3200.08816.92<.001


    		≥30.0660.041-0.1740.0551.21.23


    		≥40.7850.663-0.9070.06212.64<.001


    		51.4511.287-1.6140.08317.38<.001

    The ICCs and IICs for the Chinese WEMWBS are shown in Figures 2 and 3, respectively. The ICCs demonstrated that the sequence of the categories’ thresholds for the 14 items was as predicted, meaning that all regimentations were sufficient in including respondents; this finding, in turn, suggests that all categories were adequate based on placing a participant on the scale. The IICs displayed multimodal distribution. The shape of item 8 was the most precipitous and provided more knowledge than the other 13 items. The shape of item 4 was the flattest, indicating that the item provided the least information.

    ‎
    Figure 2. Item-category characteristic curves for the Chinese-language version of the WEMWBS. The numbers indicate each item on the scale. WEMWBS: Warwick-Edinburgh Mental Well-being Scale.
    View this figure
    ‎
    Figure 3. The item information curves for items of the Chinese-language version of the WEMWBS. The numbers indicate each item on the scale. WEMWBS: Warwick-Edinburgh Mental Well-being Scale.
    View this figure

    Principal Findings

    This is the first study to combine CTT and a GRM incorporating IRT to evaluate psychometric properties of the Chinese-language version of the WEMWBS in a sample of medical staff. Our results confirm the initial hypothesis that the WEMWBS is 1D. Since its establishment in 2006, the WEMWBS has been used in trials of patients and the general population with commendable results according to CTT and the Rasch model [32,33]. Given the broad and complicated spectrum of psychometric processes other than CTT, each with new evaluations and fixed statistical analyses in diverse models [34], we adopted the GRM to evaluate the contribution of the 14 items and their responses to the assessment of subjective well-being (SWB).

    Comparisons With Previous Studies

    The mean score for the Chinese version of the WEMWBS used in this study was 38.47 (SD 13.23), which is lower than WEMWBS scores in medical staff surveys in other countries (eg, the United Kingdom [35], Pakistan [25,36], and Northern Ireland [37]). This discrepancy may be due to the data having been collected during the outbreak of COVID-19, meaning that the SWB of the medical staff would have been impacted to a certain extent [38]. Moreover, with the aging population of China, medical staff are under a great deal of pressure and need to master multidisciplinary knowledge and skills even as their work intensity increases [39].

    The original 1D structure of the WEMWBS, as confirmed by previous studies in other countries [24,27,29], was not fully supported by earlier research in mainland China. This outcome was expected; some studies [28,40] identified a 2D structure that differed from the original assumption.

    Researchers have pointed to differences between Eastern and Western cultures to explain this: the original meaning of the individual items might be changed in translated versions, and this alteration could affect the perceived intentions of the target population [22]. Furthermore, previous studies [33] adopted the Likert ordinal interval for a comprehensive rating, in which the 14 individual item scores were added to produce a total score. Bartram [35] found that using only a CFA may lead to misunderstanding, because the total score has a serial order, and the intervals between each score are not necessarily equal. The unidimensional structure was not without problems in this study. First, the model fitting effect was insufficient, because the χ2/df was greater than 5, and the RMSEA was greater than 0.08. Only the NFI, TLI, and CFI values supported the unidimensionality of the model. However, the AVE was greater than 0.5, and the CR was greater than 0.8, suggesting a relevant result. Second, the 1D model’s factor loadings for the 14 items were similar to the 2-factor model. Third, considering that the number of factors according to the revised MAP test was 1 [31], we adopted the 1D structure. An exemplary configuration of the Chinese WEMWBS would be favorable for facilitating IRT analyses in the future. Administering the Chinese WEMWBS based on IRT could strengthen its sensitivity and precision, guaranteeing that the items reflect the participants’ SWB levels.

    The proportion of participants selecting the options “sometimes” and “often” was high in this study, suggesting that most respondents had relatively good SWB. To test the accuracy of the results, we examined the 14 items for floor and ceiling effects; we did not find extreme ceiling or floor effects, indicating that the process was reliable. There have been no reports on the distribution of responses to the WEMWBS in mainland China. In addition, the Chinese version of the WEMWBS displayed outstanding reliability, with a Cronbach α of .96, more significant than other studies for Chinese and other language versions [18,19,28,29,41].

    The GRM was the best-match IRT model in this study. No previous studies have used the GRM to evaluate the psychometric properties of the WEMWBS. Our study reinforces the use of IRT models and supports existing studies on the psychometric evaluation of the WEMWBS with IRT methods.

    The GRM analysis demonstrated that the global performance of WEMWBS items was satisfactory. The ICCs showed that the feedback categories of all the items were ordered and that all categories were presumably at the same point on the continuum [42].

    Prospects for Application of the Chinese WEMWBS

    Mental health assessment has drawn increasing attention from the Chinese government. In 2017, the Chinese government released the first guidelines to improve mental health in schools, workplaces, and hospitals. The WEMWBS has proven to be a convenient and valuable psychometric tool for academics, medical professionals, and other prominent stakeholders to measure the SWB of medical staff [43,44]. The Chinese WEMWBS has good reliability and validity with comprehensive and understandable content [15,24,26,45].

    Limitations

    There are some limitations to this study. First, our investigation concentrated on hospitals in Zhejiang and Hunan provinces, and most participants were nurses, suggesting some selection bias. Follow-up research needs a larger sample size that includes therapists, physicians, and surgeons to assess the psychometric properties of the Chinese WEMWBS. Second, the sample size was only 572, which is less than 1000; this may have caused ambiguity in evaluating the IRT model. A larger sample size is needed in future research to confirm our findings. Third, we did not discriminate between medical staff with anxiety or depression when calculating the psychometric properties of the Chinese WEMWBS, which may have caused difficulty in demonstrating the scale’s validity. The performance of the Chinese WEMWBS should be further assessed in distinct staff groups.

    Conclusion

    Detailed provisions were made for the Chinese version of the WEMWBS in this study, and its psychometric properties were evaluated in a group of medical staff. We found that the Chinese WEMWBS has good reliability and validity and that it could be used as a reliable tool to evaluate the SWB of medical staff. It is critical to adopt measures to enable decision-making departments of hospitals to reduce work pressure, improve the SWB of clinical medical staff, improve patient satisfaction, and promote the development of the medical industry in a favorable direction.

    Acknowledgments

    This work was supported by the Wenzhou Science and Technology Bureau, Wenzhou, China (Y2019038), and the Education Department of Zhejiang Province, Zhejiang, China (Y202147054), which had no role in the study design; collection, analysis, or interpretation of the data; writing the manuscript; or the decision to submit the paper for publication. The table of contents image was obtained from the Freepik image bank.

    Authors' Contributions

    All authors provided scientific input and edited and reviewed the manuscript content. All authors provided their final approval and agreed to be accountable for all aspects of the work, ensuring integrity and accuracy. WG was responsible for the manuscript; AD and JH wrote the manuscript; SL completed the data analysis; JZ tabulated the data; other authors collected case materials. All authors read and approved the final manuscript.

    Conflicts of Interest

    None declared.

    1. Diener E, Oishi S, Tay L. Advances in subjective well-being research. Nat Hum Behav 2018 Apr 12;2(4):253-260. [CrossRef] [Medline]
    2. Melnyk BM, Kelly SA, Stephens J, Dhakal K, McGovern C, Tucker S, et al. interventions to improve mental health, well-being, physical health, and lifestyle behaviors in physicians and nurses: a systematic review. Am J Health Promot 2020 Nov 27;34(8):929-941 [FREE Full text] [CrossRef] [Medline]
    3. O'Reilly M. Social media and adolescent mental health: the good, the bad and the ugly. J Ment Health 2020 Apr 28;29(2):200-206. [CrossRef] [Medline]
    4. Sim F, Mackie P. Well-being of the migrant workforce: who cares? Public Health 2018 Jul;160:A1-A2. [CrossRef] [Medline]
    5. Søvold LE, Naslund JA, Kousoulis AA, Saxena S, Qoronfleh MW, Grobler C, et al. Prioritizing the mental health and well-being of healthcare workers: an urgent global public health priority. Front Public Health 2021 May 7;9:679397 [FREE Full text] [CrossRef] [Medline]
    6. Wainberg ML, Scorza P, Shultz JM, Helpman L, Mootz JJ, Johnson KA, et al. Challenges and opportunities in global mental health: a research-to-practice perspective. Curr Psychiatry Rep 2017 May;19(5):28 [FREE Full text] [CrossRef] [Medline]
    7. Salazar de Pablo G, Vaquerizo-Serrano J, Catalan A, Arango C, Moreno C, Ferre F, et al. Impact of coronavirus syndromes on physical and mental health of health care workers: systematic review and meta-analysis. J Affect Disord 2020 Oct 01;275:48-57 [FREE Full text] [CrossRef] [Medline]
    8. Berlanda S, de Cordova F, Fraizzoli M, Pedrazza M. Risk and protective factors of well-being among healthcare staff. A thematic analysis. Int J Environ Res Public Health 2020 Sep 12;17(18):6651 [FREE Full text] [CrossRef] [Medline]
    9. Ruiz-Fernández MD, Pérez-García E, Ortega-Galán ÁM. Quality of life in nursing professionals: burnout, fatigue, and compassion satisfaction. Int J Environ Res Public Health 2020 Feb 15;17(4):1253 [FREE Full text] [CrossRef] [Medline]
    10. Zhou C, Shi L, Gao L, Liu W, Chen Z, Tong X, et al. Determinate factors of mental health status in Chinese medical staff: A cross-sectional study. Medicine (Baltimore) 2018 Mar;97(10):e0113 [FREE Full text] [CrossRef] [Medline]
    11. Shi Y, Xue H, Ma Y, Wang L, Gao T, Shi L, et al. Prevalence of occupational exposure and its influence on job satisfaction among Chinese healthcare workers: a large-sample, cross-sectional study. BMJ Open 2020 Apr 16;10(4):e031953 [FREE Full text] [CrossRef] [Medline]
    12. Ledikwe JH, Kleinman NJ, Mpho M, Mothibedi H, Mawandia S, Semo B, et al. Associations between healthcare worker participation in workplace wellness activities and job satisfaction, occupational stress and burnout: a cross-sectional study in Botswana. BMJ Open 2018 Mar 16;8(3):e018492 [FREE Full text] [CrossRef] [Medline]
    13. Yakes EA, Dean S, Labadie RF, Byrne D, Estrada C, Thompson R, et al. Factors associated with physician empowerment and well-being at an academic medical center. J Occup Environ Med 2020 Jul;62(7):478-483. [CrossRef] [Medline]
    14. Zhou H, Jiang F, Rakofsky J, Hu L, Liu T, Wu S, et al. Job satisfaction and associated factors among psychiatric nurses in tertiary psychiatric hospitals: Results from a nationwide cross-sectional study. J Adv Nurs 2019 Dec;75(12):3619-3630. [CrossRef] [Medline]
    15. Dong A, Chen X, Zhu L, Shi L, Cai Y, Shi B, et al. Translation and validation of a Chinese version of the Warwick-Edinburgh Mental Well-being Scale with undergraduate nursing trainees. J Psychiatr Ment Health Nurs 2016 Nov 18;23(9-10):554-560. [CrossRef] [Medline]
    16. Konaszewski K, Niesiobędzka M, Surzykiewicz J. Factor structure and psychometric properties of a Polish adaptation of the Warwick-Edinburgh Mental Wellbeing Scale. Health Qual Life Outcomes 2021 Mar 02;19(1):70 [FREE Full text] [CrossRef] [Medline]
    17. Lang G, Bachinger A. Validation of the German Warwick-Edinburgh Mental Well-Being Scale (WEMWBS) in a community-based sample of adults in Austria: a bi-factor modelling approach. J Public Health 2016 Dec 3;25(2):135-146. [CrossRef]
    18. Trousselard M, Steiler D, Dutheil F, Claverie D, Canini F, Fenouillet F, et al. Validation of the Warwick-Edinburgh Mental Well-Being Scale (WEMWBS) in French psychiatric and general populations. Psychiatry Res 2016 Nov 30;245:282-290. [CrossRef] [Medline]
    19. Tennant R, Hiller L, Fishwick R, Platt S, Joseph S, Weich S, et al. The Warwick-Edinburgh Mental Well-being Scale (WEMWBS): development and UK validation. Health Qual Life Outcomes 2007 Nov 27;5:63 [FREE Full text] [CrossRef] [Medline]
    20. Liddell C, Guiney C. Living in a cold and damp home: frameworks for understanding impacts on mental well-being. Public Health 2015 Mar;129(3):191-199. [CrossRef] [Medline]
    21. Tamminen N, Solin P, Stengård E, Kannas L, Kettunen T. Mental health promotion competencies in the health sector in Finland: a qualitative study of the views of professionals. Scand J Public Health 2019 Mar 12;47(2):115-120. [CrossRef] [Medline]
    22. Dong A, Zhang X, Zhou H, Chen S, Zhao W, Wu M, et al. Applicability and cross-cultural validation of the Chinese version of the Warwick-Edinburgh mental well-being scale in patients with chronic heart failure. Health Qual Life Outcomes 2019 Mar 29;17(1):55 [FREE Full text] [CrossRef] [Medline]
    23. Liu Y, Guo L, Liu K. Validity and reliability of Warwick-Edinburgh Mental Well-being Scale (WEMWBS) in older people. Chin Ment Health J 2016;30(3):174-178. [CrossRef]
    24. Fung S. Psychometric evaluation of the Warwick-Edinburgh Mental Well-being Scale (WEMWBS) with Chinese university students. Health Qual Life Outcomes 2019 Mar 14;17(1):46 [FREE Full text] [CrossRef] [Medline]
    25. Waqas A, Ahmad W, Haddad M, Taggart FM, Muhammad Z, Bukhari MH, et al. Measuring the well-being of health care professionals in the Punjab: a psychometric evaluation of the Warwick-Edinburgh Mental Well-being Scale in a Pakistani population. PeerJ 2015;3:e1264 [FREE Full text] [CrossRef] [Medline]
    26. Taggart F, Friede T, Weich S, Clarke A, Johnson M, Stewart-Brown S. Cross cultural evaluation of the Warwick-Edinburgh Mental Well-being Scale (WEMWBS) --a mixed methods study. Health Qual Life Outcomes 2013 Feb 27;11:27 [FREE Full text] [CrossRef] [Medline]
    27. Bass M, Dawkin M, Muncer S, Vigurs S, Bostock J. Validation of Warwick-Edinburgh Mental Well-being Scale (WEMWBS) in a population of people using Secondary Care Mental Health Services. J Ment Health 2016 Aug 28;25(4):323-329. [CrossRef] [Medline]
    28. Clarke A, Friede T, Putz R, Ashdown J, Martin S, Blake A, et al. Warwick-Edinburgh Mental Well-being Scale (WEMWBS): validated for teenage school students in England and Scotland. A mixed methods assessment. BMC Public Health 2011 Jun 21;11:487 [FREE Full text] [CrossRef] [Medline]
    29. Smith ORF, Alves DE, Knapstad M, Haug E, Aarø LE. Measuring mental well-being in Norway: validation of the Warwick-Edinburgh Mental Well-being Scale (WEMWBS). BMC Psychiatry 2017 May 12;17(1):182 [FREE Full text] [CrossRef] [Medline]
    30. Yamada A, Kasahara K, Ogawa Y, Samejima K, Eriguchi M, Yano H, et al. Peritonitis due to Moraxella osloensis: A case report and literature review. J Infect Chemother 2019 Dec;25(12):1050-1052. [CrossRef] [Medline]
    31. Velicer WF, Eaton CA, Fava JL. Construct explication through factor or component analysis: a review and evaluation of alternative procedures for determining the number of factors or components. In: Goffin RD, Helmes E, editors. Problem and Solutions in Human Assessment. Boston, MA: Springer; 2000.
    32. Khumalo IP, Temane QM, Wissing MP. Further validation of the General Psychological Well-being Scale among a Setswana-speaking group. In: Wissing M, editor. Well-Being Research in South Africa. Dordrecht, Netherlands: Springer; 2013:199-224.
    33. Stewart-Brown S, Tennant A, Tennant R, Platt S, Parkinson J, Weich S. Internal construct validity of the Warwick-Edinburgh Mental Well-being Scale (WEMWBS): a Rasch analysis using data from the Scottish Health Education Population Survey. Health Qual Life Outcomes 2009 Feb 19;7:15 [FREE Full text] [CrossRef] [Medline]
    34. Bilder RM, Reise SP. Neuropsychological tests of the future: How do we get there from here? Clin Neuropsychol 2019 Feb 13;33(2):220-245 [FREE Full text] [CrossRef] [Medline]
    35. Bartram DJ, Sinclair JM, Baldwin DS. Further validation of the Warwick-Edinburgh Mental Well-being Scale (WEMWBS) in the UK veterinary profession: Rasch analysis. Qual Life Res 2013 Mar 2;22(2):379-391. [CrossRef] [Medline]
    36. Ahmad W, Taggart F, Shafique MS, Muzafar Y, Abidi S, Ghani N, et al. Diet, exercise and mental-wellbeing of healthcare professionals (doctors, dentists and nurses) in Pakistan. PeerJ 2015;3:e1250 [FREE Full text] [CrossRef] [Medline]
    37. Murray MA, Cardwell C, Donnelly M. GPs’ mental wellbeing and psychological resources: a cross-sectional survey. Br J Gen Pract 2017 Jul 17;67(661):e547-e554. [CrossRef]
    38. Hu Z, Lin X, Chiwanda Kaminga A, Xu H. Impact of the COVID-19 epidemic on lifestyle behaviors and their association with subjective well-being among the general population in mainland China: cross-sectional study. J Med Internet Res 2020 Aug 25;22(8):e21176 [FREE Full text] [CrossRef] [Medline]
    39. Du J, Mayer G, Hummel S, Oetjen N, Gronewold N, Zafar A, et al. Mental health burden in different professions during the final stage of the COVID-19 lockdown in China: cross-sectional survey study. J Med Internet Res 2020 Dec 02;22(12):e24240 [FREE Full text] [CrossRef] [Medline]
    40. López MA, Gabilondo A, Codony M, García-Forero C, Vilagut G, Castellví P, et al. Adaptation into Spanish of the Warwick-Edinburgh Mental Well-being Scale (WEMWBS) and preliminary validation in a student sample. Qual Life Res 2013 Jun 27;22(5):1099-1104. [CrossRef] [Medline]
    41. Mavali S, Mahmoodi H, Sarbakhsh P, Shaghaghi A. Psychometric properties of the Warwick-Edinburgh Mental Wellbeing Scale (WEMWBS) in the Iranian older adults. Psychol Res Behav Manag 2020;13:693-700 [FREE Full text] [CrossRef] [Medline]
    42. Chorpita BF, Daleiden EL, Weisz JR. Identifying and selecting the common elements of evidence based interventions: a distillation and matching model. Ment Health Serv Res 2005 Mar;7(1):5-20. [CrossRef] [Medline]
    43. Adamou M, Goddard A, Kyriakidou N, Mooney A, O’Donoghue D, Pattani S, et al. The wellbeing thermometer: a novel framework for measuring wellbeing. Psychology 2020;11(10):1471-1480 [FREE Full text] [CrossRef]
    44. Oates J, Jones J, Drey N. Subjective well-being of mental health nurses in the United Kingdom: results of an online survey. Int J Ment Health Nurs 2017 Aug 23;26(4):391-401. [CrossRef] [Medline]
    45. Ng SSW, Lo AWY, Leung TKS, Chan FSM, Wong ATY, Lam RWT, et al. Translation and validation of the Chinese version of the short Warwick-Edinburgh Mental Well-being Scale for patients with mental illness in Hong Kong. East Asian Arch Psychiatry 2014 Mar;24(1):3-9 [FREE Full text] [Medline]

    ‎
    AVE: average variance extracted
    CFA: confirmatory factor analysis
    CFI: comparative fitting index
    CR: composite reliability
    CTT: classical test theory
    GRM: graded response model
    ICC: item characteristic curve
    IFI: incremental fit index
    IIC: item information curve
    IRT: item response theory
    MAP: mean average precision
    NFI: normed fit index
    QoL: quality of life
    RFI: relative fit index
    RMSEA: root mean square error of approximation
    SF-36 v2: 36-Item Short Form Health Survey, Version 2
    SWB: subjective well-being
    TLI: Tacker-Lewis index
    WEMWBS: Warwick-Edinburgh Mental Well-being Scale
    WHO-5: 5-item World Health Organization Well-Being Index

    Edited by R Kukafka; submitted 20.03.22; peer-reviewed by S Stewart-Brown, A Joseph, S Pesälä; comments to author 25.05.22; revised version received 07.07.22; accepted 01.11.22; published 30.11.22

    Copyright

    ©Aishu Dong, Jing Huang, Shudan Lin, Jianing Zhu, Haitao Zhou, Qianqian Jin, Wei Zhao, Lianlian Zhu, Wenjian Guo. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 30.11.2022.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.