TY  - JOUR
AU  - Zeng, Minrui
AU  - Cai, Yiyuan
AU  - Cao, Jin
AU  - He, Qianyu
AU  - Wang, Xiaohui
AU  - Lu, Yun
AU  - Liang, Huijuan
AU  - Xu, Dong
AU  - Liao, Jing
PY  - 2022
DA  - 2022/12/2
TI  - The Agreement Between Virtual Patient and Unannounced Standardized Patient Assessments in Evaluating Primary Health Care Quality: Multicenter, Cross-sectional Pilot Study in 7 Provinces of China
JO  - J Med Internet Res
SP  - e40082
VL  - 24
IS  - 12
KW  - virtual patient
KW  - unannounced standardized patient
KW  - primary health care
KW  - primary care
KW  - quality assessment
KW  - quality improvement
KW  - scenario
KW  - simulation
KW  - simulate
KW  - medical education
KW  - cross-sectional
KW  - digital health
KW  - eHealth
AB  - Background: The unannounced standardized patient (USP) is the gold standard for primary health care (PHC) quality assessment but has many restrictions associated with high human and resource costs. Virtual patient (VP) is a valid, low-cost software option for simulating clinical scenarios and is widely used in medical education. It is unclear whether VP can be used to assess the quality of PHC. Objective: This study aimed to examine the agreement between VP and USP assessments of PHC quality and to identify factors influencing the VP-USP agreement. Methods: Eleven matched VP and USP case designs were developed based on clinical guidelines and were implemented in a convenience sample of urban PHC facilities in the capital cities of the 7 study provinces. A total of 720 USP visits were conducted, during which on-duty PHC providers who met the inclusion criteria were randomly selected by the USPs. The same providers underwent a VP assessment using the same case condition at least a week later. The VP-USP agreement was measured by the concordance correlation coefficient (CCC) for continuity scores and the weighted κ for diagnoses. Multiple linear regression was used to identify factors influencing the VP-USP agreement. Results: Only 146 VP scores were matched with the corresponding USP scores. The CCC for medical history was 0.37 (95% CI 0.24-0.49); for physical examination, 0.27 (95% CI 0.12-0.42); for laboratory and imaging tests, –0.03 (95% CI –0.20 to 0.14); and for treatment, 0.22 (95% CI 0.07-0.37). The weighted κ for diagnosis was 0.32 (95% CI 0.13-0.52). The multiple linear regression model indicated that the VP tests were significantly influenced by the different case conditions and the city where the test took place. Conclusions: There was low agreement between VPs and USPs in PHC quality assessment. This may reflect the “know-do” gap. VP test results were also influenced by different case conditions, interactive design, and usability. Modifications to VPs and the reasons for the low VP-USP agreement require further study. 
SN  - 1438-8871
UR  - https://www.jmir.org/2022/12/e40082
UR  - https://doi.org/10.2196/40082
UR  - http://www.ncbi.nlm.nih.gov/pubmed/36459416
DO  - 10.2196/40082
ID  - info:doi/10.2196/40082
ER  -