Letter to the Editor
Comment in: https://www.jmir.org/2021/5/e26543
This is a corrected version. See correction statement in: https://www.jmir.org/2021/6/e31268/
We congratulate Munsch et al  on their recent publication, “Diagnostic Accuracy of Web-Based COVID-19 Symptom Checkers: Comparison Study.” The study investigated the relative performance of web-based COVID-19 screening tools between April 3 and 9, 2020. It is important to have literature comparing the relative performance of tools; however, it is also important that the approaches used in reports of this type compare “like with like” and are reported rigorously.
There are a number of important limitations in Munsch et al’s  paper, the first being that the principal results reported are not fully addressed in the publication, and the second being it is an inappropriate evaluation of COVID-19 screeners designed for use by laypersons at home. COVID-19 screeners were brought into the market at a time of need to lower the burden on health authorities and health care institutions [ ]. Overall, they were meant for use at home by people who were either worried about their symptoms or who were anxious about the pandemic in general and in need of reassurance. These people could either be put at ease that their symptoms were likely not caused by COVID-19, or be provided with appropriate localized advice through screeners tailored to their region.
The paper  does not investigate the COVID-19 screeners according to their intended purpose and compares tools with fundamentally different intended purposes. The Symptoma tool, developed by the paper’s authors, was designed to use the professional interpretation of both patients’ symptoms and the results of diagnostic tests, such as interpreted CT (computed tomography) images (eg, ground-glass opacities); both provide a predictive likelihood of COVID-19 status. All the other screeners examined in the study were designed for use at home by layperson users, providing a first line of advice to those experiencing possible COVID-19 symptoms. Importantly, none of these tools provide a diagnosis, neither their design concept nor regulatory approval includes this, and testing their diagnostic accuracy is therefore testing something they are specifically labeled as not providing. A meaningful evaluation would be to evaluate the appropriateness of the advice given according to the use context of these screeners (ie, do the screeners provide appropriate information and advice to users based on their symptoms, risk factors, exposure, and their location [including local COVID-19 status and national guidelines]?).
The ability of Symptoma to make use of the professional interpretation of clinical findings might be useful in some hospital settings as a clinical decision support tool, but COVID-19 tests are available in this setting, and at this stage in the patient journey, the tool cannot lessen the burden on the health care system. Based on the arguments above, it is critical to interpret the results in Multimedia Appendices 8-12 , which show that Symptoma is not superior in specificity and sensitivity as claimed in the main manuscript, and to remember that the appropriateness of specific advice in a given situation is more important to the user than either specificity or sensitivity.
Conflicts of Interest
All authors are employees of Ada Health GmbH.
- Munsch N, Martin A, Gruarin S, Nateqi J, Abdarahmane I, Weingartner-Ortner R, et al. Diagnostic Accuracy of Web-Based COVID-19 Symptom Checkers: Comparison Study. J Med Internet Res 2020 Oct 06;22(10):e21299 [FREE Full text] [CrossRef] [Medline]
- Ross C. Symptom checker steers Covid-19 patients to care. STAT. 2021 Apr 21. URL: https://www.statnews.com/2020/04/21/coronavirus-symptom-checker-steers-patients-to-care-far-away/ [accessed 2020-07-22]
|CT: computed tomography|
Edited by T Derrick; submitted 30.11.20; peer-reviewed by B Knapp; accepted 13.05.21; published 21.05.21Copyright
©Elizabeth Millen, Andreas Gilsdorf, Matthew Fenech, Stephen Gilbert. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 21.05.2021.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.