Determinants of Laypersons’ Trust in Medical Decision Aids: Randomized Controlled Trial

Background Symptom checker apps are patient-facing decision support systems aimed at providing advice to laypersons on whether, where, and how to seek health care (disposition advice). Such advice can improve laypersons’ self-assessment and ultimately improve medical outcomes. Past research has mainly focused on the accuracy of symptom checker apps’ suggestions. To support decision-making, such apps need to provide not only accurate but also trustworthy advice. To date, only few studies have addressed the question of the extent to which laypersons trust symptom checker app advice or the factors that moderate their trust. Studies on general decision support systems have shown that framing automated systems (anthropomorphic or emphasizing expertise), for example, by using icons symbolizing artificial intelligence (AI), affects users’ trust. Objective This study aims to identify the factors influencing laypersons’ trust in the advice provided by symptom checker apps. Primarily, we investigated whether designs using anthropomorphic framing or framing the app as an AI increases users’ trust compared with no such framing. Methods Through a web-based survey, we recruited 494 US residents with no professional medical training. The participants had to first appraise the urgency of a fictitious patient description (case vignette). Subsequently, a decision aid (mock symptom checker app) provided disposition advice contradicting the participants’ appraisal, and they had to subsequently reappraise the vignette. Participants were randomized into 3 groups: 2 experimental groups using visual framing (anthropomorphic, 160/494, 32.4%, vs AI, 161/494, 32.6%) and a neutral group without such framing (173/494, 35%). Results Most participants (384/494, 77.7%) followed the decision aid’s advice, regardless of its urgency level. Neither anthropomorphic framing (odds ratio 1.120, 95% CI 0.664-1.897) nor framing as AI (odds ratio 0.942, 95% CI 0.565-1.570) increased behavioral or subjective trust (P=.99) compared with the no-frame condition. Even participants who were extremely certain in their own decisions (ie, 100% certain) commonly changed it in favor of the symptom checker’s advice (19/34, 56%). Propensity to trust and eHealth literacy were associated with increased subjective trust in the symptom checker (propensity to trust b=0.25; eHealth literacy b=0.2), whereas sociodemographic variables showed no such link with either subjective or behavioral trust. Conclusions Contrary to our expectation, neither the anthropomorphic framing nor the emphasis on AI increased trust in symptom checker advice compared with that of a neutral control condition. However, independent of the interface, most participants trusted the mock app’s advice, even when they were very certain of their own assessment. Thus, the question arises as to whether laypersons use such symptom checkers as substitutes rather than as aids in their own decision-making. With trust in symptom checkers already high at baseline, the benefit of symptom checkers depends on interface designs that enable users to adequately calibrate their trust levels during usage. Trial Registration Deutsches Register Klinischer Studien DRKS00028561; https://tinyurl.com/rv4utcfb (retrospectively registered).

We included "Experimental Study" in the title, because -although it does involv 1a-i) Identify the mode of delivery in the title Identify the mode of delivery. Preferably use "web-based" and/or "mobile" and/or "electronic game" in the title. Avoid ambiguous terms like "online", "virtual", "interactive". Use "Internet-based" only if Intervention includes non-web-based Internet components (e.g. email), use "computer-based" or "electronic" only if offline products are used. Use "virtual" only in the context of "virtual reality" (3-D worlds). Use "online" only in the context of "online support groups". Complement or substitute product names with broader terms for the class of products (such as "mobile" or "smart phone" instead of "iphone"), especially if the application runs on different platforms.
Does your paper address subitem 1a-i? * Copy and paste relevant sections from manuscript title (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "Experimental study" was used subitem not at all important 1 2 3 4 5 essential subitem not at all important 1 2 3 4 5 essential 1a-ii) Non-web-based components or important co-interventions in title Mention non-web-based components or important co-interventions in title, if any (e.g., "with telephone support").
Does your paper address subitem 1a-ii?
Copy and paste relevant sections from manuscript title (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study not applicable, because this experiment was conducted solely web-based 1a-iii) Primary condition or target group in the title Mention primary condition or target group in the title, if any (e.g., "for children with Type I Diabetes") Example: A Web-based and Mobile Intervention with Telephone Support for Children with Type I Diabetes: Randomized Controlled Trial Does your paper address subitem 1a-iii? * Copy and paste relevant sections from manuscript title (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study We included "Laypersons" in the title 1b) ABSTRACT: Structured summary of trial design, methods, results, and conclusions NPT extension: Description of experimental treatment, comparator, care providers, centers, and blinding status.
subitem not at all important Does your paper address subitem 1b-i? * Copy and paste relevant sections from the manuscript abstract (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "Symptom checker apps are patient-facing decision support systems aimed at providing advice to laypersons on whether, where and how to seek healthcare (disposition advice)", "Subsequently, a decision aid (mock symptom checker app) provided disposition advice" & "two experimental groups using a visual framing (anthropomorphic (n = 160) vs. AI (n = 161)) and a neutral one without such framing (n = 173)."

1b-ii) Level of human involvement in the METHODS section of the ABSTRACT
Clarify the level of human involvement in the abstract, e.g., use phrases like "fully automated" vs. "therapist/nurse/care provider/physician-assisted" (mention number and expertise of providers involved, if any). (Note: Only report in the abstract what the main paper is reporting. If this information is missing from the main body of text, consider adding it) subitem not at all important 1 2 3 4 5 essential Does your paper address subitem 1b-ii?
Copy and paste relevant sections from the manuscript abstract (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Not applicable, because we did not directly evaluate a system, but presented participants a symptom checker result 1b-iii) Open vs. closed, web-based (self-assessment) vs. face-to-face assessments in the METHODS section of the ABSTRACT Mention how participants were recruited (online vs. offline), e.g., from an open access website or from a clinic or a closed online user group (closed usergroup trial), and clarify if this was a purely web-based trial, or there were face-to-face components (as part of the intervention or for assessment). Clearly say if outcomes were self-assessed through questionnaires (as common in web-based trials). Note: In traditional offline trials, an open trial (open-label trial) is a type of clinical trial in which both the researchers and participants know which treatment is being administered. To avoid confusion, use "blinded" or "unblinded" to indicated the level of blinding instead of "open", as "open" in web-based trials usually refers to "open access" (i.e. participants can self-enrol). (Note: Only report in the abstract what the main paper is reporting. If this information is missing from the main body of text, consider adding it) Does your paper address subitem 1b-iii?
Copy and paste relevant sections from the manuscript abstract (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "Through an online survey we recruited N = 494 US residents with no professional medical training from the recruitment platform Prolific." subitem not at all important 1 2 3 4 5 essential subitem not at all important 1 2 3 4 5 essential 1b-iv) RESULTS section in abstract must contain use data Report number of participants enrolled/assessed in each group, the use/uptake of the intervention (e.g., attrition/adherence metrics, use over time, number of logins etc.), in addition to primary/secondary outcomes. (Note: Only report in the abstract what the main paper is reporting. If this information is missing from the main body of text, consider adding it) Does your paper address subitem 1b-iv?
Copy and paste relevant sections from the manuscript abstract (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study

INTRODUCTION 2a) In INTRODUCTION: Scientific background and explanation of rationale
subitem not at all important 1 2 3 4 5 essential Does your paper address subitem 1b-v?
Copy and paste relevant sections from the manuscript abstract (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "Contrary to our expectation, neither the anthropomorphic framing nor the emphasis of artificial intelligence increased trust in symptom checker advice compared to a neutral control condition. However, independently of the interface, most participants trusted the mock app's advice, even when they were very certain of their own assessment. Thus, the question arises whether laypersons use such symptom checkers as substitute rather than as aid in their own decision making. This question should be explored in further research that examines real user health problems and a wider range of clinical scenarios. With trust in symptom checkers already high at baseline, the benefit of symptom checkers depends on interface designs that enable users to adequately calibrate their trust levels over time."

2a-i) Problem and the type of system/solution
Describe the problem and the type of system/solution that is object of the study: intended as stand-alone intervention vs. incorporated in broader health care program? Intended for a particular patient population? Goals of the intervention, e.g., being more cost-effective to other interventions, replace or complement other solutions? (Note: Details about the intervention are provided in " Methods" under 5) subitem not at all important 1 2 3 4 5 essential 2b) In INTRODUCTION: Specific objectives or hypotheses Does your paper address subitem 2a-i? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "As an alternative to commercial search engines, patient-facing decision support systems called symptom checkers (SC) were developed to provide a first access point to healthrelated information", "The latter assessment, the so-called disposition or urgency advice, arguably is the more important function of SCs, as it could prevent unnecessary visits and direct patients towards the appropriate healthcare facility, thus reducing the burden on the healthcare system." 2a-ii) Scientific background, rationale: What is known about the (type of) system Does your paper address subitem 2a-ii? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "The aim of this study was to examine influences of framing effects on subjective trust in SCs and the behavioral consequences of trust (i.e., dependence/following behavior), which are strongly related." METHODS 3a) Description of trial design (such as parallel, factorial) including allocation ratio 3b) Important changes to methods after trial commencement (such as eligibility criteria), with reasons Does your paper address CONSORT subitem 2b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "We hypothesized that anthropomorphism increases participants' subjective trust in the app and the proportion of participants following the app's advice (behavioral trust). We expected the same effect (higher subjective and behavioral trust) when framing the SC as an AI." Does your paper address CONSORT subitem 3a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "We used a one-factorial experimental design with the factor framing and factor levels anthropomorphic framing and framing as AI along with a control group (allocation ratio 1:1:1)." Does your paper address CONSORT subitem 3b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study We did not make any changes after trial commencement subitem not at all important 1 2 3 4 5 essential 4a) Eligibility criteria for participants 3b-i) Bug fixes, Downtimes, Content Changes Bug fixes, Downtimes, Content Changes: ehealth systems are often dynamic systems. A description of changes to methods therefore also includes important changes made on the intervention or comparator during the trial (e.g., major bug fixes or changes in the functionality or content) (5-iii) and other "unexpected events" that may have influenced study design such as staff changes, system failures/downtimes, etc. [2].
Does your paper address subitem 3b-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study There were no bugs or downtimes Does your paper address CONSORT subitem 4a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "To avoid participants' decision to be influenced by their residential country or ability to understand the scenario, only computer literate US residents fluent in English were eligible.", "We sampled participants using Prolific.", "On the first page, participants were told about the investigator, the study's purpose, what data were to be collected during the study, and where and for how long we would store them. On the second page, participants were informed about the duration of the survey (ca. 5 minutes) and received additional information about the scope and use of attention checks." subitem not at all important 1 2 3 4 5 essential subitem not at all important 1 2 3 4 5 essential 4a-i) Computer / Internet literacy Computer / Internet literacy is often an implicit "de facto" eligibility criterion -this should be explicitly clarified.
Does your paper address subitem 4a-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "To avoid participants' decision to be influenced by their residential country or ability to understand the scenario, only computer literate US residents fluent in English were eligible." 4a-ii) Open vs. closed, web-based vs. face-to-face assessments: Open vs. closed, web-based vs. face-to-face assessments: Mention how participants were recruited (online vs. offline), e.g., from an open access website or from a clinic, and clarify if this was a purely webbased trial, or there were face-to-face components (as part of the intervention or for assessment), i.e., to what degree got the study team to know the participant. In online-only trials, clarify if participants were quasi-anonymous and whether having multiple identities was possible or whether technical or logistical measures (e.g., cookies, email confirmation, phone calls) were used to detect/prevent these.
Does your paper address subitem 4a-ii? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "We sampled participants using Prolific." subitem not at all important 1 2 3 4 5 essential 4b) Settings and locations where the data were collected 4a-iii) Information giving during recruitment Information given during recruitment. Specify how participants were briefed for recruitment and in the informed consent procedures (e.g., publish the informed consent documentation as appendix, see also item X26), as this information may have an effect on user self-selection, user expectation and may also bias results.
Does your paper address subitem 4a-iii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "On the first page, participants were told about the investigator, the study's purpose, what data were to be collected during the study, and where and for how long we would store them. On the second page, participants were informed about the duration of the survey (ca. 5 minutes) and received additional information about the scope and use of attention checks." Does your paper address CONSORT subitem 4b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Participants reported their subjective trust using a questionnaire. Their behavioral trust was coded based on their stated behavioral intentions. In the study information part, affiliations were shown to participants in a Contact section.
subitem not at all important 1 2 3 4 5 essential subitem not at all important 1 2 3 4 5 essential 4b-i) Report if outcomes were (self-)assessed through online questionnaires Clearly report if outcomes were (self-)assessed through online questionnaires (as common in web-based trials) or otherwise.
Does your paper address subitem 4b-i? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Participants reported their subjective trust using a questionnaire. Their behavioral trust was coded based on their stated behavioral intentions.

4b-ii) Report how institutional affiliations are displayed
Report how institutional affiliations are displayed to potential participants [on ehealth media], as affiliations with prestigious hospitals or universities may affect volunteer rates, use, and reactions with regards to an intervention.(Not a required item -describe only if this may bias results)

Does your paper address subitem 4b-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study In the study information part, affiliations were shown to participants in a Contact section.
5) The interventions for each group with sufficient details to allow replication, including how and when they were actually administered

5-i) Mention names, credential, affiliations of the developers, sponsors, and owners
Mention names, credential, affiliations of the developers, sponsors, and owners [6] (if authors/evaluators are owners or developer of the software, this needs to be declared in a "Conflict of interest" section or mentioned elsewhere in the manuscript).

Does your paper address subitem 5-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "The project was funded by the home institutions of the last authors (MF, FB). No external funding was required."

5-ii) Describe the history/development process
Describe the history/development process of the application and previous formative evaluations (e.g., focus groups, usability testing), as these will have an impact on adoption/use rates and help with interpreting results.
subitem not at all important 1 2 3 4 5 essential subitem not at all important 1 2 3 4 5 essential Does your paper address subitem 5-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study

5-iii) Revisions and updating
Revisions and updating. Clearly mention the date and/or version number of the application/intervention (and comparator, if applicable) evaluated, or describe whether the intervention underwent major changes during the evaluation process, or whether the development and/or content was "frozen" during the trial.
Describe dynamic components such as news feeds or changing content which may have an impact on the replicability of the intervention (for unexpected events see item 3b).

Does your paper address subitem 5-iii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Not applicable, because no changes were made

5-iv) Quality assurance methods
Provide information on quality assurance methods to ensure accuracy and quality of information subitem not at all important 1 2 3 4 5 essential Does your paper address subitem 5-iv?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Diagnostic information and the correct solution were taken from a previous study (determined by an expert panel)

5-v)
Ensure replicability by publishing the source code, and/or providing screenshots/screen-capture video, and/or providing flowcharts of the algorithms used Ensure replicability by publishing the source code, and/or providing screenshots/screen-capture video, and/or providing flowcharts of the algorithms used. Replicability (i.e., other researchers should in principle be able to replicate the study) is a hallmark of scientific reporting.

Does your paper address subitem 5-v?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study

5-vi) Digital preservation
Digital preservation: Provide the URL of the application, but as the intervention is likely to change or disappear over the course of the years; also make sure the intervention is archived (Internet Archive, webcitation.org, and/or publishing the source code or screenshots/videos alongside the article). As pages behind login screens cannot be archived, consider creating demo pages which are accessible without login.

Does your paper address subitem 5-vi?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Not applicable, because we only used a results screen which can be found in the paper

5-vii) Access
Access: Describe how participants accessed the application, in what setting/context, if they had to pay (or were paid) or not, whether they had to be a member of specific group. If known, describe how participants obtained "access to the platform and Internet" [1]. To ensure access for editors/reviewers/readers, consider to provide a "backdoor" login account or demo mode for reviewers/readers to explore the application (also important for archiving purposes, see vi).

Does your paper address subitem 5-vii? *
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study " Does your paper address subitem 5-viii? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study The results screen was embedded in an online survey and gave participants advice contradicting their initial appraisal. The exact wording and design was based on a symbiosis of commonly used symptom checker apps.

5-ix) Describe use parameters
Describe use parameters (e.g., intended "doses" and optimal timing for use). Clarify what instructions or recommendations were given to the user, e.g., regarding timing, frequency, heaviness of use, if any, or was the intervention used ad libitum.
subitem not at all important 1 2 3 4 5 essential subitem not at all important 1 2 3 4 5 essential Does your paper address subitem 5-ix?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Not applicable, because the interfaces were shown once only

5-x) Clarify the level of human involvement
Clarify the level of human involvement (care providers or health professionals, also technical assistance) in the e-intervention or as co-intervention (detail number and expertise of professionals involved, if any, as well as "type of assistance offered, the timing and frequency of the support, how it is initiated, and the medium by which the assistance is delivered". It may be necessary to distinguish between the level of human involvement required for the trial, and the level of human involvement required for a routine application outside of a RCT setting (discuss under item 21 -generalizability).
Does your paper address subitem 5-x?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study No human involvement, because participants could not interact with the results screen

5-xi) Report any prompts/reminders used
Report any prompts/reminders used: Clarify if there were prompts (letters, emails, phone calls, SMS) to use the application, what triggered them, frequency etc. It may be necessary to distinguish between the level of prompts/reminders required for the trial, and the level of prompts/reminders for a routine application outside of a RCT setting (discuss under item 21 -generalizability).
subitem not at all important 1 2 3 4 5 essential 6a) Completely defined pre-specified primary and secondary outcome measures, including how and when they were assessed Does your paper address subitem 5-xi? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study There were no prompts or reminders 5-xii) Describe any co-interventions (incl. training/support) Describe any co-interventions (incl. training/support): Clearly state any interventions that are provided in addition to the targeted eHealth intervention, as ehealth intervention may not be designed as stand-alone intervention. This includes training sessions and support [1]. It may be necessary to distinguish between the level of training required for the trial, and the level of training for a routine application outside of a RCT setting (discuss under item 21 -generalizability. Does your paper address subitem 5-xii? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study There were no co-interventions subitem not at all important 1 2 3 4 5 essential Does your paper address CONSORT subitem 6a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Subjective trust (measured using the Trust in Automated Systems Survey) was the primary outcome and behavioral trust (measured by assessing whether participants changed their initial appraisal) the secondary outcome.
6a-i) Online questionnaires: describe if they were validated for online use and apply CHERRIES items to describe how the questionnaires were designed/deployed If outcomes were obtained through online questionnaires, describe if they were validated for online use and apply CHERRIES items to describe how the questionnaires were designed/deployed [9].
Does your paper address subitem 6a-i?

Copy and paste relevant sections from manuscript text
We used the validated Propensity to Trust in Technology Scale, the eHealth Literacy Scale (eHEALS) and the Trust in Automated Systems Survey. Only the eHEALS was specifically validated for online use.
subitem not at all important 1 2 3 4 5 essential subitem not at all important 1 2 3 4 5 essential 6b) Any changes to trial outcomes after the trial commenced, with reasons 6a-ii) Describe whether and how "use" (including intensity of use/dosage) was defined/measured/monitored Describe whether and how "use" (including intensity of use/dosage) was defined/measured/monitored (logins, logfile analysis, etc.). Use/adoption metrics are important process outcomes that should be reported in any ehealth trial.
Does your paper address subitem 6a-ii?
Copy and paste relevant sections from manuscript text Not applicable, because participants were shown the decision aid automatically 6a-iii) Describe whether, how, and when qualitative feedback from participants was obtained Describe whether, how, and when qualitative feedback from participants was obtained (e.g., through emails, feedback forms, interviews, focus groups).

Does your paper address subitem 6a-iii?
Copy and paste relevant sections from manuscript text Our study included an open text field where participants could enter any questions or feedback. However, we did not include qualitative data in our analysis.
7a) How sample size was determined NPT: When applicable, details of whether and how the clustering by care provides or centers was addressed subitem not at all important 1 2 3 4 5 essential 7b) When applicable, explanation of any interim analyses and stopping guidelines Does your paper address CONSORT subitem 6b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study We did not make any changes after trial commencement.

7a-i) Describe whether and how expected attrition was taken into account when calculating the sample size
Describe whether and how expected attrition was taken into account when calculating the sample size.
Does your paper address subitem 7a-i?
Copy and paste relevant sections from manuscript title (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "Yee et al. [36] found the effect size for showing a human in an interface on subjective trust measures to be d = 0.28. Based on an a priori power analysis for independent t-tests with an assumed α = .05 and a power of 1-β = .80, we aimed to sample at least N = 477 (n1 = 159, n2 = 159, n3 = 159) participants to detect difference between three groups (two experimental groups and one control group). We expected some participants to fail attention checks (items that were embedded in the survey questions and asked participants to select a particular option, eg, "Please select Disagree"), so we oversampled by 10%." 8a) Method used to generate the random allocation sequence NPT: When applicable, how care providers were allocated to each trial group 8b) Type of randomisation; details of any restriction (such as blocking and block size) 9) Mechanism used to implement the random allocation sequence (such as sequentially numbered containers), describing any steps taken to conceal the sequence until interventions were assigned Does your paper address CONSORT subitem 7b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study We stopped data collection once we reached our desired sample size.
Does your paper address CONSORT subitem 8a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "Participants were automatically randomly assigned (simple randomization) to one of these levels using the randomization tool integrated in Unipark EFS Survey." Does your paper address CONSORT subitem 8b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "Participants were automatically randomly assigned (simple randomization) to one of these levels using the randomization tool integrated in Unipark EFS Survey." 10) Who generated the random allocation sequence, who enrolled participants, and who assigned participants to interventions 11a) If done, who was blinded after assignment to interventions (for example, participants, care providers, those assessing outcomes) and how NPT: Whether or not administering co-interventions were blinded to group assignment subitem not at all important 1 2 3 4 5 essential Does your paper address CONSORT subitem 9? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "Participants were automatically randomly assigned (simple randomization) to one of these levels using the randomization tool integrated in Unipark EFS Survey." Does your paper address CONSORT subitem 10? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "Participants were automatically randomly assigned (simple randomization) to one of these levels using the randomization tool integrated in Unipark EFS Survey.", "the questionnaire was rolled out as a voluntary, open survey that was only accessible via the recruitment platform Prolific." 11a-i) Specify who was blinded, and who wasn't Specify who was blinded, and who wasn't. Usually, in web-based trials it is not possible to blind the participants [1, 3] (this should be clearly acknowledged), but it may be possible to blind outcome assessors, those doing data analysis or those administering co-interventions (if any).
subitem not at all important 1 2 3 4 5 essential 11b) If relevant, description of the similarity of interventions (this item is usually not relevant for ehealth trials as it refers to similarity of a placebo or sham intervention to a active medication/intervention) Does your paper address subitem 11a-i? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Participants could not be blinded in the only study by design, but they were not aware that other framing conditions exist while taking part in the study.
11a-ii) Discuss e.g., whether participants knew which intervention was the "intervention of interest" and which one was the "comparator" Informed consent procedures (4a-ii) can create biases and certain expectations -discuss e.g., whether participants knew which intervention was the "intervention of interest" and which one was the "comparator".

Does your paper address subitem 11a-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Participants were told that they will be presented a picture with additional supporting information. Thus, they did not know about the comparator or intervention of interest.
12a) Statistical methods used to compare groups for primary and secondary outcomes NPT: When applicable, details of whether and how the clustering by care providers or centers was addressed subitem not at all important 1 2 3 4 5 essential Does your paper address CONSORT subitem 11b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Not applicable, because we did not employ placebo.

Does your paper address CONSORT subitem 12a? *
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "For inferential analysis of continuous outcomes, we employed a one-way between-subjects analysis of variance (ANOVA). For binary outcomes, we used a dummy-coded binomial logistic regression and tested the coefficients using Wald Chi-Square tests."

12a-i) Imputation techniques to deal with attrition / missing values
Imputation techniques to deal with attrition / missing values: Not all participants will use the intervention/comparator as intended and attrition is typically high in ehealth trials. Specify how participants who did not use the application or dropped out from the trial were treated in the statistical analysis (a complete case analysis is strongly discouraged, and simple imputation techniques such as LOCF may also be problematic [4]).

12b) Methods for additional analyses, such as subgroup analyses and adjusted analyses X26) REB/IRB Approval and Ethical Considerations [recommended as subheading under "Methods"] (not a CONSORT item)
subitem not at all important 1 2 3 4 5 essential Does your paper address subitem 12a-i? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Participants that did not finish the questionnaire were excluded from the analysis. Thus, we did not have to impute missing data.

Does your paper address CONSORT subitem 12b? *
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "To test demographic and interindividual influences, we used multiple linear regression and multiple binomial logistic regression with standardized coefficients for better comparability."

X26-i) Comment on ethics committee approval
subitem not at all important 1 2 3 4 5 essential subitem not at all important 1 2 3 4 5 essential Does your paper address subitem X26-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "Approval for this study was granted by the Ethics Committee of the Department of Psychology and Ergonomics (IPA) at Technische Universität Berlin (Tracking number: FEU_9_210315)."

x26-ii) Outline informed consent procedures
Outline informed consent procedures e.g., if consent was obtained offline or online (how? Checkbox, etc.?), and what information was provided (see 4a-ii). See [6] for some items to be included in informed consent documents.

Does your paper address subitem X26-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "Participants volunteered to participate in the survey, and informed consent was required." They had to manually check a checkbox.

X26-iii) Safety and security procedures
Safety and security procedures, incl. privacy considerations, and any steps taken to reduce the likelihood or detection of harm (e.g., education and training, availability of a hotline)

RESULTS
13a) For each group, the numbers of participants who were randomly assigned, received intended treatment, and were analysed for the primary outcome NPT: The number of care providers or centers performing the intervention in each group and the number of patients treated by each care provider in each center 13b) For each group, losses and exclusions after randomisation, together with reasons Does your paper address subitem X26-iii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Not applicable, because we did not collect any identifying data.
Does your paper address CONSORT subitem 13a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study 173 in the control group, 160 in the anthropomorphic group and 161 in the AI group subitem not at all important 1 2 3 4 5 essential 14a) Dates defining the periods of recruitment and follow-up Does your paper address CONSORT subitem 13b? (NOTE: Preferably, this is shown in a CONSORT flow diagram) * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Since the completion rate was high (94%) and participants were randomized right before being presented the decision aid, we did not observe any losses. However, we excluded 27 participants (7 in the control group, 9 in the anthropomorphic group and 11 in the AI group) because they participated on a mobile phone. 8 (3 in the control group, 4 in the anthropomorphic group and 1 in the AI group) were excluded because they failed at least one embedded attention check after randomization.

13b-i) Attrition diagram
Strongly recommended: An attrition diagram (e.g., proportion of participants still logging in or using the intervention/comparator in each group plotted over time, similar to a survival curve) or other figures or tables demonstrating usage/dose/engagement.

Does your paper address subitem 13b-i?
Copy and paste relevant sections from the manuscript or cite the figure number if applicable (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Not applicable, because the decision aid was only presented once.
subitem not at all important 1 2 3 4 5 essential 14b) Why the trial ended or was stopped (early) Does your paper address CONSORT subitem 14a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "starting on Saturday May 15th at 5 pm EDT and on Sunday May 16th, 2021 at 4 pm EDT." 14a-i) Indicate if critical "secular events" fell into the study period Indicate if critical "secular events" fell into the study period, e.g., significant changes in Internet resources available or "changes in computer hardware or Internet delivery resources" Does your paper address subitem 14a-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study There were nor secular events Does your paper address CONSORT subitem 14b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study The trial ended once we reached our desired sample size.

15-i) Report demographics associated with digital divide issues
In ehealth trials it is particularly important to report demographics associated with digital divide issues, such as age, education, gender, social-economic status, computer/Internet/ehealth literacy of the participants, if known.

Does your paper address subitem 15-i? *
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Demographics such as age, gender and education are reported in the participant characteristics.
subitem not at all important 1 2 3 4 5 essential subitem not at all important 1 2 3 4 5 essential 16-i) Report multiple "denominators" and provide definitions Report multiple "denominators" and provide definitions: Report N's (and effect sizes) "across a range of study participation [and use] thresholds" [1], e.g., N exposed, N consented, N used more than x times, N used more than y weeks, N participants "used" the intervention/comparator at specific pre-defined time points of interest (in absolute and relative numbers per group). Always clearly define "use" of the intervention.
Does your paper address subitem 16-i? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study All participants were analyzed in their originally assigned groups. Data of 173 participants were analyzed in the control group, of 160 in the anthropomorphic group and of 161 in the AI group.

16-ii) Primary analysis should be intent-to-treat
Primary analysis should be intent-to-treat, secondary analyses could include comparing only "users", with the appropriate caveats that this is no longer a randomized sample (see 18-i).

Does your paper address subitem 16-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Not applicable, because we assessed patients' trust in the decision aids.
17a) For each primary and secondary outcome, results for each group, and the estimated effect size and its precision (such as 95% confidence interval) subitem not at all important 1 2 3 4 5 essential Does your paper address CONSORT subitem 17a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "Descriptively, trust in all three framing conditions was very similar (M_Anthropomorphic = 4.503, SD_Anthropomorphic = 0.922, M_AI = 4.495, SD_AI = 0.817, M_Control = 4.508, SD_Control = 0.921), see Figure 2. There was no significant effect of framing on subjective trust (F(2, 491) = 0.009, P = .99, η2 = 0.00). Based on a sensitivity power analysis (α = .05, 1β = .80, n1 = 160, n2 = 173, n3 = 161) we estimate the effect size of possible differences between the groups to not be greater than η2 = 0.018.", "Most participants followed the decision aid's advice and changed their urgency appraisal(77.7%, 384/494), see Table 2. Behavioral trust was slightly higher for the anthropomorphic system (79.4%, 127/160) compared to the control group (77.5%, 134/173), but the difference (OR = 1.120, 95% CI [0.664, 1.897]) was not statistically significant (χ2(1) = 0.18, P = .67). Behavioral trust was slightly lower for the AI system (76.4%, 123/161) compared to the control group, but the difference (OR = 0.942, 95% CI [0.565, 1.570]) was not statistically significant (χ2(1) = 0.053, P = .82), either." 17a-i) Presentation of process outcomes such as metrics of use and intensity of use In addition to primary/secondary (clinical) outcomes, the presentation of process outcomes such as metrics of use and intensity of use (dose, exposure) and their operational definitions is critical. This does not only refer to metrics of attrition (13-b) (often a binary variable), but also to more continuous exposure metrics such as "average session length". These must be accompanied by a technical description how a metric like a "session" is defined (e.g., timeout after idle time) [1] (report under item 6a).
17b) For binary outcomes, presentation of both absolute and relative effect sizes is recommended 18) Results of any other analyses performed, including subgroup analyses and adjusted analyses, distinguishing pre-specified from exploratory Does your paper address subitem 17a-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Not applicable, because participants were only presented the decision aid once.
Does your paper address CONSORT subitem 17b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "Most participants followed the decision aid's advice and changed their urgency appraisal(77.7%, 384/494), see Table 2. Behavioral trust was slightly higher for the anthropomorphic system (79.4%, 127/160) compared to the control group (77.5%, 134/173), but the difference (OR = 1.120, 95% CI [0.664, 1.897]) was not statistically significant (χ2(1) = 0.18, P = .67). Behavioral trust was slightly lower for the AI system (76.4%, 123/161) compared to the control group, but the difference (OR = 0.942, 95% CI [0.565, 1.570]) was not statistically significant (χ2(1) = 0.053, P = .82), either." subitem not at all important 1 2 3 4 5 essential Does your paper address CONSORT subitem 18? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "We observed no differences in subjective trust between participants receiving advice of greater urgency ("healthcare required") than their stand-alone initial assessment ("self-care sufficient"), and those receiving less urgent advice. Concerning behavioral trust, the proportion of participants following more urgent advice was slightly lower (76.5%) than the proportion following advice of lower urgency than their own initial stand-alone assessment (79.7%), see Table 3.", "The participants of all three groups were commonly certain about their initial stand-alone assessment (Mdn = 70, IQR = 60-81). Only 13% (63/494) were unsure (ie, indicating a certainty of less than 50% about their appraisal. No differences in patterns were observed between the framing condition.Participants' certainty in their initial assessment was not associated with subjective trust in the decision aids (R2 = .001) (see Figure 3). With increasing decisional certainty, behavioral trust decreased (OR = 0.966, 95% CI [0.952, 0.979], χ2(1) = 25.00, P < .001, McFaddens R2 = 0.055). However, behavioral trust was high and remained above 50% (56%, 19/34) even for participants indicating maximum decision certainty (100/100) (see Figure 4). There were no differences between the framing conditions.", "Neither demographic variables (age, gender and education) nor basic first aid training were associated with subjective and behavioral trust in the SC when controlling for the respective other variables. An individual's propensity to trust and their eHealth literacy, however, increased subjective trust and was statistically significant (P < .001). However, these two variables did not have a statistically significant influence on behavioral trust (see Appendix)."

18-i) Subgroup analysis of comparing only users
A subgroup analysis of comparing only users is not uncommon in ehealth trials, but if done, it must be stressed that this is a self-selected sample and no longer an unbiased sample from a randomized trial (see 16-iii).

19) All important harms or unintended effects in each group
(for specific guidance see CONSORT for harms) subitem not at all important 1 2 3 4 5 essential Does your paper address subitem 18-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Not applicable, because we did not conduct subgroup analyses.

Does your paper address CONSORT subitem 19? *
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study We did not observe any harms or unintended effects.

19-i) Include privacy breaches, technical problems
Include privacy breaches, technical problems. This does not only include physical "harm" to participants, but also incidents such as perceived or real privacy breaches [1], technical problems, and other unexpected/unintended incidents. "Unintended effects" also includes unintended positive effects [2].

Does your paper address subitem 19-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study There were no technical problems.
subitem not at all important 1 2 3 4 5 essential DISCUSSION 22) Interpretation consistent with results, balancing benefits and harms, and considering other relevant evidence NPT: In addition, take into account the choice of the comparator, lack of or partial blinding, and unequal expertise of care providers or centers in each group 19-ii) Include qualitative feedback from participants or observations from staff/researchers Include qualitative feedback from participants or observations from staff/researchers, if available, on strengths and shortcomings of the application, especially if they point to unintended/unexpected effects or uses. This includes (if available) reasons for why people did or did not use the application as intended by the developers.

Does your paper address subitem 19-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Since this study included a results screen only, we could not collect any data or observations regarding the usage.
subitem not at all important 1 2 3 4 5 essential subitem not at all important 1 2 3 4 5 essential 22-i) Restate study questions and summarize the answers suggested by the data, starting with primary outcomes and process outcomes (use) Restate study questions and summarize the answers suggested by the data, starting with primary outcomes and process outcomes (use).

Does your paper address subitem 22-i? *
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "The aim of the present study was to explore factors influencing laypersons' subjective and behavioral trust in symptom checkers. Especially, we examined the hypothesis that common features of symptom checker interfaces framing the system as either AI-based or anthropomorphic affect users trust in these systems. Our analysis does not support this hypothesis: We could not observe a difference in trust -neither subjective nor behavioralbetween a neutral SC interface (showing a mock company logo) and interfaces framed as either anthropomorphic or as using AI.", "We found most participants (77.7%) followed the decision aid's advice. This is in line with Verzantvoort et al. [17] reporting a high intention of users to follow disposition advice from a decision aid (65%).", "Although participants indicating maximum certainty in their own stand-alone assessment followed the advice less often than those indicating lower levels of certainty, the majority still changed their decision according to the decision aid's recommendation." Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "For example, it would be interesting to assess whether personalized images (eg, patients' own physician) could increase their trust.", "Future research should alter existing symptom checkers to test if our results can be replicated in practice.", "Technically, however, many other vignettes and symptoms can be entered and should thus be investigated in the future.", ". It would be interesting to see if our findings can be replicated for a variety of different cases with different gold standard urgency levels (eg, three-tiered or four-tiered urgency levels). Other decisions -such as whether emergency care is required -should also be examined, as this study cannot provide any evidence for other urgency decisions.

22-ii) Highlight unanswered new questions, suggest future research
Especially concerning the decision of whether emergency care is required or not, we consider further investigation into the question whether layperson trust is unaffected by the direction of the (contradicting) advice by a decision aid worthwhile, as here an incorrect appraisal is more consequential.", "Future studies should conduct qualitative studies on decision-making when being assisted by a symptom checker."

20-i) Typical limitations in ehealth trials
Typical limitations in ehealth trials: Participants in ehealth trials are rarely blinded. Ehealth trials often look at a multiplicity of outcomes, increasing risk for a Type I error. Discuss biases due to non-use of the intervention/usability issues, biases through informed consent procedures, unexpected events.
Does your paper address subitem 20-i? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "First, the intervention might have not been effective in producing meaningful differences. However, nearly all participants (480/494) were able to recall the picture they were presented as part of the decision aid, and thus proving that they took note of the depictions presented as part of the decision aid, and thus proving that they took note of the depictions used for framing. Moreover, the results remain consistent even if participants who could not recall the presented picture are excluded from the analysis. The framing itself represents another limitation. While we followed current practice and the manipulation extent of previous studies, other interface and framing aspects are conceivable that may not have been captured in this study. For example, it would be interesting to assess whether personalized images (eg, patients' own physician) could increase their trust.
In our study participants did not interact with the decision aid as we only presented a SC's results screen instead of letting them enter data or symptoms into an actual app. This was done to keep the survey short, avoid drop-out when entering symptoms for a longer period and to avoid introducing any bias because of different algorithmic pathways resulting from participants unreliably entering information, which is a non-negligible risk as shown by Jungmann et al. [67]. Since trust could be influenced by user experience throughout the interaction [45], we could not account for a potentially moderating role of that factor. This limitation equally applies for all experimental groups, so internal validity is not compromised by this. However, as symptom checkers commonly require extensive user interaction over a span of multiple minutes [68], ecological validity might be limited. Future research should alter existing symptom checkers to test if our results can be replicated in practice. Our participants also only evaluated a single case vignette while in other studies participants solved as many as 20 with the help of a symptom checker app. Hence, the time of exposure to the intervention was low in our study. We consider this closer to the real use case of symptom checkers, though, where users seek advice on a single set of complaints, rather than systematically testing the app by iteratively entering signs and symptoms of highly heterogeneous fictious patient descriptions. Unlike the real use case however, participants could not change their decision at a later stage. In practice, they might decide to see a healthcare professional after gathering further evidence even if they decided for self-care being sufficient when using a SC. Thus, our concept of behavioral trust only captures the users' intention after consulting a symptom checker, not their actual behavior (ie, (not) seeking healthcare according to the SC's prompt).
All participants appraised only a single case vignette which was the same across all three groups. We used only this specific case vignette because it has been used in previous studies and was ambiguous enough for patients to choose both self-care and healthcare. Technically, however, many other vignettes and symptoms can be entered and should thus be investigated in the future. The gold standard for the case vignette used in this study is self-care but visiting a healthcare professional with these symptoms is not inappropriate either, and in particular, is not unsafe. Thus, deviation from the gold standard solution may be considered wrong, but not consequential. Although the gold standard solution was assigned by a panel of physicians, the idea about the absolute correct urgency may vary for different physicians. It would be interesting to see if our findings can be replicated for a variety of different cases with different gold standard urgency levels (eg, three-tiered or fourtiered urgency levels). Other decisions -such as whether emergency care is requiredshould also be examined, as this study cannot provide any evidence for other urgency decisions. Especially concerning the decision of whether emergency care is required or not, we consider further investigation into the question whether layperson trust is unaffected by the direction of the (contradicting) advice by a decision aid worthwhile, as here an incorrect appraisal is more consequential.
It cannot be ruled out that some participants researched the correct solution online to get 21) Generalisability (external validity, applicability) of the trial findings NPT: External validity of the trial findings according to the intervention, comparators, patients, and care providers or centers involved in the trial subitem not at all important 1 2 3 4 5 essential It cannot be ruled out that some participants researched the correct solution online to get the bonus. But because this could have occurred in all groups, internal validity should not be impaired by this.
Participants did not assess their own symptoms but were presented a fictitious case vignette as a proxy for a medical case. Although this arguably reductionist approach is commonly applied when evaluating SCs [6,8,69,70]it remains unclear whether participants assess these symptoms the same way they do when experiencing them. For example, in the case of real symptoms, not only the information input might change, but also patients' mental well-being and their perceived self-efficacy to implement an action might have an impact. It is also conceivable that participants might not have empathized enough with the situation or that the urgency was assessed differently. However, online health information sources are commonly used to assess symptoms of others [3], so this use case still possesses a high degree of external validity.
Since we only collected quantitative data, we cannot explain why participants changed their decision. Future studies should conduct qualitative studies on decision-making when being assisted by a symptom checker.
Lastly, participants in this study were well educated with 54% of participants having a bachelor's degree or higher. While our sample is not representative of the US population, the average education is very close to that of symptom checker users [61]. The same applies to our participants' average age, which is very close to that of users [16] and had no impact in our exploratory analyses. "

21-i) Generalizability to other populations
Generalizability to other populations: In particular, discuss generalizability to a general Internet population, outside of a RCT setting, and general patient population, including applicability of the study results for other organizations subitem not at all important 1 2 3 4 5 essential

OTHER INFORMATION
Does your paper address subitem 21-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "While our sample is not representative of the US population, the average education is very close to that of symptom checker users [61]. The same applies to our participants' average age, which is very close to that of users [16] and had no impact in our exploratory analyses. "

21-ii) Discuss if there were elements in the RCT that would be different in a routine application setting
Discuss if there were elements in the RCT that would be different in a routine application setting (e.g., prompts/reminders, more human involvement, training sessions or other co-interventions) and what impact the omission of these elements could have on use, adoption, or outcomes if the intervention is applied outside of a RCT setting.
Does your paper address subitem 21-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "In our study participants did not interact with the decision aid as we only presented a SC's results screen instead of letting them enter data or symptoms into an actual app.", "Participants did not assess their own symptoms but were presented a fictitious case vignette as a proxy for a medical case. Although this arguably reductionist approach is commonly applied when evaluating SCs [6,8,69,70]it remains unclear whether participants assess these symptoms the same way they do when experiencing them." 23) Registration number and name of trial registry 24) Where the full trial protocol can be accessed, if available 25) Sources of funding and other support (such as supply of drugs), role of funders Does your paper address CONSORT subitem 23? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Not applicable, because we did not conduct a clinical randomized controlled trial and thus did not register our study.
Does your paper address CONSORT subitem 24? * Cite a Multimedia Appendix, other reference, or copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Not applicable, because we did not conduct a clinical randomized controlled trial and thus did not publish a trial protocol.
Does your paper address CONSORT subitem 25? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study The project was funded by the home institutions of the last authors (MF, FB). No external funding was required.

X27) Conflicts of Interest (not a CONSORT item)
subitem not at all important 1 2 3 4 5 essential About the CONSORT EHEALTH checklist yes, major changes yes, minor changes no X27-i) State the relation of the study team towards the system being evaluated In addition to the usual declaration of interests (financial or otherwise), also state the relation of the study team towards the system being evaluated, i.e., state if the authors/evaluators are distinct from or identical with the developers/sponsors of the intervention.

Does your paper address subitem X27-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Since we designed the decision aids ourselves specifically for this study without any ongoing contributions, there are no relations or conflicts of interest to declare.
As a result of using this checklist, did you make changes in your manuscript? * yes no Sonstiges: yes no Sonstiges: What were the most important changes you made as a result of using this checklist?

Meine Antwort
How much time did you spend on going through the checklist INCLUDING making changes in your manuscript * around 5 hours for answering the questions and making changes As a result of using this checklist, do you think your manuscript has improved? * Would you like to become involved in the CONSORT EHEALTH group?
This would involve for example becoming involved in participating in a workshop and writing an "Explanation and Elaboration" document