Bolstering the Business Case for Adoption of Shared Decision-Making Systems in Primary Care: Randomized Controlled Trial

Background Limited budgets may often constrain the ability of health care delivery systems to adopt shared decision-making (SDM) systems designed to improve clinical encounters with patients and quality of care. Objective This study aimed to assess the impact of an SDM system shown to improve diabetes and cardiovascular patient outcomes on factors affecting revenue generation in primary care clinics. Methods As part of a large multisite clinic randomized controlled trial (RCT), we explored the differences in 1 care system between clinics randomized to use an SDM intervention (n=8) versus control clinics (n=9) regarding the (1) likelihood of diagnostic coding for cardiometabolic conditions using the 10th Revision of the International Classification of Diseases (ICD-10) and (2) current procedural terminology (CPT) billing codes. Results At all 24,138 encounters with care gaps targeted by the SDM system, the proportion assigned high-complexity CPT codes for level of service 5 was significantly higher at the intervention clinics (6.1%) compared to that in the control clinics (2.9%), with P<.001 and adjusted odds ratio (OR) 1.64 (95% CI 1.02-2.61). This was consistently observed across the following specific care gaps: diabetes with glycated hemoglobin A1c (HbA1c)>8% (n=8463), 7.2% vs 3.4%, P<.001, and adjusted OR 1.93 (95% CI 1.01-3.67); blood pressure above goal (n=8515), 6.5% vs 3.7%, P<.001, and adjusted OR 1.42 (95% CI 0.72-2.79); suboptimal statin management (n=17,765), 5.8% vs 3%, P<.001, and adjusted OR 1.41 (95% CI 0.76-2.61); tobacco dependency (n=7449), 7.5% vs. 3.4%, P<.001, and adjusted OR 2.14 (95% CI 1.31-3.51); BMI >30 kg/m2 (n=19,838), 6.2% vs 2.9%, P<.001, and adjusted OR 1.45 (95% CI 0.75-2.8). Compared to control clinics, intervention clinics assigned ICD-10 diagnosis codes more often for observed cardiometabolic conditions with care gaps, although the difference did not reach statistical significance. Conclusions In this randomized study, use of a clinically effective SDM system at encounters with care gaps significantly increased the proportion of encounters assigned high-complexity (level 5) CPT codes, and it was associated with a nonsignificant increase in assigning ICD-10 codes for observed cardiometabolic conditions. Trial Registration ClinicalTrials.gov NCT 02451670; https://clinicaltrials.gov/ct2/show/NCT 02451670

yes: all primary outcomes were signi cantly better in intervention group vs control partly: SOME primary outcomes were signi cantly better in intervention group vs control no statistically signi cant difference between control and intervention potentially harmful: control was signi cantly better than intervention in one or more outcomes inconclusive: more research is needed Other: Approx. Percentage of Users (starters) still using the app as recommended after 3 months Identify the mode of delivery. Preferably use "web-based" and/or "mobile" and/or "electronic game" in the title. Avoid ambiguous terms like "online", "virtual", "interactive". Use "Internet-based" only if Intervention includes non-web-based Internet components (e.g. email), use "computer-based" or "electronic" only if o ine products are used. Use "virtual" only in the context of "virtual reality" (3-D worlds). Use "online" only in the context of "online support groups". Complement or substitute product names with broader terms for the class of products (such as "mobile" or "smart phone" instead of "iphone"), especially if the application runs on different platforms.
C l e a r s e l e c t i o n Does your paper address subitem 1a-i? * Copy and paste relevant sections from manuscript title (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study The mode of delivery described in the title is a "shared decision making system" for primary care. Further details of how the system works are described in detail in the manuscript. The system includes custom EHR programming and web services.

1a-ii) Non-web-based components or important co-interventions in title
Mention non-web-based components or important co-interventions in title, if any (e.g., "with telephone support"). Does your paper address subitem 1a-ii?
Copy and paste relevant sections from manuscript title (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study The intervention requires EHR connection to a web-based application and browser display, but the workflow also relies on a printing of interfaces for shared decision making. This is too much to explain in a title, but is described in detail under the intervention description.

1a-iii) Primary condition or target group in the title
Mention primary condition or target group in the title, if any (e.g., "for children with Type I Diabetes") Example: A Web-based and Mobile Intervention with Telephone Support for Children with Type I Diabetes: Randomized Controlled Trial C l e a r s e l e c t i o n Does your paper address subitem 1a-iii? * Copy and paste relevant sections from manuscript title (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study The primary target group of "primary care" is in the title. Does your paper address subitem 1b-i? * Copy and paste relevant sections from the manuscript abstract (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study The key feature of the SDM system intervention is "A shared decision-making system to improve clinical encounters with patients and quality of care" to "improve diabetes and cardiovascular outcomes." It featured "use of a clinically effective SDM system at [primary care] encounters with care gaps "

1b-ii) Level of human involvement in the METHODS section of the ABSTRACT
Clarify the level of human involvement in the abstract, e.g., use phrases like "fully automated" vs. "therapist/nurse/care provider/physician-assisted" (mention number and expertise of providers involved, if any). (Note: Only report in the abstract what the main paper is reporting. If this information is missing from the main body of text, consider adding it) C l e a r s e l e c t i o n subitem not at all important 1 2 3 4 5 essential Does your paper address subitem 1b-ii?
Copy and paste relevant sections from the manuscript abstract (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study The abstract describes use of the SDM system by primary care clinician at clinical encounters with patients. Further details about the workflow, how nurses and medical assistants are essential for the process, and how the interfaces are used are found in the manuscript.
1b-iii) Open vs. closed, web-based (self-assessment) vs. face-to-face assessments in the METHODS section of the ABSTRACT Mention how participants were recruited (online vs. o ine), e.g., from an open access website or from a clinic or a closed online user group (closed usergroup trial), and clarify if this was a purely web-based trial, or there were face-to-face components (as part of the intervention or for assessment). Clearly say if outcomes were self-assessed through questionnaires (as common in web-based trials Copy and paste relevant sections from the manuscript abstract (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study 17 clinics were randomized to "SDM intervention (n=8) versus control clinics (n=9)", and analysis included "all 24,138 encounters with care gaps targeted by the SDM system" at these clinics.  Copy and paste relevant sections from the manuscript abstract (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Conclusion describes a significant result: "increased proportion of encounters assigned high complexity CPT codes. " A "nonsignificant increase in ICD codes for cardiometaboilc conditions" was also observed.
subitem not at all important 1 2 3 4 5 essential 2a-i) Problem and the type of system/solution Describe the problem and the type of system/solution that is object of the study: intended as stand-alone intervention vs. incorporated in broader health care program? Intended for a particular patient population? Goals of the intervention, e.g., being more cost-effective to other interventions, replace or complement other solutions? (Note: Details about the intervention are provided in "Methods" under 5) C l e a r s e l e c t i o n Does your paper address subitem 2a-i? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study The introduction describes the use of "clinical decision support" technology to "promote care efficiency and shared decision making in primary care environments." The introduction describes a SDM system that involves "patient centered CDS and a workflow that early in primary care encounters presents clinicians and patients with printed information about chronic care gaps in low and high literacy formats and prioritizes care options based on potential benefits to the individual." The intervention is incorporated into clinical encounter workflow for patients with diabetes and cardiovascular risk as an adjunct to improve care.
subitem not at all important Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study The introduction provides rational for the analysis on the impact of the intervention on billing and coding documentation at primary care encounters "most of the cost burden for implementing SDM falls on the care delivery system, not the payer, and the lack of data needed to estimate the impact of SDM implementation on care delivery system revenue is often cited as a major barrier to adoption." METHODS 3a) Description of trial design (such as parallel, factorial) including allocation ratio 3b) Important changes to methods after trial commencement (such as eligibility criteria), with reasons Does your paper address CONSORT subitem 2b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study "The objective of this analysis is to evaluate the impact of a SDM system on diagnostic coding and billing at primary care encounters, because these factors substantially impact revenue generation for a care delivery system and, ultimately, can influence the case for SDM adoption." Does your paper address CONSORT subitem 3a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study The main study was a cluster randomized clinical trial with clinics randomized 1:1 within 3 care systems. "The clinic randomization was conducted using a computer-generated random allocation sequence with balance on clinic size and percentage of patients with Medicaid insurance." Only randomized clinics from "one of 3 participating medical groups was used for this exploratory analysis of SDM impact on billing and coding." subitem not at all important 1 2 3 4 5 essential 4a) Eligibility criteria for participants Does your paper address CONSORT subitem 3b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study There were no major changes to methods after trial commencement.

3b-i) Bug fixes, Downtimes, Content Changes
Bug xes, Downtimes, Content Changes: ehealth systems are often dynamic systems. A description of changes to methods therefore also includes important changes made on the intervention or comparator during the trial (e.g., major bug xes or changes in the functionality or content) (5-iii) and other "unexpected events" that may have in uenced study design such as staff changes, system failures/downtimes, etc. [2].
C l e a r s e l e c t i o n Does your paper address subitem 3b-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Yes. Changes to content are necessary due to the nature of clinical decision support that must be kept up to date. "The [intervention content and] interfaces went through multiple iterations based on feedback received from clinicians and patients during the study." subitem not at all important 1 2 3 4 5 essential Does your paper address CONSORT subitem 4a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Yes. Adults age 18-75 who had an index visit at a randomized clinic during the enrollment period and met inclusion and exclusion criteria were enrolled in the study. "Inclusion and exclusion criteria for study intervention were determined...by SDM algorithms at the start of every primary care encounter." Inclusion criteria included the presence of a major cardiovascular risk factor or a large amount of reversible cardiovascular risk. Patients were excluded if codes were identified for hospice, nursing home, active cancer, pregnancy, or cognitive impairment.

4a-i) Computer / Internet literacy
Computer / Internet literacy is often an implicit "de facto" eligibility criterion -this should be explicitly clari ed.
C l e a r s e l e c t i o n Does your paper address subitem 4a-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Computer literacy was not an eligibility criteria. "To meet a wide range of health literacy needs, the printed SDM tools included a more detailed "clinician-oriented" decision support interface as well as a companion low-literacy "patient-oriented" "interface.
subitem not at all important 1 2 3 4 5 essential subitem not at all important 1 2 3 4 5 essential 4a-ii) Open vs. closed, web-based vs. face-to-face assessments: Open vs. closed, web-based vs. face-to-face assessments: Mention how participants were recruited (online vs. o ine), e.g., from an open access website or from a clinic, and clarify if this was a purely web-based trial, or there were face-to-face components (as part of the intervention or for assessment), i.e., to what degree got the study team to know the participant. In online-only trials, clarify if participants were quasi-anonymous and whether having multiple identities was possible or whether technical or logistical measures (e.g., cookies, email con rmation, phone calls) were used to detect/prevent these.
C l e a r s e l e c t i o n Does your paper address subitem 4a-ii? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Not applicable to this study. Patients were not recruited and the study team had no interaction with them.

4a-iii) Information giving during recruitment
Information given during recruitment. Specify how participants were briefed for recruitment and in the informed consent procedures (e.g., publish the informed consent documentation as appendix, see also item X26), as this information may have an effect on user self-selection, user expectation and may also bias results. Does your paper address subitem 4a-iii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study N/A. "The IRB approved waiver of written consent for patients." Does your paper address CONSORT subitem 4b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Yes. The analysis included data from primary care encounters in 18 clinics within a large care system in Minnesota. "During the 9.5-month evaluation period, 32,735 primary care encounters with 18,070 unique adult patients" were identified.

4b-i) Report if outcomes were (self-)assessed through online questionnaires
Clearly report if outcomes were (self-)assessed through online questionnaires (as common in web-based trials) or otherwise. 5) The interventions for each group with sufficient details to allow replication, including how and when they were actually administered Does your paper address subitem 4b-i? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study No. "Encounter-level data from the SDM system were later merged with data extracted from the EHR (Epic Clarity) [24], which included the ICD-10 visit diagnostic codes and CPT level of service for the same encounters. " There were no questionnaires. Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study

4b-ii) Report how institutional affiliations are displayed
The setting is a large care system in Minnesota with no academic affiliations.
subitem not at all important 1 2 3 4 5 essential subitem not at all important 1 2 3 4 5 essential 5-i) Mention names, credential, affiliations of the developers, sponsors, and owners Mention names, credential, a liations of the developers, sponsors, and owners [6] (if authors/evaluators are owners or developer of the software, this needs to be declared in a "Con ict of interest" section or mentioned elsewhere in the manuscript).
C l e a r s e l e c t i o n

Does your paper address subitem 5-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study "This analysis occurred as part of a larger multisite clinical trial (NCT 02451670) funded by the National Institute of Mental Health (NIMH)" "The SDM system implemented for this study was developed at HealthPartners Institute through a series of federally funded research grants by a team with no financial conflicts of interest and with the main objective to improve patient outcomes."

5-ii) Describe the history/development process
Describe the history/development process of the application and previous formative evaluations (e.g., focus groups, usability testing), as these will have an impact on adoption/use rates and help with interpreting results. Does your paper address subitem 5-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Yes. This exploratory analysis occurred as part of a larger multisite clinical trial (NCT 02451670) funded by the National Institute of Mental Health (NIMH). The larger trial was published showing benefit of the intervention for patients with serious mental illness (SMI) and was referenced. It "showed that the rate of increase in total modifiable CV risk was 4% lower among intervention patients compared to control, emphasizing the value of using the SDM system to prompt management of modifiable CV risk factors in the SMI population. [13]"

5-iii) Revisions and updating
Revisions and updating. Clearly mention the date and/or version number of the application/intervention (and comparator, if applicable) evaluated, or describe whether the intervention underwent major changes during the evaluation process, or whether the development and/or content was "frozen" during the trial. Describe dynamic components such as news feeds or changing content which may have an impact on the replicability of the intervention (for unexpected events see item 3b). Does your paper address subitem 5-iii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Yes. By nature of a shared decision making system, the clinical content for this intervention is dynamic to keep it up to date. It is changed and updated based on national guidelines and clinician feedback. "The [intervention content and] interfaces went through multiple iterations based on feedback received from clinicians and patients during the study."

5-iv) Quality assurance methods
Provide information on quality assurance methods to ensure accuracy and quality of information provided [1], if applicable.
C l e a r s e l e c t i o n Does your paper address subitem 5-iv?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study "Descriptive summaries of diagnostic and billing codes were tabulated, including frequencies, means, 95% confidence intervals, and percentiles of the continuous distribution, where applicable." "....... the findings were quite consistent across all coding variables assessed. " subitem not at all important 1 2 3 4 5 essential subitem not at all important 1 2 3 4 5 essential 5-v) Ensure replicability by publishing the source code, and/or providing screenshots/screen-capture video, and/or providing flowcharts of the algorithms used Ensure replicability by publishing the source code, and/or providing screenshots/screencapture video, and/or providing owcharts of the algorithms used. Replicability (i.e., other researchers should in principle be able to replicate the study) is a hallmark of scienti c reporting.
C l e a r s e l e c t i o n Does your paper address subitem 5-v?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Screenshots of the patient and clinician interfaces were provided. Detailed diagrams outlining the technology driving the SDM system and the implementation workflow were included in the manuscript.

5-vi) Digital preservation
Digital preservation: Provide the URL of the application, but as the intervention is likely to change or disappear over the course of the years; also make sure the intervention is archived (Internet Archive, webcitation.org, and/or publishing the source code or screenshots/videos alongside the article). As pages behind login screens cannot be archived, consider creating demo pages which are accessible without login. Does your paper address subitem 5-vi?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study N/A for this intervention.

5-vii) Access
Access: Describe how participants accessed the application, in what setting/context, if they had to pay (or were paid) or not, whether they had to be a member of speci c group. If known, describe how participants obtained "access to the platform and Internet" [1]. To ensure access for editors/reviewers/readers, consider to provide a "backdoor" login account or demo mode for reviewers/readers to explore the application (also important for archiving purposes, see vi).
C l e a r s e l e c t i o n Does your paper address subitem 5-vii? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study yes. "The SDM system currently uses Epic web services to access or display the intervention interfaces. Clinicians accessed the interfaces in one of two ways" "Using the automated BPA pop up to rooming staff"....OR "Clinicians at intervention clinics also could manually view the SDM within the EHR for any adult patient." Use of the intervention was free to clinicians and patients. Clinicians were not paid or financially incentivized to use the intervention.
subitem not at all important Does your paper address subitem 5-viii? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study The SDM system used an "automated process" within a "best practice alert pop up for nurses and medical assistants" that calls the Web Service and Web application for targeted patients. "A browser [within the EHR] displays the patient and provider interfaces for viewing and printing. The interfaces were used for "provider-patient discussion and shared decision making." "The evidence-based SDM system directs patient and clinician attention to a patient's personalized care priorities at the point of care. ""The algorithms use published risk-prediction equations to calculate an individual's reversible CV risk potential and then prioritizes out-of-control CV risk factors according to which would reduce CV risk the most, if successfully addressed. Care suggestions include personalized pharmacologic and lifestyle treatment options that account for the patient's current therapy, most recent status of clinical values, distance from clinical goals, and relevant comorbidities." subitem not at all important 1 2 3 4 5 essential

5-ix) Describe use parameters
Describe use parameters (e.g., intended "doses" and optimal timing for use). Clarify what instructions or recommendations were given to the user, e.g., regarding timing, frequency, heaviness of use, if any, or was the intervention used ad libitum.
C l e a r s e l e c t i o n Does your paper address subitem 5-ix?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Yes. Use of the intervention interfaces were encouraged at every eligible primary care encounter and there may have been repeated exposure for many patients. "The SDM system is automatically triggered at adult visits when clinic rooming staff enter any blood pressure value into the EHR, as is done at over 95% of all primary care clinic visits. When web-based clinical algorithms identify a patient who meets study eligibility criteria, a flag is returned to the electronic health record (EHR) that triggers an EHR best practice advisory (BPA) popup inviting clinic rooming staff to open (one click) and print (one click) the SDM tools for patients and clinicians in intervention clinics. " subitem not at all important 1 2 3 4 5 essential 5-x) Clarify the level of human involvement Clarify the level of human involvement (care providers or health professionals, also technical assistance) in the e-intervention or as co-intervention (detail number and expertise of professionals involved, if any, as well as "type of assistance offered, the timing and frequency of the support, how it is initiated, and the medium by which the assistance is delivered". It may be necessary to distinguish between the level of human involvement required for the trial, and the level of human involvement required for a routine application outside of a RCT setting (discuss under item 21 -generalizability).
C l e a r s e l e c t i o n Does your paper address subitem 5-x?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Yes. The clinical staff including rooming staff (nurses and medical assistants) and clinicians in 17 clinics were involved. The workflow primarily involved two clicks by the rooming staff to open and print the interfaces for clinicians and patients when a best practice alert popped up for targeted patients. Use of the SDM interfaces by clinicians and patients was optional and would naturally depend on time and competing priorities for the encounter.
subitem not at all important 1 2 3 4 5 essential 5-xi) Report any prompts/reminders used Report any prompts/reminders used: Clarify if there were prompts (letters, emails, phone calls, SMS) to use the application, what triggered them, frequency etc. It may be necessary to distinguish between the level of prompts/reminders required for the trial, and the level of prompts/reminders for a routine application outside of a RCT setting (discuss under item 21 -generalizability).
C l e a r s e l e c t i o n Does your paper address subitem 5-xi? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Yes. "When web-based clinical algorithms identify a patient who meets clinical eligibility criteria, a flag is returned to the electronic health record (EHR) that triggers an EHR best practice advisory (BPA) popup inviting clinic rooming staff to open (one click) and print (one click) the SDM tools for patients and clinicians in intervention clinics. " Over the intervention period, nursing leaders at intervention clinics were given monthly reports of how often the tool was being printed for the target population. Leadership generally encouraged high printing rates as a way to improve quality of care. "From our previous implementation experience, these monthly use rate (printing) measurements were essential for promoting and maintaining high SDM use rates." subitem not at all important 1 2 3 4 5 essential 6a) Completely defined pre-specified primary and secondary outcome measures, including how and when they were assessed 5-xii) Describe any co-interventions (incl. training/support) Describe any co-interventions (incl. training/support): Clearly state any interventions that are provided in addition to the targeted eHealth intervention, as ehealth intervention may not be designed as stand-alone intervention. This includes training sessions and support [1]. It may be necessary to distinguish between the level of training required for the trial, and the level of training for a routine application outside of a RCT setting (discuss under item 21 -generalizability. C l e a r s e l e c t i o n Does your paper address subitem 5-xii? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Yes. There was a very minimal amount of training exposure for clinical staff. "Intervention clinic staff were offered a one-hour luncheon training to introduce the SDM tools and learn the recommended workflow before the SDM system became available on March 15, 2017." subitem not at all important 1 2 3 4 5 essential Does your paper address CONSORT subitem 6a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study yes. This was a post hoc secondary exploratory analysis. Outcomes were differences between clinics randomized to use a SDM intervention versus control on (a) likelihood of ICD-10 diagnostic coding for cardiometabolic conditions (The proportion of encounters with each condition and or care gap identified by the SDM system that included a corresponding visit diagnostic code) and (b) Current Procedural Terminology (CPT) billing codes (the proportion of encounters coded with a high complexity CPT level of service 5 (as opposed to other lower CPT levels 2-4). Yes. This was an intention to treat analysis and encounter data was used from encounters of all randomized clinics regardless of whether the SDM interfaces were displayed, printed, or used. However overall use was described for intervention clinics: "the SDM was printed for, on average, 75% of eligible encounters after the first month, and these rates were sustained for the duration of the intervention." 6a-iii) Describe whether, how, and when qualitative feedback from participants was obtained Describe whether, how, and when qualitative feedback from participants was obtained (e.g., through emails, feedback forms, interviews, focus groups). Does your paper address subitem 6a-iii?
Copy and paste relevant sections from manuscript text Clinician feedback was obtained throughout the intervention period through a feedback tab on the SDM display interface within the EHR. Some clinicians directly emailed the SDM system development team.
Does your paper address CONSORT subitem 6b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Yes. This was described as an exploratory analysis in 1 care system that occurred after the larger randomized trial.

7a-i) Describe whether and how expected attrition was taken into account when calculating the sample size
Describe whether and how expected attrition was taken into account when calculating the sample size.
C l e a r s e l e c t i o n 7b) When applicable, explanation of any interim analyses and stopping guidelines 8a) Method used to generate the random allocation sequence NPT: When applicable, how care providers were allocated to each trial group Does your paper address subitem 7a-i?
Copy and paste relevant sections from manuscript title (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Not relevant to this secondary exploratory analysis.
Does your paper address CONSORT subitem 7b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Not relevant to this exploratory analysis.
Does your paper address CONSORT subitem 8a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Yes. ""The clinic randomization was conducted using a computer-generated random allocation sequence" 8b) Type of randomisation; details of any restriction (such as blocking and block size) 9) Mechanism used to implement the random allocation sequence (such as sequentially numbered containers), describing any steps taken to conceal the sequence until interventions were assigned 10) Who generated the random allocation sequence, who enrolled participants, and who assigned participants to interventions Does your paper address CONSORT subitem 8b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Yes. The clinic randomization conducted "included balance on clinic size and percentage of patients with Medicaid insurance." Does your paper address CONSORT subitem 9? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Yes. The study staff other than the statistician were blinded to clinic names prior to assignment. "Clinic names were concealed until intervention assignment." 11a) If done, who was blinded after assignment to interventions (for example, participants, care providers, those assessing outcomes) and how NPT: Whether or not administering co-interventions were blinded to group assignment subitem not at all important 1 2 3 4 5 essential Does your paper address CONSORT subitem 10? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study The study statistician conducted the clinic randomization process. Patients at randomized clinics were enrolled during the routine care process without involvement of the study team.

11a-i) Specify who was blinded, and who wasn't
Specify who was blinded, and who wasn't. Usually, in web-based trials it is not possible to blind the participants [1, 3] (this should be clearly acknowledged), but it may be possible to blind outcome assessors, those doing data analysis or those administering cointerventions (if any).
C l e a r s e l e c t i o n Does your paper address subitem 11a-i? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Given the nature of the intervention, clinicians were not blinded to clinic assignment.
subitem not at all important 1 2 3 4 5 essential 11b) If relevant, description of the similarity of interventions (this item is usually not relevant for ehealth trials as it refers to similarity of a placebo or sham intervention to a active medication/intervention) 11a-ii) Discuss e.g., whether participants knew which intervention was the "intervention of interest" and which one was the "comparator" Informed consent procedures (4a-ii) can create biases and certain expectations -discuss e.g., whether participants knew which intervention was the "intervention of interest" and which one was the "comparator".
C l e a r s e l e c t i o n

Does your paper address subitem 11a-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study N/A. There were no informed consent procedures for patients in this study because the SDM content was limited to evidence based recommendations from national guidelines.
Does your paper address CONSORT subitem 11b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study No. This is not applicable to this analysis.
12a) Statistical methods used to compare groups for primary and secondary outcomes NPT: When applicable, details of whether and how the clustering by care providers or centers was addressed subitem not at all important 1 2 3 4 5 essential Does your paper address CONSORT subitem 12a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study yes. "Generalized linear mixed regression models (with a binomial distribution and logit link) were used for covariate adjustment and random intercepts to account for clustering at the provider and clinic levels. "

12a-i) Imputation techniques to deal with attrition / missing values
Imputation techniques to deal with attrition / missing values: Not all participants will use the intervention/comparator as intended and attrition is typically high in ehealth trials. Specify how participants who did not use the application or dropped out from the trial were treated in the statistical analysis (a complete case analysis is strongly discouraged, and simple imputation techniques such as LOCF may also be problematic [4]).
C l e a r s e l e c t i o n Does your paper address subitem 12a-i? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Not applicable to this analysis. All encounter level data available was used. Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study The IRB waived need for patient consent.

X26-iii) Safety and security procedures
Safety and security procedures, incl. privacy considerations, and any steps taken to reduce the likelihood or detection of harm (e.g., education and training, availability of a hotline) Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study not applicable to this data only exploratory analysis Does your paper address CONSORT subitem 13a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study This was an encounter level analysis rather than a patient analysis. "During the 9.5-month evaluation period, 32,735 primary care encounters with 18,070 unique adult patients were identified." subitem not at all important 1 2 3 4 5 essential 14a) Dates defining the periods of recruitment and follow-up Does your paper address CONSORT subitem 13b? (NOTE: Preferably, this is shown in a CONSORT flow diagram) * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Not applicable to this analysis

13b-i) Attrition diagram
Strongly recommended: An attrition diagram (e.g., proportion of participants still logging in or using the intervention/comparator in each group plotted over time, similar to a survival curve) or other gures or tables demonstrating usage/dose/engagement. Copy and paste relevant sections from the manuscript or cite the gure number if applicable (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Not applicable to this analysis subitem not at all important 1 2 3 4 5 essential 14b) Why the trial ended or was stopped (early) Does your paper address CONSORT subitem 14a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Yes. "We evaluated all eligible patient encounters occurring from March 15, 2017 to December 31, 2017 in intervention and control clinics. " 14a-i) Indicate if critical "secular events" fell into the study period Indicate if critical "secular events" fell into the study period, e.g., signi cant changes in Internet resources available or "changes in computer hardware or Internet delivery resources" C l e a r s e l e c t i o n Does your paper address subitem 14a-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study No known secular events occurred.

15) A table showing baseline demographic and clinical characteristics for each group
NPT: When applicable, a description of care providers (case volume, quali cation, expertise, etc.) and centers (volume) in each group Does your paper address CONSORT subitem 14b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study The study ended as planned per protocol.
Does your paper address CONSORT subitem 15? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study "About half of encounters (49.9%) were with female patients, and the mean age was 58.1 years (slightly older for the intervention clinics, 59.0 vs. 57.2). Seventy-five percent of patients were white. Mean BMI was 33.0 kg/m2. CVD was present in 20.4% of encounters, and of the encounters in which CVD was not identified, the mean 10-year estimated CV risk 16-i) Report multiple "denominators" and provide definitions Report multiple "denominators" and provide de nitions: Report N's (and effect sizes) "across a range of study participation [and use] thresholds" [1], e.g., N exposed, N consented, N used more than x times, N used more than y weeks, N participants "used" the intervention/comparator at speci c pre-de ned time points of interest (in absolute and relative numbers per group). Always clearly de ne "use" of the intervention.
C l e a r s e l e c t i o n Does your paper address subitem 16-i? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study All n's were reported for the denominators of the outcomes.

16-ii) Primary analysis should be intent-to-treat
Primary analysis should be intent-to-treat, secondary analyses could include comparing only "users", with the appropriate caveats that this is no longer a randomized sample (see 18-i). Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study The analysis was intention-to-treat Does your paper address CONSORT subitem 17a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Odds ratios were reported with 95% CIs.
17a-i) Presentation of process outcomes such as metrics of use and intensity of use In addition to primary/secondary (clinical) outcomes, the presentation of process outcomes such as metrics of use and intensity of use (dose, exposure) and their operational de nitions is critical. This does not only refer to metrics of attrition (13-b) (often a binary variable), but also to more continuous exposure metrics such as "average session length". These must be accompanied by a technical description how a metric like a "session" is de ned (e.g., timeout after idle time) [1] (report under item 6a).
C l e a r s e l e c t i o n 17b) For binary outcomes, presentation of both absolute and relative effect sizes is recommended 18) Results of any other analyses performed, including subgroup analyses and adjusted analyses, distinguishing pre-specified from exploratory Does your paper address subitem 17a-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study yes. Use rates defined as print rates of the intervention were reported.
Does your paper address CONSORT subitem 17b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Yes. Absolute effect sizes within the cluster randomized design were reported and adjusted odds ratios.
Does your paper address CONSORT subitem 18? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study This analysis was not pre-specified and was exploratory. Results were adjusted, "1 Age (continuous), sex (F/M), and race (White/Black/Other) were included in multivariable logistic regression models." subitem not at all important 1 2 3 4 5 essential

19) All important harms or unintended effects in each group
(for speci c guidance see CONSORT for harms)

18-i) Subgroup analysis of comparing only users
A subgroup analysis of comparing only users is not uncommon in ehealth trials, but if done, it must be stressed that this is a self-selected sample and no longer an unbiased sample from a randomized trial (see 16-iii Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Use of the intervention was high during the study and continues to be maintained since the study ended. "This SDM is currently in use at all HealthPartners and Park Nicollet primary care clinic systems in Minnesota and Wisconsin and external care organizations in rural Minnesota and 10 other states through collaborative research agreements, with over 250,000 web service calls per month. " 22-i) Restate study questions and summarize the answers suggested by the data, starting with primary outcomes and process outcomes (use) Restate study questions and summarize the answers suggested by the data, starting with primary outcomes and process outcomes (use). Does your paper address subitem 22-i? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study yes. "This analysis demonstrates that use of a SDM system of proven clinical effectiveness was associated with significantly higher levels of CPT level of service 5 coding and with consistent but nonsignficant increases in ICD-10 coding at routine primary care encounters of patients with diabetes and uncontrolled cardiometabolic conditions. An appropriate shift in CPT coding was observed with a significantly increased proportion of encounters coded as high complexity for patients with poorly controlled diabetes and with tobacco dependence. "

22-ii) Highlight unanswered new questions, suggest future research
Highlight unanswered new questions, suggest future research. Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study yes. "Future investigations should jointly consider the clinical and economic impacts of SDM technology from the point of view of both the payer and the care delivery system." "Future study is needed to assess the SDM impact with these newest CMS changes that attempt to simplify billing and coding for office-based services and compensate physicians for additional time spent with patients. Lastly, there are SDM-related factors that could positively influence revenue generation in any of these models, but they must be considered in the context of implementation and maintenance costs of SDM systems and the costs of promoting treatments that may be more intensive. "

20-i) Typical limitations in ehealth trials
Typical limitations in ehealth trials: Participants in ehealth trials are rarely blinded. Ehealth trials often look at a multiplicity of outcomes, increasing risk for a Type I error. Discuss biases due to non-use of the intervention/usability issues, biases through informed consent procedures, unexpected events.
C l e a r s e l e c t i o n Does your paper address subitem 20-i? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Yes. Limitations discussed included (a) data derived from a single care delivery system without replication (b) lack of data on physician productivity (c) financial impact on improved quality of care is not part of this analysis. Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Yes. "This SDM is currently in use at all HealthPartners and Park Nicollet primary care clinic systems in Minnesota and Wisconsin and external care organizations in rural Minnesota and 10 other states through collaborative research agreements, with over 250,000 web service calls per month." However, the manuscript acknowledges that billing and coding patterns vary a lot by care systems and the findings reported in this manuscript may not generalize.

21-ii) Discuss if there were elements in the RCT that would be different in a routine application setting
Discuss if there were elements in the RCT that would be different in a routine application setting (e.g., prompts/reminders, more human involvement, training sessions or other cointerventions) and what impact the omission of these elements could have on use, adoption, or outcomes if the intervention is applied outside of a RCT setting.

24)
Where the full trial protocol can be accessed, if available Does your paper address subitem 21-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Not applicable. The study was a pragmatic trial in a routine clinical setting with minimal training or other co-intervention.
Does your paper address CONSORT subitem 23? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Yes. "NCT02451670." 25) Sources of funding and other support (such as supply of drugs), role of funders X27) Conflicts of Interest (not a CONSORT item) subitem not at all important 1 2 3 4 5 essential Does your paper address CONSORT subitem 24? * Cite a Multimedia Appendix, other reference, or copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Yes. "URL: https://www.clinicaltrials.gov;" Does your paper address CONSORT subitem 25? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Yes. "This work was supported by a Cooperative Agreement with the National Institute of Mental Health (NIMH) grant U19MH092201."

X27-i) State the relation of the study team towards the system being evaluated
In addition to the usual declaration of interests ( nancial or otherwise), also state the relation of the study team towards the system being evaluated, i.e., state if the authors/evaluators are distinct from or identical with the developers/sponsors of the intervention.
C l e a r s e l e c t i o n About the CONSORT EHEALTH checklist yes, major changes yes, minor changes no Does your paper address subitem X27-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study The authors of this study are the developers of the intervention and leaders of the care system being evaluated.
As a result of using this checklist, did you make changes in your manuscript? * What were the most important changes you made as a result of using this checklist?

Your answer
How much time did you spend on going through the checklist INCLUDING making changes in your manuscript * I spent at least 6 hours completing this checklist. STOP -Save this form as PDF before you click submit To generate a record that you lled in this form, we recommend to generate a PDF of this page (on a Mac, simply select "print" and then select "print as PDF") before you submit it.
When you submit your (revised) paper to JMIR, please upload the PDF as supplementary le.
Don't worry if some text in the textboxes is cut off, as we still have the complete information in our database. Thank you! As a result of using this checklist, do you think your manuscript has improved? * Would you like to become involved in the CONSORT EHEALTH group?
This would involve for example becoming involved in participating in a workshop and writing an "Explanation and Elaboration" document