Review
Abstract
Background: Several countries around the world have implemented multicomponent interventions to enhance primary care, as a way of strengthening their health systems to cope with an aging chronically ill population and rising costs. Some of these efforts have included technology-based enhancements as one of the features to support the overall intervention, but their details and impacts have not been explored.
Objective: This study aimed to identify the role of digital/health technologies within wider multifeature interventions that are aimed at enhancing primary care, and to describe their aims and stakeholders, types of technologies used, and potential impacts.
Methods: A systematic review was performed following Cochrane guidelines. An electronic search, conducted on May 30, 2019, was supplemented with manual and grey literature searches in December 2019, to identify multicomponent interventions that included at least one technology-based enhancement. After title/abstract and full text screening, selected articles were assessed for quality based on their study design. A descriptive narrative synthesis was used for analysis and presentation of the results.
Results: Of 37 articles, 14 (38%) described the inclusion of a technology-based innovation as part of their multicomponent interventions to enhance primary care. The most commonly identified technologies were the use of electronic health records, data monitoring technologies, and online portals with messaging platforms. The most common aim of these technologies was to improve continuity of care and comprehensiveness, which resulted in increased patient satisfaction, increased primary care visits compared to specialist visits, and the provision of more health prevention education and improved prescribing practices. Technologies seem also to increase costs and utilization for some parameters, such as increased consultation costs and increased number of drugs prescribed.
Conclusions: Technologies and digital health have not played a major role within comprehensive innovation efforts aimed at enhancing primary care, reflecting that these technologies have not yet reached maturity or wider acceptance as a means for improving primary care. Stronger policy and financial support, and advocacy of key stakeholders are needed to encourage the introduction of efficient technological innovations, which are backed by evidence-based research, so that digital technologies can fulfill the promise of supporting strong sustainable primary care.
doi:10.2196/20195
Keywords
Introduction
Primary care is often considered a cornerstone of health care systems. Health systems with strong primary health care produce better and more equitable health outcomes, are more efficient, and can achieve higher user satisfaction in comparison to health systems with only a weak primary care orientation [
, ]. Changing demographics, an increasingly aging population, and the increased burden of noncommunicable diseases have been identified as new challenges for health systems worldwide [ - ], and strengthening primary care has been proposed as one solution to address these challenges.Many countries have implemented a wide array of innovations to enhance primary care, ranging from policy initiatives, such as capitated reimbursement, to ground level improvements, such as improving access to primary care practices and enhancing the roles of nurses to provide comprehensive primary care services [
- ]. As in other fields, such as finance, retail, and agriculture, an increasingly important domain for innovation involves the incorporation of technology. Technologies are having an impact on health service delivery and health system administration, and they promise to provide solutions for improving primary care [ , ].There have been many studies emphasizing individual digital technologies for improving specific aspects of health care and primary care. Some of these include digital health assistants to help with administrative tasks, medical chatbots to engage patients more frequently, and the use of electronic health records and telemedicine, among others [
- ]. However, no studies have explored the role of technologies within multicomponent efforts to enhance primary care, that is, whether within initiatives comprised of several features aimed at enhancing primary care, there was a technology element being introduced, and if yes, what it was.We aimed to systematically explore the role that health/digital technologies have played in multicomponent efforts designed to improve primary care by identifying (1) the types of technologies implemented, (2) the functional objective of the technology, (3) the relevant stakeholders, and (4) whether they have an impact on enhancing the defining features of primary care (ie, first contact, comprehensiveness, coordination, and continuity) [
], denoted here as the “4Cs.” We explored the overall outcomes of the multicomponent interventions in which technology is one component to attempt to discern the specific contribution of the technologies within these efforts. provides useful definitions for concepts and terms that will be used throughout the article.Useful definitions.
Multicomponent interventions/innovation environments: programs or strategies composed of several innovations/features to enhance primary care.
Innovation features: individual innovation elements included in multicomponent interventions.
Health technologies: application of scientific knowledge to solve health care–related problems, including its corresponding machinery and equipment (includes information technology, digital health, eHealth, mHealth, etc).
4Cs: the primary care core functions (first contact, comprehensiveness, coordination, and continuity).
Quadruple aim outcomes: the four types of outcomes to measure successful health system improvements (population health outcomes, health care utilization and cost outcomes, patient satisfaction, and provider satisfaction).
Methods
A systematic review was designed and performed following Cochrane guidelines for conducting systematic reviews [
]. The detailed methods for this review are described in an article that explored multicomponent interventions aimed at enhancing primary care, which identified 18 innovation strategies and provided a broader picture of the many innovation features used internationally to improve various aspects of primary care simultaneously [ ]. A summary is provided below.An electronic database search was performed in order to identify (1) multicomponent interventions or “innovation environments” aimed at enhancing primary care (with at least three innovation features); (2) factors influencing at least one of the primary care core functions (4Cs), and (3) studies reporting on any of the four basic types of outcomes of a successful health system (the so-called “quadruple aim” outcomes of population health, health care costs and utilization, patient satisfaction, and provider satisfaction) [
] and providing numerical values for at least five outcome measures. In a previous scan of the literature, we identified many specific interventions aimed at enhancing a particular aspect of primary care services (eg, the paper stamp checklist tool enhances asthma guideline knowledge and implementation by primary care physicians), and based on this, we determined that consideration of studies describing interventions with at least three distinct innovation features and measuring at least five outcome measures could ensure that the interventions were indeed “multicomponent.”A search strategy was developed, and it focused on the following three main sets of terms: (1) primary care–related terms; (2) innovation/reform/enhancement-related terms; and (3) study design filters (
). The electronic database search was performed in Ovid/MEDLINE on May 30, 2019, and it was supplemented by manual searches through the references of the included studies and by a grey literature search (ie, search through materials and documents produced by organizations outside of the traditional commercial or academic publishing and distribution channels, such as government and industry documents) in OpenGrey [ ], using “primary care” and “innovation,” on December 12, 2019. From the studies fulfilling these criteria, we selected those that had technology-based enhancements as part of the elements in their multicomponent interventions.We defined health technologies, using definitions from two World Health Organization reports, as the “application of scientific knowledge for practical purposes, including its corresponding machinery and equipment, to solve health care–related problems and improve quality of life” [
] and encompassing digital health technologies (the overarching term to include eHealth and mHealth, eg, telemedicine, electronic health records [EHRs], and wearable sensors) and their corresponding medical and assistive devices [ ].Quality evaluation of the included studies was based on study design, using the National Institutes of Health–National Health, Lung and Blood Institute’s “Study Quality Assessment Tools” [
], a comprehensive suite of study evaluation tools, which has been used in a variety of systematic reviews [ - ]. Data extraction was performed using a predefined data extraction form for study characteristics and general information (author/year, setting/country, policy influence, study design and quality, and patient population involved), primary care intervention elements, and quadruple aim outcomes, including reported magnitudes for each outcome measure. A narrative descriptive approach was utilized to identify and report the types and specific details of the implemented technologies, the involved stakeholders, whether and which 4Cs were arguably supported, and the outcomes influenced by the corresponding technology.Results
Search Results
After the electronic search, subjecting the articles to the inclusion/exclusion criteria and manual reference and grey literature searches resulted in 37 articles fulfilling the requirements for multicomponent interventions as described above. From these, 14 studies had technology-based enhancements and were included in the analysis (
).Study Characteristics
Articles were published between 2008 and 2017, and half of them were published since 2016. Most described studies performed in the United States (9/14, 64%). Additionally, four were from Europe (two from Germany and two from Spain) and one was from Argentina. Eight articles mentioned policies influencing the implementation of the innovation programs as broader country, regional, or organizational efforts to enhance primary care (
) [ - ].Study type | Author (Year) | Program name | Setting/context | Policy/ government program influencing innovation | Study design (quality evaluation rating)a | Patient population (if any) | Innovation elements included in the full interventionb | Types of outcomes studied |
Controlled intervention study | Coderch et al (2016) [ | ]—c | Integrated health care organization in the region of Girona, Spain in 2011 (128,000 residents) | Catalonia’s 2011-2015 health plan; creation of the Program for Chronic Condition Prevention and Care | Controlled, pragmatic, randomized clinical trial, with three arms: one blind control and two open intervention groups (fair) | Complex chronic patients who account for 5% of the highest risk of highest health costs each year | - Accountability mechanisms - Care plan development - Improved access - Improved specialty care access - Enhanced coordination/information exchange efforts - Provider education or training - Technology enhancements | HCd costs and utilization |
Controlled intervention study | Prestes et al (2017) [ | ]DIAPREM study | Primary care units of La Matanza County, Argentina | — | Random selection of 30 PCe providers and 30 nurses from 40 PC units (fair) | T2DMf patients | - Efforts to improve performance monitoring - Enhanced continuity/transition-based efforts - Provider education or training - Technology enhancements | Population health HC costs and utilization |
Controlled intervention study | Ruescas-Escolano et al (2014) [ | ]PROPRESE Trial | Multicentric, PC study (15 health centers), participating in the Cardiometabolic Valencian Study | — | Open randomized clinical trial with 1-year follow-up (good) | Patients with ischemic heart disease | - Efforts to improve performance monitoring - Improved patient self-management - Provider education or training - Others - Team-based care - Technology enhancements | Population health |
Observational cohort or cross-sectional study | Dale et al (2016) [ | ]Comprehensive Primary Care (CPC) Initiative | A large and diverse set of practices in seven Center for Medicare and Medicaid Services (CMS) regions (four states and three metropolitan regions in the United States) | Launching of the Centers for Medicare and Medicaid Services’ Comprehensive Primary Care Initiative, in October 2012 | Pre-post design with a comparison site (fair, classified as retrospective cohort for quality evaluation) | Medicare fee-for-service beneficiaries | - Care development plan - Case management - Improved access - Improved patient self-management - Payment-based enhancements - Social or community services engagement - Technology enhancements | HC costs and utilization Patient satisfaction |
Observational cohort or cross-sectional study | Goff et al (2017) [ | ]Buena Salud | Program implemented at Brightwood Health Center (BHC) in MA, an urban community health center with a largely Hispanic population (88%) insured primarily by either Medicaid (59%) or Medicare (28%) | — | Controlled before-and-after study (fair) | T2DM patients enrolled in the Buena Salud program | - Accountability mechanisms - Case management - Improved access - Improved patient self-management - Improved specialty care access - Social or community services engagement - Team-based care - Technology enhancements | Population health |
Observational cohort or cross-sectional study | Maeng et al (2013) [ | ]ProvenHealth Navigator | 36 Geisinger-owned PC practices, as well as seven contracted PC practices in GHP’sg provider network. Geisinger’s regional health care system is a provider to central, south-central, and northeastern Pennsylvania and southern New Jersey | PCMHh transformation in primary care | Survey of patients in “PHNi sites.” A comparable survey of patients from non-PHN sites was conducted for comparison (fair) | General patient population of PC practices enrolled in the PHN program | - Case management - Efforts to improve performance monitoring - Enhanced service capacity - Improved access - Improved patient self-management - Payment-based enhancements - Social or community services engagement - Team-based care - Technology enhancements | Patient satisfaction |
Observational cohort or cross-sectional study | Maeng et al (2012) [ | ]ProvenHealth Navigator | 36 Geisinger-owned PC practices, as well as seven contracted PC practices in GHP’s provider network. Geisinger’s regional health care system is a provider to central, south-central, and northeastern Pennsylvania and southern New Jersey | PCMH transformation in primary care | Multivariate logistic regression models with controls (members not in the program) (fair, classified as retrospective cohort for quality evaluation) | General patient population of PC practices enrolled in the PHN program | - Case management - Efforts to improve performance monitoring - Enhanced service capacity - Improved access - Improved patient self-management - Payment-based enhancements - Social or community services engagement - Team-based care - Technology enhancements | Population health |
Observational cohort or cross-sectional study | Phillips et al (2014) [ | ]The Illinois Medicaid Health Connect and Your Healthcare Plus programs | Illinois Medicaid beneficiaries, corresponding to 15% of the total state population | The Memisovski v. Maram suit (2004) ruled that Illinois had violated federal law by not providing adequate access to PC services for its Medicaid population, which made Illinois an early leader in Medicaid reform | Analysis of Medicaid claims and enrollment data from 2004 to 2010, covering both pre- and post-implementation (good, classified as retrospective cohort for quality evaluation) | Medicaid beneficiaries | - Accountability mechanisms - Care plan development - Care management - Improved access - Payment-based enhancements - Provider education or training - Technology enhancements | HC costs and utilization |
Observational cohort or cross-sectional study | Wensing et al (2017) [ | ]GPj-centered care (GPCC) program | Introduction of a program to enhance the role of general practice for patients with chronic diseases in Baden-Wuerttemberg, a German federal state with about 10.7 million inhabitants. | — | Comparative evaluation based on two cross-sectional studies at 4 and 5 years after its start (T1 and T2, respectively), based on data continuously collected for administrative control and reimbursement purposes (good) | General population aged 18 years or older with at least one primary care visit | - Accountability mechanisms - Efforts to improve performance monitoring - Enhanced coordination/ information exchange efforts - Improved access - Improved patient self-management - Inclusion of new/enhanced roles - Payment-based enhancements - Pharmacy/medication-related efforts - Provider education or training - Team-based care - Technology enhancements | HC costs and utilization |
Case-control study | Freytag et al (2016) [ | ]GP-centered program | A major Statutory Health Insurance fund AOK PLUSk, which covers 41% of the population in central Germany, established a GP-centered health care program in 2011 in the German federal state of Thuringia | In Germany, enhanced primary care programs started in 2004 with the creation of a legal framework to support “GP-centered health care” | Retrospective case-control study based on insurance claims data (fair) | General patient population | - Inclusion of new/enhanced roles - Payment-based enhancements - Pharmacy/medication-related efforts - Provider education or training - Technology enhancements | HC costs and utilization |
Pre-post study with no control | Conrad et al (2016) [ | ]Group Health Cooperative’s Access Initiative | PC practices within the integrated care delivery system that serves the Puget Sound region in Washington state | — | Pre-post implementation productivity assessment (good) | Group health cooperative’s enrollees | - Enhanced service capacity - Improved access - Improved specialty care access - Others - Payment-based enhancements - Technology enhancements | HC costs and utilization |
Pre-post study with no control | Engel et al (2016) [ | ]Geriatrics in Primary Care (GPC) | Two large medical center practices at the Veterans Affairs Boston Healthcare System in 2014 | Adoption of the Patient Aligned Care Team model of care, which is adapted from the PCMH, by the Veterans Affairs | Before-after evaluation of chart reviews (poor) | Veterans from the Veterans Affairs health system Boston, enrolled in the program | - Case/care management - Enhanced continuity/transition-based efforts - Enhanced service capacity - Improved access - Team-based care - Technology enhancements | HC costs and utilization |
Pre-post study with no control | Maeng et al (2012) [ | ]ProvenHealth Navigator | 36 Geisinger-owned PC practices, and seven contracted PC practices in GHP’s provider network. Geisinger’s regional health care system is a provider to regions of Pennsylvania and New Jersey | PCMH transformation in primary care. | Pre-post (measured at six points) and member fixed-effects model to measure within-member variation in the total cost and the PHN exposure variable over time (good) | GHP’s Medicare Advantage plan members who were at least 65 years and enrolled in clinics that became PHN sites | - Case management - Efforts to improve performance monitoring - Enhanced service capacity - Improved access - Improved patient self-management - Payment-based enhancements - Social or community services engagement - Team-based care - Technology enhancements | HC costs and utilization |
Pre-post study with no control | Ralston et al (2009) [ | ]Group Health’s Access Initiative | Adult respondents (aged ≥18 years) receiving care in Group Health’s Western Washington Integrated Delivery System | Patient-centered system reforms (such as the PCMH model of 2007) mentioned as a shift in the way access to PC is provided, which encouraged HMOsl to change their restrictive access system. | Program impact evaluation, evaluating at three time points, based on the implementation dates of the initiative’s components (fair) | Adult respondents (aged ≥18 years) receiving care in Group Health’s Western Washington Integrated Delivery System | - Accountability mechanisms - Improved access - Improved specialty care access - Others - Payment-based enhancements - Technology enhancements | HC costs and utilization Patient satisfaction Provider satisfaction |
aRatings: good/fair/poor. Study type linked to the tool used for quality evaluation.
bFull details of innovation elements are provided in
.cNot available or not reported in the articles.
dHC: health care.
ePC: primary care.
fT2DM: type 2 diabetes mellitus.
gGHP: Geisinger Health Plan.
hPCMH: Patient-Centered Medical Home.
iPHN: PatienHealthNavigator.
jGP: general practitioner.
kAOK PLUS: health insurance scheme under Germany insurer AOK.
lHMO: health maintenance organization.
In terms of study designs and quality evaluation results, three publications reported controlled interventions (two of “fair” and one of “good” quality), six reported observational cohort or cross-sectional studies with controls (four of “fair” and two of “good” quality), one reported a case-control study of “fair” quality, and four reported pre-post studies without controls (one of “poor”, one of “fair,” and two of “good” quality). Populations studied or linked to the results included the general population enrolled in the programs (in six articles), chronically ill patients with one disease or complex chronic patients (in four articles), and special populations, including elderly and disadvantaged populations (in four articles).
The interventions in the articles included between four and 11 “innovation features” (see
for definitions). The average number of features per intervention was seven (median seven), and the most common types, beside technology-based enhancements (present in all interventions), were innovations to improve access (in 11 articles), payment-based enhancements (in nine articles), and care/case management (in seven articles). In terms of the types of outcomes, the most commonly reported was health care costs and utilization (in 10 articles), followed by population health outcomes (in four articles), patient satisfaction (in three articles), and provider satisfaction (in one article). These are not mutually exclusive as one article reported on three outcomes and two reported on two outcomes each. The remaining 11 articles reported on one outcome each.Technology-Based Results
Of the 37 articles, 14 (38%) describing multicomponent interventions to enhance primary care included technology-based enhancements as one of the innovation elements (
) [ - ].Study | Technology-based on | Specific technology innovation | Aim and stakeholder (patient/provider/admin manager) | “4C” being supported by technology | General results and direction of the effects on quadruple health outcomes (of the full intervention)a |
Coderch et al (2016) [ | ]EMRsb | - Identification of patients: complex chronic patients are identified by labelling them in unique EMRs for providers - Proactive actions in PCc: individualized care plan registered in unique EMRs for providers | For providers, to be able to easily identify complex chronic patients under their care | Continuity | Health care costs and utilization ↑ (considerable increase in nonurgent primary care visits for partial and full interventions compared to each other and to control for both years 1 and 2) ↔ (mixed results for acute hospital admission and stay for year 1: considerable decrease for partial intervention compared to control, but considerable increase in full intervention compared to partial intervention; similar for readmissions <30 days in year 2, and considerable decrease for partial intervention and increase for full intervention when compared to each other) ↓ (increase in the number of prescriptions for full intervention compared to control for year 2) |
Conrad et al (2008) [ | ]Online messaging platform Online patient portal/website | - Patient-provider secure messaging through the MyGroupHealth enrollee website, including physician financial incentives for secure messaging patients - Internet access for enrollees to their EMRs through MyGroupHealth - Health promotion information on the MyGroupHealth secure website | For providers and patients, to have enhanced communication For patients, to promote self-management (through access to their medical information and health promotion information) | Continuity Comprehensiveness | Health care costs and utilization ↑ (considerable increase in panel size per FTEd and relative value unit per visit; considerable decrease in visits per FTE and per member per quarter costs) ↔ (nonrelevant increase in relative value unit per FTE) |
Dale et al (2016) [ | ]EMRs | - Optimal use of health ITe, including improving EHRf function and capability and developing practice capability for optimal use of EHR; enabling exchange of patient information to support care; and developing quality measurement and reporting from EHRs | For providers, to better use EHRs, use information to support patient care, and improve quality monitoring | Comprehensiveness Coordination Continuity | Health care costs and utilization ↑ (decrease in total Medicare expenditures [without initiative care-management fees] and considerable decrease in PC visits and diabetes patients with no tests performed) ↔ (nonrelevant effects for hospitalizations, EDg visits, specialist visits, admissions for ambulatory care–sensitive conditions, and likelihood of readmissions; no differences for tests performed for diabetes or ischemic vascular patients) Patient satisfaction ↑ (increase in satisfaction with timely appointments, self-management support, and discussion of medications) ↔ (nonrelevant differences for communication with providers, knowledge of providers of other services, and patient ratings of providers) |
Engel et al (2016) [ | ]Telephone Electronic consultations | - Proactive telephone contact with veterans and caregivers, ready access to primary care colleagues, and informed use of telephone follow-up to enhance care while reducing nonessential clinic visits - Electronic consultation for formal referrals to geriatrics in PC program | For providers, to have easier referral to services For patients, to reduce clinical visits, while enhancing care | First contact Continuity Coordination | Health care costs and utilization ↑ (decrease in the number of specialist visits after years 1 and 2, while maintaining the number of PC visits) |
Freytag et al (2016) [ | ]Medication-specific IT tool | - Obligatory use of a specific IT-pharmacotherapy tool to support rational pharmacotherapy | For providers, to support rational prescription of medicines | Comprehensiveness | Health care costs and utilization ↑ (decrease in the cost of drug prescriptions; increase in GPh consultations and decrease in specialist consultations, hospital use, and remedies; decrease in share of patients consulting more than one GP and accessing specialist without referrals; increase in the number of patients in disease management program and home visits; and decrease in the number of medical check-ups) ↓ (increase in the cost of GP consultations and specialist consultations and increase in the share of patients with five or more different medications) ↔ (no change in the number of ED hospitalizations or increase in the nursery care level) |
Goff et al (2017) [ | ]EHRs Use of insurer data | - Use of electronic health registries to identify patients in need of care and services (quarterly, reviewed the data contained in EHRs and insurer data focusing on specific care parameters in care [ie, ordered labs and mammography, scheduled PC visits, etc]) | For providers, to monitor care needs and ensure tests and visits | Continuity | Population health ↑ (considerable changes in the mean DBPi and microalbumin/creatinine ratio test within 12 months) ↔ (no relevant difference for changes in HbA1c measures, lipid measures, or other blood pressure measures; changes for HbA1c tests and lipid panels) |
Maeng et al (2012, 2012, 2013) [ | , , ]EHRs Online patient portal Online messaging platform Modeling and utilization data tools | - Preventive and chronic care optimized by health IT. - Active delivery of information to other team members at the point of care via shared EHRs - Access to the patient portal for reviewing medical records and secure messaging with providers - Predictive modelling and utilization of data tools and normative management data to improve care | For providers, to have availability of patient information for all medical team members For providers and patients, to have enhanced communication For patients, to have access to their medical records to promote self-management For practices, to have improved monitoring for population care | Comprehensiveness Coordination Continuity | Population health ↑ (decrease in amputation and end-stage renal disease in the intervention group) ↔ (no difference for myocardial infarction or stroke) Health care costs and utilization ↑ (decrease in the per member and per month allowed costs; considerable overall savings with and without Rx coverage interaction) ↓ (increase in the cost of Rx coverage, without considering other program costs) Patient satisfaction ↑ (improvement in perceived changes in care delivery, ie, “noticed difference in care coordination and higher quality,” increase in reporting of doctor’s office as usual care, and decrease in ERj visits) ↔ (no relevant changes for access to care or primary care provider performance) |
Phillips et al (2014) [ | ]Online registries/report cards | - Multiple online tools, such as registries and report cards, to assist clinicians with population-based management | For providers, to have improved monitoring and population-based management | Continuity | Health care costs and utilization ↑ (increase in estimated cost savings and rate estimated annual savings; decrease in hospitalization, bed-day, and avoidable hospitalization rates; and increase in all quality measure changes [test and screenings]) ↔ (decrease in the ED visit rate for IHCk but increase for YHPl) |
Prestes et al (2017) [ | ]Data monitoring system | - The QUALIDIAB data system was used to verify the impact of the diabetes education intervention, and the data collected are useful to allocate resources (human and financial) considering real demand | For providers, to verify the impact of the intervention and allocate resources using collected data | Continuity | Population health ↑ (considerable improvements for DBP, glycemia, HbA1c, total cholesterol, and LDL-cm and increase in the percentage of patients with target SBPn and HbA1c levels) ↔ (nonrelevant differences for SBP, creatinine, proteinuria, HDL-co, DBP <80 mmHg, glycemia <100 mg/dL, cholesterol <200 mg/dL, and triglyceride <150 mg/dL) Health care costs and utilization ↑ (considerable increase in dyslipidemia patients treated, eye tests, and cardiovascular evaluations) ↔ (nonrelevant differences for dyslipidemia treated under target or any hypertension treatments) |
Ralston et al (2009) [ | ]Online patient portal Online messaging platform | - Web access for patients that provides secure email with physicians, medical record access, medication refills, appointment scheduling, discussion groups, and health promotion information | For patients, to facilitate accessing physicians, making appointments, refilling prescriptions, accessing medical records, and supporting self-management | First contact Comprehensiveness | Health care costs and utilization ↑ (improvement in “Getting Needed Care” and “Getting Care Quickly” scores) Patient satisfaction ↑ (improvement in satisfaction with the ability to see a personal doctor; time spent on the phone and waiting time for appointment; ease of getting care; and ratings of health care, health plan, and opinion of Group Health) Provider satisfaction ↑ (improvement in the perception of providers toward Group Health’s quality and services provided and for Group Health as a good place to work) |
Ruescas-Escolano et al (2014) [ | ]EMRs | - Use of unique EMRs that allow for following control indicators and risk stratification | For providers, to monitor patient progress and manage risk | Continuity | Population health ↑ (considerable improvements in smoking status, cholesterol, and SBP) ↔ (nonrelevant differences for DBP) |
Wensing et al (2017) [ | ]Medication-specific IT tool Updated IT systems | - The practice has a data-orientated quality system and decision support for prescribing medication; prompts in software to support use of generic and discounted drugs - The practice has up-to-date IT | For providers, to support medication prescription and promote generic medication use For practices, to have better organization to support easier patient access | Comprehensiveness First contact | Health care costs and utilization ↑ (decrease in the costs of medication therapy and hospital admissions) ↑ (increase in the number of visits to family physicians and mean number of prescription drugs; decrease in the number of prescriptions that should be avoided, contacts with specialists with and without referrals, hospital admissions, avoidable hospital admissions, number of days at hospital, and readmissions) |
aExtracted from Jimenez et al [
].bEMR: electronic medical record.
cPC: primary care.
dFTE: full-time equivalent.
eIT: information technology.
fEHR: electronic health record.
gED: emergency department.
hGP: general practitioner.
iDBP: diastolic blood pressure.
jER: emergency room.
kIHC: Illinois Medicaid Health Connect.
lYHP: Your Healthcare Plus.
mLDL-c: low-density lipoprotein cholesterol.
nSBP: systolic blood pressure.
oHDL-c: high-density lipoprotein cholesterol.
According to the descriptions of the articles, we were able to identify the following six broad categories for the types of implemented technologies (description below includes intended stakeholder and use):
- 1. Enhancements leveraging electronic medical/health records [ , - , ]: it was the most common category (reported in seven studies from five interventions) and was aimed at providers. Their use is related to identifying specific groups of patients (eg, chronically ill) or specific needs of patients (eg, services needed), exchanging patient information, and developing quality measurements/control and risk stratification.
- 2. Data monitoring technologies/online registries [ , - , ]: it was the second most common category (in six studies from four interventions) and was aimed at providers and practices. It was related to the management of utilization data to allocate resources and improve care, help with population-based management, and check on the impact of programs.
- 3. Web-based online portals and messaging platforms [ , , , , ]: it was included in five studies (reporting on three interventions) and was aimed at patients to access their medical records, obtain additional health promotion information, promote self-management, and facilitate access and communication with providers.
- 4. Medication-specific eHealth/information technology tools [ , ]: it was included in two studies and was aimed at providers to support pharmacotherapy and medication prescription.
- 5. Telephone-based enhancements [ ]: it was described in one article and was aimed at providers to communicate with patients and caregivers, and provide follow-up to reduce patients’ nonessential clinic visits.
- 6. Electronic consultations between providers [ ]: it was described in one study and was aimed at enhancing geriatric referrals.
Based on the description of the technological enhancements included in the studies, we were able to link them to the 4Cs in the following way:
- 1. First contact: Three programs aimed to apply technology to impact this feature through telephone-facilitated access to primary care colleagues, facilitated appointment scheduling through web portals, and updated digital health systems for easier patient access.
- 2. Comprehensiveness: Six interventions sought to increase the ability to manage a wider range of problems with technology, including providing additional health promotion information through patients’ web portals and enhancing capacity for providers to better use electronic medical records, improve medication prescription, and provide improved preventive and chronic care.
- 3. Coordination: Three programs used technology to improve care coordination by improving EHR-enabled information exchange and by allowing electronic consultations to facilitate care among primary care providers and specialists.
- 4. Continuity: Nine interventions sought to enhance the longitudinal relationship between patients and providers by enhancing the identification and follow-up of patients for individualized care, allowing more comprehensive identification and monitoring of service needs, and improving communication between patients and providers via online messaging or telephone contact.
Outcomes
Since these technology-based innovation elements are part of wider innovation environments, which include additional enhancement features, it was not possible to attribute outcomes specifically to the identified technologies. However, we still present the outcomes of the full innovation environments in an effort to elucidate the potential role of these technologies in the outcomes. The numerical magnitudes for each outcome are presented in
(along with details of the full intervention). and the paragraphs below present a descriptive summary and general direction of the effects for these outcomes.Overall, the studies presented mixed results (ie, nonsignificant changes or significant benefits and deteriorations simultaneously for a specific outcome) for all types of outcomes, except for provider satisfaction, which was reported only in one study. The most consistent improvements per type of outcome were as follows: (1) health care costs and utilization, increased cost saving and decreased costs for some parameters (eg, Medicare expenditure decreased by US $11 per beneficiary per month [
] and drug prescriptions decreased by €44 per patient [ ]) and increased primary care visits compared to specialists; (2) population health, improved blood pressure control, improved glycated hemoglobin, decreased amputations and end-stage renal disease, and decreased smoking status; (3) patient satisfaction, increased satisfaction with timely appointments and self-management support and increased satisfaction with the ability to see the usual doctor; (4) provider satisfaction, improved perception toward place of work’s quality and services provided, and its consideration as a good place to work.The most consistent mixed results by the type of outcome were as follows: (1) health care costs and utilization, nonsignificant changes or simultaneous improvements and deteriorations depending on the study for hospital admissions, readmissions, and emergency department visits; (2) population health outcomes, nonsignificant changes for cholesterol and lipid levels, myocardial infarction, and stroke; (3) patient satisfaction, no differences for communication with providers and for primary care provider performance.
The most consistent deteriorations were found for some health care costs and utilization outcomes, such as increased number of prescriptions, increased costs for general practitioner (eg, intervention €27 more expensive than control per patient) and specialist (intervention €22 more expensive than control per patient) consultations [
], and increased costs for prescription coverage.Discussion
Principal Findings
Only 38% of our identified multicomponent interventions that aimed at enhancing primary care included technology-based enhancements, highlighting the fact that technology has not played a major role in comprehensive efforts aimed at enhancing primary care. This is not surprising, as it has been widely acknowledged that innovation in health care has always been difficult [
], especially if it has involved digital or technological efforts [ - ].Most of the included articles reported on health care costs and utilization outcomes, signaling that technology-based efforts are either aimed at decreasing costs and utilization or at least not increasing costs without contributing to other aspects of system success. In fact, the only considerable unintended consequences were increased costs for general practitioner and specialist visits, and increased costs of prescription coverage (in some studies), suggesting that introducing technologies in health care can lead to increased costs, as it has been consistently reported in the literature [
- ].The most common technology identified within these efforts was EHR, which is also not surprising given the widespread advocacy for this technology [
, ], and it was aimed mainly at providers or practices to facilitate information exchange among them and improve monitoring efforts. The only identified technology aimed at patients was the deployment of online patient portals, where they can see their records, message their providers, and access additional health information mostly for health promotion, which is in line with the idea that patients are ever more active participants in their own health care [ , ].When analyzing the interventions in terms of their impact on the 4Cs, the technologies implemented were mostly aimed at improving continuity by increasing the identification and follow-up of patients (with labels in EHRs and telephone communication), enhancing monitoring efforts for identifying care and service needs (also mostly through EHRs and online registries), and ensuring more constant communication between providers and patients via online messaging. This reflects the growing importance of continuity of care, which in the past has had weak evidence linked to its benefits, but was recently highlighted as important, especially with regard to its link to decreased mortality risk [
, ]. Technologies have been promoted to improve comprehensiveness by providing additional health promotion information for patients, improving the ability of providers to prescribe medications, reinforcing the ability of primary care providers to cover a broader number of issues themselves, and avoiding overreferring [ ].In terms of outcomes, the literature provides limited but useful information. For example, increased patient satisfaction with timeliness of care, scheduling, and better self-management support could be in part explained by the use of online patient portals. Such portals allow patients to schedule appointments, see their own medical records, and access additional prevention information. Increased primary care visits, relative to specialist visits, appear to result from innovations that enhance monitoring of services needed and follow-up of patients (identified through EHRs and/or by telephone follow-up). The introduction of medication-specific digital/information technology tools could be associated with differing impacts. While studies reported a decrease in the costs of drug prescriptions and medication therapies, they also reported an increase in the mean number of drugs prescribed, and it was also associated with more costly consultations (around €25 extra per consultation [
]).In order for digital technologies to play a more prominent role in primary care enhancement efforts, there is first a need for a responsible policy to support their development and introduction [
]. For example, some of the primary care enhancement environments have included an explicit policy encouraging the introduction of technology or information technology initiatives as part of their efforts [ , , ]. To make this happen successfully, the technology must be seen as a tool that provides needed functions in a way that is effective, humane, and sustainable. Here, the context in which the technology will be implemented must be considered. It is essential to engage relevant stakeholders to deeply understand their environment and capabilities so that the introduced technology will be truly useful, improve (or at least not disrupt) existing workflow, and have tangible value [ , ]. In order to establish value, there is a need for technologies to be linked to evidence-based positive outcomes, taking into account their potential to improve health outcomes, costs, and patient and provider satisfaction.There are some limitations for this study. The nature of the search and the specific requirements for including studies (ie, those describing multicomponent interventions aimed at enhancing primary care, which provided numerical magnitudes for reporting quadruple aim outcomes) may have made us overlook other important technological innovations aimed at improving primary care that had qualitative assessments only or did not measure quadruple aim outcomes. Similarly, since this review only focused on published and grey literature, it did not account for quality improvement interventions implemented at, for example, private primary care or accountable care organizations, which may have included technological innovations but no published results. Therefore, although technology may have a more prominent role in primary care–enhancing initiatives overall, the results from published and grey literature do not indicate so. Additionally, the fact that technological enhancements were one of many components within a primary care enhancement effort, our study eligibility criteria did not allow us to establish the actual and specific impact of the technologies on outcomes. However, it did help to situate these technologies within multicomponent innovation strategies and to gain preliminary insights into how technological enhancements may support other nontechnologically based innovation features and their impacts on the four primary care functions.
Taking all of this into account, future research should try to pinpoint the specific impact of technology-based innovation features within wider efforts aimed at enhancing primary care. This would mean including specific measures that could link outcomes to the use of such technology and quantify this impact. This exercise would also help to identify which of the 4Cs of primary care is being impacted by this technology, which would help understand the mechanisms of how these innovations are improving care. An additionally interesting research direction would be to explore how technological innovations are being leveraged at primary care private practices and/or accountable care organizations, to understand the actual role of technology in quality improvement initiatives for which there is no publicly available data. Such research could provide a more balanced view of the actual usage of technological innovations in primary care at the ground level.
Conclusions
Although technology and digital health have been proposed and encouraged as possible solutions to improve primary care, they have not played a major role in formally evaluated multicomponent interventions aimed at enhancing primary care, as reflected in the published and grey literature. Other types of nontechnologically based innovations, such as those aimed at improving access, restructuring payments for providers, and providing team-based care, have been much more widely implemented, reflecting that digital health technologies have not yet reached maturity or wider acceptance as a means for improving primary care. Leveraging technologies already in use, such as EHRs, and internet-based technologies, such as online patient portals, seems to provide promising avenues to improve continuity and comprehensiveness in primary care, which may eventually lead to better health outcomes and improved patient satisfaction. A stronger push is needed if technologies are meant to support wider efforts aimed at enhancing primary care and for them to play a more substantial role within these efforts. High-level policy and financial support must be designed to focus on the needs of a diversity of stakeholders and to encourage evidence-based research based on a coherent set of methods and measures. In this way, we can hope to fulfill the promise of technologies and digital health to enhance health care through strong sustainable primary care.
Conflicts of Interest
None declared.
Search strategy.
DOCX File , 14 KB
Innovation elements and definitions.
DOCX File , 16 KB
Details of interventions and magnitudes of outcomes.
DOCX File , 71 KBReferences
- Starfield B, Shi L, Macinko J. Contribution of primary care to health systems and health. Milbank Q 2005 Sep;83(3):457-502 [FREE Full text] [CrossRef] [Medline]
- Macinko J, Starfield B, Shi L. The contribution of primary care systems to health outcomes within Organization for Economic Cooperation and Development (OECD) countries, 1970-1998. Health Serv Res 2003 Jun;38(3):831-865 [FREE Full text] [CrossRef] [Medline]
- Haseltine W. Aging Populations Will Challenge Healthcare Systems All Over The World. Forbes Magazine. 2018. URL: https://www.forbes.com/sites/williamhaseltine/2018/04/02/aging-populations-will-challenge-healthcare-systems-all-over-the-world/?sh=1f071e852cc3 [accessed 2020-12-17]
- Mendelson DN, Schwartz WB. The effects of aging and population growth on health care costs. Health Aff (Millwood) 1993 Jan;12(1):119-125. [CrossRef] [Medline]
- Adler-Waxman A. This is the biggest challenge to our health. World Economic Forum: Agenda. 2017. URL: https://www.weforum.org/agenda/2017/12/healthcare-future-multiple-chronic-disease-ncd [accessed 2001-12-18]
- Macinko J, Montenegro H, Nebot Adell C, Etienne C, Grupo de Trabajo de Atención Primaria de Salud de la Organización Panamericana de la Salud. [Renewing primary health care in the Americas]. Rev Panam Salud Publica 2007;21(2-3):73-84. [CrossRef] [Medline]
- A vision for primary health care in the 21st century: towards universal health coverage and the Sustainable Development Goals. World Health Organization. 2018. URL: https://apps.who.int/iris/handle/10665/328065 [accessed 2019-12-18]
- Realising the Full Potential of Primary Health Care - Policy Brief. OECD Health Policy Studies. 2019. URL: http://www.oecd.org/health/health-systems/OECD-Policy-Brief-Primary-Health-Care-May-2019.pdf [accessed 2019-12-18]
- Digital technologies: shaping the future of primary health care. World Health Organization. 2018. URL: https://www.who.int/docs/default-source/primary-health-care-conference/digital-technologies.pdf [accessed 2019-12-18]
- Mitchell M, Kan L. Digital Technology and the Future of Health Systems. Health Syst Reform 2019 Mar 25;5(2):113-120. [CrossRef] [Medline]
- How Could Digital Technology Make an Impact on Primary Care? The Medical Futurist. 2018. URL: https://medicalfuturist.com/digital-technology-make-an-impact-on-primary-care/ [accessed 2019-12-18]
- Starfield B. Primary care: concept, evaluation, and policy. Oxford, United Kingdom: Oxford University Press; 1992.
- Higgings JPT, Green S, editors. Cochrane Handbook for Systematic Reviews of Interventions: Cochrane Book Series. Hoboken, New Jersey: John Wiley & Sons Ltd; 2008.
- Jimenez G, Matchar D, Koh GCH, Car J. Multicomponent interventions for enhancing primary care: a systematic review. Br J Gen Pract 2020 Nov 30. [CrossRef] [Medline]
- Bodenheimer T, Sinsky C. From triple to quadruple aim: care of the patient requires care of the provider. Ann Fam Med 2014 Nov 10;12(6):573-576 [FREE Full text] [CrossRef] [Medline]
- OpenGrey. URL: http://opengrey.org/ [accessed 2020-12-22]
- What is a health technology? World Health Organization. 2019. URL: https://www.who.int/health-technology-assessment/about/healthtechnology/en/ [accessed 2019-12-18]
- Study Quality Assessment Tools. National Heart Lung and Blood Institute and National Institutes of Health. 2018. URL: https://www.nhlbi.nih.gov/health-topics/study-quality-assessment-tools [accessed 2019-12-01]
- Cantrell A, Croot E, Johnson M, Wong R, Chambers D, Baxter SK, et al. Access to primary and community health-care services for people 16 years and over with intellectual disabilities: a mapping and targeted systematic review. Health Serv Deliv Res 2020 Jan;8(5):1-142. [CrossRef] [Medline]
- Koppen IJN, Kuizenga-Wessel S, Saps M, Di Lorenzo C, Benninga MA, van Etten-Jamaludin FS, et al. Functional Defecation Disorders and Excessive Body Weight: A Systematic Review. Pediatrics 2016 Sep 16;138(3):e20161417-e20161417 [FREE Full text] [CrossRef] [Medline]
- Hashad N, Tonna A, Perumal D, Stewart D. Antimicrobial Stewardship Program Implementation in the Gulf Cooperation Council States: A Systematic Review. Journal of Infection and Public Health 2020 Feb;13(2):324. [CrossRef]
- Coderch J, Pérez-Berruezo X, Sánchez-Pérez I, Sánchez E, Ibern P, Pérez M, et al. [Assessment of the effectiveness of a proactive and integrated healthcare programme for chronic complex patients]. Gac Sanit 2018;32(1):18-26 [FREE Full text] [CrossRef] [Medline]
- Prestes M, Gayarre MA, Elgart JF, Gonzalez L, Rucci E, Paganini JM, DIAPREM (DIAbetes Primary Care‚ Registry‚ EducationManagement). Improving diabetes care at primary care level with a multistrategic approach: results of the DIAPREM programme. Acta Diabetol 2017 Sep 17;54(9):853-861. [CrossRef] [Medline]
- Ruescas-Escolano E, Orozco-Beltran D, Gaubert-Tortosa M, Navarro-Palazón A, Cordero-Fort A, Navarro-Pérez J, et al. El estudio PROPRESE: resultados de un nuevo modelo organizativo en atención primaria para pacientes con cardiopatía isquémica crónica basado en una intervención multifactorial. Atención Primaria 2014 Jun;46:10-15 [FREE Full text] [CrossRef] [Medline]
- Dale SB, Ghosh A, Peikes DN, Day TJ, Yoon FB, Taylor EF, et al. Two-Year Costs and Quality in the Comprehensive Primary Care Initiative. N Engl J Med 2016 Jun 16;374(24):2345-2356. [CrossRef]
- Goff SL, Murphy L, Knee AB, Guhn-Knight H, Guhn A, Lindenauer PK. Effects of an enhanced primary care program on diabetes outcomes. Am J Manag Care 2017 Mar 01;23(3):e75-e81 [FREE Full text] [Medline]
- Maeng DD, Davis DE, Tomcavage J, Graf TR, Procopio KM. Improving patient experience by transforming primary care: evidence from Geisinger's patient-centered medical homes. Popul Health Manag 2013 Jun;16(3):157-163. [CrossRef] [Medline]
- Maeng DD, Graf TR, Davis DE, Tomcavage J, Bloom FJ. Can a patient-centered medical home lead to better patient outcomes? The quality implications of Geisinger's ProvenHealth Navigator. Am J Med Qual 2012 Aug 18;27(3):210-216. [CrossRef] [Medline]
- Phillips RL, Han M, Petterson SM, Makaroff LA, Liaw WR. Cost, utilization, and quality of care: an evaluation of illinois' medicaid primary care case management program. Ann Fam Med 2014 Sep 08;12(5):408-417 [FREE Full text] [CrossRef] [Medline]
- Wensing M, Szecsenyi J, Stock C, Kaufmann Kolle P, Laux G. Evaluation of a program to strengthen general practice care for patients with chronic disease in Germany. BMC Health Serv Res 2017 Jan 21;17(1):62 [FREE Full text] [CrossRef] [Medline]
- Freytag A, Biermann J, Ochs A, Lux G, Lehmann T, Ziegler J, et al. The Impact of GP-Centered Healthcare. Dtsch Arztebl Int 2016 Nov 25;113(47):791-798 [FREE Full text] [CrossRef] [Medline]
- Conrad D, Fishman P, Grembowski D, Ralston J, Reid R, Martin D, et al. Access intervention in an integrated, prepaid group practice: effects on primary care physician productivity. Health Serv Res 2008 Oct;43(5 Pt 2):1888-1905 [FREE Full text] [CrossRef] [Medline]
- Engel PA, Spencer J, Paul T, Boardman JB. The Geriatrics in Primary Care Demonstration: Integrating Comprehensive Geriatric Care into the Medical Home: Preliminary Data. J Am Geriatr Soc 2016 Apr 21;64(4):875-879. [CrossRef] [Medline]
- Maeng D, Graham J, Graf TR, Liberman JN, Dermes NB, Tomcavage J, et al. Reducing long-term cost by transforming primary care: evidence from Geisinger's medical home model. Am J Manag Care 2012 Mar;18(3):149-155. [Medline]
- Ralston JD, Martin DP, Anderson ML, Fishman PA, Conrad DA, Larson EB, et al. Group health cooperative's transformation toward patient-centered access. Med Care Res Rev 2009 Dec;66(6):703-724. [CrossRef] [Medline]
- Herzlinger RE. Why innovation in health care is so hard. Harv Bus Rev 2006 May;84(5):58-66, 156. [Medline]
- Ross J, Stevenson F, Lau R, Murray E. Factors that influence the implementation of e-health: a systematic review of systematic reviews (an update). Implement Sci 2016 Oct 26;11(1):146 [FREE Full text] [CrossRef] [Medline]
- Asthana S, Jones R, Sheaff R. Why does the NHS struggle to adopt eHealth innovations? A review of macro, meso and micro factors. BMC Health Serv Res 2019 Dec 21;19(1):984 [FREE Full text] [CrossRef] [Medline]
- Williams R. Why is it difficult to achieve e-health systems at scale? Information, Communication & Society 2016 Jan 11;19(4):540-550. [CrossRef]
- Callahan D. Chapter 17: Health Care Costs and Medical Technology. In: Crowley M, editor. From Birth to Death and Bench to Clinic: The Hastings Center Bioethics Briefing Book for Journalists, Policymakers, and Campaigns. Garrison, New York: The Hastings Center; 2008:79-82.
- Kumar R. Technology and healthcare costs. Ann Pediatr Cardiol 2011 Jan;4(1):84-86 [FREE Full text] [CrossRef] [Medline]
- Zane RD, Wiler JL. Embracing Technology to Save Primary Care. NEJM Catalyst 2018;4(4). [CrossRef]
- Adler-Milstein J, Holmgren AJ, Kralovec P, Worzala C, Searcy T, Patel V. Electronic health record adoption in US hospitals: the emergence of a digital "advanced use" divide. J Am Med Inform Assoc 2017 Nov 01;24(6):1142-1148 [FREE Full text] [CrossRef] [Medline]
- Staff. 10 Biggest Technological Advancements for Healthcare in the Last Decade. Becker's Health IT. 2015 Sep 17. URL: https://www.beckershospitalreview.com/healthcare-information-technology/10-biggest-technological-advancements-for-healthcare-in-the-last-decade.html [accessed 2019-12-18]
- Dineen-Griffin S, Garcia-Cardenas V, Williams K, Benrimoj SI. Helping patients help themselves: A systematic review of self-management support strategies in primary health care practice. PLoS One 2019 Aug 1;14(8):e0220116 [FREE Full text] [CrossRef] [Medline]
- Maarsingh OR, Henry Y, van de Ven PM, Deeg DJ. Continuity of care in primary care and association with survival in older people: a 17-year prospective cohort study. Br J Gen Pract 2016 Jun 20;66(649):e531-e539. [CrossRef]
- Berenson RA, Burton R. How Solid Is The Primary Care Foundation Of The Medical Home? Health Affairs Blog 2016 Mar 25 [FREE Full text] [CrossRef]
- Pacifico Silva H, Lehoux P, Miller FA, Denis J. Introducing responsible innovation in health: a policy-oriented framework. Health Res Policy Syst 2018 Sep 10;16(1):90 [FREE Full text] [CrossRef] [Medline]
- Carter R, Quesnel-Vallée A, Plante C, Gamache P, Lévesque JF. Effect of family medicine groups on visits to the emergency department among diabetic patients in Quebec between 2000 and 2011: a population-based segmented regression analysis. BMC Fam Pract 2016 Feb 29;17(1):23 [FREE Full text] [CrossRef] [Medline]
- Swinkels ICS, Huygens MWJ, Schoenmakers TM, Oude Nijeweme-D'Hollosy W, van Velsen L, Vermeulen J, et al. Lessons Learned From a Living Lab on the Broad Adoption of eHealth in Primary Health Care. J Med Internet Res 2018 Mar 29;20(3):e83 [FREE Full text] [CrossRef] [Medline]
Abbreviations
EHR: electronic health record |
Edited by G Eysenbach; submitted 13.05.20; peer-reviewed by M Reynolds, S Goff, N Taneja, P Rane, MDG Pimentel, M Rauws, F Palmieri, J Li; comments to author 08.07.20; revised version received 23.07.20; accepted 11.11.20; published 11.01.21
Copyright©Geronimo Jimenez, David Matchar, Choon Huat Gerald Koh, Rianne van der Kleij, Niels H Chavannes, Josip Car. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 11.01.2021.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.