This paper is in the following e-collection/theme issue:

New Methods (924) Telehealth and Telemonitoring (1612) Quality Improvement (235)

Published on in Vol 22, No 6 (2020): June

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/18203, first published .
The Impact of Digital-First Consultations on Workload in General Practice: Modeling Study

The Impact of Digital-First Consultations on Workload in General Practice: Modeling Study

The Impact of Digital-First Consultations on Workload in General Practice: Modeling Study

Authors of this article:

Chris Salisbury1 Author Orcid Image ;   Mairead Murphy1 Author Orcid Image ;   Polly Duncan1 Author Orcid Image
Article Authors Cited by (70) Tweetations (430) Metrics

    Original Paper

    Centre for Academic Primary Care, Department of Population Health Sciences, Bristol Medical School, University of Bristol, Bristol, United Kingdom

    Corresponding Author:

    Chris Salisbury, MSc, MD, FRCGP

    Centre for Academic Primary Care

    Department of Population Health Sciences, Bristol Medical School

    University of Bristol

    Canynge Hall

    39 Whatley Road

    Bristol, BS8 2PS

    United Kingdom

    Phone: 44 1173314530

    Email: c.salisbury@bristol.ac.uk


    Background: Health services in many countries are promoting digital-first models of access to general practice based on offering online, video, or telephone consultations before a face-to-face consultation. It is claimed that this will improve access for patients and moderate the workload of doctors. However, improved access could also potentially increase doctors’ workload.

    Objective: The aim of this study was to explore whether and under what circumstances digital-first access to general practice is likely to decrease or increase general practice workload.

    Methods: A process map to delineate primary care access pathways was developed and a model to estimate general practice workload constructed in Microsoft Excel (Microsoft Corp). The model was populated using estimates of key variables obtained from a systematic review of published studies. A MEDLINE search was conducted for studies published in English between January 1, 2000, and September 30, 2019. Included papers provided quantitative data about online, telephone, or video consultations for unselected patients requesting a general practice in-hours consultation for any problem. We excluded studies of general practitioners consulting specialists, consultations not conducted by doctors, and consultations conducted after hours, in secondary care, in specialist services, or for a specific health care problem. Data about the following variables were extracted from the included papers to form the model inputs: the proportion of consultations managed digitally, the proportion of digital consultations completed without a subsequent consultation, the proportion of subsequent consultations conducted by telephone rather than face-to-face, consultation duration, and the proportion of digital consultations that represent new demand. The outcome was general practice workload. The model was used to test the likely impact of different digital-first scenarios, based on the best available evidence and the plausible range of estimates from the published studies. The model allows others to test the impact on workload of varying assumptions about model inputs.

    Results: Digital-first approaches are likely to increase general practice workload unless they are shorter, and a higher proportion of patients are managed without a subsequent consultation than observed in most published studies. In our base-case scenarios (based on the best available evidence), digital-first access models using online, telephone, or video consultations are likely to increase general practitioner workload by 25%, 3%, and 31%, respectively. An important determinant of workload is whether the availability of digital-first approaches changes the demand for general practice consultations, but there is little robust evidence to answer this question.

    Conclusions: Digital-first approaches to primary care could increase general practice workload unless stringent conditions are met. Justification for these approaches should be based on evidence about the benefits in relation to the costs, rather than assumptions about reductions in workload. Given the potential increase in workload, which in due course could worsen problems of access, these initiatives should be implemented in a staged way alongside careful evaluation.

    J Med Internet Res 2020;22(6):e18203

    doi:10.2196/18203

    Keywords

    general practice; family practice; electronic consultations; remote consultation; telemedicine; telephone consultation; video; access to health care; health care quality, access, and evaluation 


    An increasing number of primary care consultations are being provided under a digital-first model, in which consultations are conducted by telephone, video, email, or online “e-consultation” systems, before offering patients a face-to-face consultation only when necessary. Examples include Doctor On Demand in the United States, Curon in Japan, Ping An Good Doctor in China, and KRY, which operates in several European countries. In England, National Health Service (NHS) policy strongly promotes the use of online consultations [1] and companies such as Babylon GP at Hand, LIVI, and Push Doctor, which offer video consultations free of charge to NHS patients, are expanding rapidly [2]. The introduction of these new access pathways in England will be accelerated by a contract reform framework that will require all general practices to offer online and video consultations by April 2021 and to allow NHS 111 (a national telephone helpline) to directly book face-to-face appointments in local practices [3]. NHS England has recently issued detailed guidance to support the introduction of online consultations [4].

    These changes are justified by two main arguments [1-5]. First, they are designed to facilitate quick and convenient access to care by patients, in line with similar changes in how consumers access almost all other services apart from health care. Second, it is argued that the introduction of “online-first” or “telephone-first” models of access will help to manage workload pressures on general practitioners (GPs). However, the twin aims to improve patient access and to manage workload pressures on GPs are likely to be in tension. Whether digital-first models of care decrease or increase general practice workload depends on factors such as the duration of the initial digital consultation and the proportion of these consultations that result in the patient needing a subsequent face-to-face consultation. The impact on general practice workload also depends on whether the demand for consultations is fixed or related to accessibility [6]. At present, many people have difficulty obtaining GP consultations and some may therefore seek help elsewhere or not obtain any professional advice [7]. If quicker and easier access to care means more people contact general practices, this supply-related demand needs to be considered alongside any efficiency gains.

    The overall impact on workload in general practice therefore depends on the relationship between several variables. We developed a model to estimate the impact of alternative access pathways on general practice workload and populated the model using a systematic review of studies of digital consultations in primary care. The aim of this study was to inform debate about to what extent, and under what circumstances, digital-first primary care consultations are likely to decrease or increase general practice workload.


    Overview

    We developed a process map to delineate the access pathway from when patients first seek a general practice consultation through to obtaining definitive assessment and care (Figure 1). Given the interconnectedness of health care systems, any such model is a simplification and must have a defined scope. Our process map begins with a patient having a health problem and considering requesting a GP consultation. They will often seek advice, which may be from family, a pharmacist, or online through an internet search, an automated symptom-checker app, or a patient forum [7]. Our model begins at the point following this, when a patient actively contacts a general practice requesting a consultation. It therefore excludes administrative issues conventionally dealt with by receptionists (for example, repeat prescriptions), consultations usually managed by nurses (for example, vaccinations), and patients who complete an online application but do not seek contact with a GP. The model includes consultations directly necessitated by a previous step (for example, a face-to-face consultation resulting from an online consultation) but ignores follow-up consultations. The model is limited to the impact on GPs, although we recognize that changes in access to GPs can have consequences for administrative and nursing staff and for demand on other parts of the health service.

    ‎
    Figure 1. Process map: impact of digital first access pathways on general practitioner (GP) workload. E-consult: online consultation. F2F: face-to-face.
    View this figure

    We populated the workload model using data from a systematic review for 3 scenarios, representing the following 3 access pathways:

    • In an online (or e-consultation) first model, patients describe their problem using an electronic form that may involve an automated algorithm or a less structured form. Administrative requests are dealt with by receptionists (excluded by our model). Requests for medical advice are reviewed by a GP who responds with a message or prescription, or a telephone, video, or face-to-face appointment.
    • In a telephone-first model, the GP attempts to resolve the problem by telephone if possible, but if not, invites the patient to a face-to-face consultation.
    • A video-first model follows a similar pattern, but there may be differences in variables such as the length of the consultation and the proportion of contacts that require a subsequent face-to-face consultation.

    Model

    The outcome for the model is percentage change in GP workload using the digital approach compared with GP workload using a conventional approach, in which most patients have face-to-face consultations, but a small proportion have telephone consultations. We created a dynamic spreadsheet model that allowed us to calculate how GP workload changes depending on the values of the key variables in the process map (such as the duration of digital consultations, or supply-related changes in demand). The results from our model are expressed as a percentage, so a result of 10% would mean 10% more hours of work than under conventional care.

    The estimates for the conventional approach came from a large and rigorous study of GP consultations in England [8]. The study showed that 86% of consultations were conducted face-to-face and 14% were conducted by telephone, with a mean duration of 9 minutes and 5 minutes, respectively. The estimates for the new digital approaches came from a systematic review, described below. The results from the model illustrate the impact of a central base-case estimate, based on the best available evidence from our systematic review. For each variable in the model, we also considered plausible upper and lower limit estimates based on outlier studies from the review or on our own informed opinion in the absence of evidence. We have made the dynamic model available online, so that readers can test the effect of their own assumptions and estimates [9].

    Systematic Review

    To include evidence-based estimates in the model, we conducted a systematic review to identify studies of any design that provided quantitative data about digital consultations in primary care, including consultations by telephone, email, e-consultation systems, or video. The quantitative variables of interest were the proportion of consultations managed digitally, completion rate (digital consultations completed without needing a subsequent GP consultation), the proportion of subsequent consultations managed by telephone rather than face-to-face following an online consultation, the duration of different types of consultation, and any indicators of changes in demand or workload after the introduction of a digital-first model.

    A change in the number of requests for health care following improved access is commonly referred to as supply-induced demand [6], which implies that a change in services has caused increased demand. However, in this study we used the more neutral term “supply-related demand” because it could equally represent underlying demand, which becomes visible only once access is improved. We sought to estimate the proportion of digital consultations that represented new supply-related demand based on (per month) the number of contacts of all types after introducing digital consultations, minus the number of contacts before introducing digital consultations, minus the number of duplicate consultations where patients had a face-to-face consultation directly resulting from a digital consultation, divided by the number of digital consultations.

    Our focus was on consultations for undifferentiated problems between a patient and a GP; therefore, we excluded studies of communication between health professionals (for example, GPs consulting specialists), after-hours consultations, specialist or secondary care consultations, consultations not conducted by GPs, studies limited to a specific type of health problem, qualitative studies, studies of patient or GP opinion, and systematic reviews that did not provide any new quantitative analyses beyond the already-included papers. Our search strategy included terms for general practice, family practice, or primary care, or papers published in leading primary care journals, combined with a wide range of terms relating to telephone, online, digital, or video consultations. We included papers identified through the bibliographies of other papers. Multimedia Appendix 1 shows the search strategy, which was conducted in MEDLINE. We restricted searches to papers published since January 1, 2000, in English, and in developed countries to focus on papers of current relevance to the United Kingdom and other similar health care systems. We did not attempt to grade the quality of the studies or assess the risk of bias. The searches were updated to September 30, 2019. CS and PD reviewed the titles and abstracts of papers independently, retaining any papers that were potentially relevant. They then independently reviewed the full text of these papers to identify those that met the inclusion criteria. CS extracted quantitative data about the variables of interest, and this was checked by PD. Disagreements between reviewers were resolved by discussion involving the third author, MM.


    The systematic review identified 1246 papers, of which 90 were judged to be potentially relevant based on their titles and abstracts (Multimedia Appendix 2). Of these, 29 papers provided data of relevance to this study (Multimedia Appendix 3) [8,10-37]. Table 1 shows the estimates for the variables included in the workload model.

    Based on our workload model, the final row in Table 1 shows workload in general practice using our central estimates for each scenario, compared with a conventional pathway based predominantly on face-to-face consultations.

    The dynamic model makes it possible to test the sensitivity of the model to different assumptions by graphically showing the impact on workload (y-axis) of changing any 2 variables simultaneously (x-axis and legend). In Figures 2 and 3, we show 2 different scenarios as examples; readers can use the model to test their own scenarios [9]. Figure 2 shows that if the average duration of a telephone call is 5 minutes, a telephone-first approach has the potential to reduce workload if at least 55% of telephone consultations are completed without needing a subsequent face-to-face consultation, assuming no supply-related increase in demand. Figure 3 demonstrates that an online-first approach could reduce GP workload only if there is minimal increased demand (<4% of online contacts represent new demand) and about 50% of all requests are resolved in one online contact, but it has the potential to substantially increase workload if there is any supply-related increase in demand. However, 2 recent UK studies suggest that only about 30% of online consultation requests are resolved entirely online [13,17], and several studies suggest that there could be a substantial inflation in demand [14,16,26,31].

    Table 1. Alternatives to face-to-face consultation: default values for variables in workload model. All values can be altered in the interactive model to test different scenarios and assumptions. Values without citations are authors’ estimates.
    VariableE-consultationaTelephoneVideo

    		Base caseLower valueUpper valueBase caseLower valueUpper valueBase caseLower valueUpper value
    Access rate: Consultations initially requested in this wayb (%)90b0.01 [17]10093 [12]10 [20]10090b50100
    Completion rate









    		Digital consultations completed without needing a subsequent consultation (%)30 [17]28 [13]70 [33]c52 [25,29]4090 [20]655083 [10]

    		Of those having a real-time consultation after e-consultation, consultations having phone rather than face-to-face consultation (%)46 [17]2090N/AdN/AN/AN/AN/AN/A
    Average time: Average time spent by GPe on this type of consultation (minutes)435 [17]5 [8]4 [11,15]6 [29]9f6 [21]15 [34]
    Supply-related demand: Alternative form consultations that are new demand (%)10–10300 [29]–10300–1030
    Total workload resource compared with conventional care, using base case assumptions (%)25N/AN/A3N/AN/A31N/AN/A

    aE-consultation: online consultation.

    bAt present, usage of e-consultation and video consultation in the United Kingdom is generally very low, so the impact is minimal. For the base cases, we have modeled a scenario in which the use of these alternatives is usual.

    cPenza et al report a 66% completion rate [33]. Completion rates of 70% are claimed by eConsult, cited by Marshall et al [2]. Longman reports similar experiences in practices using askmyGP [38].

    dN/A: Not applicable.

    eGP: general practitioner.

    fAssumed to be similar to conventional face-to-face care.

    ‎
    Figure 2. Impact of telephone consultations on general practitioner workload: varying telephone completion rate and call duration.
    View this figure
    ‎
    Figure 3. Impact of e-consultations on general practitioner workload: varying supply-related demand and e-consultation completion rate. E-consult: online consultation.
    View this figure

    Principal Results

    Current initiatives to improve access and reduce GP workload through digital-first approaches could have benefits for patients and GPs or could have entirely the opposite effect to that intended. Based on current evidence, these approaches are at least as likely to increase as to decrease workload pressure on general practice. There is potential to reduce general practice workload, but only under stringent conditions, whereby the initial assessment is short and a high proportion of contacts are managed without needing a subsequent face-to-face review. Under almost all scenarios, even modest increases in demand related to improved accessibility would lead to increases in workload.

    Comparison With Prior Work

    The estimates used to populate this model are based on the best available published evidence. Companies providing digital-first consultation systems claim that general practices using their systems achieve impressive improvements in access and reductions in workload [38,39], and anecdotal reports from some early adopter practices suggest that these benefits can be achieved [4,40]. However, research studies involving larger numbers of practices show that this is not necessarily generalizable [12,29], and UK health services that have introduced digital-first approaches have reported that they do not appear to save GP time [13,41-43]. Furthermore, 2 studies have suggested that telephone-first approaches that lead to reductions in consultations on the same day are compensated for by increased consultations over the next 28 days [11,28]. Evidence from other countries relates to consultation rates or costs rather than workload and provides conflicting findings, with some studies reporting that offering digital consultations leads to increases (including the online consultations themselves) [14,16,26,30,31], while other studies report reductions [18,22,37]. This implies that it is important to understand how and why digital-first approaches are successful in some circumstances but not others. This study helps to inform this debate.

    Strengths and Limitations

    To ensure comparability between different access pathways, the denominator for our model was patients who decide to seek a consultation with a GP, rather than all patients who use an automated symptom-checker or triage app. About one-third of people look for information online about their symptoms in a conventional consultation system [7], so the use of symptom-checker apps may be partly substituting for this use rather than replacing consultations. The possibility that the availability of an online symptom-checker or triage app could decrease or increase the number of requests for contact with a GP is taken into account using the supply-related demand parameter in our model. Although some providers of online consultation services report that a high proportion of contacts can be managed entirely online [38,39], it is important to consider the denominator, since some of these contacts might not have been made at all or would not have led to a consultation under conventional care. These effects can be considered by comparing the total number of online, telephone, and face-to-face contacts with the number of consultations under a previous conventional approach (see formula in “Systematic Review” in the Methods section).

    Some of the estimates used in this model could be challenged because they depend on the context in which the access pathway is introduced. For example, some of the estimates in the model come from the ESTEEM trial of telephone triage of requests for same-day consultations rather than all consultations [11,12]. Second, some clinicians suggest that the duration of a face-to-face consultation might be shorter after a prior online or telephone assessment than under conventional care [19] (although the evidence suggests this is not the case) [23,29]. Third, the proportion of consultations that can be successfully completed online or by telephone might be higher when patients are given the choice to consult in this way, rather than in systems where all patients have to go through this step before accessing a face-to-face consultation. A strength of this study is that by making our model freely available, anyone can explore how workload varies under different assumptions.

    The need to limit the scope of the workload model means that it does not consider the effect on other services. The availability of digital consultations could be efficient if it reduces consultations in hospital emergency departments, but the evidence so far provides little support for this hypothesis [12,29,34,44]. The use of digital-first approaches could reduce GP workload by directing patients needing face-to-face care to other primary care professionals, such as nurses and pharmacists. However, we were unable to include this in the model because of a lack of evidence about how much this delegation occurs (compared with the extent to which receptionists direct patients to these professionals under conventional appointment systems) and the proportion of patients that would need to be transferred back to a GP after a nurse or pharmacist consultation. It will be increasingly important to consider the impact on nonmedical workloads in general practice as roles and responsibilities evolve. It is important to note that although delegation to other staff may reduce GP workload, it is not necessarily more efficient for the health care service overall [45].

    Finally, we recognize that this study was designed from the perspective of the NHS in the United Kingdom, and usual care is different in other countries. However, by making the workload model freely available, including allowing changes to estimates such as the duration of face-to-face consultations under usual care, we hope that our model will be useful in various settings.

    Implications for Clinicians and Policy Makers

    If initiatives to improve access to care do lead to increased GP workload, this is not necessarily an argument against them. Ensuring good access to health care is a core purpose of primary care, and additional consultations might represent a response to previously unmet need. However, these initiatives should be justified on the grounds of benefits to patients rather than claims about reductions in GP workload [46]. As with most medical interventions, the key issue is whether the additional benefits are justified in relation to any extra costs. Furthermore, our analysis shows that assumptions about efficiency savings may be misplaced and general practice may need more resources to implement digital-first pathways.

    It is important to consider how the benefits of different access pathways are felt by different segments of the population. Digital consultations are predominantly used by patients in the 20-44 age group [17,34], which is a group with generally fewer health care needs. If improving access for them requires more GP time, this will decrease rather than increase the time available for patients with more complex problems, as well as decreasing access for those without internet access. To reduce the potential for worsening health inequalities, it is important to prioritize the use of technology to improve access for the groups of patients with the greatest health care needs, such as older adults, carers, and people with disabilities.

    Digital-first access models may have other potential advantages and disadvantages. Improved access could help reverse the decline in public satisfaction with NHS general practice [47] and help avoid inappropriate use of expensive hospital care [3]. Triage systems may offer GPs a greater sense of control over their working day [4]. Technologies to allow GPs to work from home could expand the workforce by unlocking the potential contribution of doctors who cannot work fixed hours in conventional settings [34]. On the other hand, a shift in working patterns toward significant amounts of time spent consulting online or by telephone could lead more GPs to leave the workforce than to join it.

    Apart from the impact on workload, there are very important unknowns about the quality and safety of alternatives to face-to-face consultations, as well as the acceptability of these access pathways to different patient groups [2]. These questions should be a priority for research.

    Conclusions

    This study has highlighted that efficiency gains or losses from the use of digital-first access pathways are finely balanced, and the main impact on workload will be determined by whether these pathways change demand. Digital-first services could increase demand through improved supply or surfacing previously unmet need, or could reduce demand by encouraging patients to self-care or use other services. This is, therefore, an issue of critical importance, but about which we currently have the least evidence. It may take several years for these effects to become manifest. Given that it will be difficult to lower expectations and demand after these have been raised, this suggests the need for careful and staged implementation alongside evaluation rather than universal implementation of digital-first access pathways as soon as possible, as advocated by current UK policy [3].

    Acknowledgments

    CS is a senior investigator at the National Institute for Health Research. The views expressed in this article are those of the authors, not necessarily those of the National Institute for Health Research or the Department of Health and Social Care.

    CS had the idea for the paper, reviewed papers, extracted data, developed the first draft of the workload calculator, and wrote the first draft of this paper. MM devised the first draft of the process map for the access pathway and contributed to development of the interactive workload calculator. PD reviewed papers and extracted data for the systematic review. All authors provided critical intellectual input and approved the final paper. CS is guarantor and affirms that the manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as originally planned have been explained.

    Conflicts of Interest

    None declared.

    ‎
    Multimedia Appendix 1

    Search strategy for literature in MEDLINE (Ovid SP).

    DOCX File , 18 KB
    ‎
    Multimedia Appendix 2

    PRISMA Flow Diagram.

    DOCX File , 38 KB
    ‎
    Multimedia Appendix 3

    Data about model inputs from relevant papers.

    DOCX File , 34 KB
    1. NHS England. 2020. Online Consultation Funding   URL: https://www.england.nhs.uk/gp/digital-first-primary-care/online-consultation-funding/ [accessed 2020-05-22]
    2. Marshall M, Shah R, Stokes-Lampard H. Online consulting in general practice: making the move from disruptive innovation to mainstream service. BMJ 2018 Mar 26;360:k1195. [CrossRef] [Medline]
    3. NHS England. 2019 Jan 31. Investment and evolution: A five-year framework for GP contract reform to implement The NHS Long Term Plan   URL: https://www.england.nhs.uk/wp-content/uploads/2019/01/gp-contract-2019.pdf [accessed 2020-05-19]
    4. Bakhai M, Croney L, Waller O, Henshall N, Felstead C. NHS England. 2019 Sep 26. Using online consultations in primary care: implementation toolkit   URL: https:/​/www.​england.nhs.uk/​publication/​using-online-consultations-in-primary-care-implementation-toolkit/​ [accessed 2020-05-22]
    5. NHS England. 2016. General Practice Forward View   URL: https://www.england.nhs.uk/wp-content/uploads/2016/04/gpfv.pdf [accessed 2020-05-19]
    6. Rosen R. Nuffield Trust. 2014 Jun 30. Meeting need or fuelling unnecessary demand? Understanding the impact of improved access to primary care   URL: https:/​/www.​nuffieldtrust.org.uk/​research/​meeting-need-or-fuelling-unnecessary-demand-understanding-the-impact-of-improved-access-to-primary-care#partners [accessed 2020-05-19]
    7. NHS England. 2019 Jul. GP Patient Survey   URL: http://www.gp-patient.co.uk/surveysandreports [accessed 2020-05-19]
    8. Hobbs FDR, Bankhead C, Mukhtar T, Stevens S, Perera-Salazar R, Holt T, et al. Clinical workload in UK primary care: a retrospective analysis of 100 million consultations in England, 2007-14. Lancet 2016 Jun 04;387(10035):2323-2330 [FREE Full text] [CrossRef] [Medline]
    9. Salisbury C, Murphy M. University of Bristol. 2020 May. Calculate the impact of online and phone consultations on GP workload   URL: http://www.bristol.ac.uk/primaryhealthcare/resources/wlc/ [accessed 2020-05-22]
    10. Brunett PH, DiPiero A, Flores C, Choi D, Kum H, Girard DE. Use of a voice and video internet technology as an alternative to in-person urgent care clinic visits. J Telemed Telecare 2015 Jun;21(4):219-226. [CrossRef] [Medline]
    11. Campbell JL, Fletcher E, Britten N, Green C, Holt T, Lattimer V, et al. The clinical effectiveness and cost-effectiveness of telephone triage for managing same-day consultation requests in general practice: a cluster randomised controlled trial comparing general practitioner-led and nurse-led management systems with usual care (the ESTEEM trial). Health Technol Assess 2015 Feb;19(13):1-212, vii [FREE Full text] [CrossRef] [Medline]
    12. Campbell JL, Fletcher E, Britten N, Green C, Holt TA, Lattimer V, et al. Telephone triage for management of same-day consultation requests in general practice (the ESTEEM trial): a cluster-randomised controlled trial and cost-consequence analysis. Lancet 2014 Nov 22;384(9957):1859-1868 [FREE Full text] [CrossRef] [Medline]
    13. Carter M, Fletcher E, Sansom A, Warren FC, Campbell JL. Feasibility, acceptability and effectiveness of an online alternative to face-to-face consultation in general practice: a mixed-methods study of webGP in six Devon practices. BMJ Open 2018 Feb 15;8(2):e018688 [FREE Full text] [CrossRef] [Medline]
    14. Chen C, Garrido T, Chock D, Okawa G, Liang L. The Kaiser Permanente Electronic Health Record: transforming and streamlining modalities of care. Health Aff (Millwood) 2009;28(2):323-333. [CrossRef] [Medline]
    15. de Groot RA, de Haan J, Bosveld HEP, Nijland A, Meyboom-de Jong B. The implementation of a call-back system reduces the doctor's workload, and improves accessibility by telephone in general practice. Fam Pract 2002 Oct;19(5):516-519. [CrossRef] [Medline]
    16. Dexter EN, Fields S, Rdesinski RE, Sachdeva B, Yamashita D, Marino M. Patient-Provider Communication: Does Electronic Messaging Reduce Incoming Telephone Calls? J Am Board Fam Med 2016;29(5):613-619 [FREE Full text] [CrossRef] [Medline]
    17. Edwards HB, Marques E, Hollingworth W, Horwood J, Farr M, Bernard E, et al. Use of a primary care online consultation system, by whom, when and why: evaluation of a pilot observational study in 36 general practices in South West England. BMJ Open 2017 Nov 22;7(11):e016901 [FREE Full text] [CrossRef] [Medline]
    18. Ekman B. Cost Analysis of a Digital Health Care Model in Sweden. Pharmacoecon Open 2018 Sep;2(3):347-354 [FREE Full text] [CrossRef] [Medline]
    19. Farr M, Banks J, Edwards HB, Northstone K, Bernard E, Salisbury C, et al. Implementing online consultations in primary care: a mixed-method evaluation extending normalisation process theory through service co-production. BMJ Open 2018 Mar 19;8(3):e019966 [FREE Full text] [CrossRef] [Medline]
    20. Gonzalez F, Cimadevila B, Garcia-Comesaña J, Cerqueiro S, Andion E, Prado J, et al. Telephone consultation in primary care. J Health Organ Manag 2018 Apr 09;32(2):321-337. [CrossRef] [Medline]
    21. Hammersley V, Donaghy E, Parker R, McNeilly H, Atherton H, Bikker A, et al. Comparing the content and quality of video, telephone, and face-to-face consultations: a non-randomised, quasi-experimental, exploratory study in UK primary care. Br J Gen Pract 2019 Sep;69(686):e595-e604 [FREE Full text] [CrossRef] [Medline]
    22. Bergmo TS, Kummervold PE, Gammon D, Dahl LB. Electronic patient-provider communication: will it offset office visits and telephone consultations in primary care? Int J Med Inform 2005 Sep;74(9):705-710. [CrossRef] [Medline]
    23. Holt TA, Fletcher E, Warren F, Richards S, Salisbury C, Calitri R, et al. Telephone triage systems in UK general practice: analysis of consultation duration during the index day in a pragmatic randomised controlled trial. Br J Gen Pract 2016 Mar;66(644):e214-e218 [FREE Full text] [CrossRef] [Medline]
    24. Huygens MWJ, Swinkels ICS, Verheij RA, Friele RD, van Schayck OCP, de Witte LP. Understanding the use of email consultation in primary care using a retrospective observational study with data of Dutch electronic health records. BMJ Open 2018 Jan 21;8(1):e019233 [FREE Full text] [CrossRef] [Medline]
    25. Jiwa M, Mathers N, Campbell M. The effect of GP telephone triage on numbers seeking same-day appointments. Br J Gen Pract 2002 May;52(478):390-391 [FREE Full text] [Medline]
    26. Katz SJ, Moyer CA, Cox DT, Stern DT. Effect of a triage-based E-mail system on clinic resource use and patient and physician satisfaction in primary care: a randomized controlled trial. J Gen Intern Med 2003 Sep;18(9):736-744 [FREE Full text] [CrossRef] [Medline]
    27. McKinstry B, Hammersley V, Burton C, Pinnock H, Elton R, Dowell J, et al. The quality, safety and content of telephone and face-to-face consultations: a comparative study. Qual Saf Health Care 2010 Aug;19(4):298-303. [CrossRef] [Medline]
    28. McKinstry B, Walker J, Campbell C, Heaney D, Wyke S. Telephone consultations to manage requests for same-day appointments: a randomised controlled trial in two practices. Br J Gen Pract 2002 Apr;52(477):306-310 [FREE Full text] [Medline]
    29. Newbould J, Abel G, Ball S, Corbett J, Elliott M, Exley J, et al. Evaluation of telephone first approach to demand management in English general practice: observational study. BMJ 2017 Sep 27;358:j4197 [FREE Full text] [CrossRef] [Medline]
    30. North F, Crane SJ, Chaudhry R, Ebbert JO, Ytterberg K, Tulledge-Scheitel SM, et al. Impact of patient portal secure messages and electronic visits on adult primary care office visits. Telemed J E Health 2014 Mar;20(3):192-198 [FREE Full text] [CrossRef] [Medline]
    31. Palen TE, Ross C, Powers JD, Xu S. Association of online patient access to clinicians and medical records with use of clinical services. JAMA 2012 Nov 21;308(19):2012-2019. [CrossRef] [Medline]
    32. Peleg R. Off-the-cuff cellular phone consultations in a family practice. J R Soc Med 2001 Jun;94(6):290-291 [FREE Full text] [CrossRef] [Medline]
    33. Penza KS, Murray MA, Pecina JL, Myers JF, Furst JW. Electronic Visits for Minor Acute Illnesses: Analysis of Patient Demographics, Prescription Rates, and Follow-Up Care Within an Asynchronous Text-Based Online Visit. Telemed J E Health 2018 Mar;24(3):210-215. [CrossRef] [Medline]
    34. Quigley A, Hex N, Aznar C, Salisbury C. NHS Hammersmith and Fulham CCG and NHS England. 2019 May. Evaluation of Babylon GP at hand: Final evaluation report   URL: https:/​/www.​hammersmithfulhamccg.nhs.uk/​media/​156123/​Evaluation-of-Babylon-GP-at-Hand-Final-Report.​pdf [accessed 2020-05-19]
    35. Richards DA, Meakins J, Tawfik J, Godfrey L, Dutton E, Richardson G, et al. Nurse telephone triage for same day appointments in general practice: multiple interrupted time series trial of effect on workload and costs. BMJ 2002 Nov 23;325(7374):1214 [FREE Full text] [CrossRef] [Medline]
    36. Stevens S, Bankhead C, Mukhtar T, Perera-Salazar R, Holt TA, Salisbury C, NIHR School for Primary Care Research. Patient-level and practice-level factors associated with consultation duration: a cross-sectional analysis of over one million consultations in English primary care. BMJ Open 2017 Nov 16;7(11):e018261 [FREE Full text] [CrossRef] [Medline]
    37. Zhou YY, Garrido T, Chin HL, Wiesenthal AM, Liang LL. Patient access to an electronic health record with secure messaging: impact on primary care utilization. Am J Manag Care 2007 Jul;13(7):418-424 [FREE Full text] [Medline]
    38. Longman H. askmyGP. 2017 Oct 08. Cut your workload by 10%, without fail   URL: https://askmygp.uk/cut-workload-10-without-fail/ [accessed 2020-05-19]
    39. eConsult is the most used online consultation platform by NHS GP practices in the UK. eConsult   URL: https://econsult.net/primary-care/ [accessed 2020-05-19]
    40. NHS England. Releasing time for care   URL: https://www.england.nhs.uk/gp/gpfv/redesign/gpdp/releasing-time/ [accessed 2020-05-19]
    41. Banks J, Farr M, Salisbury C, Bernard E, Northstone K, Edwards H, et al. Use of an electronic consultation system in primary care: a qualitative interview study. Br J Gen Pract 2018 Jan;68(666):e1-e8 [FREE Full text] [CrossRef] [Medline]
    42. Patel A. Pulse. 2019 Aug 21. No evidence that online consultations save time, says CCG   URL: http:/​/www.​pulsetoday.co.uk/​news/​all-news/​no-evidence-that-online-consultations-save-time-says-ccg/​20039267.​article [accessed 2020-05-19]
    43. Lawless M, Wright E, Davidson J. A collaborative approach to improving patient access in general practice: impact of three different pilot schemes in 12 general practices in Greenwich. London J Prim Care (Abingdon) 2016;8(4):56-65 [FREE Full text] [CrossRef] [Medline]
    44. Bunn F, Byrne G, Kendall S. The effects of telephone consultation and triage on healthcare use and patient satisfaction: a systematic review. Br J Gen Pract 2005 Dec;55(521):956-961 [FREE Full text] [Medline]
    45. Hollinghurst S, Horrocks S, Anderson E, Salisbury C. Comparing the cost of nurse practitioners and GPs in primary care: modelling economic data from randomised trials. Br J Gen Pract 2006 Jul;56(528):530-535 [FREE Full text] [Medline]
    46. Downey A. Digital Health. Digital solutions 'should be encouraged' to ease pressure on GPs   URL: https://www.digitalhealth.net/2019/08/digital-solutions-ease-pressure-gps/ [accessed 2020-05-19]
    47. Robertson R, Appleby J, Evans H. Nuffield Trust. 2018 Feb. Public satisfaction with the NHS and social care in 2017: Results and trends from the British Social Attitudes survey   URL: https://www.nuffieldtrust.org.uk/files/2018-02/nut-kf-bsa-2018-web.pdf [accessed 2020-05-19]

    ‎
    GP: general practitioner

    Edited by G Eysenbach; submitted 11.02.20; peer-reviewed by M Carter, F North; comments to author 20.03.20; revised version received 26.03.20; accepted 28.03.20; published 16.06.20

    Copyright

    ©Chris Salisbury, Mairead Murphy, Polly Duncan. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 16.06.2020.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.