Viewpoint
Abstract
This article aims to introduce emerging measurement domains made feasible through the electronic health record (EHR) use metadata, to inform the changing landscape of health care delivery. We reviewed emerging domains in which EHR metadata may be used to measure health care delivery, outlining a framework for evaluating measures based on desirability, feasibility, and viability. We argue that EHR use metadata may be leveraged to develop and operationalize novel measures in the domains of team structure and dynamics, workflows, and cognitive environment to provide a clearer understanding of modern health care delivery. Examples of measures feasible using metadata include quantification of teamwork and collaboration, patient continuity measures, workflow conformity measures, and attention switching. By enabling measures that can be used to inform the next generation of health care delivery, EHR metadata may be used to improve the quality of patient care and support clinician well-being. Careful attention is needed to ensure that these measures are desirable, feasible, and viable.
J Med Internet Res 2025;27:e64721doi:10.2196/64721
Keywords
Introduction
Since the widespread adoption of electronic health records (EHRs) following the enactment of the Health Information Technology for Economic and Clinical Health (HITECH) Act, clinical care is increasingly mediated by health information technology (HIT) []. The digitization of health care has enabled not only the capture of more clinical data for use in clinical research [], but also metadata about how those data are produced and used, giving a window into the process of health care delivery [-].
Established health care metrics focus largely on the patient process and outcome measurements (eg, health maintenance screening and preventable admissions) or outcomes for individual health care workers (eg, turnover and professional burnout). While these measures reflect important outcomes of a complex health care system, they do not reliably indicate upstream aspects and contextual factors of health care delivery, which may be leveraged to improve patient and clinician outcomes. EHR use metadata (eg, audit logs, orders metadata, documentation and communication metadata, and patient encounters metadata) contain valuable details on the complex system in which health care is delivered, with early insights primarily focused on discrete action or time-based measures [-].
This paper describes 3 examples of emerging domains of measurement through EHR use metadata, identified through a series of workshops facilitated by the Measures Workgroup of the National Research Network for EHR Audit Log Data []. These domains, health care teams, workflows, and cognitive environments, may provide a clearer understanding of modern health care delivery. We expect the value of these measures to outweigh the resources required to develop them, informing the next generation of health care delivery that improves both the quality of patient care and supports clinician well-being.
Data and Data Sources
Traditional sources of health care delivery data (eg, time and motion studies, surveys, or administrative claims) [,] often require high resource expenditure to capture accurately and in detail. In contrast, continuously collected EHR use metadata can facilitate widespread measurement of several domains of health care delivery [,,], providing valuable data that otherwise would require direct observations that are virtually impossible to collect at scale or for extended durations [].
At a more granular level, EHR audit logs and other event logs offer a detailed account of actions that took place within the EHR at what time and by whom [], enabling exploration of the “path” that a clinician or team took in the EHR to complete that task. In turn, these details can be used to delineate certain workflows or team structures contributing to task efficiency and clinical outcomes, which may be the result of vendor-derived or investigator-derived measures []. Such approaches have been described for a variety of EHR vendors, clinical scenarios [], methodologies [], and health professions [].
Although metadata are not identical across EHR vendors, and institutions often have differing customizations and third-party modules, commonalities among metadata structures make it feasible to normalize measures across institutions with sufficient considerations. In addition, recent studies from 2 of the largest EHR vendors suggest that clinician variation far exceeds organizational variation [,], indicating that metadata-derived EHR use measures are more reflective of individual workflows than organizational configurations. We posit here that these new horizons of measurement warrant dedicated development and prioritization of the key clinical and operational questions for which EHR use metadata can be brought to bear.
Principles of Domain Measurement
Innovative ideas need to be feasible, viable, and desirable to be successful []. While originally developed for commercial products, this framework can also guide EHR measure development. First, EHR use metadata to enable new measures of health care delivery that are technically feasible but also require proof that such measures are valid and reliable. Second, analyzing EHR metadata takes less time and money than many other methods of observing health care delivery, but measurement schemes based on EHR metadata still need to prove their viability (providing more value than the effort required to produce them) by generating insights that tangibly improve care []. Finally, measures need to be desirable, aligning with human values and providing insight into phenomena people care about. Desirable measures may answer research questions, contribute to more efficient and reliable single-center cross-sectional or longitudinal operational and quality reporting [], or facilitate multi-institutional comparisons. This perspective demonstrates that, in health care delivery, these phenomena include not only how people spend their time but also how they interact (team structure and dynamics), organize their work (workflows), and direct their attention (cognitive environment; 
Figure 1) [,,-].
Making decisions based on social measures is fraught with risk, with a natural tendency to focus on what is easiest to measure (feasible) rather than what is most useful (viable) or meaningful (desirable) as an indicator of high-quality processes or outcomes []. Yet, most measures are only rough proxies for more complex phenomena, and improving the measure (eg, patient satisfaction scores) may not improve the phenomenon of interest (eg, patient satisfaction) []. Furthermore, poor selection of a proxy can promote bias (eg, using predicted health care costs as a proxy for clinical risk underestimates the needs of those with less access to care []), and measures used for social decision-making are more likely to be gamed [].
These risks do not require that we avoid measuring health care delivery. As statistician George Box observed, “Essentially, all models are wrong, but some are useful” []. Developing useful measures while acknowledging the ways in which they are wrong requires careful attention to the social aspects of what cannot be easily measured and the unintended consequences of putting such measures into practice []. Here we describe 3 domains of health care delivery where EHR use metadata measures may be feasible, viable, and desirable, though not without risk.
Measuring Team Structure and Dynamics
Most patient care is delivered through teams of nurses, medical assistants, technicians, pharmacists, physicians, and many other staff members all of whom are integral to care delivery and outcomes. Fortunately, a byproduct of EHR auditing requirements means that EHR use metadata record details on both the clinicians and patients for whom they perform actions, making it feasible to reconstruct team structures and dynamics. This can be viewed as (1) clinician-centric models of health care provider teams, and (2) patient-centric models of clinicians involved in their care (synchronously and not), recognizing that each feeds back on the other through complex relationships. Identifying these team structures relies on inference based on shared patient access, physical locations of EHR access, temporal proximity, or direct communication records [,].
Health Care Provider Teams
EHR uses metadata in relation to defined team structures to allow investigation into patterns of how health care teams interact with each other. The “strength” of team structure can then be weighted (eg, through repeat interactions) to quantify team familiarity that might promote shared mental models that are essential for effective teamwork.
Researchers have the opportunity to consider the timing, intensity, and content of team-based interactions as proxies for effective teamwork. This requires a robust understanding of how EHR-based teamwork takes place in a broader framework of in-person and HIT-mediated collaboration, allowing for variance in the phenotypes of team collaboration that can reliably still be associated with provider satisfaction and patient care quality []. This work also requires the development of outcome measures more sensitive to the quality of team collaboration, such as care delays or therapy cycling. Valid and reliable understanding of the connections between team structure, processes, and outcomes is foundational to each measure’s viability and desirability, insofar as it supports health care managers in making operational decisions and designing institutional supports that enhance team collaboration and quality of care [].
Clinician-Patient Teams
The clinician-patient relationship is the cornerstone of health care provision, yet many essential measures of this relationship have been prohibitively difficult to measure at scale before the emergence of EHR use metadata. For example, continuity of care is associated with patient satisfaction and a host of improved outcomes [,], attributed to the trust and knowledge in the clinician-patient relationship that supports tailored evaluations, accurate diagnosis, customized treatment plans, and increased medication adherence. EHR use of metadata has made more nuanced measurements in this domain feasible because system-wide records of all clinician-patient interactions now enable quantifying repeated interactions with the same clinician at a more granular level than visits, procedures, or diagnoses. These records include asynchronous messages through patient portals and documentation of telephone encounters. With the addition of domain knowledge, these detailed metadata also enable assessments of the appropriateness of a patient’s encounters with primary care versus subspecialty care. These concepts of continuity of care [-] and comprehensiveness [,,] are particularly viable and desirable as they may help physicians and operational leaders optimize care to achieve the quintuple aim [,] goals of quality, cost, satisfaction, patient and care team experience, and equity.
Through improved measures and stakeholder engagement, health care systems can work to optimize the relationship between clinicians and patients, to promote health care that is not only more effective and efficient but also more satisfying and rewarding for both clinicians and patients.
Health care team structure and dynamics are complex and not fully reflected within EHR use metadata, raising important limitations. The balance between HIT-facilitated and in-person interactions varies among settings, with important implications for the validity of extrapolating teamwork measures from EHR use metadata within a particular use case. In addition, value judgments on the appropriateness or quality of a particular team structure or clinician-patient interaction require additional clinical and system knowledge, much of which is not readily available within EHR data or metadata.
Measuring Workflows
EHR audit logs and other event logs contain records of timestamped granular interactions with an EHR as part of larger clinical tasks (eg, refilling a prescription or ordering bloodwork), making feasible large-scale identification of common pathways for routine clinical practices []. These pathways may then be evaluated for relative time spent on clinical tasks, the complexity of action sequences, engagement with decision support, variation for particular workflows, specialties, providers, or combinations of these (eg, variation in outpatient ordering workflow among physicians within 1 clinic) [].
To be viable and desirable, workflow measures must account for the tension between unnecessary and necessary deviations of behaviors from the norm []. Reducing unnecessary deviations can improve efficiency and patient safety through standardized best practices [,]. For example, heterogeneity in physician work with medical scribes may negatively impact physician efficiency []. While highly variable stroke care pathways may result in therapeutic delays []. Yet, enabling necessary deviations from the norm may promote tailored approaches to unique or complex scenarios, promoting clinician autonomy, professional satisfaction, and precision medicine.
Key to this measurement domain is the risk of misclassifying an unhelpful workflow as “normative” or a necessary variation as “deviant,” highlighting the importance of establishing clinical expertise, stakeholder buy-in, and relationships to important outcomes in the development and deployment of any workflow measures. In addition, although efforts at automated task identification within audit log data have been made [,,], the diversity of clinical workflows makes reliable task identification challenging. Furthermore, due to differences in actions and tracking methods available in particular EHR instances, such measures may be poorly generalizable outside of an institution or EHR vendor unless carefully designed to be context-agnostic and may require recalibration over time with new technologies (eg, large language models and generative artificial intelligence), clinical approaches, or software functionality.
Measuring Cognitive Environment
Clinical work is cognitively complex; clinicians must synthesize ever increasing amounts of information to make diagnosis and treatment decisions amidst competing demands for their attention. Cognitive overload, which has been linked to errors and burnout, can occur easily in such environments [,]. Because clinical work is largely mediated by the EHR [], EHR use metadata offers opportunities to measure clinician cognitive load at scale without laborious data collection. These measurements are critical for understanding how the clinical cognitive environment can be improved.
Cognitive load includes an intrinsic component related to the complexity of a particular task (eg, patient complexity), and an extrinsic component related to the work environment (eg, interruptions, poor EHR, or workplace design []). Both components may be feasible to measure using EHR use metadata. For example, intrinsic cognitive load can be approximated using audit log–derived measures of patient load [,], EHR time [], and encounter or task complexity []. Extrinsic cognitive load is more challenging to measure; while some interruptions like secure messaging use can be observed directly within domain-specific or third-party event logs [], many occur outside of HIT. Instead, efforts have focused on capturing task fragmentation [,], such as attention switching [] or undivided attention [,,,,-].
While this domain has considerable promises, there are several barriers to progress. Few of the audit log-derived cognitive load measures have been validated using established instruments from the cognitive science literature (eg, surveys like the NASA-TLX [National Aeronautics and Space Administration–Task Load Index] or physiological measurements like pupillometry) []. In addition, difficulties with the related concept of task identification limit the ability to subsequently measure task fragmentation, with further validation of task identification needed before task fragmentation can be reliably measured.
Conclusion
EHR uses metadata to provide remarkable potential for measuring and improving health care delivery by making feasible previously inaccessible insights. By measuring team structure and dynamics using intersecting metadata on clinician EHR use, health care provider teams may be studied and optimized and clinician-patient teams can be improved to prioritize continuity of care and comprehensiveness where appropriate. Health care workflows may be better understood through better identification of tasks and measurement of paths taken to achieve them. Finally, proxies of cognitive load may be measured, to inform interventions to reduce overload and the resultant risks for burnout and medical errors. Each of these measurement domains will require further tuning to fully realize their promise of improving health care delivery through sustainably scalable measurement.
Acknowledgments
This work was funded in part by a grant (K08HS027837) from the Agency for Health care Research and Quality and an EHR (electronic health record) Use Metrics Research Grant from the American Medical Association. The views expressed in this paper do not reflect those of the funding agencies. They had no role in writing or reviewing the work.
Conflicts of Interest
Adam Rule reports receiving grants, honoraria, and travel support from the American Medical Association outside the reported work.
References
- Washington V, DeSalvo K, Mostashari F, Blumenthal D. The HITECH era and the path forward. N Engl J Med. 2017;377(10):904-906. [CrossRef] [Medline]
- Hripcsak G, Albers DJ. Next-generation phenotyping of electronic health records. J Am Med Inform Assoc. 2013;20(1):117-121. [FREE Full text] [CrossRef] [Medline]
- Tawfik D, Bayati M, Liu J, Nguyen L, Sinha A, Kannampallil T, et al. Predicting primary care physician burnout from electronic health record use measures. Mayo Clin Proc. 2024;99(9):1411-1421. [FREE Full text] [CrossRef] [Medline]
- Adler-Milstein J, Adelman JS, Tai-Seale M, Patel VL, Dymek C. EHR audit logs: a new goldmine for health services research? J Biomed Inform. 2020;101:103343. [FREE Full text] [CrossRef] [Medline]
- Adler-Milstein J, Zhao W, Willard-Grace R, Knox M, Grumbach K. Electronic health records and burnout: time spent on the electronic health record after hours and message volume associated with exhaustion but not with cynicism among primary care clinicians. J Am Med Inform Assoc. 2020;27(4):531-538. [FREE Full text] [CrossRef] [Medline]
- Kannampallil T, Adler-Milstein J. Using electronic health record audit log data for research: insights from early efforts. J Am Med Inform Assoc. 2022;30(1):167-171. [FREE Full text] [CrossRef] [Medline]
- Sinsky C, Rule A, Cohen G, Arndt B, Shanafelt T, Sharp C, et al. Metrics for assessing physician activity using electronic health record log data. J Am Med Inform Assoc. 2020;27(4):639-643. [FREE Full text] [CrossRef] [Medline]
- Eschenroeder HC, Manzione LC, Adler-Milstein J, Bice C, Cash R, Duda C, et al. Associations of physician burnout with organizational electronic health record support and after-hours charting. J Am Med Inform Assoc. 2021;28(5):960-966. [FREE Full text] [CrossRef] [Medline]
- Bartek B, Lou SS, Kannampallil T. Measuring the cognitive effort associated with task switching in routine EHR-based tasks. J Biomed Inform. 2023;141:104349. [CrossRef] [Medline]
- Dyrbye L, Gordon J, O'Horo J, Belford S, Wright M, Satele D, et al. Relationships between EHR-based audit log data and physician burnout and clinical practice process measures. Mayo Clin Proc. 2023;98(3):398-409. [CrossRef] [Medline]
- Lou SS, Lew D, Harford DR, Lu C, Evanoff BA, Duncan JG, et al. Temporal associations between EHR-derived workload, burnout, and errors: a prospective cohort study. J Gen Intern Med. 2022;37(9):2165-2172. [FREE Full text] [CrossRef] [Medline]
- Rule A, Chiang MF, Hribar MR. Using electronic health record audit logs to study clinical activity: a systematic review of aims, measures, and methods. J Am Med Inform Assoc. 2020;27(3):480-490. [FREE Full text] [CrossRef] [Medline]
- Rule A, Melnick ER, Apathy NC. Using event logs to observe interactions with electronic health records: an updated scoping review shows increasing use of vendor-derived measures. J Am Med Inform Assoc. 2022;30(1):144-154. [FREE Full text] [CrossRef] [Medline]
- Magon HS, Helkey D, Shanafelt T, Tawfik D. Creating conversion factors from EHR event log data: a comparison of investigator-derived and vendor-derived metrics for primary care physicians. AMIA Annu Symp Proc. 2023;2023:1115-1124. [FREE Full text] [Medline]
- Sinha A, Stevens LA, Su F, Pageler NM, Tawfik DS. Measuring electronic health record use in the pediatric ICU using audit-logs and screen recordings. Appl Clin Inform. 2021;12(4):737-744. [FREE Full text] [CrossRef] [Medline]
- Center CIaIR. National research network for EHR audit log data. University of California San Francisco. URL: https://cliir.ucsf.edu/portfolio/national-research-network-ehr-audit-log-data [accessed 2024-07-17]
- Steinwachs DM, Hughes RG. Health services research: scope and significance. In: Patient Safety and Quality: An Evidence-Based Handbook for Nurses. Rockville. Agency for Healthcare Research and Quality (US); 2008.
- Horwitz LI, Partovian C, Lin Z, Grady JN, Herrin J, Conover M, et al. Development and use of an administrative claims measure for profiling hospital-wide performance on 30-day unplanned readmission. Ann Intern Med. 2014;161(10 Suppl):S66-S75. [FREE Full text] [CrossRef] [Medline]
- Baxter SL, Apathy NC, Cross DA, Sinsky C, Hribar MR. Measures of electronic health record use in outpatient settings across vendors. J Am Med Inform Assoc. 2021;28(5):955-959. [FREE Full text] [CrossRef] [Medline]
- Rule A, Kannampallil T, Hribar MR, Dziorny AC, Thombley R, Apathy NC, et al. Guidance for reporting analyses of metadata on electronic health record use. J Am Med Inform Assoc. 2024;31(3):784-789. [CrossRef] [Medline]
- Rose C, Thombley R, Noshad M, Lu Y, Clancy HA, Schlessinger D, et al. Team is brain: leveraging EHR audit log data for new insights into acute care processes. J Am Med Inform Assoc. 2022;30(1):8-15. [FREE Full text] [CrossRef] [Medline]
- Lou SS, Liu H, Harford D, Lu C, Kannampallil T. Characterizing the macrostructure of electronic health record work using raw audit logs: an unsupervised action embeddings approach. J Am Med Inform Assoc. 2023;30(3):539-544. [FREE Full text] [CrossRef] [Medline]
- Tiase VL, Sward KA, Facelli JC. A scalable and extensible logical data model of electronic health record audit logs for temporal data mining (RNteract): model conceptualization and formulation. JMIR Nurs. 2024;7:e55793. [FREE Full text] [CrossRef] [Medline]
- Cross DA, Holmgren AJ, Apathy NC. The role of organizations in shaping physician use of electronic health records. Health Serv Res. 2024;59(1):e14203. [FREE Full text] [CrossRef] [Medline]
- Overhage JM, Qeadan F, Choi EHE, Vos D, Kroth PJ. Explaining variability in electronic health record effort in primary care ambulatory encounters. Appl Clin Inform. 2024;15(2):212-219. [FREE Full text] [CrossRef] [Medline]
- Brown T. Change by Design: How Design Thinking Transforms Organizations and Inspires Innovation. New York. HarperCollins; 2009.
- Bodenheimer T, Sinsky C. From triple to quadruple aim: care of the patient requires care of the provider. Ann Fam Med. 2014;12(6):573-576. [FREE Full text] [CrossRef] [Medline]
- Saraswathula A, Merck SJ, Bai G, Weston CM, Skinner EA, Taylor A, et al. The volume and cost of quality metric reporting. JAMA. 2023;329(21):1840-1847. [FREE Full text] [CrossRef] [Medline]
- Borgatti SP, Jones C. A measure of past collaboration. Connections. 1996;19(1):58-60. [FREE Full text]
- Rodriguez HP, Marshall RE, Rogers WH, Safran DG. Primary care physician visit continuity: a comparison of patient-reported and administratively derived measures. J Gen Intern Med. 2008;23(9):1499-1502. [FREE Full text] [CrossRef] [Medline]
- O'Malley AS, Rich EC, Shang L, Rose T, Ghosh A, Poznyak D, et al. Practice-site-level measures of primary care comprehensiveness and their associations with patient outcomes. Health Serv Res. 2021;56(3):371-377. [FREE Full text] [CrossRef] [Medline]
- Noshad M, Rose CC, Chen JH. Signal from the noise: a mixed graphical and quantitative process mining approach to evaluate care pathways applied to emergency stroke care. J Biomed Inform. 2022;127:104004. [FREE Full text] [CrossRef] [Medline]
- Chen Y, Adler-Milstein J, Sinsky CA. Measuring and maximizing undivided attention in the context of electronic health records. Appl Clin Inform. 2022;13(4):774-777. [FREE Full text] [CrossRef] [Medline]
- Muller JZ. The Tyranny of Metrics. New Jersey. Princeton University Press; 2018.
- Harris M, Tayler B. Don't let metrics undermine your business: an obsession with the numbers can sink your strategy. Harvard Business Review. 2019;97(5):62-70. [FREE Full text]
- Obermeyer Z, Powers B, Vogeli C, Mullainathan S. Dissecting racial bias in an algorithm used to manage the health of populations. Science. 2019;366(6464):447-453. [FREE Full text] [CrossRef] [Medline]
- Campbell DT. Assessing the impact of planned social change. Evaluation and Program Planning. 1979;2(1):67-90. [CrossRef]
- Box GEP, Draper NR. Empirical Model-Building and Response Surfaces. Hoboken, New Jersey. Wiley; 1987.
- Burden M, Keniston A, Pell J, Yu A, Dyrbye L, Kannampallil T. Unlocking inpatient workload insights with electronic health record event logs. J Hosp Med. 2025;20(1):79-84. [CrossRef] [Medline]
- Mundt MP, Gilchrist VJ, Fleming MF, Zakletskaia LI, Tuan WJ, Beasley JW. Effects of primary care team social networks on quality of care and costs for patients with cardiovascular disease. Ann Fam Med. 2015;13(2):139-148. [FREE Full text] [CrossRef] [Medline]
- Kim S, Song H, Valentine MA. Learning in temporary teams: the varying effects of partner exposure by team member role. Organization Science. 2023;34(1):433-455. [CrossRef]
- Huckman RS, Staats BR. Fluid tasks and fluid teams: the impact of diversity in experience and team familiarity on team performance. M&SOM. 2011;13(3):310-328. [CrossRef]
- Chan KS, Wan EYF, Chin WY, Cheng WHG, Ho MK, Yu EYT, et al. Effects of continuity of care on health outcomes among patients with diabetes mellitus and/or hypertension: a systematic review. BMC Fam Pract. 2021;22(1):145. [FREE Full text] [CrossRef] [Medline]
- Reddy A, Pollack CE, Asch DA, Canamucio A, Werner RM. The effect of primary care provider turnover on patient experience of care and ambulatory quality of care. JAMA Intern Med. 2015;175(7):1157-1162. [FREE Full text] [CrossRef] [Medline]
- Wolinsky FD, Bentler SE, Liu L, Geweke JF, Cook EA, Obrizan M, et al. Continuity of care with a primary care physician and mortality in older adults. J Gerontol A Biol Sci Med Sci. 2010;65(4):421-428. [FREE Full text] [CrossRef] [Medline]
- Waters A. Patients treated at practices with high GP turnover have worse outcomes, finds study. BMJ. 2023;380:177. [CrossRef] [Medline]
- Parisi R, Lau YS, Bower P, Checkland K, Rubery J, Sutton M, et al. Predictors and population health outcomes of persistent high GP turnover in English general practices: a retrospective observational study. BMJ Qual Saf. 2023;32(7):394-403. [FREE Full text] [CrossRef] [Medline]
- Sinsky CA, Shanafelt TD, Dyrbye LN, Sabety AH, Carlasare LE, West CP. Health care expenditures attributable to primary care physician overall and burnout-related turnover: a cross-sectional analysis. Mayo Clin Proc. 2022;97(4):693-702. [FREE Full text] [CrossRef] [Medline]
- Nyweide DJ, Bynum JP. Relationship between continuity of ambulatory care and risk of emergency department episodes among older adults. Ann Emerg Med. 2017;69(4):407-415.e3. [FREE Full text] [CrossRef] [Medline]
- Frandsen BR, Joynt KE, Rebitzer JB, Jha AK. Care fragmentation, quality, and costs among chronically ill patients. Am J Manag Care. 2015;21(5):355-362. [FREE Full text] [Medline]
- Chen CC, Tseng CH, Cheng SH. Continuity of care, medication adherence, and health care outcomes among patients with newly diagnosed type 2 diabetes: a longitudinal analysis. Med Care. 2013;51(3):231-237. [CrossRef] [Medline]
- Yao S, Lix L, Teare G, Evans C, Blackburn D. An integrated continuity of care measure improves performance in models predicting medication adherence using population-based administrative data. PLoS One. 2022;17(3):e0264170. [FREE Full text] [CrossRef] [Medline]
- Pereira Gray DJ, Sidaway-Lee K, White E, Thorne A, Evans PH. Continuity of care with doctors-a matter of life and death? A systematic review of continuity of care and mortality. BMJ Open. 2018;8(6):e021161. [FREE Full text] [CrossRef] [Medline]
- Rich EC, Peris K, Luhr M, Ghosh A, Molinari L, O'Malley AS. Association of the range of outpatient services provided by primary care physicians with subsequent health care costs and utilization. J Gen Intern Med. 2023;38(15):3414-3423. [CrossRef] [Medline]
- Bazemore A, Petterson S, Peterson LE, Phillips RL. More comprehensive care among family physicians is associated with lower costs and fewer hospitalizations. Ann Fam Med. 2015;13(3):206-213. [FREE Full text] [CrossRef] [Medline]
- Nundy S, Cooper LA, Mate KS. The quintuple aim for health care improvement: a new imperative to advance health equity. JAMA. 2022;327(6):521-522. [CrossRef] [Medline]
- Mandal S, Wiesenfeld BM, Mann DM, Szerencsy AC, Iturrate E, Nov O. Quantifying the impact of telemedicine and patient medical advice request messages on physicians' work-outside-work. NPJ Digit Med. 2024;7(1):35. [FREE Full text] [CrossRef] [Medline]
- Sinsky CA, Bavafa H, Roberts RG, Beasley JW. Standardization vs customization: finding the right balance. Ann Fam Med. 2021;19(2):171-177. [FREE Full text] [CrossRef] [Medline]
- Cohen GR, Friedman CP, Ryan AM, Richardson CR, Adler-Milstein J. Variation in physicians' electronic health record documentation and potential patient harm from that variation. J Gen Intern Med. 2019;34(11):2355-2367. [FREE Full text] [CrossRef] [Medline]
- Rodríguez-Fernández JM, Loeb JA, Hier DB. It's time to change our documentation philosophy: writing better neurology notes without the burnout. Front Digit Health. 2022;4:1063141. [FREE Full text] [CrossRef] [Medline]
- Rule A, Chiang MF, Hribar MR. Medical scribes have a variable impact on documentation workflows. Stud Health Technol Inform. 2022;290:892-896. [FREE Full text] [CrossRef] [Medline]
- Perros I, Yan X, Jones JB, Sun J, Stewart WF. Using the PARAFAC2 tensor factorization on EHR audit data to understand PCP desktop work. J Biomed Inform. 2020;101:103312. [FREE Full text] [CrossRef] [Medline]
- Chen B, Alrifai W, Gao C, Jones B, Novak L, Lorenzi N, et al. Mining tasks and task characteristics from electronic health record audit logs with unsupervised machine learning. J Am Med Inform Assoc. 2021;28(6):1168-1177. [FREE Full text] [CrossRef] [Medline]
- Graber ML, Kissam S, Payne VL, Meyer AND, Sorensen A, Lenfestey N, et al. Cognitive interventions to reduce diagnostic error: a narrative review. BMJ Qual Saf. 2012;21(7):535-557. [CrossRef] [Medline]
- Harry E, Sinsky C, Dyrbye L, Makowski M, Trockel M, Tutty M, et al. Physician task load and the risk of burnout among US physicians in a national survey. Jt Comm J Qual Patient Saf. 2021;47(2):76-85. [FREE Full text] [CrossRef] [Medline]
- Tai-Seale M, Olson CW, Li J, Chan AS, Morikawa C, Durbin M, et al. Electronic health record logs indicate that physicians split time evenly between seeing patients and desktop medicine. Health Aff (Millwood). 2017;36(4):655-662. [FREE Full text] [CrossRef] [Medline]
- Harry EP, Kneeland P, Huang G, Stein J, Sweller J. Cognitive load and its implciations for health care. NEJM Catalyst. 2018. URL: https://catalyst.nejm.org/doi/full/10.1056/CAT.18.0233 [accessed 2025-02-19]
- Mai MV, Orenstein EW, Manning JD, Luberti AA, Dziorny AC. Attributing patients to pediatric residents using electronic health record features augmented with audit logs. Appl Clin Inform. 2020;11(3):442-451. [FREE Full text] [CrossRef] [Medline]
- Rotenstein LS, Holmgren AJ, Downing NL, Bates DW. Differences in total and after-hours electronic health record time across ambulatory specialties. JAMA Intern Med. 2021;181(6):863-865. [FREE Full text] [CrossRef] [Medline]
- Lou SS, Baratta LR, Lew D, Harford D, Avidan MS, Kannampallil T. Anesthesia clinical workload estimated from electronic health record documentation vs billed relative value units. JAMA Netw Open. 2023;6(8):e2328514. [FREE Full text] [CrossRef] [Medline]
- Baratta LR, Harford D, Sinsky CA, Kannampallil T, Lou SS. Characterizing the patterns of electronic health record-integrated secure messaging use: cross-sectional study. J Med Internet Res. 2023;25:e48583. [FREE Full text] [CrossRef] [Medline]
- Moy AJ, Cato KD, Kim EY, Withall J, Rossetti SC. A Computational framework to evaluate emergency department clinician task switching in the electronic health record using event logs. AMIA Annu Symp Proc. 2023;2023:1183-1192. [FREE Full text] [Medline]
- Shah M, De Arrigunaga S, Forman LS, West M, Rowe SG, Mishuris RG. Cumulated time to chart closure: a novel electronic health record-derived metric associated with clinician burnout. JAMIA Open. 2024;7(1):ooae009. [FREE Full text] [CrossRef] [Medline]
- Lou SS, Kim S, Harford D, Warner BC, Payne PR, Abraham J, et al. Effect of clinician attention switching on workload and wrong-patient errors. Br J Anaesth. 2022;129(1):e22-e24. [FREE Full text] [CrossRef] [Medline]
- Kissler MJ, Kissler K, Burden M. Toward a medical "Ecology of Attention". N Engl J Med. 2021;384(4):299-301. [FREE Full text] [CrossRef] [Medline]
- Zulman DM, Haverfield MC, Shaw JG, Brown-Johnson CG, Schwartz R, Tierney AA, et al. Practices to foster physician presence and connection with patients in the clinical encounter. JAMA. 2020;323(1):70-81. [CrossRef] [Medline]
- Savoy A, Patel H, Murphy DR, Meyer AND, Herout J, Singh H. Electronic health records' support for primary care physicians' situation awareness: a metanarrative review. Hum Factors. 2023;65(2):237-259. [FREE Full text] [CrossRef] [Medline]
- Shreffler J, Huecker M. Physician flow at work: examining work absorption, clinical flow, work fulfillment, and flow thieves. Workplace Health Saf. 2022;70(11):484-491. [CrossRef] [Medline]
- Nijor S, Rallis G, Lad N, Gokcen E. Patient safety issues from information overload in electronic medical records. J Patient Saf. 2022;18(6):e999-e1003. [FREE Full text] [CrossRef] [Medline]
- Moy AJ, Aaron L, Cato KD, Schwartz JM, Elias J, Trepp R, et al. Characterizing multitasking and workflow fragmentation in electronic health records among emergency department clinicians: using time-motion data to understand documentation burden. Appl Clin Inform. 2021;12(5):1002-1013. [FREE Full text] [CrossRef] [Medline]
- Lieu TA, Warton EM, East JA, Moeller MF, Prausnitz S, Ballesca M, et al. Evaluation of attention switching and duration of electronic inbox work among primary care physicians. JAMA Netw Open. 2021;4(1):e2031856. [FREE Full text] [CrossRef] [Medline]
- Melnick ER, Harry E, Sinsky CA, Dyrbye LN, Wang H, Trockel MT, et al. Perceived electronic health record usability as a predictor of task load and burnout among US physicians: mediation analysis. J Med Internet Res. 2020;22(12):e23382. [FREE Full text] [CrossRef] [Medline]
- Westbrook JI, Raban MZ, Walter SR, Douglas H. Task errors by emergency physicians are associated with interruptions, multitasking, fatigue and working memory capacity: a prospective, direct observation study. BMJ Qual Saf. 2018;27(8):655-663. [FREE Full text] [CrossRef] [Medline]
- Orru G, Longo L. The evolution of cognitive load theory and the measurement of its intrinsic, extraneous and Germane loads: a review. In: Human Mental Workload: Models and Applications. Cham. Springer International Publishing; 2019.
Abbreviations
| EHR: electronic health record |
| HITECH: Health Information Technology for Economic and Clinical Health |
| HIT: health information technology |
| NASA-TLX: National Aeronautics and Space Administration–Task Load Index |
Edited by A Mavragani; submitted 24.07.24; peer-reviewed by K McVeigh, A Yazdanian, M Mun, J Hron, D Chrimes; comments to author 17.12.24; revised version received 15.01.25; accepted 11.02.25; published 06.03.25.
Copyright©Daniel Tawfik, Adam Rule, Aram Alexanian, Dori Cross, A Jay Holmgren, Sunny S Lou, Eugenia McPeek Hinz, Christian Rose, Ratnalekha V N Viswanadham, Rebecca G Mishuris, Jorge M Rodríguez-Fernández, Eric W Ford, Sarah T Florig, Christine A Sinsky, Nate C Apathy. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 06.03.2025.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research (ISSN 1438-8871), is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.