Published on in Vol 21, No 8 (2019): August

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/14086, first published .
Mapping the Access of Future Doctors to Health Information Technologies Training in the European Union: Cross-Sectional Descriptive Study

Mapping the Access of Future Doctors to Health Information Technologies Training in the European Union: Cross-Sectional Descriptive Study

Mapping the Access of Future Doctors to Health Information Technologies Training in the European Union: Cross-Sectional Descriptive Study

Original Paper

1University of Oulu, Oulu, Finland

2University College Dublin School of Public Health, Physiotherapy and Sports Science, Dublin, Ireland

3Insight Centre for Data Analytics, Dublin, Ireland

4Ministry of Health, Zagreb, Croatia

5Universidad de Sevilla, Seville, Spain

Corresponding Author:

Guido Giunti, MD, PhD

University of Oulu

Pentti Kaiteran katu 1

Oulu, 90570

Finland

Phone: 358 294 48 0000

Fax:358 294 48 0000

Email: drguidogiunti@gmail.com


Background: Health information technologies (HITs) such as electronic health records (EHR) and telemedicine services are currently used to assist clinicians provide care to patients. There are many barriers to HIT adoption, including mismatches between investments and benefits, disruptions in the workflow, and concerns about privacy and confidentiality. The lack of HIT training of health professionals as a workforce is an increasingly recognized and understudied barrier.

Objective: The purpose of this study is to describe what courses on HIT topics are available at the graduate level for future health professionals in the European Union (EU) and to explore possible determining factors for their exposure to these courses.

Methods: A cross-sectional descriptive study of EU medical schools was performed to explore the prevalence of HIT courses. The curricula of all identified higher learning institutions that offer a medical degree were manually explored to identify graduate-level courses that offer specific training on HIT topics. HIT topics were defined as courses or subjects that provided knowledge on the design, development, use, and implementation of HIT. Associations among potential factors such as population, yearly medical graduates, total number of physicians, EHR presence, and gross domestic product (GDP) were explored.

Results: A total of 302 medical schools from the 28 member states of the EU were explored. Only about one-third (90/302, 29.80%) of all medical degree curricula offered any kind of HIT course at the graduate level; in the medical schools that offered HIT courses, the courses were often mandatory (58/90, 64.44%). In most EU countries, HIT courses are offered in less than half of the medical schools, regardless of the country’s GDP per capita. Countries with the highest percentages of HIT course presence have the lowest GDP per capita. There seems to be a weak inverse correlation (–0.49) between the two variables (GDP per capita and HIT course presence). There is a trend between the availability of medical human resources and an increase in the presence of HIT courses, with Romania, Croatia, and Greece as outliers in this respect.

Conclusions: The current state of medical training in the EU leaves much room for improvement. Further studies are required for in-depth analysis of the content and manner of instruction that would fit present and future needs of HIT.

J Med Internet Res 2019;21(8):e14086

doi:10.2196/14086

Keywords



Health information technologies (HITs) [1] are used in health care institutions to assist clinicians provide care with tools such as electronic health records (EHR) and telemedicine services [2]. The increasingly vast amounts of health data available provide health professionals access to new ways to collect, analyze, and use that information [3,4]. The use of EHRs for documentation allows data to be organized around diseases or quality indicators, machine learning and artificial intelligence enable population health analytics to identify predictive characteristics and factors for diseases, and remote tools facilitate disease management in point-of-care and home settings [5].

There are many barriers to HIT adoption, including mismatches between investments and benefits, disruptions in the workflow, and concerns about privacy and confidentiality [6-8]. However, while many health professionals and students see the potential benefit of these technologies in health care, many are also frustrated [9,10], struggling to adapt, without knowing the underlying science of information in these new tools. The lack of HIT training of health professionals as a workforce is an increasingly recognized and understudied barrier [4,11,12].

Focused and concerted educational training in health informatics is essential for health professionals to realize the full benefit of the data and tools that are already part of the practice of medicine and to help develop new and improved tools of the future. There are some initiatives worldwide to provide specialization training on medical informatics, such as the American Medical Informatics Association in the United States, with nearly 1700 board certified professionals [13], or England’s efforts with the Topol review [14]; many other countries have not, or at least not sufficiently, established such opportunities until now [4]. A recent publication in the British Medical Journal proposes that literacy in informatics should be a formal requirement of all medical education, biomedical research, and public health training [15].

The current state of how future doctors are trained in HIT contents is largely unknown in many European Union (EU) countries. Further, although there may be opportunities for obtaining education in this field, most are targeted at the postgraduate level, leaving HIT training to be pursued as a professional interest and not a core skill. The purpose of this study is to describe what courses on HIT topics are available at the graduate level for future health professionals in the EU and to explore possible determining factors for their exposure to these courses.


Study Design

A cross-sectional descriptive study of the state of EU medical education was performed to explore the prevalence of specific HIT courses offered to future doctors. Cross-sectional studies are carried out at one time point or over a short period to estimate the prevalence of the outcome of interest for a given population [16].

The curricula of all identified higher learning institutions such as faculties of medicine, schools of medicine, or universities that offer the medical degree, hereon referred to as “medical schools,” were explored to identify graduate-level courses that may offer would-be physicians training on any HIT topics. Potential factors such as population, yearly medical graduates, total number of physicians, EHR presence, and GDP were explored for associations.

Settings

The EU is a political and economic union of 28 member states, with an estimated population of over 513 million and almost 2 million practicing physicians [17]. During the first quarter of 2018, the medical degree programs from all EU member states were systematically explored and classified.

Data Sources

No official list of medical schools in EU exists; therefore, a preliminary list was obtained and refined from gray literature [18]. Multimedia Appendix 1 shows the definitive list of medical schools used.

Data on EU population statistics, GDP per capita, EHR presence, and medical graduates and doctors were extracted from their respective official sources [17,19,20]. Latest available information was used, and cases where no information was available were marked.

Selection Criteria

For the purpose of this study, and given the many different terms in use in the field, the terms medical informatics, health information and communications technology (ICT), eHealth, mHealth, biomedical and health informatics, consumer health informatics, digital health, and other variations were considered to be encapsulated by the term “HIT.”

HIT topics were defined as courses or subjects that provided knowledge on the design, development, use, and implementation of HIT. The different course names and programs (where available) were revised for mentions of HIT contents. Courses that used ICT tools to enhance medical education on other subjects (eg, anatomy lessons or clinical case virtual simulations), or who only taught ICT content as a means to an end (eg, programming in R for biostatistics) were not considered to be focused on HIT training and were therefore not included.

A small random sample (10%) was independently reviewed to assess clarity of the selection criteria, and interrater reliability was calculated using the Fleiss-Cohen Coefficient. Once this was established, the remaining data were explored. Disagreements were resolved by consensus involving a third reviewer, when necessary.

Data Extraction and Classification

Medical education data were manually extracted from each medical school’s website to obtain the latest publicly available curricula and program. A multilingual and multicultural team (GG, EG, and HB) conducted the data extraction, and any language barrier issue was resolved using the Google Translate feature. The team independently reviewed and classified the extracted information using structured forms. In Multimedia Appendix 2, the raw data are presented.

Statistical Methods

Categorical variables are presented as absolute and relative frequencies. Quantitative variables are presented as mean and SD or median and interquartile range, depending on the distribution. The Landis and Koch standards for the Fleiss-Cohen coefficient were used [21]. Statistical analysis was performed using R (Vienna, Austria, R Foundation for Statistical Computing; 2013).


Interrater reliability was determined using the Cohen kappa statistic by extracting a random sample (n=32) and independently reviewing the selection criteria. Kappa was found to be more than acceptable at 0.91 (SE 0.06, 95% CI 0.79-1.0).

Table 1. Summary of the presence of health information technology courses in each European Union member state.
CountryTotal medical schools (N=302), n (%)Medical schools with HITa courses (n=90), n (%)Medical schools with HIT courses, where the course is mandatory (n=58), n (%)EHRb availability
Austria6 (2.0)1 (16.67)1 (100)Nationwide project in progress
Belgium10 (3.3)1 (10.00)0 (0)Nationwide project in progress
Bulgaria6 (2.0)2 (33.33)1 (50)Yes
Croatia4 (1.3)3 (75.00)3 (100)Nationwide project in progress
Cyprus4 (1.3)1 (25.00)1 (100)Nationwide project in progress
Czech Republic9 (2.9)3 (33.33)2 (66.67)Nationwide project in progress
Denmark4 (1.3)0 (0.00)N/AcYes
Estonia1 (0.3)0 (0.00)N/AYes
Finland5 (1.7)1 (20.00)1 (100)Yes
France34 (11.26)10 (29.41)7 (70)Nationwide project in progress
Germany38 (12.6)16 (42.11)8 (50)Nationwide project in progress
Greece7 (2.3)6 (85.71)5 (83.33)Nationwide project in progress
Hungary4 (1.3)1 (25.00)1 (100)Yes
Ireland6 (2.0)2 (33.33)1 (50)Nationwide project in progress
Italy41 (13.6)7 (17.07)7 (100)Nationwide project in progress
Latvia2 (0.7)0 (0)N/ANationwide project in progress
Lithuania2 (0.7)0 (0)N/AYes
Luxembourg1 (0.3)0 (0)N/ANationwide project in progress
Malta1 (0.3)0 (0)N/AYes
Netherlands9 (3.0)3 (33.33)0 (0)Nationwide project in progress
Poland19 (6.3)7 (36.84)7 (100)Nationwide project in progress
Portugal7 (2.3)2 (28.57)1 (50)Nationwide project in progress
Romaniad13 (4.3)10 (76.92)5 (50)Nationwide project in progress
Slovakiad3 (1.0)0 (0)0 (0)Nationwide project in progress
Slovenia2 (0.7)2 (100)1 (50)Nationwide project in progress
Spain25 (8.3)8 (32)4 (50)Nationwide project in progress
Sweden7 (2.3)0 (0)N/AYes
United Kingdom32 (10.6)2 (6.25)2 (100)Yes

aHIT: health information technology.

bEHR: electronic health record.

cN/A: not available.

dInformation on the medical school curricula in one school each in Romania and Slovakia was not available.

A total of 302 medical schools from the 28 member states of the EU were explored. Only one-third (90/302, 29.80%) of all medical degree curricula offered any kind of HIT course on the graduate level; in medical schools that offered HIT courses, the courses were often mandatory (58/90, 64.44%). The prevalence of HIT courses for medical degree students in EU member states is very low. As an indicator of system-wide informatization, we also compared our findings with reports and literature on each EU country’s process of EHR implementation. The state of EHR implantation in the EU is a work in progress in most cases (only one-third have nationwide systems). Table 1 presents the information according to each country.

HIT courses offered in most medical schools were titled along the lines of “Medical informatics,” “Telemedicine,” “e-Health,” or “Health informatics” and usually paired with statistics content. Some universities also provided courses on relatively advanced topics such as “AI in Medicine” (Medical University of Lublin, Poland), “Robotics programming with LEGO” (University of Duisburg-Essen, Germany), or workshops on “Application scenarios of Virtual Reality” (University of Ulm, Germany). Entrepreneurship was related to HIT in some cases, for example, in the Galway School of Medicine (National University of Ireland, Ireland), courses on “Becoming a Medical Innovator” are offered. Figure 1 shows a map with EU member states and the presence of HIT courses. A list of all the course names and their frequency can be found in Table 2.

Figure 1. Presence of health information technology courses in the European Union member states.
View this figure
Table 2. List of all the names of health information technology courses and their frequencies.
Course nameFrequency
Medical informatics22
Epidemiology, medical biometry and medical informatics10
Medical informatics and biostatistics9
Introduction to Medical Informatics7
Information Technologies5
Research and New Technologies3
Biomedical Information Systems2
Evaluation of methods of analysis applied to life and health sciences2
Medical ITa2
Advanced Medical Technology1
Basics of informatics in the health sector1
Basics of Medical Informatics1
Becoming a Medical Innovator1
Bioinformatics1
Biomedical Research and New Technologies1
Biometrics and Epidemiology1
Clinical Informatics and Biostatistics1
eHealthb1
eHealth & Medical Informatics1
Health informatics1
Health Technology Assessment1
Healthcare Imaging and Information Systems,1
ICTc for Medicine1
Informatics and Applications of Medical Informatics1
IT Resources; Telemedicine1
Lecture Epidemiology and Medical Informatics1
Legal and Organisational Aspects of Medicine - includes Health IT1
Medical and Scientific Methodology (includes Bioengineering and Medical Informatics)1
Medical Applied Informatics1
Medical Computer Science1
Medical Informatics and Internet Computer Certificate1
Medical Informatics, Biomedical Statistics, and scientific English1
Medical informatics, e-Healthb and medical statistics1
Medicine and Technology1
Modern Informatics in Biomedicine1
New technologies in biomedicine1
Robotics and Programming with Lego - An introductory Course to Robotics and Programming for Medical Students1
Statistics and Bioinformatics1
Tele-Health & Health Information Technologies In Public Health1
Telemedicine and eHealth1
Telemedicine: internet technologies for health1
Telemedicine: Possibilities and Limitations1
The Industry Perspectives on Innovative Medicine Intensive Summer Course1

aIT: information technology.

beHealth/e-Health: electronic health.

cICT: information and communication technology.

We also explored possible associations between the presence of HIT courses and EU population statistics, GDP per capita, EHR presence, and yearly new medical graduates per country. Most countries seem to offer HIT courses in less than half of their medical schools, regardless of their GDP per capita. Countries with the highest percentages of HIT course offerings are among the ones with the lowest GDP per capita in EU. There seems to be a weak inverse correlation (–0.49) between the two variables (GDP per capita and HIT course offer). In Figure 2, we show a scatter plot with the relationship between GDP per capita and percentage of medical schools with HIT courses. The exploration of population size, the main demographic variable, resulted in no relation between the variables.

We explored the association of a country’s population over the number of physicians and the presence of HIT course offerings. There seems to be a trend between the availability of medical human resources and an increase in the presence of HIT courses, with Romania, Croatia, and Greece as outliers (Figure 3).

Figure 2. Relationship between European Union member states' GDP per capita and presence of HIT courses. The surface represents the number of medical graduates per year over 100,000 inhabitants. Countries where no information on yearly medical graduates was available (Cyprus and Luxembourg) are not shown. GDP: gross domestic product; HIT: health information technology.
View this figure
Figure 3. Relationship between the total number of physicians per European Union member state and the presence of HIT courses. Countries where information was missing (Cyprus and Luxembourg) are not shown. HIT: health information technology.
View this figure

Principal Findings

This work provides an overview of the state of HIT courses available for graduate-level medical students in the EU. It offers unique insight into the prevalence of HIT courses for each country and explores possible associated factors. Medical students in the EU seem to find it difficult to gain exposure to HIT courses, with only one-third of all medical schools offering any courses on these topics at all. The uneven distribution of training opportunities for medical students on HIT contents between the different countries highlights the need of further work on improving the medical curricula.

Comparison With Prior Work

Despite their promise, HITs have proven difficult to implement [22], and there seems to be a failure to achieve widespread understanding of the benefits of technologies such as electronic record keeping and information exchange [23]. Although technology is increasingly being used in health care settings, health care professionals remain reticent to adopting HIT at different levels [24].

The need to improve the HIT competences of the health care workforce has been frequently emphasized by policymakers at an international levels [25]. One of the projects aiming to identify health care workforce IT skill needs is the CAMEI (Coordination Actions in the scientific era of Medical Education Informatics) project, which is a collaboration between the United States and Europe [26]. Further, the importance of training for the health care workforce in the use of new technologies was also acknowledged in many studies [25,26]. Several authors have pointed out that the training and competences that health care professionals receive regarding ICTs, in general, and as HIT end users, in particular, act as key factors in HIT adoption [27]. The limited exposure that medical students in the EU receive on HIT contents can easily explain the resistance that HIT implementations face later in health services settings. Recommendations from the International Medical Informatics Association [4] suggest that education on HIT contents should be available to all types of health care professionals, regardless of the types of specialization and levels of education as a way of emphasizing the formal penetration of the discipline and its concepts.

A significant factor for the slow uptake of HIT for health care professionals also seems to be the lack of customization that software vendors offer for their solutions [28,29]. The concept of user-centered design places the needs and characteristics of end users at the center of software design [30-32]. However, involving stakeholders in the design and development of technologies is not yet common practice for HIT solutions.

The introduction of HIT should not be viewed as a problem in technology exclusively, but rather as a problem in organizational change [33]. The implementation of HIT in the different EU member states is very diverse, particularly regarding the development of EHRs [27], a central component of an integrated HIT [34]. This heterogeneity seems to also apply to how health professionals are trained at the graduate level with regard to HIT concepts, with countries ranging from no training at all offered to all medical schools requiring the approval of HIT courses to obtain an medical degree.

In addition, it is known that health care spending per capita is positively correlated with GDP per capita [35] and that the use of health information systems may be a method to control increasing health spending [35]. Although more data would be needed to conclude a definitive casual relation, an interesting finding of our study was a weak inverse relationship between the presence of HIT courses and a country’s GDP. It would be interesting to ascertain what other factors and how the design of medical program curricula can impact a country’s health care expenditure.

Many factors that determine how a degree’s curriculum is structured, and while there is a need for change, political factors that prevent much needed modifications may exist. It is important to remember that wide-scale implementation of EHRs and other HIT systems has often been the result of state or national legislations [19]. Initiatives like these exist in many EU countries, but they are relatively recent and the change to curricula takes time. For example, digital competence has been acknowledged as one of the eight key competences for lifelong learning by the EU [36]. Under this framework, governments should encourage universities, particularly medical schools, to introduce formal teaching activities within undergraduate medical studies.

Limitations

The findings of this study should be interpreted in the context of its limitations. The main limitation of this study is that it focuses on courses that specifically teach HIT contents and does not account for how ICTs can be used in different teaching methods such as problem-based learning, or other subjects where HIT concepts may be taught indirectly. Additionally, this study did not perform an in-depth analysis of the medical degree curricula; therefore, it is possible that HIT lessons are included or integrated in other courses. However, given that medical education is usually divided in preclinical studies (eg, basic sciences such as anatomy, physiology, and biochemistry) and clinical studies (eg, various areas of clinical medicine such as internal medicine, pediatrics, obstetrics, and gynecology), it is unlikely that foundational HIT contents would be found within these modules.

Analysis and interpretation was performed using the retrieved information. Newer data could be available through internal channels and publications of each institution or country. Nevertheless, our study used publicly available data, which would also be available to other researchers.

In recent years, new bachelor and master degrees of an interdisciplinary nature have been developed (eg, biomedical engineering). Universities that hold these degrees may have contents or courses available for medical degree students to access; however, as these courses would not appear in the medical degree curricula, these were not explored or included. Further, this study only focuses on medical degree students and does not cover other health care professionals (eg, nurses or physiotherapists) or disciplines that could have some overlap (eg, biomedical engineering).

Finally, an interplay of varying complex factors determines how a degree program and curriculum are designed, both within and outside academic institutions such as local, national, and regional needs; legislations and ruling legal frameworks; and political leadership and visions. These were not included, as they would be too many and exceed the scope of this study.

Conclusions

Technology is transforming health care by slowly changing the way medicine is practiced. As HITs become more prevalent, expectations for health professionals to be fluent users of health technologies will continue to increase. The current state of medical training in the EU in this regard leaves much room for improvement. Further studies are required for in-depth analysis on the type of contents, proper placement in the medical curricula, and manner of instruction that would fit present and future needs of HIT.

Taking into consideration that the use of HIT is likely to continue to increase even further in most European health care systems, the need of future physicians to be literate in both medical sciences and HIT will soon no longer be optional. Empowering the following generations of doctors to allow them to take advantage of the full benefits that technologies have to offer is ever more important. A thorough look at the way we design our medical school curricula is needed.

Acknowledgments

GG and EG-F gratefully acknowledge the grant (number 676201) for the Connected Health Early-stage researcher Support System (CHESS ITN) from the Horizon 2020 Framework Programme of the European Commission. This article is based upon work from COST Action ENJECT TD 1405, supported by COST (European Cooperation in Science and Technology; www.cost.eu).

We would like to thank Luis Fernandez-Luque, PhD, Prof Minna Isomursu, Prof Brian Caulfield, Tim Sangster, Joaquin Chacon-Galvez, Analia Baum, and Diego Giunta for their cooperation and support.

Conflicts of Interest

None declared.

Multimedia Appendix 1

List of medical schools in the EU.

XLSX File (Microsoft Excel File), 47KB

Multimedia Appendix 2

Data on medical schools.

XLSX File (Microsoft Excel File), 22KB

  1. Lorenzi NM. E-Health Strategies Worldwide. Yearb Med Inform 2018 Mar 06;14(01):157-164. [CrossRef]
  2. Tang PC, Ash JS, Bates DW, Overhage JM, Sands DZ. Personal health records: definitions, benefits, and strategies for overcoming barriers to adoption. J Am Med Inform Assoc 2006 Mar;13(2):121-126 [FREE Full text] [CrossRef] [Medline]
  3. Greenes RA. Medical Informatics. JAMA 1990 Feb 23;263(8):1114. [CrossRef]
  4. Mantas J, Ammenwerth E, Demiris G, Hasman A, Haux R, Hersh W, IMIA Recommendations on Education Task Force. Recommendations of the International Medical Informatics Association (IMIA) on Education in Biomedical and Health Informatics. First Revision. Methods Inf Med 2010 Jan 07;49(2):105-120. [CrossRef] [Medline]
  5. Adler-Milstein J, Cohen G. Implementing the IT infrastructure for health reform: adoption of health IT among patient-centered medical home practices. AMIA Annu Symp Proc 2013;2013:11-16 [FREE Full text] [Medline]
  6. Ball M, Silva J, Bierstock S, Douglas JV, Norcio AF, Chakraborty J, et al. Failure to provide clinicians useful IT systems: opportunities to leapfrog current technologies. Methods Inf Med 2008;47(1):4-7. [Medline]
  7. Hersh W. Health Care Information Technology. JAMA 2004 Nov 10;292(18):2273. [CrossRef]
  8. Poon EG, Blumenthal D, Jaggi T, Honour MM, Bates DW, Kaushal R. Overcoming barriers to adopting and implementing computerized physician order entry systems in U.S. hospitals. Health Aff (Millwood) 2004;23(4):184-190 [FREE Full text] [Medline]
  9. Payne T, Corley S, Cullen T, Gandhi TK, Harrington L, Kuperman GJ, et al. Report of the AMIA EHR-2020 Task Force on the status and future direction of EHRs. J Am Med Inform Assoc 2015 Sep;22(5):1102-1110 [FREE Full text] [CrossRef] [Medline]
  10. Kern J, Fister K. Medical Informatics Meets Medical Education: the Croatian Experience. Eur J Biomed Informatics 2011;7(2):58-63 [FREE Full text] [CrossRef]
  11. Hersh W. Who are the informaticians? What we know and should know. J Am Med Inform Assoc 2006;13(2):166-170 [FREE Full text] [CrossRef] [Medline]
  12. Hersh W. Health and biomedical informatics: opportunities and challenges for a twenty-first century profession and its education. Yearb Med Inform 2008:157-164. [Medline]
  13. Detmer D, Lumpkin J, Williamson J. Defining the medical subspecialty of clinical informatics. J Am Med Inform Assoc 2009;16(2):167-168 [FREE Full text] [CrossRef] [Medline]
  14. NHS. 2018. Preparing the healthcare workforce to deliver the digital future   URL: https://topol.hee.nhs.uk/wp-content/uploads/HEE-Topol-Review-2019.pdf [accessed 2019-08-01]
  15. Fridsma D. Health informatics: a required skill for 21st century clinicians. BMJ 2018 Jul 12;362:k3043. [CrossRef] [Medline]
  16. Thelle D, Laake P. Research in medical and biological sciences. Cambridge, MA: Academic Press; 2015:275-320.
  17. European Commission. Eurostat - statistics explained. Healthcare Personnel Statistics - Physicians   URL: http:/​/ec.​europa.eu/​eurostat/​statistics-explained/​index.php/​Healthcare_personnel_statistics_-_physicians#Healthcare_personnel [accessed 2019-08-01]
  18. Wikipedia: The Free Encyclopedia. List of medical schools in Europe   URL: https://en.wikipedia.org/w/index.php?title=List_of_medical_schools_in_Europe&oldid=868758984 [accessed 2019-08-01]
  19. European Comission. 2013. Overview of the National Laws on Electronic Health Records in the EU Member States and Their Interaction with the Provision of Cross-Border EHealth Services   URL: https://ec.europa.eu/health/ehealth/projects/nationallaws_electronichealthrecords_en [accessed 2019-08-01]
  20. World Health Organization. 2014 Dec. From Innovation to Implementation: eHealth in the WHO European Region   URL: http:/​/www.​euro.who.int/​__data/​assets/​pdf_file/​0012/​302331/​From-Innovation-to-Implementation-eHealth-Report-EU.​pdf [accessed 2019-08-01]
  21. Landis JR, Koch GG. The Measurement of Observer Agreement for Categorical Data. Biometrics 1977 Mar;33(1):159. [CrossRef]
  22. Taylor R, Bower A, Girosi F, Bigelow J, Fonkych K, Hillestad R. Promoting Health Information Technology: Is There A Case For More-Aggressive Government Action? Health Affairs 2005 Sep;24(5):1234-1245. [CrossRef]
  23. Achieving Efficiency Improvements in the Health Sector through the Implementation of Information and Communication Technologies. 2010. EHealthNews.eu   URL: https://tinyurl.com/y4zybwda [accessed 2019-08-01]
  24. Gagnon M, Légaré F, Labrecque M, Frémont P, Pluye P, Gagnon J, et al. Interventions for promoting information and communication technologies adoption in healthcare professionals. Cochrane Database Syst Rev 2009 Jan 21(1):1-100 [FREE Full text] [CrossRef] [Medline]
  25. Li S, Bamidis PD, Konstantinidis ST, Traver V, Car J, Zary N. Setting priorities for EU healthcare workforce IT skills competence improvement. Health Informatics J 2017 Apr 25;25(1):174-185. [CrossRef]
  26. Traver V, Konstantinidis S, Bamidis P, Zary N. Analysis of EU-USA cooperation opportunities on IT skills for healthcare workforce. Stud Health Technol Inform 2015;210:561-563. [Medline]
  27. Lluch M. Healthcare professionals' organisational barriers to health information technologies-a literature review. Int J Med Inform 2011 Dec;80(12):849-862. [CrossRef] [Medline]
  28. Westbrook J, Braithwaite J, Iedema R, Coiera E. Evaluating the impact of information communication technologies on complex organizational systems: a multi-disciplinary, multi-method framework. Stud Health Technol Inform 2004;107(Pt 2):1323-1327. [Medline]
  29. Ford EW, Menachemi N, Phillips MT. Predicting the Adoption of Electronic Health Records by Physicians: When Will Health Care be Paperless? Journal of the American Medical Informatics Association 2006 Jan 01;13(1):106-112. [CrossRef]
  30. International Organization for Standardization. 2010. Ergonomics of human-system interaction -- Part 210: Human-centred design for interactive systems   URL: https://www.iso.org/standard/52075.html [accessed 2019-08-01]
  31. De Vito Dabbs A, Myers BA, Mc Curry KR, Dunbar-Jacob J, Hawkins RP, Begey A, et al. User-centered design and interactive health technologies for patients. Comput Inform Nurs 2009;27(3):175-183 [FREE Full text] [CrossRef] [Medline]
  32. Pruitt J, Adlin T. The Persona Lifecycle: Keeping People In Mind Throughout Product Design (interactive Technologies). NYC: Morgan Kaufmann; 2019.
  33. Wears RL, Berg M. Computer Technology and Clinical Work. JAMA 2005 Mar 09;293(10):1261. [CrossRef]
  34. Black AD, Car J, Pagliari C, Anandan C, Cresswell K, Bokun T, et al. The impact of eHealth on the quality and safety of health care: a systematic overview. PLoS Med 2011 Jan 18;8(1):e1000387 [FREE Full text] [CrossRef] [Medline]
  35. Anderson GF, Frogner BK, Johns RA, Reinhardt UE. Health Care Spending And Use Of Information Technology In OECD Countries. Health Affairs 2006 May;25(3):819-831. [CrossRef]
  36. Carretero S, Vuorikari R, Punie Y. European Commission. 2017. DigComp 2.1: The Digital Competence Framework for Citizens with eight proficiency levels and examples of use   URL: http://publications.jrc.ec.europa.eu/repository/bitstream/JRC106281/web-digcomp2.1pdf_(online).pdf [accessed 2019-08-01]


CAMEI: Coordination Actions in the scientific era of Medical Education Informatics
eHealth: electronic health
EHR: electronic health record
EU: European Union
GDP: gross domestic product
HIT: health information technology
ICT: information and communication technology
IT: information technology


Edited by B Caulfield; submitted 21.03.19; peer-reviewed by V Traver Salcedo, WH Ho, Y Acharya, C Perrin; comments to author 16.04.19; revised version received 05.06.19; accepted 19.06.19; published 12.08.19

Copyright

©Guido Giunti, Estefania Guisado-Fernandez, Hrvoje Belani, Juan R Lacalle-Remigio. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 12.08.2019.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.