This paper is in the following e-collection/theme issue:

e-Learning and Medical Education (645) eHealth Literacy / Digital Literacy (351) User Training and Skills (45)
Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/19827, first published .
Perceptions of Digital Health Education Among European Medical Students: Mixed Methods Survey

Perceptions of Digital Health Education Among European Medical Students: Mixed Methods Survey

Perceptions of Digital Health Education Among European Medical Students: Mixed Methods Survey

Authors of this article:

Felix Machleid1, 2 Author Orcid Image ;   Robert Kaczmarczyk1, 2 Author Orcid Image ;   Doreen Johann3 Author Orcid Image ;   Justinas Balčiūnas1, 4 Author Orcid Image ;   Beatriz Atienza-Carbonell1, 5 Author Orcid Image ;   Finn von Maltzahn1, 6 Author Orcid Image ;   Lina Mosch1, 7 Author Orcid Image
Article Authors Cited by (72) Tweetations (19) Metrics

    Original Paper

    1Association Européenne des Étudiants en Médecine (EMSA), Brussels, Belgium

    2School of Medicine, Technical University of Munich, Munich, Germany

    3Geography Department, Humboldt-Universität zu Berlin, Berlin, Germany

    4Lithuanian University of Health Sciences, Kaunas, Lithuania

    5Medical Faculty, University of Valencia, Valencia, Spain

    6Medical Faculty, Rheinisch-Westfälische Technische Hochschule Aachen University, Aachen, Germany

    7Charité - Universitätsmedizin Berlin (corporate member of Freie Universität Berlin, Humboldt-Universität zu Berlin, and Berlin Institute of Health), Berlin, Germany

    *these authors contributed equally

    Corresponding Author:

    Lina Mosch

    Charité - Universitätsmedizin Berlin (corporate member of Freie Universität Berlin, Humboldt-Universität zu Berlin, and Berlin Institute of Health)

    Charitéplatz 1

    Berlin, 10117

    Germany

    Phone: 49 15757206906

    Email: lina.mosch@charite.de


    Background: Digital health technologies hold promise to enhance patient-related outcomes, to support health care staff by reducing their workload, and to improve the coordination of care. As key users of digital health technologies, health care workers are crucial to enable a meaningful digital transformation of health care. Digital health literacy and digital skills should become prerequisite competencies for health professionals to facilitate the implementation and leverage the potential of digital technologies to improve health.

    Objective: We aimed to assess European medical students’ perceived knowledge and opinions toward digital health, the status of digital health implementation in medical education, and the students’ most pressing needs.

    Methods: The explanatory design of our mixed methods study was based on an online, anonymous, self-administered survey targeted toward European medical students. A linear regression analysis was used to identify the influence of the year of medical studies on the responses. Additional analysis was performed by grouping the responses by the self-evaluated frequency of eHealth technology use. Written responses to four qualitative questions in the survey were analyzed using an inductive approach.

    Results: The survey received a total of 451 responses from 39 European countries, and there were respondents for every year of medical studies. The majority of respondents saw advantages in the use of digital health. While 40.6% (183/451) felt prepared to work in a digitized health care system, more than half (240/451, 53.2%) evaluated their eHealth skills as poor or very poor. Medical students considered lack of education to be the reason for this, with 84.9% (383/451) agreeing or strongly agreeing that more digital health education should be implemented in the medical curriculum. Students demanded introductory and specific eHealth courses covering data management, ethical aspects, legal frameworks, research and entrepreneurial opportunities, role in public health and health systems, communication skills, and practical training. The emphasis lay on tailoring learning to future job requirements and interprofessional education.

    Conclusions: This study shows a lack of digital health-related formats in medical education and a perceived lack of digital health literacy among European medical students. Our findings indicate a gap between the willingness of medical students to take an active role by becoming key players in the digital transformation of health care and the education that they receive through their faculties.

    J Med Internet Res 2020;22(8):e19827

    doi:10.2196/19827

    Keywords

    medical students; medical education; eHealth; mixed method; health workforce; digital literacy; curriculum 


    Health care systems around the world are facing challenges connected to an aging population, multimorbidity, an increase of preventable noncommunicable diseases, and health workforce shortages [1-3]. Digital health technologies are seen as a key solution to address these challenges, reinforced by the public health emergency of coronavirus disease 2019 (COVID-19), by having the potential to change the way health services are delivered and promoting the health and well-being of millions of citizens [1,4-6]. For instance, open source technologies have enabled low-cost dissemination and access to data and health information, telehealth technologies have offered communication channels for citizens and health care workers besides physical consultations, and nanotech products have been developed to improve diagnosis and treatment of COVID-19 [7]. However, extensive and sustainable implementation of digital technologies, both into specific clinical settings [8,9] and into national health systems [4,10,11], has been advancing slowly.

    Health care professionals play a crucial role in assisting their patients in using digital health technologies appropriately [12,13]. Thus, the need to improve the digital competencies of health workers and citizens to take advantage of digital technologies and facilitate implementation has been emphasized frequently on the international policy level [1,12,14-16].

    Major barriers to the successful implementation of digital health technologies are (1) the lack of coordinated, formal education and (2) health care professionals’ skepticism and unwillingness toward implementing digital technologies [17-19]. Engaging with these challenges, medical education, and an effective culture of learning could drive the meaningful digitalization of health care [14]. However, to effectively introduce respective topics in medical education, the needs of key stakeholders and the status of the medical curriculum should be considered.

    In this paper, we present and discuss the results of a European-wide study assessing the expectations and needs of medical students regarding digital health competencies and the implementation of digital health in the medical curriculum.


    Setting

    The European Medical Students’ Association (Association Européenne des Étudiants en Médecine; EMSA) is a nonprofit, nongovernmental organization representing the voice of medical students from over 110 faculties in 30 countries across Europe [20]. Data were collected according to the European Union General Data Protection Regulation 2016/679 [21].

    Study Design

    We conducted a cross-sectional mixed methods online survey, following an explanatory approach [22-24]. The survey questions were developed during four online discussions after conducting literature research and collecting feedback from external collaborators. We developed 48 questions, of which 29 were quantitative (including 6 questions on demographic data), and 17 were qualitative (see Multimedia Appendix 1).

    Data Collection and Analysis

    Data were collected via an informed consent–based online survey in English from June 2018 until August 2018. The target group consisted of medical students in Europe from the first to the seventh year of studies. The survey was distributed via medical faculty mailing lists mapped by EMSA, social media channels, and personal connections.

    For the statistical analysis, R (version 3.5.0) and R Studio (version 1.1a) were used [25]. A linear regression analysis was used to identify the influence of the year of medical studies on the responses. Additional analysis was performed by grouping the students depending on the answers to the question “How often are you using eHealth technologies (for example health apps) in your daily life?” P values<.05 were deemed statistically significant. For questions where the possible answers ranged from 0 (“strongly disagree”) to 6 (“strongly agree”), the answer options “undecided” and “I do not feel informed enough” were treated adjacently in the linear regression model, as they were situated exactly in the middle of the extremes 0 and 6.

    For the linear regression model, the answers “undecided” and “I feel not informed enough” were arbitrarily located adjacent to one another and between the first two and subsequent two options in questions where both these options were included. The dependent variables represented the answers to each question, the independent variable, if not stated otherwise, was the year of medical studies, ranging from first to seventh. Most quantitative questions included an ordinal response format (eg, a Likert-type scale), and in one case, a categorical response format was used. Likert-type scale results were rounded to the next significant digit.

    Written responses to four qualitative questions in the survey were analyzed using an inductive approach [26]. The coding and categorization of responses was performed using MaxQDA (version 2020; VERBI GmbH) qualitative data analysis software (Multimedia Appendix 2, Multimedia Appendix 3, Multimedia Appendix 4, and Multimedia Appendix 5) [27]. The results were summarized in paragraphs of continuous text, according to the established code system.

    To maintain reflexivity, the research team discussed and established codes and coding in three face-to-face discussions and documented the process of analysis in research diaries throughout the study.


    Summary

    Our study sample consisted of students (n=451) from 39 European countries in all years of medical studies, most aged 18-24 years (344/451, 76.3%). Respondents indicated a need for more eHealth implementation into medical curriculum (agree or strongly agree: 383/451, 84.9%) and a subjective lack of digital skills among the surveyed medical students (evaluated eHealth skills as poor or very poor: 240/451, 53.2%). One quarter (110/451, 24.4%) stated that their faculty provides no eHealth-related courses at all. Surveyed students reported courses on ethical discussions (247/451, 54.8%), research opportunities (211/451, 46.8%), computer science (138/451, 30.6%), usage of eHealth technologies (83/451, 18.4%), start-up possibilities (69/451, 15.3%), and other (36/451, 8.0%).

    Qualitatively, we found variability in the responses to the question “Please define eHealth in your own words,” both in the level of detail, scope, and specification. According to the respondents, implementing eHealth in their curriculum would prepare students for their future working environment, keep education up-to-date, and reduce their doubts about eHealth. The already high density of learning content in the medical curriculum was seen as a counterpoint to this. The feeling of being ready to work in a digitized health care system was based on the students’ own technical interests and motivations rather than on adequate training in their faculties. The respondents emphasized being willing to learn about eHealth. They indicated a need for an introduction to eHealth and for specific courses (data management, practical training with eHealth technologies, courses on informatics, ethical aspects, legal frameworks, research and entrepreneurial opportunities, role in public health and health systems, communication skills). Content-wise, the interest lay in learning about recent developments and technologies, health information systems, and artificial intelligence in health. Also, suggestions for cross-disciplinary courses, for teaching eHealth as a separate discipline were made, and that learning should be tailored to future job requirements and interprofessional education.

    Demographic Data

    In total, 459 replies were received, of which 2 (0.4%) were left empty and 6 (1.3%) were not willing to participate after reading the initial survey description, resulting in 451 (98.3%) respondents. Our sample was evenly distributed between the first and the sixth year of medical studies. There were fewer seventh-year medical students (22/451, 4.9%) since medical programs with a duration of seven years exist in only a few European countries. The majority of respondents were between 18 and 24 years old (344/451, 76.3%), followed by 25-34 years (98/451, 21.7%). In total, we received responses from 39 countries in the European region with most responses coming from Germany (134/451, 29.0%), Portugal (49/451, 10.9%), and Turkey (39/451, 8.6%).

    Quantitative Results

    In general, more than half of the respondents (239/451, 53.0%) strongly agreed or agreed on being familiar with the term eHealth. Together, almost two-thirds (274/451, 60.8%) of the respondents claimed to never use eHealth technologies or only every other week. Overall, they had positive expectations toward eHealth: they saw mainly or more advantages in mHealth (362/451, 80.3%), telehealth (314/451, 69.6%), and big data (302/451, 67.0%). The respondents strongly agreed or agreed (272/451, 60.3%) that health care professionals should be responsible for eHealth knowledge and skills of their patients. More than half of the respondents (240/451, 53.2%) evaluated their eHealth skills regarding working with eHealth technologies as poor or very poor (Figure 1).

    ‎
    Figure 1. Self-evaluation in response to the statement: "I evaluate my eHealth skills (eg, working with clinical decision support systems, remote patient monitoring systems, artificial intelligence, applications in radiology) as...".
    View this figure

    Respondents who used eHealth technologies more frequently evaluated their eHealth skills on average as better (P<.001) (Figure 2). A majority of respondents (183/451, 40.6%) stated they felt prepared for working in a digitized health care system. Regarding the implementation of eHealth in medical education, we found that 81.8% (369/451) of the students received between 0 and 5 hours of eHealth training during their medical studies. The majority of students (383/451, 84.9%) agreed or strongly agreed that eHealth should be increasingly included in the medical curriculum; 8.9% (40/451) were undecided (Figure 3). Regarding eHealth-related topics provided by the faculty, students stated they received courses on ethical discussions (247/451, 54.8%), followed by research opportunities (211/451, 46.8%), computer science (138/451, 30.6%), usage of eHealth technologies (83/451, 18.4%), start-up possibilities (69/451, 15.3%), and other (36/451, 8.0%). One-quarter (110/451, 24.4%) of the respondents stated that their faculty provided no eHealth-related courses at all. One-third (149/451, 33.0%) said that they were not informed enough to answer this question (Figure 4).

    ‎
    Figure 2. Self-evaluation of the respondents' eHealth skills in relation to their time spent using eHealth technologies in response to the statement: "I evaluate my eHealth skills (eg, working with clinical decision support systems, remote patient monitoring systems, artificial intelligence, applications in radiology) as...".
    View this figure
    ‎
    Figure 3. Self-evaluation in response to "Regarding the statement 'I would like eHealth to be more implemented in the medical curriculum,' do you...?".
    View this figure
    ‎
    Figure 4. Overview of eHealth-related topics provided by the respondents’ faculty (ie, response to the question: "What eHealth-related topics does your faculty provide courses on?).
    View this figure

    Qualitative Results

    Defining eHealth

    Regarding the task “Please define eHealth in your own words,” 30.3% (137/451) did not give a definition and 1.5% (7/451) admitted not knowing how to define the term eHealth. The resulting 68.1% (307/451) provided definitions that were coded and categorized using a grounded theory approach. Of these, 41.7% (128/307) provided a definition coded as Usage of technologies in health (Figure 5, Multimedia Appendix 2). The specifications were related to the purpose of technology application in health: (1) delivery of health, (2) monitoring and documentation of health, (3) communication (between health care professionals as well as between doctors and patients), and (4) public health (eg, health promotion, patient-centered health care); 18.9% (58/307) of the definitions fell into the category Health care services for patients and health care professionals where eHealth was seen either as a complement to doctors or as a replacement for them (eg, image recognition, clinical decision support systems); 15.6% (48/307) provided definitions of eHealth assigned the code Technologies affecting the health care sector, naming examples of software, hardware, internet tools, and health apps. The last category, eHealth as a Field of medicine putting digital technologies into practice and research accounted for 7.1% (22/307). The remaining 16.6% (51/307) could not be allocated to any of the preceding categories and fell under the category Other.

    ‎
    Figure 5. Responses to the statement: "Please define eHealth in your own words." The percentages are in relation to the number of definitions given.
    View this figure
    Reasoning For and Against eHealth in Medical Education

    The task “Regarding the statement ‘I would like eHealth to be more implemented in the medical curriculum,’ do you strongly agree, agree, undecided, disagree, strongly disagree? Why?” resulted in a total amount of 151 qualitative answers (151/451, 33.5%). The majority of all respondents (383/451, 84.9%) wished for a stronger implementation of eHealth into the medical curriculum. The reasons provided were that it would prepare them for their future work environment and that learning about eHealth in medical school would be part of keeping the curriculum up-to-date with the latest developments in medicine.

    Medicine is a science that should always follow and walk hand by hand with the progress in other resources. Medical school should adapt those changes and provide the best knowledge to the upcoming physicians.
    [survey response]

    Furthermore, medical students justified their wish for personal benefits such as digital health literacy as a job qualification and to maintain physicians’ responsibilities and power within the hospital environment and society.

    ...doctors have to be in charge not IT companies, doctors have to fight for their interest; this is only possible if we have the knowledge and skills...
    [survey response]

    Respondents stated that the implementation of eHealth into medical education could decrease the students’ doubts about eHealth technologies. Education on eHealth would drive its effective implementation into health care systems and innovation processes in health. Respondents who were undecided said this was because of a lack of capacities due to an already high workload in medical studies. Some felt that both the quality of education and the resources at their university were not sufficient to sustain the implementation of digital health into the curriculum. Among the reasons for respondents disagreeing were arguments that eHealth topics were already implemented in their curriculum or that digital health literacy should have already been taught in earlier education.

    No, this starts much earlier. Dealing with modern technology needs to be taught properly in schools. At university level, students should have basic abilities to deal with ehealth on their own!
    [survey response]
    Reasons for Feeling or Not Feeling Prepared to Work With eHealth Technologies

    The results to the question of whether the respondents felt prepared to work in a digitized health care system (and why or why not) showed mixed opinions. The majority of all respondents (183/451, 40.6%) stated they felt prepared; however, this was not based on adequate training in their faculty but rather on their own technical skills and interests. The respondents agreed that eHealth should be addressed in more medical curricula, both technically-theoretically and practically.

    The generation now studying medicine was raised during digitalization so we are very skilled in using its products.
    [survey response]
    My ability to use electronic devices and software in general is good, still it would be nice to be prepared for specific software and situations. In university digitization is not considered at all, neither in teaching nor in patient care.
    [survey response]

    Respondents not feeling prepared for working in a digitized health care system (130/451, 28.8%) justified this with no or only little eHealth education in their curriculum. They emphasized their willingness to learn more about eHealth.

    I haven’t received enough eHealth knowledge or practical skills in order to be able to work appropriately in a digitized health care system. In my opinion, I do not feel completely prepared for working in a digitalized health care system as there is a lack of training in our medical curricula and poor sources in our health care system.
    [survey response]

    Respondents who were undecided (129/451, 28.6%) stated that they did not experience any or enough training to evaluate their expertise. They, however, were willing to gain new or extend existing skills:

    I’m not sure what a digitized health care system is and I don’t remember if I ever had to interact with it. However, I think it’s something I might be able to handle with my present knowledge.
    [survey response]
    Opinions on How to Implement eHealth in Medical Education

    The question “What eHealth-related courses would you like to have in your university’s curriculum? (free text)” received 451 answers. Of those, 14.2% (64/451) stated they did not know which courses should be implemented, 9.1% (41/451) gave no answer, and 4.0% (18/451) did not want to have any courses on eHealth. The remaining 72.7% (328/451) were coded and divided into the following categories: Courses, Technologies, and Strategy (Figure 6); hereby, one response could be affiliated with several categories.

    Regarding Courses, respondents pointed out their need for an introduction to eHealth. For specific courses, education on data management including big data analysis, data sharing, and data security was mentioned most often. Furthermore, practical training with eHealth technologies and courses in computer science (eg, programming languages, app development), the operating principles of eHealth technologies ethical aspects, legal frameworks, research and entrepreneurial opportunities related to eHealth, and the role of eHealth in public health and in health systems were requested. Several respondents wished for courses focusing on communication skills, in particular on how to advise and guide patients using digital health tools. Regarding Technologies, respondents wanted to learn about recent examples of eHealth technologies (eg, telehealth and mHealth applications, virtual reality simulations, and robotic applications), health information systems (eg, electronic patient records), and artificial intelligence in health care (eg, clinical decision support systems). Regarding Strategy (of implementing eHealth into the medical curriculum), respondents suggested the introduction of cross-disciplinary courses (eg, eHealth in radiology, cardiology), or on the other hand, that eHealth be made a new individual discipline. In both cases, it was suggested that training should be tailored to future job requirements and taught interprofessionally, by involving information technology specialists.

    ‎
    Figure 6. Free text categories derived from responses to the question: "What specific eHealth courses would you like to be implemented in your university’s curriculum?".
    View this figure

    Principal Findings

    The results of this survey illustrated a gap between digital health literacy among medical students and the lack of training and education, despite the willingness of medical students to become key players in the digital transformation of health care. Qualitative analysis of eHealth definitions showed that the student understanding of the subject of eHealth varied considerably. Still, we found all domains of eHealth as described previously by Shaw et al [28] to be covered by the responses. The study showed that student needs in terms of digital health education and training are global and specific; students in the study wished for the implementation of courses ranging from programming languages, legal aspects, and guiding patients using digital health technologies. Students also demanded training with already established health information technology such as health information systems. We found mixed opinions on whether eHealth education should be implemented as a separate discipline or in an integrated form. The respondents highlighted the importance of interprofessional education.

    Digital Health Literacy and Skills for the Future Health Workforce

    Among the major factors enabling the successful implementation of eHealth are digital health literacy, digital skills of health care professionals, along with health care professionals’ trust in the potential of novel digital health solutions [9,29]. Furthermore, education and training of health care professionals have been identified as key facilitators of digital health implementation [4,8]. The promotion of these factors requires a multifaceted and diverse approach in order to engage with different stakeholders [1,12,16].

    Identifying health care professionals and health care students as central connectors within a challenging digitalization of the health care sector is paramount to its efficacy. In accordance with previous research on the digital skills of health care professionals [12,16,30], our survey showed a gap between the overall willingness of students to become key players in a meaningful digitalization of health care and the competencies and skills they have acquired through their learning.

    Future-Proof Health Care Curriculum

    In 2019, the Topol Review [14] identified the top digital technologies affecting 80% of the health workforce in 2040: digital medicine (eg, telemedicine and mHealth), artificial intelligence, robotics, and genomics. Future health care professionals have to become aware of the ethical and patient safety considerations posed by the digital transformation of health care [31]. Additionally, the shift away from technical tasks toward a more patient-centered medicine accompanied by the change in the traditional doctor-patient relationship requires different communication skills [13]. Continuous education and training of health care professionals regarding digital health literacy and skills help to ensure the most effective application of digital health to clinical workflows [32,33]. However, medical students do not necessarily need to become experts in programming and data science as the medical profession will probably continue to be a social one. In line with these considerations and recent publications [14,34,35], the findings of this study indicate that medical education should incorporate basic courses on digital medicine, artificial intelligence in health care, genomics, and data science.

    Including the various dimensions of digital health in health curricula is a challenging, yet urgently needed task. Education providers can and should build upon preliminary work [36-41]. Priorities and action plans for the improvement of information technology skills in the EU health care workforce have recently been set out [37].

    The recognition of digital skills as key competencies by national training commissions and examination authorities is an essential factor for driving the implementation of this content in health care education [42]. To date, accreditation of digital health literacy and skills in national medical education frameworks is lacking [36,42]. The renowned CanMEDs framework, outlining core competencies for future physicians and serving as orientation for medical educators globally, mentions the term digital only in the context of communication and documentation of health data [43]. Our findings support suggestions to revise the CanMEDs and other national frameworks according to the requirements that arise in relation to digital health technologies [13,35,42,44].

    Practicing physicians are educators whose proficiency in digital health has a direct impact on the learning outcomes of undergraduates [37]. Therefore, teaching digital health literacy and skills should follow a holistic approach and be integrated into both undergraduate and continuing medical education [45-48].

    Interprofessional Collaboration and International Best Practice Exchange

    The creation of future-proof health curricula involves the integration of interprofessional collaboration and supranational coordination and monitoring. Evolving through the disruptive changes brought about by digitalization, health care engages many different professions [44,49]. Topics related to engineering, computer science, and entrepreneurship have become increasingly relevant for medical professionals [49,50]. Our study indicates that medical students are eager to learn interprofessionally and wish to include input from other fields to their education.

    In Europe, several networks have been established to support the digitalization of health care and its implementation in health care education [51-54]. Approaches coordinated by the European Commission or EU member states take into account the heterogeneity of European health care systems and the different pace of digitalization in EU member states [16,51]. Additionally, the European exchange of best practices on an institutional level is focused on driving advances in digital health literacy for health care professionals and digital health implementation [14,55]. For instance, platforms such as the European Institute of Innovation and Technology Health, the European Deans’ meeting Training Future-Proof Doctors for the Digital Society, and respective medical education conferences [31,53,56,57] bring together experts and stakeholders in the field. Such platforms for interprofessional collaboration and best practice exchange as well as cross-disciplinary training are essential to ensure continuous improvements in a rapidly changing field.

    Limitations

    Our survey was answered by first- to seventh-year medical students throughout Europe. Some of the questions may not have been fully answered by first-year medical students, as some may not yet have had a complete overview of their medical curriculum. Moreover, it is important to mention that the students’ perception of their curriculum might differ from the actual courses offered by their university. The lack of awareness among students suggests faculties need to promote already existing courses.

    Our findings may even overestimate the digital health literacy of European medical students. This may be due to two aspects. First, the voluntary mode of participation may suggest that respondents were already aware of and interested in digital health. Thus, it is possible that they do not represent the average knowledge and skill level of medical students regarding digital health and are already more confident about their e-skills abilities than their colleagues would be. Second, people with weak skills tend to overestimate their skills and expected performance [58]. Further research on the eHealth literacy of medical students using validated assessment tools [59,60] would be necessary in order to obtain a clearer and more objective picture.

    Due to the limited number of respondents, our results can be considered neither complete nor representative of the European region. Rather, the findings should be seen as a starting point for further research in this area. In a follow-up study, various results could be specifically queried again and linked to current developments in health care education. In addition, findings can be an impetus for policymakers and stakeholders in health care education to revise their approach to meet the needs of future health care professionals with regard to digital skills and eHealth literacy.

    The study was conducted in English only, which may have resulted in misunderstandings, inaccuracies, and flawed answers due to a language barrier. Despite the definitions of technical terms given in the survey, their complexity and ambiguity may have caused difficulties in understanding, and subsequently, in answering survey questions. Furthermore, when responding to a Likert scale, it has been shown that respondents were more likely to pick answers in the middle of the scale when responding to an English-language questionnaire compared to a questionnaire in their native language. In turn, questionnaires in a respondents’ native language provoked a higher level of extreme responses [61]. Next to this, the field of eHealth is multidimensional and often not directly labeled as such [28,62], which could have led to misunderstandings among respondents, and thus, biased the survey results.

    Conclusion

    This study was the first pan-European approach assessing the needs of medical students regarding digital health literacy and digital skills in medical education. We revealed that the majority of European medical students have a positive attitude toward the digitalization of health care and are willing to play an active role and take responsibility especially as mediators of digital health literacy to patients. However, we also found a lack of knowledge and skills regarding the adequate use and evaluation of digital health technologies, attributed to a lack of respective topics in the medical curriculum. We showed the students’ demand for new, additional teaching concepts ranging from technical-theoretical issues (data management, computer science, legal, and ethical aspects) to practical training with specific technologies and patient communication.

    The apparent gap between the overall willingness of medical students to become key players in the digitalization of health care and the education they receive poses a significant challenge to the successful implementation of digital technologies into health care settings. Education providers and policymakers should acknowledge the central role of future health care professionals in health innovation, develop interprofessional concepts ensuring continuous learning, and evaluate them in a continual exchange among themselves and with their students, adapting to the latest scientific and technological developments.

    Further research on the specific needs of health care professionals is necessary as new challenges in the growing field of digital health continuously arise. The medical curriculum is essential to create preparatory experiences regarding digital health literacy and digital skills before students enter their professional life. Our findings support the call for faculties and medical education institutions to collaboratively establish targeted, customized, and efficient education and training on digital health.

    Acknowledgments

    We express our gratitude to the respondents of our survey.

    We would like to thank the following students who have contributed to our work: Paulius Povilonis, Cristina-Nelida Bora, Maria Roque, Aykut Uçar, Dilara Tüzün, Friederike Münn, Sebastian Weyn Banningh, Philippa Lantwin, Leu Huang, Sophia Hodgkinson, Jolan van Maaren, Orsolya Süli, and Tuğçe Çetin. Furthermore, we would like to thank advisors Felix Balzer, Akira-Sebastian Poncette, Armin Ott, Bianca Jansky, Michael Rosenblatt, Carole Rouaud, Jehona Krasniqi for their support in the conception of the study and survey design.

    Authors' Contributions

    FM, JB, and LM conceived the study. The survey was developed by FM, JB, FVM, and LM. RK performed the statistical analysis. LM and DJ conducted the qualitative data analyses. FM, RK, and LM wrote the manuscript, supported by JB, BA, and FVM. LM and FM supervised all parts of the study. All authors critically reviewed and approved the manuscript.

    Conflicts of Interest

    None declared.

    ‎
    Multimedia Appendix 1

    Catalog of all survey questions.

    PDF File (Adobe PDF File), 237 KB
    ‎
    Multimedia Appendix 2

    Codes and codings for "Please define eHealth in your own words.".

    PDF File (Adobe PDF File), 222 KB
    ‎
    Multimedia Appendix 3

    Codes and codings for "Regarding the statement 'I would like eHealth to be more implemented in the medical curriculum,' do you... strongly agree, agree, undecided, disagree, strongly disagree, no answer - Why?".

    PDF File (Adobe PDF File), 160 KB
    ‎
    Multimedia Appendix 4

    Codes and codings for "Regarding the statement 'I feel prepared for working in a digitized health care system,' do you... strongly agree, agree, undecided, disagree, strongly disagree, no answer - Please elaborate.".

    PDF File (Adobe PDF File), 123 KB
    ‎
    Multimedia Appendix 5

    Codes and codings for "What specific eHealth-related courses would you like to have in your university’s curriculum?".

    PDF File (Adobe PDF File), 274 KB
    ‎
    Multimedia Appendix 6

    Supplementary figures.

    PDF File (Adobe PDF File), 508 KB
    1. European C. Communication on enabling the digital transformation of health and care in the Digital Single Market; empowering citizens and building a healthier society. https://ec.europa.eu.: European Commission; 2018 Apr 25.   URL: https://ec.europa.eu/newsroom/dae/document.cfm?doc_id=51628 [accessed 2020-04-03]
    2. Richard S, John R B, Ties B, Somnath C. Health in an ageing world—what do we know? The Lancet. 2005 Nov.   URL: https://www.thelancet.com/journals/lancet/article/PIIS0140-6736(14)61597-X/fulltext [accessed 2019-06-22]
    3. Non CDI. Noncommunicable diseases. World Health Organization. 2018 Jun 01.   URL: https://www.who.int/news-room/fact-sheets/detail/noncommunicable-diseases [accessed 2019-06-22]
    4. World Health Organization. WHO Guideline: recommendations on digital interventions for health system strengthening. World Health Organization. WHO guideline; 2019.   URL: https://apps.who.int/iris/bitstream/handle/10665/311941/9789241550505-eng.pdf?ua=1 [accessed 2020-04-02]
    5. McCall B. COVID-19 and artificial intelligence: protecting health-care workers and curbing the spread. The Lancet Digital Health 2020 Apr;2(4):e166-e167. [CrossRef]
    6. Greenhalgh T, Wherton J, Shaw S, Morrison C. Video consultations for covid-19. BMJ 2020 Mar 12:m998. [CrossRef]
    7. Kritikos M. Ten technologies to fight coronavirus. EPRS | European Parliamentary Research Service.: European Parliament; 2020 Apr.   URL: https://www.europarl.europa.eu/RegData/etudes/IDAN/2020/641543/EPRS_IDA(2020)641543_EN.pdf [accessed 2020-05-22]
    8. Wachter R. Making IT Work: Harnessing the Power of Health Information Technology to Improve Care in England. Report of the National Advisory Group on Health Information Technology in England.: The National Advisory Group on Health Information Technology in England; 2016.   URL: https:/​/assets.​publishing.service.gov.uk/​government/​uploads/​system/​uploads/​attachment_data/​file/​550866/​Wachter_Review_Accessible.​pdf [accessed 2019-06-22]
    9. Ross J, Stevenson F, Lau R, Murray E. Factors that influence the implementation of e-health: a systematic review of systematic reviews (an update). Implementation Sci 2016 Oct 26;11(1):146. [CrossRef]
    10. Hüsers J, Hübner U, Esdar M, Ammenwerth E, Hackl WO, Naumann L, et al. Innovative Power of Health Care Organisations Affects IT Adoption: A bi-National Health IT Benchmark Comparing Austria and Germany. J Med Syst 2017 Feb;41(2):33 [FREE Full text] [CrossRef] [Medline]
    11. Bertelsmann Stiftung. Summary #SmartHealthSystems - Focus Europe. https://www.bertelsmann-stiftung.de/en/publications.: Bertelsmann Stiftung; 2019.   URL: https:/​/www.​bertelsmann-stiftung.de/​fileadmin/​files/​Projekte/​Der_digitale_Patient/​VV_SHS_Europe_eng.​pdf [accessed 2020-04-02]
    12. European HP. Digital Skills for Health Profesionals. European Health Parliament. 2016.   URL: https:/​/www.​healthparliament.eu/​wp-content/​uploads/​2017/​09/​Digital-skills-for-health-professionals.​pdf [accessed 2020-04-02]
    13. Masters K. Preparing medical students for the e-patient. Medical Teacher 2017 May 22;39(7):681-685. [CrossRef]
    14. Topol E. Preparing the healthcare workforce to deliver the digital future Internet. NHS Health Education England. 2019 Feb.   URL: https://topol.hee.nhs.uk/wp-content/uploads/HEE-Topol-Review-2019.pdf [accessed 2019-06-22]
    15. European Commission. Transatlantic eHealth/health IT Cooperation Roadmap. https://ec.europa.eu.: European Comission; 2013 Mar 15.   URL: https:/​/ec.​europa.eu/​digital-single-market/​en/​news/​transatlantic-ehealthhealth-it-cooperation-roadmap [accessed 2019-06-22]
    16. Keane L. D6.3 Report on eSkills for Professionals Information Note; WP 6 Enhancing Continuity of Care. 15th eHN meeting, June 2019.: eHAction - Joint Action supporting the eHealth Network; 2019 Jun.   URL: https://ec.europa.eu/health/sites/health/files/ehealth/docs/ev_20190611_co322_en.pdf [accessed 2019-09-13]
    17. The Evidence Centre for Skills for Health. How do new technologies impact on workforce organisation? https://www.skillsforhealth.org.uk/. 2011.   URL: https://www.skillsforhealth.org.uk/index.php?option=com_mtree&task=att_download&link_id=101&cf_id=24 [accessed 2019-06-22]
    18. GOVUK Department for Digital, Culture, Media & Sport. Digital skills and inclusion - giving everyone access to the digital skills they need Internet. GOVUK Department for Digital, Culture, Media & Sport. 2017 Mar.   URL: https:/​/www.​gov.uk/​government/​publications/​uk-digital-strategy/​2-digital-skills-and-inclusion-giving-everyone-access-to-the-digital-skills-they-need [accessed 2019-06-22]
    19. Parry E, Battista V. The impact of emerging technologies on work: a review of the evidence and implications for the human resource function. Emerald Open Res 2019 Jan 28;1:5. [CrossRef]
    20. European Medical Students' Association. European Medical Students' Association - Association Européenne des Étudiants en Médecine (EMSA). https://emsa-europe.eu/.   URL: https://emsa-europe.eu/ [accessed 2019-09-13]
    21. Regulation. Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46. https://eur-lex.europa.eu/. 2016.   URL: https://eur-lex.europa.eu/eli/reg/2016/679/2016-05-04 [accessed 2019-06-22]
    22. Jansen H. The Logic of Qualitative Survey Research and its Position in the Field of Social Research Methods. Forum Qual Sozialforschung Forum Qual Soc Res 2010 Mar 21;Vol 11:No 2 ( 2010;Visualising Migration and Social Division:2-18 [FREE Full text] [CrossRef]
    23. Guetterman TC, Babchuk WA, Howell Smith MC, Stevens J. Contemporary Approaches to Mixed Methods–Grounded Theory Research: A Field-Based Analysis. Journal of Mixed Methods Research 2017 Jun 02;13(2):179-195. [CrossRef]
    24. Watling CJ, Lingard L. Grounded theory in medical education research: AMEE Guide No. 70. Med Teach 2012 Jan;34(10):850-861. [CrossRef] [Medline]
    25. The R Project for Statistical Computing. https://www.r-project.org/.   URL: https://www.r-project.org/ [accessed 2019-06-22]
    26. Boyatzis R. Transforming Qualitative Information: Thematic Analysis and Code Development. London and New Delhi: SAGE; 1998:-978.
    27. MAXQDA 2020. MAXQDA 2020. Berlin, Germany: VERBI Software; 2019.   URL: https://maxqda.com [accessed 2020-04-02]
    28. Shaw T, McGregor D, Brunner M, Keep M, Janssen A, Barnet S. What is eHealth (6)? Development of a Conceptual Model for eHealth: Qualitative Study with Key Informants. J Med Internet Res 2017 Oct 24;19(10):e324 [FREE Full text] [CrossRef] [Medline]
    29. Lennon MR, Bouamrane M, Devlin AM, O'Connor S, O'Donnell C, Chetty U, et al. Readiness for Delivering Digital Health at Scale: Lessons From a Longitudinal Qualitative Evaluation of a National Digital Health Innovation Program in the United Kingdom. J Med Internet Res 2017 Feb 16;19(2):e42 [FREE Full text] [CrossRef] [Medline]
    30. Stellefson M, Hanik B, Chaney B, Chaney D, Tennant B, Chavarria EA. eHealth literacy among college students: a systematic review with implications for eHealth education. J Med Internet Res 2011 Dec;13(4):e102 [FREE Full text] [CrossRef] [Medline]
    31. European Commission. Digital is transforming medical doctors? daily work: deans and student doctors agree on common principles to adapt medical doctors? training Internet. European Commission. 2019.   URL: https:/​/ec.​europa.eu/​digital-single-market/​en/​news/​digital-transforming-medical-doctors-daily-work-deans-and-student-doctors-agree-common [accessed 2019-06-22]
    32. Torous J, Chan S, Luo J, Boland R, Hilty D. Clinical Informatics in Psychiatric Training: Preparing Today's Trainees for the Already Present Future. Acad Psychiatry 2018 Oct;42(5):694-697. [CrossRef] [Medline]
    33. Cresswell KM, Lee L, Mozaffar H, Williams R, Sheikh A. Sustained User Engagement in Health Information Technology: The Long Road from Implementation to System Optimization of Computerized Physician Order Entry and Clinical Decision Support Systems for Prescribing in Hospitals in England. Health Serv Res 2016 Oct 07;52(5):1928-1957. [CrossRef]
    34. Fridsma DB. Health informatics: a required skill for 21st century clinicians. BMJ 2018 Jul 12:k3043. [CrossRef]
    35. Rampton V, Mittelman M, Goldhahn J. Implications of artificial intelligence for medical education. The Lancet Digital Health 2020 Mar;2(3):e111-e112. [CrossRef]
    36. Gray K, Dattakumar A, Maeder A, Butler-Henderson K, Chenery H. Advancing ehealth education for the clinical health professions: final report. Australia, Office for Learning and Teaching. 2014.   URL: https:/​/research.​bond.edu.au/​files/​26149651/​Advancing_Ehealth_Education_for_the_Clinical_Health_Professions_Final_Report_2014.​pdf [accessed 2019-09-12]
    37. Li S, Bamidis PD, Konstantinidis ST, Traver V, Car J, Zary N. Setting priorities for EU healthcare workforce IT skills competence improvement. Health Informatics J 2017 Apr 25;25(1):174-185. [CrossRef]
    38. Mesko B, Győrffy Z, Kollár J. Digital Literacy in the Medical Curriculum: A Course With Social Media Tools and Gamification. JMIR Medical Education 2015 Oct 01;1(2):e6. [CrossRef]
    39. Milano CE, Hardman JA, Plesiu A, Rdesinski RE, Biagioli FE. Simulated electronic health record (Sim-EHR) curriculum: teaching EHR skills and use of the EHR for disease management and prevention. Acad Med 2014 Mar;89(3):399-403 [FREE Full text] [CrossRef] [Medline]
    40. Brunner M, McGregor D, Keep M, Janssen A, Spallek H, Quinn D, et al. An eHealth Capabilities Framework for Graduates and Health Professionals: Mixed-Methods Study. J Med Internet Res 2018 May 15;20(5):e10229 [FREE Full text] [CrossRef] [Medline]
    41. Varghese J, Röhrig R, Dugas M. Which competencies in medical informatics are required by physicians? An update of the catalog of learning objectives for medical students. GMS Med Inform Biom Epidemiol German Medical Science GMS Publishing House 2020;16(1):-3 [FREE Full text] [CrossRef]
    42. Edirippulige S, Brooks P, Carati C, Wade VA, Smith AC, Wickramasinghe S, et al. It’s important, but not important enough: eHealth as a curriculum priority in medical education in Australia. J Telemed Telecare 2018 Oct 22;24(10):697-702. [CrossRef]
    43. Frank J, Snell L, Sherbino J. CanMEDS 2015: physician competency framework. In: Royal College of Physicians and Surgeons of Canada. Ottawa, Canada: Royal College of Physicians and Surgeons of Canada; 2015:1-978.
    44. Ahmadvand A, Gatchel R, Brownstein J, Nissen L. The Biopsychosocial-Digital Approach to Health and Disease: Call for a Paradigm Expansion. J Med Internet Res 2018 May 18;20(5):e189. [CrossRef]
    45. Hilty DM, Turvey C, Hwang T. Lifelong Learning for Clinical Practice: How to Leverage Technology for Telebehavioral Health Care and Digital Continuing Medical Education. Curr Psychiatry Rep 2018 Dec 12;20(3):15. [CrossRef] [Medline]
    46. Cullen MW, Geske JB, Anavekar NS, McAdams JA, Beliveau ME, Ommen SR, et al. Reinvigorating Continuing Medical Education: Meeting the Challenges of the Digital Age. Mayo Clinic Proceedings 2019 Dec;94(12):2501-2509. [CrossRef]
    47. Schmuter G. Envirotyping in Medical Schools: A Movement Toward Digital Health Literacy. American Journal of Ophthalmology 2019 Nov;207:427-428. [CrossRef]
    48. Stathis K, Li S, Traver V, Zary N, Bamidis P. Actions to Empower Digital Competences in Healthcare Workforce: A Qualitative Approach. Stud Health Technol Inform Netherlands: IOS Press 2017:253-256. [CrossRef]
    49. Nelson R, Staggers N. Health informatics: an interprofessional approach. In: Health informatics: an interprofessional approach. Amsterdam, Netherlands: Elsevier; 2018:a.
    50. Salminen L, Lindberg E, Gustafsson M, Heinonen J, Leino-Kilpi H. Entrepreneurship Education in Health Care Education. Education Research International 2014;2014:1-8. [CrossRef]
    51. European Commission. eHealth Network. https://ec.europa.eu/.   URL: https://ec.europa.eu/health/ehealth/policy/network_en [accessed 2019-07-22]
    52. eHAction - Joint Action to Support the eHealth Network Internet. eHAction.   URL: http://ehaction.eu/ [accessed 2019-09-12]
    53. EIT Health - about Internet. EIT Health.   URL: https://eithealth.eu/who-we-are/ [accessed 2019-09-12]
    54. European C. 2020 Thematic Network Digital Skills for future-proof doctors – Digital Doc. https://ec.europa.eu/.   URL: https://ec.europa.eu/health/sites/health/files/policies/docs/ev_20200618_ag_en.pdf [accessed 2020-06-18]
    55. Abbott PA, Foster J, Marin HDF, Dykes PC. Complexity and the science of implementation in health IT--knowledge gaps and future visions. Int J Med Inform 2014 Jul;83(7):e12-e22. [CrossRef] [Medline]
    56. AMEE - conferences. AMEE.   URL: https://amee.org/conferences [accessed 2019-09-12]
    57. Society for Simulation in Europe. SESAM.   URL: https://www.sesam-web.org [accessed 2019-09-12]
    58. Dunning D, Johnson K, Ehrlinger J, Kruger J. Why People Fail to Recognize Their Own Incompetence. Curr Dir Psychol Sci 2016 Jun 23;12(3):83-87. [CrossRef]
    59. Karnoe A, Furstrand D, Christensen KB, Norgaard O, Kayser L. Assessing Competencies Needed to Engage With Digital Health Services: Development of the eHealth Literacy Assessment Toolkit. J Med Internet Res 2018 May 10;20(5):e178 [FREE Full text] [CrossRef] [Medline]
    60. Norman CD, Skinner HA. eHEALS: The eHealth Literacy Scale. J Med Internet Res 2006 Nov;8(4):e27 [FREE Full text] [CrossRef] [Medline]
    61. Harzing A. Response Styles in Cross-national Survey Research. Int'l Jnl of Cross Cultural Management 2016 Jul 24;6(2):243-266. [CrossRef]
    62. Chris S, Christian N. How Should We Define eHealth, and Does the Definition Matter? Studies in Health Technology and Informatics 2012;180:881-884 [FREE Full text] [CrossRef]

    ‎
    EMSA: The European Medical Students’ Association

    Edited by G Eysenbach; submitted 04.05.20; peer-reviewed by W Ahmed, S Mehta Ambalal; comments to author 12.06.20; revised version received 24.06.20; accepted 25.06.20; published 14.08.20

    Copyright

    ©Felix Machleid, Robert Kaczmarczyk, Doreen Johann, Justinas Balčiūnas, Beatriz Atienza-Carbonell, Finn von Maltzahn, Lina Mosch. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 14.08.2020.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.