This paper is in the following e-collection/theme issue:

JMIR Theme Issue: COVID-19 Special Issue (2391) New Methods (909) Outbreak and Pandemic Preparedness and Management (1675)
Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/25283, first published .
Interdisciplinary Online Hackathons as an Approach to Combat the COVID-19 Pandemic: Case Study

Interdisciplinary Online Hackathons as an Approach to Combat the COVID-19 Pandemic: Case Study

Interdisciplinary Online Hackathons as an Approach to Combat the COVID-19 Pandemic: Case Study

Authors of this article:

Katarina Braune1, 2, 3 Author Orcid Image ;   Pablo-David Rojas2 Author Orcid Image ;   Joscha Hofferbert2 Author Orcid Image ;   Alvaro Valera Sosa2, 4, 5 Author Orcid Image ;   Anastasiya Lebedev2 Author Orcid Image ;   Felix Balzer6, 7 Author Orcid Image ;   Sylvia Thun3 Author Orcid Image ;   Sascha Lieber6, 8 Author Orcid Image ;   Valerie Kirchberger8 Author Orcid Image ;   Akira-Sebastian Poncette2, 3, 6, 7 Author Orcid Image
Article Authors Cited by (25) Tweetations (6) Metrics

    Original Paper

    1Department of Paediatric Endocrinology and Diabetes, Charité – Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin, Humboldt-Universität zu Berlin, and Berlin Institute of Health, Berlin, Germany

    2Hacking Health Berlin, Berlin, Germany

    3Berlin Institute of Health, Berlin, Germany

    4CityLAB Berlin, Building Health Lab, Berlin, Germany

    5Department of Design and Typologies, Technische Universität Berlin, Berlin, Germany

    6Charité – Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin, Humboldt-Universität zu Berlin, and Berlin Institute of Health, Department of Anesthesiology and Intensive Care Medicine, Berlin, Germany

    7Einstein Center Digital Future, Berlin, Germany

    8Executive Board, Charité – Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin, Humboldt-Universität zu Berlin, and Berlin Institute of Health, Berlin, Germany

    Corresponding Author:

    Katarina Braune, MD

    Department of Paediatric Endocrinology and Diabetes

    Charité – Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin

    Humboldt-Universität zu Berlin, and Berlin Institute of Health

    Augustenburger Platz 1

    Berlin, 13353

    Germany

    Phone: 49 30450616454

    Email: katarina.braune@charite.de


    Background: The COVID-19 outbreak has affected the lives of millions of people by causing a dramatic impact on many health care systems and the global economy. This devastating pandemic has brought together communities across the globe to work on this issue in an unprecedented manner.

    Objective: This case study describes the steps and methods employed in the conduction of a remote online health hackathon centered on challenges posed by the COVID-19 pandemic. It aims to deliver a clear implementation road map for other organizations to follow.

    Methods: This 4-day hackathon was conducted in April 2020, based on six COVID-19–related challenges defined by frontline clinicians and researchers from various disciplines. An online survey was structured to assess: (1) individual experience satisfaction, (2) level of interprofessional skills exchange, (3) maturity of the projects realized, and (4) overall quality of the event. At the end of the event, participants were invited to take part in an online survey with 17 (+5 optional) items, including multiple-choice and open-ended questions that assessed their experience regarding the remote nature of the event and their individual project, interprofessional skills exchange, and their confidence in working on a digital health project before and after the hackathon. Mentors, who guided the participants through the event, also provided feedback to the organizers through an online survey.

    Results: A total of 48 participants and 52 mentors based in 8 different countries participated and developed 14 projects. A total of 75 mentorship video sessions were held. Participants reported increased confidence in starting a digital health venture or a research project after successfully participating in the hackathon, and stated that they were likely to continue working on their projects. Of the participants who provided feedback, 60% (n=18) would not have started their project without this particular hackathon and indicated that the hackathon encouraged and enabled them to progress faster, for example, by building interdisciplinary teams, gaining new insights and feedback provided by their mentors, and creating a functional prototype.

    Conclusions: This study provides insights into how online hackathons can contribute to solving the challenges and effects of a pandemic in several regions of the world. The online format fosters team diversity, increases cross-regional collaboration, and can be executed much faster and at lower costs compared to in-person events. Results on preparation, organization, and evaluation of this online hackathon are useful for other institutions and initiatives that are willing to introduce similar event formats in the fight against COVID-19.

    J Med Internet Res 2021;23(2):e25283

    doi:10.2196/25283

    Keywords

    hackathon; COVID-19; digital health; mentoring; interdisciplinarity; interoperability; SARS-CoV-2; public health; innovation; collaboration; hack; mentor; case study; online health care; challenge; implementation; plan; collaboration 


    The COVID-19 outbreak is first and foremost a human tragedy, affecting the health of millions of people and overwhelming health care systems and the global economy [1]. This devastating pandemic has brought together communities across the globe to tackle this issue in an unprecedented manner.

    Hackathons are collaborative multiday events that bring people from different professional and personal backgrounds face to face in a venue to work on specific challenges. Previous publications have shown how health hackathons work as an effective model of interdisciplinary collaboration, capable of forwarding recommendations and viable solutions effectively [2-36]; however, they often require several months of preparation and considerable financing [6,34].

    Little is so far known about how health hackathons can be executed remotely and in quick response to ongoing circumstances, as well as how they can contribute to solving challenges related to the COVID-19 pandemic, considering the amount and fast pace of evidence gathered on its impact and possible solutions [37-48].

    The following case study describes the execution of an online hackathon dedicated to the challenges of the COVID-19 pandemic and participant experience, which will be useful to other organizations and institutions in the health care sector.


    Study Setting

    Hacking Health is a global nonprofit organization that pairs innovators with health care experts [6]. Its Berlin chapter organized its first online hackathon in partnership with Charité – Universitätsmedizin Berlin (the largest university hospital in Europe), the Berlin Institute of Health, Diabetes Center Berne, and Data Natives (an online community of data scientists and developers). The event, called EasterHack [49], ran from Friday evening to Monday afternoon over the Easter weekend in 2020. Participants included physicians, health care professionals, researchers, patients, entrepreneurs, engineers, designers, developers, etc. Participation was open to everyone worldwide and was free of charge.

    Challenges

    The most critical issues caused by, or associated with, COVID-19 and the current state of possible solutions were reviewed and discussed in a series of five virtual meetings held across the interdisciplinary Hacking Health Berlin community in cooperation with the hospital’s executive board and different departments of Charité – Universitätsmedizin Berlin.

    A summary of this review was sent to, and discussed with, frontline clinicians and researchers for consideration, to ponder on its relevance, and to raise other questions related to the most pressing issues in their line of work. It was highly relevant for identifying unmet needs and urgent problems among health care professionals, vulnerable patient groups, and more generally on a population level, rather than ideas for solutions. Thereafter, a total of six challenges were identified by the Hacking Health Berlin team (Table 1) in close cooperation with the university hospital’s executive board and other institutes and departments (Institute of Hygiene and Tropical Medicine, Department of Anesthesiology and Intensive Care Medicine, Department of Psychosomatic Medicine, Department of Paediatric Endocrinology and Diabetes of Charité – Universitätsmedizin Berlin) and the open-source initiative CoEpi.org, an international online community of volunteers with different skillsets working on open-source solutions for privacy-first contact tracing. Participants were allowed to use their choice of technology to solve these challenges, without constraint or limitation.

    Table 1. Overview of the hackathon’s challenges. A total of six COVID-19–related challenges were identified in close cooperation with frontline physicians and researchers.
    ChallengeQuestions/considerations
    1. Protect high-risk populations
    • How can we better support the medical needs of high-risk populations (eg, people with pre-existing conditions such as cancer, diabetes, and immunodeficiencies)?
      • ‎
    • How can we identify high-risk patients and isolate them from the general population?
      • ‎
    • How can we support high-risk patients with preventive ambulatory treatment?
      • ‎
    • How can we deliver care for people with chronic conditions during a pandemic?
      • ‎
    2. Protect health care workers
    • How can we prevent medical staff from getting infected with SARS-CoV-2 in health care facilities?
      • ‎
    • How can we manage the increasing demand for personal protective equipment in patient care?
      • ‎
    3. Privacy-first contact tracing and digital epidemiology
    • What can we build on top of open-source solutions (such as CoEpi.org and COVID-watch.org) to allow users to opt in to disclose anonymous, nonidentifiable information that public health officials would find useful?
      • ‎
    • How can we authenticate positive COVID-19 tests without requiring public health authorities to do extra work to authenticate them for us?
      • ‎
    • How can we collect more information on Bluetooth low-energy proximity contact event duration and distance to inform risk scores?
      • ‎
    • How can we automate the provisional diagnosis of COVID-19 based solely on self-reported symptoms and self-collected data?
      • ‎
    4. Improve intensive care
    • How can mechanical ventilation be improved to reduce long-term damage to the lungs?
      • ‎
    • How can we ease prone positioning in ventilated patients?
      • ‎
    • How can we reduce the formation of aerosols?
      • ‎
    • How can we visually monitor agitated and hyperactive patients in the ICUa to prevent them from extubating or hurting themselves?
      • ‎
    • How can we give confused ICU patients in-room, time, and situation orientation?
      • ‎
    • How can we make the ICU a more natural and pleasant environment?
      • ‎
    • How can we apply evidence-based medicine in all ICUs by tele-ICU?
      • ‎
    • How can we prevent long-term complications following ICU treatment such as controlled cortical impact and postintensive care syndrome?
      • ‎
    5. Protect our mental health
    • How can we support health care workers in dealing with the stress and trauma that they might experience during a pandemic?
      • ‎
    • How can we identify and offer support to people, families, friends, and/or coworkers who are running into mental health issues?
      • ‎
    • How can we predict critical mental states through early detection?
      • ‎
    6. Your idea
    • Find a novel solution for your individual COVID-19 challenge, if it has not been listed above
      • ‎

    aICU: intensive care unit.

    Preparation

    The core organizing team consisted of 7 volunteers to set up digital tools; scheduling; marketing outreach strategy for recruitment of participants, cooperating partners, participant and mentor communication; and coordination. An implementation strategy previously designed and used for in-person hackathons by Charité – Universitätsmedizin Berlin and Hacking Health Berlin [6] was adjusted accordingly.

    Due to the novelty and urgency of the topic, organization of the hackathon began only 2 weeks prior to the event being completely prepared and executed online, which is different from former hackathons. In April 2020, travel and assembly restrictions, as well as social distancing recommendations, did not allow in-person meetings in Europe. However, the online nature of the event allowed participants and mentors from all over the world to participate regardless of their location, avoiding traveling costs and time.

    Participant Engagement

    Recent innovation studies have demonstrated interdisciplinarity among participants and within-team compositions are essential for the success of hackathon projects [50,51]. Therefore, it is essential to recruit participants from various relevant fields of expertise such as biomedical research, health care, psychology, education, design, economics, data science, computer science, engineering, social sciences, and politics. Future end-users of the solutions created at a hackathon should be included as early as possible to actively contribute to the cocreation process.

    To attract and recruit a diverse and interdisciplinary group of participants, previously existing multidisciplinary networks on LinkedIn [52] and Slack [53] actively engaged in health care innovation, and cooperating with Hacking Health Berlin, were targeted. This provided a great opportunity to reach participants, mentors, and cooperation partners who supported organizers and participating teams in developing projects before, during, and after the hackathon. As one of the largest local networks of innovation in health care, Hacking Health Berlin was able to recruit a large number of interdisciplinary skilled participants within only 5 days. Professionals working in relevant fields and interested in digital health were also targeted through promotional campaigns using LinkedIn, Facebook, Meetup, and other social media channels.

    Participants and mentors were able to sign up via an online form, where they provided information on their skillset, shared their experience level and accomplishments from previous hackathons, their motivations to participate, their desired team role during the hackathon, and a brief outline of their project idea.

    Online Tools

    To prepare for the hackathon, the organizers used Airtable [54] to arrange schedules, responsibilities, marketing, registration procedures, jury criteria definition, and the collection and structuring of all information provided to the teams before, throughout, and after the event. The Hacking Health Berlin team and external co-organizers had access to this platform.

    The online instant messenger tool Slack [53] was used for communication during the event to coordinate actions and share data via text and video. Public and closed subchannels were created for the organizing team, mentors, judges, and participants to communicate and execute activities. Additionally, participating teams created their private subchannels on Slack to communicate internally.

    Mentoring sessions were coordinated and held on the Mentornity platform [55], as described in a subsequent point. For the opening and closing ceremonies, moderations, pitches, and prototype showcasing sessions, video conferences were held via Zoom and live streamed to the public on Facebook and YouTube.

    The Hackathon

    The hackathon was conducted over a period of 4 days, from Friday to Monday afternoon. An example schedule is shown in Figure 1.

    ‎
    Figure 1. Activities and schedule of a 4-day online hackathon.
    View this figure

    A keynote from the director of the interoperability unit of the Berlin Institute of Health opened the event, motivated the teams, and pointed out the importance of this online hackathon to create solutions for the challenges of the COVID-19 pandemic.

    Following the opening ceremony, individual participants and pre-existing teams pitched their ideas for the challenges they selected. Additional participants joined the team of their choice. Team composition is a crucial part of hackathons. As previous research shows, highly diverse and multidisciplinary teams are more likely to be effective in developing creative and sustainable solutions [50,51]. This is particularly relevant in the complex field of digital health, where experts from different fields and areas are needed for development such as health care workers, patients and caregivers, developers, engineers, data scientists, designers, and experts in economics, legal and regulatory affairs. Team building was supported by volunteers from the Hacking Health Berlin team to foster interdisciplinary and international cooperation. A maximum of 6 people per team was recommended.

    After teams were formed, the idea and prototype were developed collaboratively over the course of 4 days aided by mentoring sessions. Daily morning and evening stand-up sessions were held virtually, open to all organizers, mentors, and participants.

    On day 4, solutions and prototypes were presented to the public and evaluated by an interdisciplinary jury entitled to award the most elaborate and promising solutions based on the criteria described below.

    Mentoring

    Mentors provided participants with valuable expert knowledge about medicine, design, business, IT (information technology), and other areas of expertise. They indicated their availability for mentoring within the 48-hour span of days 2 and 3. This setting was well embraced, especially by health care professionals—despite the pressing pandemic situation—and other professionals working from home.

    One week prior to the hackathon event, mentors with previous mentoring experience at Hacking Health Berlin events were recruited via email. New mentors were recruited through various local and international networks with the use of social media campaigns and direct messaging via networking platforms. In both cases, mentors were asked to fill in an online form indicating fields of expertise and availability (ie, convenient time slots). This information was transferred to the online platform Mentornity. The platform showed each mentor’s profile and facilitated video call sessions for each team. Simultaneously, participants created team profiles for their project, searched mentors by expertise area, and booked at least one mandatory mentoring session made available by the mentor. The platform’s analysis feature tracked the number of mentoring sessions held by every mentor and team. Likewise, mentors were encouraged to sign up for Slack—the hackathon’s communication platform—to keep everyone informed on sessions prior to and during the event.

    Two experts in the field of communication and public speaking offered dedicated pitch training to help teams develop focused final pitches that adhered to the 3-minute time limit.

    Jury and Project Evaluation

    The interdisciplinary jury was composed of 10 members: a professor of anesthesiology and computer science, a pediatrician and hospital executive board member, a professor of medical informatics, an entrepreneur with a background in open-source technology, a medical doctor and high-risk patient, a senior doctor in hygiene, a digital health connector, a designer, a chief information and digital officer, and the managing director of a research institution.

    During the final presentations, teams were asked to first name the challenge or problem intended to be solved, then showcase the solution developed, and how it can be integrated into health care systems. The winning teams were awarded monetary prizes of 5000 EUR (US $6059) in total following the evaluation criteria shown in Table 2.

    Table 2. Evaluation criteria for the projects presented in the final pitch session at the end of the hackathon.
    CriterionConsiderations
    InnovationHow innovative is the idea? Does the approach stand out from solutions that are available so far?
    ImpactHow high is the benefit for the patient and other stakeholders of the health care system? Does the solution generate impact for our society? How viable is the idea?
    ApplicabilityHow user-friendly is the solution? Is the solution convenient to use? How viable is the solution?
    FeasibilityHow realistic is the implementation of the idea? Can the solution be brought to life by further research projects or by founding a company?
    ExecutionHow sophisticated and well-made is the technical implementation?

    Data Collection and Analysis

    Two online surveys were designed by the Hacking Health Berlin team, one survey targeting participants and another one for mentors of the hackathon (Multimedia Appendices 1 and 2). The research team members comprised health care professionals, scientists, and entrepreneurs with backgrounds in anesthesiology, intensive care medicine, pediatrics, psychology, microbiology, medical informatics, neuroscience, human factors, health care building design, human-centered design, and marketing. Google Forms was used to collect responses. CHERRIES (Checklist for Reporting Results of Internet E-Surveys) was used to guide survey development (Multimedia Appendix 3) [56].

    The participant survey comprised 17 plus 5 optional items (using branching logic, dependent on responses to previous questions). The questions assessed quality experience throughout the online hackathon, including the mentorship sessions and future plans regarding the projects presented. Furthermore, mentors provided feedback to the organizers via an online survey through Mentornity subsequently to each session (Multimedia Appendix 1). All data were collected during a single event with surveys being distributed to mentors and participants at the end of the hackathon. Surveys were anonymous and voluntary and included electronic consent. Participant responses were collected, managed, and analyzed using Google Forms. A thematic hackathon T-shirt was offered as a reward for completing the surveys.


    Participants

    In total, 171 participants, of whom 42% (n=72) were female, expressed their interest and preregistered online within 5 days prior to the hackathon. Of the total, 84 participants had joined the Zoom event for the opening ceremony, 62 of whom formed 23 different teams. Finally, 14 teams comprising 48 members pitched their completed projects at the end of the hackathon.

    Participants based in at least 8 countries participated, including Germany (n=25, 52%), the United States (n=5, 10%), Bosnia and Herzegovina (n=3, 6%), Canada (n=3, 6%), and others (n=5, 10%) like Belgium, Sri Lanka, Switzerland, and the United Kingdom, as well as 7 participants (15%) from unknown locations. Various professional backgrounds were represented, with IT (n=8, 17%), research (n=5, 10%), project management (n=4, 8%), and software engineering (n=4, 8%) being the most frequent work fields. Other backgrounds (n=27, 56%) included business development, data science, entrepreneurship, health care, human resource management, mechanical engineering, quality management, education, and UI/UX (user interface/user experience) design. Half of the participants (n=24) had never joined a hackathon before.

    Mentoring

    A total of 60 mentors preregistered online over a period of 6 days prior to the hackathon. Of them, 52 completed their profiles and became active on the Mentornity platform, of whom 46 joined Slack to communicate with teams and organizers. A briefing session was held to inform mentors about the online hackathon concept and how to offer expertise and express their availability to participants. In total, 75 video mentoring sessions took place on Mentornity where 35 mentors and 19 teams interacted. The actual number of video sessions is estimated to have been higher since it was known that other video conference platforms such as Zoom, Slack, and Skype were used and not tracked by Mentornity.

    Of all mentors, 24 had no previous experience mentoring, 12 had experience from a previous Hacking Health Berlin hackathon, and the remaining 16 had experience mentoring but were new to Hacking Health Berlin events. The group of mentors had broad professional backgrounds as shown in Figure 2. The top fields of expertise comprised business, medicine, and software development. Along those lines, the most booked mentoring sessions reflected a high interest in these fields.

    ‎
    Figure 2. Sessions booked in the different fields of expertise available for mentorship (red line) and the number of mentors with knowledge in the respective disciplines (bars). Participants booked mentoring sessions in the field of tech (blue bars); health care, research, and education (red bars); entrepreneurship and management (gray bars); public speaking and public relations (teal bars); user research and design (yellow bars). API: application programming interface.
    View this figure

    Feedback forms (Multimedia Appendix 1) were automatically created on Mentornity for each mentoring session conducted. Of the mentors, 13 completed the feedback forms, covering a total of 25 mentoring sessions involving 13 teams. The feedback showed that mentors were largely satisfied with the online mentoring experience despite the new format. Of 25 mentoring sessions, 22 took place and were evaluated with ratings ranging from 3 to 5, scoring an average of 4.68 (SD 0.57). Three surveys indicated that the mentoring session did not occur for various reasons: cancellation of the meeting, difficulties with the platform, or the teams did not attend the scheduled mentoring session. The teams that received mentoring reported gaining valuable insights, although they also highlighted that online mentoring was a new format and brought interaction and scheduling challenges. A number of positive comments by mentored teams reflected appreciation for mentors’ willingness to share their experience, which enabled them to take a broader perspective of their project. However, during the hackathon, some teams and mentors reported difficulties in scheduling sessions or connecting via the platform, resulting in some frustration.

    Projects

    A total of 14 projects were developed and presented to the jury and public audience. The scope of the created solutions included telemedicine services, personal risk assessment tools, trackers for COVID-19–specific symptoms and mental health impairments, tracers for population density and risk contacts, immunity passports, hardware for safe reuse of personal protective equipment, platforms for sharing food and other essential products, and educational tools.

    The most innovative solution [57] consisted of a mobile app for knowledge transfer between researchers, the public, and health care workers during and after the pandemic. It tackled the need to access clear and timely COVID-19 scientific evidence for researchers and practitioners to have a clear picture of the situation. App users were able to share information through short videos and visual graphics, aided by curated information and deep learning. The team presented a functional website presenting an novel artificial intelligence–based tool where users could enter specifics about their professional background and their individual research question in a free-text field to obtain summarized research findings from peer-reviewed scientific literature.

    The most impactful solution [58] was a service platform with a video-based hotline to provide instant psychological support for health care professionals exposed to stress, a high workload, and moral dilemmas during the pandemic. With restricted time, health care workers lack the capacity to reach out to existing psychosocial support structures. By bringing psychological support to health care professionals directly via a ​video chat hotline, the instant psychological support was made possible by the voluntary commitment of a broad network of psychotherapists. With a well-planned concept presented on a functional website, the project straightforwardly addressed an imminent user need as identified by frontline health care professionals and outlined in challenge 5.

    The most applicable solution [59] consisted of an app that allowed frontline health care workers involved in COVID-19 patient care to perform regular virtual health self-checks and seek support to ensure their mental, as well as physical health, was prioritized. The presented mock-up of the app had a simple user interface, self-checks required very little time, and support tools were easy to access.

    The app rated as most feasible [60] helped to redistribute critical resources and services. As the pandemic has disrupted supply chains around the world, at the time plenty of resources available were not arriving at locations where needed. The platform solved the issue by allowing people to easily know what was needed, indicate where to send these resources, and track the resource shipping progress. A web portal ready for immediate implementation and use, as well as an outline of short and long-term goals of the project, was presented.

    The best executed solution [61] focused on privacy-first contact tracing and digital epidemiology. It created a multiservice platform connecting patients to self-assessment tools, dashboards, and personal analytics. Users would be able to easily track their logins on their mobile phones and see how symptoms changed. The secure and anonymous contact tracing would allow patients to be aware in their residential area and avoid potential high-risk areas. Finally, information like location, age, and test results would be collected and anonymized into a large database for further analyses and data modeling. The team presented a working web app prototype with 3 features (self-assessment for tracking COVID-19–specific symptoms, contact tracing, warning function for population-dense areas) with a user-friendly and professional-looking user interface, as well as a clear roadmap for the future implementation of their product.

    Approaches by other teams included sterilization methods to reuse protective equipment for health care workers, virtual reality visualizations of epidemiological data, and tools to support social distancing in public spaces. The results presented varied from concepts and click dummies to working prototypes of mobile apps.

    Hackathon Experience

    According to the feedback (n=30), participants were made aware of the event by word of mouth (n=14, 47%), social media (n=12, 40%), event platforms (n=3, 10%), email, and postings on Hacking Health Berlin’s and Data Natives’ existing online communication channels on Slack (n=4, 13%).

    Participants were overall satisfied with the execution of the remote hackathon, evaluating the experience with 4.23 points out of 5 on average. They appreciated the general atmosphere and team spirit, diversity and skillsets of the participants and mentors, the flexibility permitted by the remote nature of the event, and felt that occurring problems could be quickly resolved by the organizing team. However, the variety of online platforms that were used was perceived to be overwhelming to some, and technical issues with the tools were reported. Furthermore, icebreaker and team-building activities at the beginning of the event were desired to get familiar with the general concept of online hackathons and to get to know the other participants (Figure 3). Most participants (n=21, 70%) felt that it was an advantage that the hackathon was held over a long holiday weekend, but opinions were mixed about whether an in-person event of a longer duration than the usual 48 hours would be desirable, with 40% (n=12) in favor of the idea.

    Before the hackathon, participants rated their confidence in starting a digital health venture or research project with 2.6 points out of 5 on average. After the hackathon, the participants’ confidence increased significantly to 3.7 on average. Of the participants, 60% (n=18) would not have started their projects without attending this hackathon, and agreed that it enabled them to progress faster by (1) offering new insights and feedback provided by the mentors, (2) creating a functional prototype, and (3) finding the right members for their team.

    ‎
    Figure 3. Experiences of participants specifically regarding the remote and online nature of the hackathon.
    View this figure

    Main Findings

    This case study demonstrates the feasibility and effectiveness of remote hackathons as an approach to combat the challenges of the COVID-19 pandemic. The event was prepared in a much shorter time and conducted entirely online at significantly lower costs, with similar results as health hackathons held in person. To our knowledge, this is one of the first studies to report and critically discuss experiences of, and provide guidance on, online hackathons and hackathons specifically targeted to solve the challenges around COVID-19, as well as to report on the impact of expert mentoring during health hackathons.

    Cultural Change and Opportunities of Online Collaborations

    The circumstances of the COVID-19 pandemic and the necessity for social distancing have led to a cultural change in work settings, communication, and collaboration. As a consequence, professional meetings and conferences are more likely to take place in an online setting. Based on the high number of participants and mentors, the medical and scientific way in which the challenges were addressed, and the feedback from participants who had experience in face-to-face hackathons, we can conclude that remote online hackathons are feasible and resemble some of the dynamics and synergies of the in-person ones. Online hackathons effectively bring people together at a significantly lower cost in comparison to a conventional in-person hackathon. The EasterHack event enabled participants not only to collaborate in an interdisciplinary manner but also across diverse geographic locations, avoiding the possible inconveniences and costs of travel. The transition toward an online and federated event allowed for a more diverse and global audience. There is substantial research showing a relation between increased diversity in teamwork leads and benefits, such as increased creativity and better problem-solving abilities [50,51].

    Since the first online hackathons to fight COVID-19 were launched in March 2020, more than 50 online events summing up to 28,000 participants [62-67] have been held. The impact is evident in terms of a higher participation level and number of collaborating partners, which brings a multibeneficial bottom line of fewer logistical, financial, geographical, and time constraints, and reduced carbon dioxide output.

    The Impact of Expert Mentoring

    In recent years, mentoring has been shown to have a positive impact on the way participants gain insights, interact, execute multidisciplinary tasks, and build their prototypes during hackathons [3,6,8,28,36]. They act as guiding lights during the process by encouraging exploration and providing information in areas where participants have questions and interests. The main benefit is reflected in the holistic and complementary nature of each hack, often embracing different disciplines and resulting in outstanding solutions and pitches with a strong technical background (software or hardware); addressing health-related challenges pragmatically; and putting the whole idea into a well-polished business plan. On those grounds, the recruitment of expert mentors takes place depending on the scope of the hackathon.

    In general, challenges tackling the COVID-19 pandemic are complex and require critical advice from biomedical experts and frontline health care professionals. These hurdles can be tackled through expert mentorship, complemented with knowledge from specialists for additional areas of expertise aiming to increase the feasibility and applicability of solutions. Availability of mentors is important; for example, health care staff and other essential workers may have commitments directly related to the pandemic, which limits their availability. With online mentoring programs, mentors can help participants with flexible and short mentorship sessions without altering their working schedules and personal time. Likewise, mentors located in different regions and time zones were able to support participants, which in an in-person format would not have been the case. The interdisciplinary nature and outcome of the event was reflected in the diversity of sessions booked; this also confirms the relevance of mentoring and our endeavor to have medical expertise as the backbone. For these reasons, we strongly recommend online mentorship to be considered in both virtual and on-site hackathons.

    Challenges of Conducting Hackathons Remotely

    Online hackathons might have some limitations such as data security concerns when it comes to personal information, especially with video conference tools, communication breakdowns, technical issues, and access to essential hardware including microphones and cameras. Only recently, the video conference service Zoom reported security issues concerning unwanted guests who disturbed some virtual meeting rooms [67-69]. While the preparation time for online hackathons can be relatively short, all tools and technical solutions need to be tested thoroughly in advance to avoid frustration among organizers, participants, and mentors. Technical glitches may also affect the ability to communicate properly, share announcements, recruit survey participants, and marketing. Furthermore, participants who work remotely might become distracted more easily with the screen being the only visual feedback platform and other work-related and personal commitments at the same time. Therefore, a certain level of media competence is recommended. Online team building activities require different approaches compared to events in physical presence. In addition, the dynamics of networking and small talk might be different. Although participants of EasterHack reported positive experiences regarding group synergies and team spirit during the event, there were 14 individuals who did not complete their projects. Further research should address drop-out reasons and challenges of attending online hackathons from the participant perspective, and elaborate on how these challenges might be resolved.

    Limitations

    This case study was based on data that were captured in a single online event and not directly compared with in-person events. Although participants were from various geographical locations, the sample size of 48 participants limits broad generalizations. Further studies in the course of online hackathons should be conducted to validate the stated findings.

    Conclusions

    Online hackathons offer an ideal setting for rapid and goal-oriented collaborative work, and a format to react quickly to urgent and unknown situations like the COVID-19 pandemic.

    EasterHack participants generally reported a positive experience during the online hackathon format, increased confidence, and a higher likelihood of continuing to work on their projects. Despite remotely held events posing challenges to organizers, participants, and mentors, they can be executed much faster and at lower costs compared to in-person meetings, as well as foster team diversity and increase cross-regional collaboration. Setting up a research project and creating IT solutions within health care or research facilities can be a time-consuming and onerous process; however, concept-to-solution processes can be achieved during a single online hackathon weekend. As highlighted by an avid supporter of hackathons [70]:

    Governments need to change. Institutions need to change. But, in the meantime, we have hackathons, where you can rapidly prototype. [...] Do you get a perfect solution? No. But anything is better than nothing. Done is better than perfect.

    Acknowledgments

    Our thanks and appreciation go out to the volunteers of Hacking Health Berlin, Data Natives, and Charité – Universitätsmedizin Berlin, as well as to our mentors and judges, who have greatly contributed to this event. The hackathon received funding from the Berlin Institute of Health, Diabetes Centre Berne, and Paul Hartmann. We further acknowledge support from the German Research Foundation and the Open Access Publication Funds of Charité – Universitätsmedizin Berlin for the publication of this paper.

    Authors KB and A-SP are fellows of the Charité Digital Clinician Scientist Program funded by the Charité – Universitätsmedizin Berlin and the Berlin Institute of Health. This program was initiated and led by Prof. Dr. Duska Dragun to enable resident physicians to pursue a career in digital health and academic research. With great sadness we have received the news that Prof. Dragun passed away on December 28th of 2020. This publication is dedicated to her as a mentor, friend, role model and stellar scientist.

    Authors' Contributions

    KB, ASP, and JH performed the literature search. KB, AL, and PDR designed the surveys and collected the data. KB analyzed the data. KB, AVS, JH, ASP, PDR, and AL wrote the initial draft of the paper, supported by ST, FB, SL, and VK. All authors have critically revised the paper and approved its final version.

    Conflicts of Interest

    None declared.

    ‎
    Multimedia Appendix 1

    Postsession feedback form for mentors to respond to on the Mentornity platform.

    DOCX File , 13 KB
    ‎
    Multimedia Appendix 2

    Feedback survey for hackathon participants.

    DOCX File , 15 KB
    ‎
    Multimedia Appendix 3

    Checklist for Reporting Results of Internet E-Surveys (CHERRIES).

    DOCX File , 19 KB
    1. Impact of COVID-19 on people's livelihoods, their health and our food systems: Joint statement by ILO, FAO, IFAD and WHO. World Health Organization. 2020 Oct 13.   URL: https:/​/www.​who.int/​news/​item/​13-10-2020-impact-of-covid-19-on- people's-livelihoods-their-health-and-our-food-systems#:~:text=The%20economic%20and%20social%20disruption, the%20end%20of%20the%20year [accessed 2020-10-18]
    2. Cardwell FS, Bisung E, Clarke AE, Elliott SJ. Hacking systemic lupus erythematosus (SLE): outcomes of the Waterlupus hackathon. Health Promot Chronic Dis Prev Can 2020 Aug;40(7-8):235-244 [FREE Full text] [CrossRef] [Medline]
    3. Ruzgar NM, Ahuja C, Yu KE, Sallam A, Rosenthal R, Killelea B. How We Do It: Creation of a Workforce Development-Focused Track at a Surgical Hackathon. J Surg Educ 2020 Sep;77(5):1028-1032. [CrossRef] [Medline]
    4. Ahmed AE, Mpangase PT, Panji S, Baichoo S, Souilmi Y, Fadlelmola FM, et al. Organizing and running bioinformatics hackathons within Africa: The H3ABioNet cloud computing experience. AAS Open Res 2018 Apr 18;1:9. [CrossRef]
    5. Schoeb D, Suarez-Ibarrola R, Hein S, Dressler FF, Adams F, Schlager D, et al. Use of Artificial Intelligence for Medical Literature Search: Randomized Controlled Trial Using the Hackathon Format. Interact J Med Res 2020 Mar 30;9(1):e16606 [FREE Full text] [CrossRef] [Medline]
    6. Poncette A, Rojas P, Hofferbert J, Valera Sosa A, Balzer F, Braune K. Hackathons as Stepping Stones in Health Care Innovation: Case Study With Systematic Recommendations. J Med Internet Res 2020 Mar 24;22(3):e17004 [FREE Full text] [CrossRef] [Medline]
    7. Mitra S, Ashby J, Muhumuza A, Ndayishimiye I, Wasserman I, Santhirapala V, et al. Surgathon: a new model for creating a surgical innovation ecosystem in low-resource settings. BMJ Glob Health 2020 Feb 16;5(2):e002162 [FREE Full text] [CrossRef] [Medline]
    8. Li C, Xiong Y, Sit HF, Tang W, Hall BJ, Muessig KE, et al. A Men Who Have Sex With Men-Friendly Doctor Finder Hackathon in Guangzhou, China: Development of a Mobile Health Intervention to Enhance Health Care Utilization. JMIR Mhealth Uhealth 2020 Feb 27;8(2):e16030 [FREE Full text] [CrossRef] [Medline]
    9. Pathanasethpong A, Areemit R, Teerakulpisut D, Morley K, Morley M. Health hackathon as a venue for interprofessional education: a qualitative interview study. J Interprof Care 2019 Dec 21;34(6):1-3. [CrossRef] [Medline]
    10. Ferreira GC, Oberstaller J, Fonseca R, Keller TE, Adapa SR, Gibbons J, et al. Iron Hack - A symposium/hackathon focused on porphyrias, Friedreich's ataxia, and other rare iron-related diseases. F1000Res 2019 Jul 19;8:1135 [FREE Full text] [CrossRef] [Medline]
    11. McLeod P, Cunningham Q, DiazGranados D, Dodoiu G, Kaplan S, Keyton J, et al. Hacking teamwork in health care: Addressing adverse effects of ad hoc team composition in critical care medicine. Health Care Manage Rev 2019 Dec 03:A. [CrossRef] [Medline]
    12. Fecho K, Ahalt SC, Arunachalam S, Champion J, Chute CG, Davis S, Biomedical Data Translator Consortium. Sex, obesity, diabetes, and exposure to particulate matter among patients with severe asthma: Scientific insights from a comparative analysis of open clinical data sources during a five-day hackathon. J Biomed Inform 2019 Dec;100:103325 [FREE Full text] [CrossRef] [Medline]
    13. Connor R, Brister R, Buchmann J, Deboutte W, Edwards R, Martí-Carreras J, et al. NCBI's Virus Discovery Hackathon: Engaging Research Communities to Identify Cloud Infrastructure Requirements. Genes (Basel) 2019 Sep 16;10(9):714 [FREE Full text] [CrossRef] [Medline]
    14. Panchapakesan C, Sheldenkar A, Wimalaratne P, Wijayamuni R, Lwin MO. Developing a Digital Solution for Dengue Through Epihack: Qualitative Evaluation Study of a Five-Day Health Hackathon in Sri Lanka. JMIR Form Res 2019 Aug 29;3(3):e11555 [FREE Full text] [CrossRef] [Medline]
    15. Ramadi KB, Srinivasan S, Atun R. Health diplomacy through health entrepreneurship: using hackathons to address Palestinian-Israeli health concerns. BMJ Glob Health 2019 Jul 10;4(4):e001548. [CrossRef]
    16. Bell J, Murray F, Davies E. An investigation of the features facilitating effective collaboration between public health experts and data scientists at a hackathon. Public Health 2019 Aug;173:120-125. [CrossRef] [Medline]
    17. Lwin MO, Sheldenkar A, Panchapakesan C, Ng JS, Lau J, Jayasundar K, et al. Epihack Sri Lanka: development of a mobile surveillance tool for dengue fever. BMC Med Inform Decis Mak 2019 Jun 13;19(1):111 [FREE Full text] [CrossRef] [Medline]
    18. Aungst TD, Patel R, Pugliese R, Patel I, Boutari C. From ideation to practice: How pharmacists and students can leverage hackathons and innovation labs to accelerate innovation in pharmacy. J Am Pharm Assoc (2003) 2019 Mar;59(2S):S25-S29. [CrossRef] [Medline]
    19. Wang JK, Pamnani RD, Capasso R, Chang RT. Correction to: An Extended Hackathon Model for Collaborative Education in Medical Innovation. J Med Syst 2018 Dec 10;43(1):15. [CrossRef] [Medline]
    20. Wang JK, Roy SK, Barry M, Chang RT, Bhatt AS. Institutionalizing healthcare hackathons to promote diversity in collaboration in medicine. BMC Med Educ 2018 Nov 20;18(1):269 [FREE Full text] [CrossRef] [Medline]
    21. Tucci C, Viscusi G, Gautschi H. Translating Science Into Business Innovation: The Case of Open Food and Nutrition Data Hackathons. Front Nutr 2018 Oct 22;5:96 [FREE Full text] [CrossRef] [Medline]
    22. Wang JK, Pamnani RD, Capasso R, Chang RT. An Extended Hackathon Model for Collaborative Education in Medical Innovation. J Med Syst 2018 Oct 17;42(12):239. [CrossRef] [Medline]
    23. Brown A, Jauregui J, Chipman A, Ilgen J. Using Hackathons to Transform Complex Educational Problems Into Innovative Prototypes. J Grad Med Educ 2018 Aug;10(4):465-466 [FREE Full text] [CrossRef] [Medline]
    24. Ghouila A, Siwo GH, Entfellner JD, Panji S, Button-Simons KA, Davis SZ, DREAM of Malaria Hackathon Participants, et al. Hackathons as a means of accelerating scientific discoveries and knowledge transfer. Genome Res 2018 May 12;28(5):759-765 [FREE Full text] [CrossRef] [Medline]
    25. Lyndon MP, Cassidy MP, Celi LA, Hendrik L, Kim YJ, Gomez N, et al. Hacking Hackathons: Preparing the next generation for the multidisciplinary world of healthcare technology. Int J Med Inform 2018 Apr;112:1-5. [CrossRef] [Medline]
    26. Cooper K, Siefert A, Weinreb J. Skills Beyond the Reading Room: Training in Innovation and Collaboration at a Radiology Hackathon. J Am Coll Radiol 2018 Mar;15(3 Pt A):466-468. [CrossRef] [Medline]
    27. Firenze A, Provenzano S, Santangelo O, Alagna E, Piazza D, Torregrossa M. Hackathon Public Health. Clin Ter 2017;168(6):e421-e427 [FREE Full text] [CrossRef] [Medline]
    28. Pathanasethpong A, Soomlek C, Morley K, Morley M, Polpinit P, Dagan A, et al. Tackling Regional Public Health Issues Using Mobile Health Technology: Event Report of an mHealth Hackathon in Thailand. JMIR Mhealth Uhealth 2017 Oct 16;5(10):e155 [FREE Full text] [CrossRef] [Medline]
    29. Madhani S, Farooqi WH, Mian AI. Stimulating innovation through the hackathon concept in paediatrics: our experience at the Aga Khan University. Arch Dis Child 2017 Oct 16;102(10):994-994. [CrossRef] [Medline]
    30. Kohli M, Morrison JJ, Wawira J, Morgan MB, Hostetter J, Genereaux B, et al. Creation and Curation of the Society of Imaging Informatics in Medicine Hackathon Dataset. J Digit Imaging 2018 Feb 20;31(1):9-12 [FREE Full text] [CrossRef] [Medline]
    31. hackseq Organizing Committee 2016. Catalyzing collaboration between biological and computational scientists via hackathon. F1000Res 2017 Feb 28;6:197 [FREE Full text] [CrossRef] [Medline]
    32. Angelidis P, Berman L, Casas-Perez MDLL, Celi LA, Dafoulas GE, Dagan A, et al. The hackathon model to spur innovation around global mHealth. J Med Eng Technol 2016 Aug 10;40(7-8):392-399 [FREE Full text] [CrossRef] [Medline]
    33. Kapur T, Pieper S, Fedorov A, Fillion-Robin J, Halle M, O'Donnell L, et al. Increasing the impact of medical image computing using community-based open-access hackathons: The NA-MIC and 3D Slicer experience. Med Image Anal 2016 Oct;33:176-180 [FREE Full text] [CrossRef] [Medline]
    34. Silver JK, Binder DS, Zubcevik N, Zafonte RD. Healthcare Hackathons Provide Educational and Innovation Opportunities: A Case Study and Best Practice Recommendations. J Med Syst 2016 Jul 8;40(7):177 [FREE Full text] [CrossRef] [Medline]
    35. Verschoore JR. coordenação de esforços coletivos para enfrentar a pandemia do novo coronavírus: um estudo de caso sobre o hackathon hack for Brazil | COVID-19. REAd Rev eletrôn adm (Porto Alegre) 2020 Aug;26(2):238-264. [CrossRef]
    36. Bacq S, Geoghegan W, Josefy M, Stevenson R, Williams TA. The COVID-19 Virtual Idea Blitz: Marshaling social entrepreneurship to rapidly respond to urgent grand challenges. Bus Horiz 2020 Nov;63(6):705-723 [FREE Full text] [CrossRef] [Medline]
    37. Song Y, Zhang M, Yin L, Wang K, Zhou Y, Zhou M, et al. COVID-19 treatment: close to a cure? A rapid review of pharmacotherapies for the novel coronavirus (SARS-CoV-2). Int J Antimicrob Agents 2020 Aug;56(2):106080 [FREE Full text] [CrossRef] [Medline]
    38. Esakandari H, Nabi-Afjadi M, Fakkari-Afjadi J, Farahmandian N, Miresmaeili S, Bahreini E. A comprehensive review of COVID-19 characteristics. Biol Proced Online 2020 Aug 4;22(1):19 [FREE Full text] [CrossRef] [Medline]
    39. Tang Y, Liu J, Zhang D, Xu Z, Ji J, Wen C. Cytokine Storm in COVID-19: The Current Evidence and Treatment Strategies. Front Immunol 2020 Jul 10;11:1708 [FREE Full text] [CrossRef] [Medline]
    40. Kai H, Kai M. Interactions of coronaviruses with ACE2, angiotensin II, and RAS inhibitors-lessons from available evidence and insights into COVID-19. Hypertens Res 2020 Jul 27;43(7):648-654 [FREE Full text] [CrossRef] [Medline]
    41. Sanders J, Monogue M, Jodlowski T, Cutrell J. Pharmacologic Treatments for Coronavirus Disease 2019 (COVID-19): A Review. JAMA 2020 May 12;323(18):1824-1836. [CrossRef] [Medline]
    42. Pascarella G, Strumia A, Piliego C, Bruno F, Del Buono R, Costa F, et al. COVID-19 diagnosis and management: a comprehensive review. J Intern Med 2020 Aug 13;288(2):192-206 [FREE Full text] [CrossRef] [Medline]
    43. Al-Quteimat O, Amer A. The Impact of the COVID-19 Pandemic on Cancer Patients. Am J Clin Oncol 2020 Jun;43(6):452-455 [FREE Full text] [CrossRef] [Medline]
    44. Zhou M, Xie X, Peng Y, Wu M, Deng X, Wu Y, et al. From SARS to COVID-19: What we have learned about children infected with COVID-19. Int J Infect Dis 2020 Jul;96:710-714 [FREE Full text] [CrossRef] [Medline]
    45. Godri Pollitt KJ, Peccia J, Ko AI, Kaminski N, Dela Cruz CS, Nebert DW, et al. COVID-19 vulnerability: the potential impact of genetic susceptibility and airborne transmission. Hum Genomics 2020 May 12;14(1):17 [FREE Full text] [CrossRef] [Medline]
    46. Randolph HE, Barreiro LB. Herd Immunity: Understanding COVID-19. Immunity 2020 May 19;52(5):737-741 [FREE Full text] [CrossRef] [Medline]
    47. Ali I, Alharbi OM. COVID-19: Disease, management, treatment, and social impact. Sci Total Environ 2020 Aug 01;728:138861 [FREE Full text] [CrossRef] [Medline]
    48. Sood S, Aggarwal V, Aggarwal D, Upadhyay SK, Sak K, Tuli HS, et al. COVID-19 Pandemic: from Molecular Biology, Pathogenesis, Detection, and Treatment to Global Societal Impact. Curr Pharmacol Rep 2020 Jul 27;6(5):1-16 [FREE Full text] [CrossRef] [Medline]
    49. Hacking Health - EasterHack. 2020.   URL: https://hacking-health.org/easter-hackathon2020/ [accessed 2020-10-26]
    50. Usher M, Barak M. Team diversity as a predictor of innovation in team projects of face-to-face and online learners. Computers & Education 2020 Jan;144:103702. [CrossRef]
    51. Bouncken R, Brem A, Kraus S. Multi-cultural teams as sources for creativity and innovation: the role of cultural diversity on team performance. Int J Innov Mgt 2016 Feb 15;20(01):1650012. [CrossRef]
    52. LinkedIn.   URL: http://www.linkedin.com [accessed 2020-11-20]
    53. Slack.   URL: http://www.slack.com/ [accessed 2020-11-18]
    54. Airtable.   URL: http://www.airtable.com/ [accessed 2020-11-18]
    55. Mentornity.   URL: http://www.mentornity.com/ [accessed 2020-11-18]
    56. Eysenbach G. Improving the quality of Web surveys: the Checklist for Reporting Results of Internet E-Surveys (CHERRIES). J Med Internet Res 2004 Sep 29;6(3):e34 [FREE Full text] [CrossRef] [Medline]
    57. THM - The Health Manager.   URL: https://hackcorona-berlin.sparkboard.com/project/5e9033478757d6002407f0d7 [accessed 2020-12-15]
    58. Psychische Unterstützung während Krisenzeiten. mAIndcraft.   URL: https://hotline.maindcraft.com/ [accessed 2020-12-15]
    59. MyCare.   URL: https://hackcorona-berlin.sparkboard.com/project/5e90a7424a73ae002467088a [accessed 2020-12-15]
    60. contribute2.us.   URL: https://hackcorona-berlin.sparkboard.com/project/5e90a9d74a73ae002467088f [accessed 2020-12-15]
    61. reCOVer.   URL: https://hackcorona-berlin.sparkboard.com/project/5e9001868757d6002407f0cf [accessed 2020-12-15]
    62. Pan-European Matchathon.   URL: http://www.euvsvirus.org [accessed 2020-10-18]
    63. #WirVsVirus - #WirVSVirus.   URL: http://www.wirvsvirus.org [accessed 2020-10-18]
    64. Talbert T. Space Apps COVID-19 Hackathon Brings 15,000 Together Worldwide. NASA. 2020 Jun 9.   URL: https://www.nasa.gov/feature/space-apps-covid-19-hackathon-brings-15000-together-worldwide [accessed 2020-10-18]
    65. Kinder F, Harvey A. Covid-19: the medical students responding to the pandemic. BMJ 2020 Jun 15;369:m2160. [CrossRef] [Medline]
    66. Luo EM, Newman S, Amat M, Charpignon M, Duralde ER, Jain S, et al. MIT COVID-19 Datathon: data without boundaries. BMJ Innov 2021 Jan 31;7(1):231-234 [FREE Full text] [CrossRef] [Medline]
    67. Wagenseil P. Zoom security issues: Here's everything that's gone wrong (so far). Tom's Guide. 2021 Jan 5.   URL: https://www.tomsguide.com/news/zoom-security-privacy-woes [accessed 2021-01-28]
    68. Paul K. ‘Zoom is malware’: why experts worry about the video conferencing platform. The Guardian. 2020 Apr 2.   URL: http:/​/www.​theguardian.com/​technology/​2020/​apr/​02/​zoom-technology-security-coronavirus-video-conferencing [accessed 2020-10-18]
    69. O'Flaherty K. Zoom Security: 100 New Features Later, Can You Trust Zoom? Forbes. 2020 Jul 2.   URL: https:/​/www.​forbes.com/​sites/​kateoflahertyuk/​2020/​07/​02/​zoom-security-100-new-features-later-can-you-trust-zoom-yet/​ [accessed 2020-10-18]
    70. Garfinkel J. Hacking COVID-19. The Walrus. 2020 Oct 7.   URL: https://thewalrus.ca/hacking-covid-19 [accessed 2020-10-18]

    ‎
    CHERRIES: Checklist for Reporting Results of Internet E-Surveys
    IT: information technology
    UI: user interface
    UX: user experience

    Edited by G Eysenbach; submitted 23.11.20; peer-reviewed by H Park, A Al-Hindawi; comments to author 14.12.20; revised version received 02.01.21; accepted 16.01.21; published 08.02.21

    Copyright

    ©Katarina Braune, Pablo-David Rojas, Joscha Hofferbert, Alvaro Valera Sosa, Anastasiya Lebedev, Felix Balzer, Sylvia Thun, Sascha Lieber, Valerie Kirchberger, Akira-Sebastian Poncette. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 08.02.2021.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.