This paper is in the following e-collection/theme issue:

e-Learning and Medical Education (688) Telehealth and Telemonitoring (1668) Innovations in Exercise Rehabilitation (213) Technology in Physiotherapy (134) Focus Groups and Qualitative Research for Human Factors Research (848)

Published on in Vol 23, No 4 (2021): April

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/25872, first published .
Evaluation of a Novel e-Learning Program for Physiotherapists to Manage Knee Osteoarthritis via Telehealth: Qualitative Study Nested in the PEAK (Physiotherapy Exercise and Physical Activity for Knee Osteoarthritis) Randomized Controlled Trial

Evaluation of a Novel e-Learning Program for Physiotherapists to Manage Knee Osteoarthritis via Telehealth: Qualitative Study Nested in the PEAK (Physiotherapy Exercise and Physical Activity for Knee Osteoarthritis) Randomized Controlled Trial

Evaluation of a Novel e-Learning Program for Physiotherapists to Manage Knee Osteoarthritis via Telehealth: Qualitative Study Nested in the PEAK (Physiotherapy Exercise and Physical Activity for Knee Osteoarthritis) Randomized Controlled Trial

Authors of this article:

Sarah E Jones1 Author Orcid Image ;   Penny K Campbell1 Author Orcid Image ;   Alexander J Kimp1 Author Orcid Image ;   Kim Bennell1 Author Orcid Image ;   Nadine E Foster2, 3 Author Orcid Image ;   Trevor Russell4 Author Orcid Image ;   Rana S Hinman1 Author Orcid Image
Article Authors Cited by (26) Tweetations (17) Metrics

    Original Paper

    1Department of Physiotherapy, School of Health Sciences, Centre for Health, Exercise and Sports Medicine, The University of Melbourne, Melbourne, Australia

    2Primary Care Centre Versus Arthritis, School of Medicine, Keele University, Keele, United Kingdom

    3STARS Education and Research Alliance, School of Health and Behavioural Sciences, The University of Queensland, Brisbane, Australia

    4RECOVER Injury Research Centre, The University of Queensland, Brisbane, Australia

    Corresponding Author:

    Rana S Hinman, PhD

    Department of Physiotherapy, School of Health Sciences

    Centre for Health, Exercise and Sports Medicine

    The University of Melbourne

    Alan Gilbert Building, 161 Barry St

    The University of Melbourne

    Melbourne, 3010

    Australia

    Phone: 61 03 8344 3223

    Email: ranash@unimelb.edu.au


    Background: The delivery of physiotherapy via telehealth could provide more equitable access to services for patients. Videoconference-based telehealth has been shown to be an effective and acceptable mode of service delivery for exercise-based interventions for chronic knee pain; however, specific training in telehealth is required for physiotherapists to effectively and consistently deliver care using telehealth. The development and evaluation of training programs to upskill health care professionals in the management of osteoarthritis (OA) has also been identified as an important priority to improve OA care delivery.

    Objective: This study aims to explore physiotherapists’ experiences with and perceptions of an e-learning program about best practice knee OA management (focused on a structured program of education, exercise, and physical activity) that includes telehealth delivery via videoconferencing.

    Methods: We conducted a qualitative study using individual semistructured telephone interviews, nested within the Physiotherapy Exercise and Physical Activity for Knee Osteoarthritis randomized controlled trial, referred to as the PEAK trial. A total of 15 Australian physiotherapists from metropolitan and regional private practices were interviewed following the completion of an e-learning program. The PEAK trial e-learning program involved self-directed learning modules, a mock video consultation with a researcher (simulated patient), and 4 audited practice video consultations with pilot patients with chronic knee pain. Interviews were audio recorded and transcribed verbatim. Data were thematically analyzed.

    Results: A total of five themes (with associated subthemes) were identified: the experience of self-directed e-learning (physiotherapists were more familiar with in-person learning; however, they valued the comprehensive, self-paced web-based modules. Unwieldy technological features could be frustrating); practice makes perfect (physiotherapists benefited from the mock consultation with the researcher and practice sessions with pilot patients alongside individualized performance feedback, resulting in confidence and preparedness to implement new skills); the telehealth journey (although inexperienced with telehealth before training, physiotherapists were confident and able to deliver remote care following training; however, they still experienced some technological challenges); the whole package (the combination of self-directed learning modules, mock consultation, and practice consultations with pilot patients was felt to be an effective learning approach, and patient information booklets supported the training package); and impact on broader clinical practice (training consolidated and refined existing OA management skills and enabled a switch to telehealth when the COVID-19 pandemic affected in-person clinical care).

    Conclusions: Findings provide evidence for the perceived effectiveness and acceptability of an e-learning program to train physiotherapists (in the context of a clinical trial) on best practice knee OA management, including telehealth delivery via videoconferencing. The implementation of e-learning programs to upskill physiotherapists in telehealth appears to be warranted, given the increasing adoption of telehealth service models for the delivery of clinical care.

    J Med Internet Res 2021;23(4):e25872

    doi:10.2196/25872

    Keywords

    osteoarthritis; knee; physiotherapy; exercise; e-learning; qualitative; telehealth; pain; education 


    Background

    Physiotherapy care is traditionally delivered via in-person consultations. However, for many people, access to physiotherapy is limited by geographical isolation, or limited local services, or both [1]. To this end, experts in Australia [2] and the United Kingdom National Health Service [3] have identified the need to make full use of digital technologies to facilitate patient convenience and access to care. Such services have become even more critical during the COVID-19 pandemic. Social distancing requirements and lockdown restrictions have affected the delivery of in-person health care worldwide, including for noncommunicable diseases, where rehabilitation services have been among the hardest hit [4]. The COVID-19 pandemic has accelerated the drive toward telehealth service delivery as a safe and viable model for physiotherapy services [5,6].

    Telehealth is defined by the World Health Organization as the “delivery of health care services, where patients and providers are separated by distance. Telehealth uses information communication technology for the exchange of information for the diagnosis and treatment of diseases and injuries, research and evaluation, and for the continuing education of health professionals” [7]. Data suggest that only a minority of physiotherapists were providing telehealth services prepandemic [8,9], highlighting the lack of telehealth experience within the profession. Delivering physiotherapy via telehealth requires new technical skills and new clinical skills to adapt clinical practice to treating a patient located remotely from the clinician [10,11]. Although generally computer literate, clinicians delivering rehabilitation services for chronic pain typically have limited confidence and knowledge in the use of telehealth [12]. Furthermore, clinician acceptance and confidence in telehealth increases with both training and repeated exposure to telehealth practice [10,12,13]. Thus, specific training in telehealth is required for physiotherapists to effectively and consistently deliver care via this medium [14].

    Advancing technologies allow education and training for medical and allied health students and professionals to overcome learning barriers such as distance and the limited availability of specialist training staff while also providing a standardized experience for learners [15]. e-Learning broadly relates to the delivery of educational material through information and communication technology (ICT) [16,17], using the internet to wholly or partially replace the need for a human instructor [18,19]. e-Learning can be synchronous (mediated in real time, eg, videoconference), asynchronous (self-directed, self-paced learning), or a combination of both. A systematic review of studies in practicing or trainee doctors and other health professionals compared e-learning interventions with noninternet learning or no learning interventions. Findings revealed that e-learning was associated with large positive effects on education outcomes (knowledge and skills) compared with no learning, whereas only small, inconsistent effects were seen between e-learning and noninternet learning [18], suggesting that e-learning may be similarly effective to more traditional teaching methods.

    Osteoarthritis (OA) is a common and often debilitating chronic joint disease and is one of the leading causes of pain and disability in Australia [20] and worldwide [21]. Symptoms can become increasingly debilitating over time and can greatly affect quality of life, contributing to feelings of dependence and loss of autonomy in older people [22]. As a result of an aging population, combined with increasing obesity rates, the disease burden associated with knee OA is forecast to increase substantially over the coming decade [23]. There is no cure for OA. However, improvements in pain, physical function, and quality of life have been demonstrated with exercise-based interventions [24,25]. Thus, clinical guidelines consistently emphasize education, exercise, weight loss (if required), and support for self-management to alleviate knee OA symptoms before using surgical or pharmacological interventions [26-30].

    Given the central role of exercise in disease management, physiotherapists in primary care settings play an important role in providing care to people with knee OA. However, physiotherapists often feel underprepared to manage OA, lacking knowledge about evidence-based practice and confidence in implementing recommendations into routine care [31-34]. The development and evaluation of training programs to upskill health care professionals in OA management has thus been identified as an important priority for improving OA care [35-37]. Telephone- and videoconference-based telehealth interventions have been shown to be effective and acceptable [14,38-40] modes of service delivery for exercise-based interventions aimed at relieving chronic knee pain and improving physical dysfunction and are as effective as in-person care for adults with musculoskeletal pain [41]. Australia’s National Osteoarthritis Strategy [20] has also called for the increased implementation of remotely delivered evidence-based OA services.

    Objectives

    In this study, we explore the experiences of physiotherapists with and their perceptions of an e-learning program aimed at educating physiotherapists about best practice knee OA management, including the implementation of a protocolized management program focused on education, exercise, and physical activity, and how to deliver such care remotely using a videoconferencing platform.


    Design

    A qualitative study nested within an ongoing randomized controlled trial (RCT; Australian New Zealand Clinical Trials Registry: ACTRN12619001240134) [42] was conducted. The Physiotherapy Exercise and Physical Activity for Knee OA RCT (known and referred to as the PEAK trial) is a noninferiority trial comparing physiotherapist-delivered in-person consultations with physiotherapist-delivered video consultations for people with knee OA. For the RCT, 15 physiotherapists were provided with a structured program of e-learning in best practice knee OA management, including the implementation of a protocolized management program focused on education, exercise, and physical activity as well as how to deliver such care remotely using a videoconferencing platform. In this study, a qualitative approach was chosen to explore the physiotherapist’s experience of e-learning as well as its impact and perceived effectiveness. The research design was centered around a constructivist paradigm, which asserts that people generate their own understanding and knowledge subjectively through experience and reflection [43]. The Standards for Reporting Qualitative Research checklist guided the reporting [44].

    Participants

    All 15 physiotherapists recruited to deliver trial interventions for the PEAK RCT participated in this qualitative study. Eligibility criteria for physiotherapist participation in the RCT included current registration to practice as a physiotherapist, private practice located in metropolitan or regional Victoria or Queensland (Australia), access to a computer with internet connection, suitable workspace for confidential video consultations, and some previous experience with videoconferencing software, for example, Skype, Zoom, or Facetime (not necessarily for delivering clinical care). Physiotherapists who met the eligibility criteria were considered for the study, and participating physiotherapists were selected based on their availability and location to ensure geographical spread across metropolitan and regional areas of Victoria and Queensland. All participants provided written informed consent, and the qualitative study was approved by the Institutional Human Research Ethics Committee, separate from the RCT.

    Training Program

    The PEAK trial training program was developed by the research team, using an e-learning approach featuring both asynchronous and synchronous learning, developed in line with the Miller Pyramid (Figure 1), a learning model designed for use in health education [45]. The model proposes that the foundation of the pyramid is knowledge (knows), with learners understanding the relevant facts and theories. The next level is competence (knows how), with learners developing skills necessary to apply knowledge, demonstrated via a mock video consultation with a researcher (simulated patient). Next is performance (shows how), with learners demonstrating, via practice video consultations with pilot patients, that they can apply knowledge and skills in a realistic clinical context. The final stage represents action (does)—the stage at which the learner has integrated their knowledge, skills, and attitudes and is prepared to integrate the competency into their clinical practice. Results from the linked RCT will provide data on the effectiveness of the implementation.

    ‎
    Figure 1. Schematic representation of the PEAK (Physiotherapy Exercise and Physical Activity for Knee Osteoarthritis) training components mapped to the Miller Pyramid learning model. PEAK: Physiotherapy Exercise and Physical Activity for Knee Osteoarthritis. RCT: randomized controlled trial.
    View this figure

    Physiotherapists first completed approximately 5 hours of self-directed e-learning modules delivered at the University of Melbourne Learning Management System (LMS; Canvas LMS by Instructure, 2019) covering OA management best practices (including a structured physiotherapy treatment protocol), telehealth (the delivery of care via Zoom videoconferencing), and trial procedures. Each module included a quiz to help reinforce user’s knowledge and learning. e-Learning was completed at the user’s own self-selected pace (ideally within 4 weeks) and delivered knowledge as the foundation of the physiotherapist’s learning. The PEAK training program e-learning modules have since been released for use by clinicians more widely [46].

    The e-learning then moved to practical synchronous components, whereby physiotherapists participated in a mock initial consultation via videoconferencing with a physiotherapist researcher (AJK; simulated patient), who provided immediate verbal feedback on consultation quality and performance and evaluated the physiotherapist’s competency according to a standardized competency checklist (Multimedia Appendix 1). The next stage of practical synchronous e-learning required physiotherapists to complete 4 practice video consultations, with 2 pilot patients with chronic (>3 months) knee pain (recruited by research staff) to practice their video consultation skills and apply the education, exercise, and physical activity management program taught in the e-learning modules in a realistic scenario. The research staff conducted spot-checks of consultation recordings to assess the fidelity of the practice consultations to the protocolized treatment plan. Written feedback on common mistakes was provided to participants, and personalized feedback was provided verbally to individual physiotherapists after all pilot patient consultations had been completed. Physiotherapists were financially compensated for the time away from private practice that was dedicated to all elements of participating in the training program and the time taken to complete the interviews.

    Semistructured Interviews

    Semistructured interviews were conducted after physiotherapists completed all components of the training program and commenced treating participants in the PEAK RCT. The interview schedule (Table 1) was developed based on a constructivist schema whereby physiotherapist expectations, training experience, and perceptions of effectiveness of the e-learning program in changing confidence, imparting knowledge, and addressing learning needs were explored. Questions were loosely aligned with the theoretical framework of acceptability [47], focusing on the framework’s five most relevant constructs to the e-learning program (affective attitude, burden, perceived effectiveness, intervention coherence, and self-efficacy). All interviews were conducted over the telephone by SEJ (a researcher who is not a physiotherapist but is trained in qualitative research) who was not involved in the RCT or in developing the training program and was otherwise unknown to the participants. Interviews lasted approximately 30 minutes and were audio recorded and then transcribed verbatim by a third party. Audio recordings and transcripts were deidentified, with transcripts assigned gender-matched pseudonyms to ensure participant confidentiality. All data were stored in digital format on a password-protected university server.

    Table 1. Semistructured interview schedule (loosely aligned with the theoretical framework of acceptability).
    Topic and constructQuestions (prompts)
    Expectations

    		Affective attitude; self-efficacy


    		1. Can you tell me how confident you were in your ability to deliver an OAa management program before the PEAKb trial training? Previous OA management training? Usual management of knee OA before training?


    		2. Can you tell me about how confident you were in your ability to deliver physiotherapy to OA patients via videoconferencing, before you started the PEAK training? Training or professional development or experience in telehealth?


    		3. What were your overall impressions and experience of the training program? Expectations? Comparison with previous professional development?
    Training experience

    		Burden


    		4. What did you like and dislike about the training program? Consider both the web-based modules and the practical aspects of training (ie, the competency video consultation and 4 practice patient video consultations)? Time taken for web-based modules? Volume of material?

    		Intervention coherence


    		5. To what extent did you feel your learning needs were met (or not) through the PEAK training program? Was there anything else you felt need to be covered or a required a greater focus in the training? Was there anything included that you felt was unnecessary?


    		6. What were your thoughts on the content and presentation of the content in the PEAK web-based modules? How did you feel about the volume of text presented? How would you prefer to see the content delivered? If you were to change the content or its presentation, what would you suggest?


    		7. What were your impressions about the usability of the web-based training platform? Like/dislikes/changes?


    		8. How did you find the competency video consultation with the research staff member? Was it useful?


    		9. Can you tell me a little bit about what you thought about the practice video consultation sessions with the pilot patients with knee pain? Were these consultations useful?

    		Perceived effectiveness


    		10. Can you tell me overall how useful you found the PEAK trial training program? Intent to implement changes to usual clinical practice?


    		11. How confident do you feel in delivering OA management to patients now, after completing the PEAK trial training? What elements do you feel most/least confident with?


    		12. How confident do you feel in delivering physiotherapy to your OA patients over videoconference, now that you have completed the PEAK training? What elements do you feel most/least confident with?
    Concluding remarks


    		13. Given the restrictions on clinical practice imposed by the current COVID-19 pandemic, has the PEAK training program enabled you to make any changes to your own current clinical practice?


    		14. Thank you very much for all your time today and all your time and efforts with this study. Is there anything else you would like to add about your experiences with the PEAK training program?

    aOA: osteoarthritis.

    bPEAK: Physiotherapy Exercise and Physical Activity for Knee Osteoarthritis.

    Data Analysis

    Data analysis was conducted using a thematic approach [48]. Shortly after transcription, interview transcripts were read by SEJ to familiarize with the data. Transcripts were then reread and coded, with text indexed into topics, each identified with a short descriptor. To demonstrate the credibility and confirmability of the emergent topics and patterns, coding was also performed by a second researcher PKC (not a physiotherapist but assisted with developing the training program and recruited physiotherapists into the RCT, and trained in qualitative research). Topics identified by both SEJ and PKC were reviewed, and in collaboration, closely related topics were collated to generate emergent themes within the data. All transcripts were also read by RSH (a physiotherapist who led the development of the training program and an experienced qualitative researcher) who confirmed the relevance of emergent themes across transcripts. Emergent themes were then further refined, ensuring clear and encompassing definitions were generated when naming final themes and subthemes. Themes and subthemes were presented with exemplary quotes from interviews to demonstrate the transferability of the results [49].


    Physiotherapist Characteristics

    All 15 physiotherapists recruited to deliver care as part of the PEAK RCT were invited to participate in the qualitative interviews. The sample comprised physiotherapists who worked in private practices across 2 Australian states. Physiotherapist characteristics are summarized in Table 2, including the number of PEAK RCT participant consultations conducted by each physiotherapist at the time of the interview. Participating physiotherapists were more often male (11/15, 73%). There was an even divide between those who worked in major cities (8/15, 53%) and those who worked in regional areas (7/15, 47%). Of the 14 physiotherapists, 8 of them (60%) had no previous experience with telehealth. The mean (SD) of years of clinical experience was 11 years (SD 4), and the mean numbers of PEAK RCT participants who physiotherapists had already treated at the time of the interview were 2 (SD 1) via videoconferencing and 2 (SD 2) in-person.

    Table 2. Physiotherapists’ characteristics (n=15).
    PseudonymSexGeographical locationaClinical experience (years)Previous experience with telehealthPEAKb RCTc videoconferencing participants at the time of the interview, nPEAK RCT in-person participants at the time of the interview, n
    DanielMaleOuter regional, Victoria19Yes10
    EdwardMaleMajor city, Queensland10No10
    GregoryMaleMajor city, Victoria4No00
    JasonMaleInner regional, Queensland12Yes52
    RobertMaleOuter regional, Victoria12No22
    StevenMaleInner regional, Victoria9No23
    NicoleFemaleInner regional, Victoria15No35
    AnthonyMaleMajor city, Queensland6Yes23
    MarkMaleMajor city, Queensland9Yes23
    DouglasMaleMajor city, Victoria18No43
    CarolineFemaleMajor city, Victoria7No14
    LeslieFemaleMajor city, Queensland9No01
    BrianMaleInner regional, Queensland14Yes30
    WilliamMaleMajor city, Victoria10No11
    VickiFemaleOuter regional, Queensland5Yes14

    aLevel of remoteness, based on residential postcode, in accordance with the Australian Statistical Geographical Classification-Remoteness Area.

    bPEAK: Physiotherapy Exercise and Physical Activity for Knee Osteoarthritis.

    cRCT: randomized controlled trial.

    Emergent Themes

    A total of five themes, with associated subthemes, emerged and are described in Multimedia Appendix 2.

    Theme 1: The Experience of Self-Directed e-Learning

    Physiotherapists found the self-directed e-learning modules to be of high quality, comprehensive, and user-friendly and that they would “benefit the least experienced physiotherapist and the most experienced physiotherapist”. Although physiotherapists were more familiar with “hands-on” in-person professional development training for furthering their knowledge and clinical skills, they highly valued the web-based structure. In particular, the self-paced nature of the program was highly regarded and enabled the physiotherapists to fit the training into their busy daily lives and complete it as time allowed. However, the physiotherapists spoke of feeling annoyed and frustrated by some unwieldy features of the delivery platform (including log-in and navigation). Although most physiotherapists spoke highly of the web-based modules, 3 physiotherapists with divergent views spoke of the e-learning modules being “slow going”, “tedious”, or “fatiguing”. The telehealth learning module content, in particular, was felt to be “dry” and “overtly obvious”.

    Theme 2: Practice Makes Perfect

    Physiotherapists frequently referenced the benefit gained from individualized performance feedback from the research team during or after the practical components of the training. They spoke of “learning from applying” and that feedback was helpful ahead of implementing their new skills in a true clinical scenario. Physiotherapists found the mock consultation with the researcher (simulated patient) facilitated the transition from theory to implementation, commenting it “gave us a feel of what to expect for the upcoming pilots” and that it prevented them having to “worry about looking silly with the patients.” Similarly, the practice consultations with the pilot patients with chronic knee pain were valued, with physiotherapists reporting that this stage of practical e-learning was “hugely beneficial” and “consolidated everything” ahead of consulting with actual patients. Upon completion of the practical components of the e-learning program, physiotherapists expressed a high level of confidence and readiness to go into their first patient consultations using their new skills and knowledge. Two physiotherapists with divergent views felt that the mock consultation with the researcher was “daunting” or “confronting”, with one feeling that it was not necessary. The same physiotherapist also felt that the practice consultations with pilot patients were “surplus to needs”.

    Theme 3: The Telehealth Journey

    For the most part, physiotherapists were inexperienced in telehealth before training. Experience was largely constrained to videoconferencing for social purposes rather than for health care delivery. Physiotherapists in general were nervous or uncertain about providing care to patients via telehealth before undertaking the training program. However, 3 physiotherapists felt that they were moderately confident in telehealth before training, with one stating that “it didn’t seem like too much of a leap to deliver these services [exercise prescription via telehealth] to people.” After completing the training, most physiotherapists were much more confident and reported that they felt ready to deliver OA care via telehealth using videoconferencing. Despite improved confidence and feelings of preparedness, physiotherapists still felt that telehealth posed some challenges, particularly in supporting patients with OA to be able to navigate videoconferencing technology effectively.

    Theme 4: The Whole Package

    Physiotherapists highlighted the benefits of the structured approach of the e-learning program. They found that the combination of self-directed e-learning modules, followed by practical components (mock consultation and pilot patient consultations), was very effective as a “whole package” to develop the knowledge and skills required for best practice OA care via telehealth. Physiotherapists valued the resources and patient information booklets that accompanied the training program, stating that the resources “made it really easy” to navigate the consultations and supported the implementation of their knowledge and skills into practice. One physiotherapist with a divergent view appreciated the package but felt that they would get more out of in-person training because of their learning style.

    Theme 5: Implementation in Broader Clinical Practice

    Although the greatest knowledge gains from the training program were regarding the practical implementation of telehealth for the PEAK randomized trial, physiotherapists also described their intent to apply new knowledge and skills in OA management more broadly to their own private practice. They spoke of how the training program filled “knowledge gaps”, particularly regarding patient education, and provided them with a more structured approach to OA management that could be used when treating their own clients. Given that the training was completed before the COVID-19 pandemic, physiotherapists felt they were “well equipped for this COVID situation” and expressed feeling “ahead of the curve” when it came to implementing telehealth physiotherapy services more broadly, given social distancing and lockdown restrictions. They felt that the skills they had learned allowed them to confidently switch their private practice service delivery to telehealth as well as teach their colleagues about telehealth. Overall, 4 physiotherapists commented that despite the COVID-19 pandemic, they had not ramped up telehealth services or experienced demand for telehealth in their clinical practice.


    Principal Findings

    Our study shows that physiotherapists accepted e-learning despite their lack of familiarity with professional development delivered entirely via the internet. Physiotherapists valued both the theoretical and practical components of training, which together formed the whole package. Most physiotherapists described increasing confidence in providing OA management via telehealth as they progressed through the elements of the e-learning program (Figure 2). The e-learning approach first built a foundation based on new knowledge, before facilitating a transition from theory to practice, then consolidating knowledge in a clinical scenario, and ultimately resulting in the implementation of telehealth into their broader clinical practice.

    ‎
    Figure 2. Schematic representation of emergent themes and their contribution to the development of confidence in delivering osteoarthritis care via telehealth. OA: osteoarthritis.
    View this figure

    Physiotherapists perceived the asynchronous aspect of the self-directed e-learning modules to be generally user-friendly; however, the unwieldy technological features of the delivery platform could be frustrating. A previous systematic review of enablers and barriers affecting e-learning in health science education found that 33% (8/24) of the studies identified the lack of user-friendly ICT as a barrier to successful e-learning [50]. This highlights the importance of a delivery platform that is simple and user-friendly when developing e-learning resources to maximize user engagement. Similarly, a previous qualitative study identified user-friendly training and technology as key factors in the successful implementation of telehealth in diabetic foot care [51]. Physiotherapists valued the comprehensive and self-paced nature of the web-based modules, which is consistent with other research. A previous study evaluated the perceptions of physiotherapy students on asynchronous e-learning for the management of chronic health conditions [52]. Students consistently highlighted the flexibility to work at their own pace and time and access to comprehensive information as advantages of e-learning. Collectively, these findings suggest that the ability to self-pace a person’s learning should be prioritized when creating adult e-learning materials.

    Telehealth practical components of our e-learning program were valued by physiotherapists, which supports previous research highlighting the importance of practical components in health care education, given the need to develop clinical or hands-on skills alongside knowledge [53-56]. In a study evaluating e-learning for physiotherapy students on intensive care work placements [57], students perceived the e-learning modules to be helpful in preparing them for their clinical rotation and reducing their anxiety. However, e-learning modules could only sufficiently prepare students when integrated with an in-person clinical placement, where practical experience was considered necessary to build clinical reasoning skills. Similarly, other studies have noted that supervised practice and clinical simulations are valuable for building confidence and preparedness for physiotherapy students to provide care in clinical situations [58,59]. A qualitative study showed that physiotherapy students felt that clinically oriented learning, focusing on knowledge, skills, and learning through doing, was an important feature to include in an e-learning program about the management of chronic health conditions [52]. It is likely that the benefit of and amount of practical training required might differ depending on the level of the learner’s previous relevant clinical experience as well as the nature of the clinical skill being taught, and therefore, all of these factors should be considered when developing e-learning programs. From a pedagogical perspective, learning approaches incorporating both knowledge and skills practice are consistent with adult learning theories centered around the concept of competency being developed in sequential stages of learning, grounded in theory with proficiency built through experience [45,60,61]. Our ongoing PEAK RCT will provide additional insights into the fidelity and clinical effectiveness of the structured physiotherapy treatment plan delivered using telehealth compared with the same treatment plan delivered in person.

    Although most physiotherapists described increased confidence in the delivery of telehealth after training, we do not have quantitative measures in this study documenting changes in telehealth proficiency over time. Thus, we cannot draw strong conclusions about how clinical skills in telehealth changed as a result of the training. A previous study of e-learning in physiotherapists showed that asynchronous e-learning was effective in increasing quantitatively measured confidence and knowledge in delivering self-management interventions for OA and lower back pain to patients and that the intervention was delivered with high fidelity despite a lack of supervised practical training [62]. Future research incorporating quantitative measures of intervention confidence, knowledge, and fidelity may be warranted to compare outcomes between e-learning approaches with and without synchronous practical elements of training. Future research may also aim to explore innovative alternatives to synchronous practice consultations. Such examples might include encouraging the learner to practice skills with friends or family, including completion of self-audit and self-reflection exercises.

    Our sample of Australian physiotherapists was inexperienced with telehealth before training and lacked confidence in delivering care remotely. This is unsurprising given that the uptake of telehealth across Australian health services before 2020 was minimal [63,64], despite recognition of its potential to facilitate efficient health service delivery [65-67]. The COVID-19 pandemic has driven a step change in health care delivery, with rapid adoption of telehealth services by health care providers [4], coupled with increased funding for telehealth services (including physiotherapy) in several countries, including Australia [6,68] and the United Kingdom [69]. Our findings showed that the e-learning program facilitated most physiotherapists to make a rapid switch to telehealth with the sudden onset of the COVID-19 pandemic. This suggests that e-learning courses may be an effective means to train physiotherapists in best practice telehealth delivery. Education providers may also wish to incorporate e-learning telehealth training into the entry-to-practice curriculum to prepare emerging practitioners for the evolving digital health landscape.

    Strengths and Limitations

    Strengths of our study include the evaluation of an e-learning program for which the e-learning modules are now freely accessible to clinicians globally. Our qualitative evaluation allowed for a thorough understanding of participants’ experiences and perceptions of participating in the e-learning program. Our interviews were conducted and the analysis was led by a person who was unknown to the participants and who was not a physiotherapist, minimizing the chances of personal or professional bias influencing findings. Limitations include that our sample was of limited size and comprised solely of physiotherapists who had applied and been selected to deliver the intervention as part of the PEAK RCT. Participants were bound by their clinical trial agreement to complete the training, and they were financially compensated for their time. Therefore, we cannot generalize our findings to the general population of physiotherapists who may be unwilling or unmotivated to complete training in their own time. As the training program is also only available in English, we cannot generalize the findings to non-English speakers nor to countries that may have a different scope of physiotherapy practice to that of Australia.

    Conclusions

    In conclusion, this study provides evidence for the perceived effectiveness and acceptability of an e-learning program to train physiotherapists (in the context of a clinical trial) on best practice knee OA management, including telehealth delivery via videoconferencing. The implementation of e-learning programs to upskill physiotherapists in telehealth appears to be warranted, given the increasing adoption of telehealth service models for the delivery of clinical care.

    Acknowledgments

    This work was supported by the National Health and Medical Research Council (grant 1157977). RSH was supported by the National Health and Medical Research Council Senior Research Fellowship (grant 1154217). KB was supported by the National Health and Medical Research Council (grant 1174431). NEF is a senior investigator at the UK National Institute for Health Research.

    Conflicts of Interest

    None declared.

    ‎
    Multimedia Appendix 1

    Standardized competency checklist used to provide personalized verbal feedback to physiotherapists during a mock consultation with a researcher (simulated patient).

    DOCX File , 17 KB
    ‎
    Multimedia Appendix 2

    Themes, subthemes, and exemplary quotes.

    DOCX File , 23 KB
    1. Papandony MC, Chou L, Seneviwickrama M, Cicuttini F, Lasserre K, Teichtahl A, et al. Patients' perceived health service needs for osteoarthritis (OA) care: a scoping systematic review. Osteoarthr Cartil 2017 Jul;25(7):1010-1025 [FREE Full text] [CrossRef] [Medline]
    2. Bradford NK, Caffery LJ, Smith AC. Telehealth services in rural and remote Australia: a systematic review of models of care and factors influencing success and sustainability. Rural Remote Health 2016;16(4):4268 [FREE Full text] [Medline]
    3. NHS five year forward view. National Health Service (NHS) UK. 2014.   URL: https://www.england.nhs.uk/publication/nhs-five-year-forward-view/ [accessed 2020-11-03]
    4. Rapid assessment of service delivery for NCDs during the COVID-19 pandemic. World Health Organization. 2020.   URL: https:/​/www.​who.int/​publications/​m/​item/​rapid-assessment-of-service-delivery-for-ncds-during-the-covid-19-pandemic [accessed 2020-10-14]
    5. Duckett S. What should primary care look like after the COVID-19 pandemic? Aust J Prim Health 2020;26(3):207. [CrossRef]
    6. Stanhope J, Weinstein P. Learning from COVID-19 to improve access to physiotherapy. Aust J Prim Health 2020;26(4):271. [CrossRef]
    7. Telehealth. World Health Organisation. 2016.   URL: https://www.who.int/gho/goe/telehealth/en/ [accessed 2020-02-11]
    8. Lee AC, Davenport TE, Randall K. Telehealth physical therapy in musculoskeletal practice. J Orthop Sports Phys Ther 2018 Oct;48(10):736-739. [CrossRef] [Medline]
    9. Victorian Allied Health Workforce Research Project; Allied Health Assistance Workforce Report. State of Victoria, Department of Health and Human Services. 2016.   URL: https://www2.health.vic.gov.au/Api/downloadmedia/%7BB0FE1403-FB25-4F2A-8FE5-4FECE377B257%7D [accessed 2021-03-31]
    10. Cottrell MA, Russell TG. Telehealth for musculoskeletal physiotherapy. Musculoskelet Sci Pract 2020 Aug;48 [FREE Full text] [CrossRef] [Medline]
    11. Bishop A, Gamlin J, Hall J, Hopper C, Foster NE. PhysioDirect: supporting physiotherapists to deliver telephone assessment and advice services within the context of a randomised trial. Physiotherapy 2013 Jun;99(2):113-118 [FREE Full text] [CrossRef] [Medline]
    12. Cottrell MA, Hill AJ, O'Leary SP, Raymer ME, Russell TG. Clinicians' perspectives of a novel home-based multidisciplinary telehealth service for patients with chronic spinal pain. Int J Telerehabil 2018 Dec 11;10(2):81-88 [FREE Full text] [CrossRef] [Medline]
    13. Lawford BJ, Delany C, Bennell KL, Hinman RS. "I Was Really Pleasantly Surprised": firsthand experience and shifts in physical therapist perceptions of telephone-delivered exercise therapy for knee osteoarthritis-a qualitative study. Arthritis Care Res (Hoboken) 2019 Apr 28;71(4):545-557. [CrossRef] [Medline]
    14. Hinman RS, Lawford BJ, Bennell KL. Harnessing technology to deliver care by physical therapists for people with persistent joint pain: telephone and video‐conferencing service models. J Appl Behav Res 2018 Oct 30;24(2). [CrossRef]
    15. Munro V, Morello A, Oster C, Redmond C, Vnuk A, Lennon S, et al. E-learning for self-management support: introducing blended learning for graduate students - a cohort study. BMC Med Educ 2018 Sep 24;18(1):219 [FREE Full text] [CrossRef] [Medline]
    16. Lawn S, Zhi X, Morello A. An integrative review of e-learning in the delivery of self-management support training for health professionals. BMC Med Educ 2017 Oct 10;17(1):183 [FREE Full text] [CrossRef] [Medline]
    17. Sinclair PM, Kable A, Levett-Jones T, Booth D. The effectiveness of Internet-based e-learning on clinician behaviour and patient outcomes: a systematic review. Int J Nurs Stud 2016 May;57:70-81. [CrossRef] [Medline]
    18. Cook DA, Levinson AJ, Garside S, Dupras DM, Erwin PJ, Montori VM. Internet-based learning in the health professions: a meta-analysis. J Am Med Assoc 2008 Sep 10;300(10):1181-1196. [CrossRef] [Medline]
    19. Gagnon K, Young B, Bachman T, Longbottom T, Severin R, Walker MJ. Doctor of physical therapy education in a hybrid learning environment: reimagining the possibilities and navigating a "New Normal". Phys Ther 2020 Aug 12;100(8):1268-1277 [FREE Full text] [CrossRef] [Medline]
    20. Feng Y. National osteoarthritis strategy. National Osteoarthritis Strategy Project Group. 2018.   URL: https://ibjr.sydney.edu.au/wp-content/uploads/2019/05/National-Osteoarthritis-Strategy.pdf [accessed 2021-03-31]
    21. Vos T, Flaxman AD, Naghavi M, Lozano R, Michaud C, Ezzati M, et al. Years lived with disability (YLDs) for 1160 sequelae of 289 diseases and injuries 1990-2010: a systematic analysis for the Global Burden of Disease Study 2010. Lancet 2012 Dec 15;380(9859):2163-2196 [FREE Full text] [CrossRef] [Medline]
    22. Wallis JA, Taylor NF, Bunzli S, Shields N. Experience of living with knee osteoarthritis: a systematic review of qualitative studies. BMJ Open 2019 Sep 24;9(9) [FREE Full text] [CrossRef] [Medline]
    23. Ackerman IN, Pratt C, Gorelik A, Liew D. Projected burden of osteoarthritis and rheumatoid arthritis in Australia: a population-level analysis. Arthritis Care Res (Hoboken) 2018 Jun 12;70(6):877-883. [CrossRef] [Medline]
    24. Fransen M, McConnell S, Harmer AR, Van der Esch M, Simic M, Bennell KL. Exercise for osteoarthritis of the knee: a Cochrane systematic review. Br J Sports Med 2015 Dec 24;49(24):1554-1557. [CrossRef] [Medline]
    25. Uthman OA, van der Windt DA, Jordan JL, Dziedzic KS, Healey EL, Peat GM, et al. Exercise for lower limb osteoarthritis: systematic review incorporating trial sequential analysis and network meta-analysis. Br Med J 2013 Sep 20;347(sep20 1):5555 [FREE Full text] [CrossRef] [Medline]
    26. Bannuru RR, Osani MC, Vaysbrot EE, Arden NK, Bennell K, Bierma-Zeinstra SM, et al. OARSI guidelines for the non-surgical management of knee, hip, and polyarticular osteoarthritis. Osteoarthr Cartil 2019 Nov;27(11):1578-1589 [FREE Full text] [CrossRef] [Medline]
    27. Fernandes L, Hagen KB, Bijlsma JW, Andreassen O, Christensen P, Conaghan PG, European League Against Rheumatism (EULAR). EULAR recommendations for the non-pharmacological core management of hip and knee osteoarthritis. Ann Rheum Dis 2013 Jul;72(7):1125-1135 [FREE Full text] [CrossRef] [Medline]
    28. Kolasinski SL, Neogi T, Hochberg MC, Oatis C, Guyatt G, Block J, et al. 2019 American College of Rheumatology/Arthritis Foundation guideline for the management of osteoarthritis of the hand, hip, and knee. Arthritis Rheumatol 2020 Feb 06;72(2):220-233 [FREE Full text] [CrossRef] [Medline]
    29. National Clinical Guideline Centre (UK). Osteoarthritis: Care and Management in Adults. London, United Kingdom: London: National Institute for Health and Care Excellence (UK); 2014 Feb. National Institute for Health and Clinical Excellence: Guidance; 2014.
    30. Guideline for the management of knee and hip osteoarthritis, 2nd edn. The Royal Australian College of General Practitioners. 2018.   URL: https:/​/www.​racgp.org.au/​clinical-resources/​clinical-guidelines/​key-racgp-guidelines/​view-all-racgp-guidelines/​knee-and-hip-osteoarthritis [accessed 2021-03-31]
    31. Briggs AM, Hinman RS, Darlow B, Bennell KL, Leech M, Pizzari T, et al. Confidence and attitudes toward osteoarthritis care among the current and emerging health workforce: a multinational interprofessional study. ACR Open Rheumatol 2019 Jun 22;1(4):219-235 [FREE Full text] [CrossRef] [Medline]
    32. Briggs AM, Houlding E, Hinman R, Desmond L, Bennell K, Darlow B, et al. Health professionals and students encounter multi-level barriers to implementing high-value osteoarthritis care: a multi-national study. Osteoarthr Cartil 2019 May;27(5):788-804. [CrossRef] [Medline]
    33. Egerton T, Diamond L, Buchbinder R, Bennell K, Slade S. A systematic review and evidence synthesis of qualitative studies to identify primary care clinicians' barriers and enablers to the management of osteoarthritis. Osteoarthr Cartil 2017 May;25(5):625-638 [FREE Full text] [CrossRef] [Medline]
    34. Selten EM, Vriezekolk JE, Nijhof MW, Schers HJ, van der Meulen-Dilling RG, van der Laan WH, et al. Barriers impeding the use of non-pharmacological, non-surgical care in hip and knee osteoarthritis. J Clin Rheumatol 2017;23(8):405-410. [CrossRef]
    35. Eyles JP, Hunter DJ, Bennell KL, Dziedzic KS, Hinman RS, van der Esch M, Joint Effort Initiative Members. Priorities for the effective implementation of osteoarthritis management programs: an OARSI international consensus exercise. Osteoarthr Cartil 2019 Sep;27(9):1270-1279 [FREE Full text] [CrossRef] [Medline]
    36. Eyles JP, Hunter DJ, Briggs AM, Hinman RS, Fitzpatrick J, March L, et al. National osteoarthritis strategy brief report: living well with osteoarthritis. Aust J Gen Pract 2020 Jul 01;49(7):438-442. [CrossRef]
    37. National strategic action plan for arthritis. Australian Government, Department of Health. 2019.   URL: https://www.health.gov.au/resources/publications/national-strategic-action-plan-for-arthritis [accessed 2021-03-31]
    38. Bennell KL, Nelligan R, Dobson F, Rini C, Keefe F, Kasza J, et al. Effectiveness of an internet-delivered exercise and pain-coping skills training intervention for persons with chronic knee pain. Ann Intern Med 2017 Feb 21;166(7):453. [CrossRef]
    39. Hinman RS, Campbell PK, Lawford BJ, Briggs AM, Gale J, Bills C, et al. Does telephone-delivered exercise advice and support by physiotherapists improve pain and/or function in people with knee osteoarthritis? Telecare randomised controlled trial. Br J Sports Med 2020 Jul 20;54(13):790-797. [CrossRef] [Medline]
    40. Hinman RS, Nelligan RK, Bennell KL, Delany C. "Sounds a Bit Crazy, But It Was Almost More Personal:" A qualitative study of patient and clinician experiences of physical therapist-prescribed exercise for knee osteoarthritis via Skype. Arthritis Care Res (Hoboken) 2017 Dec 02;69(12):1834-1844. [CrossRef] [Medline]
    41. Salisbury C, Montgomery AA, Hollinghurst S, Hopper C, Bishop A, Franchini A, et al. Effectiveness of PhysioDirect telephone assessment and advice services for patients with musculoskeletal problems: pragmatic randomised controlled trial. Br Med J 2013 Jan 29;346(jan29 3):43 [FREE Full text] [CrossRef] [Medline]
    42. Hinman RS, Kimp AJ, Campbell PK, Russell T, Foster NE, Kasza J, et al. Technology versus tradition: a non-inferiority trial comparing video to face-to-face consultations with a physiotherapist for people with knee osteoarthritis. Protocol for the PEAK randomised controlled trial. BMC Musculoskelet Disord 2020 Aug 07;21(1):522 [FREE Full text] [CrossRef] [Medline]
    43. Baxter P, Jack S. Qualitative case study methodology: study design and implementation for novice researchers. Qual Rep 2015 Jan 14. [CrossRef]
    44. O’Brien BC, Harris IB, Beckman TJ, Reed DA, Cook DA. Standards for reporting qualitative research. Acad Med 2014;89(9):1245-1251. [CrossRef]
    45. Miller GE. The assessment of clinical skills/competence/performance. Acad Med 1990 Sep;65(9 Suppl):S63-S67. [CrossRef] [Medline]
    46. PEAK training. The University of Melbourne, Melbourne School of Health Sciences.   URL: https:/​/healthsciences.​unimelb.edu.au/​departments/​physiotherapy/​chesm/​clinician-resources/​peak-training [accessed 2021-04-09]
    47. Sekhon M, Cartwright M, Francis JJ. Acceptability of healthcare interventions: an overview of reviews and development of a theoretical framework. BMC Health Serv Res 2017 Jan 26;17(1):88 [FREE Full text] [CrossRef] [Medline]
    48. Morse JM, Field PA. Qualitative Research Methods for Health Professionals. Thousand Oaks, United States: SAGE Publications Inc; 1995:1-272.
    49. Mays N, Pope C. Rigour and qualitative research. Br Med J 1995 Jul 08;311(6997):109-112 [FREE Full text] [CrossRef] [Medline]
    50. Regmi K, Jones L. A systematic review of the factors - enablers and barriers - affecting e-learning in health sciences education. BMC Med Educ 2020 Mar 30;20(1):91 [FREE Full text] [CrossRef] [Medline]
    51. Kolltveit BH, Gjengedal E, Graue M, Iversen MM, Thorne S, Kirkevold M. Conditions for success in introducing telemedicine in diabetes foot care: a qualitative inquiry. BMC Nurs 2017 Jan 13;16(1):2 [FREE Full text] [CrossRef] [Medline]
    52. Gardner P, Slater H, Jordan JE, Fary RE, Chua J, Briggs AM. Physiotherapy students' perspectives of online e-learning for interdisciplinary management of chronic health conditions: a qualitative study. BMC Med Educ 2016 Mar 16;16(1):62 [FREE Full text] [CrossRef] [Medline]
    53. Duque G, Demontiero O, Whereat S, Gunawardene P, Leung O, Webster P, et al. Evaluation of a blended learning model in geriatric medicine: a successful learning experience for medical students. Australas J Ageing 2013 Jun 17;32(2):103-109. [CrossRef] [Medline]
    54. Nartker AJ, Stevens L, Shumays A, Kalowela M, Kisimbo D, Potter K. Increasing health worker capacity through distance learning: a comprehensive review of programmes in Tanzania. Hum Resour Health 2010 Dec 31;8(1):30 [FREE Full text] [CrossRef] [Medline]
    55. Rowe M, Frantz J, Bozalek V. The role of blended learning in the clinical education of healthcare students: a systematic review. Med Teach 2012 Mar 28;34(4):216-221. [CrossRef]
    56. Zolfaghari M, Negarandeh R, Eybpoosh S. Developing a blended learning program for nursing and midwifery students in Iran: process and preliminary outcomes. Iran J Nurs Midwifery Res 2013 Jan;18(1):20-26 [FREE Full text] [Medline]
    57. Major ME, Ramaekers SP, Engelbert RH, Van der Schaaf M. Preparing undergraduate students for clinical work in a complex environment: evaluation of an e-learning module on physiotherapy in the intensive care unit. BMC Med Educ 2020 Apr 28;20(1):130 [FREE Full text] [CrossRef] [Medline]
    58. Ohtake PJ, Lazarus M, Schillo R, Rosen M. Simulation experience enhances physical therapist student confidence in managing a patient in the critical care environment. Phys Ther 2013 Mar;93(2):216-228. [CrossRef] [Medline]
    59. Thomas EM, Rybski MF, Apke TL, Kegelmeyer DA, Kloos AD. An acute interprofessional simulation experience for occupational and physical therapy students: key findings from a survey study. J Interprof Care 2017 May 28;31(3):317-324. [CrossRef] [Medline]
    60. Dreyfus SE, Dreyfus HL. A five stage model of the mental activities involved in directed skill acquisition. Department of Industrial Engineering and Operations Research University of California, Berkeley. 1980.   URL: https://apps.dtic.mil/dtic/tr/fulltext/u2/a084551.pdf [accessed 2021-03-31]
    61. Dreyfus SE. The five-stage model of adult skill acquisition. Bull Sci Technol Soc 2016 Jul 24;24(3):177-181. [CrossRef]
    62. Hurley DA, Keogh A, Mc Ardle D, Hall AM, Richmond H, Guerin S, et al. Evaluation of an e-learning training program to support implementation of a group-based, theory-driven, self-management intervention for osteoarthritis and low-back pain: pre-post study. J Med Internet Res 2019 Mar 07;21(3) [FREE Full text] [CrossRef] [Medline]
    63. Smith AC, Armfield NR, Croll J, Gray LC. A review of Medicare expenditure in Australia for psychiatric consultations delivered in person and via videoconference. J Telemed Telecare 2012 Feb 23;18(3):169-171. [CrossRef]
    64. Wade V, Soar J, Gray L. Uptake of telehealth services funded by Medicare in Australia. Aust Health Review 2014;38(5):528. [CrossRef]
    65. Nelson R. Telemedicine and telehealth: the potential to improve rural access to care. Am J Nurs 2017 Jul;117(6):17-18. [CrossRef] [Medline]
    66. Polinski JM, Barker T, Gagliano N, Sussman A, Brennan TA, Shrank WH. Patients' satisfaction with and preference for telehealth visits. J Gen Intern Med 2016 Mar 13;31(3):269-275 [FREE Full text] [CrossRef] [Medline]
    67. Shigekawa E, Fix M, Corbett G, Roby DH, Coffman J. The current state of telehealth evidence: a rapid review. Health Aff (Millwood) 2018 Dec;37(12):1975-1982. [CrossRef] [Medline]
    68. COVID-19 National Health Plan – Primary Care Package – MBS telehealth services and increased practice incentive payments. Australian Government, Department of Health. 2020.   URL: https:/​/www.​health.gov.au/​resources/​publications/​covid-19-national-health-plan-primary-care-package-mbs-telehealth-services-and-increased-practice-incentive-payments [accessed 2020-10-14]
    69. Tack CT, Grodon J, Shorthouse F, Spahr N. "Physio anywhere": digitally-enhanced outpatient care as a legacy of coronavirus 2020. Physiotherapy 2021 Mar;110:26-28 [FREE Full text] [CrossRef] [Medline]

    ‎
    ICT: information and communication technology
    LMS: Learning Management System
    OA: osteoarthritis
    PEAK: Physiotherapy Exercise and Physical Activity for Knee Osteoarthritis
    RCT: randomized controlled trial

    Edited by R Kukafka; submitted 19.11.20; peer-reviewed by S Kardes, A Samuel; comments to author 29.12.20; revised version received 14.01.21; accepted 18.03.21; published 30.04.21

    Copyright

    ©Sarah E Jones, Penny K Campbell, Alexander J Kimp, Kim Bennell, Nadine E Foster, Trevor Russell, Rana S Hinman. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 30.04.2021.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.