This paper is in the following e-collection/theme issue:

e-Learning and Medical Education (691) Mobile Health (mhealth) (2939) Games and Gamification for Health (650) Serious Games for Health and Medicine (683) Serious Games for Education (149) Training Tools for Medical Students (130) Mobile Approaches to Medical Education (57) mHealth in Medical Education and Training (77)

Published on in Vol 20, No 2 (2018): February

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/8987, first published .
Cognitive Style and Mobile E-Learning in Emergent Otorhinolaryngology-Head and Neck Surgery Disorders for Millennial Undergraduate Medical Students: Randomized Controlled Trial

Cognitive Style and Mobile E-Learning in Emergent Otorhinolaryngology-Head and Neck Surgery Disorders for Millennial Undergraduate Medical Students: Randomized Controlled Trial

Cognitive Style and Mobile E-Learning in Emergent Otorhinolaryngology-Head and Neck Surgery Disorders for Millennial Undergraduate Medical Students: Randomized Controlled Trial

Authors of this article:

Li-Ang Lee1, 2, 3 Author Orcid Image ;   Yi-Ping Chao4, 5 Author Orcid Image ;   Chung-Guei Huang6, 7 Author Orcid Image ;   Ji-Tseng Fang1, 2, 8 Author Orcid Image ;   Shu-Ling Wang2, 9 Author Orcid Image ;   Cheng-Keng Chuang1, 2, 10 Author Orcid Image ;   Chung-Jan Kang1, 3 Author Orcid Image ;   Li-Jen Hsin1, 3 Author Orcid Image ;   Wan-Ni Lin1, 3 Author Orcid Image ;   Tuan-Jen Fang1, 3 Author Orcid Image ;   Hsueh-Yu Li1, 3 Author Orcid Image
Article Authors Cited by (20) Tweetations (4) Metrics

    Original Paper

    1Faculty of Medicine, Chang Gung University, Taoyuan, Taiwan

    2Graduate Institute of Clinical Medicine Sciences, Chang Gung University, Taoyuan, Taiwan

    3Department of Otorhinolaryngology-Head and Neck Surgery, Linkou Chang Gung Memorial Hospital, Taoyuan, Taiwan

    4Department of Computer Science and Information Engineering, Graduate Institute of Medical Mechatronics, Chang Gung University, Taoyuan, Taiwan

    5Department of Neurology, Linkou Chang Gung Memorial Hospital, Taoyuan, Taiwan

    6Department of Laboratory Medicine, Linkou Chang Gung Memorial Hospital, Taoyuan, Taiwan

    7Department of Medical Biotechnology and Laboratory Science, Graduate Institute of Biomedical Sciences, Chang-Gung University, Taoyuan, Taiwan

    8Department of Nephrology, Linkou Chang Gung Memorial Hospital, Taoyuan, Taiwan

    9Graduate Institute of Digital Learning and Education, National Taiwan University of Science and Technology, Taipei, Taiwan

    10Department of Surgery, Linkou Chang Gung Memorial Hospital, Taoyuan, Taiwan

    Corresponding Author:

    Li-Ang Lee, MD, MSc

    Department of Otorhinolaryngology-Head and Neck Surgery

    Linkou Chang Gung Memorial Hospital

    No. 5, Fu-Hsing Street, Guishan District

    Taoyuan, 33305

    Taiwan

    Phone: 886 33281200 ext 3972

    Fax:886 33979361

    Email: 5738@cgmh.org.tw


    Background: Electronic learning (e-learning) through mobile technology represents a novel way to teach emergent otorhinolaryngology-head and neck surgery (ORL-HNS) disorders to undergraduate medical students. Whether a cognitive style of education combined with learning modules can impact learning outcomes and satisfaction in millennial medical students is unknown.

    Objective: The aim of this study was to assess the impact of cognitive styles and learning modules using mobile e-learning on knowledge gain, competence gain, and satisfaction for emergent ORL-HNS disorders.

    Methods: This randomized controlled trial included 60 undergraduate medical students who were novices in ORL-HNS at an academic teaching hospital. The cognitive style of the participants was assessed using the group embedded figures test. The students were randomly assigned (1:1) to a novel interactive multimedia (IM) group and conventional Microsoft PowerPoint show (PPS) group matched by age, sex, and cognitive style. The content for the gamified IM module was derived from and corresponded to the textbook-based learning material of the PPS module (video lectures). The participants were unblinded and used fully automated courseware containing the IM or PPS module on a 7-inch tablet for 100 min. Knowledge and competence were assessed using multiple-choice questions and multimedia situation tests, respectively. Each participant also rated their global satisfaction.

    Results: All of the participants (median age 23 years, range 22-26 years; 36 males and 24 females) received the intended intervention after randomization. Overall, the participants had significant gains in knowledge (median 50%, interquartile range [IQR]=17%-80%, P<.001) and competence (median 13%, IQR=0%-33%, P=.006). There were no significant differences in knowledge gain (40%, IQR=13%-76% vs 60%, IQR=20%-100%, P=.42) and competence gain (0%, IQR= −21% to 38% vs 25%, IQR=0%-33%, P=.16) between the IM and PPS groups. However, the IM group had a higher satisfaction score (8, IQR=6-9 vs 6, IQR=4-7, P=.01) compared with the PPS group. Using Friedman’s two-way nonparametric analysis of variance, cognitive styles (field-independent, field-intermediate, or field-dependent classification) and learning modules (IM or PPS) had significant effects on both knowledge gain (both adjusted P<.001) and satisfaction (both adjusted P<.001).

    Conclusions: Mobile e-learning is an effective modality to improve knowledge of emergent ORL-HNS in millennial undergraduate medical students. Our findings suggest the necessity of developing various modules for undergraduate medical students with different cognitive styles.

    Trial Registration: Clinicaltrials.gov NCT02971735; https://clinicaltrials.gov/ct2/show/NCT02971735 (Archived by WebCite at http://www.webcitation.org/6waoOpCEV)

    J Med Internet Res 2018;20(2):e56

    doi:10.2196/jmir.8987

    Keywords

    cognitive style; e-learning; mobile technology; randomized controlled trial 


    e-Learning Can Provide an Opportunity for Active Self-Directed Learning

    The large investment involved in undergraduate medical education (UME) for students and society has led medical schools worldwide to seek strategies and methods to improve their students’ progress [1-3]. The cost of medical school tuition continues to increase annually [4], and medical students face substantial financial stress in the United States and Taiwan [5]. A reduction in medical training time has been shown to reduce medical school tuition fees [6]. Innovative curricula, quality of teaching, and primary care education are three major issues in UME [4,7], and novel UME should empower undergraduate medical students to use different learning strategies and learn outside the classroom by promoting self-directed learning [8]. For instance, electronic learning (e-learning) can provide an opportunity for active self-directed learning and the dissemination of knowledge in an interactive fashion [9].

    Cognitive Style Is Underresearched in the Context of Medical Education

    Certain learning characteristics have been positively correlated with academic success; for instance, strong motivation and enjoying studying have been identified as positive predictors during the undergraduate year of medical school [10]. However, it has also been suggested that the predictive power of learning strategies such as cognitive style is underresearched in the context of medical education [11]. At least some undergraduate medical student–initiated learning situations have been reported to be consistent with their individual cognitive and instructional preferences [12]. The Group Embedded Figure Test (GEFT) was first applied to assess cognitive style and instructional materials for medical students in 1981 [13]. Field-independent (FI) learners have been shown to prefer and have better performance in problem-based learning and computer-assisted learning [14], and the preinstructional determination of cognitive style may help to select suitable instructional materials for these students while providing other instructional tools for field-dependent (FD) learners. However, when teaching millennials (also known as digital natives) who are paradoxically motivated by self-interest, the current understanding of these students may be inaccurate because of the considerable diversity in background, personality, and learning preference [15,16]. Therefore, educational reforms such as competency-based medical education have been implemented to better suit the current millennial generation of undergraduate medical students [17].

    Mobile Technology in e-Learning Has Gained Popularity

    Mobile technology has gained popularity in recent years as a means of immediate interactive multimedia (IM) communication and to access the Internet. Embedded e-learning in a smartphone or tablet can affect the educational environment. In this context, it has been termed “mobile technology in e-learning (M-TEL),” and it has been reported to represent the next natural frontier in the evolution of e-learning [18]. Almost all e-learning today can be accessed from mobile devices, including medical education [19], patient education [20], and the development of mobile medical educators [21]. However, clinical teachers looking to use M-TEL therefore need to ensure that it will meet both the needs of their millennial learners and the requirements of the program.

    Students Need to Spend More of Their Time Outside the Classroom to Learn Otorhinolaryngology-Head and Neck Surgery

    Reducing training time can limit the number of topics taught in an UME curriculum, including otorhinolaryngology-head and neck surgery (ORL-HNS). However, at least 20% of primary care complaints are related to ORL-HNS, and a substantial downstream effect on managing ORL-HNS problems has been reported in family medical practice [22]. In Taiwan, a reduction of approximate 20% in classroom lectures in 6-year medical programs was implemented in 2013. Therefore, the students need to be encouraged to spend more of their time outside the classroom to learn. In our pilot study [23], we found that M-TEL using IM modules could be an effective and satisfactory way to learn about emergent ORL-HNS disorders. In this study, we hypothesized that FI learners would prefer M-TEL technology compared with FD learners, and that they would have a better performance with a novel IM module (cases) compared with a conventional Microsoft PowerPoint show (PPS) module (controls). The control group also received identical instructional materials using the same mobile device.


    Study Design

    We conducted this prospective study from August 1, 2015 to July 31, 2017 at a university (Department of ORL-HNS, Faculty of Medicine, Chang Gung University, Taoyuan, Taiwan). This study included two parts: (1) pilot system-design study, and (2) validation study. This study was approved by the institutional review board of Chang Gung Medical Foundation (No: 105-5290C), and all procedures were conducted in compliance with the Declaration of Helsinki 1975. The participants were informed about the aims of the study, and written informed consent was obtained from all participants. The study proposal was registered at ClinicalTrials.gov (NCT02971735).

    Setting

    In the pilot system-design study [23], we established the instructional materials, including essential knowledge and competence of the 10 most common emergent ORL-HNS disorders using the analysis, design, development, implementation, and evaluation models [24] to design effective instruction for e-learning (Figure 1). All of the materials were developed according to the results of needs assessment in a focus group of undergraduate students and revised using a two-round modified Delphi method to develop the instructional content and assess the relative importance of each item. Storyboards and courseware of the IM and PPS modules were developed using the same user interface (Figure 2).

    The Novel IM Module

    Using the IM module, the learners could operate a leading character to search for and interact with other nonplayer characters to procure instructional materials, to review acquired instructional slides (maximal 80), and to win five small game-based quizzes (Figure 3). The instructional slides were briefly explained using scrolling text. The content for the novel IM module was derived from and corresponded to the textbook-based learning material of the conventional PPS module. Game-based quizzes with different contexts from the multiple-choice questions (MCQs) and multimedia situational tests (MSTs) provided small repetitive summaries of the emergent ORL-HNS disorders. The learners could use a learning map to assess their progress in each section or a bar chart to assess their progress overall.

    The Conventional PPS Module

    In the PPS module, the learners chose and watched 10 visual-auditory text-image videos of emergent ORL-HNS disorders (a total of 80 min) by themselves. Video lectures were created by recording Microsoft PowerPoint presentations with audio narrations, timings, and ink gestures using Camtasia Studio software version 8 (TechSmith, Okemos, MI, USA). The learners were free to watch the videos at any time, and they could also rewind and fast-forward the videos as needed (Figure 4).

    ‎
    Figure 1. Analysis, design, development, implementation, and evaluation (ADDIE) model for designing effective instruction of mobile technology in electronic learning (e-learning).
    View this figure

    During the course, all of the participants arbitrarily reviewed 80 slides by themselves. The instructional content of the two learning modules was confirmed to be correlated and equivalent by 2 investigators from the study team (r=.91, P<.001, Spearman correlation test) using the Software Evaluation Checklist [25]. This checklist uses seven criteria (curriculum connections, age/grade appreciates, investment justification, lay-out, support materials, instructional content, and graphics/multimedia) with two (yes, no) Likert-type scales (a total of 28 questions). Qualitative feedback from the participants was obtained through feedback forms in the pilot study [23]. Major bug fixes were performed before the validation study.

    The validation study was a prospective, parallel-controlled, randomized clinical trial assessing the impact of cognitive styles and learning modules using M-TEL on knowledge gain, competence gain, satisfaction, and learning experience.

    Selection of Participants

    A total of 60 consecutive volunteers were recruited from a teaching clinic for the validation study from November 23, 2016 to July 5, 2017. All of the volunteers had at least a basic level of computer literacy, and they were shown the practical aspects of using tablets and apps. The inclusion criteria were as follows: (1) age >20 years; and (2) undergraduate medical students (clerkship). The exclusion criteria were: (1) previous ORL-HNS training; and (2) declining to participate.

    ‎
    Figure 2. Start of the apps. Learners read the adventure story and objectives (story symbol), played four instructional domains (red arrow symbol), reviewed instructional materials (book symbol), assessed learning progress (bar chart symbol), and got the helps (hint symbol) on the start screen.
    View this figure
    ‎
    Figure 3. Screenshots of the interactive multimedia module. Learners arbitrarily operated a leading character to run, jump, and interact with other nonplayer characters (up) to procure instructional materials (middle). After a small session, learners need to complete small game-based quizzes (low).
    View this figure

    Methods of Measurement

    There were four different face-to-face assessments. The cognitive style of the participants was assessed using the 25-item GEFT after enrollment [26]. The GEFT has a relatively high Spearman-Brown reliability coefficient of 0.82 [27]. On the basis of the number of correct answers given by the participants, the GEFT scores ranged from 0 (the most FD) to 18 (the most FI). We stratified the students into two subgroups: “classical FD” (GEFT score ≤12) and “classical FI” (GEFT score >12) [26].

    Pretests including a 15-min 10-question standard MCQs to evaluate the students’ existing knowledge (range 0-100) and a 15-min 5-question MSTs to assess their existing competence (range 0-100) with regard to “emergent ORL-HNS disorders” were given to the students. Each textbook-based MCQ was designed to be answered within 90 seconds and was preselected according to the results of item analysis. The MSTs presented the learners with written descriptions of five scenarios with or without images/videos and asked them to select the appropriate responses from 5 MCQs for one emergent ORL-HNS disorder using the methodology described in a previous publication of key features approach [28]. The MST was developed to assess clinical reasoning competence [23]. After a 100-min learning course, the participants were again requested to answer a different set of MCQs and MSTs posttest. These assessments were comparable with respect to psychometric properties [26]. Two members of staff confirmed that these questions could be sufficiently answered after reviewing the instructional content of the M-TEL.

    The students were then asked to complete a global satisfaction score (GSS) (range 0-10) questionnaire.

    Randomization and Blinding

    Blinding to the purpose of the study during recruitment was maintained to minimize preparation bias. After the participants had provided consent and completed the GEFT and pretests, we randomly assigned them (1:1) to the IM group and PPS group (Figure 5). A balanced design with regard to cognitive style, sex, and age was assured by the randomization procedure. Computer-generated lists of random numbers were created using the Random Number Generator in IBM SPSS software version 23 (IBM, Armonk, NY, USA) for the allocation of the students, who were stratified by center with a 1:1 allocation using a fixed block size of 6 in both parallel subgroups. The allocation sequence was concealed before implementation of the M-TEL module, and the module adhered to our computer-generated randomization protocol.

    Intervention

    After randomization, the participants were unblinded and used fully automated courseware containing IM or PPS module on a 7-inch tablet in an ordinary office environment for 100 min. Before using the courseware, the functionality of the tablet was explained to the participants. The IM group participated in a parkour course to find and read the instructional materials and played small quiz games that were different from the MCQ and MST questions. The students in the PPS group used an app to read and listen to instructional materials in 10 linear-designed sessions. After completing the brief sessions, the IM and PPS learners could review simple slides of the instructional materials.

    Outcome Measures

    The primary outcome measure was the percentage change in MCQ score (ie, “knowledge gain”) after the M-TEL. Other outcomes were the percentage changes in MST (ie, “competence gain”) and GSS.

    Sample Size

    A priori sample size was estimated using primary outcome effects (percentage change in MCQ score) based on a pilot study (IM module: 43% [SD 18%]; PPS module: 35% [SD 21%]). A two-tailed Wilcoxon signed-rank test to calculate the sample size of 26 in each group (normal parent distribution; calculated effect size: 0.41; type I error: 0.05; power: 80%). Assuming a 10% dropout rate to fulfill the criteria of intention-to-treat analysis, we needed at least 29 participants in each group. Accordingly, we decided to enroll a total of 60 students to show the difference in percentage change in MCQ score.

    Statistical Analysis

    Because the primary outcome measure (percentage change in MCQ score) was not normally distributed according to the D’Agostino-Pearson omnibus normality test, percentage changes ([after value-before value]/[before value] × 100) in MCQ, MST, and GSS; and AttrakDiff2 scores were compared between groups using the Wilcoxon signed-rank test, Mann-Whitney U test, or Kruskal-Wallis test as appropriate. Categorical variables were analyzed using Fisher exact test, and the Spearman correlation test was used to analyze the relationship between variables of interest. Friedman test (two-way nonparametric analysis of variance) was used to compare the effect of multiple levels of two factors [29]. All tests were two-tailed, and statistical significance was established at P<.05. Statistical analyses were performed using G*Power 3.1.9.2 software (Heinrich-Heine University, Dusseldorf, Germany), Statistical Package for the Social Sciences for Windows version 23.0 (SPSS Inc., Chicago, IL, USA), and GraphPad Prism for Windows version 7.0 (GraphPad Software Inc., San Diego, CA, USA).

    ‎
    Figure 4. Screenshots of the PowerPoint Show module. Learners watched 10 visual-auditory text-image videos of emergent otorhinolaryngology-head and neck surgery (ORL-HNS) disorders. The instructional slides of this module were identical to those of the interactive multimedia module and arranged linearly.
    View this figure
    ‎
    Figure 5. The Consolidated Standards of Reporting Trials flow diagram.
    View this figure

    Study Participants

    A total of 60 undergraduate medical students were screened, all of whom (median age 23 years, range 22-26 years; 36 males, 60% and 24 females , 40%) were randomized 1:1 to the IM group or PPS group as shown in the Consolidated Standards of Reporting Trials flow diagram (Figure 3; Multimedia Appendix 1 [30]). Table 1 summarizes the variables of interest for the overall study cohort. There were no significant differences in age, sex, cognitive style, MCQ, or MST scores between the two groups at baseline. All of the participants (100%, 60/60) received the intended intervention after randomization, and there was no deviation from the study protocol.

    Primary and Secondary Outcomes

    Overall, all of the participants showed significant improvements in MCQ score (P<.001) and MST score (P=.006) after 100 min of e-learning (Table 1). The median percentage changes in MCQ and MST scores were 50% (interquartile range 17%-80%, P<.001) and 13% (interquartile range 0%-33%, P=.006), respectively. The M-TEL positively impacted the GSS (P<.001). The PPS group had significant improvements in knowledge (P<.001) and competence (P=.001), whereas the IM group had a significant improvement in knowledge (P<.001) but not in competence (P=.53). There were no significant differences in the percentage changes in the MCQ or MST score between the two groups (P=.42 and P=.16, respectively). Notably, the IM group had a significantly higher GSS compared with the PPS group (P=.01).

    Differences in Outcomes Between the Classical Field-Dependent and Field-Independent Learners

    Using the original definition of FD and FI defined by Witkin [29], 5 (8%) participants had the classical FD cognitive style and 55 (92%) had the classical FI cognitive style. Table 2 summarizes comparisons of the variables of interest between the classical FD and FI learners. There were no significant differences in age, sex, M-TEL module, or MCQ or MST scores between the classical FD and FI groups at baseline. After 100 min of M-TEL, increases in MCQ and MST scores in the classical FD learners did not reach statistical significance (P=.14 and P=.85, respectively), whereas they were significantly increased in the classical FI learners (P<.001 and P=.003, respectively). However, the percentage changes in MCQ and MST scores were not significantly different between the classical FD and FI groups (P=.90 and P=.68, respectively). Even though the differences were not statistically significant, the classical FD learners had a lower GSS than the classical FI learners.

    Table 1. Demographics, cognitive style, learning outcomes, satisfaction, and experience.
    VariablesOverall, N=60Interactive multimedia group, N=30PowerPoint show group, N=30P valuea
    Demographics




    		Age in years, median (IQRb)23 (23-24)23 (23-24)23 (23-24).21

    		Male sex, n (%)36 (60)20 (67)16 (53).43
    Cognitive style




    		Group embedded figures test score, median (IQR)17 (15-18)18 (15-18)17 (16-18).78

    		Field-dependence, n (%)5 (8)3 (10)2 (7)>.99
    Learning outcomes




    		Multiple-choice questions-before, median (IQR)40 (40-50)c40 (40-60)c45 (30-50)c.47

    		Multiple-choice questions-after, median (IQR)70 (60-80)c70 (58-70)c70 (60-80)c.72

    		Percentage change in multiple-choice questions, median (IQR)50 (17-80)d40 (13-76)d60 (20-100)d.42

    		Multimedia situational test-before, median (IQR)80 (60-80)c80 (60-80)80 (60-80)c.84

    		Multimedia situational test-after, median (IQR)80 (80-100)c80 (60-80)80 (80-100)c.003

    		Percentage change in multimedia situational test, median (IQR)13 (0-33)d0 (−21 to 38)25 (0-33)d.16
    Learning satisfaction




    		Global satisfaction score, median (IQR)7 (5-9)d8 (6-9)d6 (4-7).01

    aMann-Whiney U test (continuous variables) or Fisher exact test (categorical variables)

    bIQR: interquartile range.

    cP<.05, before versus after, Wilcoxon signed-rank test (two-tailed).

    dP<.05, compared with a neutral value (“0” for multiple-choice questions and multimedia situational test, or “5” for “global satisfaction score”), Wilcoxon signed-rank test (two-tailed).

    Table 2. Comparisons of demographics, learning model, outcomes, satisfaction, and experience between classical cognitive styles.
    VariablesClassical field-dependent, N=5Classical field-independent, N=55P valuea
    Demographics



    		Age in years, median (IQRb)23 (22-24)23 (23-24).45

    		Male sex, n (%)2 (40)34 (62).38

    		Group embedded figures test score, median (IQR)9 (4-12)18 (17-18)<.001
    Learning model



    		Interactive multimedia, n (%)3 (60)28 (51)>.99
    Learning outcomes



    		Multiple-choice questions-before, median (IQR)40 (25-60)40 (40-50)c.53

    		Multiple-choice questions-after, median (IQR)60 (50-80)70 (60-80)c.70

    		Percentage change in multiple-choice question, median (IQR)67 (−7 to 200)50 (17-80)d.90

    		Multimedia situational test-before, median (IQR)80 (50-100)80 (60-80)c.63

    		Multimedia situational test-after, median (IQR)80 (70-90)80 (80-100)c.92

    		Percentage change in multimedia situational test, median (IQR)0 (−30 to 92)25 (0-33)d.68
    Learning satisfaction



    		Global satisfaction score, median (IQR)6 (4-7)7 (5-9)d.25

    aMann-Whiney U test (continuous variables) or Fisher exact test (categorical variables).

    bIQR: interquartile range.

    cP<.05, before versus after, Wilcoxon signed-rank test (two-tailed).

    dP<.05, compared with a neutral value (“0” for multiple-choice question and multimedia situational test, or “5” for “global satisfaction score”), Wilcoxon signed-rank test (two-tailed).

    Post Hoc Analysis

    In this study, most of the participants were categorized as classical FI learners, and we were unable to determine which FI learners were more suitable for M-TEL using classical classification [26]. El-Banna proposed that a field-intermediate (FINT) category exists between FD and FI categories [31]. Accordingly, we adopted this modified classification of cognitive style (FD: <mean GEFT score − standard deviation [SD] × 0.25; FINT: ≥mean GEFT score − SD × 0.25 and ≤mean GEFT score + SD × 0.25; FI: >mean GEFT score + SD × 0.25); thereby resulting in three modified categories : FD: <16, n=15 (25%); FINT: ≥16 and ≤17, n=17 (28%0; FI: >17, n=28 (47%). Table 3 illustrates comparisons of these modified FD, FINT, and FI groups using modern classification. There were no significant differences in age, sex, M-TEL module, or MCQ or MST scores among the modified FD, FINT, and FI groups at baseline. Modified FI was independent of the M-TEL module and learning outcomes (all P>.05). Increases in MCQ and MST were significant in all three of these cognitive style groups. Although the differences in knowledge and competence gains were not statistically significant among the modified cognitive styles, the FINT group had a significantly higher satisfaction with M-TEL than the modified FD group regardless of which M-TEL module they used. The modified FINT learners had a significantly positive attitude toward M-TEL in terms of GSS, whereas the modified FD learners had the lowest GSS. Furthermore, the FINT learners using the IM module had a significantly higher GSS than those using the PPS module (P=.005). The modified FI learners had a positive attitude toward M-TEL in terms of GSS, and a significantly higher GSS when using the IM module compared with the PPS module (P=.02). We further compared the effect of modified cognitive style and M-TEL module on outcomes using Friedman test (Table 4). Both modified cognitive style and M-TEL module had significant effects on percentage changes in MCQ score and GSS. The M-TEL module had significant effects on MST score, whereas modified cognitive style did not have any significant effect.

    Table 3. Comparisons of demographics, learning models, outcomes, satisfaction, and experience among modified cognitive styles.
    VariablesModified field-dependent, N=15Modified field-intermediate, N=17Modified field-independent, N=28P valuea
    Demographics




    		Age in years, median (IQRb)23 (22-24)23 (23-24)23 (23-24).74

    		Male sex, n (%)8 (53)12 (71)16 (57).56
    Cognitive style




    		Group embedded figures test score, median (IQR)14 (12-15)17 (17-17)18<.001
    Learning module




    		Interactive multimedia, n (%)8 (53)7 (41)15 (54).69
    Learning outcomes




    		Multiple-choice questions-before, median (IQR)50 (30-60)c40 (40-50)c40 (40-50)c.47

    		Multiple-choice questions-after, median (IQR)70 (60-80)c70 (65-80)c70 (53-70)c.48

    		Percentage change in multiple-choice question, median (IQR)40 (17-100)d75 (33-100)d45 (15-75)d.34

    		Multimedia situational test-before, median (IQR)80 (60-80)c80 (60-80)c70 (60-80)c.74

    		Multimedia situational test-after, median (IQR)80 (80-80)c80 (70-100)c80 (63-95)c.83

    		Percentage change in multimedia situational test, median (IQR)25 (0-33)d25 (0-29)d0.0 (0-37)d.82
    Learning satisfaction




    		Global satisfaction score, median (IQR)6 (3-7)8 (7-10)d7 (5-8)d.02

    aMann-Whiney U test (continuous variables) or Fisher exact test (categorical variables).

    bIQR: interquartile range.

    cP<.05, before versus after, Wilcoxon signed-rank test (two-tailed).

    dP<.05, compared with a neutral value (“0” for multiple-choice question and multimedia situational test, or “5” for “global satisfaction score”), Wilcoxon signed-rank test (two-tailed).

    Table 4. Comparisons of the effect of modified cognitive style and module of mobile technology in electronic learning on outcomes.
    OutcomesaTest statisticsStandard errorStandard test statisticsP valueAdjusted P value
    Percentage change in multiple-choice question





    		Modified cognitive style–learning module−0.530.18−2.92.003.01

    		Modified cognitive style–percentage change0.710.18−3.88<.001<.001

    		Learning module–percentage change−1.240.18−6.80<.001<.001
    Percentage change in multimedia situational test





    		Modified cognitive style–learning module−0.570.18−3.10.002.006

    		Modified cognitive style–percentage change0.110.180.59.55>.99

    		Learning module–percentage change−0.460.182.51.012.04
    Global satisfaction score

    		−


    		Modified cognitive style–learning module−0.520.18−2.83.005.01

    		Modified cognitive style–global satisfaction score−1.190.18−6.53<.001<.001

    		Learning module–global satisfaction score−1.710.18−9.36<.001<.001

    aFriedman’s two way analysis of variance test.


    Principal Findings

    The main findings of this study are that M-TEL outside the classroom can help undergraduate medical students to strengthen their knowledge and competence of emergent ORL-HNS disorders, and to provide an enjoyable learning experience overall. In addition, our findings suggest that millennials can significantly gain knowledge rather than reinforce competence using an IM module. Despite the similar efficacy of both modules, the students preferred the IM module to the PPS module because of it being more efficient and enjoyable to use. Although the classical classification of cognitive style [26] did not seem to be associated with learning preference or outcomes, the modified FINT learners had the highest knowledge gain and satisfaction with M-TEL (especially the IM module) compared with the modified FD and FI learners [32]. With further controlling for the modified cognitive style, the PPS module enhanced competence compared with the IM module.

    Limitations

    There are several limitations to this randomized controlled trial. First, this study was quasi-experimental because of the lack of probability sampling. Even though we selected individuals based on their availability to the investigators, the sample size was representative of the target population (>50% were classmates). Second, the study used different posttest questions to measure learning outcomes, and the interaction between taking a pretest and the intervention itself may threaten the external validity. A design which does not use a pretest would have been preferable [33]. Third, we did not survey learning preferences, cognitive load, or self-direction in this short-term study. These factors have been reported to effect learning outcomes in modern medical curricula [16,34], and they should be closely monitored when students use this type of M-TEL app.

    Comparison With Prior Work

    As mentioned above, both the IM and PPS modules used the same textbook-based learning material. However, the IM module applied game-design elements and game principles in nongame contexts (gamification) [35]. The gamification of medical education is a rapidly growing field [32,36], and it has been shown to have the potential to improve knowledge [37], and increase motivation and engagement [38]. Our results demonstrate that M-TEL for medical education can facilitate the learning of complex topics with promising results in terms of gains in knowledge, competence, and satisfaction compared with other forms of e-learning [39-41]. However, the learners using the gamified IM model still struggled with performance in MSTs. Therefore, M-TEL may not be an approach that is suitable for all.

    Teaching and learning processes are thought to be affected by various cognitive variables. If medical students receive training, which has been designed according to their individual needs, they may develop a sense of competence and positive self-perception [42]. Previous studies have shown that FI learners have higher levels of achievement and better problem-solving ability [43,44]. In this study, relatively few of our undergraduate medical students were classical FD learners, they did not seem to significantly gain knowledge or competence after the 100-min M-TEL module, and they were neutrally satisfied with this learning method. This may be due to the small sample size or suboptimal classification of cognitive style. Therefore, we applied the modified classification of cognitive style that has been used for gifted students [45] and classified our participants accordingly. Using this modified classification, we found that all of the participants significantly gained knowledge and competence after the M-TEL course regardless of cognitive style. In this short-term M-TEL course, the FINT learners had the significantly highest knowledge gain and satisfaction regardless of which M-TEL module they used compared with the modified FD and FI learners, who needed a more specific design of instructional material. For example, the FD learners needed an easy-to-use and follow style of M-TEL, whereas the FI learners wanted a more vigorous style of M-TEL, including engaging quiz games [23].

    Medical teachers in Taiwan have traditionally assumed that medical students can automatically adapt to the instructional modality and material by themselves to learn a topic, regardless of whether or not they are effective. Although we previously postulated that both learning module and cognitive style could affect competence gain for UME, we found the undergraduate medical students using the PPS learning module had significantly higher improvements in competence, whereas there was no change in competence in the IM learners. Even though the IM module was more attractive and satisfactory than the PPS module, more integrated formats of instructional material were needed to reduce extraneous cognitive load to facilitate problem-solving performance [44]. Bertini et al [46] concluded that FI learners can effectively transfer tasks when they must be transferred to a novel situation and that they can identify the important aspects of ambiguous or disorganized information. However, in this study, modified cognitive styles did not significantly affect the MST outcomes whereas the learning module did (using Friedman test). Even though most of our participants had the FI cognitive style making them better suited to game-based tasks, but insensitive to situational cues, and the minority had the FD cognitive style resulting in divided attention and increased cognitive load in game-based processing, all of them had a similar posttest MST performance. Nevertheless, the high overall baseline MST score suggests that the study participants were previously equipped with an understanding of the clinical reasoning process [47] and that this helped them to predict the features of clinical situations, to determine the appropriate course of action, and to choose correct solutions in the MST. Even though they were novices in ORL-HNS, the high baseline scores created a buffer to achieve a significant increase in MST performance.

    Since learners can start and stop M-TEL at any time or place of their choosing [48], this learning modality may be superior to traditional classroom lectures with regard to self-directed effort management and organized study, and allow them to achieve deep understanding by repeatedly reviewing the instructional materials. Because learning preference and satisfaction allow students to learn outside the classroom, cognitive style should be taken into consideration to design curricula to suit the style of the individual undergraduate medical student, thereby enhancing the effectiveness of M-TEL [49].

    Conclusions

    M-TEL using conventional PPS and novel IM modules seems to be an effective method to teach emergent ORL-HNS disorders to undergraduate medical students. The PPS module represented a formal, serious learning modality, whereas the IM module represented a satisfactory, enjoyable way for the millennial students to learn. Cognitive style and M-TEL module significantly affected knowledge gain and satisfaction, and the modified FINT learners had the highest gains in knowledge and satisfaction when using the IM module. These findings support the development of M-TEL, including various learning modules for undergraduate medical students with different cognitive styles.

    Acknowledgments

    This study was financially supported by grants from the Ministry of Science and Technology, Taiwan, ROC (104-2511-S-182-010 & 105-2511-S-182A-006) and a grant from the Chang Gung Medical Foundation, Taiwan, ROC (CMRPG3F1091). The funding source had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and the decision to submit the manuscript for publication.

    We are indebted to our trained study staff, especially Chung-Fang Hsiao, Hsiu-Yen Yu, Cheng-Hao Yang, Hao-Xiang Zhang, and Ming-Xuan Chen.

    Authors' Contributions

    LAL, YPC, SLW, CKC, and HYL participated in the conception and design of this work. LAL, YPC, CJK, LJH, WNL, TJF, and HYL collected data. LAL, YPC, CGH, JTF, SLW, CKC, and HYL analyzed and interpreted data. LAL, YPC, SLW, CJK, LJH, WNL, TJF, and HYL carried out the development of the project software. All authors participated in the writing of the manuscript and take public responsibility for it. LAL, YPC, CGH, JTF, SLW, CKC, and HYL revised it critically for important intellectual content. All authors reviewed the final version of the manuscript and approve it for publication. All authors attested to the validity and legitimacy of the data in the manuscript and agree to be named as authors of the manuscript.

    Conflicts of Interest

    None declared.

    ‎

    Multimedia Appendix 1

    CONSORT-EHEALTH checklist V 1.6.1.

    PDF File (Adobe PDF File), 701KB
    1. Dodds AE, Reid KJ, Conn JJ, Elliott SL, McColl GJ. Comparing the academic performance of graduate- and undergraduate-entry medical students. Med Educ 2010 Feb;44(2):197-204. [CrossRef] [Medline]
    2. Bickerdike A, O'Deasmhunaigh C, O'Flynn S, O'Tuathaigh C. Learning strategies, study habits and social networking activity of undergraduate medical students. Int J Med Educ 2016 Jul 17;7:230-236 [FREE Full text] [CrossRef] [Medline]
    3. Boileau E, St-Onge C, Audétat M. Is there a way for clinical teachers to assist struggling learners? A synthetic review of the literature. Adv Med Educ Pract 2017;8:89-97 [FREE Full text] [CrossRef] [Medline]
    4. Royal KD. Quality teaching matters more than innovative curricula. Am J Med 2017 Apr;130(4):e167. [CrossRef] [Medline]
    5. Gil JA, Park SH, Daniels AH. Variability in United States allopathic medical school tuition. Am J Med 2015 Nov;128(11):1257-1262. [CrossRef] [Medline]
    6. Emanuel EJ, Fuchs VR. Shortening medical training by 30%. J Am Med Assoc 2012 Mar 21;307(11):1143-1144. [CrossRef] [Medline]
    7. Long T, Grant RA. If training time is shortened, don't forget incentives for shortage areas and primary care. Am J Med 2014 Aug;127(8):e21. [CrossRef] [Medline]
    8. Wolff M, Wagner MJ, Poznanski S, Schiller J, Santen S. Not another boring lecture: engaging learners with active learning techniques. J Emerg Med 2015 Jan;48(1):85-93. [CrossRef] [Medline]
    9. Gill P, Kitney L, Kozan D, Lewis M. Online learning in paediatrics: a student-led web-based learning modality. Clin Teach 2010 Mar;7(1):53-57. [CrossRef] [Medline]
    10. Al SL, Abulaban AA, Abulaban AA, Merdad A, Baghlaf S, Algethami A, et al. Factors potentially influencing academic performance among medical students. Adv Med Educ Pract 2015;6:65-75 [FREE Full text] [CrossRef] [Medline]
    11. Ferguson E, James D, Madeley L. Factors associated with success in medical school: systematic review of the literature. Br Med J 2002 Apr 20;324(7343):952-957 [FREE Full text] [Medline]
    12. Curry L. Individual differences in cognitive style, learning styleinstructional preference in medical education. In: Norman GR, Van Der Vleuten CP, Newble DI, editors. International Handbook of Research in Medical Education. Volume 7. Dordrecht: Springer; 2002:263-276.
    13. Scott N, Smith DU, Rosenberg IK. Cognitive style and instructional materials for medical students. J Med Educ 1981 Jul;56(7):565-571. [Medline]
    14. Chapman DM, Calhoun JG. Validation of learning style measures: implications for medical education practice. Med Educ 2006 Jun;40(6):576-583. [CrossRef] [Medline]
    15. DiLullo C, McGee P, Kriebel RM. Demystifying the Millennial student: a reassessment in measures of character and engagement in professional education. Anat Sci Educ 2011;4(4):214-226. [CrossRef] [Medline]
    16. Holen A, Manandhar K, Pant DS, Karmacharya BM, Olson LM, Koju R, et al. Medical students' preferences for problem-based learning in relation to culture and personality: a multicultural study. Int J Med Educ 2015 Jul 19;6:84-92 [FREE Full text] [CrossRef] [Medline]
    17. Desy JR, Reed DA, Wolanskyj AP. Milestones and millennials: a perfect pairing-competency-based medical education and the learning preferences of generation Y. Mayo Clin Proc 2017 Feb;92(2):243-250. [CrossRef] [Medline]
    18. Archbold Hufty Alegría D, Boscardin C, Poncelet A, Mayfield C, Wamsley M. Using tablets to support self-regulated learning in a longitudinal integrated clerkship. Med Educ Online 2014 Jan;19(1):23638. [CrossRef] [Medline]
    19. Siebert JN, Ehrler F, Combescure C, Lacroix L, Haddad K, Sanchez O, et al. A mobile device app to reduce time to drug delivery and medication errors during simulated pediatric cardiopulmonary resuscitation: a randomized controlled trial. J Med Internet Res 2017 Feb 01;19(2):e31 [FREE Full text] [CrossRef] [Medline]
    20. Schooley B, San NT, Burkhard R. Patient-provider communications in outpatient clinic settings: a clinic-based evaluation of mobile device and multimedia mediated communications for patient education. JMIR Mhealth Uhealth 2015;3(1):e2 [FREE Full text] [CrossRef] [Medline]
    21. Rusatira JC, Tomaszewski B, Dusabejambo V, Ndayiragije V, Gonsalves S, Sawant A, et al. Enabling access to medical and health education in Rwanda using mobile technology: needs assessment for the development of mobile medical educator apps. JMIR Med Educ 2016 Jun 01;2(1):e7 [FREE Full text] [CrossRef] [Medline]
    22. Glicksman JT, Brandt MG, Parr J, Fung K. Needs assessment of undergraduate education in otolaryngology among family medicine residents. J Otolaryngol Head Neck Surg 2008 Oct;37(5):668-675. [Medline]
    23. Lee LA, Wang SL, Chao YP, Tsai MS, Hsin L, Kang C, et al. Mobile technology in e-learning for undergraduate medical education on emergent otorhinolaryngology-head and neck surgery disorders: a pilot study. JMIR Med Educ 2018:A (forthcoming). [CrossRef]
    24. Morrison GR, Ross SM, Kalman HK, Kemp JE, editors. Designing Effective Instruction. 7th edition. Hoboken, NJ: Wiley Inc; 2013.
    25. Cennamo K, Ross J, Ertmer P, editors. Technology Integration for Meaningful Classroom Use: A Standards-Based Approach. 2nd ed. Belmont, CA: Wadsworth, Cengage Learning; 2013.
    26. Witkin HA, Oltman PK, Raskin E, Karp SA, editors. A Manual for the Embedded Figures Tests. Palo Alto, CA: Consulting Psychologists Press; 1971.
    27. Chapelle C, Green P. Field independence/dependence in second-language acquisition research. Lang Learn 1992 Mar;42(1):47-83. [CrossRef]
    28. Farmer EA, Page G. A practical guide to assessing clinical decision-making skills using the key features approach. Med Educ 2005 Dec;39(12):1188-1194. [CrossRef] [Medline]
    29. Daniel WW. Friedman two-way analysis of variance by ranks. In: Daniel WW, editor. Applied Nonparametric Statistics. 2nd edition. Boston, MA: PWS-Kent; 1990:262-274.
    30. Eysenbach G, CONSORT-EHEALTH Group. CONSORT-EHEALTH: improving and standardizing evaluation reports of Web-based and mobile health interventions. J Med Internet Res 2011 Dec;13(4):e126 [FREE Full text] [CrossRef] [Medline]
    31. El-Banna H. Ethos.bl.uk. Scotland: University of Glasgow; 1987. The development of a predictive theory of science education based upon information processing theory   URL: http://ethos.bl.uk/OrderDetails.do?uin=uk.bl.ethos.382005 [accessed 2017-09-01] [WebCite Cache]
    32. Ahmed M, Sherwani Y, Al-Jibury O, Najim M, Rabee R, Ashraf M. Gamification in medical education. Med Educ Online 2015;20:29536 [FREE Full text] [Medline]
    33. Dimitrov DM, Rumrill Jr PD. Pretest-posttest designs and measurement of change. Work 2003;20(2):159-165. [Medline]
    34. Peine A, Kabino K, Spreckelsen C. Self-directed learning can outperform direct instruction in the course of a modern German medical curriculum - results of a mixed methods trial. BMC Med Educ 2016 Jun 03;16:158 [FREE Full text] [CrossRef] [Medline]
    35. Deterding S, Dixon D, Khaled R, Nacke L. From game design elements to gamefulness: defining gamification. New York: ACM Press 9; 2011 Presented at: Proceedings of the 15th International Academic MindTrek Conference on Envisioning Future Media Environments; September 28-30; Tampere, Finland. [CrossRef]
    36. McCoy L, Lewis JH, Dalton D. Gamification and multimedia for medical education: a landscape review. J Am Osteopath Assoc 2016 Jan;116(1):22-34. [CrossRef] [Medline]
    37. Ingadottir B, Blondal K, Thue D, Zoega S, Thylen I, Jaarsma T. Development, usability, and efficacy of a serious game to help patients learn about pain management after surgery: an evaluation study. JMIR Serious Games 2017 May 10;5(2):e10 [FREE Full text] [CrossRef] [Medline]
    38. Muntasir M, Franka M, Atalla B, Siddiqui S, Mughal U, Hossain IT. The gamification of medical education: a broader perspective. Med Educ Online 2015 Dec 17;20:30566 [FREE Full text] [Medline]
    39. Worm BS. Learning from simple ebooks, online cases or classroom teaching when acquiring complex knowledge. A randomized controlled trial in respiratory physiology and pulmonology. PLoS One 2013;8(9):e73336 [FREE Full text] [CrossRef] [Medline]
    40. Gaupp R, Körner M, Fabry G. Effects of a case-based interactive e-learning course on knowledge and attitudes about patient safety: a quasi-experimental study with third-year medical students. BMC Med Educ 2016 Jul 11;16:172 [FREE Full text] [CrossRef] [Medline]
    41. Back DA, von Malotky J, Sostmann K, Hube R, Peters H, Hoff E. Superior gain in knowledge by podcasts versus text-based learning in teaching orthopedics: A randomized controlled trial. J Surg Educ 2017;74(1):154-160. [CrossRef] [Medline]
    42. Karbownik MS, Wiktorowska-Owczarek A, Kowalczyk E, Kwarta P, Mokros L, Pietras T. Board game versus lecture-based seminar in the teaching of pharmacology of antimicrobial drugs--a randomized controlled trial. FEMS Microbiol Lett 2016 Apr;363(7):pii: fnw045. [CrossRef] [Medline]
    43. Xu J, Si JW, Zhang W. Distractors interfere with numerical estimation in Chinese college students as a function of field-dependent or field-independent cognitive style. Psychol Rep 2011 Jun;108(3):825-842. [CrossRef] [Medline]
    44. Angeli C. Examining the effects of field dependence-independence on learners' problem-solving performance and interaction with a computer modeling tool: implications for the design of joint cognitive systems. Comput Educ 2013 Mar;62:221-230. [CrossRef]
    45. Üstünel H, Uçar E, Civelek T, Umut İ. The relationships between field dependent/independent cognitive styles and information & communication technologies based programs in gifted education. Int J Hum Sci 2015;12(2):266. [CrossRef]
    46. Bertini M, Pizzamiglio L, Wapner S, Wapner S, editors. Field Dependence in Psychological Theory, Research and Application: Two Symposia in Memory of Herman A. Witkin. 1st ed. Hilisdale, NJ: Lawrence Eribaum Associates; 1986.
    47. Charlin B, Tardif J, Boshuizen HP. Scripts and medical diagnostic knowledge: theory and applications for clinical reasoning instruction and research. Acad Med 2000 Feb;75(2):182-190. [Medline]
    48. Eggermont S, Bloemendaal PM, van Baalen JM. E-learning any time any place anywhere on mobile devices. Perspect Med Educ 2013 Mar 30;2(2):95-98 [FREE Full text] [CrossRef] [Medline]
    49. Riding RJ, Rayner SG, editors. Cognitive Styles and Learning Strategies: Understanding Style Differences in Learning and Behaviour. London, UK: David Fulton Publishers; 1998.

    ‎
    FD: field-dependent
    FI: field-independent
    FINT: field-intermediate
    GEFT: group embedded figure test
    GSS: global satisfaction score
    IM: interactive multimedia
    IQR: interquartile range
    MCQ: multiple-choice questions
    MST: multimedia situational test
    M-TEL: mobile technology in e-learning
    ORL-HNS: otorhinolaryngology-head and neck surgery
    PPS: PowerPoint show
    UME: undergraduate medical education

    Edited by G Eysenbach; submitted 16.09.17; peer-reviewed by K Royal, B Worm; comments to author 12.10.17; revised version received 13.11.17; accepted 01.12.17; published 13.02.18

    Copyright

    ©Li-Ang Lee, Yi-Ping Chao, Chung-Guei Huang, Ji-Tseng Fang, Shu-Ling Wang, Cheng-Keng Chuang, Chung-Jan Kang, Li-Jen Hsin, Wan-Ni Lin, Tuan-Jen Fang, Hsueh-Yu Li. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 13.02.2018.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.