Evaluation of the Educational Value of YouTube Videos About Physical Examination of the Cardiovascular and Respiratory Systems

Introduction

Current medical curricula are paying increasing attention to learning how to conduct physical examinations and to early exposure of students to clinical skills. The aim is to highlight physical examination as a core competency for students and to ensure that the content and delivery of this component in the curriculum has been mastered before graduation [1,2]. Learning about physical examination of the cardiovascular and respiratory systems, as is the case with other elements of clinical skills, cannot be achieved from reading textbooks or attending lectures. Such skills are usually developed through observation of clinicians performing examinations and through practice on simulated and real patients [3-5].

For many years, students relied on their clinical teachers as one of the main sources for learning such skills [3,4]. However, with the introduction of problem-based learning (PBL) and self-directed learning into most medical curricula, more emphasis has been placed on changing the learning and teaching pedagogy [5-7]. In these courses, students use a range of learning resources, including review papers, journal articles, textbooks, museum specimens, simulated patients, computer-aided learning programs, and multimedia [8-10]. The Internet has become an easily available resource of up-to-date information worldwide [9,11-15]. Medical students usually rely on Google and YouTube as the first resources in their research [12-14].

YouTube is the largest Internet video-sharing site and is a useful tool in social communication, business, advertising, and news as well as a promising learning resource for students and the general public [16,17]. In 2005, YouTube was created as an arena for personal/social communication and for distribution of commercial content. Although similar video-sharing sites are available for public use, YouTube has become the most popular worldwide. Statistical data offered by YouTube provide evidence of its popularity [17]. For example, YouTube is the largest video site with over 4 billion videos watched around the globe every day and one hour of video is uploaded to YouTube every second [17].

Recently, the quality of YouTube videos has been evaluated in a number of areas related to medical and patient health information and medical skills [18], including first aid information on thermal burns [19], human papilloma virus vaccinations [20], investigation into the mechanisms of elbow dislocation [21], clinical procedures [22], rheumatoid arthritis [23], as a learning resource for nurses [24], surface anatomy [25], cardiopulmonary resuscitation [26], dental education [27], and infantile spasms [28]. However, not all researchers found YouTube videos to be educationally useful videos; on the contrary, a number of researchers warned that there are thousands of videos on YouTube that promote misleading information and could possibly endanger some viewers. For example, there are videos encouraging anorexia as a healthy life-style choice [29] and non-suicidal self-injury videos that may foster normalization of non-suicidal self-injury and potentially reinforce the behavior through regular viewing [30,31].

With these limitations in mind, there is no doubt that educationally designed videos have the advantage of explaining difficult concepts through using simulation, graphic diagrams, dynamic illustrations, analogies, and simulated patients. The learning/teaching benefits of videos will be enhanced if videos are well-designed, explore scientifically correct content, feature a clear presentation, and address students’ learning needs. Searching for educationally useful videos on YouTube may be time consuming and requires knowledge from researchers about what makes an educationally useful video [32-35].

No studies assessing the usefulness of YouTube videos on examination of the cardiovascular system (CVS) and the respiratory system (RS) could be found in PubMed. This study is aimed at assessing YouTube videos regarding physical examination of the cardiovascular and respiratory systems.

Methods

Searching YouTube

From November 2, 2011 to December 2, 2011, the YouTube site was searched using the following key words: “cardiovascular system examination”, “cardiovascular clinical examination”, “cardiovascular physical examination”, “respiratory system examination”, “respiratory clinical examination”, and “respiratory physical examination”. When searching YouTube, quotation marks were used with these terms to specify that these terms must be present. Only the first 10 pages of results for each search term were screened for related videos. The reason for not searching videos beyond page 10 of the search term results is that one is less likely to find videos covering the key words and users are less likely to go that far in their search [26]. We observed that videos not related to search words could be seen by the third and the fourth pages. By pages 8 to 10, videos not related to the search were found. Therefore, it was decided to go only as far as the first 10 pages for each search. Videos in the English language only were identified and the related URL was recorded.

The search was conducted by three authors (HAA, RAA, and SMA) independently using the search key words. The search results were evaluated and used to compile a common pool that was used in further analysis. The inclusion criteria were videos covering clinical examination of the cardiovascular or respiratory systems in adults. Videos were excluded if they were (1) not in the English language, (2) in the form of a lecture, (3) an advertisement or news, (4) discussing signs or symptoms of diseases affecting the cardiovascular or respiratory systems, (5) about simulated patients reflecting on their experiences or roles, (6) about patients with cardiovascular or respiratory disorders, (7) about drugs used in treating patients with cardiovascular or respiratory disorders, or (8) in the form of seminars or reviews of cardiovascular or respiratory systems. Duplicated videos were also excluded and repeats were treated as a single file for analysis. The repeat file with the greatest number of hits was used for analysis.

Collected Data

For each video, the following data were collected: title, duration of the video, number of days on YouTube, total number of viewers, and name of uploader/creator (organization, group of people, one person). The number of “likes” and “dislikes” (a crude scoring system that viewers can use to assign to each video they watched, though it is not necessarily used by all viewers) for each video was recorded. Because the number of days on YouTube varies widely among videos, we decided to calculate viewership/day as a more accurate parameter compared to total number of viewers. The viewership per day is the ratio of number of viewers to the number of days a video is on YouTube. The number of days is calculated from the day of uploading on YouTube up to December 1, 2011. This calculation of viewership/day was conducted for each video. The page number on which a video was placed was recorded. This is because the YouTube search algorithm is designed to show videos according to their relevance to key words used in the search and hence videos on the first three pages are more likely to be watched than videos on lower pages. Thus, the location of the videos may indirectly affect the number of “likes” and “dislikes” given by viewers.

Analysis of Videos

The criteria used in evaluating the videos have been described in detail in an earlier work [25,35] with some modification to suit the study. In summary, the design of the criteria is based on four main domains: video content, technical aspects, authority/creator, and pedagogy used. The items in the criteria are grouped under two categories: major and minor. The major criteria comprise: (1) the video uses simulated patients or patients to demonstrate the examination, (2) contents about clinical examination are scientifically correct, (3) images are clear, (4) the creator and/or organization providing the video are mentioned, and (5) sounds are clear and background is free from noise. The minor criteria comprise: (1) the video covers the topic identified in the title, (2) designed at the level of undergraduate medical science students, (3) time to download is reasonable (about 10-15 minutes at the maximum, not interrupted or challenging to download as reported by the three evaluators), (4) information about the creator is up-to-date, (5) the educational objectives are stated, and (6) the topic is clearly presented. The criteria were used to categorize videos into educationally useful and non-educationally useful videos. We mean by “educationally useful” that a video provides scientifically correct and up-to-date knowledge and clinical instructions/skills accepted by educators in other teaching hospitals about cardiovascular and respiratory examinations. As per the basis of the evaluation criteria, educationally useful videos should fulfill the four domains (scientific content, technical aspects, authority/creator, and pedagogy used). Two scores were allocated for each item in the major criteria and one score was allocated to each item under the minor criteria. If an item was fulfilled, an allocated score was given; if not fulfilled, a zero was given. No half scores were used. As per our previous research work, educationally useful videos should fulfill all major criteria items as the minimum requirements plus at least three items under the minor criteria [35].

Testing the Criteria

To standardize the evaluation of the content of each video and the process of clinical examination, the assessors used the textbook and video by Talley and O’Connor, “Clinical Examination”, as a reference to guide their assessment [36]. The content element in the criteria comprised the following: examiner introduces him/herself to the patient; patient is correctly positioned in bed; examination is conducted within time frame; examination covers the sequence of inspection, palpation, percussion, and auscultation; physical signs are correctly elicited; and examination is conducted in a professional manner. Prior to applying the criteria, we piloted its use. A total of 25 videos were randomly selected and used for this purpose. The criteria were applied independently by three assessors (SMA, HAA, and RAA). None of the assessors shared their findings or discussed the outcome of their evaluation. An Excel spreadsheet covering the three evaluations was examined by a fourth researcher (SAA). The agreement among the assessors was in the range of 96-98%. The findings were discussed among the researchers. The criteria were tested again independently by three assessors for another 25 videos. Videos were then rated independently by three assessors (SMA, HAA, and RAA). When videos were difficult to classify or when there was a disagreement among assessors, all researchers reviewed such videos in a meeting and reached a final agreement. This study was approved by the ethics committee at King Saud University College of Medicine.

Data Analysis

The data were entered into Microsoft Excel 2010 and were checked before conducting any analysis. Analysis was conducted with SPSS software (version 18.0 for MS Windows) and was reported via mean, SD, percentage, and minimum and maximum; t tests and ANOVA (analysis of variance) were performed to determine significant differences. To assess the degree to which different judges or raters agreed in their assessment decisions, Cohen’s kappa for inter-rater reliability was used [37]. Pearson’s correlation coefficient (r) was calculated to determine if the viewership per day was correlated to the total scores given to each video. This relationship has also been examined to see if there is a correlation between the number of “likes” given by viewers to a video and the total scores given [38,39]. For all calculations, a P value <.05 was considered significant.

Results

Videos on CVS and RS Examinations

A total of 1920 YouTube videos were found on initial search and, on applying the inclusion criteria and visual examination of the videos, only 56 videos were relevant to clinical examination of the CVS (20 videos) and the RS (36 videos) (Table 1). Figure 1 summarizes how the YouTube searches and the number of videos in each category were refined on the basis of the inclusion and exclusion criteria. Examples of screenshots of these videos are shown in Figures 2-4.

‎View this figure

‎View this figure

‎View this figure

‎View this figure

Analysis of Videos

The total duration of these video clips was 349 minutes, 11 seconds. The use of the criteria for categorizing the videos into useful and non-useful videos revealed that there were 9 CVS and 7 RS videos that provided useful information on clinical examination. These videos scored mean 14.9 (SD 0.33) and mean 15.0 (SD 0.00), respectively. The total duration of useful videos was 128 minutes, 6 seconds (71 minutes, 18 seconds for the CVS videos and 56 minutes, 48 seconds for the RS videos). The total viewers of all videos were 1,458,436. For both body systems, useful videos were viewed by 542,833 viewers (37.22% of total viewers, 542,833/1,458,436). Table 2 reveals detailed information about the 56 videos included in the study and the multimedia links show examples of useful videos about examination of the cardiovascular system (Multimedia Appendix 1) and the respiratory system (Multimedia Appendix 2).

Useful videos were created by doctors or professional bodies; a link to an organization such as ImedrxTV, geekymedics123, MedicalStudentApps, mosaicism105, or the name of the creator and his or her credentials, such as Professor S. Femando from the Open University of Sri Lanka, were shown. Some videos were linked to universities and known teaching institutes such as Manchester Medical School, St. George’s University of London, Central Manchester University Hospitals, the National Health Service, and the University of Wisconsin School of Medicine and Public Health.

Non-educationally useful videos scored mean 11.1 (SD 1.08) for CVS examination and mean 11.2 (SD 1.29) for the RS examination. Compared to educationally useful videos, the differences were significant (P<.001 for both CVS and RS videos) (Table 1). For both body systems, non-useful videos were viewed by 915,603 (62.78% of total viewers, 915,603/1,458,436). Non-educationally useful videos failed due to a number of reasons. The majority of the non-educationally useful videos failed to fulfill one of the major criterion items. Among those, more than 80% (33/40) were due to the image lacking clarity or no mention of the creator of the video. The concordance between the assessors on applying the criteria was 0.89, with a kappa score >.86.

The correlations between the total video scores and the number of viewers/day were not significant for the CVS videos (r=.06, P=.059) and the RS videos (r=.027, P=.827). Similarly, no correlation was found between likes or dislikes and the total number of scores given to CVS videos (r=.06, P=.059) or RS videos (r=.02, P=.832). No significant correlation was found between the video location on the first pages and the scores for like or dislike.

Discussion

Principal Findings

The aim of this study was to conduct an analysis of YouTube videos about CVS and RS physical examinations and to categorize these videos for educational purposes. Several methods have been described in evaluating videos [40-43]. However, none of these systems were useful for classifying videos about clinical skills. The system used in this study is simple, easy to apply, and covers four key elements, namely: scientific content, technicality, authority, and pedagogy parameters. It has also been tested in two previous studies and has shown a high level of inter-rater correlation while covering the key elements required for an educationally useful video [25,35].

The findings in this study indicated that there are 9 educationally useful videos on CVS examination and 7 on RS examination. These videos provided approximately 71 minutes, 18 seconds of CVS examination and 56 minutes, 48 seconds of RS examination that can be used in clinical learning and teaching purposes. Useful videos were linked mainly to universities or educational institutes. This indicates the involvement of universities and teaching institutes in promoting the use of educational videos as a resource to learners. This is particularly important with the shift of most universities toward self-directed learning and student-centered programs.

YouTube’s search engine, despite all precautions taken to target the search results to videos about examination of these two body systems, delivered more than 1800 videos that were not related to the search terms. Furthermore, the lack of correlation between the total score given to a video and the location of the video when searched (the page number) highlights the fact that the YouTube search algorithm is not well calibrated and unrelated videos usually appear despite the search filter provided by YouTube. Also, the lack of correlation between the total score given to a video to the “like” and “dislike” numbers as well as to viewership/day suggests the possibility that many viewers were watching substandard videos in their learning. Assuming that the majority of these viewers were medical/health students and trainees, these results highlight the need for YouTube to improve its search algorithm system to generate more accurate lists of videos that match with the search terms.

Much emphasis has been placed on physical examination in current medical curricula. To learn such skills, students usually rely on observing clinicians conducting examinations and then practicing on simulated patients and real patients. Recently, Duvivier et al (2012) in a study from Maastricht University reported that on average students devote 20% of their self-study time to skills training on physical examination [3]. They found that students use textbooks, examination guidelines, scientific articles, the Internet, videos/DVD, and scoring forms from previous OSCEs as their learning resources. Although the Internet and videos were among these resources, the study did not explore the sources of these videos and whether YouTube videos were used. Also, the study did not specify how much time they spend on average on resources such as videos and the Internet [3].

There is no substitute for witnessing a physical examination or a clinical procedure being performed live. Neither static images nor a description in a book outlining these techniques can offer the same impact as personal experience. Videos, however, are a medium that can transfer the experience and help in mastering such skills through repeated watching of techniques used and information provided. Online videos have become a routine and important tool in a student’s preparation for clinical skills. Attending clinical skills sessions, reading textbooks along with watching online videos, and practicing skills learned on patients have become important learning strategies in most clinical schools [44,45]. Based on cognitive psychological research, the use of videos will help expose students to the techniques of clinical examination, approaches for examining patients, and how to manage the sequence of technical steps in such examinations [45-50].

Considering the increasing number of learners using the Internet as their primary source of information, medical educators and clinical teachers should recognize the importance of YouTube in education and invest in using Web 2.0 in learning and teaching activities such as clinical teaching [8,13-14]. Although there are other links on the Web that provide free videos, we decided to examine the videos on YouTube for a number of reasons. First, YouTube is popular and usually preferred by users compared to other websites. Second, it is relatively easy to share videos on YouTube. Third, there is continuous improvement in the design of the YouTube site and each video is accompanied by useful data that reflect the evolution of online social networking and can be of use to researchers and viewers. Finally, YouTube has succeeded in providing social networking and enabling discussion among viewers, which could be useful to viewers as they share useful videos with each other.

This study shows that clinical teachers who are competent in clinical teaching and clinical educators, such as those who created the videos we identified in this study, can offer great service to medical and health students worldwide by placing their work on sharing websites such as YouTube. This can be part of knowledge transfer and scholarly work as it is created by academics, shared, and peer-reviewed [50]. Recently, YouTube launched YouTube EDU, an area of YouTube where video creators must possess high-quality credentials and provide evidence of significant mass of resources before they qualify to have their content included in YouTube EDU. This may have a major impact on improving successful search strategies particularly for YouTube videos designed for educational purposes.

Limitations

This study represents a snapshot of available resources during November/December 2011, and, since then, there may have been more videos uploaded and made available. Given the continuous upload of videos on YouTube and the volume of new videos added to the system on a daily basis, further studies are needed to assess whether new videos of high quality and coverage have been added.

The small number of videos used in this study is a limitation to this study. The study was limited to videos in the English language and only those covering examination of adults, which may have contributed to the smaller number of videos found. However, the majority of videos uploaded on YouTube are in the English language. Also, there is the possibility that some videos were not labeled as such and thus were unidentifiable under our search terms, despite all our precautions including conducting the search of YouTube over 30 days independently by three researchers and using six search keywords to compile a common pool. The search in this study was limited to YouTube and there is the possibility that videos on other websites, such as those of medical and other health professional societies and medical journals, were not included.

Conclusions

This is possibly the first study assessing the educational value of YouTube videos about physical examination of the CVS and RS systems. Despite the small number of videos identified and found educationally useful, these videos can be used by medical students for independent learning and by clinical teachers as learning resources. The scoring system utilized by this study is simple, easy to apply, and could be used by other researchers on similar topics. The authors encourage other researchers to assess the tool and contribute to its improvement.