Published on in Vol 18, No 7 (2016): July

The Effect of a Freely Available Flipped Classroom Course on Health Care Worker Patient Safety Culture: A Prospective Controlled Study

The Effect of a Freely Available Flipped Classroom Course on Health Care Worker Patient Safety Culture: A Prospective Controlled Study

The Effect of a Freely Available Flipped Classroom Course on Health Care Worker Patient Safety Culture: A Prospective Controlled Study

Original Paper

1Prince of Wales Hospital, Department of Anaesthesia and Intensive Care, Shatin, China (Hong Kong)

2The Chinese University of Hong Kong, Department of Anaesthesia and Intensive Care, Shatin, China (Hong Kong)

Corresponding Author:

Anna Lee, MPH, PhD

The Chinese University of Hong Kong

Department of Anaesthesia and Intensive Care

4th Floor, Main Clinical Block and Trauma Centre

Prince of Wales Hospital


China (Hong Kong)

Phone: 852 2632 2735

Fax:852 2637 2422


Background: Patient safety culture is an integral aspect of good standard of care. A good patient safety culture is believed to be a prerequisite for safe medical care. However, there is little evidence on whether general education can enhance patient safety culture.

Objective: Our aim was to assess the impact of a standardized patient safety course on health care worker patient safety culture.

Methods: Health care workers from Intensive Care Units (ICU) at two hospitals (A and B) in Hong Kong were recruited to compare the changes in safety culture before and after a patient safety course. The BASIC Patient Safety course was administered only to staff from Hospital A ICU. Safety culture was assessed in both units at two time points, one before and one after the course, by using the Hospital Survey on Patient Safety Culture questionnaire. Responses were coded according to the Survey User’s Guide, and positive response percentages for each patient safety domain were compared to the 2012 Agency for Healthcare Research and Quality ICU sample of 36,120 respondents.

Results: We distributed 127 questionnaires across the two hospitals with an overall response rate of 74.8% (95 respondents). After the safety course, ICU A significantly improved on teamwork within hospital units (P=.008) and hospital management support for patient safety (P<.001), but decreased in the frequency of reporting mistakes compared to the initial survey (P=.006). Overall, ICU A staff showed significantly greater enhancement in positive responses in five domains than staff from ICU B. Pooled data indicated that patient safety culture was poorer in the two ICUs than the average ICU in the Agency for Healthcare Research and Quality database, both overall and in every individual domain except hospital management support for patient safety and hospital handoffs and transitions.

Conclusions: Our study demonstrates that a structured, reproducible short course on patient safety may be associated with an enhancement in several domains in ICU patient safety culture.

J Med Internet Res 2016;18(7):e180



Good medical practice is based on the classic maxim of “primum non nocere,” and yet it is estimated that at least 1 in 10 patients may be harmed by adverse events during their hospital stay [1,2]. The landmark US Institute of Medicine’s report in 1999, To Err is Human: Building a Safer Health System, ignited the interest in improving patient safety. Studies into harm suggest that a significant proportion of adverse events is preventable [3]. These include prescription mistakes, handover lapses, surgical errors, diagnostic mishaps, and other errors attributable to the human factor [4-7].

Changing and adopting health care technology has been shown to reduce medical errors and improve patient safety [8]. However, technology itself improves patient safety only to a limited extent, and further error reduction requires human factors and organizational change [9]. Clinical human interventions such as additional pharmacist inspection of electronic prescriptions can further reduce medication errors [6]. Using targeted education to change clinical practice seems to be effective as well. Specific interventions adopted for central venous catheter insertions have been shown to reduce central venous access-related infection and improve patient outcome [10]. However, it has also been shown that although targeted educational interventions could improve clinical staff knowledge, this did not translate to improved outcomes [11]. This highlights multiple challenges in studying the effect of education. First, knowledge itself may be a prerequisite for safety culture. However, attitudes and perception are equally important but more difficult to measure and define. Second, education is often not standardized, as such the findings may not be generalizable. Third, it is difficult to conduct blinded randomized trials with appropriate controls. Studies on patient safety education programs have generally not assessed the effectiveness of these interventions with adequate controls and rigor [12].

Recent reports on the failings of a hospital in the United Kingdom have highlighted issues with a lack of an appropriate patient safety culture [13]. The likely causal relationship between poor culture and poor patient care stresses the importance of improving culture to improve standards of care. Therefore, we decided to conduct a prospective controlled study to assess the impact of a standardized, free license, patient safety course on patient safety culture.

Study Design and Hospitals

The study protocol was approved by the Chinese University of Hong Kong Survey and Behavioural Research Ethics Committee. This was a prospective controlled, before and after, study design that used the Hospital Survey on Patient Safety Culture (HSOPSC) questionnaire instrument to evaluate the impact of the BASIC (Basic Assessment and Support in Intensive Care) Patient Safety Course on safety culture. The course was delivered to doctors, nurses, and health care assistants in the Intensive Care Unit (ICU) of hospital A (ICU A) only. In order to control for temporal changes or changes resulting from policies implemented across the entire public hospital system, the questionnaire was also administered to equivalent staff in the ICU of hospital B, a neighboring hospital (ICU B). The two hospitals are publicly funded, located in the same hospital cluster, and share a common cluster chief executive. Hospital A is a tertiary teaching hospital with 1400 beds and 22 ICU beds. Hospital B is an acute general hospital with 600 inpatient beds and 14 ICU beds. A comparison of clinical data between ICU A and ICU B is shown in Table 1.

Table 1. Comparison Statistics of ICU A and B.

Admissions per yearSeverity of illness (APACHE III Acute Physiology Score)Average ICU length of stayRisk-adjusted hospital mortality ratio
ICU A1500504 days0.80
ICU B600555 days0.75

BASIC Patient Safety Course

The BASIC Patient Safety course is a blended learning course that uses a flipped classroom approach in which didactic teaching is carried out prior to participants attending face-to-face teaching. This allows face-to-face time to be dedicated to interactive sessions involving application of the knowledge already acquired. In our course, preparatory material consists of a short printed course manual and e-learning.

The e-learning material comprises short narrated lectures, typically based around a modified clinical case, formative assessment, an interactive electronic lesson, and a video of an incident involving a serious medication error. The e-lectures were created in PowerPoint and Camtasia and were produced as MP4 files so that they could be played on different platforms including Windows, iOS, and Android. The files were uploaded to a Moodle 2.0 platform that was configured to be accessible both on personal computers and mobile devices. Each lecture is supplemented by a formative assessment, in the form of a multiple choice test, which emphasizes the key points covered in the lecture. The system is configured to allow participants to access the assessment only after watching the corresponding lecture for its entire duration.

The interactive lesson is a more complex form of formative assessment. Each participant’s individual pathway through the lesson is dependent on their answers to questions posed during the lesson. Candidates with a poorer understanding of the material take a longer pathway, receiving greater explanation of basic aspects before moving on to more complex issues, while those with a greater understanding rapidly progress to the more complex material. Teaching is thus adjusted to the participant’s needs. All activity on the e-learning site is automatically logged (with the knowledge of participants). Completion of all e-learning is required before attending the face-to-face teaching. Computers are made available to participants who are unable to access the material on their own devices, but no participants required this facility during the study.

Face-to-face teaching consists of small group teaching involving a simulated emergency (to practice communication and team and leadership skills), practicing Situation Background Assessment Recommendation (SBAR) communication, practicing breaking news of an error, discussion of the video shown on the e-learning site, and reflection on patient safety in one’s own unit. This was followed by a group debriefing session. The face-to-face component of the course lasts 3 hours.

The course was written specifically for health care professionals whose primary function is to provide clinical care. It is not aimed at those with a predominantly managerial role. Participants are expected to gain detailed knowledge of the definitions and scope of patient safety, human factors engineering and why it is important to patient safety; cause and reduction of errors, preventing errors leading to harm, cognition, communication with colleagues and patients and the importance of full disclosure, root cause analysis, quality improvement, teamwork, medication safety, and coping with errors. They are expected to enhance the skills required to be an effective team player, understand and learn from errors, understand and manage clinical risk, engage with patients and caregivers, and communicate with full disclosure after adverse events. The course material is available free of charge, and the course is disseminated on a train-the-trainers basis. Facilitators/instructors for the small group teaching are given detailed written guidance on the content for each discussion/tutorial. One instructor is required for every 6 participants.

Between April to December 2011, 117 participants attended the course, of whom 91 nurses and 8 doctors worked in ICU A. No staff from ICU B attended the course. The course was taught predominantly by senior nursing and medical staff from ICU A.


A convenience sample of doctors, nurses, and health care assistants in the ICU from the two hospitals were asked to complete a Hong Kong Chinese version of the HSOPSC before and after the course was implemented to measure their attitudes towards patient safety. The survey consisted of 44 questions with 5-point Likert response scale of agreement of either strongly disagree to strongly agree, never to always, or failing to excellent. Our version contained an extra question about whether the survey participant would feel safe being treated as a patient in the respective hospital. These questions were grouped into 13 dimensions and the overall positive scores were calculated by categorizing strongly agree/agree, excellent/very good, or always/most of time as positive responses. For questions that were negatively worded, positive responses equated to strongly disagree/disagree or never/rarely. The survey also asked the frequency of event reporting by the participant over the past year.

The pre-course (baseline) survey was carried out immediately before the first time the course was run, and the post course survey within 3 months of completion of the series of courses. We wanted to assess the effects of the course on general staff patient safety culture rather than specifically on the attitudes of course attendees. Therefore, the survey respondents were selected randomly from the staff of ICU A and ICU B. Course feedback was collected from participants using anonymized electronic feedback forms.

Statistical Analyses

We entered the responses into the AHRQ Hospital Survey on Patient Safety Culture Excel tool (version 1.5). Positive responses were coded according to the Survey User’s Guide. For each hospital, the percentage change was estimated as the follow-up percentage minus the baseline percentage. The 95% confidence interval (95% CI) around the positive response percentage for each AHQR patient safety domain was estimated, and baseline results were compared to the 2012 AHRQ ICU sample of 36,120 respondents to provide a contextual reference for interpretation of the applicability of our findings.

Separate generalized estimating equations (GEE) were used to account for the correlation in participants responding to both pre- and post-workshop questionnaires. Separate difference-in-differences models were constructed for each of the 12 AHRQ patient safety domains using participant-level data [14]. The outcome “positive response” was modelled as a function of ICU (A or B), period (baseline or post-intervention), and an interaction term between ICU and period, adjusted for duration of work in Intensive Care (≤10 years vs >10 years). The coefficient for the interaction term in the GEE model indicated whether ICU A improved more or less than ICU B from baseline to follow up. All statistical analysis was carried out using SPSS version 22.0.

Response Rates and Participant Characteristics

The total number of questionnaires distributed across the two hospitals was 127 with an overall response rate of 74.8% (95 respondents). Three respondents answered both pre- and post-workshop questionnaires. The pre- and post-intervention response rates from ICU A were 88% (37/42) and 79% (23/29), respectively. The response rates from ICU B were 63% for both pre- (20/32) and post-intervention (15/24) survey.

Of the 95 participants, 78 were registered ICU nurses, 11 patient care assistants, and 6 physicians. Most respondents (90/95, 95%) had direct contact or interactions with patients. Over half (53/95, 56%) had worked in the hospital system for less than 10 years. There was no difference in the proportion of participants working less than or equal to 10 years in the current work area/unit between hospitals: ICU A 58% (33/57) versus ICU B 74% (28/38), P=.12.

Survey Responses

ICU A had lower positive responses at baseline on 7 of 12 domains when compared to baseline responses of ICU B (see Table 2). These included overall perception of safety (P=.007), organization learning/continuous improvement (P=.03), teamwork within hospital units (P=.003), communication openness (P=.03), feedback and communication about error (P<.001), staffing (P<.001), and hospital management support for patient safety (P=.02).

After the safety course, ICU A had significantly improved responses in teamwork within hospital units (P=.008) and hospital management support for patient safety (P<.001), but decreased in the frequency of reporting mistakes (P=.006) compared to baseline. For ICU B, there was a decrease in the proportion of positive responses in 6 measured domains during the same period (see Table 2).

There was a significant interaction between ICUs and period, after adjusting for the duration of work in current area/unit, indicating that ICU A showed greater improvement in positive responses than ICU B in 5 domains (see Table 3).

Table 2. Unadjusted difference in positive responses at baseline and follow-up between hospitals.
DomainBaseline responses (%)Follow-up responses (%)Changes from baseline (%)a
Frequency of reporting32/60 (53.3)43/108 (39.8)14/43 (32.6)13/66 (19.7)-20.8.04-20.1.006
Overall perception of safety39/80 (48.8)42/148 (28.3)21/60 (35.0)37/92 (40.2)-13.8.1011.9.06
Supervisor/manager expectations and actions promoting safety55/80 (68.8)82/148 (55.4)33/60 (55.0)60/92 (65.2)-
Organization learning/continuous improvement45/60 (75.0)61/111 (55.0)27/45 (60.0)44/69 (63.8)-
Teamwork within hospital units71/80 (88.8)91/147 (61.9)40/60 (66.7)72/92 (78.3)-22.1.00116.4.008
Communication openness31/60 (51.7)34/111 (30.6)12/45 (26.7)21/69 (30.4)-25.0.01-0.2.98
Feedback and communication about error47/60 (78.3)40/111 (36.0)18/45 (40.0)34/69 (49.3)-38.3<.00113.3.08
Nonpunitive response19/60 (31.7)19/111 (17.1)6/45 (13.3)10/69 (14.5)-18.4.03-2.6.64
Staffing48/80 (60.0)42/147 (28.6)24/60 (40.0)32/90 (35.6)-
Hospital management support for patient safety39/60 (65.0)44/108 (40.7)27/45 (60.0)48/69 (69.6)-5.0.6028.9<.001
Teamwork across hospital units41/80 (51.3)68/144 (47.2)23/60 (38.3)39/92 (42.4)-13.0.13-4.8.47
Hospital handoffs and transitions40/79 (50.6)70/143 (49.0)34/59 (57.6)43/92 (46.7)-7.0.42-2.3.74

aFollow-up percentage minus the baseline percentage. Denominators for each item are the product of the number of questions in that domain and the number of respondents. Numerators are the total number of positive responses to all questions in that domain.

Table 3. Relative risk (95% CI) of improvement in patient safety domains: Baseline to follow-up in hospitals with and without educational intervention.
DomainRelative risk (95% CI for difference between groups)aP value
Frequency of reporting0.90 (0.33-2.49).84
Overall perception of safety1.94 (1.11-3.37).02
Supervisor/manager expectations and actions promoting safety1.48 (0.99-2.20).06
Organization learning/continuous improvement1.45 (0.96-2.20).08
Teamwork within hospital units1.55 (1.10-2.19).01
Communication openness1.66 (0.73-3.76).23
Feedback and communication about error2.47 (1.28-4.80).007
Nonpunitive response1.68 (0.54-5.18).37
Staffing1.92 (1.15-3.19).01
Hospital management support for patient safety1.88 (1.16-3.04).01
Teamwork across hospital units1.23 (0.75-2.00).41
Hospital handoffs and transitions0.86 (0.44-1.70).67

aAdjusted for duration of work in current area/unit (≤10 years vs >10 years)

Participants in ICU A were seven times more likely to report “feeling safe being treated in this hospital as a patient” than those in ICU B after adjusting for duration of work in the current area/unit (P=.01; see Table 4).

Table 4. Response to statement “I would feel safe being treated in this hospital as a patient.”
Baseline responses (%)Follow-up responses (%)Changes from baseline responses (%)Relative riska (95% CI)
Feel safe11/19 (57.9)14/35 (40.0)3/15 (20.0)12/22 (54.5)-37.914.57.29 (1.52-34.94)

aInteraction effect (risk ratio of improvement from base to follow-up between ICUs, adjusted for duration of work in current area/unit (≤10 years vs >10 years).

Pooled data of all participants from both ICUs indicate that patient safety culture was poorer than the average ICUs in the 2012 AHRQ database (see Table 5). Reponses from both ICUs were lower (at least 5% point difference) for every individual domain except hospital management support for patient safety and hospital handoffs and transitions.

Table 5. Domain-level comparative average percentage (95% CI) positive responses of Hong Kong ICUs (N=95) to 2012 AHRQ database (N=36,120).
DomainHong Kong ICUs,
% (95% CI)
2012 AHRQ ICUs, %
Frequency of reporting37 (28-47)59
Overall perception of safety37 (28-47)60
Supervisor/manager expectations and actions promoting safety61 (51-70)73
Organization learning/continuous improvement62 (52-71)72
Teamwork within hospital units72 (62-80)84
Communication openness34 (25-44)61
Feedback and communication about error49 (40-59)60
Nonpunitive response19 (12-28)40
Staffing39 (30-49)58
Hospital management support for patient safety56 (46-65)64
Teamwork across hospital units45 (36-55)57
Hospital handoffs and transitions50 (39-59)51
Average across domains47 (38-57)62

Of the 117 safety course participants, 90 (77%) answered the course feedback questionnaire. The vast majority of these 90 participants agreed or strongly agreed with positive statements about the course (see Figure 1), with 32% strongly agreeing and another 56% agreeing that the course was useful to improve patient safety. Notably, participants were as likely to agree or strongly agree with positive statements about the electronic lectures as they were to agree or strongly agree with positive statements about other aspects of the course.

Figure 1. Participants responses to the feedback questionnaire.
View this figure

Principal Findings

Our data suggest that a structured, reproducible, short blended learning course on patient safety may improve perceived ICU patient safety culture. After controlling for duration of working in the respective ICUs, there was a significant improvement in 5 of 12 domains and a trend towards improvement in 2 others in the ICU where the course was given. Furthermore there was a substantial difference in the change of response to the additional statement “I would feel safe being treated in this hospital as a patient” in favor of ICU A.

However, it is notable that there was a deterioration in patient safety culture in ICU B during the study period. This could be due to factors unique to that ICU or hospital or to systemic changes that would also have affected ICU A. If the decline in ICU B was due to systemic factors, then our results suggest that our course not only arrested but largely reversed the deterioration. On the other hand, if the deterioration was due to unique factors affecting only ICU B, then the positive effect of our course could be less than our results would suggest. Nevertheless, if one compares only pre- and post-course results in ICU A in isolation (see Table 2), it can be seen that there were still significant improvements in the domains of teamwork within hospital units and hospital management support for patient safety, and weak evidence supporting an improvement in the feedback and communication about error and overall perception of safety domains.

There was a significant deterioration in the frequency of reporting domain in both ICUs during the study period. In ICU B, this may simply reflect the general deterioration seen across multiple domains. In ICU A, the change is difficult to explain as it is the only domain in which there was a significant reduction. One possibility is that the changes in ICU B did reflect a systemic deterioration in patient safety culture across the two hospitals that was reversed in most, but not all, domains in ICU A by the course. However, we cannot exclude the possibility that this was an inadvertent adverse effect of the course.


Our study has a number of other weaknesses. Similar to all before and after studies, we cannot exclude the possibility of confounding factors that affected only ICU A. Rather than a direct comparison between two ICUs, this study evaluated the temporal change in each unit. Therefore the relevant confounding factors are ones that affect temporal changes rather than the baseline differences between the two ICUs. What we have shown is a temporal relationship between our course and changes in patient safety attitudes, not a causal relationship. The feedback data suggest that the course may have changed attitudes. However, it is possible that the responses of some of the participants from ICU A may have been influenced by the fact that the course was taught by senior staff from the same ICU, even though the feedback was anonymous. Furthermore, any change in attitude may have been due to the involvement of senior staff signaling to other staff that patient safety is an important issue rather than the educational content of the course itself.

We studied only two ICUs; therefore, our results may not be generalizable to other ICUs or to other hospital departments. Previous studies on patient safety culture in Chinese countries such as China and Taiwan showed some important ethnic and cultural factors that may result in differences to western patient safety culture [15-18]. Furthermore, the baseline data suggest that the patient safety culture was poor in both ICUs, relative to ICUs contributing to the AHRQ database, and it is possible that the course may have little effect in units where patient safety culture is well developed. Finally, we have studied only the short-term effect of the course, and although appropriate patient safety culture is considered a pre-requisite to patient safety behavior in practice, this has not been rigorously tested [19].

Study Strengths

Our study does have the advantage that we studied the effect of a standardized, freely available, educational intervention with a parallel control group [14,20]. The standardized nature of the course both facilitates further research (ie, the same intervention is tested each time) and the applicability of the results (course material may be obtained directly from the authors). Studies of educational interventions that are not in the public domain cannot be reproduced, and it is unclear whether the results can be applied to other individual educational packages. Previous studies of educational interventions to improve patient safety show variable results suggesting that the exact nature of the intervention may be important [10,11,21-24]. In particular, our course incorporates e-learning, interactive modules, and formative assessments. Active involvement and formative assessment are key elements for effective adult learning. A similar educational approach has been used by the Canadian “Managing Obstetric Risks Efficiently” safety program and proven to be effective in advancing safety knowledge (culture was not examined) [25].

Although the course is labor-intensive with a high ratio of instructors to participants, the face-to-face contact time is short as a result of the pre-course reading and e-learning. This facilitates its use as part of in-service training, by minimizing disruption to clinical staffing. It is notable that the e-learning components of the course were highly rated by participants. Although we did not test the specific effect of the e-learning, a study of one of our other courses suggests it enhances learning [26]. The same study revealed that participants value the flexibility of listening to e-lectures at their own convenience and the ability to re-play lectures.


Our results suggest that the course may improve patient safety culture. Further research is required to establish whether the temporal association can be reproduced when more units are studied in a variety of different cultures and if so, an attempt should be made to determine whether the relationship is causal. In conclusion, introduction of a standardized patient safety course was temporally associated with an improvement in several domains of patient safety culture in a single ICU.


We thank the Hong Kong Hospital Authority and the Chinese University of Hong Kong Grants Committee Knowledge Transfer Project Fund (KTP091126F06) for funding and support.

Authors' Contributions

LLWL drafted the manuscript and made substantial revisions. AL performed the statistical analyses and had full access to all the data in the study. CDG, GMJ, and AL were involved in the study concept and design of the study. WS collected the data. All authors interpreted the data, made critical revisions to the manuscript for important intellectual content, and approved the final version of the manuscript.

Conflicts of Interest

CDG and GMJ were involved in developing and teaching the BASIC course.

  1. Davis P, Lay-Yee R, Briant R, Ali W, Scott A, Schug S. Adverse events in New Zealand public hospitals I: occurrence and impact. N Z Med J 2002 Dec 13;115(1167):U271. [Medline]
  2. de Vries EN, Ramrattan MA, Smorenburg SM, Gouma DJ, Boermeester MA. The incidence and nature of in-hospital adverse events: a systematic review. Qual Saf Health Care 2008 Jun;17(3):216-223 [FREE Full text] [CrossRef] [Medline]
  3. Vincent C, Neale G, Woloshynowych M. Adverse events in British hospitals: preliminary retrospective record review. BMJ 2001 Mar 3;322(7285):517-519 [FREE Full text] [Medline]
  4. Leape LL, Brennan TA, Laird N, Lawthers AG, Localio AR, Barnes BA, et al. The nature of adverse events in hospitalized patients. Results of the Harvard Medical Practice Study II. N Engl J Med 1991 Feb 7;324(6):377-384. [CrossRef] [Medline]
  5. Thomas EJ, Studdert DM, Burstin HR, Orav EJ, Zeena T, Williams EJ, et al. Incidence and types of adverse events and negligent care in Utah and Colorado. Med Care 2000 Mar;38(3):261-271. [Medline]
  6. Abdel-Qader DH, Harper L, Cantrill JA, Tully MP. Pharmacists' interventions in prescribing errors at hospital discharge: an observational study in the context of an electronic prescribing system in a UK teaching hospital. Drug Saf 2010 Nov 1;33(11):1027-1044. [CrossRef] [Medline]
  7. Starmer AJ, Sectish TC, Simon DW, Keohane C, McSweeney ME, Chung EY, et al. Rates of medical errors and preventable adverse events among hospitalized children following implementation of a resident handoff bundle. JAMA 2013 Dec 4;310(21):2262-2270. [CrossRef] [Medline]
  8. Ball MJ, Garets DE, Handler TJ. Leveraging information technology towards enhancing patient care and a culture of safety in the US. Methods Inf Med 2003;42(5):503-508. [CrossRef] [Medline]
  9. Anderson JG, Ramanujam R, Hensel D, Anderson MM, Sirio CA. The need for organizational change in patient safety initiatives. Int J Med Inform 2006 Dec;75(12):809-817. [CrossRef] [Medline]
  10. Bion J, Richardson A, Hibbert P, Beer J, Abrusci T, McCutcheon M, Matching Michigan Collaboration & Writing Committee. 'Matching Michigan': a 2-year stepped interventional programme to minimise central venous catheter-blood stream infections in intensive care units in England. BMJ Qual Saf 2013 Feb;22(2):110-123 [FREE Full text] [CrossRef] [Medline]
  11. Santana SL, Furtado GH, Wey SB, Medeiros EA. Impact of an education program on the incidence of central line-associated bloodstream infection in 2 medical-surgical intensive care units in Brazil. Infect Control Hosp Epidemiol 2008 Dec;29(12):1171-1173. [CrossRef] [Medline]
  12. Halligan M, Zecevic A. Safety culture in healthcare: a review of concepts, dimensions, measures and progress. BMJ Qual Saf 2011 Apr;20(4):338-343. [CrossRef] [Medline]
  13. Investigation into mid Staffordshire NHS Foundation Trust. United Kingdom: Healthcare Commission; 2009 Mar.   URL: [accessed 2015-10-16] [WebCite Cache]
  14. Dimick JB, Ryan AM. Methods for evaluating changes in health care policy: the difference-in-differences approach. JAMA 2014 Dec 10;312(22):2401-2402. [CrossRef] [Medline]
  15. Liu Y, Kalisch BJ, Zhang L, Xu J. Perception of safety culture by nurses in hospitals in china. J Nurs Care Qual 2009;24(1):63-68. [CrossRef] [Medline]
  16. Nie Y, Mao X, Cui H, He S, Li J, Zhang M. Hospital survey on patient safety culture in China. BMC Health Serv Res 2013;13:228 [FREE Full text] [CrossRef] [Medline]
  17. Chen I, Li HH. Measuring patient safety culture in Taiwan using the Hospital Survey on Patient Safety Culture (HSOPSC). BMC Health Serv Res 2010;10:152 [FREE Full text] [CrossRef] [Medline]
  18. Zhu J, Li L, Li Y, Shi M, Lu H, Garnick DW, et al. What constitutes patient safety culture in Chinese hospitals? Int J Qual Health Care 2012 Jun;24(3):250-257 [FREE Full text] [CrossRef] [Medline]
  19. Wakefield JG, McLaws ML, Whitby M, Patton L. Patient safety culture: factors that influence clinician involvement in patient safety behaviours. Qual Saf Health Care 2010 Dec;19(6):585-591. [CrossRef] [Medline]
  20. Villa JM. diff: Simplifying the estimation of difference-in-differences treatment effects. The Stata Journal 2016 Mar;16(1):52-71.
  21. Warren DK, Zack JE, Cox MJ, Cohen MM, Fraser VJ. An educational intervention to prevent catheter-associated bloodstream infections in a nonteaching, community medical center. Crit Care Med 2003 Jul;31(7):1959-1963. [CrossRef] [Medline]
  22. Warren DK, Zack JE, Mayfield JL, Chen A, Prentice D, Fraser VJ, et al. The effect of an education program on the incidence of central venous catheter-associated bloodstream infection in a medical ICU. Chest 2004 Nov;126(5):1612-1618. [CrossRef] [Medline]
  23. Sherertz RJ, Ely EW, Westbrook DM, Gledhill KS, Streed SA, Kiger B, et al. Education of physicians-in-training can decrease the risk for vascular catheter infection. Ann Intern Med 2000 Apr 18;132(8):641-648. [Medline]
  24. Yilmaz G, Caylan R, Aydin K, Topbas M, Koksal I. Effect of education on the rate of and the understanding of risk factors for intravascular catheter-related infections. Infect Control Hosp Epidemiol 2007 Jun;28(6):689-694. [CrossRef] [Medline]
  25. Milne JK, Lalonde AB. Patient safety in women's health-care: professional colleges can make a difference. The Society of Obstetricians and Gynaecologists of Canada MORE(OB) program. Best Pract Res Clin Obstet Gynaecol 2007 Aug;21(4):565-579. [CrossRef] [Medline]
  26. Gomersall CD, Lam P, Joynt GM. Electronic interactive learning to supplement acute care teaching. Med Educ 2010 May;44(5):526. [CrossRef] [Medline]

AHRQ: Agency for Healthcare Research and Quality
BASIC: Basic Assessment and Support in Intensive Care
GEE: Generalized Estimating Equations
HSOPSC: Hospital Survey on Patient Safety Culture
ICU: Intensive Care Unit
SBAR: Situation Background Assessment Recommendation

Edited by G Eysenbach; submitted 25.11.15; peer-reviewed by WC Su, ML Lypson; comments to author 03.03.16; revised version received 22.04.16; accepted 04.06.16; published 05.07.16


©Lowell Ling, Charles David Gomersall, Winnie Samy, Gavin Matthew Joynt, Czarina CH Leung, Wai-Tat Wong, Anna Lee. Originally published in the Journal of Medical Internet Research (, 05.07.2016.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on, as well as this copyright and license information must be included.