This paper is in the following e-collection/theme issue:

e-Learning and Medical Education (645) Continuing Medical Education (CME) for Doctors (120) Training Tools for Medical Students (124)

Published on in Vol 23, No 5 (2021): May

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/27124, first published .
Effects of an Animated Blood Clot Technology (Visual Clot) on the Decision-Making of Users Inexperienced in Viscoelastic Testing: Multicenter Trial

Effects of an Animated Blood Clot Technology (Visual Clot) on the Decision-Making of Users Inexperienced in Viscoelastic Testing: Multicenter Trial

Effects of an Animated Blood Clot Technology (Visual Clot) on the Decision-Making of Users Inexperienced in Viscoelastic Testing: Multicenter Trial

Authors of this article:

Sadiq Said1 Author Orcid Image ;   Tadzio Raoul Roche1 Author Orcid Image ;   Julia Braun2 Author Orcid Image ;   Micheal Thomas Ganter3 Author Orcid Image ;   Patrick Meybohm4 Author Orcid Image ;   Johannes Herrmann4 Author Orcid Image ;   Kai Zacharowski5 Author Orcid Image ;   Florian Jürgen Raimann5 Author Orcid Image ;   Florian Piekarski5 Author Orcid Image ;   Eva Rivas6 Author Orcid Image ;   Manuel López-Baamonde6 Author Orcid Image ;   Donat R Spahn1 Author Orcid Image ;   Christoph Beat Nöthiger1 Author Orcid Image ;   David Werner Tscholl1 Author Orcid Image
Article Authors Cited by (7) Tweetations (1) Metrics

    Original Paper

    1Institute of Anesthesiology, University Hospital Zurich, University of Zurich, Zurich, Switzerland

    2Department of Epidemiology, Epidemiology, Biostatistics and Prevention Institute, University of Zurich, Zurich, Switzerland

    3Institute of Anesthesiology and Pain Therapy, Cantonal Hospital Winterthur, Winterthur, Switzerland

    4Department of Anesthesiology, Intensive Care, Emergency, and Pain Medicine, University Hospital Wuerzburg, University of Wuerzburg, Wuerzburg, Germany

    5Department of Anesthesiology, Intensive Care Medicine, and Pain Therapy, University Hospital Frankfurt, Goethe University Frankfurt, Frankfurt, Germany

    6Department of Anesthesiology, Intensive Care Medicine, and Pain Therapy, Hospital Clinic of Barcelona, University of Barcelona, Barcelona, Spain

    Corresponding Author:

    Tadzio Raoul Roche

    Institute of Anesthesiology

    University Hospital Zurich

    University of Zurich

    Raemistrasse 100

    Zurich, 8091

    Switzerland

    Phone: 41 432530255

    Email: Tadzioraoul.roche@usz.ch


    Background: Viscoelastic test–guided coagulation management has become increasingly important in assessing hemostasis. We developed Visual Clot, an animated, 3D blood clot that illustrates raw rotational thromboelastometry (ROTEM) parameters in a user-centered and situation awareness–oriented method.

    Objective: This study aimed to evaluate the applicability of Visual Clot by examining its effects on users that are novices in viscoelastic-guided resuscitation.

    Methods: We conducted an investigator-initiated, international, multicenter study between September 16, 2020, and October 6, 2020, in 5 tertiary care hospitals in central Europe. We randomly recruited medical students and inexperienced resident physicians without significant prior exposure to viscoelastic testing. The 7 participants per center managed 9 different ROTEM outputs twice, once as standard ROTEM tracings and once as the corresponding Visual Clot. We randomly presented the 18 viscoelastic cases and asked the participants for their therapeutic decisions. We assessed the performance, diagnostic confidence, and perceived workload in managing the tasks using mixed statistical models and adjusted for possible confounding factors.

    Results: Analyzing a total of 630 results, we found that the participants solved more cases correctly (odds ratio [OR] 33.66, 95% CI 21.13-53.64; P<.001), exhibited more diagnostic confidence (OR 206.2, 95% CI 93.5-454.75; P<.001), and perceived less workload (coefficient –41.63; 95% CI –43.91 to –39.36; P<.001) using Visual Clot compared to using standard ROTEM tracings.

    Conclusions: This study emphasizes the practical benefit of presenting viscoelastic test results in a user-centered way. Visual Clot may allow inexperienced users to be involved in the decision-making process to treat bleeding-associated coagulopathy. The increased diagnostic confidence, diagnostic certainty, reduced workload, and positive user feedback associated with this visualization may promote the further adoption of viscoelastic methods in diverse health care settings.

    J Med Internet Res 2021;23(5):e27124

    doi:10.2196/27124

    Keywords

    avatar technology; coagulation management; hemostasis; intuitive design; rotational thromboelastometry; user-centered design; Visual Clot; testing 


    Since Hartert invented viscoelastic testing in 1948 [1] and its later clinical introduction in the 1980s [2,3], viscoelastic coagulation monitoring has become increasingly important in assessing acute bleeding in patients. To this end, several leading guidelines have proposed the use of viscoelastic-guided transfusion algorithms [4,5]. Compared to standard laboratory coagulation assays, rotational thromboelastometry (ROTEM) is faster [6,7], reduces inappropriate blood transfusions [8], and is more cost-efficient overall [9]. Further, previous studies showed that goal-directed viscoelastic hemostatic resuscitation improved patient outcomes in various different surgical specialties [9-13]. However, despite its evident importance, widespread acceptance, and increasing use, correctly interpreting ROTEM outputs remains a significant challenge for inexperienced physicians.

    Hence, our research department aimed to simplify viscoelastic test outputs by developing Visual Clot technology. This animated, 3D blood clot illustrates the raw ROTEM parameters in a user-centered and situation awareness–oriented method. Visual Clot displays different coagulation components as either present or absent based on empirical ROTEM cutoff values, without making the final decision for the user. Figure 1 shows the functionality of the Visual Clot technology. In a previous, prospective, dual-center study, Visual Clot helped anesthesia and intensive care physicians in Germany and Switzerland to improve their therapeutic decisions in coagulation management [14]. In a computer-based environment, the physicians were faster, exhibited more confidence, and experienced less workload in managing the hypothetical ROTEM outputs [14]. After their initial experiences, the same physicians considered Visual Clot as intuitive, easy to learn, and useful for the decision-making process [15].

    ‎
    Figure 1. Example of the animated blood clot, Visual Clot. Blood drops are shown in cases of deficient hemostatic components.
    View this figure

    In contrast to the previous research [14,15], this study aimed to evaluate the applicability of Visual Clot by examining its effects on users that are novices in viscoelastic-guided resuscitation. Without giving any instructions on analyzing or interpreting viscoelastic results, we tested the performance of using Visual Clot compared to that of using standard ROTEM readings. We hypothesized that these inexperienced users would solve more simulated bleeding scenarios correctly, with more diagnostic confidence and less perceived workload using this avatar technology. The results of this study may support the concept of Visual Clot technology and demonstrate its potential for involving inexperienced physicians in coagulation diagnostics and management. Moreover, this study promotes the further development of user-centered, situation awareness–oriented visualization technologies.


    This was an investigator-initiated, computer-based, within-subject, international multicenter study comparing standard ROTEM results with a corresponding animated viscoelastic visualization in simulated bleeding situations. We conducted this study between September 2020 and October 2020 in 5, large, tertiary care hospitals. The Cantonal Hospital Winterthur and University Hospital Zurich in Switzerland, the University Hospital Frankfurt and University Hospital Wuerzburg in Germany, and Hospital Clinic de Barcelona in Spain participated as different centers. The leading ethics committee in Zurich waived this study as it was not within the scope of the human research act (no. BASEC-Nr. Req-2020-00906). The other centers in Germany and Spain also waived ethical approval. All participants agreed in writing to the further use of obtained data for research purposes.

    Study Procedure

    For participants, we included medical students in their last or penultimate year of study and inexperienced resident physicians without significant prior exposure to viscoelastic coagulation testing. Further inclusion criteria were that they had never seen Visual Clot before and had no or minimal self-declared ROTEM skills. All participants worked in the respective hospitals. Participant selection was according to availability in clinical practice and inexperience in viscoelastic resuscitation. This subject population was completely different from that of the previous Visual Clot study [14], in which we investigated the technology’s effect on experienced anesthesia and intensive care physicians.

    We prepared 9 different bleeding scenarios that indicated specific coagulation disorders or a normal hemostatic state. In Multimedia Appendix 1, we provide a detailed list of all scenarios including their recommended therapeutic options. We showed each scenario twice, either as standard ROTEM readings or as the corresponding Visual Clot. Figure 2 shows an example of a standard ROTEM presentation with a corresponding Visual Clot, while Multimedia Appendix 2 shows the Visual Clot instructional video. We programmed Visual Clot according to the Zurich coagulation algorithm (Multimedia Appendix 3), which was validated in clinical practice [11]. When showing the bleeding scenarios as standard ROTEM readings, we provided the participants with the same coagulation algorithm’s normal values [11]. At the beginning of the study, we first asked the participants to fill out a demographic survey on personal data, such as age, gender, and educational level. We then randomly presented the 9 different coagulation scenarios twice without any instructions on how to interpret them. We used online software [16] to randomize the sequence of these 18 scenarios, providing each participant with a unique set. We showed all ROTEM readings or the respective Visual Clot on an Apple MacBook (Apple Inc). Using exclusively the respective viscoelastic results presentation, we asked the participants to choose their targeted therapeutic recommendations from a total of 6 given answer options. It was possible that multiple therapeutic interventions were necessary for sufficient treatment. We provided the answers as checkboxes in multiple choice form using the app iSurvey (Harvest Your Data) displayed on an iPad (Apple Inc) [17]. We encouraged all participants to submit their therapeutic recommendations as quickly and accurately as possible. After each scenario, the participants rated their diagnostic confidence and perceived workload in fulfilling the given task. At the end of the study session, we asked the participants to rate 4 statements on a 5-point Likert scale (from strongly disagree to strongly agree). These statements aimed to obtain a deeper understanding of the participants’ opinions about Visual Clot technology.

    ‎
    Figure 2. Example of a standard rotational thromboelastometry (ROTEM) presentation with a corresponding Visual Clot. Adhering to the coagulation algorithm used at the University Hospital Zurich, this hemostatic state represents fibrin deficiency.
    View this figure

    Outcomes

    The primary outcome of this study was performance, a binary outcome defined as correctly or incorrectly solved scenarios. In scenarios with multiple required therapeutic elements, we considered them to be correctly managed if the participant selected all the correct therapeutic options and no incorrect ones. As secondary outcomes of this study, we assessed the diagnostic confidence binary as unconfident or confident, and the perceived workload using the raw NASA (National Aeronautics and Space Administration) Task Load Index questionnaire. This subjective workload assessment tool has been validated in many different areas, including health care [18-23]. The raw questionnaire defines the total perceived workload as the arithmetic mean of 6 different, workload-associated subscores [23-25]. This study did not investigate the physical demand subscore, as our tasks were not physically challenging.

    Statistical Analysis

    As a first unadjusted analysis, we applied the McNemar test to compare the numbers of correctly and incorrectly solved cases with ROTEM and Visual Clot. We calculated mixed logistic regression models with a random intercept for each participant for the binary outcome variables, performance as correct or incorrect, and confidence as unconfident or confident. Further, we calculated a linear mixed model with a random intercept per participant for the continuous outcome regarding perceived workload as measured by the overall NASA Task Load Index scores. Apart from the variable denoting the respective viscoelastic modality (ROTEM vs Visual Clot), we adjusted all models for the center, gender, and job experience as confounders. We did not include the respective scenario as a confounding factor because the order was completely randomized for each participant and would not have consequently influenced the overall results.

    As we expected similar or greater performance differences as in the previously published Visual Clot study with experienced ROTEM users, we conducted an a priori sample size calculation based on these previous results [14]. We had reported a median of 44% (317/720) correct decisions for ROTEM and 100% (720/720) correct decisions using the Visual Clot. Based on the McNemar test and assuming the proportions of correct solutions were found in the pilot study, we calculated a sample size of 15 participants to achieve a significance level of 5% and a power of 90%. As the data collection was very cost- and time efficient, we decided to include 7 participants in each center to adjust the analyses for the different centers.

    We examined all data using R Version 3.6.2 (R Foundation for Statistical Computing) and created graphs using GraphPad Prism Version 9.0.0 (GraphPad Software Inc). Statistical significance was considered at a P value <.05.


    Between September 16, 2020, and October 6, 2020, each of the 5 study centers included 7 participants, amounting to 35 participants in total. We exposed each participant to 18 coagulation management scenarios, providing ROTEM results in 9 cases and Visual Clot in the remaining corresponding cases. No data were excluded in the final analysis. We investigated a total of 630 results, 315 per viscoelastic output modality. Regarding the participants, 49% (17 of 35) were female and none had previous contact with Visual Clot technology. Table 1 displays further study and participant characteristics. In Multimedia Appendix 4, we provide the full statistical analysis of this study.

    Table 1. Study and participant characteristics (N=35).
    CharacteristicValue
    Study centers, n5
    Age (years), median (IQR, range)28 (25-32, 24-36)
    Self-rated theoretical ROTEMa knowledgeb, median (IQR, range)0 (0-10, 0-20)
    Number of ROTEMs interpreted per yearc, median (IQR, range)0 (0-0, 0-6])
    Experience, n (%)

    		Penultimate year of medical studies3 (9%)

    		Last year of medical studies10 (29%)

    		First year resident physician19 (54%)

    		Second year resident physician2 (6%)

    		Third year resident physician1 (3%)

    aROTEM: rotational thromboelastometry.

    bThe self-rated ROTEM knowledge scale ranges from 0 (very low) to 100 (very high).

    cThe number of ROTEM interpreted per year ranges from 0 (very low) to 100 (very high).

    Regarding our primary outcome, binary performance, the mixed logistic regression provided very strong evidence for a difference between the 2 viscoelastic modalities. The odds of correctly solving the case were about 33 times as high when using Visual Clot compared to when using conventional ROTEM tracings (odds ratio [OR] 33.66, 95% CI 21.13-53.64; P<.001). In Figure 3, we illustrate the unadjusted comparison between the viscoelastic modalities using the McNemar test. There was no significant difference in performance between the genders (OR 0.63, 95% CI 0.38-1.03; P=.06) or the different study centers (all P values >.05; Cantonal Hospital Winterthur P=.41; University Hospital Frankfurt P=.65; University Hospital Würzburg P=.60; Hospital Clinic de Barcelona P=.69). The job experience, represented by the educational level, did not differ significantly in terms of performance either (all P>.05; last year of studies P=.59; first year of residency P=.64; second year of residency P=.96; third year of residency P=.75).

    ‎
    Figure 3. Donut charts displaying the binary performance defined as the number of correctly and incorrectly solved cases using standard ROTEM results (left donut) or Visual Clot (right donut). The unadjusted analysis using the McNemar test showed very strong evidence for a difference between the 2 viscoelastic modalities (P<.001). N=315 per viscoelastic modality. ROTEM: rotational thromboelastometry.
    View this figure

    For the analysis of the participants’ diagnostic confidence ratings, our results showed very strong evidence in favor of Visual Clot. The odds of being diagnostically confident were about 200 times higher than when using ROTEM printouts (OR 206.2, 95% CI 93.5-454.75; P<.001). Regarding this outcome, no significant differences were not found for the ratings of genders (P=.96), centers (Cantonal Hospital Winterthur P=.10; University Hospital Frankfurt P=.42; University Hospital Würzburg P=.12; Hospital Clinic de Barcelona P=.08), or job experiences (last year of studies P=.73; first year of residency P=.97; second year of residency P=.32; third year of residency P=.18).

    Finally, mixed linear regression yielded very strong evidence for a difference between the 2 viscoelastic modalities regarding perceived workload ratings. The overall raw NASA Task Load Index scores were on average about 40 points lower if Visual Clot was used, with a coefficient of –41.63 (95% CI –43.91 to –39.36l; P<.001). In Figure 4, we provide the analysis of the overall perceived workload and its subscores as boxplots.

    ‎
    Figure 4. Boxplots representing the analysis of the participants’ perceived workload after using the respective viscoelastic modality. The overall workload and its subscores were evaluated using the modified, raw NASA Task Load Index questionnaire. Low workload scores correspond to low perceived workload. The box represents the first and third quartiles, with the line indicating the median. N=315 per viscoelastic modality. The whiskers represent the 5th and 95th percentile. NASA: National Aeronautics and Space Administration; ROTEM: rotational thromboelastometry; TLX: Task Load Index.
    View this figure

    Figure 5 illustrates the 4 statements regarding the participants’ opinions about Visual Clot and the results of their assessment. All following results are presented as agree (strongly agree or agree), neutral, or disagree (strongly disagree or disagree). Out of 35 participants, 34 (97%) agreed that the interpretation of Visual Clot was simple and 33 (94%) agreed that Visual Clot helped them feel better prepared to interpret viscoelastic test results. Moreover, all participants (35/34, 100%) agreed that they would use Visual Clot technology in a real bleeding situation, and 28 of 35 (80%) would want their treating physician to use Visual Clot if they were experiencing acute bleeding.

    ‎
    Figure 5. Graphical presentation of the participants’ rated survey statements as 10 x 10 parts of whole dot plots. Results are presented as median and IQR. N=35 in each rated statement.
    View this figure

    This study compared 315 within-subject therapeutic decisions of 35 participants who were inexperienced in viscoelastic-guided hemostatic resuscitation and who did not receive any instructions in interpreting the result printouts. Using the animated blood clot, Visual Clot, the participants interpreted more viscoelastic test results correctly than when using the standard ROTEM tracings. Moreover, they felt more confident in their therapeutic decision and perceived less workload using Visual Clot technology. None of the participants had any previous contact with the avatar-based presentation before attending the study. We examined the results using mixed models and adjusted for different confounders.

    Analysis of the participants’ performance revealed that the participants selected the correct therapeutic options in over 90% (282 of 315) of the Visual Clot cases, while they made the same choices in only about 22% (69 of 315) using the corresponding ROTEM. This resulted in a relative risk ratio of about 4 and 33-times higher odds of correctly interpreting the test results when using Visual Clot technology. This positive effect persisted both across the different study centers and across the participants’ educational levels. These results are even more pronounced than those in the primary Visual Clot study, where we investigated viscoelastic-experienced physicians [14]. There, the odds of a correct therapeutic decision using the avatar technology were around 22 times increased. Visual Clot appears to have significant positive effects even without any instructions for interpretation and even for users with minimal viscoelastic-guided resuscitation knowledge.

    Indeed, perceived usability is a prerequisite for user acceptance of innovative technologies [26]. Both users’ intuition (ie, unconscious reasoning) and the technology’s characteristics, such as shape, color, and presentation of its features, directly influence how people interact with unfamiliar devices [27]. Out of 35 participants, 33 agreed that interpreting Visual Clot was simple. In a previous study investigating physicians’ opinions on Visual Clot, the participants found the avatar useful, easy to learn, and intuitive [15]. Likewise, they mentioned that it allowed a faster overview of complex coagulation situations [15]. Visual Clot dichotomizes a disease state or the presence of substrates necessary for optimal clotting based on pathological, self-determinable ROTEM thresholds of an integrated, adequate coagulation algorithm. Indicating essential factors as present or absent and demonstrating them in a playful, user-centered manner reduces information complexity, leading to increased cognitive reception [28]. This binary illustration facilitates decision-making, may help to enforce local coagulation guidelines, and can reduce uncertainty in ambiguous situations through its clear presentation. On the other hand, the standard ROTEM printouts require that the physician understands the numerical results and the data-driven tracings in the context of the coagulation algorithm’s normal values to form a mental model of the current bleeding situation. This seems to cause more insecurity and incorrect therapeutic decisions that may affect the treatment of patients.

    Our analysis showed a significant reduction of participants’ perceived workload scores when using Visual Clot than when using traditional ROTEM results. Furthermore, our analyses showed 200 times higher odds of being confident when using Visual Clot. Again, these results are more pronounced in these novice users than in the experienced anesthesiologists and intensive care specialists of the previous Visual Clot study [14]. In large hospitals, inexperienced resident physicians may be confronted with acute bleeding situations and coagulopathy even before they possess sufficient medical training in this field. This may cause a high-pressure working environment, which is known to degrade performance [29] and lead to fatigue from perceived work overload [30]. Staff well-being directly influences the prevalence of medical errors [31], and confidence positively affects performance [32]. We should strive to minimize workload and promote the staff’s diagnostic confidence to ensure better patient outcomes.

    In this study, all of the participants agreed that they would use Visual Clot in a real bleeding situation. Further, 80% (28 of 35) agreed that they would want their treating physician to use this technology if they were experiencing acute bleeding themselves. It seems that the participants trust the technology and accept its application. However, we designed Visual Clot to complement the quantitative ROTEM data as a graphical representation, rather than to replace them.

    This study had several limitations. Using a computer-based simulation design, we generated ROTEM printouts and corresponding Visual Clot animations that were clearly attributable to a coagulation disorder or normal hemostatic state. Viscoelastic results in real clinical bleeding may be less distinctive. Future studies are needed to confirm the results of this study in real bleeding-associated coagulopathic situations. However, simulation studies are considered an optimal environment to train and assess new methods [33]. Further, we performed this study in tertiary care hospitals in central Europe, and the results may differ elsewhere in the world. However, we consider this unlikely as all participants were novices to viscoelastic-guided management and therefore did not yet benefit from those large facilities’ medical training.

    This study also possesses several strengths. The analyses were adequately powered due to the a priori sample size calculation. Furthermore, the within-subject comparisons may largely rule out alternative explanations for our findings. The multicenter design and balanced participant selection across the 5 study centers minimized selection bias.

    This study emphasizes the relevance of designing viscoelastic test results in a user-centered, situation awareness–oriented method. The avatar-based blood clot presentation enabled users with no or minimal knowledge in viscoelastic-guided coagulation management and without any prior training to solve almost all coagulation scenarios correctly. It further improved the participants’ diagnostic confidence and reduced their perceived workload. Straightforward and confident interpretation may benefit new and experienced users in a wide range of treatment settings and promote the adoption of viscoelastic methods. The most significant benefits will likely be gained by inexperienced users and users who need to make quick decisions in stressful situations, such as on the battlefield, in spaceflight, or in the emergency room. The potential benefits of this technology and the emerging use of viscoelastic testing with its evidence-based importance justifies further investigation of Visual Clot in real clinical bleeding, with the ultimate aim of improving patient outcomes.

    Acknowledgments

    The authors are sincerely grateful to the staff of all participating study centers for their support in conducting this study. Further, the authors thank all study participants for their time and effort. Instrumentation Laboratory Company/Werfen Corporation had no role in the design of the study, the data collection, analysis and interpretation of data, or the decision to submit this article for publication.

    This study was funded by the Institute of Anesthesiology at the University Hospital Zurich, Zurich, Switzerland.

    Authors' Contributions

    SS, TRR, JB, DRS, CBN, and DWT designed the study.

    SS, TRR, MTG, PM, JH, KZ, ER, MLB, FJR, FP, DRS, CBN, and DWT contributed to the acquisition or interpretation of data.

    SS, TRR, JB, DRS, and DWT performed data analysis.

    SS, TRR, JB, MTG, PM, JH, KZ, ER, MLB, FJR, FP, DRS, CBN, and DWT drafted the manuscript or provided critical revision.

    All authors gave approval of the final version of the manuscript for publication.

    All authors agree to be accountable for all aspects of the work, including the accuracy and integrity of the work.

    Conflicts of Interest

    The academic department of DRS is receiving grant support from the Swiss National Science Foundation, Berne, Switzerland; the Swiss Society of Anesthesiology and Reanimation (SGAR), Berne, Switzerland; the Swiss Foundation for Anesthesia Research, Zurich, Switzerland; and Vifor SA, Villars-sur-Glâne, Switzerland. DRS is cochair of the ABC-Trauma Faculty, sponsored by unrestricted educational grants from Novo Nordisk Health Care AG, Zurich, Switzerland; CSL Behring GmbH, Marburg, Germany; LFB Biomédicaments, Courtaboeuf Cedex, Franc; and Octapharma AG, Lachen, Switzerland. DRS received honoraria/travel support for consulting or lecturing from Danube University of Krems, Austria; the US Department of Defense, Washington, USA; the European Society of Anaesthesiology, Brussels, Belgium; the Korean Society for Patient Blood Management, Seoul, Korea; the Korean Society of Anesthesiologists, Seoul, Korea; the Network for the Advancement of Patient Blood Management, Haemostasis and Thrombosis, Paris, France; Baxter AG, Volketswil, Switzerland; Baxter S.p.A., Roma, Italy; Bayer AG, Zürich, Switzerland; Bayer Pharma AG, Berlin, Germany; B. Braun Melsungen AG, Melsungen, Germany; Boehringer Ingelheim GmbH, Basel, Switzerland; Bristol-Myers-Squibb, Rueil-Malmaison Cedex, France and Baar, Switzerland; CSL Behring GmbH, Hattersheim am Main, Germany and Berne, Switzerland; Celgene International II Sàrl, Couvet, Switzerland; Curacyte AG, Munich, Germany; Daiichi Sankyo AG, Thalwil, Switzerland; GlaxoSmithKline GmbH & Co. KG, Hamburg, Germany; Haemonetics, Braintree, MA, USA; Instrumentation Laboratory (Werfen), Bedford, MA, USA; LFB Biomédicaments, Courtaboeuf Cedex, France; Merck Sharp & Dohme, Kenilworth, New Jersey, USA; Octapharma AG, Lachen, Switzerland; Organon AG, Pfäffikon/SZ, Switzerland; PAION Deutschland GmbH, Aachen, Germany; Pharmacosmos A/S, Holbaek, Denmark; Photonics Healthcare BV, Utrecht, Netherlands; Pierre Fabre Pharma, Alschwil, Switzerland; Roche Diagnostics International Ltd, Reinach, Switzerland; Roche Pharma AG, Reinach, Switzerland, Sarstedt AG & Co, Sevelen, Switzerland and Nümbrecht, Germany; Schering-Plough International, Inc, Kenilworth, New Jersey, USA; Tem International GmbH, Munich, Germany; Verum Diagnostica GmbH, Munich, Germany; Vifor Pharma, Munich, Germany, Vienna, Austria, and Villars-sur-Glâne, Switzerland; Vifor (International) AG, St. Gallen, Switzerland; and Zuellig Pharma Holdings, Singapore. DRS, CBN, and DWT are the designated inventors of Visual Clot technology, for which the University of Zurich holds various patents and trademarks. The University of Zurich signed a letter of intent for a cooperation and licensing agreement with Instrumentation Laboratory Company/Werfen Corporation, Bedford, MA, USA and Barcelona, Spain. Under this and future agreements, DRS, CBN, and DWT may receive royalties. DRS, CBN, and DWT received travel support for consulting Instrumentation Laboratory, Bedford, MA, USA. CBN and DWT are designated inventors of Visual Patient technology, for which the University of Zurich holds various patents and trademarks. There are cooperation and licensing agreements with Philips Medizin Systeme Böblingen GmbH, Böblingen, Germany; Koninklijke Philips NV, Amsterdam, The Netherlands; and Philips Research/Philips Electronics Nederland BV, Eindhoven, The Netherlands. Under these agreements, CBN and DWT may receive royalties. MTG received honoraria/travel support for consulting or lecturing from Vifor Pharma of Munich, Germany; Vienna, Austria; and Villars-sur-Glâne, Switzerland. FJR received funding from HemoSonics LLC, Charlottesville, VA, USA. FJR received honoraria/travel support for consulting or lecturing from Pharma-Consult-Petersohn, Köln, Nordrhein-Westfalen, Germany; Boehringer Ingelheim, Ingelheim am Rhein, Rheinland-Pfalz, Germany; Keller Medical GmbH, Bad Soden/Ts, Hessen, Germany; and HELIOS Klinikum Krefeld, Krefeld, Germany. PM received research grants from the German Research Foundation (no. ME 3559/1-1 and ME 3559/3-1), BMBF (no. 01KG1815), the International Anesthesia Research Society, the German Society of Anaesthesiology and Intensive Care Medicine, and the European Society of Anaesthesiology; and from B. Braun Melsungen, CSL Behring, Fresenius Kabi, and Vifor Pharma for the implementation of Frankfurt‘s Patient Blood Management Program; and from Pfizer and Dr. F. Köhler Chemie GmbH for an investigator-initiated trial. PM further received honoraria for scientific lectures from Abbott GmbH & Co KG, Aesculap Academy, B. Braun Melsungen, Biotest AG, Vifor Pharma, Ferring, CSL Behring, German Red Cross/ Institute of Transfusion Medicine, HCCM Consulting GmbH, Heinen & Löwenstein, and Pharmacosmos, Siemens Healthcare; along with funding from Aktionsbündnis Patientensicherheit, the European Society of Anaesthesiology, Lohfert-Stiftung AG, Masimo Patient Safety Foundation, and MSD-Gesundheitspreis. KZ received research grants from the European Union Horizon 2020 - Research and Innovation Framework Programme (ENVISION); Aesculap Akademie GmbH, Tuttlingen, Germany; Affinites Sante, Paris, France; Ashai Kasai Pharma, Waltham, MA, USA; B. Braun AG, Melsungen, Germany; B. Braun Avitum AG, Melsungen, Germany; Bayer AG, Leverkusen, Germany; Biotest AG; Dreieich, Germany; Christian Doppler Stiftung, Wien, Österreich; CSL Behring GmbH, Marburg, Germany; Cyto Sorbents GmbH, Berlin, Germany; Edward Lifescience Corporation, Unterschleißheim, Germany; Executive Insight AG, Baar, Switzerland; Fresenius Kabi GmbH, Bad Homburg, Germany; Fresenius Medical Care, Bad Homburg, Germany; Haemonetics Corporation, Braintree, MA, USA; Hartmannbund Landesverband, Berlin, Germany; Health Advances GmbH, Zug, Switzerland; Löwenstein Medical GmbH, Bad Ems, Germany; Hexal AG, Holzkirchen, Germany; INC Research, München, Germany; Johnson & Johnson, Norderstedt, Germany; Josef Gassner, Weißenkirchen im Attergau, Austria; Maquet GmbH, Rastatt, Germany; Markus Lücke Kongress Organisation, Walsrode, Germany; Masimo International, Neuchatel, Switzerland; med Update GmbH, Wiesbaden, Germany; Medizin & Markt Gesundheitsnetzwerk, München, Germany; MSD Sharp & Dohme GmbH, Haar, Germany; Nordic Group, Hoofddorp, the Netherlands; Nordic Pharma, Ismaning, Germany; Novo Nordisk Pharma GmbH, Mainz, Germany; Pfizer Pharma GmbH, Berlin, Germany; Pharmacosmos, Wiesbaden, Germany; Ratiopharm GmbH, Ulm, Germany; Salvia Medical GmbH, Kronberg, Germany; Schering Stiftung, Berlin, Germany; Schöchl Medical Österreich, Mattsee, Austria; Serumwerke, Bernburg, Germany; Verlag für Printmedien und PR, Forum Sanitas, Leopoldshöhe, Germany; Vifor Pharma GmbH, München, Germany; Wellington, München, Germany; and Werfen, München, Germany. The other authors have no conflicts of interest to declare.

    ‎
    Multimedia Appendix 1

    All scenarios including their recommended therapeutic options.

    PDF File (Adobe PDF File), 2082 KB
    ‎
    Multimedia Appendix 2

    Instructional video showing the functionality of Visual Clot.

    MP4 File (MP4 Video), 7648 KB
    ‎
    Multimedia Appendix 3

    Visual Clot algorithm according to the local rotational thromboelastometry thresholds at the University Hospital Zurich.

    PDF File (Adobe PDF File), 152 KB
    ‎
    Multimedia Appendix 4

    Full statistical analysis of this study.

    PDF File (Adobe PDF File), 131 KB
    1. Hartert H. Blutgerinnungsstudien mit der thrombelastographie, einem neuen untersuchungsverfahren. Klin Wochenschr 1948 Oct;26(37-38):577-583. [CrossRef]
    2. Kang YG, Martin DJ, Marquez J, Lewis JH, Bontempo FA, Shaw BW, et al. Intraoperative changes in blood coagulation and thrombelastographic monitoring in liver transplantation. Anesth Analg 1985 Sep;64(9):888-896 [FREE Full text] [Medline]
    3. Shen L, Tabaie S, Ivascu N. Viscoelastic testing inside and beyond the operating room. J Thorac Dis 2017 Apr;9(Suppl 4):S299-S308 [FREE Full text] [CrossRef] [Medline]
    4. Kozek-Langenecker SA, Ahmed AB, Afshari A, Albaladejo P, Aldecoa C, Barauskas G, et al. Management of severe perioperative bleeding: guidelines from the European Society of Anaesthesiology: First update 2016. Eur J Anaesthesiol 2017 Jun;34(6):332-395. [CrossRef] [Medline]
    5. Spahn DR, Bouillon B, Cerny V, Duranteau J, Filipescu D, Hunt BJ, et al. The European guideline on management of major bleeding and coagulopathy following trauma: fifth edition. Crit Care 2019 Mar 27;23(1):98 [FREE Full text] [CrossRef] [Medline]
    6. Haas T, Spielmann N, Mauch J, Madjdpour C, Speer O, Schmugge M, et al. Comparison of thromboelastometry (ROTEM®) with standard plasmatic coagulation testing in paediatric surgery. Br J Anaesth 2012 Jan;108(1):36-41 [FREE Full text] [CrossRef] [Medline]
    7. Haas T, Fries D, Tanaka KA, Asmis L, Curry NS, Schöchl H. Usefulness of standard plasma coagulation tests in the management of perioperative coagulopathic bleeding: is there any evidence? Br J Anaesth 2015 Feb;114(2):217-224 [FREE Full text] [CrossRef] [Medline]
    8. Ak K, Isbir CS, Tetik S, Atalan N, Tekeli A, Aljodi M, et al. Thromboelastography-based transfusion algorithm reduces blood product use after elective CABG: a prospective randomized study. J Card Surg 2009;24(4):404-410. [CrossRef] [Medline]
    9. Weber CF, Görlinger K, Meininger D, Herrmann E, Bingold T, Moritz A, et al. Point-of-Care testing. Anesthesiology 2012;117(3):531-547. [CrossRef]
    10. Gonzalez E, Moore EE, Moore HB, Chapman MP, Chin TL, Ghasabyan A, et al. Goal-directed hemostatic resuscitation of trauma-induced coagulopathy: a pragmatic randomized clinical trial comparing a viscoelastic assay to conventional coagulation assays. Ann Surg 2016 Jun;263(6):1051-1059 [FREE Full text] [CrossRef] [Medline]
    11. Stein P, Kaserer A, Sprengel K, Wanner GA, Seifert B, Theusinger OM, et al. Change of transfusion and treatment paradigm in major trauma patients. Anaesthesia 2017 Nov;72(11):1317-1326 [FREE Full text] [CrossRef] [Medline]
    12. Mallaiah S, Barclay P, Harrod I, Chevannes C, Bhalla A. Introduction of an algorithm for ROTEM-guided fibrinogen concentrate administration in major obstetric haemorrhage. Anaesthesia 2014 Oct 07;70(2):166-175. [CrossRef]
    13. Deppe A, Weber C, Zimmermann J, Kuhn EW, Slottosch I, Liakopoulos OJ, et al. Point-of-care thromboelastography/thromboelastometry-based coagulation management in cardiac surgery: a meta-analysis of 8332 patients. J Surg Res 2016 Jun 15;203(2):424-433. [CrossRef] [Medline]
    14. Rössler J, Meybohm P, Spahn DR, Zacharowski K, Braun J, Nöthiger CB, et al. Improving decision making through presentation of viscoelastic tests as a 3D animated blood clot: the Visual Clot. Anaesthesia 2020 Feb 06:14985. [CrossRef] [Medline]
    15. Roche TR, Said S, Rössler J, Gozdzik M, Meybohm P, Zacharowski K, et al. Physicians' perceptions of a situation awareness-oriented visualization technology for viscoelastic blood coagulation management (visual clot): mixed methods study. JMIR Serious Games 2020 Dec 04;8(4):e19036 [FREE Full text] [CrossRef] [Medline]
    16. Urbaniak GC, Plous S. Research randomizer. Social Psychology Network.   URL: http://www.randomizer.org [accessed 2020-08-01]
    17. Tscholl DW, Weiss M, Spahn DR, Noethiger CB. How to conduct multimethod field studies in the operating room: the iPad combined with a survey app as a valid and reliable data collection tool. JMIR Res Protoc 2016 Jan 05;5(1):e4 [FREE Full text] [CrossRef] [Medline]
    18. Grier RA. How high is high? A meta-analysis of NASA-TLX global workload scores. 2015 Presented at: Proceedings of the Human Factors and Ergonomics Society Annual Meeting; 2015 Oct 26-30; Los Angeles. [CrossRef]
    19. Lowndes BR, Forsyth KL, Blocker RC, Dean PG, Truty MJ, Heller SF, et al. NASA-TLX assessment of surgeon workload variation across specialties. Ann Surg 2020 Apr;271(4):686-692. [CrossRef] [Medline]
    20. Qiao F, Zhang R, Yu L. using NASA-Task Load Index to assess drivers' workload on freeway guide sign structures. In: American Society of Civil Engineers. 2011 Presented at: 11th International Conference of Chinese Transportation Professional (ICCTP); Aug 14 2011; Nanjing , China p. 53. [CrossRef]
    21. Ruiz-Rabelo JF, Navarro-Rodriguez E, Di-Stasi LL, Diaz-Jimenez N, Cabrera-Bermon J, Diaz-Iglesias C, et al. Validation of the NASA-TLX score in ongoing assessment of mental workload during a laparoscopic learning curve in bariatric surgery. Obes Surg 2015 Dec;25(12):2451-2456. [CrossRef] [Medline]
    22. Yurko YY, Scerbo MW, Prabhu AS, Acker CE, Stefanidis D. Higher mental workload is associated with poorer laparoscopic performance as measured by the NASA-TLX tool. Simul Healthc 2010 Oct;5(5):267-271. [CrossRef] [Medline]
    23. Said S, Gozdzik M, Roche TR, Braun J, Rössler J, Kaserer A, et al. Validation of the raw national aeronautics and space administration Task Load Index (NASA-TLX) questionnaire to assess perceived workload in patient monitoring tasks: pooled analysis study using mixed models. J Med Internet Res 2020 Sep 07;22(9):e19472 [FREE Full text] [CrossRef] [Medline]
    24. Hendy KC, Hamilton KM, Landry LN. Measuring subjective workload: When is one scale better than many? Hum Factors 2016 Nov 23;35(4):579-601. [CrossRef]
    25. Nygren TE. Psychometric properties of subjective workload measurement techniques: implications for their use in the assessment of perceived mental workload. Hum Factors 2016 Nov 23;33(1):17-33. [CrossRef]
    26. Holden H, Rada R. Understanding the influence of perceived usability and technology self-efficacy on teachers’ technology acceptance. Journal of Research on Technology in Education 2011 Jun;43(4):343-367. [CrossRef]
    27. Blackler A, Popovic V, Mahar D. Investigating users' intuitive interaction with complex artefacts. Appl Ergon 2010 Jan;41(1):72-92. [CrossRef] [Medline]
    28. Agarwal R, Karahanna E. Time flies when you're having fun: cognitive absorption and beliefs about information technology usage. MIS Quarterly 2000 Dec;24(4):665. [CrossRef]
    29. Grier RA, Warm JS, Dember WN, Matthews G, Galinsky TL, Parasuraman R. The vigilance decrement reflects limitations in effortful attention, not mindlessness. Hum Factors 2003;45(3):349-359. [CrossRef] [Medline]
    30. Rose DM, Seidler A, Nübling M, Latza U, Brähler E, Klein EM, et al. Associations of fatigue to work-related stress, mental and physical health in an employed community sample. BMC Psychiatry 2017 May 5;17(1):50. [CrossRef]
    31. Landrigan CP, Rothschild JM, Cronin JW, Kaushal R, Burdick E, Katz JT, et al. Effect of reducing interns' work hours on serious medical errors in intensive care units. N Engl J Med 2004 Oct 28;351(18):1838-1848. [CrossRef] [Medline]
    32. Endsley M, Jones W. Wright-Patterson Air Force Base, OH: United States Airforce. 1997.   URL: https:/​/www.​researchgate.net/​publication/​235167825_Situation_Awareness_Information_Dominance_Information_Warfare [accessed 2020-12-03]
    33. Moorthy K, Vincent C, Darzi A. Simulation based training. BMJ 2005 Mar 03;330(7490):493-494. [CrossRef]

    ‎
    NASA: National Aeronautics and Space Administration
    OR: odds ratio
    ROTEM: rotational thromboelastometry

    Edited by G Eysenbach; submitted 12.01.21; peer-reviewed by R Marshall; comments to author 02.02.21; revised version received 03.02.21; accepted 11.04.21; published 03.05.21

    Copyright

    ©Sadiq Said, Tadzio Raoul Roche, Julia Braun, Micheal Thomas Ganter, Patrick Meybohm, Johannes Herrmann, Kai Zacharowski, Florian Jürgen Raimann, Florian Piekarski, Eva Rivas, Manuel López-Baamonde, Donat R Spahn, Christoph Beat Nöthiger, David Werner Tscholl. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 03.05.2021.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.