Physicians’ Perceptions of a Situation Awareness–Oriented Visualization Technology for Viscoelastic Blood Coagulation Management (Visual Clot): Mixed Methods Study

Background Viscoelastic tests enable a time-efficient analysis of coagulation properties. An important limitation of viscoelastic tests is the complicated presentation of their results in the form of abstract graphs with a multitude of numbers. We developed Visual Clot to simplify the interpretation of presented clotting information. This visualization technology applies user-centered design principles to create an animated model of a blood clot during the hemostatic cascade. In a previous simulation study, we found Visual Clot to double diagnostic accuracy, reduce time to decision making and perceived workload, and improve care providers’ confidence. Objective This study aimed to investigate the opinions of physicians on Visual Clot technology. It further aimed to assess its strengths, limitations, and clinical applicability as a support tool for coagulation management. Methods This was a researcher-initiated, international, double-center, mixed qualitative-quantitative study that included the anesthesiologists and intensive care physicians who participated in the previous Visual Clot study. After the participants solved six coagulation scenarios using Visual Clot, we questioned them about the perceived pros and cons of this new tool. Employing qualitative research methods, we identified recurring answer patterns, and derived major topics and subthemes through inductive coding. Based on them, we defined six statements. The study participants later rated their agreement to these statements on five-point Likert scales in an online survey, which represented the quantitative part of this study. Results A total of 60 physicians participated in the primary Visual Clot study. Among these, 36 gave an interview and 42 completed the online survey. In total, eight different major topics were derived from the interview field note responses. The three most common topics were “positive design features” (29/36, 81%), “facilitates decision making” (17/36, 47%), and “quantification not made” (17/36, 47%). In the online survey, 93% (39/42) agreed to the statement that Visual Clot is intuitive and easy to learn. Moreover, 90% (38/42) of the participants agreed that they would like the standard result and Visual Clot displayed on the screen side by side. Furthermore, 86% (36/42) indicated that Visual Clot allows them to deal with complex coagulation situations more quickly. Conclusions A group of anesthesia and intensive care physicians from two university hospitals in central Europe considered Visual Clot technology to be intuitive, easy to learn, and useful for decision making in situations of active bleeding. From the responses of these possible future users, Visual Clot appears to constitute an efficient and well-accepted way to streamline the decision-making process in viscoelastic test–based coagulation management.


Introduction
For optimal perioperative bleeding management, a quick and reliable assessment of the patient's blood coagulation function is of utmost importance [1][2][3]. As laboratory coagulation testing can take more than an hour [4] to produce results, viscoelastic point-of-care devices such as rotational thromboelastometry (ROTEM, Instrumentation Laboratory/Werfen Bedford, MA, USA) and thromboelastography (TEG, Haemonetics, Boston, MA, USA) play an important role in guiding hemostatic interventions in a timeefficient manner [5]. Previous studies showed that the utilization of viscoelastic testing can improve patient outcome in bleeding trauma patients [3,[6][7][8][9]. Hence, the European guidelines on trauma management recommend viscoelastic testing to enable goal-directed bleeding management [10].
However, the abstract presentation of the viscoelastic test results complicates its handling in clinical routine. This in turn shows the demand for a tool that facilitates the interpretation of viscoelastic readings. To serve this apparent need, Visual Clot technology [15] was developed. Visual Clot is an alternative situation awareness-oriented visualization technology created adhering to the principles of user-centered design [16]. Instead of abstract shapes and a multitude of numbers, as in the conventional presentation of the ROTEM results, Visual Clot technology displays a three-dimensional, animated model of the blood clot that corresponds to the real phenomena one would see when looking at the blood clot through an electron microscope. In an international, dual center study [15], Visual Clot technology enabled the participating physicians to make twice as many correct diagnoses, quicker, with improved diagnostic confidence, and with reduced perceived cognitive workload compared to standard viscoelastic test results.
This study aimed to explore and capture the opinions of the 60 anesthesiologists and intensive care specialists of the previous Visual Clot study [15] on this new visualization technology. Its results will help define the strengths and limitations of this visual decision aid. Further, this study examines the applicability of Visual Clot as an additional tool for coagulation management from the viewpoint of its potential future users.

Methods
The leading ethics committee, the Cantonal Ethics Committee of the Canton of Zurich in Switzerland, reviewed the study protocol and issued a declaration of no objection (Business Management System for Ethics Committees Number 2018-00933). We obtained written informed consent from each participant for the use of their data.

Study design
We conducted this study at the University Hospital Frankfurt (UKF) in Germany and the University Hospital Zurich (USZ) in Switzerland. We included anesthesiologists and intensive care physicians from these two hospitals. Both study centers routinely use ROTEM-guided hemostatic resuscitation.
This study is a researcher-initiated, international, dual center, mixed qualitative quantitative study about the opinions of anesthesia and intensive care physicians regarding Visual Clot technology. The methods employed included interviewing physicians after their first contact with Visual Clot technology and jointly generating field notes based on their answers. Further, we created statements derived from these field note responses and asked the same group of physicians to rate the statements in an online survey.

Previous Visual Clot study and participant interviews
We held interviews for the qualitative part of this study at the end of the data collection sessions of the previously published Visual Clot study. In mentioned study, the participating physicians evaluated 12 coagulation scenarios in randomized order. They solved each scenario twice, once only observing Visual Clot and once only using the conventional viscoelastic test results. Before the testing took place, the participants received a brief individual training and subsequently solved six scenarios using Visual Clot In an undisturbed environment, we asked the physicians about the advantages and disadvantages of Visual Clot. There was no further guidance by the investigators, who all had previous experience with qualitative research. While the physicians were encouraged to freely verbalize their thoughts, the data collector simultaneously took field notes in a Microsoft Word (Microsoft Corporation, Redmond, WA, USA) document. After the interview session, the field notes were presented to the participants. They were asked whether they agreed with them and were encouraged to determine final adjustments and changes.

Qualitative analysis
We used a template approach in the qualitative analysis [17]. This involves grouping topics using a coding template. The topics are often predefined in advance. During further data analysis, new topics are added or existing ones are revised [17]. First we translated the original answers from German to English using Deepl.com (DeepL GmbH, Cologne, Germany). All translated field notes are available as Multimedia Appendix 2. After translation, we used Microsoft Word to identify the most common words and created a tag cloud ( Figure 2) as its quantitative graphic illustration using Wordle.net. We ignored common English words such as "the, and, to", and unified word groups with the same word root. The two physicians and study authors TRR and DWT were both involved in the qualitative analysis of the field notes. TRR is a resident anesthesiologist who did not partake in the interviewing process. Study author DWT is a senior anesthesiologist with previous experience in qualitative and patient safety research. Adhering to the criteria for reporting qualitative research [18], TRR and DWT developed a coding template as a rating system, using both the word count and inductive coding based on recurring topics in the field note responses. We modified the initial coding template by successive reading, coding of the data and discussions with other authors. After multiple data coding, we agreed upon a final template. The two study authors mentioned above then independently rated all field notes using illustrated final coding template, which we provide in Figure 3. Inter-rater reliability was calculated to investigate the consistency of the implementation of the rating system. In cases of disagreement between these two examiners, a final code for the respective field note response was jointly determined.

Quantitative analysis
Literature provides several arguments why qualitative data could be combined with a quantitative analysis. One of the reasons for this is that quantitative data can help to generalize and confirm specific observations made in a qualitative analysis [19]. To further examine the qualitative research part of this study and Visual Clot applicability, we performed a quantitative assessment of the derived statements using a Web-based online survey. We defined six, partially generalized statements and asked the participants to rate them on five point Likert scales ranging from "strongly disagree" to "strongly agree".
Out of all six statements, four directly refer to the previously identified major topics. We specifically added two additional statements to the questioning which we considered necessary. These two aimed to get at a deeper understanding of Visual Clot technology applicability. Using SurveyMonkey (SVMK Inc., San Matteo, CA, USA), we created the open online survey and tested its usability before sending the study link via e-mail to all physicians who participated in the previous Visual Clot study and still worked at the respective institutions. We informed the participants that the survey takes about one minute to fill out. Participation was voluntary, no incentives were offered and no personal identifying information was collected. The participants were able to review and change their answers before completion. A single reminder to complete the survey was sent after two weeks. Multimedia Appendix 3 displays the translated wording of the survey invitation announcement as well as the reminder mail.
We completed the data collection one week after the reminder period expired.

Statistical analysis
We provide the quantitative analysis of the online survey as median and interquartile ranges of all statements. Using the Wilcoxon signed-rank test, we evaluated the difference between the median for each of the statements and the neutral answer. We considered P <.05 to indicate statistical significance.
The qualitative part of this study aimed to identify the pros and cons using the new visualization technology. With the evaluated statements of the online survey, we aimed to further quantify the agreeor disagreement. This procedure additionally increases the relevance of the statements made.

Data Sharing Statement
The translated interview field notes are available as Multimedia Appendix 2. We report all other data in this manuscript.

Study and participant characteristics
This study includes the same participants as the previous Visual Clot study [15]. A total of 60 anesthesia and intensive care physicians participated, half of them working at USZ and the other half at UKF. After collecting data from the first 24 participants of the Visual Clot study, we decided to conduct systematic interviews as we found that the participants gave very informative feedback on the technology. We interviewed all of the participants at the UKF and six participants at the USZ, making it a total of 36 out of 60 (60%). Regarding the Web-based online survey, we sent out the study link as an invitation to all physicians who still worked at the respective institutions. We checked the IP address displayed on the SurveyMonkey website to identify potential duplicate entries from the same user. Since the computers in the hospitals are shared and the workspace usually only changes daily, we considered a difference of at least 24 hours appropriate for entries from the same IP address. Of all invitations sent out for the online survey, the participation rate was 42 out of 49 (86%). Table 1 further outlines the study and participant characteristics in detail.

Study characteristics Values
Total number of participants, N1 60 Total number of interviewed participants (field notes), n of N1 (%) 36 of 60 (60) Online survey participation rate, n of N2 (%) 42 of 49 (86) Online survey completion rate, n of N3 (%) 42 of 42 (100)   Figure 2 provides the tag cloud created from the word count of the field note responses.

Participant characteristics Values
Through inductive, free coding, we identified eight major topics with associated subthemes. Figure 3 displays the generated coding template, which was used as a coding system. The two study authors TRR and DWT rated all thirty-six field note responses independently. A total of 131 codes were assessed.
Inter-rater reliability was 71%, with a Cohen's Kappa of 0.665 agreement. After the first run, TRR and DWT discussed all differences and agreed on a coding for the case of disagreement. The second run showed an inter-rater agreement of 100% between the coding of TRR and DWT. Table 2

Themes
Positive design features. Of the total of thirty-six participants, twenty-nine (81%) made comments that fit the major topic positive design features. After additional inductive, free coding, this topic is further divided into the three subthemes "intuitive design", "clear visualization" and "innovative idea". Some participants perceived the visualization as intuitive, while others pointed out its clear visualization of standard ROTEM results. Participant #7 stated that there are no unnecessary information to distract.
Others indicated Visual Clot as innovative as they perceived this idea to be missing. Saves cognitive resources. Fifteen participants of 36 (42%) indicated that using Visual Clot, the user saves cognitive resources compared to standard ROTEM readings. The two subthemes "less brainwork needed" and "easy understanding of complex topic" are derived from this major topic. Some participants mentioned that they did not have to think as much using Visual Clot. Another participant reasoned that using Visual Clot, he did not need to memorize the cut-off values.
Easy to learn. Fourteen of 36 participants (39%) remarked that Visual Clot technology is easy to learn.
Participant #25 felt that after a short instruction, one is already able to use Visual Clot correctly.

Accelerating treatment. Thirteen of 36 participants (36%) found Visual Clot to accelerate treatment.
This major topic is subdivided into the aspects "quick recognition of situation" and "time saving". Some participants mentioned that using Visual clot most of the information is visible at a single look. Further, it helps them to interpret the coagulation status faster.
User groups. Nine of 36 participants (25%) made comments that fit the major topic user groups. All of them shared the opinion that Visual Clot would especially help an inexperienced user in the

Part II: Quantitative analysis of statements rated in an online survey
We chose six statements for the online survey and illustrate the ratings from the online survey in Figure   4. The sample medians of statements two to six all differed statistically significantly from neutral. The first statement showed that half of the participants considered the use of ROTEM readings in the management of coagulopathies difficult, whereas the other half did not.
Below we list the six statements with the evaluation of the online survey rating results: 1. "I find the interpretation of the conventional presentation of the ROTEM and the management of coagulopathies difficult." This statement does not differ statistically significantly from neutral (P =.54).
2. "I find Visual Clot intuitive and I could learn the interpretation easily." Thirty-nine of the 42 participants (93%) agreed or strongly agreed to this statement (P <.001).
3. "Visual Clot allows me to quickly grasp complex clotting situations." Thirty-six of the 42 participants (86%) agreed or strongly agreed to this statement (P <.001).

"I think Visual
Clot is helpful in the decision making process for a targeted therapy." Thirtyfour of the 42 participants (81%) agreed or strongly agreed to this statement (P <.001).
5. "I required more cognitive resources for the interpretation of Visual Clot than for the interpretation of the conventional ROTEM." One of the 42 participants (2%) agreed to this statement (P <.001).
6. "I would find it helpful to have the visualization of Visual Clot and the conventional ROTEM on the screen next to each other." Thirty-eight of the 42 participants (90%) agreed or strongly agreed to this statement (P <.001).

Discussion
This mixed methods study analyzed the opinions of physicians regarding Visual Clot, a new situationawareness oriented visualization technology for viscoelastic coagulation management. The main findings were that the participants perceived Visual Clot as intuitive, easy to learn and helpful in the decision making process for ROTEM-guided coagulation management. Further, they found that Visual Clot gave them a good overview of the clotting situation. The main criticism concerned its missing quantification. The participants would prefer this visualization on a screen next to the conventional ROTEM readings. In an environment, where health care providers are confronted with increasingly polymorbid patients and complicated diagnostic tools [20,21], more efficient assistive technologies leading to safer transmission of critical information will be of lasting importance for human performance and patient safety.
Our study showed a high level of physician acceptance and satisfaction with the new tool. Eighty-one percent of the participants regarded the animated blood clot as useful in the decision making process for coagulation management. Only one participant (2%) agreed to the statement "I required more cognitive resources for the interpretation of Visual Clot than for the interpretation of conventional ROTEM." These participant responses corresponded to the reactions to the similarly well-accepted Visual Patient technology [22]. There, only 11% of the questioned subjects considered that technology not helpful in patient monitoring after their first contact with it [23]. Visual Patient is a situation awareness-oriented visualization for patient vital signs with comparable effect size in information transfer improvement as Visual Clot [24][25][26][27][28]. Ninety-three percent of the participants considered Visual Clot intuitive to interpret and easy to learn. In the Visual Patient study [23], 82% percent of the respondents attributed the same characteristics to Visual Patient. The higher complexity of Visual Patient with far more visualizations may have caused the 10% difference in this aspect between the two technologies. Intuition enables humans to apply a new tool utilizing mainly unconscious processing and previously learned experiential knowledge [29]. Cognitive ease when learning a new technology is crucial for user acceptance [30].
Indeed, previous medical visualization technologies failed because they were too difficult to understand [31,32]. We drafted Visual Clot and its visualizations based on principles of logic, human-computer interaction, and results of prior work in medical interface and user-centered design [33,34]. According to Wittgenstein's theory [35], a coherent image or model has a meaningful commonality with the reality it is intended to reflect. To achieve this, we designed Visual Clot as a model of a blood clot. This philosophy is in line with results from Wachter et al., which show that an anatomically correct interface is particularly intuitive [36]. The referenced NASA publication outlines the various hierarchical levels of information representation [37]. It highlights the "order of wholeness" achieved by integrating the required information into a single display as the highest level of such presentation. In order to increase the cognitive receptiveness of information, we intentionally animated Visual Clot in a playful way [29,38]. This enhances user accessibility to the otherwise dry data-driven conventional result printouts.

Limitations
This study has several limitations. The opinions obtained cannot be extrapolated to a larger population because qualitative research does not investigate statistical significance. Its only goal is to collect and present the broadest possible range of opinions and views. Secondly, we asked the participants after their first contact with the new technology in a controlled instruction setting. Its use in the everyday clinical routine could alter the care providers` opinion. However, we consider it unlikely that Visual Clot would not be perceived as positively in clinical routine as it was in this study setting, as we observed widespread approval by anesthesiologists and intensive care physicians, the potential future users.
Nevertheless, further research is needed to identify the strengths and weaknesses of Visual Clot technology in a clinical setting. Another limitation concerns the selection of participants. We did not include them randomly, but according to their availability in the daily clinical routine. However, the high participation rate in the quantitative analysis reduces potential selection bias. Finally, we conducted this study in university hospitals with high standards of care in central Europe. Users' opinions may be different in other parts of the world. More research is needed in this respect as well.

Conclusions
A group of anesthesiologists and intensive care physicians regarded Visual Clot to be intuitive, easy to learn, and useful for the decision making process in acute clotting disorders. The limitations of the technology resulting from the translation of continuous measurement values into categorical ones was the most frequently mentioned potential disadvantage of the visualization technology. A split-screen implementation may be used to combine the advantages of both technologies. In this study, Visual Clot appeared to be a well-accepted decision support tool for ROTEM-based coagulation management.
Further research is needed to investigate its potential in clinical practice and medical education.