This paper is in the following e-collection/theme issue:

New Methods (963) Human Factors and Usability Case Studies (809) Clinical Communication, Electronic Consultation and Telehealth (615) Decision Support for Health Professionals (1210) Safety and Error Prevention in Health (206) Digital Health, Telehealth and e-Innovation in Clinical Settings (421) Mobile tools for surgery and perioperative medicine (85) Design of Processes and Workflows (108) Surgery, Trauma and Surgical Sciences (146) Disaster Medicine and Mass Casualty Management (24)

Published on in Vol 26 (2024)

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/65728, first published .
Evaluation of an App-Based Mobile Triage System for Mass Casualty Incidents: Within-Subjects Experimental Study

Evaluation of an App-Based Mobile Triage System for Mass Casualty Incidents: Within-Subjects Experimental Study

Evaluation of an App-Based Mobile Triage System for Mass Casualty Incidents: Within-Subjects Experimental Study

Authors of this article:

Martin Schmollinger1 Author Orcid Image ;   Jessica Gerstner2 Author Orcid Image ;   Eric Stricker2 Author Orcid Image ;   Alexander Muench2 Author Orcid Image ;   Benjamin Breckwoldt2 Author Orcid Image ;   Manuel Sigle3 Author Orcid Image ;   Peter Rosenberger2 Author Orcid Image ;   Robert Wunderlich2, 4 Author Orcid Image
Article Authors Cited by Tweetations Metrics

    Original Paper

    1School of Informatics, Reutlingen University, Reutlingen, Germany

    2University Department of Anesthesiology and Intensive Care Medicine, University Hospital Tübingen, Eberhard Karls University, Tübingen, Germany

    3Department of Cardiology and Angiology, University Hospital Tübingen, Eberhard Karls University, Tübingen, Germany

    4German Society for Disaster Medicine (Deutsche Gesellschaft für Katastrophenmedizin), Kirchseeon, Germany

    *these authors contributed equally

    Corresponding Author:

    Robert Wunderlich, MSc, MD, Dr med

    University Department of Anesthesiology and Intensive Care Medicine

    University Hospital Tübingen

    Eberhard Karls University

    Hoppe-Seyler-Str.3

    Tübingen, 72076

    Germany

    Phone: 49 70712981116

    Fax:49 7071295530

    Email: Robert.Wunderlich@med.uni-tuebingen.de


    Background: Digitalization in disaster medicine holds significant potential to accelerate rescue operations and ultimately save lives. Mass casualty incidents demand rapid and accurate information management to coordinate effective responses. Currently, first responders manually record triage results on patient cards, and brief information is communicated to the command post via radio communication. Although this process is widely used in practice, it involves several time-consuming and error-prone tasks. To address these issues, we designed, implemented, and evaluated an app-based mobile triage system. This system allows users to document responder details, triage categories, injury patterns, GPS locations, and other important information, which can then be transmitted automatically to the incident commanders.

    Objective: This study aims to design and evaluate an app-based mobile system as a triage and coordination tool for emergency and disaster medicine, comparing its effectiveness with the conventional paper-based system.

    Methods: A total of 38 emergency medicine personnel participated in a within-subject experimental study, completing 2 triage sessions with 30 patient cards each: one session using the app-based mobile system and the other using the paper-based tool. The accuracy of the triages and the time taken for each session were measured. Additionally, we implemented the User Experience Questionnaire along with other items to assess participants’ subjective ratings of the 2 triage tools.

    Results: Our 2 (triage tool) × 2 (tool order) mixed multivariate analysis of variance revealed a significant main effect for the triage tool (P<.001). Post hoc analyses indicated that participants were significantly faster (P<.001) and more accurate (P=.005) in assigning patients to the correct triage category when using the app-based mobile system compared with the paper-based tool. Additionally, analyses showed significantly better subjective ratings for the app-based mobile system compared with the paper-based tool, in terms of both school grading (P<.001) and across all 6 scales of the User Experience Questionnaire (all P<.001). Of the 38 participants, 36 (95%) preferred the app-based mobile system. There was no significant main effect for tool order (P=.24) or session order (P=.06) in our model.

    Conclusions: Our findings demonstrate that the app-based mobile system not only matches the performance of the conventional paper-based tool but may even surpass it in terms of efficiency and usability. This advancement could further enhance the potential of digitalization to optimize processes in disaster medicine, ultimately leading to the possibility of saving more lives.

    J Med Internet Res 2024;26:e65728

    doi:10.2196/65728

    Keywords

    disaster medicine; mass casualty incidents; digitalization; triage; Germany; mobile triage app 


    Background

    In recent years, the number of disasters involving mass casualty incidents (MCIs) has significantly increased [1], driven by factors such as terrorism, climate change, and war. While disaster medicine procedures are well-established and effective, there remains potential for optimization through digitalization [2]. Currently, information collected manually and on paper during an MCI is often incomplete, sometimes disorganized, and potentially unreliable. This includes critical details such as the number of patients, their locations, identifying characteristics, and injury severity. These limitations can hinder decision makers’ ability to obtain timely and accurate information essential for an effective response. This information, especially from rescue teams who are first on the scene, is crucial for effective mass casualty and disaster management, as it can save lives and provide valuable insights for scientific research to enhance future operations [3]. First responders must prioritize patients based on injury severity to ensure that those in critical condition receive immediate attention. Incident commanders coordinate rescue resources and communicate with hospitals about capacity, requiring a comprehensive overview of patient numbers, injury severity, and locations. Upon arrival at the incident scene, rescue teams search for casualties and categorize patients [4] using triage algorithms such as Simple Triage and Rapid Treatment (START) [5], modified Simple Triage and Rapid Treatment (mSTART) [6], and jumpSTART for children [7,8], classifying them by health status with color codes (Textbox 1).

    Textbox 1. Triage categories following the modified Simple Triage and Rapid Treatment algorithm.
    1. Red
      • Patients with life-threatening injuries that immediately need an adequate treatment.
    2. Yellow
      • Patients that are seriously injured and not able to walk. The treatment is still urgent but can be delayed after the first group.
    3. Green
      • Patients that are slightly injured, able to walk, and can be treated later. Sometimes they are also called “walking wounded.”
    4. Black
      • Deceased casualties or casualties that are dying and only need supportive treatment.
    5. BLUE
      • Sometimes blue is used for patients that have no chance of surviving under the given circumstances considering the available resources.

    Biographic data and initial treatments are recorded on handwritten cards attached to patients, aiding in communication and tracking. All information is relayed to incident commanders via radio, who coordinate with various rescue organizations and manage resources and patient transport based on the triage data [9,10]. Rescue services establish areas for admission, treatment, decontamination, and transportation, where patients are re-triaged to monitor changes in their health status [11]. This process ensures that the most critical patients receive timely and appropriate care.

    Weaknesses and Problems With the Current Emergency Medical Care During MCIs

    Triage data collected by rescue teams are currently communicated to the command post via radio, which must be concise and minimal due to time constraints [12]. Radio communication is not suitable for transmitting large volumes of data. Communication from rescue teams to incident commanders and dispatch centers follows a many-to-one pattern, requiring recipients to manually compile information—a process that is both time-consuming and prone to error. During triage, rescue teams gather additional data, including patient biographic information and minor treatment details. Transmitting all this information via radio is too time-consuming, so only the most critical data are communicated, resulting in incomplete information that can hinder effective disaster management [13,14]. Incident commanders often need to contact rescue teams for missing details, adding to communication time. Furthermore, verbal descriptions of patient locations are less precise than GPS tracking, complicating rapid evacuations. Rescue teams move quickly from patient to patient and can only intermittently update the command post on their findings, causing delays in situational awareness. This slow development of an overall view of the disaster scene can lead to outdated or incorrect decisions, requiring reorganization and resulting in time loss [15]. The manual, communication-intensive processes reduce work quality and increase task execution times, which can dramatically impact patient outcomes. Automating the process with a software system can help address these issues by providing a common, centralized data source accessible to all involved parties [16].

    The App-Based Mobile System “KatApp”

    The weaknesses in the current rescue process stem from the absence of a common, centralized data source accessible to all involved parties. If rescue teams utilized software for triaging patients, the results could be immediately stored in a shared data source. Incident commanders connected to this data source would have access to real-time information tailored to their needs, thereby eliminating several weaknesses through process automation. This would eliminate the need for manual compilation of information and facilitate the creation of complete data sets, minimizing data incompleteness and reducing additional communication costs. The software could also automatically track GPS coordinates of triage locations and present real-time data, enhancing the overall quality and speed of the process. By removing several manual tasks, the overall quality of work would improve, reducing errors and allowing more time for critical tasks, thereby accelerating process execution. From a functional perspective, developing such a software system may seem straightforward; however, several nonfunctional requirements must also be considered, including usability, data security, data privacy, and system availability. The software must be intuitive and easy to use under stressful conditions, such as low visibility (eg, bright sunlight or darkness). Additionally, data security and privacy safeguards must be in place to prevent unauthorized access to sensitive patient information, and the system must be reliable and instantly accessible in real-time disaster scenarios.

    These requirements are essential for the system to be effective in real disaster scenarios.

    To address these requirements, we utilized software prototyping [17] to design and develop a prototype named “KatApp.” This prototype was tested in several disaster medicine training sessions, during which we collected both quantitative data and qualitative feedback for evaluation. The system uses smartphones for triage and tablets for the command post, all connected to a shared cloud-based architecture. Applications running on the client side are part of the front-end layer and utilize services from the backend layer to read or write data, with the backend services relying on a persistence layer for permanent data storage. To connect patients with their digital data records during a disaster, rescue teams scan a patient-specific QR code before starting triage, seamlessly linking patients to their digital records (Figure 1).

    ‎
    Figure 1. Digital triage procedure using the KatApp Software System.

    The software architecture of the KatApp system follows a modern cloud-based architecture style, integrating backend and front-end components. The front-end layer comprises 2 client applications: a native app for triaging patients, used by rescue teams, and a web application that implements a dashboard for the command post. Both client applications access services from the backend layer, with the native app using a REST (Representational State Transfer) application programming interface and the web application connecting via a web socket connection.

    The triage app is developed using Google’s Flutter framework [18], which supports multiple platforms from a single codebase—an essential feature for training and future use by different organizations. The app’s user interface is designed for ease of use during triage, transitioning from flowchart-based designs to a page flow that represents the triage algorithms. In addition to implementing the triage algorithms (mSTART and jumpSTART), the app includes several features such as day and night modes; using the smartphone flashlight to easily scan the QR code, even at night; and an overview of completed triages. The command post dashboard was developed as a web application using the React framework [19,20]. It displays information on the exact GPS locations of patients using OpenStreetMap [21], along with a list of patients who have been triaged and the time elapsed since the start of triage handling, together with the current time.

    The dashboard application connects to the backend service using a web socket connection. As soon as new triage records are received by the backend service from the rescue teams’ apps, they are pushed to the dashboard application through this connection. This ensures that the command post always has the most current information without the need to manually pull data. The system’s design is well-suited for serverless computing [22,23], as it is only required during real disasters or training sessions. This allows for billing based solely on active usage, which aligns with the sporadic nature of MCIs and has the potential to reduce overall operational expenses.

    After prototyping KatApp, several functionality tests were conducted under realistic conditions, including the use of waterproof devices and gloves. These tests were followed by training sessions aimed at gathering qualitative feedback to refine the prototype.

    Logging functionalities were subsequently added to measure triage time and accuracy. Early usability insights were incorporated, and the system was standardized for evaluation in a within-subjects experimental study comparing it with the paper-based triage method. Table 1 presents the early implementations of our prototype in pilot studies. Several usability insights for the app and dashboard were promptly added to the backlog. To standardize the evaluation of our refined prototype against the conventional paper-based triage tool, we conducted an experiment using a within-subjects design.

    Based on the insights gained from these early implementations, we made several improvements to the KatApp prototype. These refinements included enhancements to the user interface, optimization of the triage algorithms, and improvements to the data logging capabilities. The resulting version of the app, which incorporated feedback from all 3 training sessions, was used in the subsequent comparative experiment.

    Table 1. Early-stage implementations of the KatApp prototype.
    Training sessionObjectiveSampleProcedureResults
    Training 1Obtain first qualitative user feedbackN=15 rescue service personnel, emergency physicians, and medical studentsParticipants triaged 60 patient cards at different locations (ie, at the location of the incident, at the treatment area, and before the release of the patient)
    • 150 triages were made
    • More than 90% (11/12) of the app users and 100% (4/4) of the dashboard users stated that they would like to use such a system in a real disaster
    Training 2Use a prototype in an evacuation training (earthquake) and finish the evacuation within 90 minutesN=50 medical students as first responder teams, supervised by experienced emergency physiciansParticipants triaged 18 patients (played by actors)
    • 48 triages were made
    • The scenario could be finished in time
    • 2 patients were triaged in the wrong category
    • Average time of a triage: 42 seconds
    Training 3Use prototype in an evacuation training (flood warning + gas explosion) and finish the evacuation within 75 minutesN=50 rescue service personnel and emergency physiciansParticipants triaged 23 patients (played by actors)
    • 50 triages were made
    • All patients were triaged in the correct category
    • Average time of a triage: 37 seconds

    Research Objective

    The study aimed to evaluate the KatApp as a triage tool for emergency and disaster medicine, which was previously designed and developed by our research group, as described in the “Background” section. The objective was to determine whether the KatApp is superior to paper-based triage tools in terms of speed, accuracy, and user experience among emergency medicine experts. We hypothesized that the KatApp would be at least as accurate as the paper-based tool, while also being faster and more user-friendly.


    Ethical Considerations

    The Ethics Committee of the Faculty of Medicine at Tübingen University Hospital approved the study (approval number 260/2024BO2). All methods were implemented in accordance with the Declaration of Helsinki, and written informed consent was obtained from all participants.

    Study Design and Setting

    The study was designed as a within-subjects experimental design, with the triage tool serving as a within-subjects factor (comparing the paper-based tool and KatApp). The order of the triage tools was counterbalanced to mitigate carryover effects.

    The study was conducted at the German Red Cross in Kirchentellinsfurt on behalf of the University Hospital Tübingen, in collaboration with Reutlingen University, in June 2024. Participation in the study was voluntary.

    Recruitment

    Invitations to the triage training course and calls for participation in the study were sent to a mailing list from the Tübingen District Association of the German Red Cross, which includes approximately 500 email addresses of rescue service personnel, physicians, and medical students with German Red Cross Emergency Medical Technician (EMT) qualifications.

    To determine the sample size for the study, a statistical power analysis was conducted using G*Power (version 3.1 for MacOS; Universität Düsseldorf) [24]. The analysis indicated a required sample size of 45 participants, with an effect size of Cohen d=0.5, a statistical power of 0.95, and a significance level (P value) of .05. It is important to note that, due to the limited number of similar publications, the statistical power analysis was based on general considerations regarding the trade-offs between the ability to detect medium-sized effects and the feasibility of obtaining a sufficiently large sample. To account for potential no-shows and dropouts, we decided to recruit an additional 10 participants, bringing the total to 55.

    Study Procedures

    After participants registered for the triage training course and study, they received instructions on how to install the app and details about the study procedure 1 week before the course.

    To familiarize participants with the mSTART algorithm using both the paper-based tool and the KatApp, an introduction was provided at the beginning of the course. Participants were then asked to complete 2 triage sessions set in the context of a simulated MCI, involving a terrorist driving into the fan zone of a public screening of the UEFA (Union of European Football Associations) European Football Championship 2024. In one session, participants used the KatApp, while in the other session, they used the conventional paper-based tool. The order of the tools was randomized. Each session involved classifying 30 patient cards into 1 of 4 triage categories, with the cards distributed along a course of exactly 96.32 m for each triage tool. The 2 sessions were conducted simultaneously, with participants starting their sessions every 5 minutes. Refer to Figure 2 for a sketch of the experimental setup, Multimedia Appendix 1 for a drone photograph of the setting, and Multimedia Appendix 2 for a drone video.

    After each session, participants completed a web-based questionnaire using SurveyMonkey (SurveyMonkey Inc.). Following a recovery period of approximately 30 minutes, they began the second session with the alternate triage tool.

    ‎
    Figure 2. Sketch of the experimental setup. Each session consisted of 30 patient cards (marked as triangles in this sketch) that were distributed along a course of 96.32 m. Emergency vehicles and a fog machine blocked the view between the 2 courses. In addition, simulated sounds of a mass casualty incident were played via connected sound boxes that were placed in the middle of each course. Immediately after the sessions, the participants were asked to scan a QR code to complete the follow-up questionnaire.

    Patient Cards

    The patient cards were initially developed using a preexisting artificial patient database [25,26] and subsequently adapted by the study team. Each card featured a pictogram to illustrate the injury pattern, along with details on injury severity (Abbreviated Injury Scale) and vital parameters, including the Glasgow Coma Scale, respiratory rate, systolic blood pressure, radial pulse, capillary refill time, walking ability, and age. Additionally, a short storyboard provided further context regarding the circumstances of the accident (Multimedia Appendix 3). To ensure comparability between the 2 sessions, the number of patient cards in each triage category was kept identical: red (6/30, 20%), yellow (13/30, 43%), green (9/30, 30%), and black (2/30, 7%). This categorization was based on empirical data from MCIs [27]. Additionally, we paired patient cards to match them in terms of overall injury severity (Abbreviated Injury Scale) and injury pattern, minimizing the impact of any differences in the patient cards on the outcome variables between the triage tools. The order of patient cards in each course was randomized using the online tool RANDOM.ORG (Randomness and Integrity Services Limited), ensuring that the randomization was consistent for each participant.

    Triage Tools

    Paper-Based Tool

    When using the conventional paper-based tool, participants were required to complete a casualty card (see Multimedia Appendix 4) from their equipment. This process involved manually indicating the triage category by folding the card and attaching it to the patient card. Additionally, participants needed to record the triage category on a documentation sheet. To assist with the triage process, they were provided with an mSTART checklist [6].

    KatApp

    When using the KatApp, participants were required to attach a QR code from their equipment to each patient. They then scanned the QR code with the KatApp’s QR code scanner and conducted the triage process as previously described (see Figure 1 and Multimedia Appendix 5). Participants used their smartphones, which were connected to the mobile network, allowing the triage results to be immediately forwarded to a central dashboard.

    System Architecture Behind the KatApp

    All software components are managed using repositories on GitHub (GitHub, Inc.). Automatic deployments are facilitated by GitHub Actions and custom scripts [28]. The iOS app is deployed to TestFlight (Apple Inc.), while the Android app is deployed to Firebase (Google LLC/Alphabet Inc.). Rescue teams can install the desired version from these platforms before training. The dashboard web application and the backend are deployed on Amazon Web Services (AWS; Amazon Web Services, Inc.). The dashboard functions as a web application hosted on AWS, while the backend services are implemented as AWS Lambda functions [29]. These services utilize AWS DynamoDB for user management, logging, and triage records, and AWS S3 for binary data storage. Figure 3 provides a detailed depiction of the architecture.

    The KatApp system operates as a distributed system, dependent on the connectivity of its various nodes, which include the mobile devices used by rescue teams, the tablets at the command post, and the backend services. In standard operation mode, it is assumed that an internet connection is accessible at the incident scene, enabling the mobile apps used by rescue teams to access the web application and backend services hosted in the cloud. The dashboard web application is accessible through browsers on tablets or computers at the command post, allowing it to establish a web socket connection to the KatApp backend. In an ideal scenario, this operational mode would suffice. However, as highlighted by the fallacies of distributed computing [30], networks can often be unreliable, particularly during disasters, when essential communication infrastructure may be compromised or overwhelmed, or in areas with inadequate network coverage. To tackle these challenges, the KatApp system incorporates a caching mechanism within the triage app, enabling rescue teams to continue triaging patients even in the absence of an internet connection. Triage records are stored locally and are automatically transmitted once connectivity is restored. Furthermore, the KatApp system features an operational mode that functions independently of an internet connection, ensuring that critical triage processes can continue uninterrupted during emergencies. A local wireless network can be established by the command post or a technical organization on-site, featuring a server that hosts the dashboard application and backend services. Mobile devices can connect to this local network, enabling the system to operate as intended. As the backend services were developed as AWS Lambda functions, they cannot be executed locally. To address this limitation, an additional abstraction layer was implemented, allowing the use of alternative open-source or custom-built components for each AWS service. This ensures that the KatApp system remains functional and efficient, even in environments where internet connectivity is compromised.

    ‎
    Figure 3. Schematic overview of the KatApp architecture without deployment and identity and access management. API: application programming interface; AWS: Amazon Web Services; DB: database; SPA: single-page application.

    Measuring and Outcomes

    Duration and Quality of Triages

    The study staff recorded the time at the beginning and end of each course as participants crossed the start and finish lines. This timing encompassed the entire triage process, including walking between patients, applying QR codes or attaching paper triage cards (depending on the method used), and conducting the triage assessment itself. Additionally, the number of correct triages per session was evaluated by analyzing participants’ documentation sheets from the paper-based tool, as well as the data collected by the KatApp.

    Subjective Ratings

    Participants were required to rate both triage tools using the German grading system, where a score of 1 represents “excellent” and a score of 6 indicates “insufficient.” They were also asked to express their preference between the 2 tools through a forced-choice item.

    User Experience Questionnaire

    We used the German version of the User Experience Questionnaire (UEQ) [31] to evaluate participants’ experiences with each triage tool. The UEQ comprises 26 items organized into 6 scales: Attractiveness, Perspicuity, Efficiency, Dependability, Stimulation, and Novelty. Each item is presented as a semantic differential, requiring participants to rate their responses on a 7-point Likert scale, which is coded from –3 to +3 (Multimedia Appendix 6). Attractiveness refers to the overall impression of the product (Does a user like or dislike the product?), whereas Perspicuity assesses the ease with which users can get familiar with the product and learn how to use it. Efficiency indicates whether users can complete their tasks without unnecessary effort, while Dependability refers to the sense of control, security, and predictability in the interaction. Stimulation measures how exciting and motivating the product is to use it, while Novelty assesses whether the product is creative and captures the interest of users. Figure 4 illustrates the overall experimental design.

    ‎
    Figure 4. Within-subjects experimental design that was used for the evaluation of the KatApp compared with the conventional paper-based tool. Each participant conducted 2 sessions in which they had to triage 30 patient cards. The tool order was counterbalanced to compensate for possible carry-over effects. UEQ: User Experience Questionnaire.

    Statistical Analysis

    SPSS version 29 for MacOS (IBM Corp.) was utilized for data management and statistical analyses.

    To analyze the differences between the conventional paper-based tool and the KatApp, we performed a mixed multivariate analysis of variance (MANOVA) with the triage tool as a within-subjects factor (paper-based tool vs KatApp). To account for potential interactions between the triage tool and tool order, we included tool order (KatApp used in the first session vs the second session) as a between-subjects factor. As dependent variables, we included the overall time taken to complete each triage session, the number of correct triages, the participants’ subjective ratings of the triage tools using the German school grading system, and the results from the 6 scales of the UEQ.

    In a separate repeated-measures MANOVA, we utilized the measurement time point (first vs second session) as a within-subjects factor for the same 9 dependent variables to evaluate differences between the 2 time points.

    We reported the Pillai trace as the F-statistic in our models and conducted post hoc univariate ANOVAs for each dependent variable.

    Despite some outcome variables violating the assumptions of normal distribution, we calculated MANOVAs, as this method is relatively robust against such violations to conduct MANOVAs instead of multiple ANOVAs because MANOVAs account for the complex relationships between dependent variables, offer greater statistical power, and help mitigate the issue of alpha error accumulation compared with multiple testing.

    As an exploratory analysis, we calculated nonparametric correlations (Kendall tau) between age and the outcome variables to evaluate the suitability of including age as a covariate in our model. This assessment aimed to investigate the potential influences of age on the use of the app, as younger generations may be more accustomed to using smartphones and apps compared with older individuals. However, no statistically significant correlations were found (see Multimedia Appendix 7). Additionally, we chose not to include other demographic data, such as gender or profession, in our model, as they did not provide additional value concerning our study objectives and could further decrease statistical power, as previously discussed.

    We established the significance level at α=.05 for our analyses.

    We created raincloud plots in R (R Foundation) using the raincloudplots package [32]. This approach visualizes differences between the 2 triage tools, incorporating raw data, probability density, and key summary statistics in a transparent format.

    The study was conducted in accordance with the Template for Intervention Description and Replication (TIDieR) checklist and guideline recommendations. The completed checklist is available as Multimedia Appendix 8.


    Sample Size and Exclusions

    In total, 55 individuals initially registered for the course, resulting in 48 participants who attended, which reflects a no-show rate of 13% (7/55). Among those who attended, 6 participants were excluded from the statistical analyses due to incomplete triage documentation (n=3 with the paper-based tool, n=2 with the KatApp, and n=1 with both tools). Additionally, 2 participants did not complete questionnaire 1 and 2 did not complete questionnaire 2. Consequently, the final sample size for further statistical analyses comprised 38 participants.

    Sample Characteristics

    Demographic characteristics of the sample are presented in Table 2. The average age of the participants was 32.5 (SD 9.2) years. Of the total 38 participants, 24 (63%) were male and 15 (39%) reported having previous experience with triaging. In terms of occupation, 14 (37%) were physicians, 17 (45%) worked as rescue service personnel, 4 (11%) were medical students with German Red Cross EMT qualifications, and 3 (8%) belonged to other groups (eg, standby service or honorary positions).

    Table 2. Demographic characteristics of the sample.
    CharacteristicsValue
    Age (years), mean (SD)32.5 (9.2)
    Gender, n/N (%)

    		Female14/38 (37)

    		Male24/38 (63)
    Previous experience, n/N (%)

    		Yes15/38 (39)

    		No23/38 (61)
    Occupation, n/N (%)

    		Physician14/38 (37)

    		Rescue service personnel17/38 (45)

    		Medical students with EMTa qualifications4/38 (11)

    		Other3/38 (8)
    Specialization (physicians), n/N(%)

    		Anesthesiology10/14 (71)

    		Surgery2/14 (14)

    		Neurology1/14 (7)

    		Pediatrics1/14 (7)
    Additional qualification in emergency medicine (physicians), n/N (%)

    		Yes11/14 (79)

    		No3/14 (21)
    Highest qualification (rescue service personnel), n/N (%)

    		EMT-basic12/17 (71)

    		EMT-intermediate1/17 (6)

    		Paramedic4/17 (24)

    aEMT: Emergency Medical Technician.

    Evaluation Results

    Overall

    Findings from the 2 (triage tool) × 2 (tool order) mixed MANOVA revealed a significant main effect for the within-subjects factor of the triage tool (F9,28=70.585, P<.001, η2p=.958). Additionally, there was a significant interaction between the triage tool and the tool order (F9,28=4.432, P=.001, η2p=.588); however, no significant main effect was found for the between-subjects factor of tool order (F9,28=1.382, P=.24, η2p=.308).

    Duration

    The results of the post hoc univariate ANOVA indicated that participants were significantly faster when using the KatApp compared with the paper-based tool (F1,36=229.769, P<.001, η2p=.865). The mean difference is illustrated in Figure 5A. In addition, a significant interaction was found between the triage tool and the tool order (F1,36=15.052, P<.001, η2p=.295), suggesting that the time taken for each tool was lower in the second session, although the time was consistently less for the KatApp.

    ‎
    Figure 5. Results of the User Experience Questionnaire (UEQ) comparing the conventional paper-based tool with the KatApp regarding the 6 scales (A) Attractiveness, (B) Perspicuity, (C) Efficiency, (D) Dependability, (E) Stimulation, and (F) Novelty.
    Accuracy

    The post hoc univariate ANOVA also demonstrated that participants assigned a significantly greater number of patient cards to the correct category when using the KatApp compared with the paper-based tool (F1,36=8.979, P=.005, η2p=.2), as illustrated in Figure 5B. Additionally, a significant interaction was observed between the triage tool and the tool order concerning triage quality (F1,36=4.731, P=.04, η2p=.116). This interaction indicates that the number of correct triages was higher in the second session for both tools, with the KatApp consistently yielding a greater number of correct triages.

    Subjective Ratings

    When participants rated the triage tools using the German school grading system, the post hoc univariate ANOVA revealed significantly better ratings for the KatApp (rated as excellent) compared with the paper-based tool (rated as satisfactory; F1,36=116.281, P<.001, η2p=.764). There was no significant interaction between the triage tool and the tool order (F1,36=0.007, P=.93, η2p=.000). The mean difference is illustrated in Figure 5C.

    User Experience Questionnaire

    Regarding the 6 scales of the UEQ, post hoc univariate ANOVAs indicated significantly better ratings for the KatApp compared with the paper-based tool on each scale (Attractiveness: F1,36=128.812, P<.001, η2p=.782; Efficiency: F1,36=105.542, P<.001, η2p=.744; Perspicuity: F1,36=26.390, P<.001, η2p=.423; Dependability: F1,36=61.693, P<.001, η2p=.631; Stimulation: F1,36=58.422, P<.001, η2p=.619; and Novelty: F1,36=132.687, P<.001, η2p=.787). There was no significant interaction between the triage tool and the tool order for any of the UEQ scales (Attractiveness: F1,36=0.214, P=.65, η2p=.006; Efficiency: F1,36=0.006, P=.94, η2p=.000; Perspicuity: F1,36=0.636, P=.43, η2p=.017; Dependability: F1,36=1.762, P=.19, η2p=.047; Stimulation: F1,36=0.434, P=.51, η2p=.012; Novelty: F1,36=2.361, P=.13, η2p=.062). Figure 6 illustrates the differences between the tools regarding the 6 scales of the UEQ.

    The mean values and corresponding statistics for the outcome measures between the 2 triage tools are presented in Table 3 and Multimedia Appendix 9 (also see [33]).

    ‎
    Figure 6. Comparison of the conventional paper-based tool with the KatApp regarding (A) duration to complete each session in minutes, (B) triage accuracy as number of correct triages out of 30, and (C) the participants’ subjective ratings as German school grades from 1 (excellent) to 6 (insufficient). UEQ: User Experience Questionnaire.
    Table 3. Post hoc univariate analysis of variance comparison between the paper-based tool and the KatApp (N=38).
    Outcome variableAnnotations in the paper-based tool, mean (SD)Annotations in the KatApp, mean (SD)F test (df)P valueη2p
    Duration (minutes)43.71 (9.816)25.22 (4.737)229.769 (1, 36)<.0010.865
    Triage accuracy28.92 (1.099)29.45 (0.724)8.979 (1, 36).0050.200
    Subjective rating3.26 (0.860)1.53 (0.506)116.281 (1, 36)<.0010.764
    User Experience Questionnaire





    		Attractiveness–0.58 (1.175)1.96 (0.764)128.812 (1, 36)<.001.782

    		Efficiency–0.65 (1.224)1.86 (0.900)104.542 (1, 36)<.001.744

    		Perspicuity0.94 (1.146)2.09 (0.861)26.390 (1, 36)<.001.423

    		Dependability0.49 (0.937)1.88 (0.600)61.693 (1, 36)<.001.631

    		Stimulation0.14 (1.251)1.76 (0.997)58.422 (1, 36)<.001.619

    		Novelty–1.45 (0.964)1.54 (0.999)132.687 (1, 36)<.001.787
    Preference

    When participants were asked to express their preference using a forced-choice item, an overwhelming majority (36/38, 95%) favored the KatApp, while the remaining 2 (5%) preferred the paper-based tool.

    Controlling for Order Effects

    The results of the 2 (triage tool) × 2 (tool order) mixed MANOVA indicated no significant main effect for the between-subjects factor of tool order (F9,28=1.382, P=.24, η2p=.308). However, univariate ANOVAs for the between-subjects effect revealed no significant differences across the measures, except for the UEQ scale novelty (F1,36=6.625, P=.01, η2p=.155). This finding suggests that when the KatApp was used in the second session, novelty ratings were significantly higher than when it was used first.

    Additionally, the repeated measures MANOVA for outcome measures at each time point, with the session as a within-subjects factor, did not show a significant main effect for session (F9,29=2.094, P=.06, η2p=.394). None of the post hoc analyses indicated significant differences between the first and second sessions for any of the outcome measures (Table 4).

    Table 4. Post hoc univariate multivariate analysis of variance comparison between the results of the first and second sessions.
    Compared variablesF test (df)P valueη2p
    Triage quality3.357 (1, 37).07.083
    Time duration3.357 (1, 37).26.035
    Subjective rating0.104 (1, 37).75.003
    User Experience Questionnaire



    		Attractiveness0.253 (1, 37).62.007

    		Efficiency0.101 (1, 37).75.003

    		Perspicuity0.165 (1, 37).69.004

    		Dependability1.151 (1, 37).29.030

    		Stimulation0.443 (1, 37).51.012

    		Novelty0.190 (1, 37).66.005

    Principal Findings

    In this paper, we present an innovative approach to triage that replaces traditional paper-based classification with a mobile app–based system for categorizing patients in the context of MCIs. After prototyping the mobile app, we tested it in several training sessions and refined its features. The final prototype was then evaluated against the conventional paper-based tool regarding time duration, triage quality, and subjective ratings, such as user experience, using a within-subjects experimental study. Our results suggest that using this software system enables triage to be completed more quickly, easily, and accurately compared with the conventional paper-based tool. This approach addresses the time-consuming and error-prone issues associated with current procedures in disaster medicine.

    The most significant and noticeable difference was the time participants required to complete the 2 triage sessions, each consisting of 30 patient cards. Using the KatApp, participants were, on average, over 18 minutes faster than when using the paper-based tool. Additionally, because command posts receive information in real time, critical decisions can be made sooner, allowing for faster patient treatment, which could ultimately save more lives. It is important to note that participants did not have to transmit their information via radio communication to the command post during the paper-based session, which would have added even more time. The requirement to fill out casualty cards manually and complete the documentation sheet likely contributed to this time difference.

    Participants also found the KatApp to be more user-friendly than the conventional paper-based tool. In each of the 6 scales of the UEQ, participants rated the KatApp as “excellent” or at least “good” according to the corresponding benchmarks [33]. By contrast, the paper-based tool received ratings of “bad” or “below average.” This trend is reflected in the school grades awarded by participants: the KatApp was perceived as “excellent,” while the paper-based tool averaged a “satisfactory” grade. These results suggest that, in addition to the time advantage, the KatApp was perceived as easier to use than the paper-based tool. Furthermore, the significance of digitizing disaster medicine processes is underscored by the fact that 36 out of 38 (95%) participants indicated a preference for the KatApp over the traditional triage tool.

    Surprisingly, there was also a statistically significant difference in triage accuracy between the 2 tools, indicating that participants triaged more patients correctly when using the KatApp compared with the paper-based tool. However, because the effect size for this difference was smaller than that of other measures, this result should be interpreted with caution. It is possible that the patient cards varied slightly, even though they were matched in terms of injury severity, injury pattern, and triage category. Additionally, the number of correctly assigned patient cards was high for both tools. However, some participants particularly valued the support provided by the app during triage, which may have contributed to fewer errors due to the automated application of the mSTART algorithm.

    The results of our within-subjects experimental study support our hypotheses, demonstrating that the KatApp system is at least as useful and reliable as the conventional paper-based tool for triaging in disasters involving MCIs. The advantages of a digital triage tool—such as automatic forwarding of information to a shared data source and central dashboard, collection of real-time data, tracking of patients’ geolocations, and gathering of complex data to optimize rescue operations—are clearly evident. In addition to our quantitative results, we observed several additional advantages in our study. During the early implementations of the KatApp and in the experimental study, participants did not receive extensive training before using the system. Instead, they were given only a brief introduction right before the training began. This highlights the overall usability of the KatApp system.

    Furthermore, qualitative feedback from the command posts after the training sessions confirmed that the information presented in the dashboard significantly simplified their work. In particular, the real-time overview of patients along with their triage categories, as well as the map displaying patient locations, was highly appreciated.

    Next Steps and Further Adaptions

    The next steps will include enhancing interconnectivity between smartphones when mobile networks are unavailable and incorporating emergency resources such as ambulances, helicopters, and hospitals into the backend. We also plan to conduct final tests in realistic scenarios, including the use of full personal protective equipment and simulated body fluids. Following these tests, we will initiate a pilot phase in real, nonsimulated settings with the goal of introducing the live system as quickly as possible.

    Comparison With Prior Work

    To our knowledge, this is the first paper addressing the challenges of current rescue operations in mass casualty disasters through the development, implementation, and experimental evaluation of a mobile triage app compared with a conventional paper-based tool. A systematic review of triage apps for health emergencies [34] indicates that there have been previous attempts to develop similar approaches. For example, in 2014, a research group developed a mobile-based system to support triage in MCIs, incorporating a Cox proportional hazard model to present patients’ survival curves and aid in triage and transportation decisions [35]. Another approach involved using smart glasses along with a specific Android app to assist in triage during MCIs, which was evaluated in a randomized controlled simulation trial [36]. The TRIAGIST mobile app [37,38] was developed as a low-cost system that dispatch centers or rescue teams in Thailand could use for developmental purposes. However, none of these approaches have been further implemented for use in disaster medicine to date. Additionally, there has been a decline in studies focused on apps for catastrophe triage in recent years, and the few existing triage apps on the market that are based on scientific and academic research often lack free access [34]. In response to the call for an accessible app that provides patient geolocation and assists rescue teams by automatically applying a triage algorithm [34], our KatApp system aims to address a significant gap in the field. This contribution is of great practical importance for improving triage processes in disaster scenarios.

    Limitations

    In our study, we utilized patient cards instead of real patients or actors, which may limit the external validity of our findings. Our primary goal was to compare both triage tools within a standardized procedure and analyze a large number of triages, resulting in a total of 2280 triages. Involving 60 actors would have been more costly with only marginal additional benefit. The KatApp still needs to be evaluated in real disaster operations or large-scale simulations. Regarding the within-subjects design, we acknowledge that order effects, such as practice or fatigue, cannot be completely ruled out. To address this, we counterbalanced the tool order and included it as a between-subjects factor in our MANOVA. We also conducted a second repeated-measures MANOVA with measurement time points as a within-subjects factor. Our analyses revealed no significant influence of order on the primary outcomes. While practice effects were observed, participants consistently performed better in terms of time and accuracy with the KatApp. Despite these order effects, we chose a within-subjects design for its greater statistical power, better control over individual differences, and smaller sample size requirement. However, within-subjects designs may also make it easier for participants to infer the hypotheses, potentially distorting results due to heightened expectations. Finally, some assumptions for our statistical analyses, such as the normal distribution of variables, were partially violated. However, MANOVA is generally considered robust to such violations [39]. Although the overall missing data rate was high (10/48, 21%), we still reported large effect sizes in our results.

    Conclusions

    In this paper, we present research that addresses the weaknesses and challenges of the current triage process in MCIs through the development, implementation, and evaluation of a mobile system called KatApp.

    We demonstrated that the KatApp is a mobile triage app designed for disaster medicine that not only matches the performance of the conventional paper-based tool but may even surpass it in terms of efficiency and usability. This advancement highlights the potential of digitalization to optimize processes in disaster medicine, ultimately leading to the possibility of saving more lives. While it is clear that this novel approach reduces time-consuming and error-prone manual tasks, we are currently unable to fully quantify the impact of the KatApp system on efficiency in real disaster situations, as it has only been implemented in simulated training scenarios thus far. Further research and refinement of this approach are needed to address this gap.

    Acknowledgments

    As the development of the KatApp system was a purely academic project, several information technology students contributed to the development of the KatApp system. We would like to mention the following: Ralf Arnold, Marvin Bermel, Niklas Digeser, Nico Döbele, Hai Yen Dang, Berend Fuchs, Diana Gilmanov, Laurenz Guth, Alexander Heinz, David Hötten, Jarne Jost, Nico Kantas, Hannes Kececi, Marvin Kotzian, Jona Krumrein, Fabian Just, Jan Meckelholt, Pascal Nerger, Sait Ok, Michael Pfeiffer, Ansam Qasem, Benjamin Reiser, Philipp Riedel, Hannes Wagner, and Tim Weiß. We also thank Dominik Neumann for cosupervising the students and the German Red Cross for hosting us at their center for the experiment. To improve language, we used Perplexity AI (Perplexity AI, Inc.) as generative artificial intelligence to rephrase and translate text passages. We acknowledge the support from the Open Access Publication Fund of the University of Tübingen.

    Data Availability

    The data sets generated and analyzed during this study are available from the corresponding author upon reasonable request. The software code for the KatApp cannot be made publicly available. However, interested researchers may inquire about specific aspects of the system’s functionality and architecture. We are committed to ensuring that the data supporting the findings of this study are accessible to the scientific community.

    Authors' Contributions

    RW, ES, and M Schmollinger initiated and coordinated the study. RW and JG coordinated the experiment, including data collection. JG and RW interpreted the data. M Schmollinger and JG wrote the initial draft of the manuscript. PR, AM, BB, and M Sigle critically reviewed and substantially improved the manuscript. All authors reviewed and approved the final version of the manuscript before submission. All authors had access to all data and analysis in the study. All authors had final responsibility for the decision to submit for publication.

    Conflicts of Interest

    None declared.

    Multimedia Appendix 1

    Experiment overview picture.

    PNG File , 1675 KB
    Multimedia Appendix 2

    Experiment overview film.

    MOV File , 39934 KB
    Multimedia Appendix 3

    Patient cards.

    PDF File (Adobe PDF File), 593 KB
    Multimedia Appendix 4

    Paper-based casualty cards.

    PDF File (Adobe PDF File), 749 KB
    Multimedia Appendix 5

    KatApp mock-ups.

    PDF File (Adobe PDF File), 1066 KB
    Multimedia Appendix 6

    German version of the User Experience Questionnaire as used in our study with English translations.

    DOCX File , 16 KB
    Multimedia Appendix 7

    Kendall tau correlations between age and the outcome variables to check the conditions for a multivariate analysis of covariance.

    PDF File (Adobe PDF File), 258 KB
    Multimedia Appendix 8

    TIDieR (Template for Intervention Description and Replication) checklist.

    PDF File (Adobe PDF File), 53 KB
    Multimedia Appendix 9

    Post hoc univariate analysis of variance comparison between the paper-based tool and the KatApp (N=38).

    DOCX File , 16 KB
    1. Centre for Research on the Epidemiology of Disasters (CRED). 2022 Disasters in numbers. CRED. Brussels, Belgium. Institute Health and Society – UCLouvain; 2023. URL: https://cred.be/sites/default/files/2022 _EMDAT_report.pdf [accessed 2024-08-06]
    2. Sever MS, Remuzzi G, Vanholder R. Disaster medicine and response: optimizing life-saving potential. Am J Disaster Med. 2018;13(4):253-264. [CrossRef] [Medline]
    3. Cuthbertson J, Archer F, Robertson A, Rodriguez-Llanes JM. Improving disaster data systems to inform disaster risk reduction and resilience building in Australia: a comparison of databases. Prehosp Disaster Med. Sep 16, 2021;36(5):511-518. [CrossRef]
    4. Cheng T, Staats K, Kaji AH, D'Arcy N, Niknam K, Donofrio-Odmann JJ. Comparison of prehospital professional accuracy, speed, and interrater reliability of six pediatric triage algorithms. J Am Coll Emerg Physicians Open. Feb 2022;3(1):e12613. [FREE Full text] [CrossRef] [Medline]
    5. Franc JM, Kirkland SW, Wisnesky UD, Campbell S, Rowe BH. METASTART: a systematic review and meta-analysis of the diagnostic accuracy of the Simple Triage and Rapid Treatment (START) algorithm for disaster triage. Prehosp Disaster Med. Feb 2022;37(1):106-116. [CrossRef] [Medline]
    6. Kanz KG, Hornburger P, Kay MV, Mutschler W, Schäuble W. mSTaRT-algorithmus für sichtung, behandlung und transport bei einem massenanfall von verletzten. Notfall Rettungsmed. May 2006;9(3):264-270. [CrossRef]
    7. Romig LE. Pediatric triage. A system to JumpSTART your triage of young patients at MCIs. JEMS. Jul 2002;27(7):52-8, 60. [Medline]
    8. Smith M. Get smart: jumpSTART! Emerg Med Serv. May 2001;30(5):46-8, 50. [Medline]
    9. Spöttl P, Ziegler A. Triage: prioritization of injured after a major accident or mass casualty incident. Med Klin Intensivmed Notfmed. Feb 3, 2015;110(1):9-14. [CrossRef] [Medline]
    10. Dirks B. Management des massenanfalls von verletzten/erkrankten durch den leitenden notarzt. Notfall Rettungsmed. May 2006;9(3):333-346. [CrossRef]
    11. Rauch L, Dotzer M, Händl T. [Management of a mass casualty incident according to the recommendations of the Federal Office of Civil Protection and Disaster Assistance and the principles of the Terror and Disaster Surgical Care Concept in a regional trauma center]. Anaesthesiologie. Mar 27, 2024;73(3):193-203. [CrossRef] [Medline]
    12. Auf der Heide E. The importance of evidence-based disaster planning. Ann Emerg Med. Jan 2006;47(1):34-49. [CrossRef] [Medline]
    13. De Cauwer H, Barten D, Willems M, Van der Mieren G, Somville F. Communication failure in the prehospital response to major terrorist attacks: lessons learned and future directions. Eur J Trauma Emerg Surg. Aug 10, 2023;49(4):1741-1750. [CrossRef] [Medline]
    14. Sadat SJ, Afrasiabifar A, Khorasani-Zavarehg D, Moradian MJ, Vafaeenasab M, Dehghani Tafti A, et al. Exploring barriers and facilitators of inter-organizational management in response to mass casualty traffic incidents: a qualitative study. Bull Emerg Trauma. Apr 2021;9(2):86-95. [FREE Full text] [CrossRef] [Medline]
    15. Bania A, Iatrellis O, Samaras N. Information communication technologies (ICTs) and disaster risk management (DRM): systematic literature review. Berlin, Germany. Springer; 2023. Presented at: CSUM 2022; August 31-September 2, 2022:1779-1794; Skiathos Island, Greece. URL: https://link.springer.com/book/10.1007/978-3-031-23721-8 [CrossRef]
    16. Sakurai M, Murayama Y. Information technologies and disaster management – benefits and issues. Progress in Disaster Science. Jul 2019;2:100012. [CrossRef]
    17. Davis A. Software prototyping. Advances in Computers. 1995;40:39-63. [CrossRef]
    18. Białkowski D, Smołka J. Evaluation of Flutter framework time efficiency in context of user interface tasks. jcsi. Dec 30, 2022;25:309-314. [CrossRef]
    19. Karić A, Durmić N. Comparison of JavaScript frontend frameworks – Angular, React, and Vue. International Journal of Innovative Science and Research Technology (IJISRT). Jul 1, 2024:1383-1390. [CrossRef]
    20. CACM Staff. React: Facebook's functional turn on writing JavaScript. Commun ACM. Dec 2016;59(12):56-62. [CrossRef]
    21. Herfort B, Lautenbach S, Porto de Albuquerque J, Anderson J, Zipf A. The evolution of humanitarian mapping within the OpenStreetMap community. Sci Rep. Feb 04, 2021;11(1):3037. [FREE Full text] [CrossRef] [Medline]
    22. Castro P, Ishakian V, Muthusamy V, Slominski A. The rise of serverless computing. Commun ACM. Nov 21, 2019;62(12):44-54. [CrossRef]
    23. Aluvala R, Maheswari V U. Serverless Computing Concepts, Technology and Architecture. Hershey, PA. IGI Global; Apr 2024:1-19.
    24. Faul F, Erdfelder E, Buchner A, Lang A. Statistical power analyses using G*Power 3.1: tests for correlation and regression analyses. Behavior Research Methods. Nov 2009;41(4):1149-1160. [CrossRef]
    25. Sigle M, Berliner L, Richter E, van IM, Gorgati E, Hubloue I. Artificial patient database used for the publication "Development of an anticipatory triage-ranking algorithm using dynamic simulation of the expected time course of patients with trauma: modeling and simulation study". Zenodo. 2022. URL: https://zenodo.org/records/7134900 [accessed 2024-08-06]
    26. Sigle M, Berliner L, Richter E, van Iersel M, Gorgati E, Hubloue I, et al. Development of an anticipatory triage-ranking algorithm using dynamic simulation of the expected time course of patients with trauma: modeling and simulation study. J Med Internet Res. Jun 15, 2023;25:e44042. [FREE Full text] [CrossRef] [Medline]
    27. Heller AR, Neidel T, Klotz PJ, Solarek A, Kowalzik B, Juncken K, et al. Validation of secondary triage algorithms for mass casualty incidents-a simulation-based study-German version. Anaesthesiologie. Jul 2023;72(7):467-476. [FREE Full text] [CrossRef] [Medline]
    28. Kinsman T, Wessel M, Gerosa M, Treude C. How do software developers use GitHub actions to automate their workflows? 2021. Presented at: The 2021 Mining Software Repositories Conference; May 17-19, 2021; Virtual. URL: https://arxiv.org/abs/2103.12224 [CrossRef]
    29. Adzic G, Chatley R. Serverless computing: economic and architectural impact. New York City, NY. Association for Computing Machinery; 2017. Presented at: ESEC/FSE'17: Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering; September 4-8, 2017; Paderborn, Germany. [CrossRef]
    30. Rotem-Gal-Oz A. Fallacies of distributed computing explained. ResearchGate. 2008. URL: https://www.researchgate.net/publication/322500050_Fallacies_of_Distributed_Computing_Explained [accessed 2024-08-06]
    31. Laugwitz B, Held T, Schrepp M. Construction and evaluation of a user experience questionnaire. In: HCI and Usability for Education and Work. Berlin, Heidelberg. Springer; 2008:66-76.
    32. Allen M, Poggiali D, Whitaker K, Marshall TR, van Langen J, Kievit RA. Raincloud plots: a multi-platform tool for robust data visualization. Wellcome Open Res. 2019;4:63. [FREE Full text] [CrossRef] [Medline]
    33. Schrepp M, Hinderks A, Thomaschewski J. Construction of a benchmark for the User Experience Questionnaire (UEQ). IJIMAI. 2017;4(4):40. [CrossRef]
    34. Montano IH, de la Torre Díez I, López-Izquierdo R, Villamor MAC, Martín-Rodríguez F. Mobile triage applications: a systematic review in literature and play store. J Med Syst. Aug 13, 2021;45(9):86. [FREE Full text] [CrossRef] [Medline]
    35. Tian Y, Zhou T, Wang Y, Zhang M, Li J. Design and development of a mobile-based system for supporting emergency triage decision making. J Med Syst. Jun 2014;38(6):65. [CrossRef] [Medline]
    36. Follmann A, Ohligs M, Hochhausen N, Beckers SK, Rossaint R, Czaplik M. Technical support by smart glasses during a mass casualty incident: a randomized controlled simulation trial on technically assisted triage and telemedical app use in disaster medicine. J Med Internet Res. Jan 03, 2019;21(1):e11939. [FREE Full text] [CrossRef] [Medline]
    37. Sutham K, Khuwuthyakorn P, Thinnukool O. Thailand medical mobile application for patients triage base on criteria based dispatch protocol. BMC Med Inform Decis Mak. Apr 09, 2020;20(1):66. [FREE Full text] [CrossRef] [Medline]
    38. Nimmolrat A, Sutham K, Thinnukool O. Patient triage system for supporting the operation of dispatch centres and rescue teams. BMC Med Inform Decis Mak. Feb 19, 2021;21(1):68. [FREE Full text] [CrossRef] [Medline]
    39. Finch H. Comparison of the performance of nonparametric and parametric MANOVA test statistics when assumptions are violated. Methodology. Jan 2005;1(1):27-38. [CrossRef]

    ‎
    AWS: Amazon Web Services
    EMT: Emergency Medical Technician
    MANOVA: multivariate analysis of variance
    MCI: mass casualty incident
    mSTART: modified Simple Triage and Rapid Treatment
    REST: Representational State Transfer
    START: Simple Triage and Rapid Treatment
    TIDieR: Template for Intervention Description and Replication
    UEQ: User Experience Questionnaire

    Edited by A Mavragani; submitted 23.08.24; peer-reviewed by J Murphy; comments to author 23.09.24; revised version received 30.09.24; accepted 08.10.24; published 21.11.24.

    Copyright

    ©Martin Schmollinger, Jessica Gerstner, Eric Stricker, Alexander Muench, Benjamin Breckwoldt, Manuel Sigle, Peter Rosenberger, Robert Wunderlich. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 21.11.2024.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research (ISSN 1438-8871), is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.