Review
Abstract
Background: Digital health interventions (DHIs) are changing the dynamic of health care by providing personalized, private, and instantaneous solutions to end users. However, the explosion of digital health has been fraught with challenges. The approach to co-design with end users varies across a diverse domain of stakeholders, often resulting in siloed approaches with no clear consensus. The concept of validating user experiences contrasts greatly between digital stakeholders (ie, user experience and retention) and health stakeholders (ie, safety and efficacy). Several methodologies and frameworks are being implemented to address this challenge to varying degrees of success.
Objective: We aimed to broadly examine the advancements and challenges to co-design DHIs with end users over the last decade. This task was undertaken to identify the key problem areas at the domain level, with the ultimate goal of creating recommendations for better approaches to co-design DHIs with end users.
Methods: We conducted a systematic search of key databases for co-design studies involving end users in DHIs. Searches were divided into 3 relevant streams: health behavior, user experience, and digital methodologies and frameworks. The eligibility criteria were guided by the PerSPEcTiF framework and the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) checklist. In line with this framework, studies were included in this review that (1) address research on DHIs; (2) focus on interaction and co-design with end users; (3) explain results such that uptake, effectiveness, satisfaction, and health outcomes are discernible, positively or negatively; and (4) describe actionable procedures for better DHI design. The search was conducted in a diverse group of 6 bibliographical databases from January 2015 to May 2024: PsycINFO, PubMed (MEDLINE), Web of Science, CINAHL, Institute of Electrical and Electronics Engineers Xplore, and Scopus. From the 13,961 studies initially screened for titles and abstracts, 489 (3.6%) were eligible for a full-text screening, of which 171 (1.2%) studies matched the inclusion criteria and were included in a qualitative synthesis.
Results: Of the 171 studies analyzed across 52 journals, we found 5 different research approaches, spanning 8 different digital health solution types and 5 different design methodologies. These studies identified several core themes when co-designing with end users: advancements, which included participatory co-design; challenges, which included participatory co-design, environment and context, testing, and cost and scale; and gaps, which included a pragmatic hybridized framework and industry implementability.
Conclusions: This research supports a pragmatic shift toward using mixed methods approaches at scale, methods that are primed to take advantage of the emerging big data era of digital health co-design. This organic outlook should blend the vision of digital health co-designers with the pragmatism of Agile design methodology and the rigor of health care metrics.
Trial Registration: PROSPERO CRD42021238164; https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42021238164
International Registered Report Identifier (IRRID): RR2-10.2196/28083
doi:10.2196/50178
Keywords
Introduction
Background
Digital health interventions (DHIs) are revolutionizing health care delivery by providing personalized, private, and instantaneous solutions to end users. An umbrella term, a DHI is a mobile- or web-based digital solution that facilitates a diverse array of health interventions—be they emotional, decisional, or behavioral. A DHI is designed for a complex set of end users—those who use the intervention, including patients, caregivers, and health care professionals []. With the ever-increasing ubiquity of mobile health (mHealth) technologies, cost-effective scalable DHIs are emerging almost daily []. These solutions range from self-monitoring to diagnostic, educational, and behavior change–based interventions. At the heart of a DHI is the contextual adaptation [] around the needs of end users, those who use the technology directly to impact their health [], such as a patient with diabetes monitoring blood sugar on an mHealth app. This is an exciting yet challenging shift in health care design. It promises more autonomy to the end user through a bottom-up [] design approach; however, it ruptures the top-down [] traditional health care approach that positions experts as the central designers of interventions. This evolution marks a shift from visiting health facilities for health services toward the ubiquitous digital participation of end users and health experts. This has produced a new era of health literacy built around diagnostics, services, and self-help. This reciprocal, empowering approach is creating a more mutual understanding [], evolving the dynamic [] among health care professionals, engineers, designers, and end users in DHI co-design, ultimately shifting the balance of power toward the patient. Nonetheless, the promise of DHIs is tempered by the fact that 26% of apps are discarded after first use []. To address this concern, numerous design approaches such as user-centered and human-centered design, among others, are being used to varying degrees of success []. Each approach seeks to incorporate end users, from the primary stages of ideation to the final testing stages of production, gathered around multidisciplinary teams of digital and health care professionals. It is within this focus that we have examined the prominent plus points (ie, advancements), pain points (ie, challenges), and system-wide gaps to incorporate end users in co-design, issues that cut across the entire range of design approaches used in digital health at present.
The Complexity of Hybridizing Digital and Health Approaches to Incorporate End Users
Despite today’s digital sphere increasingly being one of digital natives, the path toward a fluid, organic digital health design space has been fraught with challenges. From a digital perspective, rapid Agile development methods evolve to garner end-user satisfaction in an evolving form of iterative qualitative evaluation. From a health care perspective, slow, safe expert-led studies produce large quantitative datasets toward validating patient involvement []. Digital health, in its nascent stages, sits somewhat uncomfortably between these two domains. From the macrolevel of policy down to the microlevel of co-design workshops, the very language of collaboration may differ. Hybridizing these 2 contrarian methodologies is challenging. The concept of building validation around the user or patient varies, depending on the given perspective. These challenges, along with others, have led to a multiplicity of design approaches, a disparity in conceptualizing the end user, and competing agendas [,,,] among DHI stakeholders that muddy the waters of digital health design. In this regard, we were inspired to conduct this systematic review on the basis of the initial analysis of our previous study titled, “The Challenges Toward Real-world Implementation of Digital Health Design Approaches” []. In our primary study, we found that irrespective of the domain, design methodology, or user definition (), there are common pain points (ie, repeated points of conflict or displeasure resulting in a poor user experience [UX]) emerging that impact digital health at the domain level.
In addition, numerous studies [,,,,] have noted the challenge to design frameworks that effectively define and implement “co-design” or “co-leadership” around end-user involvement in DHI design. On one hand, health stakeholders may lament the lack of alignment of UX design to health care delivery []; on the other hand, digital stakeholders may insist that user-centered design (UCD) is essential toward acceptability []. In both cases, each stakeholder is viewing efficacy through a different lens. This cuts to the heart of the matter of co-designing DHIs with end users; the overarching philosophy and methods used are starkly different. Therefore, there is a need to better understand the common pain points in this collaboration, which in turn may lead to a better perspective on why, when, and how to incorporate end users in the design of DHIs.
Objective of This Study
This study seeks to compile the core challenges to incorporate end users in the design of DHIs over the last decade. We sought to thematically analyze current approaches, to analyze evidence of pain points that presented a challenge or conflict in the incorporation of end users in the co-design process, and to explore gaps and opportunities toward a more organic co-design approach in digital health. This includes a broad stroke of studies, including quantitative, qualitative, mixed methods, and research through design. It features the perspectives of health care, psychology, engineering, development, and design in the production of mHealth, eHealth, and extended reality DHIs. This intentionally broad approach is reflective of the nascency of digital health as a field and the open-mindedness of this study, such that the divergence and convergence of methodologies and tools is viewed as a prosperous and necessary growing pain toward a more organic digital health approach to end users.
It is hoped that through critical analysis of these advancements, challenges, and gaps, synthesized findings from the following questions lead to an improved approach toward incorporating end users in DHI co-design:
- How are end users currently incorporated into iterative DHI design, and what is the effectiveness of current methods?
- What are the most common pain points (ie, challenges) encountered while incorporating end users in DHI design activities?
- What are the current gaps in end-user research at the domain level, and how can these be addressed to better integrate end users into the iterative design process in digital health, aiming to improve efficacy and uptake?
Methods
Study Design
This systematic review followed an a priori published protocol with detailed methods registered with PROSPERO (CRD42021238164) []. The review followed the Cochrane Handbook for Systematic Reviews [] and was reported using the PRISMA-P (Preferred Reporting Items for Systematic Reviews and Meta-Analyses Protocols) guidelines [].
We followed six stages in this systematic review: (1) literature search, (2) article selection, (3) data extraction, (4) quality appraisal, (5) data analysis, and (6) data synthesis.
Eligibility Criteria
Considering the complexity of DHIs, we used the PerSPEcTiF [] guidelines for intervention (
Table 1). We selected it due to its suitability for qualitative synthesis in the health care domain.
In line with this framework, studies were included in this review that (1) address research on DHIs; (2) focus on interaction and co-design with end users; (3) explain results such that uptake, effectiveness, satisfaction, and health outcomes are discernible, positively or negatively; and (4) describe actionable procedures for better DHI co-design.
| Initials | Definition | Detail |
| Per | Perspective | From the perspective of end users |
| S | Setting | In the setting of digital health |
| P | Phenomenon of interest or problem | What are the most prominent pain points |
| E | Environment | Within an environment of designing DHIsa |
| C | Comparison (optional) | —b |
| Ti | Time | During ideation and co-designing |
| F | Findings | In relation to understanding the challenges of successfully incorporating end users in the successful design of DHIs |
aDHI: digital health intervention.
bNot applicable.
Search Strategy
We systematically searched 6 electronic databases: PsycINFO, PubMed (MEDLINE), Web of Science, CINAHL, Institute of Electrical and Electronics Engineers Xplore, and Scopus. We selected these databases according to preliminary searches and consultations with experts and librarians in this field. Keywords related to DHIs from preliminary searches were compiled and refined. We adapted the search strategy as needed to return a breadth of papers without retrieving an unmanageably large number of irrelevant articles.
A list of the search terms used in this review were grouped into 3 clusters (Table 2). In addition, we have included all search strings for replicability ().
| Categories | Keywords |
| DHIsa | “digital health” or “mHealth” or “eHealth” and “end user*” or “end user*” |
| User experience | “user experience” or “UX” or “user centred” or “user centered” or “human centred” or “human centered” or “patient centred” or “patient centered” or “person based” or “person centered” or “person centred” or “participatory design” or “involvement” and “digital health” or “mHealth” or “eHealth” |
| Methodologies and frameworks | “agile” or “scrum” or “kanban” and “digital health” or “mHealth” or “eHealth” |
aDHI: digital health intervention.
Cluster 1 includes DHIs (owing to behavior change in health care resulting from DHIs). Cluster 2 includes UX (owing to engineering, development, and design in digital health). Cluster 3 includes methodologies and frameworks (owing to digital project facilitation and to health care intervention design and policy).
Inclusion and Exclusion Criteria
The inclusion and exclusion criteria were based on the peer-reviewed studies outlined in 
Textbox 1.
Inclusion criteria
- Qualitative studies
- Case studies
- Observational studies, including cross-sectional surveys
- Nonrandomized studies (eg, before and after studies and interrupted time series studies)
- Cohort studies
Exclusion criteria
- Noncritical analysis of end-user involvement in digital health design (ie, no results or discussion on end-user incorporation)
- Single person studies
- Nonprimary research (eg, reviews and commentaries)
- Randomized controlled trials
- Nonhuman studies
Due to the rapid innovation of digital health, only studies in the English language from January 1, 2015, to May 31, 2024, were included. Broadly put, the inclusion criteria included any population group that involved the critical analysis of end-user incorporation (ie, inclusive of terms such as patient, person, and human) in the co-design of a DHI. Studies that did not meet this criteria were excluded, including those that included end users but did not provide any critical analysis of the process; reviews and commentaries due to them being nonobservational or nonexperimental; single-user studies due to the inability to generalize; randomized controlled trials (RCTs) due to a lack of focus on the end-user integration in the design of a DHI; and nonhuman studies, which are impertinent to the design of a DHI.
Screening and Article Selection
All articles identified and selected from database searches were stored in the reference management software Paperpile (Paperpile LLC), which was used to eliminate duplicates and to tag and organize the research findings. The initial screening process involved 2 independent reviewers (AD and either IR or NB) who screened titles and abstracts of all studies for possible inclusion using an Excel (Microsoft Corp) spreadsheet ( and ) to ensure no studies met the exclusion criteria. After filtering results on the basis of the possible inclusion criteria, the second stage involved a full-text review of the remaining articles, which was conducted by 3 independent reviewers (AD and IR or AS). Upon scrutiny of the full-text selections, reviewers (AD and IR or AS) removed items that did not meet the inclusion criteria. In the third and final stage, reviewers (AD and IR or AS) met to cross-reference eligibility of the final inclusion list, as well as to settle any conflicts. This cross-reference involved the individual review of all items to white list them per the inclusion criteria. For items in conflict, reviewers (AD and IR or AS) simultaneously scanned items side by side to determine whether the items met the inclusion criteria. Since this was the third round of filtering, the majority of items met the inclusion criteria, so the determining factor in conflicts was to collectively review each item to see whether the studies actively involved end users in the design of the DHI and whether critical analysis of this was present in the results or discussion sections of the respective study. This final step removed any subjectivity and resolved conflict.
Data Extraction, Analysis, and Synthesis
A data extraction template was created by one of the authors (AD) that provided a record of the study characteristics. The categorization of themes and subthemes emerged organically from the findings (). The collection, extraction, and subsequent analysis and synthesis of data is outlined in Table 3.
Raw content (ie, quotes and reflective analysis) was extracted and collected in a research document (). Data were extracted and analyzed relative to the goals of the research questions, following guidance from the Cochrane Handbook for Systematic Reviews of Interventions []. These data are synthesized and presented in narrative form in the Results section. Full characteristics of all included end-user studies are available in .
| Category | Description | |||
| Data extraction | ||||
| Summary of characteristics | Full citation, author, PMIDa, year, and journal | |||
| Study type | Case study, quantitative study, qualitative study, research article, and mixed methods study | |||
| Study details | Target population, end users, duration (h), and solution | |||
| Design methodology | User-centered design, human-centered design, patient-centered design, person-based approach, patient-led design, and general participatory design. | |||
| Project facilitation method | Kanban, Scrumban, Waterfall, XPb, Agile inspired, design thinking, and unspecified | |||
| Data analysis and synthesis (themes and subthemes) | ||||
| Key plus points (ie, positives) | ||||
| Participatory co-design | Collaboration, ideation, improved mode of delivery, behavior modification, and in situ environment | |||
| Key pain points (ie, negatives) | ||||
| Participatory co-design | Collaboration, the language of digital health, co-design, competing interests, hierarchical versus democratic design, and methodological misalignment | |||
| Environment and context | Untested in situ and multiplicity of user types | |||
| Testing | Lack of participants, quality of participants, condensed time frame, lack of comprehension, technology, and cultural sensitivity | |||
| Cost and scale | ||||
| Challenges (ie, gaps in the research) | Organic co-design approaches, fluidity of digital health efficacy evaluation, Agile digital health methods, industry implementability, supply chain, medical device versus consumer app, and research and development limitations | |||
aPMID: PubMed identifier.
bXP: extreme programming.
Results
Search Results
The search strategy identified 13,961 studies. After removing duplicates (n=794), a total of 13,167 (94%) studies were screened for title and abstract. After the initial title and abstract screening phase, 12,678 (91%) studies were excluded, leaving 489 (4%) potentially relevant papers. In the second full-text screening phase, 318 (2%) studies were excluded. This resulted in the inclusion of 171 (1%) relevant studies in this systematic review. Reasons for exclusion included a lack of critical analysis of end-user involvement in digital health co-design and nonprimary research (eg, reviews and op-eds, RCTs, and single-user case studies). A PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) flow diagram (
Figure 1) was used to record the details of the screening and selection process so that the study can be reproduced.
Study Characteristics
Before addressing our research questions, we present a record of characteristics detailing the diversity of digital health studies retrieved (Table 4).
| Study characteristics | Studies, n (%) | |||||
| Year | ||||||
| 2015 | 4 (2.3) | |||||
| 2016 | 7 (4.1) | |||||
| 2017 | 14 (8.2) | |||||
| 2018 | 18 (10.5) | |||||
| 2019 | 25 (14.6) | |||||
| 2020 | 38 (22.2) | |||||
| 2021 | 10 (5.8) | |||||
| 2022 | 26 (15.2) | |||||
| 2023 | 24 (14) | |||||
| 2024 | 5 (2.9) | |||||
| Type and solution | ||||||
| Qualitative | 107 (62.6) | |||||
| Quantitative | 6 (3.5) | |||||
| Mixed methods | 43 (25.1) | |||||
| Case study | 12 (7) | |||||
| Research article | 4 (2.3) | |||||
| Methodology and methods (project facilitationa) | ||||||
| Agile | 24 (14) | |||||
| Scrum | 9 (5.3) | |||||
| Design thinking | 10 (5.8) | |||||
| Unspecified | 133 (77.8) | |||||
| Target population and end users | ||||||
| Patients | 91 (53.2) | |||||
| Older adults | 23 (13.4) | |||||
| Health professionals | 15 (8.8) | |||||
| Young people | 11 (6.4) | |||||
| Adults | 9 (5.3) | |||||
| Patients and health professionals | 5 (2.9) | |||||
| Other | 17 (9.9) | |||||
| Journal type | ||||||
| JMIR | 94 (55) | |||||
| BMCb | 11 (6.4) | |||||
| IEEEc | 6 (3.5) | |||||
| Other | 60 (35.1) | |||||
| Digital solution | ||||||
| mHealthd | 97 (56.7) | |||||
| Web based | 35 (20.5) | |||||
| eHealth | 22 (12.9) | |||||
| Device | 7 (4.1) | |||||
| Serious gaming | 6 (3.5) | |||||
| Other | 4 (2.4) | |||||
| Design methodologye | ||||||
| User-centered design | 84 (49.1) | |||||
| Participatory design | 58 (33.9) | |||||
| Human-centered design | 18 (10.5) | |||||
| Person-centered design | 9 (5.3) | |||||
| Person-based approach | 4 (2.3) | |||||
| Person-led design | 1 (0.6) | |||||
| Number of end users involved | ||||||
| 1-5 | 21 (12.3) | |||||
| 6-10 | 44 (25.7) | |||||
| 11-15 | 28 (16.4) | |||||
| 16-20 | 15 (8.8) | |||||
| 21-30 | 25 (14.6) | |||||
| 31-40 | 7 (4.1) | |||||
| 41-50 | 6 (3.5) | |||||
| ≥51 | 9 (5.3) | |||||
| Unspecified | 16 (9.4) | |||||
aThree studies identified with Scrum, which is a subset of Agile.
bBMC: Biomed Central.
cIEEE: Institute of Electrical and Electronics Engineers.
dmHealth: mobile health.
eThree studies identified more than one methodology.
Study Types
Owing to the diversity of digital health, the 171 studies identified span 5 different research approaches. Most approaches were qualitative (107/171, 62.6%), followed by mixed methods (43/171, 25.1%), indicative of the social and emotional influence on DHIs. These studies were collected from 52 different journals across multiple research domains, with JMIR accounting for the majority (94/171, 55%).
Study Details
The DHIs spanned 8 different solution types with mHealth (97/171, 56.7%) representing the majority. This development is in line with the growing dominance of mobile digital devices with up to 92% of internet traffic at present being mobile based [].
These solutions were used in 15 different populations with patient-focused populations being the most prominent (91/171, 53.2%). There was considerable variation in the number of end users involved in a DHI co-design with 6 to10 (44/171, 25.7%) end users being the most prominent, followed by 11 to15 (28/171, 16.4%), 21 to 30 (25/171, 14.6%), and 1 to 5 (21/171, 12.3%) end users. Only 9 studies (5.3%) involved >50 end users.
Study Design Methodology
As a guiding design methodology, UCD (84/171, 49.1%) was most prominent, followed by the broader approach of participatory design (58/171, 33.9%), then human-centered design (18/171, 10.5%). Other approaches such as patient-centered design, patient-based approach, and patient-led design were used sparingly.
Study Project Facilitation Method
Finally, in addition to tracking the design methodology, we tracked the project facilitation method used. The vast majority of studies did not specify a project facilitation method (133/171, 77.8%) that was used. Of those that did, Agile was used the most (24/171, 14%), followed by design thinking (10/171, 5.8%). Scrum, a subset of Agile, was next (9/171, 5.3%), while none specified other common industry methods such as Kanban, Extreme Programming, or Waterfall. This is particularly interesting when comparing how digital health prototypes are facilitated in academic research environments versus typical industry methods.
We present the aforementioned study characteristics to illustrate the diversity of investigating the incorporation of end users in co-design. As discussed, this rigorous approach sought to incorporate as many vantage points as possible to identify common advancements, challenges, and opportunities for improvement in the domain. A thematic analysis of the plus points (ie, advantages), pain points (ie, disadvantages), and challenges to incorporate end users in the co-design of DHIs (
Figure 2) as addressed in the 3 research questions is presented later.
A brief definition of the resultant themes and subthemes is presented in Table 5. This is followed by an explanation of the results relevant to each of the 3 research questions in this study.
| Section and theme | Description | Studies, n (%) | |||||||
| Plus points | |||||||||
| Theme: Participatory co-design | Positive advancements from the active participation of end users in the co-design workshop phases to ideate and test DHIsa | 95 (55.5) | |||||||
| Subthemes | |||||||||
| Collaboration | Collaboration among stakeholders (eg, developers, health professionals, and end users) in the co-design of a DHI | 69 (72.6) | |||||||
| Ideation | Creative process to construct ideas, requirements, and design concepts in the nascent stages of co-design with end users | 55 (57.9) | |||||||
| Immediate behavior modification | The benefits of instant behavior changes obtained by end users through the knowledge enhancing collaboration of DHI design workshops | 16 (16.8) | |||||||
| In situ environment | Ideating and testing with end users in the actual environment the intervention is designed for (ie, hospital, care homes, among others) | 7 (7.4) | |||||||
| Pain points | |||||||||
| Theme: Participatory co-design | Failures when incorporating end users in the active participation of co-design workshop phases to ideate and test DHIs | 70 (40.9) | |||||||
| Subthemes | |||||||||
| Collaboration challenges | Failures to curate and implement collaboration approaches among stakeholders (eg, developers, health professionals and end users) in the co-design of a DHI | 14 (20) | |||||||
| The language of digital health | Difficulties communicating a common language of co-design considering the various semantics that stem from differences in collaborator perspectives (ie, digital, health care, and end users) | 14 (20) | |||||||
| Cultural sensitivity | Understanding the nuances of culturally appropriate approaches that are used to describe the content and functionality of DHIs with end users | 13 (18.6) | |||||||
| Competing interests | Contrasting objectives defined by health and digital goals that create friction in the process of designing around the end users | 18 (25.7) | |||||||
| Methodological misalignment | Differences in digital health intervention design stemming from methodological misalignment (ie, digital, health or behavioral approaches to co-design) | 13 (18.6) | |||||||
| Hierarchical versus democratic design | The challenging contrast between the slow, steady, and data-driven approach to health care design and the laissez-faire rapid approach to digital design innovation, making processes and evaluation difficult in digital health | 17 (24.3) | |||||||
| Theme: Environment and context | The collaboration and testing environment of the end user and the contextual understanding of who the end user is | 38 (22.2) | |||||||
| Subthemes | |||||||||
| Untested in situ | The lack of co-designing or testing in the contextual environment of the user | 23 (60.5) | |||||||
| Multiplicity of user types | Lack of understanding of the complexity of user types (ie, patient, person, clinician, therapist, and caregiver) | 16 (42.1) | |||||||
| Theme: testing | Problems incurred during the testing phase with end-user collaborators that led to questions on the validity, reliability, and generalizability of results | 113 (66.1) | |||||||
| Subthemes | |||||||||
| Lack of participants | In reference to a lack of end-user testers leading to questions on the validity and reliability of results and determinations | 75 (66.4) | |||||||
| Quality of participants | In reference to a lack of broader representation of user types (ie, patient user, caregiver user, and clinician user), including demographic concerns (ie, ethnic diversity), thus weakening the generalizability of results | 58 (51.3) | |||||||
| Condensed timeframe and limited participation | The condensed time frame leading to limited participation of end users in collaboration workshops and testing, raising questions on rigor and applicability of results broadly | 25 (22.1) | |||||||
| Lack of technological comprehension | Inclusive of knowledge of technical terminology and digital device variance (ie, iPhone vs Android) | 14 (12.4) | |||||||
| Challenges | |||||||||
| Theme: Pragmatic hybridized frameworks | The need for new frameworks that are friction reducing, pragmatic (ie, fit for purpose), and hybridized (ie, a collective fluid combination of digital, health and behavioral objectives), leading toward more transdisciplinary approaches in DHI design | 60 (35.1) | |||||||
| Subthemes | |||||||||
| The value of mixed method approaches to end-user data | Despite the advantages that mixed methods approaches bring to DHIs (ie, blending emotional and functional feedback for better design decisions), the uptake in mixed methods approaches to data collection is low and inhibits more pragmatic approaches to DHI evaluation | 24 (40) | |||||||
| Organic co-design approaches | The challenge to cocreate approaches to end-user involvement in co-design that are not a sum-of-its-parts approach; siloing digital (ie, usability) and health care (ie, safety) elevation as unrelated metrics | 25 (41.7) | |||||||
| Fluid digital health efficacy evaluation | The challenge to amalgamate the fail fast approach of digital design and the first do no harm approach of health care design around end-user design involvement | 39 (65) | |||||||
| Agile digital health project facilitation | The difficulty to implement Agile industry-styled co-design practices with the rigor and requirements of health care safety and efficacy | 18 (30) | |||||||
| Theme: Industry implementability | The lack of consideration when co-designing with end users for downstream industry implementation within the complexity of the health care system (ie, government policy and industry constraints and regulations that impact design) | 13 (7.6) | |||||||
| Subthemes | |||||||||
| Medical device vs consumer app | The challenge to navigate consumer (ie, end user) preferences that collide with medical device regulation | 5 (38.5) | |||||||
| Supply chain | Health care delivery involves the collaboration of industry suppliers and government regulatory bodies that bring DHIs to the end users. The lack of health care business analysts in the co-design process with end users negates business modeling factors that could reduce downstream deployment challenges. | 6 (46.2) | |||||||
| Research and no development | The lack of recognition and resolution of the aforementioned failures and challenges often leads to prototypes that never reach the development or deployment stages | 5 (38.5) | |||||||
aDHI: digital health intervention.
Research Question 1: How Are End Users Currently Incorporated Into Iterative DHI Co-Design, and What Is the Effectiveness of Current Methods?
Theme: Participatory Co-Design
The most positive and influential approach to the incorporation of end users in the co-design of DHIs can broadly be termed as participatory co-design (ie, an approach wherein end users participate in the co-design process). In total, 95 of the 171 (55.5%) included studies discussed the advantages and advancements of participatory co-design in the digital health domain, irrespective of the domain approach (ie, engineering and health among others). This overarching methodological approach culminated in subthemes such as collaboration, ideation, environment, and improved behavior modification.
Subtheme: Collaboration
Collaboration (69/171, 40.4%) [,,,,-] was identified as a key benefit of participatory co-design. Studies explained that collaboration in DHI design allows complex teams to co-create dynamic solutions [] that uniquely combine a variety of digital and health approaches around the end user [,,]. Collaboration may take the form of interviews, design workshops, and crowdsourcing (ie, web-based surveys and mass testing). This constructive, creative, iterative [], and engaging process reflects the diverse mosaic of digital health stakeholders, while increasing the understanding of end-user needs due to the variety of vantage points []. Increasingly, studies showed that collaborating with a diverse stakeholder group around the end user, from ideation to production, increased end-user design participation [,,], improving evidence-based approaches []. This development is increasingly important, since earlier attempts to incorporate end users in late-production testing have been widely found to be ineffective []. Collaboration, particularly through the early low-fidelity stages of prototype design, increased adaptability of the solution in a low-cost phase []. It also created mutual understanding among all stakeholders regarding their roles and how they relate to the DHI as a sum of its parts rather than as siloed experts [,]. This approach is increasing sensitivity to the cultural context and personal preferences of end users, which in turn is improving the common language of communication in digital health solutions []. In short, collaboration is the element of participatory co-design that is closing the gap between the patient, clinician, and designer [].
Subtheme: Ideation
Ideation (55/171, 32.2%) was identified as a key benefit in participatory co-design. Loosely defined as the ability to generate new ideas [], studies used it as a hands-on approach to incorporate users within digital health teams in workshops [] to cocreate and prioritize end-user preferences [,] and subsequent features in a DHI. This process was used early in the DHI design process, often encapsulated in design sprints or health game jams [] that empathetically seek to understand end-user needs in a process of divergence and convergence [,], whereby finding differences is seen as a necessary building block on the path to agreeing on a robust co-designed solution.
Subtheme: Immediate Behavior Modification
Immediate behavior modification (16/171, 9%) [,,,,,,,,,,,-] in participating end users was identified as a beneficial yet perhaps unexpected advantage of in situ collaboration and ideation. Studies observed instant behavior modification from incorporating end users in DHI workshops. The multidisciplinary nature of rapid iteration that organically collects design, development, health care, and end-user experience data in a decentralized experimental environment allowed for persuasive design and behavior change in real time. This reciprocal process allowed digital and health stakeholders to educate end users on how a DHI can improve their experience, while end users were able to educate other stakeholders on how and what makes a positive UX for them.
Subtheme: In Situ Environment
Co-designing in situ (7/171, 4%) [,,,,,,] such that on-site collaboration with end users took place in hospitals, clinical environments, or homes [] increased the realism of stakeholder participation, leading to a more natural evaluation of the intervention. In such studies, this led to a better ability to co-conceptualize solutions []. This shift is in response to studies showing limitations on the quality of data and the potential effectiveness of DHIs that were commonly not tested in situ (eg, university laboratory environments), which challenges external validity [,]. In situ co-design increased familiarity of surroundings, enhanced comfort, and provided a more genuine, meaningful interaction with end users [].
Research Question 2: What Are the Most Common Pain Points (That Is, Challenges) Encountered While Incorporating End Users in DHI Design Activities?
Theme: Participatory Co-Design
Participatory co-design was simultaneously a core theme of the positive advancements as well as common failures to incorporate end users in the design of DHIs (70/171, 40.9%). This dichotomy is not unexpected considering the nascency of the burgeoning digital health industry. These challenges are related to collaboration, communication (ie, the language of digital health), competing interests, hierarchical versus democratic design approaches, and methodological misalignment. Broadly speaking, these challenges are growing pains toward blending digital and health care approaches in co-design.
Subtheme: Collaboration Challenges
Collaboration challenges (14/171, 8%) [,,,,,,,,,,-] in participatory co-design ranged from the challenge to curate collaboration approaches based on the given use-case scenario, to variance in stakeholders’ knowledge bases (ie, technology, health care, and UX), to miscalculating the need for onboarding co-designers in the workshop process, to mediating disagreements [], and misallocating stakeholder types (ie, too many clinicians, not enough end users, or vice versa) []. These factors led to disagreements, increased timelines, and overbudgeting that challenged the effectiveness of the co-design around the end user. These collaboration challenges are resultant of a clash of cultures (eg, digital and health care) and are defined by the following subthemes.
Subtheme: The Language of Digital Health Co-Design
Challenges to form a common language in co-design (14/171, 8%) [,,,,,,-,,-] identified the difficulty of blending the semantics of health care, digital design, and the colloquialisms of end users. Design “sprints” [] sometimes felt rushed to health care stakeholders and end users if unexplained. While safety may indicate imminent danger to end users, in health care, safety is more indicative of preventing harm from happening []. Conceptualizing what a prototype was or explaining what a minimum viable product represents in nascent design stages was not a given for nondigital stakeholders. Usability (ie, technical), UX (ie, emotional), and patient efficacy (ie, medical) are all by-products of the co-design process, but the language, interpretation, and implementation in the workshop meant different things to different stakeholders leading to conflicting understandings.
Subtheme: Cultural Sensitivity
Stemming from the common language issue, studies identified the challenge to collaborate with end users in a way that is culturally sensitive and appropriate (13/171, 7.6%) [,,,,,,,,-]). Cultural sensitivity has been observed in 4 main categories: content, functionality, technology, and user interface []. Studies cited the challenge of using inclusive digital media in workshops that is culturally befitting, as well as addressing sensory needs (eg, vision and hearing deficiencies) of patient end users []. For example, the use of gamified incentives and rewards in mHealth apps [] for communities that suffer from gambling addictions may be a critical oversight. Studies that incorporated cultural frameworks built with end users showed promise toward negating downstream challenges in the design cycle, particularly concerning appropriate content, features, and imagery in the interface design [,].
Subtheme: Competing Interests
Competing interests (18/171, 10.5%) [,,,,,,,,,,-,,-] were found to be a prominent challenge within participatory co-design with end users. This can be summarized as the focus on health systems and services versus the needs of matching digital technologies to user preferences []. These competing lenses [] led to contrasting perspectives on how to incorporate end users in DHI co-design. Broadly, it speaks to the clash of cultures as expert-led health care approaches and innovative user-driven approaches collide. The concept of “shared ownership” proved uncomfortable in numerous studies [,,,]. Although there was a shared vision on improving quality and safety around end users, exactly how this was achieved is a point of conflict []. Disease management, improved well-being, and UX are all facets of an intelligent design, yet they speak to different objectives []. Stakeholders often felt internal pressure to deliver aspects of the solution that reflect their domain (ie, digital, health care, and policy), which challenged the co- in co-design. Where a patient stakeholder may desire anonymous data tracking (eg, for privacy), a health care professional may desire mapped data tracking (eg, for safety and electronic records), and a digital stakeholder may need analytics (eg, to drive sales) []. The inability to corral these competing interests led to disengagement or abandonment of the collaboration in some cases [].
Subtheme: Methodological Misalignment
Ironically, a paradox exists in that the emergence of diverse stakeholder groups seeking to co-design with end users has led to difficulties in communicating and co-designing []. Methodological misalignment (13/171, 7.6%) [,,,,,,,,,-] in the approach to evaluating safety, usability, and efficacy differed vastly between health care and digital experts, respectively. The goals of these contrasting stakeholders were quite divergent []. In this regard, digital health is a domain struggling to formulate its own unique design culture around end users. Contextual barriers [,] exist such that the qualitative nature of rapid design sprints does not quantify the risks, safety, or efficacy of a longitudinal study, which spans large datasets (ie, RCTs). Simply put, it can be difficult to gain buy-in due to the vast variation in research approaches.
Subtheme: Hierarchical Versus Democratic Design
Methodological misalignment surfaced most prominently as a conflict between hierarchical and democratic concepts of designing DHIs (17/171, 9.94%) [,,,,,,,,,,,-]. Traditionally, a more paternalistic relationship [] exists between health care experts and patients (ie, end users). This hierarchical construct often relies on expert-driven “hard data” decisions. This is contrary to the more democratic, inclusive approach to digital design workshops, where the end user and other stakeholders are often seen as equal. The concept of leadership and followership, such that design workshops often see different stakeholders take the lead based on a given exercise [], is a new and rupturing approach to health care interventions. Evidence-based and experience-based approaches accentuated the juggling act of balancing perspectives and soothing egos. On the design side, certain biases related to the pragmatic mode of using user-driven exercises conflicted with evidence-based research in health care, especially in the arena of safety and clinical efficacy. Design workshop approaches to user personas and user journeys tell a story about a user and their preferences, but they do not account for clinical perspectives on harm prevention and safety. Where digital experts may insist “the solution must occur organically” around the end user, this democratic approach may be viewed as oversimplification by health experts who may insist that a “first do no harm” approach must involve quantitative data that exemplifies safety and efficacy. In this context, the aforementioned challenges culminate in the battle between innovation and clinical reality [].
Theme: Environment and Contextualization
After understanding the various challenges that participatory co-design poses to stakeholders working with end users in the design of a DHI, an emergent theme that developed was the impact of the collaboration environment and the contextualization of the end user (38/171, 22.2%).
Subtheme: Untested In Situ
Numerous studies observed the challenging impact of not designing a DHI with end users in situ (23/171, 13.4%) [,,,,,,,,,,,,,-]. In situ here refers to co-designing or testing in the contextual environment of the user. For example, a triage digital tablet in a waiting room or a sleep diary app in a user’s bedroom. This limited generalizability, challenged the value [] of design workshops, and weakened the ability to evaluate the true context of use [,,-,-]. Without testing DHIs in their natural environment, the context of the data collected was sometimes considered incomplete. Furthermore, it does not observe how end users would interact with the data and experiences they are creating, in real-world settings [,,]. For many mHealth solutions, app use was found to vary by time and location, with various environmental factors impacting how end users may interact with the intervention [,]. This made it increasingly difficult to generalize both design considerations and testing conclusions from a controlled laboratory environment. For digital stakeholders, this may render UX evaluation speculative. For health stakeholders, clinical efficacy evaluation may be incomplete.
Subtheme: Multiplicity of User Types
Another factor impacting the contextualization of the DHI was the multiplicity of user types (16/171, 9.4%) [,,,,-,,,,,,-]. End users were viewed as patients, persons, clinicians, therapists, and passive users (ie, an observer such as a caregiver), among others. Numerous DHIs had more than one end user, each using the solution for a different objective. Unlike traditional collaborative design workshops that involve 1 end-user type or a potential customer, mHealth solutions often provide services to numerous types of end users. For instance, in heart medication apps, a patient may objectify a knowledge base with push notifications and, as a user goal, a therapist may objectify behavior change as a user goal and a physician may objectify metrics to track medication history as a user goal—all 3 stakeholders may be end users of the same DHI []. Therefore, positioning the patient as the sole end user may insufficiently reflect the scope of end-user incorporation in the design process.
Theme: Testing
Foundational challenges in participatory co-design, inclusive of environmental and contextual issues, unsurprisingly culminated with pain points downstream in testing with end users (113/171, 66.1%). Testing subthemes included a lack of participants, the quality of participants, a lack of testing comprehension, and condensed time frames.
Subtheme: Lack of Participants
A lack of participants (ie, end users) challenged the generalizability of results (75/171, 43.8%) [,,,,,,,,,,,,-,-, ,,-,,,,,,,,,,,,, ,,,,,,-,,,-]. This limitation in the number of participants is a common challenge for co-design workshops []. The reliance on 5 test participants [,,] is a common practice in qualitative research, where findings from 5 end users are often assumed to reflect the experiences of 50 or 500 end users. This approach is foreign to traditionally quantifiable health research, and is admittedly speculative and potentially biased, as observed in many of the study limitations. Therefore, despite the richness in emotional feedback derived from and around end users in co-design workshops, the lack of quantifiable evidence limits and weakens the broad applicability of results []. Terms such as “sufficient” [] or “generally provable” were used alongside admissions of lack of generalizability. The rapid, cost-effective, human-centered approach [] to design workshops concedes larger sample sizes, edge cases, and an overall breadth of data obtained from large-scale patient trials in health care. Ironically, while aiming to use methods that address “the needs of end users,” [,,,,,,] these same methods inherently reduce the scope of end user needs, conceding large-scale feedback that could further advocate for design workshops emphasizing “acceptability” and “usability,” with large-scale, quantifiable validation from a broader pool of end users.
Subtheme: Lack of Diversity of Participants
The lack of diversity among participants (58/171, 33.9%) [,,,,,,,,-,-,,-,, ,-,,,,,,,,,,,,,-] was evident in a third of studies. Studies with small sample sizes were not always able to account for various cultural and educational user differences within a demographic [,,,,,,,,]. The lack of representability, both in variety of end-user types (eg, patient, clinician, caregiver, and others) and skillset (ie, tech savvy superusers [] vs standard users) contributed to consistent validity and generalizability concerns. With the clinician (ie, patient advisor), patient, and caregiver (ie, observer) often being co-users of the DHI, excluding one end-user group or more from the co-design or testing phases was seen as a risk of biasing results [,,]. The user test of a patient and that of a clinician may vary drastically due to contrary perspectives and user needs, despite them both being end users of the same DHI [,,].
Subtheme: Condensed Time Frame and Limited Participation
In some cases, the quantity and quality of participants was directly impacted by the rapid, condensed time frame (25/171, 14.6%) [,,,,,,-,,,,,,, ,,,,,,,-] of co-design workshops built around end users. Some studies recognized that despite the overtly flexible approach to iterating with end users, involving them alongside clinicians and digital stakeholders, from conception to validation, is a time-constrained process affected by availability and budget [,,,,,,,]. The challenge to coordinate health stakeholders’ involvement in end-user co-design was more apparent, considering that design workshops are not a part of the day-to-day workflow of health professionals []. In addition, patient health often dictated availability and digital teams often floated between multiple projects simultaneously []. The short time frames often resulted in only low-fidelity prototypes with a limited scope of input, often making it difficult to speculate whether higher fidelity finished products would garner the same feedback [].
Subtheme: Lack of Technological Comprehension
The previously mentioned pain point on the language of digital health co-design translated to further challenges downstream when testing ideas and prototypes due to a lack of technological comprehension (14/171, 8.2%) [,,,,,,-,,,,] among some stakeholders. Assumptions connecting personal experiences and digital health solutions were sometimes unvalidated, creating a disconnect among stakeholders [,]. In addition, familiarity with testing devices (eg, iPhone vs Android or extended reality interfaces) [] was not ubiquitous. The lack of perception and clarity of objectives sometimes intimidated end users [] and increased the difficulty in blending clinical, usability, and UX feedback in a constrained timeframe.
In summary, participatory co-design with DHI end users has highlighted challenges on the difficulty of amalgamating common interests, common language, and the collision of systematic and organic design approaches, culminating in methodological misalignments that produce varying testing approaches and hoped outcomes. This challenges the broader goal of defining a digital health framework around end-user involvement in co-design.
Research Question 3: What Are the Current Gaps in End-User Research at the Domain Level, and How Can These Be Addressed to Better Integrate End Users Into the Iterative Design Process in Digital Health, Aiming to Improve Efficacy and Uptake?
After assessing plus points and pain points in the first 2 research questions, our final question sought to understand the limitations and gaps that are challenging the advancement of end-user incorporation in the design of DHIs.
Theme: Pragmatic Hybridized Framework
Despite positive advancements in incorporating end users in digital health co-design, the various aforementioned flash points led to advocating for a more pragmatic hybridized digital health framework (60/171, 35%). Subthemes centered around the need for a more mixed method, adaptable, and contextual approach to incorporating end users; one that is a fluid, whole-of-system digital health approach.
Subtheme: The Value of Mixed-Method Approaches to End-User Data
Numerous studies identified taking a more mixed-method approach (2/171, 14%) [,,,,,,,,,,, ,,,,,,,-] as a cornerstone toward hybridizing digital and health approaches to incorporate end users. Unsurprisingly, mixed methods studies suffered less limitations in the form of validity and generalizability. Notably, studies found that there were discrepancies between what end users verbalized versus what was being observed from them [], indicative of a mismatch between end-user desires and end-user actions. This observation strengthened the need for other forms of quantitative assessment for correlation. Another study found that pooling larger datasets through the System Usability Scale [,] and weighing them against smaller qualitative datasets was seen as a more prudent method of testing assumptions and weighing feedback. A study by He et al [] found that end-user preferences for a tutorial (documented in qualitative interviews) was implemented, but they were subsequently found to be ignored by end users during quantitative data use analysis of the next iteration. Gaining metrics from hypothesis testing [] was seen as an emotionally detached compliment to qualitative workshop approaches to gain personal insights from end-user preferences and behavior. An additional benefit to this approach was comparing data between experiment and control groups []. In some cases, the “hard data” metrics from mHealth app use were used to support or reject findings from interviews and observations. This was strongly exemplified in a study by Fico et al [] in which a user-centered workshop design (ie, a qualitative approach) was complemented by bio-signal sensors to implement “a reliable information workflow.” In this regard, valuable emotional feedback from end users was cross-referenced with sensory data analytics for stronger conclusions [,]. In essence, the pragmatic, synthesized coevaluation of quantitative and qualitative data may only serve to strengthen the reliability of studies by blending quantifiable statistics with emotional insights from end-user analysis.
Subtheme: Organic Co-Design Approaches
Building upon the value of richer mixed method approaches to evaluating end-user co-design involvement, the need for more organic co-design approaches (25/171, 14.6%) [,,,,,,,,,-,,,,,,,,,,,,] was discussed in a number of studies. Organic, as in a holistic co-design approach, one that combines digital, health, and behavioral metrics into a more unified process. This is a response to the problem of misaligning usability and health care design objectives and the recognition that “end users” are not one single homogenous group []. Co-design workshops were found to have benefited from a funneling of methodologies and methods, whether qualitative or quantitative, that reflect digital (ie, technical and usability) and health care (ie, medical and psychoanalytical) around end-user involvement. Those studies that found this to be successful noted that it is only possible where diverse stakeholder involvement around the end users is consistent throughout the iterative design process [,,]. Values such as impartiality [] and interdisciplinarity [,] were identified as key elements toward hybridizing approaches. These values recognized that the needs of stakeholders and end users may be highly disparate []. Therefore, if acceptability [] is a ubiquitous term that satisfies digital goals of usability, health care goals of efficacy, and end-user goals of usefulness, then refining a co-design approach that is organically suited to the needs of the digital health ecosystem is paramount.
Subtheme: Fluid Digital Health Efficacy Evaluation
The call for a more organic, digital health co-design approach was backed by calls to integrate digital and health efficacy metrics into a singular framework for evaluation (39/171, 22.8%) [,,,,,,,,,,,,,, ,-,,,,-,,,,,,,, ,,,,,,-]. Various studies found a siloed approach that demarcated digital and health stakeholders was ineffective toward curating interventions. Studies recognized that design workshops did not always account for technical usability, behavior changes or patient engagement approaches that are all part of the digital health evaluation ecosystem [,,,,,,,]. Determining user motivation and confidence did not account for health system constraints [], and determining health safety did not account for user engagement. Essentially, studies found a need to bridge [,,,,] digital concepts such as increased usability, with health care concepts, such as reduced risk, in a workshop environment that fluidly and holistically facilitates both needs around the end user.
Subtheme: Agile Digital Health Project Facilitation
Numerous studies (18/171, 10.5%) [,,,,,,,,,,,,,,,,,] discussed the challenge to funnel digital and health co-design approaches into an adaptation of Agile methodology. Agile has been used in DHIs as a rapid design framework in response to legacy methods such as RCTs, which span years []. Agile uses short-cycle iterations (ie, sprints) with small sample sizes (ie, teams) [] producing mainly qualitative results. However, efforts to port Agile “out of the box” into digital health have been met with challenges. While Agile presents exciting methods to incorporate end users and produce rapid emotional responses, studies have found that this qualitative analysis struggles to incorporate the rigor of health care development []. For example, some studies found that in Agile design sprints, patient end users were unsure of what a health solution should look like []. This exemplified the difficulty of entirely democratizing Agile digital health workshops. This, coupled with the small sample size of Agile teams, raised questions on validity and generalizability []. Findings exemplified the need for having project leads, such as Scrum masters and product owners, who are familiar with health care contexts [] and can effectively blend digital health design sprints that better satisfy the needs of all stakeholders. In this context, evidence-based and theory driven approaches can meet around user needs and preferences. As Schwartz et al [] summarized in their study, “experimenting with agile adaptation in digital health will require a transparency of the development process, collecting shared difficulties and solutions toward a more standardized implementation suited to mHealth interventions.”
Theme: Industry Implementability
Industry implementability (13/171, 7.6%) [,,,,,,,,,,,,] represented downstream awareness of the need to develop a more fluid digital health hybridized design framework around end users. This is being fueled by the emerging policy shift toward the digital empowerment of patient end users. This new environment is further integrating government policy and administration, eHealth records, and encrypted end-to-end access through various collaborations between government bodies, industries, and end users (ie, citizens). Understanding the challenge to co-design for medical devices versus consumer devices was highlighted by studies that found conventional UCD processes hitting the wall of medical device regulation []. Nonetheless, medical regulation is part of the digital health ecosystem and needs to be considered in the ideation “design features” []. The needs of distributed health services were found to differ considerably with end-user preferences []. This reality addressed the complexity of propelling nascent stage DHI co-design toward the market. From industry suppliers to government regulatory bodies to end users, potential blockers in postdevelopment stages loomed large due to misalignment with clinical workflow and health policy [,]. Cooperation between public and private sectors to deliver DHIs added layers of bureaucracy []. In some cases, conceptual prototypes were not implementable in a given health care information technology ecosystem [] due to security or privacy policies. This challenge exposed a gap in the business model to identify critical success factors [] that are often missing in digital health stakeholder collaborations around end users.
Discussion
Principal Findings
This review examined the advancements and challenges of co-designing DHIs with end users, highlighting key gaps and limitations that cut through a wide range of design methodologies. The burgeoning growth of the digital health industry is a major advancement in health care delivery, one that is seeing exciting developments and major investment. Participatory co-design approaches that promote collaboration and ideation around the end users, often in situ, are promoting positive behavior change by seeking to absorb a diverse stakeholder group into a more fluid digital health ethos. These iterative design approaches are often centered around UCD methodologies and other similar approaches (eg, human-, person-, and patient- centered approaches) that emphasizes a more holistic empathetic approach, positioning the end users as coequal in the design phase (). Co-design is an exploratory process that experiences divergence and convergence. Through this disruptive process, the collective brain of a diverse stakeholder group is sought out in order to resolve conflicts and to amalgamate design needs. It is hoped that through these approaches, a common language and improved cultural context may form, leading to a better co-design outcome, one that provides both better efficacy and UX.
These promising developments are tempered by the sheer complexity of combining digital and health approaches into one funneled, organic design around the end user. The rapidity of co-design workshops, with their limited cohort size, has raised many questions about the validity and reliability of findings. While an RCT is the gold standard in health care validation, its timeframe often outlives the development of a digital solution []. In response to this, researchers are experimenting within the nascent stages of digital health design, porting a variety of research methodologies (ie, observational and experimental, whether qualitative or quantitative) and design methods (ie, Agile and behavior change, among others), mixing and matching concepts seeking out how best to validate DHIs with end users.
Exactly how this incorporation occurs is the flash point of conflict. Digital design perspectives that are heavily borrowed from the mainstream digital industry are rooted in UX approaches. Health design perspectives focus on long-term safety and efficacy (). While digital approaches are suited to rapid innovation, pivoting changes in real-time, health approaches are rooted in rigor and safety, implementing change slowly with longitudinal data. The variance between qualitative and quantitative approaches, UX and behavior change, and efficacy and usability accentuates the downstream challenges that occur in ideation, testing, and implementation of DHIs. This culture clash calls for pragmatism. Hybridizing digital and health approaches such that the demarcation between them eventually dissolves is the apex of DHI co-design. To better achieve this, the digital health domain must become comfortable with the uncomfortable, allowing the tensions of hierarchical health approaches to collide with flat-management digital approaches. Finding common ground between complaints of digital UX approaches that oversimplify health concerns and health approaches that are bureaucratic and time intensive will be paramount in creating an equilibrium that blends competing agendas.
Understanding the Advancements, Failures, and Challenges to Incorporate End Users in DHI Co-Design
Advancements
In response to the call for better contextual design [] to curate better health outcomes [] around end users, various digital, health, and behavioral approaches are being implemented in digital health. From a digital perspective, collaboration and ideation with end users is a response to traditional top-down waterfall approaches that backload testing and feedback late in the development process, when it is often too late to properly incorporate fundamental changes []. From a health care perspective, there is a push toward digital health citizens []; persons who take more control of their own health care, whether observing metrics, building their knowledge base, or proactively engaging health care professionals digitally. Our findings highlight that these 2 motivating factors are driving improvements in collaborative ideation and testing with end users in their natural environments, leading to improved behavior modification. These positive findings in participatory co-design are tunneling digital health toward a more organic design process with end users, one that is seeking to balance health care and purposeful design more fluidly. These positive developments are exemplifying theoretical concepts that advocate for more reciprocal, self-empowering approaches to co-design [] with end users, as digital health began to emerge a decade ago. Nonetheless, this is still very much a nascent exploratory stage for digital health, one that through the process of discovery continues to uncover new challenges.
Failures and Challenges
The findings in this study indicate that participatory co-design with DHI end users has interconnected challenges that underscore the difficulty of amalgamating common interests, common language among stakeholders, and both systematic and innovative design approaches. These challenges culminate in methodological misalignments that produce variance in ideating and testing, resulting in divergent goals. These themes complicate the broader objective of defining a digital health framework around end-user involvement in co-design. Optimistically, some studies focused on the core theme of pragmatism, seeking to implement digital health solutions around end users that are fit for purpose, albeit more theoretically than experimentally. To resolve the key challenges identified in the Introduction section, namely the multiplicity of design approaches, resolving disparate understandings of the end user, and dissolving competing agendas [,,,], DHIs require frameworks that are truly representative of all aspects of digital health delivery, breaking through the common silo disconnects. From the messiness of brainstorming ideation to the effectiveness of user testing right through the product implementation and system-wide distribution, an evolution is needed, one that provides broader, richer data points to better understand end-user incorporation and to provide better validation of iterative design decisions. The need for more data points and better testing methods is supported by the fact that 112 of 171 (65.5%) studies in this review encountered failures or challenges in testing phases citing either a lack of end users or the limited diversity among them. This speaks to the lack of data impacting digital health design decisions and its overarching philosophies.
The complexity of bringing DHIs to market cannot be accomplished solely with small, often speculative user studies. Downstream implementation and policy concerns can render years of research fruitless. New frameworks must emerge that are holistic and consider a global outlook on digital health distribution. Doing so may require evolving the digital health ecosystem by dissolving the tribalism of qualitative versus quantitative approaches, instead mixing methods pragmatically to better account for the sheer breadth of digital health success factors. Theoretically, hybridizing the 3 pillars—digital (eg, experience and usability), health (eg, medical and psychoanalytical), and business (eg, critical success factors)—may be crucial for better collectivizing design insights, improving industry implementability, and reducing downstream failures in product market fit. Practically, there is a need for co-designing frameworks and platforms that provide rapidly accessible, contextual, and demographic data to better justify iterative design decisions around end users. The end user is a person, a patient, a user, and a consumer. Their incorporation spans many personas and multiple user journeys that beget numerous types of data points relative to the evaluation of a DHI, from preconception to postdelivery; a “whole of system” co-design approach around health end users will be paramount to the future.
On the basis our findings (
Figure 3), we propose the core recommendations illustrated in subsequent sections.
Recommendation 1: Agile Health Design Workshops
Considering that Agile methodology is a project facilitation approach conceptualized outside of health care, unsurprisingly it has faced challenges when ported out of the box. To combine lightweight, rapid, qualitative Agile methods with more rigorous quantitative health methods, we recommend Agile workshops that triangulate emotional and observational feedback with statistical biosensor and analytical feedback that may soothe concerns about validity, generalizability, and attrition. This topic in itself could spawn a study on a mixed methods Agile framework design for digital health. Building on the calls for more Agile efficient approaches to digital health co-design suggested by Peiris et al [] and Wilson et al [], we propose the recommendations outlined in Textbox 2.
- Involving digital health agilists who have an overarching and empathetic view of the needs of digital, health, and end-user stakeholders. Coordinators of digital health projects (ie, Scrum masters and product owners) need to align the collective goals that are unique to digital health, ones that may not be solely reflected in traditional user experience design workshops.
- A preiteration sprint (ie, sprint 0) that involves creating shared digital health milestones and backlog grooming that combines the goals of health stakeholders (eg, clinical assessment, safety, and efficacy), digital stakeholders (eg, usability and technical validation), and end users (eg, user experience and acceptability). This preliminary approach may also account for discrepancies in technical ability and comprehension, as well as cultural sensitivities.
- From this, the selection of co-design activities that seek to combine these goals into an organic digital health workshop process may occur []. These activities should acquire metrics, whether qualitative or quantitative, that pragmatically fit the co-design objectives of a digital health team.
- Subsequent sprint retrospectives and sprint planning that continuously engages and reevaluates the collective goals of the digital health objectives and milestones, merging user experience validators with health and behavioral assessments.
Using the analogy of the toolbox, much like a carpenter must be pragmatic when selecting the given tools for the task at hand, digital health workshops must use a similar logic by hybridizing the methodologies, methods, and metrics that are fit for purpose []. Standardizing this process has proven difficult []. Shoehorning standard Agile practices into digital health continues to prove challenging, most notably in the struggle to validate studies with small participant groups in health care []. Therefore, there is a need to produce rapid quantifiable methods within the Agile design cycle, which points toward crowdsourcing ideation and testing data at scale. Charting a path that is reflective of collective co-designed digital health goals may reduce previously observed methodological misalignment, improving the flow of the co-design and building toward a more organic set of collective hoped outcomes. We do not suggest this as a magic elixir or standardized framework but as a flexible Agile health toolkit that seeks to amalgamate digital health approaches around the end user in the Agile workflow. This process must be tailored, balancing what participants say, do, imagine, test, and evolve, without being exhaustive []. As Schwartz et al [] summarized in their study, experimenting with Agile adaptation in digital health will require a transparency of the development process, collecting shared difficulties and solutions toward a more standardized implementation suited to mHealth interventions. As a result, this may reduce downstream challenges of divergent objectives, hoped outcomes and failed implementation.
Recommendation 2: Account for the Multiplicity of End-User Types With Broader, More Rapid Data Collection
While traditional UCD workshops hone in on the end user as a single entity, DHIs are often more complex, with multiple types of end users. With 112 of 171 studies (65.5%) in this review citing challenges in either the number or variance of end users involved, recent digital health studies highlight the drawbacks of traditional UCD workshops, particularly in health care, where the “user (ie, patient) does not always have answers” [] to clinical interventions []. The oversimplification of what a digital health end user is may deeply affect the integrity of the co-design. Considering that an mHealth app for heart medication may be used for tracking medication use; analytical data may serve multiple end users (eg, patient, caregiver, pharmacist, and clinician). Each type of end user has different goals, user experiences, and hoped outcomes. This reality was summarized well by Honary et al [] where they keenly recognized that relatives of mHealth end users have different roles and responsibilities in the DHI. This growing pain in digital health is coming to the forefront as a study by Mak et al [] advocated for mixed methods approaches that can better substantiate findings in diverse populations. To address this challenge and to account for the complexity of user types, broader pools of co-design data are needed; data that drill down on end-user demographics to create personas from rapid database-driven results. In this regard, a study by Tyo and Desroches [] has experimented with the quantification of user personas to analyze clusters to better inform design requirements. These emerging studies speak to the need for quantifiable digital co-design platforms that can be deployed at scale.
Recommendation 3: Implement a Leadership and Followership Approach to Co-Design Workshops
A key flash point in the co-design process is the concept of democratic versus hierarchical approaches. The bottom-up (ie, shared and innovative) approach of design workshops is diametrically opposed to the top-down (ie, systematic and preplanned) approach of health care. In our findings, the high value of “equal coleadership” [,,] was often championed as a guiding light in qualitative workshops. This has advantages and disadvantages, such that the democratic process brings many voices to the table, but this same process can eliminate evidence-based research from the solution []. Where criticism may exist of clinician-led studies being narrow minded, entirely democratic qualitative workshops can also be challenged for simplemindedness. For example, when a patient insists on a design preference [] that conflicts with a safety issue backed up by years of quantifiable empirical data collected by clinicians. As Mirkovic et al [] found, there is a need for equity over equality when it comes to workshop design with experts and end users. Undoubtedly, the value of a diverse array of stakeholders is of high importance; however, considering them all coequal through all stages of the design process does not serve to strengthen the design around the end user, rather it may weaken it if popular is preferred over practical, or entertainment over effectiveness. The plain democratization of the design process may lead to safety and efficacy concerns being voted off the island so to speak. Similarly, patient concerns with UX could be demoted since they are not central to behavior change or health policy goals.
The process of divergence and convergence in design workshops can instead benefit from a leadership and followership approach, such that design stakeholders lead the facilitation of UX objectives, health stakeholders lead the facilitation of safety and efficacy objectives, and patient stakeholders may lead discussion on their experiences and lifestyles [,,]. In each example, the collaborative approach is central; however, there is identifiable leadership and followership such that a given expertise is provided enough ownership [] to guide the workshop forward more effectively. This workshop approach may shift the perspective from coequals to coleaders, jointly seeking to learn from one another in a collaborative space. The goal of this approach would be to better balance person-centeredness, clinical acceptability, and UX [] without eliminating expertise through a democratic design process.
Recommendation 4: Collectivizing Digital, Health, and Behavioral Objectives
A more pragmatic flexible framework that better understands the complexity of end-user types, one that seeks to blend the hoped outcomes of a disparate group of stakeholders, by necessity may benefit from collectivizing digital, health, and behavioral metrics and goals. Of the 171 studies, 70 (40.9%) studies listed various co-design challenges in collaboration and alignment. As the study by Ledel et al [] found, both evidence-based (ie, health) and experience-based (ie, digital) approaches have merit for the end user, but friction exists when aligning these objectives. To reduce analysis in a vacuum and to align goals collectively, we suggest workshop co-design activities that blend objectives, such as safety (ie, health based) and usability (ie, digital based) can be blended into safe usability; behavior modification (ie, health based) and UX (ie, digital based) into user behavior; and efficacy (ie, health based) and uptake (ie, digital based) into effective uptake.
If the digital health end product is a fusion of digitally delivered health objectives, then the co-design process should suitably reflect this both philosophically and practically, aiming to reduce friction and combine goals and milestones. As Donovan et al [] summarized in their study, “DHIs do not exist in isolation; they are integrated into existing clinical pathways.” Considering that stakeholder disengagement and disparity in hoped outcomes is a frequent challenge, the aim of designing interviews, observations, and testing, whether qualitative or quantitative, that blend approaches would be to increase the fluidity of the co-design process with end users. This fluidity may reduce the challenge of balancing the innovation equilibrium; letting designers innovate and health stakeholders evaluate, with patient experiences provide the link between the two. This environment can better accommodate clinical reality, incorporating biomedical knowledge, clinical workflows, and overarching health organizational requirements [] around Agile practices that improve user collaboration, ideation, and rapid testing. This is very much an experimental process, but it begins with a change in thinking that can prevent implementation challenges downstream.
Recommendation 5: Include Business Case Modeling in the Co-Design Process for Downstream Implementability
Our final recommendation looks forward toward the downstream challenges of industry implementation and scalability. Rarely, in any of the studies observed, did we find co-design workshops that included health care policy or business stakeholders who addressed the clinical workflow challenges of delivering DHIs at scale to end users. Only 13 of 171 (7.6%) studies described the challenge of industry implementability, yet presumably every DHI project seeks to bring a viable product to market. In 2018, Hetrick et al [] found that despite the success of their mHealth youth co-design, barriers exist toward the implementation of the app in clinical care settings that factor various service and governance factors. This identified the need to include health care analysts and policy makers in the co-design phase. Elements of public and private sector collaboration are interwoven into the fabric of health care service delivery that have their own set of needs that uniquely affect the constraints of a DHI. This factor needs to be incorporated into the co-design process to prevent blue-sky prototypes being discarded. A study by Vial et al [] called for a transdisciplinary approach that includes business model definition coaching as a complement to the co-design team []. This accountability [] recognizes the final leg of a DHI, the health care delivery platform. The inability to recognize this prevents scalability [] and does not factor the ever-increasing integration of DHIs with government objectives to deliver ubiquitous services at scale.
Preparing for the Era of Digital Health Citizens
These 5 recommendations are responses to the challenges identified in this review, with an eye toward the emerging era of digital health citizenry, in which techno-sociality and bio-sociality are undemarcating the boundaries of digital health []. This “health citizen” agenda envisions digital health as more of a public good that provides tools, opportunities, responsibilities, and rights []. This transformation is built upon the prosumerist nature of today’s technologies, such that any given DHI may be an end-user experience of patients, clinicians, caregivers, therapists, and passive observers collectively, facilitated through algorithmically driven big data solutions (ie, artificial intelligence and machine learning) that are intuitive, customizable, and adaptable. The delivery of health care is increasingly blending government (eg, health records and policy), medical (eg, public and private sector solutions), and citizens (eg, end users) into one fluid infrastructure. This relationship is becoming increasingly reciprocal, more productive than facilitative in nature []. The potential for crowdsourcing data in tomorrow’s era of digital health seems boundless.
Needless to say, this vision needs to be brought to the forefront of the messier, creative stages of co-design workshops. The reliance on small qualitative datasets does not leverage the vastness of crowdsourcing end-user feedback that is equally capable of collecting quantifiable, demographically contextual statistics, while garnering emotional qualitative data in the form of asynchronous digital co-design platforms. If there is one word to define the evolution of digital health moving forward, it may be pragmatism. Leveraging the right tools to incorporate end users in the design of DHIs should mean removing any boundaries to the type of data collected. This means removing any allegiances to particular schools of thought on qualitative or quantitative, observational, or statistical, instead focusing on a pragmatic “patient activation” [], approach to garner positive involvement from end users at scale. There is no need for competing interests methodologically; a combination of metrics that increases the understanding of digital health users only serves to strengthen the confidence of design solutions, reducing attrition rates and increasing validity and generalizability. The digital health citizen is envisioned not as a user, an expert, or a novice, but as an active participant in the health care ecosystem, one who constantly learns and adapts to their health care needs. It is fluid and proactive rather than static and receptive. Therefore, there is a clear need to move the incorporation of end users in DHI co-design from small participant workshops objective to a dynamic socio-digital platform of ideating, testing, and evaluating in real time, at scale. Looking forward, this approach involves a new outlook that leverages the vision of the digital health citizen methodology, with the pragmatism of Agile tools, and the rigors of health care metrics. Experimenting with processes and tools that expand the range of data collection may better identify end-user types, better triangulate preferences and challenges, and provide stronger integration with health policies and implementation objectives downstream. Refocusing co-design approaches within this lens serves to better use the emerging potential of big data, potentially increasing the user perspectives of all stakeholders across the digital health spectrum, without sacrificing the rapidity and innovation of the design environment.
Strengths and Limitations
This systematic review drew from a large swath of studies, using various methodologies and techniques to incorporate end users in DHIs. The strength of this approach was identifying a large number of studies for analysis and highlighting key global issues that digital health stakeholders face, including points of failure and challenges. We used the PerSPEcTiF framework due to its suitability to qualitative studies in health care. However, many of these studies were limited in the number of participants involved, bringing the validity and reliability of findings from these studies into question. The largely qualitative approach to studies selected, with vast variance in data collection approaches, did not allow for sensitivity analysis on the basis of measurement criteria (ie, setting, duration, and demographics).
A limitation also exists in the generalizability of findings toward specific scenarios (ie, mental health and patient aids) in which domain-specific usages weigh heavily on the context of end-user incorporation. We recommend further research to investigate these themes as subsets within specific digital health communities (ie, mental health and serious gaming) In addition, in many cases, studies did not provide perspectives on risk of harm when using approaches to incorporate end users. This issue requires critical analysis itself. With digital health being a hybrid domain, there is no standardized approach to quality assessment, leaving the potential for risk of bias. These factors complicate data extraction and interpretation. This may be improved by using a health-centric qualitative framework in such studies, such as Confidence in the Evidence from Reviews of Qualitative Research [].
Conclusions
The challenge to incorporate end users in the co-design of DHIs is complicated by a variety of methodological influences on problem definition, testing approaches, and hoped outcomes. The porting of legacy Agile, behavioral, and analytical methods has often resulted in conflicts that weaken conclusions that can be drawn from the co-design. In this study, we sought to draw a line through the common pain points and gaps in the literature that are omnipresent regardless of the design approach. The subsequent recommendations stemming from this systematic review serve as guidelines that may help soothe the path toward the big data environment that is emerging through a socio-technical digital health citizen approach into the mid-21st century.
Acknowledgments
SM received funding from Simon Fraser University Community Trust Endowment Fund and AGE-WELL, Canada’s technology and aging network. AD received funding from the Graduate Dean’s Entrance Scholarship at the School of Interactive Arts and Technology at Simon Fraser University. The authors wish to thank Ms Adena Brons, liaison librarian: interactive arts and technology, Simon Fraser University Library, for guiding our search strategy.
Data Availability
All data generated or analyzed during this study are included in this published manuscript and its supplementary information files.
Authors' Contributions
The study protocol was conceived by AD, GC, and SM. AD prepared and drafted the manuscript. All authors provided input into the design, edits, and revision of the manuscript. All authors read and approved the final manuscript. AD is the guarantor of the review.
Conflicts of Interest
None declared.
Multimedia Appendix 1
A Venn diagram of the interdisciplinary intersection of digital health interventions.
PDF File (Adobe PDF File), 826 KBMultimedia Appendix 2
End users in design of digital health interventions: search string queries.
XLSX File (Microsoft Excel File), 69 KBMultimedia Appendix 3
End users in design of digital health interventions: screening template (2015 to 2021).
XLSX File (Microsoft Excel File), 973 KBMultimedia Appendix 4
End users in design of digital health interventions: screening template (2022 to May 31, 2024).
XLSX File (Microsoft Excel File), 7325 KBMultimedia Appendix 5
End users in design of digital health interventions: data extraction template for categorization of themes and codes.
XLSX File (Microsoft Excel File), 225 KBMultimedia Appendix 6
End users in design of digital health interventions: raw data content (ie, quotes and reflective analysis) extracted from eligible studies in the review.
PDF File (Adobe PDF File), 1075 KBMultimedia Appendix 7
Characteristics of included end-user studies.
XLSX File (Microsoft Excel File), 67 KBMultimedia Appendix 8
Comparison of evaluation in digital and health-based perspectives.
PDF File (Adobe PDF File), 705 KBMultimedia Appendix 9
PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) checklist.
PDF File (Adobe PDF File), 118 KBReferences
- Aref-Adib G, McCloud T, Ross J, O'Hanlon P, Appleton V, Rowe S, et al. Factors affecting implementation of digital health interventions for people with psychosis or bipolar disorder, and their family and friends: a systematic review. Lancet Psychiatry. Mar 2019;6(3):257-266. [CrossRef] [Medline]
- Murray E, Hekler EB, Andersson G, Collins LM, Doherty A, Hollis C, et al. Evaluating digital health interventions: key questions and approaches. Am J Prev Med. Nov 2016;51(5):843-851. [FREE Full text] [CrossRef] [Medline]
- Ospina-Pinillos L, Davenport TA, Navarro-Mancilla AA, Cheng VW, Cardozo Alarcón AC, Rangel AM, et al. Involving end users in adapting a Spanish version of a web-based mental health clinic for young people in Colombia: exploratory study using participatory design methodologies. JMIR Ment Health. Feb 08, 2020;7(2):e15914. [FREE Full text] [CrossRef] [Medline]
- Van Velsen L, Wentzel J, Van Gemert-Pijnen JE. Designing eHealth that matters via a multidisciplinary requirements development approach. JMIR Res Protoc. Jun 24, 2013;2(1):e21. [FREE Full text] [CrossRef] [Medline]
- Ector GI, Westerweel PE, Hermens RP, Braspenning KA, Heeren BC, Vinck OM, et al. The development of a web-based, patient-centered intervention for patients with chronic myeloid leukemia (CMyLife): design thinking development approach. J Med Internet Res. May 15, 2020;22(5):e15895. [FREE Full text] [CrossRef] [Medline]
- Kerkhof Y, Pelgrum-Keurhorst M, Mangiaracina F, Bergsma A, Vrauwdeunt G, Graff M, et al. User-participatory development of FindMyApps; a tool to help people with mild dementia find supportive apps for self-management and meaningful activities. Digit Health. Mar 26, 2019;5:2055207618822942. [FREE Full text] [CrossRef] [Medline]
- Ledel Solem IK, Varsi C, Eide H, Kristjansdottir OB, Børøsund E, Schreurs KM, et al. A user-centered approach to an evidence-based electronic health pain management intervention for people with chronic pain: design and development of EPIO. J Med Internet Res. Jan 21, 2020;22(1):e15889. [FREE Full text] [CrossRef] [Medline]
- Arvidsson S, Gilljam BM, Nygren J, Ruland CM, Nordby-Bøe T, Svedberg P. Redesign and validation of Sisom, an interactive assessment and communication tool for children with cancer. JMIR Mhealth Uhealth. Jun 24, 2016;4(2):e76. [FREE Full text] [CrossRef] [Medline]
- Curtis KE, Lahiri S, Brown KE. Targeting parents for childhood weight management: development of a theory-driven and user-centered healthy eating app. JMIR Mhealth Uhealth. Jun 18, 2015;3(2):e69. [FREE Full text] [CrossRef] [Medline]
- Duffy A, Christie GJ, Moreno S. The challenges toward real-world implementation of digital health design approaches: narrative review. JMIR Hum Factors. Sep 09, 2022;9(3):e35693. [FREE Full text] [CrossRef] [Medline]
- van Kessel K, Babbage DR, Kersten P, Drown J, Sezier A, Thomas PW, et al. Design considerations for a multiple sclerosis fatigue mobile app MS Energize: a pragmatic iterative approach using usability testing and resonance checks. Internet Interv. Apr 2021;24:100371. [FREE Full text] [CrossRef] [Medline]
- Mirkovic J, Jessen S, Kristjansdottir OB, Krogseth T, Koricho AT, Ruland CM. Developing technology to mobilize personal strengths in people with chronic illness: positive codesign approach. JMIR Form Res. Jun 05, 2018;2(1):e10774. [FREE Full text] [CrossRef] [Medline]
- Ravn Jakobsen P, Hermann AP, Søndergaard J, Wiil UK, Clemensen J. Development of an mHealth application for women newly diagnosed with osteoporosis without preceding fractures: a participatory design approach. Int J Environ Res Public Health. Feb 13, 2018;15(2):330. [FREE Full text] [CrossRef] [Medline]
- Latulippe K, Hamel C, Giroux D. Co-design to support the development of inclusive eHealth tools for caregivers of functionally dependent older persons: social justice design. J Med Internet Res. Nov 09, 2020;22(11):e18399. [FREE Full text] [CrossRef] [Medline]
- Patel D, Sarlati S, Martin-Tuite P, Feler J, Chehab L, Texada M, et al. Designing an information and communications technology tool with and for victims of violence and their case managers in San Francisco: human-centered design study. JMIR Mhealth Uhealth. Aug 24, 2020;8(8):e15866. [FREE Full text] [CrossRef] [Medline]
- Hetrick SE, Robinson J, Burge E, Blandon R, Mobilio B, Rice SM, et al. Youth codesign of a mobile phone app to facilitate self-monitoring and management of mood symptoms in young people with major depression, suicidal ideation, and self-harm. JMIR Ment Health. Jan 23, 2018;5(1):e9. [FREE Full text] [CrossRef] [Medline]
- Duffy A, Christie G, Moreno S. Examining challenges to the incorporation of end users in the design of digital health interventions: protocol for a systematic review. JMIR Res Protoc. Jul 26, 2021;10(7):e28083. [FREE Full text] [CrossRef] [Medline]
- Higgins JP, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, et al. Cochrane Handbook for Systematic Reviews of Interventions. 2nd edition. Hoboken, NJ. John Wiley & Sons; 2019.
- Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. Jul 21, 2009;6(7):e1000097. [FREE Full text] [CrossRef] [Medline]
- Booth A, Noyes J, Flemming K, Moore G, Tunçalp Ö, Shakibazadeh E. Formulating questions to explore complex interventions within qualitative evidence synthesis. BMJ Glob Health. Jan 25, 2019;4(Suppl 1):e001107. [FREE Full text] [CrossRef] [Medline]
- Moon D. Network traffic characteristics and analysis in recent mobile games. Appl Sci. Feb 08, 2024;14(4):1397. [FREE Full text] [CrossRef]
- Boyd AD, Moores K, Shah V, Sadhu E, Shroff A, Groo V, et al. My interventional drug-eluting stent educational app (MyIDEA): patient-centered design methodology. JMIR Mhealth Uhealth. Jul 02, 2015;3(3):e74. [FREE Full text] [CrossRef] [Medline]
- Abujarad F, Alfano S, Bright TJ, Kannoth S, Grant N, Gueble M, et al. Building an informed consent tool starting with the patient: the patient-centered virtual multimedia interactive informed consent (VIC). AMIA Annu Symp Proc. 2017;2017:374-383. [FREE Full text] [Medline]
- Brox E, Konstantinidis ST, Evertsen G. User-centered design of serious games for older adults following 3 years of experience with exergames for seniors: a study design. JMIR Serious Games. Jan 11, 2017;5(1):e2. [FREE Full text] [CrossRef] [Medline]
- Gabrielli S, Dianti M, Maimone R, Betta M, Filippi L, Ghezzi M, et al. Design of a mobile app for nutrition education (TreC-LifeStyle) and formative evaluation with families of overweight children. JMIR Mhealth Uhealth. Apr 13, 2017;5(4):e48. [FREE Full text] [CrossRef] [Medline]
- Harte R, Quinlan LR, Glynn L, Rodríguez-Molinero A, Baker PM, Scharf T, et al. Human-centered design study: enhancing the usability of a mobile phone app in an integrated falls risk detection system for use by older adult users. JMIR Mhealth Uhealth. May 30, 2017;5(5):e71. [FREE Full text] [CrossRef] [Medline]
- Allin S, Shepherd J, Tomasone J, Munce S, Linassi G, Hossain SN, et al. Participatory design of an online self-management tool for users with spinal cord injury: qualitative study. JMIR Rehabil Assist Technol. Mar 21, 2018;5(1):e6. [FREE Full text] [CrossRef] [Medline]
- Børøsund E, Mirkovic J, Clark MM, Ehlers SL, Andrykowski MA, Bergland A, et al. A stress management app intervention for cancer survivors: design, development, and usability testing. JMIR Form Res. Sep 06, 2018;2(2):e19. [FREE Full text] [CrossRef] [Medline]
- Jessen S, Mirkovic J, Ruland CM. Creating gameful design in mHealth: a participatory co-design approach. JMIR Mhealth Uhealth. Dec 14, 2018;6(12):e11579. [FREE Full text] [CrossRef] [Medline]
- Ahmed R, Toscos T, Rohani Ghahari R, Holden RJ, Martin E, Wagner S, et al. Visualization of cardiac implantable electronic device data for older adults using participatory design. Appl Clin Inform. Aug 18, 2019;10(4):707-718. [FREE Full text] [CrossRef] [Medline]
- Birnie KA, Campbell F, Nguyen C, Lalloo C, Tsimicalis A, Matava C, et al. iCanCope PostOp: user-centered design of a smartphone-based app for self-management of postoperative pain in children and adolescents. JMIR Form Res. Apr 22, 2019;3(2):e12028. [FREE Full text] [CrossRef] [Medline]
- Gonsalves PP, Hodgson ES, Kumar A, Aurora T, Chandak Y, Sharma R, et al. Design and development of the " POD adventures" smartphone game: a blended problem-solving intervention for adolescent mental health in india. Front Public Health. Aug 23, 2019;7:238. [FREE Full text] [CrossRef] [Medline]
- Jefferson UT, Zachary I, Majee W. Employing a user-centered design to engage mothers in the development of a mHealth breastfeeding application. Comput Inform Nurs. Oct 2019;37(10):522-531. [CrossRef] [Medline]
- Aufegger L, Bùi KH, Bicknell C, Darzi A. Designing a paediatric hospital information tool with children, parents, and healthcare staff: a UX study. BMC Pediatr. Oct 08, 2020;20(1):469. [FREE Full text] [CrossRef] [Medline]
- Cornet VP, Toscos T, Bolchini D, Rohani Ghahari R, Ahmed R, Daley C, et al. Untold stories in user-centered design of mobile health: practical challenges and strategies learned from the design and evaluation of an app for older adults with heart failure. JMIR Mhealth Uhealth. Jul 21, 2020;8(7):e17703. [FREE Full text] [CrossRef] [Medline]
- Davies A, Mueller J, Hennings J, Caress A, Jay C. Recommendations for developing support tools with people suffering from chronic obstructive pulmonary disease: co-design and pilot testing of a mobile health prototype. JMIR Hum Factors. May 15, 2020;7(2):e16289. [FREE Full text] [CrossRef] [Medline]
- Fico G, Martinez-Millana A, Leuteritz JP, Fioravanti A, Beltrán-Jaunsarás ME, Traver V, et al. User centered design to improve information exchange in diabetes care through eHealth : results from a small scale exploratory study. J Med Syst. Nov 18, 2019;44(1):2. [FREE Full text] [CrossRef] [Medline]
- Jessen S, Mirkovic J, Nes LS. MyStrengths, a strengths-focused mobile health tool: participatory design and development. JMIR Form Res. Jul 24, 2020;4(7):e18049. [FREE Full text] [CrossRef] [Medline]
- Slater H, Stinson JN, Jordan JE, Chua J, Low B, Lalloo C, et al. Evaluation of digital technologies tailored to support young people's self-management of musculoskeletal pain: mixed methods study. J Med Internet Res. Jun 05, 2020;22(6):e18315. [FREE Full text] [CrossRef] [Medline]
- Mendes-Santos C, Nunes F, Weiderpass E, Santana R, Andersson G. Development and evaluation of the usefulness, usability, and feasibility of iNNOV breast cancer: mixed methods study. JMIR Cancer. Feb 15, 2022;8(1):e33550. [FREE Full text] [CrossRef] [Medline]
- Liu S, La H, Willms A, Rhodes RE. A "No-Code" app design platform for mobile health research: development and usability study. JMIR Form Res. Aug 18, 2022;6(8):e38737. [FREE Full text] [CrossRef] [Medline]
- Adler RF, Morales P, Sotelo J, Magasi S. Developing an mHealth app for empowering cancer survivors with disabilities: co-design study. JMIR Form Res. Jul 26, 2022;6(7):e37706. [FREE Full text] [CrossRef] [Medline]
- Sunjaya AP, Martin A, Jenkins C. A design thinking approach to developing a clinical decision support system for breathlessness in primary care. Stud Health Technol Inform. Jun 06, 2022;290:839-843. [CrossRef] [Medline]
- Khowaja K, Waheeda Syed W, Singh M, Taheri S, Chagoury O, Al-Thani D, et al. A participatory design approach to develop visualization of wearable actigraphy data for health care professionals: case study in Qatar. JMIR Hum Factors. Apr 08, 2022;9(2):e25880. [FREE Full text] [CrossRef] [Medline]
- Preuhs K, van Keulen H, Andree R, Wins S, van Empelen P. A tailored web-based video intervention (ParentCoach) to support parents with children with sleeping problems: user-centered design approach. JMIR Form Res. Apr 19, 2022;6(4):e33416. [FREE Full text] [CrossRef] [Medline]
- Moltrecht B, Patalay P, Bear HA, Deighton J, Edbrooke-Childs J. A transdiagnostic, emotion regulation app (Eda) for children: design, development, and lessons learned. JMIR Form Res. Jan 19, 2022;6(1):e28300. [FREE Full text] [CrossRef] [Medline]
- Morse B, Soares A, Kwan BM, Allen M, Lee RS, Desanto K, et al. A transgender health information resource: participatory design study. JMIR Hum Factors. Jun 15, 2023;10:e42382. [FREE Full text] [CrossRef] [Medline]
- Mak WW, Ng SM, Leung FH. A web-based stratified stepped care platform for mental well-being (TourHeart+): user-centered research and design. JMIR Form Res. Mar 22, 2023;7:e38504. [FREE Full text] [CrossRef] [Medline]
- Brown M, Lord E, John A. Adaptation of ACTivate your wellbeing, a digital health and well-being program for young persons: co-design approach. JMIR Form Res. Apr 13, 2023;7:e39913. [FREE Full text] [CrossRef] [Medline]
- Yusupov I, Vandermorris S, Plunkett C, Astell A, Rich JB, Troyer AK. An agile development cycle of an online memory program for healthy older adults. Can J Aging. Dec 2022;41(4):647-656. [CrossRef] [Medline]
- Mueller EL, Cochrane AR, Campbell ME, Nikkhah S, Miller AD. An mHealth app to support caregivers in the medical management of their child with cancer: co-design and user testing study. JMIR Cancer. Mar 16, 2022;8(1):e33152. [FREE Full text] [CrossRef] [Medline]
- Lawrence K, Cho J, Torres C, Alfaro-Arias V. Building virtual health training tools for residents: a design thinking approach. Front Digit Health. Jun 13, 2022;4:861579. [FREE Full text] [CrossRef] [Medline]
- Fox S, Brown LJ, Antrobus S, Brough D, Drake RJ, Jury F, et al. Co-design of a smartphone app for people living with dementia by applying agile, iterative co-design principles: development and usability study. JMIR Mhealth Uhealth. Jan 14, 2022;10(1):e24483. [FREE Full text] [CrossRef] [Medline]
- O'Connor A, Tai A, Brinn M, Thuc Hien Hoang N, Cataldi D, Carson-Chahhoud K. Co-design of an augmented reality asthma inhaler educational intervention for children: development and usability study. JMIR Pediatr Parent. Jul 25, 2023;6:e40219. [FREE Full text] [CrossRef] [Medline]
- Taxter A, Johnson L, Tabussi D, Kimura Y, Donaldson B, Lawson E, et al. Co-design of an electronic dashboard to support the coproduction of care in pediatric rheumatic disease: human-centered design and usability testing. J Particip Med. Apr 22, 2022;14(1):e34735. [FREE Full text] [CrossRef] [Medline]
- Liang Z, Melcer E, Khotchasing K, Hoang NH. Co-design personal sleep health technology for and with university students. Front Digit Health. Apr 9, 2024;6:1371808. [FREE Full text] [CrossRef] [Medline]
- Ekstedt M, Kirsebom M, Lindqvist G, Kneck Å, Frykholm O, Flink M, et al. Design and development of an eHealth service for collaborative self-management among older adults with chronic diseases: a theory-driven user-centered approach. Int J Environ Res Public Health. Dec 30, 2021;19(1):391. [FREE Full text] [CrossRef] [Medline]
- Donovan G, Hall N, Smith F, Ling J, Wilkes S. Two-way automated text messaging support from community pharmacies for medication taking in multiple long-term conditions: human-centered design with nominal group technique development study. JMIR Form Res. Dec 21, 2022;6(12):e41735. [FREE Full text] [CrossRef] [Medline]
- Pfeuffer N, Beyer A, Penndorf P, Leiz M, Radicke F, Hoffmann W, et al. Evaluation of a health information exchange system for geriatric health care in rural areas: development and technical acceptance study. JMIR Hum Factors. Sep 15, 2022;9(3):e34568. [FREE Full text] [CrossRef] [Medline]
- Lundell S, Toots A, Sönnerfors P, Halvarsson A, Wadell K. Participatory methods in a digital setting: experiences from the co-creation of an eHealth tool for people with chronic obstructive pulmonary disease. BMC Med Inform Decis Mak. Mar 18, 2022;22(1):68. [FREE Full text] [CrossRef] [Medline]
- Vial S, Boudhraâ S, Dumont M, Tremblay M, Riendeau S. Developing a mobile app with a human-centered design lens to improve access to mental health care (mentallys project): protocol for an initial co-design process. JMIR Res Protoc. Aug 22, 2023;12:e47220. [FREE Full text] [CrossRef] [Medline]
- Tessarolo F, Petsani D, Conotter V, Nollo G, Conti G, Nikolaidou M, et al. Developing ambient assisted living technologies exploiting potential of user-centred co-creation and agile methodology: the CAPTAIN project experience. J Ambient Intell Human Comput. Apr 09, 2022;15(5):2661-2676. [FREE Full text] [CrossRef]
- McCann L, Hewitt C, McMillan KA. Developing an e-prehabilitation system of care for young adults diagnosed with cancer: user-centered design study. JMIR Cancer. Mar 30, 2023;9:e41441. [FREE Full text] [CrossRef] [Medline]
- Arigo D, Lobo AF, Ainsworth MC, Baga K, Pasko K. Development and initial testing of a personalized, adaptive, and socially focused web tool to support physical activity among women in midlife: multidisciplinary and user-centered design approach. JMIR Form Res. Jul 26, 2022;6(7):e36280. [FREE Full text] [CrossRef] [Medline]
- Schwartz-Lasfargues C, Roux-Gendron C, Edomskis P, Marque I, Bayon Y, Lange JF, et al. Development of a connected sensor system in colorectal surgery: user-centered design case study. JMIR Hum Factors. Jul 08, 2022;9(3):e31529. [FREE Full text] [CrossRef] [Medline]
- Turesson C, Liedberg G, Björk M. Development of a digital support application with evidence-based content for sustainable return to work for persons with chronic pain and their employers: user-centered agile design approach. JMIR Hum Factors. Mar 14, 2022;9(1):e33571. [FREE Full text] [CrossRef] [Medline]
- Østervang C, Lassen A, Schmidt T, Coyne E, Dieperink KB, Jensen CM. Development of a health information system to promote emergency care pathways: a participatory design study. Digit Health. Dec 26, 2022;8:20552076221145856. [FREE Full text] [CrossRef] [Medline]
- Ghorayeb A, Comber R, Gooberman-Hill R. Development of a smart home interface with older adults: multi-method co-design study. JMIR Aging. Jun 16, 2023;6:e44439. [FREE Full text] [CrossRef] [Medline]
- Johansen SK, Kanstrup AM, Thomsen JL, Christensen MN, Rathleff MS. Exploring the barriers and facilitators for supporting adolescents with knee pains adherence to mobile health apps: a think-aloud study. Digit Health. Oct 19, 2023;9:20552076231205750. [FREE Full text] [CrossRef] [Medline]
- Weber P, Mahmood F, Ahmadi M, Von Jan V, Ludwig T, Wieching R. Fridolin: participatory design and evaluation of a nutrition chatbot for older adults. i-com. 2023;22(1):33-51. [CrossRef]
- Holmqvist M, Ros A, Lindenfalk B, Thor J, Johansson L. How older persons and health care professionals co-designed a medication plan prototype remotely to promote patient safety: case study. JMIR Aging. Apr 07, 2023;6:e41950. [FREE Full text] [CrossRef] [Medline]
- Woodward M, Dixon-Woods M, Randall W, Walker C, Hughes C, Blackwell S, Thiscovery Authorship Group, et al. How to co-design a prototype of a clinical practice tool: a framework with practical guidance and a case study. BMJ Qual Saf. Mar 25, 2024;33(4):258-270. [FREE Full text] [CrossRef] [Medline]
- Isangula K, Pallangyo ES, Ndirangu-Mugo E. Interventions co-designed by healthcare providers and clients for improving therapeutic relationships in maternal and child healthcare: a pilot study using human centered design in rural Tanzania. BMC Nurs. Sep 14, 2023;22(1):314. [FREE Full text] [CrossRef] [Medline]
- Tran-Nguyen K, Berger C, Bennett R, Wall M, Morin SN, Rajabiyazdi F. Mobile app prototype in older adults for postfracture acute pain management: user-centered design approach. JMIR Aging. Oct 17, 2022;5(4):e37772. [FREE Full text] [CrossRef] [Medline]
- Sönnerfors P, Skavberg Roaldsen K, Lundell S, Toots A, Wadell K, Halvarsson A. Preferences for an eHealth tool to support physical activity and exercise training in COPD: a qualitative study from the viewpoint of prospective users. BMC Pulm Med. Feb 13, 2023;23(1):65. [FREE Full text] [CrossRef] [Medline]
- Young SR, Lattie EG, Berry AB, Bui L, Byrne GJ, Yoshino Benavente JN, et al. Remote cognitive screening of healthy older adults for primary care with the MyCog mobile app: iterative design and usability evaluation. JMIR Form Res. Jan 10, 2023;7:e42416. [FREE Full text] [CrossRef] [Medline]
- De Sutter E, Geerts D, Yskout K, Verreydt S, Borry P, Barbier L, et al. Testing and practical implementation of a user-friendly personalized and long-term electronic informed consent prototype in clinical research: mixed methods study. J Med Internet Res. Dec 19, 2023;25:e46306. [FREE Full text] [CrossRef] [Medline]
- Deady M, Collins D, Gayed A, Harvey SB, Bryant R. The development of a smartphone app to enhance post-traumatic stress disorder treatment in high-risk workers. Digit Health. Feb 22, 2023;9:20552076231155680. [FREE Full text] [CrossRef] [Medline]
- Webb MM, Bridges P, Aruparayil N, Chugh C, Beacon T, Singh T, et al. The RAIS device for global surgery: using a participatory design approach to navigate the translational pathway to clinical use. IEEE J Transl Eng Health Med. 2022;10:3700212-3700212. [FREE Full text] [CrossRef] [Medline]
- Yau KW, Tang TS, Görges M, Pinkney S, Amed S. Using human-centered design and cocreation to create the live 5-2-1-0 mobile app to promote healthy behaviors in children: app design and development. JMIR Pediatr Parent. May 17, 2023;6:e44792. [FREE Full text] [CrossRef] [Medline]
- Ludlow K, Russell JK, Ryan B, Brown RL, Joynt T, Uhlmann LR, et al. Co-designing a digital mental health platform, "Momentum", with young people aged 7-17: a qualitative study. Digit Health. Nov 28, 2023;9:20552076231216410. [FREE Full text] [CrossRef] [Medline]
- van de Riet L, Aris AM, Verouden NW, van Rooij T, van Woensel JB, van Karnebeek CD, et al. Designing eHealth interventions for children with complex care needs requires continuous stakeholder collaboration and co-creation. PEC Innov. Dec 2024;4:100280. [FREE Full text] [CrossRef] [Medline]
- Dahildahil RO, Isip-Tan IT, Marcelo PG, Grepo LC. User-centered design in time and resource-limited settings: enhancing the usability of 'hearing for life' (HeLe) device. Acta Med Philipp. Sep 27, 2023;57(9):39-53. [FREE Full text] [CrossRef] [Medline]
- Bonet-Olivencia S, Carrillo-Leal J, Rao A, Sasangohar F. User-centered design of a diabetes self-management tool for underserved populations. J Diabetes Sci Technol. Jan 18, 2024;18(1):22-29. [CrossRef] [Medline]
- Reimer LM, Nissen L, von Scheidt M, Perl B, Wiehler J, Najem SA, et al. User-centered development of an mHealth app for cardiovascular prevention. Digit Health. 2024;10:20552076241249269. [FREE Full text] [CrossRef] [Medline]
- Appleton KM, Bray J, Price S, Liebchen G, Jiang N, Mavridis I, et al. A mobile phone app for the provision of personalized food-based information in an eating-out situation: development and initial evaluation. JMIR Form Res. Nov 04, 2019;3(4):e12966. [FREE Full text] [CrossRef] [Medline]
- van den Berg LN, Hallensleben C, Chavannes NH, Versluis A. Developing a smartphone application that promotes responsible short-acting Beta2-agonist use in People with asthma: a participatory design. Int J Environ Res Public Health. Jul 12, 2022;19(14):63. [FREE Full text] [CrossRef] [Medline]
- Donald M, Beanlands H, Straus SE, Smekal M, Gil S, Elliott MJ, et al. A web-based self-management support prototype for adults with chronic kidney disease (my kidneys my health): co-design and usability testing. JMIR Form Res. Feb 09, 2021;5(2):e22220. [FREE Full text] [CrossRef] [Medline]
- Ehrler F, Lovis C, Blondon K. A mobile phone app for bedside nursing care: design and development using an adapted software development life cycle model. JMIR Mhealth Uhealth. Apr 11, 2019;7(4):e12551. [FREE Full text] [CrossRef] [Medline]
- Humphries SM, Rondung E, Norlund F, Sundin Ö, Tornvall P, Held C, et al. Designing a web-based psychological intervention for patients with myocardial infarction with nonobstructive coronary arteries: user-centered design approach. J Med Internet Res. Sep 17, 2020;22(9):e19066. [FREE Full text] [CrossRef] [Medline]
- Andersson SR, Hassanen S, Momanyi AM, Onyango DK, Gatwechi DK, Lutukai MN, et al. Using human-centered design to adapt supply chains and digital solutions for community health volunteers in nomadic communities of Northern Kenya. Glob Health Sci Pract. Mar 15, 2021;9(Suppl 1):S151-S167. [FREE Full text] [CrossRef] [Medline]
- Fleisher L, Bass SB, Shwarz M, Washington A, Nicholson A, Alhajji M, et al. Using theory and user-centered design in digital health: the development of the mychoice communication tool to prepare patients and improve informed decision making regarding clinical trial participation. Psychooncology. Jan 02, 2020;29(1):114-122. [CrossRef] [Medline]
- Saparamadu AA, Fernando P, Zeng P, Teo H, Goh A, Lee JM, et al. User-centered design process of an mHealth app for health professionals: case study. JMIR Mhealth Uhealth. Mar 26, 2021;9(3):e18079. [FREE Full text] [CrossRef] [Medline]
- Ahtinen A, Andrejeff E, Vuolle M, Väänänen K. Walk as you work: user study and design implications for mobile walking meetings. In: Proceedings of the 9th Nordic Conference on Human-Computer Interaction. 2016. Presented at: NordiCHI '16; October 23-27, 2016:1-10; Gothenburg, Sweden. URL: https://dl.acm.org/doi/10.1145/2971485.2971510 [CrossRef]
- Balli F. Developing digital games to address airway clearance therapy in children with cystic fibrosis: participatory design process. JMIR Serious Games. Nov 21, 2018;6(4):e18. [FREE Full text] [CrossRef] [Medline]
- Tyo MB, Desroches ML. Content analysis of qualitative interviews for user-centered design of a prototype mobile health app for direct support professionals' resilience. Disabil Health J. Apr 2024;17(2):101544. [FREE Full text] [CrossRef] [Medline]
- Shanmuganathan C, Anish TP, Thamizharasi M, Joseph M. An iterative constraint spectral model for ophthalmic disease detection using transfer learning. In: Proceedings of the 2022 International Conference on Data Science, Agents & Artificial Intelligence. 2022. Presented at: ICDSAAI '22; December 8-10, 2022:1-7; Chennai, India. URL: https://ieeexplore.ieee.org/document/10028807 [CrossRef]
- Schwartz LA, Psihogios AM, Henry-Moss D, Daniel LC, Ver Hoeve ES, Velazquez-Martin B, et al. Iterative development of a tailored mHealth intervention for adolescent and young adult survivors of childhood cancer. Clin Pract Pediatr Psychol. Mar 2019;7(1):31-43. [FREE Full text] [CrossRef] [Medline]
- Ortet C, Vale Costa L. "Listen to your immune system when it's calling for you": monitoring autoimmune diseases using the iShU app. Sensors (Basel). May 18, 2022;22(10):3834. [FREE Full text] [CrossRef] [Medline]
- Ramsey AT, Bray M, Acayo Laker P, Bourdon JL, Dorsey A, Zalik M, et al. Participatory design of a personalized genetic risk tool to promote behavioral health. Cancer Prev Res (Phila). Jul 2020;13(7):583-592. [FREE Full text] [CrossRef] [Medline]
- Nota I, Drossaert CH, Melissant HC, Taal E, Vonkeman HE, Haagsma CJ, et al. Development of a web-based patient decision aid for initiating disease modifying anti-rheumatic drugs using user-centred design methods. BMC Med Inform Decis Mak. Apr 26, 2017;17(1):51. [FREE Full text] [CrossRef] [Medline]
- Stratton E, Choi I, Peters D, Calvo RA, Harvey SB, Glozier N. Co-designing a web-based decision aid tool for employees disclosure of mental health conditions: a participatory study design using employee and organizational preferences. JMIR Form Res. Nov 06, 2020;4(11):e23337. [FREE Full text] [CrossRef] [Medline]
- van Leersum CM, Moser A, van Steenkiste B, Reinartz M, Stoffers E, Wolf JR, et al. What matters to me - a web-based preference elicitation tool for clients in long-term care: a user-centred design. BMC Med Inform Decis Mak. Mar 17, 2020;20(1):57. [FREE Full text] [CrossRef] [Medline]
- Poncette AS, Mosch LK, Stablo L, Spies C, Schieler M, Weber-Carstens S, et al. A remote patient-monitoring system for intensive care medicine: mixed methods human-centered design and usability evaluation. JMIR Hum Factors. Mar 11, 2022;9(1):e30655. [FREE Full text] [CrossRef] [Medline]
- Engdahl P, Svedberg P, Lexén A, Tjörnstrand C, Strid C, Bejerholm U. Co-design process of a digital return-to-work solution for people with common mental disorders: stakeholder perception study. JMIR Form Res. Jan 18, 2023;7:e39422. [FREE Full text] [CrossRef] [Medline]
- Verhoeven F, Cremers A, Schoone M, Van Dijk J. Mobiles for mobility: participatory design of a ‘happy walker’ that stimulates mobility among older people. Gerontechnology. 2016;15(1):32-44. [FREE Full text] [CrossRef]
- Couture B, Lilley E, Chang F, DeBord Smith A, Cleveland J, Ergai A, et al. Applying user-centered design methods to the development of an mHealth application for use in the hospital setting by patients and care partners. Appl Clin Inform. Apr 09, 2018;9(2):302-312. [FREE Full text] [CrossRef] [Medline]
- Rai HK, Schneider J, Orrell M. An individual cognitive stimulation therapy app for people with dementia: development and usability study of thinkability. JMIR Aging. Nov 16, 2020;3(2):e17105. [FREE Full text] [CrossRef] [Medline]
- Schiro J, Pelayo S, Martinot A, Dubos F, Beuscart-Zéphir MC, Marcilly R. Applying a human-centered design to develop a patient prioritization tool for a pediatric emergency department: detailed case study of first iterations. JMIR Hum Factors. Sep 04, 2020;7(3):e18427. [FREE Full text] [CrossRef] [Medline]
- Abubakar Kawu A, Kahuhu Koruhama A, Arinze Ikwunne T, Pretorius C. Gamifying the “unspoken”: designing to resolve myths on contraceptives among the Himba people of Namibia. In: Proceedings of the 22nd International Conference on Human-Computer Interaction with Mobile Devices and Services. 2020. Presented at: MobileHCI '20; October 5-8, 2020:1-4; Oldenburg, Germany. URL: https://dl.acm.org/doi/10.1145/3406324.3424592 [CrossRef]
- Tobias G, Spanier AB. Developing a mobile app (iGAM) to promote gingival health by professional monitoring of dental selfies: user-centered design approach. JMIR Mhealth Uhealth. Aug 14, 2020;8(8):e19433. [FREE Full text] [CrossRef] [Medline]
- Tonkin E, Jeffs L, Wycherley TP, Maher C, Smith R, Hart J, et al. A smartphone app to reduce sugar-sweetened beverage consumption among young adults in Australian remote indigenous communities: design, formative evaluation and user-testing. JMIR Mhealth Uhealth. Dec 12, 2017;5(12):e192. [FREE Full text] [CrossRef] [Medline]
- Ospina-Pinillos L, Davenport T, Mendoza Diaz A, Navarro-Mancilla A, Scott EM, Hickie IB. Using participatory design methodologies to co-design and culturally adapt the Spanish version of the mental health eClinic: qualitative study. J Med Internet Res. Aug 02, 2019;21(8):e14127. [FREE Full text] [CrossRef] [Medline]
- Hou M, Hindriks K, Eiben AE, Baraka K. A process-oriented framework for robot imitation learning in human-centered interactive tasks. In: Proceedings of the 32nd IEEE International Conference on Robot and Human Interactive Communication. 2023. Presented at: RO-MAN '23; August 28-31, 2023:1745-1752; Busan, Korea. URL: https://ieeexplore.ieee.org/document/10309326 [CrossRef]
- Wannheden C, Revenäs Å. How people with Parkinson’s disease and health care professionals wish to partner in care using eHealth: co-design study. J Med Internet Res. Sep 21, 2020;22(9):e19195. [FREE Full text] [CrossRef] [Medline]
- Giles SJ, Reynolds C, Heyhoe J, Armitage G. Developing a patient-led electronic feedback system for quality and safety within Renal PatientView. J Ren Care. Mar 19, 2017;43(1):37-49. [FREE Full text] [CrossRef] [Medline]
- Bevan Jones R, Thapar A, Rice F, Beeching H, Cichosz R, Mars B, et al. A web-based psychoeducational intervention for adolescent depression: design and development of MoodHwb. JMIR Ment Health. Feb 15, 2018;5(1):e13. [FREE Full text] [CrossRef] [Medline]
- Bradway M, Morris RL, Giordanengo A, Årsand E. How mHealth can facilitate collaboration in diabetes care: qualitative analysis of co-design workshops. BMC Health Serv Res. Nov 30, 2020;20(1):1104. [FREE Full text] [CrossRef] [Medline]
- Naeemabadi M, Søndergaard JH, Klastrup A, Schlünsen AP, Lauritsen RE, Hansen J, et al. Development of an individualized asynchronous sensor-based telerehabilitation program for patients undergoing total knee replacement: participatory design. Health Informatics J. Dec 2020;26(4):2492-2511. [FREE Full text] [CrossRef] [Medline]
- Kim JE, Lee J, Kwon H, Bessho M, Sakamura K. Towards supporting childhood atopic dermatitis management: a user-centered design approach. In: Proceedings of the 33rd International Symposium on Computer-Based Medical Systems. 2020. Presented at: CBMS '20; July 28-30, 2020:368-373; Rochester, MN. URL: https://ieeexplore.ieee.org/document/9183058 [CrossRef]
- Allsop MJ, Johnson O, Taylor S, Hackett J, Allen P, Bennett MI, et al. Multidisciplinary software design for the routine monitoring and assessment of pain in palliative care services: the development of PainCheck. JCO Clin Cancer Inform. Sep 2019;3:1-17. [FREE Full text] [CrossRef] [Medline]
- Schleimer E, Pearce J, Barnecut A, Rowles W, Lizee A, Klein A, et al. A precision medicine tool for patients with multiple sclerosis (the Open MS BioScreen): human-centered design and development. J Med Internet Res. Jul 06, 2020;22(7):e15605. [FREE Full text] [CrossRef] [Medline]
- Terp M, Laursen BS, Jørgensen R, Mainz J, Bjørnes CD. A room for design: through participatory design young adults with schizophrenia become strong collaborators. Int J Ment Health Nurs. Dec 2016;25(6):496-506. [CrossRef] [Medline]
- Carr EC, Babione JN, Marshall D. Translating research into practice through user-centered design: an application for osteoarthritis healthcare planning. Int J Med Inform. Aug 2017;104:31-37. [FREE Full text] [CrossRef] [Medline]
- Hesseldal L, Christensen JR, Olesen TB, Olsen MH, Jakobsen PR, Laursen DH, et al. Long-term weight loss in a primary care-anchored eHealth lifestyle coaching program: randomized controlled trial. J Med Internet Res. Sep 23, 2022;24(9):e39741. [FREE Full text] [CrossRef] [Medline]
- Marko-Holguin M, Cordel SL, Van Voorhees BW, Fogel J, Sykes E, Fitzgibbon M, et al. A two-way interactive text messaging application for low-income patients with chronic medical conditions: design-thinking development approach. JMIR Mhealth Uhealth. May 01, 2019;7(5):e11833. [FREE Full text] [CrossRef] [Medline]
- Guay M, Labbé M, Séguin-Tremblay N, Auger C, Goyer G, Veloza E, et al. Adapting a person's home in 3D using a mobile app (MapIt): participatory design framework investigating the app's acceptability. JMIR Rehabil Assist Technol. May 11, 2021;8(2):e24669. [FREE Full text] [CrossRef] [Medline]
- Kildea J, Battista J, Cabral B, Hendren L, Herrera D, Hijal T, et al. Design and development of a person-centered patient portal using participatory stakeholder co-design. J Med Internet Res. Feb 11, 2019;21(2):e11371. [FREE Full text] [CrossRef] [Medline]
- Smaradottir B, Holen-Rabbersvik E, Thygesen E, Fensli R, Martinez S. User-centred design of the user interface of a collaborative information system for inter-municipal dementia team. In: Proceedings of the 2015 International Conference on Health Informatics. 2015. Presented at: HEALTHINF '15; January 12-15, 2015:22-31; Lisbon, Portugal. URL: https://www.scitepress.org/Link.aspx?doi=10.5220/0005222704460453 [CrossRef]
- Smaradottir B, Gerdes M, Martinez S, Fensli R. The EU-project United4Health: user-centred design of an information system for a Norwegian telemedicine service. J Telemed Telecare. Oct 09, 2016;22(7):422-429. [FREE Full text] [CrossRef] [Medline]
- Uddin AA, Morita PP, Tallevi K, Armour K, Li J, Nolan RP, et al. Development of a wearable cardiac monitoring system for behavioral neurocardiac training: a usability study. JMIR Mhealth Uhealth. Apr 22, 2016;4(2):e45. [FREE Full text] [CrossRef] [Medline]
- Jibb LA, Cafazzo JA, Nathan PC, Seto E, Stevens BJ, Nguyen C, et al. Development of a mHealth real-time pain self-management app for adolescents with cancer: an iterative usability testing study [formula: see text]. J Pediatr Oncol Nurs. 2017;34(4):283-294. [CrossRef] [Medline]
- Rodrigues AM, Sniehotta FF, Birch-Machin MA, Olivier P, Araújo-Soares V. Systematic and iterative development of a smartphone app to promote sun-protection among holidaymakers: design of a prototype and results of usability and acceptability testing. JMIR Res Protoc. Jun 12, 2017;6(6):e112. [FREE Full text] [CrossRef] [Medline]
- Ospina-Pinillos L, Davenport TA, Ricci CS, Milton AC, Scott EM, Hickie IB. Developing a mental health eClinic to improve access to and quality of mental health care for young people: using participatory design as research methodologies. J Med Internet Res. May 28, 2018;20(5):e188. [FREE Full text] [CrossRef] [Medline]
- Afrizal SH, Hidayanto AN, Hakiem N, Sartono A, Priyambodo S, Eryando T. Design of mHealth application for integrating antenatal care service in primary health care: a user-centered approach. In: Proceedings of the 4th International Conference on Informatics and Computing. 2019. Presented at: ICIC '19; October 16-17, 2019:1-6; Semarang, Indonesia. URL: https://ieeexplore.ieee.org/document/8985911 [CrossRef]
- Sedlmayr B, Schöffler J, Prokosch HU, Sedlmayr M. User-centered design of a mobile medication management. Inform Health Soc Care. Mar 05, 2019;44(2):152-163. [CrossRef] [Medline]
- Laine TH, Normark J, Lindvall H, Lindqvist AK, Rutberg S. A distributed multiplayer game to promote active transport at workplaces: user-centered design, implementation, and lessons learned. IEEE Trans Games. Dec 2020;12(4):386-397. [CrossRef]
- Tonga E, Williamson E, Srikesavan C, Özen T, Sarıtaş F, Lamb SE. A hand exercise mobile app for people with rheumatoid arthritis in Turkey: design, development and usability study. Rheumatol Int. Jun 18, 2021;41(6):1151-1160. [CrossRef] [Medline]
- Honary M, Fisher NR, McNaney R, Lobban F. A web-based intervention for relatives of people experiencing psychosis or bipolar disorder: design study using a user-centered approach. JMIR Ment Health. Dec 07, 2018;5(4):e11473. [FREE Full text] [CrossRef] [Medline]
- Span M, Hettinga M, Groen-van de Ven L, Jukema J, Janssen R, Vernooij-Dassen M, et al. Involving people with dementia in developing an interactive web tool for shared decision-making: experiences with a participatory design approach. Disabil Rehabil. Jun 2018;40(12):1410-1420. [CrossRef] [Medline]
- Schild S, Sedlmayr B, Schumacher AK, Sedlmayr M, Prokosch H, St Pierre M, et al. German Cognitive Aid Working Group. A digital cognitive aid for anesthesia to support intraoperative crisis management: results of the user-centered design process. JMIR Mhealth Uhealth. Apr 29, 2019;7(4):e13226. [FREE Full text] [CrossRef] [Medline]
- Buck B, Wingerson M, Whiting E, Snyder J, Monroe-DeVita M, Ben-Zeev D. User-centered development of bolster, an mHealth intervention for early psychosis caregivers: needs assessment, prototyping, and field trial. JMIR Ment Health. Nov 30, 2023;10:e50522. [FREE Full text] [CrossRef] [Medline]
- Ahufinger S, Pequeno E, Herrero P. Mob-EEG: a user-centered design for wireless electroencephalogram in intensive care units. In: Proceedings of the 32nd International Symposium on Computer-Based Medical Systems. 2019. Presented at: CBMS '19; June 5-7, 2019:204-207; Cordoba, Spain. URL: https://ieeexplore.ieee.org/abstract/document/8787513 [CrossRef]
- Alberts NM, Badawy SM, Hodges J, Estepp JH, Nwosu C, Khan H, et al. Development of the InCharge health mobile app to improve adherence to hydroxyurea in patients with sickle cell disease: user-centered design approach. JMIR Mhealth Uhealth. May 08, 2020;8(5):e14884. [FREE Full text] [CrossRef] [Medline]
- Corazza F, Snijders D, Arpone M, Stritoni V, Martinolli F, Daverio M, et al. Development and usability of a novel interactive tablet app (PediAppRREST) to support the management of pediatric cardiac arrest: pilot high-fidelity simulation-based study. JMIR Mhealth Uhealth. Oct 01, 2020;8(10):e19070. [FREE Full text] [CrossRef] [Medline]
- Erguera XA, Johnson MO, Neilands TB, Ruel T, Berrean B, Thomas S, et al. WYZ: a pilot study protocol for designing and developing a mobile health application for engagement in HIV care and medication adherence in youth and young adults living with HIV. BMJ Open. May 05, 2019;9(5):e030473. [FREE Full text] [CrossRef] [Medline]
- Kerkhof YJ, den Ouden ME, Soeteman S, Scholten A, Ben Allouch S, Willems CG. Development of a memory application for structuring and supporting daily activities of clients with intellectual disabilities. Technol Disabil. Jul 07, 2017;29(1-2):77-89. [CrossRef]
- Marien S, Legrand D, Ramdoyal R, Nsenga J, Ospina G, Ramon V, et al. A User-Centered design and usability testing of a web-based medication reconciliation application integrated in an eHealth network. Int J Med Inform. Jun 2019;126:138-146. [CrossRef] [Medline]
- Martin S, Armstrong E, Thomson E, Vargiu E, Solà M, Dauwalder S, et al. A qualitative study adopting a user-centered approach to design and validate a brain computer interface for cognitive rehabilitation for people with brain injury. Assist Technol. Jul 14, 2018;30(5):233-241. [CrossRef] [Medline]
- Navarro-Alamán J, Lacuesta R, García-Magariño I, Gallardo J, Ibarz E, Lloret J. Close2U: an app for monitoring cancer patients with enriched information from interaction patterns. J Healthc Eng. Jul 15, 2020;2020:3057032-3057013. [FREE Full text] [CrossRef] [Medline]
- Newton A, Bagnell A, Rosychuk R, Duguay J, Wozney L, Huguet A, et al. A mobile phone-based app for use during cognitive behavioral therapy for adolescents with anxiety (MindClimb): user-centered design and usability study. JMIR Mhealth Uhealth. Dec 08, 2020;8(12):e18439. [FREE Full text] [CrossRef] [Medline]
- Noordman J, Driesenaar JA, van Bruinessen IR, van Dulmen S. ListeningTime; participatory development of a web-based preparatory communication tool for elderly cancer patients and their healthcare providers. Internet Interv. Sep 2017;9:51-56. [FREE Full text] [CrossRef] [Medline]
- Petersen M, Hempler NF. Development and testing of a mobile application to support diabetes self-management for people with newly diagnosed type 2 diabetes: a design thinking case study. BMC Med Inform Decis Mak. Jun 26, 2017;17(1):91. [FREE Full text] [CrossRef] [Medline]
- Pham Q, Graham G, Lalloo C, Morita PP, Seto E, Stinson JN, et al. An analytics platform to evaluate effective engagement with pediatric mobile health apps: design, development, and formative evaluation. JMIR Mhealth Uhealth. Dec 21, 2018;6(12):e11447. [FREE Full text] [CrossRef] [Medline]
- Sin J, Henderson C, Woodham LA, Sesé Hernández A, Gillard S. A multicomponent eHealth intervention for family carers for people affected by psychosis: a coproduced design and build study. J Med Internet Res. Aug 06, 2019;21(8):e14374. [FREE Full text] [CrossRef] [Medline]
- Stevenson JE, Oscarsson M. User-centred iterative design to develop an evidence-based communication application for maternity care. Health Informatics J. May 24, 2021;27(2):14604582211014579. [FREE Full text] [CrossRef] [Medline]
- Vilardaga R, Rizo J, Zeng E, Kientz JA, Ries R, Otis C, et al. User-centered design of learn to quit, a smoking cessation smartphone app for people with serious mental illness. JMIR Serious Games. Jan 16, 2018;6(1):e2. [FREE Full text] [CrossRef] [Medline]
- Willard S, Cremers G, Man YP, van Rossum E, Spreeuwenberg M, de Witte L. Development and testing of an online community care platform for frail older adults in the Netherlands: a user-centred design. BMC Geriatr. Apr 07, 2018;18(1):87. [FREE Full text] [CrossRef] [Medline]
- Pienkowska A, Ang CS, Mammadova M, Mahadzir MD, Car J. A diabetes education app for people living with type 2 diabetes: co-design study. JMIR Form Res. Sep 18, 2023;7:e45490. [FREE Full text] [CrossRef] [Medline]
- Alvarez SJ, Pencheva B, Westfall E, Mwalija C, Parsell M, Greenleaf M, et al. A novel mobile health application to support cancer surveillance needs of patients and families with cancer predisposition syndromes. Pediatr Blood Cancer. Oct 06, 2023;70(10):e30537. [FREE Full text] [CrossRef] [Medline]
- Shanmugavel A, Shakya PR, Shrestha A, Nepal J, Shrestha A, Daneault J, et al. Designing and developing a mobile app for management and treatment of gestational diabetes in Nepal: user-centered design study. JMIR Form Res. Jan 17, 2024;8:e50823. [FREE Full text] [CrossRef] [Medline]
- Dannenberg MD, Bienvenida JC, Bruce ML, Nguyen T, Hinn M, Matthews J, et al. End-user views of an electronic encounter decision aid linked to routine depression screening. Patient Educ Couns. Mar 2019;102(3):555-563. [CrossRef] [Medline]
- Rao N, Perdomo S, Jonassaint C. A novel method for digital pain assessment using abstract animations: human-centered design approach. JMIR Hum Factors. Jan 07, 2022;9(1):e27689. [FREE Full text] [CrossRef] [Medline]
- Zhang M, Heng S, Song G, Fung DS, Smith HE. Co-designing a mobile gamified attention bias modification intervention for substance use disorders: participatory research study. JMIR Mhealth Uhealth. Oct 03, 2019;7(10):e15871. [FREE Full text] [CrossRef] [Medline]
- Calvillo-Arbizu J, Roa-Romero LM, Estudillo-Valderrama MA, Salgueira-Lazo M, Aresté-Fosalba N, Del-Castillo-Rodríguez NL, et al. User-centred design for developing e-Health system for renal patients at home (AppNephro). Int J Med Inform. May 2019;125:47-54. [CrossRef] [Medline]
- van Velsen L, van Weering MD, Luub F, Neven A. Travelling with my SOULMATE: participatory design of an mHealth travel companion for older adults. In: Proceedings of the 5th International Conference on Information and Communication Technologies for Ageing Well and e-Health. 2019. Presented at: ICT4AWE '19; May 2-4, 2019:38-47; Crete, Greece. URL: https://www.scitepress.org/Link.aspx?doi=10.5220/0007680200380047 [CrossRef]
- Stawarz K, Preist C, Tallon D, Wiles N, Kessler D, Turner K, et al. Design considerations for the integrated delivery of cognitive behavioral therapy for depression: user-centered design study. JMIR Ment Health. Sep 03, 2020;7(9):e15972. [FREE Full text] [CrossRef] [Medline]
- Stoyanov SR, Zelenko O, Staneva A, Kavanagh DJ, Smith C, Sade G, et al. Development of the Niggle app for supporting young people on their dynamic journey to well-being: co-design and qualitative research study. JMIR Mhealth Uhealth. Apr 20, 2021;9(4):e21085. [FREE Full text] [CrossRef] [Medline]
- Hoffman AS, Bateman DR, Ganoe C, Punjasthitkul S, Das AK, Hoffman DB, et al. Development and field testing of a long-term care decision aid website for older adults: engaging patients and caregivers in user-centered design. Gerontologist. Jul 15, 2020;60(5):935-946. [FREE Full text] [CrossRef] [Medline]
- Backman C, Harley A, Kuziemsky C, Mercer J, Peyton L. MyPath to home web-based application for the geriatric rehabilitation program at Bruyère continuing care: user-centered design and feasibility testing study. JMIR Form Res. Sep 14, 2020;4(9):e18169. [FREE Full text] [CrossRef] [Medline]
- Korpershoek YJ, Hermsen S, Schoonhoven L, Schuurmans MJ, Trappenburg JC. User-centered design of a mobile health intervention to enhance exacerbation-related self-management in patients with chronic obstructive pulmonary disease (Copilot): mixed methods study. J Med Internet Res. Jun 15, 2020;22(6):e15449. [FREE Full text] [CrossRef] [Medline]
- Da Silva Júnior JL, Biduski D, Bellei EA, Becker OH, Daroit L, Pasqualotti A, et al. A bowling exergame to improve functional capacity in older adults: co-design, development, and testing to compare the progress of playing alone versus playing with peers. JMIR Serious Games. Jan 29, 2021;9(1):e23423. [FREE Full text] [CrossRef] [Medline]
- Dirin M, Dirin A, Laine TH. User-centered design of a context-aware nurse assistant (CANA) at Finnish elderly houses. In: Proceedings of the 9th International Conference on Ubiquitous Information Management and Communication. Proceedings of the 9th International Conference on Ubiquitous Information Management and Communication. IMCOM ?15. Association for Computing Machinery; 2015. Presented at: IMCOM '15; January 8-10, 2015:1-8; Bali, Indonesia. URL: https://dl.acm.org/doi/10.1145/2701126.2701225 [CrossRef]
- Lyles CR, Altschuler A, Chawla N, Kowalski C, McQuillan D, Bayliss E, et al. User-centered design of a tablet waiting room tool for complex patients to prioritize discussion topics for primary care visits. JMIR Mhealth Uhealth. Sep 14, 2016;4(3):e108. [FREE Full text] [CrossRef] [Medline]
- Marien S, Legrand D, Ramdoyal R, Nsenga J, Ospina G, Ramon V, et al. A web application to involve patients in the medication reconciliation process: a user-centered usability and usefulness study. J Am Med Inform Assoc. Nov 01, 2018;25(11):1488-1500. [FREE Full text] [CrossRef] [Medline]
- He X, Zhang R, Rizvi R, Vasilakes J, Yang X, Guo Y, et al. ALOHA: developing an interactive graph-based visualization for dietary supplement knowledge graph through user-centered design. BMC Med Inform Decis Mak. Aug 08, 2019;19(Suppl 4):150. [FREE Full text] [CrossRef] [Medline]
- Martin A, Caon M, Adorni F, Andreoni G, Ascolese A, Atkinson S, et al. A mobile phone intervention to improve obesity-related health behaviors of adolescents across Europe: iterative co-design and feasibility study. JMIR Mhealth Uhealth. Mar 02, 2020;8(3):e14118. [FREE Full text] [CrossRef] [Medline]
- Mathenjwa T, Adeagbo O, Zuma T, Dikgale K, Zeitlin A, Matthews P, et al. Development and acceptability of a tablet-based app to support men to link to HIV care: mixed methods approach. JMIR Mhealth Uhealth. Nov 24, 2020;8(11):e17549. [FREE Full text] [CrossRef] [Medline]
- van Bruinessen I, van Klooster JW, Boessen A, Heide L, Vollenbroek-Hutten M, van Dulmen S. Involving end-users in the development and implementation of a web-based, physical activity platform for elderly. In: Proceedings of the 2017 International Conference on e-Health. 2017. Presented at: eHealth '17; October 12-15, 2017:145-149; Dalian, China. URL: https://www.iadisportal.org/digital-library/involving-end-users-in-the-development-and-implementation-of-a-web-based-physical-activity-platform-for-elderly
- Chan NH, Merali HS, Mistry N, Kealey R, Campbell DM, Morris SK, et al. Development of a novel mobile application, HBB prompt, with human factors and user-centred design for helping babies breathe skills retention in Uganda. BMC Med Inform Decis Mak. Feb 04, 2021;21(1):39. [FREE Full text] [CrossRef] [Medline]
- Balatsoukas P, Porat T, Sassoon I, Essers K, Kokciyan N, Chapman M, et al. User involvement in the design of a data-driven self-management decision support tool for stroke survivors. In: Proceedings of the 18th International Conference on Smart Technologies. 2019. Presented at: IEEE EUROCON '19; July 1-4, 2019:1-6; Novi Sad, Serbia. URL: https://ieeexplore.ieee.org/document/8861812 [CrossRef]
- Woods L, Cummings E, Duff J, Walker K. Conceptual design and iterative development of a mHealth app by clinicians, patients and their families. Stud Health Technol Inform. 2018;252:170-175. [Medline]
- Powell J, Newhouse N, Boylan AM, Williams V. Digital health citizens and the future of the NHS. Digit Health. Oct 10, 2016;2:2055207616672033. [FREE Full text] [CrossRef] [Medline]
- Kress HG, Aldington D, Alon E, Coaccioli S, Collett B, Coluzzi F, et al. A holistic approach to chronic pain management that involves all stakeholders: change is needed. Curr Med Res Opin. 2015;31(9):1743-1754. [CrossRef] [Medline]
- Peiris D, Miranda JJ, Mohr DC. Going beyond killer apps: building a better mHealth evidence base. BMJ Glob Health. 2018;3(1):e000676. [FREE Full text] [CrossRef] [Medline]
- Wilson K, Bell C, Wilson L, Witteman H. Agile research to complement agile development: a proposal for an mHealth research lifecycle. NPJ Digit Med. Sep 13, 2018;1(1):46. [FREE Full text] [CrossRef] [Medline]
- Petrakaki D, Hilberg E, Waring J. The cultivation of digital health citizenship. Soc Sci Med. Feb 2021;270:113675. [FREE Full text] [CrossRef] [Medline]
- Lewin S, Booth A, Glenton C, Munthe-Kaas H, Rashidian A, Wainwright M, et al. Applying GRADE-CERQual to qualitative evidence synthesis findings: introduction to the series. Implement Sci. Jan 25, 2018;13(Suppl 1):2. [FREE Full text] [CrossRef] [Medline]
Abbreviations
| DHI: digital health intervention |
| mHealth: mobile health |
| PRISMA: Preferred Reporting Items for Systematic Reviews and Meta-Analyses |
| PRISMA-P: Preferred Reporting Items for Systematic Reviews and Meta-Analyses Protocols |
| RCT: randomized controlled trial |
| UCD: user-centered design |
| UX: user experience |
Edited by A Mavragani; submitted 21.06.23; peer-reviewed by E Baker, E Rosenfeld; comments to author 19.09.23; revised version received 06.12.23; accepted 07.12.24; published 14.03.25.
Copyright©Anthony Duffy, Nazanin Boroumandzad, Alfredo Lopez Sherman, Gregory Christie, Indira Riadi, Sylvain Moreno. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 14.03.2025.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research (ISSN 1438-8871), is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.