This paper is in the following e-collection/theme issue:

Web-based and Mobile Health Interventions (3092) Theoretical Frameworks and Concepts (230) Behavior Change (652)

Published on in Vol 21, No 10 (2019): October

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/13606, first published .
Identifying Frameworks for Validation and Monitoring of Consensual Behavioral Intervention Technologies: Narrative Review

Identifying Frameworks for Validation and Monitoring of Consensual Behavioral Intervention Technologies: Narrative Review

Identifying Frameworks for Validation and Monitoring of Consensual Behavioral Intervention Technologies: Narrative Review

Authors of this article:

François Carbonnel1, 2, 3, 4 Author Orcid Image ;   Gregory Ninot1, 2, 5 Author Orcid Image
Article Authors Cited by (16) Tweetations (3) Metrics

    Review

    1Research Unit EA4556 Epsylon, University of Montpellier, Montpellier, France

    2Plateforme Universitaire Collaborative d’Evaluation des Programmes de Prévention et de Soins de Support, University of Montpellier, Montpellier, France

    3University Department of General Practice, University of Montpellier, Montpellier, France

    4University Multiprofessional Health Center Avicenne, Perpignan, France

    5Montpellier Cancer Institute, Montpellier, France

    *all authors contributed equally

    Corresponding Author:

    François Carbonnel, MD, PhD

    University Department of General Practice

    University of Montpellier

    641 Avenue du Doyen Giraud

    UFR Médecine site Nord

    Montpellier, 34093

    France

    Phone: 33 684014834

    Email: francois.carbonnel@gmail.com


    Background: Changing health behaviors, such as smoking, unhealthy eating, inactivity, and alcohol abuse, may have a greater impact on population health than any curative strategy. One of the suggested strategies is the use of behavioral intervention technologies (BITs). They open up new opportunities in the area of prevention and therapy and have begun to show benefits in the durable change of health behaviors in patients or those at risk. A consensual and international paradigm was adopted by health authorities for drugs 50 years ago. It guides their development from research units to their authorization and surveillance. BITs’ generalization brings into question their upstream evaluation before being placed on the market and their downstream monitoring once on the market; this is especially the case in view of the marketing information provided by manufacturers and the scarcity and methodological limits of scientific studies on these tools.

    Objective: This study aims to identify and categorize the frameworks for the validation and monitoring of BITs proposed in the literature.

    Methods: We conducted a narrative literature review using MEDLINE, PsycINFO, and Web of Science. The review items included the following: name, publication year, name of the creator (ie, first author), country, funding organization, health focus, target group, and design (ie, linear, iterative, evolutive, and/or concurrent). The frameworks were then categorized based on (1) translational research thanks to a continuum of steps and (2) the three paradigms that may have inspired the frameworks: biomedical, engineering, and/or behavioral.

    Results: We identified 46 frameworks besides the classic US Food and Drug Administration (FDA) five-phase drug development model. A total of 57% (26/46) of frameworks were created in the 2010s and 61% (28/46) involved the final user in an early and systematic way. A total of 4% (2/46) of frameworks had a linear-only sequence of their phases, 37% (17/46) had a linear and iterative structure, 33% (15/46) added an evolutive structure, and 24% (11/46) were associated with a parallel process. Only 12 out of 46 (26%) frameworks covered the continuum of steps and 12 (26%) relied on the three paradigms.

    Conclusions: To date, 46 frameworks of BIT validation and surveillance coexist, besides the classic FDA five-phase drug development model, without the predominance of one of them or convergence in a consensual model. Their number has increased exponentially in the last three decades. Three dangerous scenarios are possible: (1) anarchic continuous development of BITs that depend on companies amalgamating health benefits and usability (ie, user experience, data security, and ergonomics) and limiting implementation to several countries; (2) the movement toward the type of framework for drug evaluation centered on establishing its effectiveness before marketing authorization to guarantee its safety for users, which is heavy and costly; and (3) the implementation of a framework reliant on big data analysis based on a posteriori research and an autoregulation of a market, but that does not address the safety risk for the health user, as the market will not regulate safety or efficacy issues. This paper recommends convergence toward an international validation and surveillance framework based on the specificities of BITs, not equivalent to medical devices, to guarantee their effectiveness and safety for users.

    J Med Internet Res 2019;21(10):e13606

    doi:10.2196/13606

    Keywords

    behavioral intervention technology; validation; surveillance; paradigm; framework; nonpharmacological interventions 


    According to the World Health Organization (WHO), changing health-related behaviors, such as smoking, unhealthy diet, physical inactivity, and alcohol abuse, could avoid up to 80% of heart diseases, strokes, and type 2 diabetes as well as more than 30% of cancers [1]. These behaviors explain 50% of premature mortality and morbidity in the United States [2]. Improving the efficacy of the interventions dedicated to sustainably change these behaviors will have a greater impact on population health than any therapeutic strategy [3]. One of the possible solutions is the use of nonpharmacological interventions such as digital health interventions [4]. Among eHealth tools, mHealth includes mobile phone health apps and connected health devices [5,6]. Between 20% and 80% of adults are equipped with connected health devices, from one country to another [7-11]. Behavioral intervention technologies (BITs) “employ a broad range of technologies, such as mobile phones, the Web, and sensors, to support users in changing behaviors and cognitions related to health, mental health, and wellness” [12]. Digital health interventions open up new opportunities in the area of prevention and therapeutics: quantified-self activities and behaviors; sharing of clinical, psychological, and behavioral data (ie, short message service [SMS] and social networks); real-time analysis of health data; delivering of health promotion messages (ie, mobile phone and Web apps); involvement of health professionals (ie, telemedicine); e-coaching; social support of the family; and peers in the social environment [13,14]. Several observational studies indicate benefits of BITs on health behavior change in patients with chronic disease or at-risk people using interventions to assist with stronger drug compliance, smoking cessation, alcohol consumption reduction, weight management, and better diets [15-18]. Does the existence of a benefit provide evidence of an effective and safe tool and the only approach to generalize it?

    The evaluation of BITs before market and their ongoing monitoring remain questionable, especially in a world propelled by marketing strategies and the lack of regulation of health solutions that do not belong to the category of medical devices. According to the US Food and Drug Administration (FDA) [19,20], the validation process for drugs and implantable medical devices involves the “collection and evaluation of data, from the process design stage through commercial production, which establishes scientific evidence that a process is capable of consistently delivering quality product.” To assess the efficacy and safety of a health product, there is a need to use a specific scientific paradigm, which is “a set of principles and methods shared by a scientific community” [21]. A framework is a model for planning processes or for action plans, which brings a systematic approach to developing, managing, and evaluating interventions [22]. Regulators, researchers, and manufacturers share a consensual paradigm for drugs [19,20]. This clinical trial framework guides the development from lab to authorization and monitoring. It is organized in five phases: Phase 0 (ie, preclinical) to identify mechanisms; Phase 1 to determine tolerance in healthy humans; Phase 2 to identify the optimal dose for a small number of patients (ie, pilot trial); Phase 3 to demonstrate evidence of efficacy and safety (ie, randomized controlled trial); and Phase 4 to ensure long-term safety [23]. The number of studies assessing BITs is growing exponentially. However, their methodological designs do not follow an established validation framework. They flow from individual or research team choice, context, and/or opportunity.

    The objective of this study was to identify the frameworks for the validation and monitoring of BITs proposed in the literature, besides the five-phase drug development model, and to categorize them.


    Data Collection

    We have conducted a narrative literature review of articles published up to April 4, 2019, of the validation and monitoring frameworks of BITs. The main databases that we searched were MEDLINE, PsycINFO, Web of Science, Science Direct, and the Journal of Medical Internet Research (JMIR) database. They have been chosen as reference databases in biomedicine and psychology and comprise the largest general scientific database [24]. The search keywords were as follows: (“model” OR “framework” OR “process” OR “evaluation process” OR “validation process”) AND (“behavior” OR “behavior change” OR “behavioral intervention”) AND (“digital health” OR “ehealth” OR “mhealth” OR “connected health” OR “medical app” OR “smartphone” OR “iphone” OR “email” OR “text message” OR “SMS” OR “mobile app” OR “smartphone app” OR “connected health” OR “wearable”). New articles were extracted based on analyzed articles.

    Description of Frameworks

    The frameworks were listed in chronological order. Items for each framework were as follows: name of the framework—if none was given, a phrase describing it in the original article was written within quotation marks; publication year; name of the first author, called the creator here; country of the creator; organization having funded the creator; health focus or lack of health focus; the target group (ie, population for whom the framework had been initially designed, for example, researchers, health professionals, or software designers); and its design. For the latter, we noted the involvement of the final users (ie, early stage or systematic) and the chain of the development stages (ie, linear, iterative, evolutive, and/or concurrent) [25]. A linear process executes the different stages in a sequential order. An iterative process combines one or more stages before chaining. An evolutive process executes them in a circular pattern in order to obtain, with each revolution, a more mature version of the product. Finally, a parallel process executes one stage or more in a concurrent way. If a framework had evolved through time and had been described by a new publication, the latter was also written in the synthesis table but its creator, as well as the creator’s country and funding organization, remained the original one.

    Categorization of Frameworks and Paradigms

    The frameworks were categorized according to the accepted procedure of translational research [26,27]. It may be described as a continuum of steps. It starts with a prototyping step, which has a background in engineering, followed by a mechanisms step based on a theoretical approach to health behavior change. Concept follows, which is a proof-of-concept step regarding health impact based on an exploratory intervention trial. This is followed by a demonstration step, evidence of health efficacy and effectiveness, which is based on a confirmatory interventional trial. Finally, a surveillance step is implemented for market use based on implementation and dissemination studies.

    The frameworks follow three general paradigms: (1) the biomedical paradigm, with its essential phase of clinical research to identify the benefits and risks; (2) the engineering paradigm, with its essential phase to improve the device; and (3) the behavioral paradigm, with its essential phase to evaluate the impact on health behaviors.


    Overview

    The literature review identified 46 frameworks, besides the five-phase drug development model, that met the research criteria (see Table 1).

    Table 1. Validation and monitoring frameworks of BITs. BIT: behavioral intervention technology.
    FrameworksAuthors, year
    1. Waterfall modelRoyce, 1970 [28]
    2. PRECEDE-PROCEEDa modelGreen, 1974 [29]
    3. Prototyping modelFloyd, 1984 [30]
    4. Five-phase cancer control modelGreenwald and Cullen, 1985 [31]
    5. Flay’s eight-stage health promotion modelFlay, 1986 [32]
    6. V life cycle modelRook, 1986 [33]
    7. Spiral life cycle modelBoehm, 1988 [34]
    8. Star life cycle modelHarston and Dix, 1989 [35]
    9. Rapid application developmentMartin, 1991 [36]
    10. National Institute on Drug Abuse’s (NIDA) stage modelOnken et al, 1997 [37]
    11. Intervention mappingBartholomew et al, 1998 [38]
    12. Usability engineering life cycleMayhew, 1999 [39]
    13. Agile software managementBeck et al, 2001 [40]
    14. Information technology (IT) implementation frameworkKukafka et al, 2003 [41]
    15. Multiphase Optimization STrategy (MOST)Collins et al, 2005 [42]
    16. Framework for evaluating emergent eHealth resourcesPagliari, 2007 [43]
    17. Consolidated Standards of Reporting Trials (CONSORT) statement for nonpharmacologic treatmentsBoutron et al, 2008 [44] (updated in Boutron et al, 2017 [45])
    18. Iterative and incremental modelCockburn, 2008 [46]
    19. Medical Research Council (MRC) complex interventionCraig et al, 2008 [47]
    20. eHealth interventions evaluation processCatwell and Sheikh, 2009 [48]
    21. Center for eHealth Research (CeHRes) roadmap for the development of eHealth technologiesVan Gemert-Pijnen et al, 2011 [49], and Van Velsen et al, 2013 [50]
    22. The behavior change wheelMichie et al, 2011 [51]
    23. Consolidated Standards of Reporting Trials of Electronic and Mobile HEalth Applications and onLine TeleHealth (CONSORT-EHEALTH)Eysenbach et al, 2011 [52], and Eysenbach et al, 2013 [53]
    24. mHealth development and evaluation frameworkWhittaker et al, 2012 [17]
    25. Explore Values, Operationalize and Learn, and eValuate Efficacy (EVOLVE) mixed-methods modelPeterson et al, 2013 [54]
    26. Development process of Young and ActiveRiiser et al, 2013 [55]
    27. It’s LiFe! User-centered design processVan der Weegen et al, 2013 [56]
    28. DoTTIb development frameworkSmits et al, 2014 [57]
    29. National Institutes of Health (NIH) Stage modelOnken et al, 2014 [58]
    30. Behavioral intervention technology modelMohr et al, 2014 [12]
    31. Five-step content validity processKassam-Adams et al, 2015 [59]
    32. Steps for developing a text-messaging programAbroms et al, 2015 [60]
    33. Person-based approachRiiser et al, 2013 [55], and Yardley et al, 2015 [61]
    34. Obesity-Related Behavioral Intervention Trials (ORBIT) modelVan der Weegen et al, 2013 [56], and Czajkowski et al, 2015 [62]
    35. Pragmatic Framework for developing just-in-time adaptive interventions (JITAIs)Smits et al, 2014 [57], and Nahum-Shani et al, 2015 [63]
    36. TElehealth in CHronic disease (TECH) conceptual modelSalisbury et al, 2015 [64]
    37. Network for the Improvement of Addiction Treatment (NIATx) modelGustafson et al, 2016 [65]
    38. Integrate, Design, Assess, and Share (IDEAS) frameworkMummah et al, 2016 [66]
    39. Chronic disease mHealth app intervention design frameworkWilhide III et al, 2016 [67]
    40. Three-phase human-centered design methodologyHarte et al, 2017 [68]
    41. DREAM-GLOBALc frameworkMaar et al, 2017 [69]
    42. Processes and recommendations for creating mHealth apps for low-income populationsStephan et al, 2017 [70]
    43. Accelerated Creation-To-Sustainment (ACTS) modelMohr et al, 2017 [71]
    44. User-centered design processVilardaga et al, 2018 [72]
    45. Eight-step scoping frameworkDavidson et al, 2019 [73]
    46. Targeting, Understanding, Designing, Evaluating, and Refining (TUDER) frameworkWang et al, 2019 [74]

    aPRECEDE-PROCEED: Predisposing, Reinforcing, and Enabling Constructs in Educational Diagnosis and Evaluation—Policy, Regulatory, and Organizational Constructs in Educational and Environmental Development.

    bDoTTI: Design and develOpment, Testing early iterations, Testing for effectiveness, Integration and implementation.

    cDREAM-GLOBAL: Diagnosing hypeRtension—Engaging Action and Management in Getting LOwer Bp in Aboriginal and LMIC (lower- and middle-income countries).

    Description of Frameworks

    Frameworks

    The results showed that, out of 46 frameworks, 2 (4%) were designed in the 1970s, 6 (13%) in the 1980s, 4 (9%) in the 1990s, 8 (17%) in the 2000s, and 26 (57%) in the 2010s (see Multimedia Appendix 1). Their creators were mainly from the United States (24/46, 52%) and the United Kingdom (10/46, 21%). The Netherlands, Germany, Canada, Australia, Brazil, France, Ireland, Norway, and New Zealand were the remaining creators’ countries. A total of 8 frameworks out of 46 (17%) were created by private software company workers. A total of 4 frameworks out of 46 (9%) were designed by a national or a supranational public organization: the National Institutes of Health (NIH), the Medical Research Council (MRC), the National Health Service (NHS), and the Consolidated Standards of Reporting Trials (CONSORT). Universities were the main institutions (34/46, 74%), with 16 out of 46 (35%) from the United States. A total of 29 out of 46 frameworks (63%) were supported by a research grant: out of 46 frameworks, 29 (63%) were funded by a public grant, 12 (26%) were funded by the NIH, and 6 (13%) were funded by the English public health system. Out of 46 frameworks, 5 (11%) were funded by universities, 3 (7%) were funded by the private sector, and 3 (7%) were funded by a public-private partnership.

    Purpose and Target Population

    A total of 36 frameworks out of 46 (78%) were created with an individual health focus. Among them, the NIH Stage model was created to be relevant for clinical sciences [58] and the CONSORT statement was created for nonpharmacologic treatments for all nonpharmacological interventions [44]. Out of 46 frameworks, 3 (7%) aimed to validate health promotion programs [29,32,38]. A total of 23 of 46 (50%) were targeted to validate health behavior change interventions: of these 23, 15 (65%) were aimed at eHealth interventions in general [12,41,43,48,49,52,59,61,64-66,71-74], 2 (9%) at Internet interventions [55,57], and 6 (26%) at mobile interventions [17,56,60,67,69,70]. Among the 8 frameworks out of 23 (35%) aimed at health behavior change that were not focused on eHealth, 5 (63%) were created for behavior change in general [42,47,51,54,63] and 3 (38%) were dedicated to diseases such as cancer [31], addiction [37], and all chronic disease [62]. A total of 10 out of 46 frameworks (22%) were created with an original purpose that was not associated with health, but was associated with software development [28,33,34,36,40,46], human-computer interfaces [35,39], or engineering in general [30,68].

    Out of 46 frameworks, 17 (37%) were designed explicitly for intervention designers: 11 (24%) for software designers [17,30,33,35,36,39,43,46,48,49,64], 1 of which was for software evaluators [66], and 6 (13%) for designers of health behavior change interventions [12,41,51,61-63], 2 of which were based on technologies to change health behaviors [12,41]. A total of 9 out of 46 frameworks (20%) were aimed at researchers; these included researchers in general [54,58], scientific paper authors or scientific journal editors [44,51], researchers and stakeholders [32,71], researchers and industry professionals [66], and health researchers and software designers [43,60]. Finally, 13 frameworks out of 46 (28%) were intended for all stakeholders: professionals, researchers, users, clinicians, and other health helpers [42,46,55-57,59,65,67,68-70,72,73]. In 6 out of 46 cases (13%), the target of the framework was not mentioned; however, the reading of these articles was directed toward software designers [28,34,40], researchers, stakeholders, and health program planners [29,31,37]. Out of 46, 25 frameworks (54%) were created for software designers and 24 (52%) were created for researchers.

    Organization

    In 28 out of 46 frameworks (61%), the final user was involved early and systematically (ie, at each step). In 27 of them (59%), he or she played an active role in the BIT’s specification and assessment [17,35,36,39,40,43,48,49,54-57,59-62,64-74]. In the V life cycle model, the role was only as an evaluator [33].

    A total of 2 frameworks out of 46 (4%) had a linear sequence in their phases. The Waterfall model adopts a seven-phase structure: identification of the system specifications, identification of the software specifications, analysis, program design, coding, tests, and operations [28]. Inspired by the latter, the V life cycle model has a V-shaped structure, which matches development and testing phases and involves clients, followed by final users, in the development [33]. In addition to the linear sequence, 17 out of 46 frameworks (37%) had an iterative structure [12,17,31,32,42,44,52,54-56,61,62,66,69,70,72,73], allowing a refining step at each phase in the case of suboptimal results. The iterative structure is one of the assets used by the Multiphase Optimization STrategy (MOST) and the Obesity-Related Behavioral Intervention Trials (ORBIT) model frameworks to optimize interventions [42,62]. A total of 15 out of 46 frameworks (33%) were also associated with an evolutive process, creating a cyclic organization to improve the product progressively [30,34,35,37-39,43,47-49,51,58,60,67,68]. Finally, 11 out of 46 frameworks (24%) also integrated a parallel process [29,36,40,41,46,57,59,64,65,71,74]; 1 of those was the Agile software management framework, which aims for a high degree of adaptability to satisfy the clients [40].

    Framework Categorization

    The categorization of frameworks, according to their coverage of the continuum of steps, is shown in Figure 1.

    The Waterfall model is the only one that covers only the prototyping step [28]. Other frameworks cover the prototyping and surveillance steps by integrating the final user [30,33-36,39,40,46,48,49,68].

    The behavior change wheel and the pragmatic framework for developing just-in-time adaptive interventions (JITAIs) are focused on the mechanisms step [51,63]. The BIT model, the chronic disease mHealth app intervention design framework, and the Network for the Improvement of Addiction Treatment (NIATx) model cover the prototyping and mechanisms steps by associating the conception of both numeric and behavioral interventions [12,17,65].

    The Explore Values, Operationalize and Learn, and eValuate Efficacy (EVOLVE) and MOST frameworks cover the mechanisms, concept, and evidence steps. A total of 9 frameworks out of 46 (20%) are extended over the mechanisms, concept, evidence, and surveillance steps [29,31,32,37,44,47,49,58,62]. Finally, the whole continuum of steps is covered by 12 out of 46 (26%) frameworks [17,41,43,52,56,57,61,64,66,70,71,74].

    ‎
    Figure 1. Proposed frameworks to validate and monitor behavioral intervention technologies (BITs). ACTS: Accelerated Creation-To-Sustainment; CeHRes: Center for eHealth Research; DoTTI: Design and develOpment, Testing early iterations, Testing for effectiveness, Integration and implementation; DREAM-GLOBAL: Diagnosing hypeRtension—Engaging Action and Management in Getting LOwer Bp in Aboriginal and LMIC (lower- and middle-income countries); EVOLVE: Explore Values, Operationalize and Learn, and eValuate Efficacy; IDEAS: Integrate, Design, Assess, and Share; IT: information technology; JITAI: just-in-time adaptive intervention; MOST: Multiphase Optimization STrategy; MRC: Medical Research Council; NIATx: Network for the Improvement of Addiction Treatment; NIDA: National Institute on Drug Abuse; NIH: National Institutes of Health; ORBIT: Obesity-Related Behavioral Intervention Trials; PRECEDE-PROCEED: Predisposing, Reinforcing, and Enabling Constructs in Educational Diagnosis and Evaluation—Policy, Regulatory, and Organizational Constructs in Educational and Environmental Development; TECH: TElehealth in CHronic disease; TUDER: Targeting, Understanding, Designing, Evaluating, and Refining.
    View this figure

    Paradigm Categorization

    Figure 1 illustrates the categorization of frameworks based on their paradigms. Most belonged to one paradigm (ie, biomedical, engineering, or behavioral), while few referred to two or three paradigms.

    Out of 46 frameworks, 12 (26%) fell within the paradigm of engineering [28,30,33-36,39,40,46,48,49,68] and 3 (7%) were based on the behavioral paradigm [38,51,63]. A total of 9 (20%) frameworks were based on these two latter paradigms [12,55,59,60,65,67,70-72]. A total of 5 (11%) frameworks were based on the biomedical paradigm, which draws on the clinical trial, specifically the randomized controlled trial [28,31,32,44,52]. A total of 4 others (9%) also incorporated a theoretical approach of behavior change [37,47,54,58]. The framework for evaluating emergent eHealth resources was based on the biomedical and engineering paradigms [43]. A total of 12 (26%) frameworks mixed three paradigms [17,41,42,56,57,61-63,67,69,73,74].


    Principal Findings

    Our narrative review of the literature showed the absence of a unique and consensual validation and surveillance framework for health behavior intervention technology. This conclusion is in keeping with the statement of Bradway et al about mHealth assessment: “too many initiatives, too few answers” [75]. To date, 46 frameworks coexist, besides the five-phase drug development model, without a predominance of, or a convergence toward, one of them. Their number has increased exponentially over time since the 1970s. The United States and the United Kingdom are the main countries that have proposed models, probably due to their university research productivity and new-technology industry leadership [76]. They are motivated, on the one hand, by the plethoric and easily available offer of digital tools and networks and, on the other hand, by the failure to prevent unhealthy diet, addiction, and physical inactivity [2].

    Beyond the quantitative increase of these frameworks, this review underlines an increased heterogeneity. One surprising finding is that the benefit-to-risk ratio is not a main goal. Contrary to drug claims regarding societal recognition and reimbursement for their impact on health outcomes [77], showing evidence of safety and efficacy is paradoxically not a priority for BITs. This fact can explain why researchers have noted that medical mobile phone apps are lightly used by patients in the medium term, have little involvement from health professionals, lack characterization of their content, and have a low integration of behavior change theories [78]. The same problem can be found for connected health devices. The lack of evidence of their efficacy leads to their nonprescription by general practitioners, who picture them mainly as a supplementary organizational constraint [79]. Most frameworks are inspired by the paradigms of engineering and behavioral sciences. Engineering is focused on technicity (ie, release criteria, ergonomics, user experience, and data security), which leads to uncertainties about these technologies’ health contributions and turns them into entertainment products or gadgets [80]. The behavioral sciences bring an understanding of behavior change mechanisms and skills techniques without determining and comparing their health impacts [81,82]. This explains how translational research is unusual in this area compared to that for drugs [83]. To date, for instance, there is no validation model for psychotherapies [84].

    Beyond the quantitative increase and diversification of frameworks, we have noted the emergence of common principles. The first one is an assumed superiority with respect to health and health behavior change of some BITs compared to others. This hypothesis aims to address the plethoric and exponential offer of digital solutions sustained by marketing. This growing need to know the efficacy of each BIT and to compare them to other solutions to change health behaviors comes from both health professionals, who seek to recommend or even prescribe evidence-based solutions, and health users, who seek solutions beyond marketing and/or users’ experiences from social networks.

    The second is the requirement of user involvement during the beginning of BIT development and after market access. This principle exists in every recent framework based on the behavioral science, engineering, or biomedical paradigms. It requires a coconstruction with the user and considers his or her experience. It involves the end user through a user-centered design from engineering [85,86]. It requires a patient-centered approach to the provision of personalized care in the biomedical area [87]. The patient’s preferences are considered and may allow improved chances of stable and efficient use.

    The third is the ambition to evaluate BITs beyond simple satisfaction. This is more or less oriented by manufacturers to aim at effectiveness in medicine becoming more and more predictive, preventive, personalized, and participative; this is called 4P medicine [88,89].

    The fourth is the shortening of the delay for upstream validation, until its complete suppression in some engineering frameworks, and a multiplication of the downstream surveillance methods, such as implementation studies, experience feedback (eg, to a learned society or health authority), and big data analyses combining multimodal data. The shortening of the upstream period is linked, on the one hand, to a supposedly low dangerousness of these health solutions and, on the other hand, to the short life cycle of the technologies involved [90]. The development time of a drug, from the lab bench to its marketing authorization, is between 17 and 24 years [91-93]. It is incompatible with digital innovation. Digital industrialists assume that hundreds of millions of users will allow for the evaluation of BITs’ effectiveness on a large scale thanks to big data analyses [94,95].

    The fifth is the introduction of hybrid frameworks integrating the development and updating processes from engineering, behavior change theories, techniques from behavioral sciences, and the rigorous approach of validation from the biomedical area (eg, the FDA and the Environmental Management Association). This hybridization contributes to the creation of interventions standing between medical devices and products in the category of general commodities. The scientific process to develop and validate them, the rules for marketing authorization, and surveillance have not yet been defined for a country or a continent and are still under construction. Many digital industrialists wish to avoid the validation process for medical devices in view of its constraints and costs; at the same time, they aim to indicate to health professionals and users the health impact of their solutions. They are encouraged to address this by mutual insurances, other insurances, and banks.

    We stress two extreme positions for the studied frameworks that may become problematic for the development of BITs in the health sector. The first is that of permanent technological innovation. It is embodied in the Agile software management framework. Flexibility, based on user demand, dictates each evolution. Health is a market like any other. If this approach was to win in this area, for example, to make substantial savings in clinical research, no counterpart could be asked for, concerning health benefits. Big data analyses will not guarantee a relevant comparability between BITs [96]. The second problem at the opposite extreme is that marketing authorization of the BIT must be conditional on the completion of a well-designed randomized controlled trial. This process has been a part of the success of drugs in the last century. These trials are long, costly, and debatable in terms of methodology for BITs, if only for the choice of control group [97].

    Limitations and Strengths

    To identify every validation and surveillance framework of BITs is challenging. This area is new, it is based on a number of different paradigms, and develops in an uncontrolled way from one country to another and from one industry to another. One of the identified difficulties within these publications is the lack of common terminology [98,99]. The originality of our review is the spanning of the three approaches (ie, engineering, behavioral sciences, biomedical sciences). Although they are often opposed, the concern here was to note the borrowings and contributions that they could mutually bring through transversal thinking.

    Conclusions

    Validating a BIT for health has the following specific challenges: achieve an efficient and quick development; understand and promote long-term adherence; be supported by behavior change theories and techniques; evaluate effectiveness and cost-effectiveness; and ensure rigorous management in terms of regulation, ethics, and information [93]. Patient expectations are relayed through the media and through the arrival on the health market of digital players proposing short life-cycle products due to rapid obsolescence of technology; this justifies the need for a consensual validation and surveillance framework for nonpharmacological interventions. This framework will necessarily differ from that used for drugs. Our review has identified 46 frameworks, none of which dominates for BITs. Three paths are opened: (1) the anarchic continuous development of new competing frameworks that prevents any convergence in a standardized validation and surveillance framework, with its consequential recognition by health authorities; (2) the movement toward the type of framework for drug evaluation centered on establishing its effectiveness before marketing authorization to guarantee its safety for users, which is heavy and costly; and (3) the implementation of a framework reliant on big data analysis, based on a posteriori research and an autoregulation of a market; however, that does not address the safety risk for the health user, as the market will not regulate safety or efficacy issues. This article calls for a minimal upstream clinical phase and an increased surveillance of BITs to address the risk of semantic amalgams, inappropriate prescriptions, and induced misuses [16,100]. A BIT cannot be a simple tool but must be a complex strategy integrated in a given environment [101]. A BIT must make sense by virtue of its interaction with the context, building a system, and must be evaluated as such [100].

    Acknowledgments

    The authors would like to thank the members of the Plateforme universitaire Collaborative d’Evaluation des programmes de Prévention et de Soins de support (CEPS) for their help in better defining and using nonpharmacological interventions for patients and the SIRIC Montpellier Cancer (Grant INCa Inserm DGOS 12553).

    Conflicts of Interest

    None declared.

    ‎
    Multimedia Appendix 1

    Table of frameworks.

    PDF File (Adobe PDF File), 225 KB
    1. Global Action Plan for the Prevention and Control of Noncommunicable Diseases 2013-2020. Geneva, Switzerland: World Health Organization; 2013.   URL: http://apps.who.int/iris/bitstream/handle/10665/94384/9789241506236_eng.pdf?sequence=1.Archived [accessed 2019-09-06] [WebCite Cache]
    2. Institute of Medicine, Committee on Quality of Health Care in America. Crossing The Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academies Press; 2001.
    3. Sabaté E, De Geest S. Adherence to long-term therapies management: A call for cardiovascular nursing managers and policymakers. Prog Cardiovasc Nurs 2004;19(1):28-29. [Medline]
    4. CEPS Platform.   URL: https://www.iceps2020.com/iceps-congres/ceps-platform/ [accessed 2019-09-12] [WebCite Cache]
    5. Fifty-Eighth World Health Assembly. Geneva, Switzerland: World Health Organization; 2005.   URL: http://apps.who.int/gb/ebwha/pdf_files/WHA58-REC1/english/A58_2005_REC1-en.pdf [accessed 2019-09-06]
    6. World Health Organization. mHealth: New Horizons for Health Through Mobile Technologies. Global Observatory for eHealth Series - Volume 3. Geneva, Switzerland: World Health Organization; 2011.
    7. Gandhi M, Wang T. Rock Health. 2015. Digital health consumer adoption: 2015   URL: https://rockhealth.com/reports/digital-health-consumer-adoption-2015/ [accessed 2019-09-06] [WebCite Cache]
    8. Fox S, Duggan M. Pew Research Center. Washington, DC: Pew Internet & American Life Project; 2012 Nov 08. Mobile health 2012   URL: http://www.pewinternet.org/2012/11/08/mobile-health-2012 [accessed 2019-09-06] [WebCite Cache]
    9. Kontos E, Blake KD, Chou WS, Prestin A. Predictors of eHealth usage: Insights on the digital divide from the Health Information National Trends Survey 2012. J Med Internet Res 2014 Jul 16;16(7):e172 [FREE Full text] [CrossRef] [Medline]
    10. mHealth App Developer Economics 2016: The Current Status and Trends of the mHealth App Market. Berlin, Germany: Research 2 Guidance; 2016 Oct.   URL: http:/​/research2guidance.​com/​wp-content/​uploads/​2016/​10/​mHealth-App-Developer-Economics-2016-v17-Preview-1-1.​pdf [accessed 2019-09-06] [WebCite Cache]
    11. Danova T. Business Insider France. 2013 Oct 02. Morgan Stanley: 75 billion devices will be connected to the Internet of Things by 2020   URL: http:/​/www.​businessinsider.fr/​us/​75-billion-devices-will-be-connected-to-the-internet-by-2020-2013-10/​ [accessed 2019-09-06] [WebCite Cache]
    12. Mohr DC, Schueller SM, Montague E, Burns MN, Rashidi P. The behavioral intervention technology model: An integrated conceptual and technological framework for eHealth and mHealth interventions. J Med Internet Res 2014 Jun 05;16(6):e146 [FREE Full text] [CrossRef] [Medline]
    13. Klasnja P, Pratt W. Healthcare in the pocket: Mapping the space of mobile-phone health interventions. J Biomed Inform 2012 Feb;45(1):184-198 [FREE Full text] [CrossRef] [Medline]
    14. Petit A, Cambon L. Exploratory study of the implications of research on the use of smart connected devices for prevention: A scoping review. BMC Public Health 2016 Jul 11;16:552 [FREE Full text] [CrossRef] [Medline]
    15. Free C, Phillips G, Galli L, Watson L, Felix L, Edwards P, et al. The effectiveness of mobile-health technology-based health behaviour change or disease management interventions for health care consumers: A systematic review. PLoS Med 2013;10(1):e1001362 [FREE Full text] [CrossRef] [Medline]
    16. Perski O, Blandford A, West R, Michie S. Conceptualising engagement with digital behaviour change interventions: A systematic review using principles from critical interpretive synthesis. Transl Behav Med 2017 Jun;7(2):254-267 [FREE Full text] [CrossRef] [Medline]
    17. Whittaker R, Merry S, Dorey E, Maddison R. A development and evaluation process for mHealth interventions: Examples from New Zealand. J Health Commun 2012;17 Suppl 1:11-21. [CrossRef] [Medline]
    18. Lau PW, Lau EY, Wong DP, Ransdell L. A systematic review of information and communication technology-based interventions for promoting physical activity behavior change in children and adolescents. J Med Internet Res 2011 Jul 13;13(3):e48 [FREE Full text] [CrossRef] [Medline]
    19. Agalloco J. Validation: An unconventional review and reinvention. PDA J Pharm Sci Technol 1995;49(4):175-179. [Medline]
    20. Process Validation: General Principles and Practices. Silver Spring, MD: US Food and Drug Administration; 2011 Jan.   URL: https://www.fda.gov/media/71021/download [accessed 2019-09-06] [WebCite Cache]
    21. Kuhn TS. The Structure of Scientific Revolutions. Chicago, IL: University Of Chicago Press; 1996.
    22. Green L, Kreuter M. Health Program Planning: An Educational and Ecological Approach. 4th edition. New York, NY: McGraw-Hill Humanities/Social Sciences/Languages; 2004.
    23. ClinicalTrials.gov. Phase   URL: https://clinicaltrials.gov/ct2/help/glossary/phase [accessed 2019-09-06] [WebCite Cache]
    24. Chapman D. Health-related databases. J Can Acad Child Adolesc Psychiatry 2009 May;18(2):148-149 [FREE Full text] [Medline]
    25. Pressman RS. Software Engineering: A Practitioner's Approach. 5th edition. New York, NY: McGraw-Hill Education; 2001.
    26. Fang FC, Casadevall A. Lost in translation: Basic science in the era of translational research. Infect Immun 2010 Feb;78(2):563-566 [FREE Full text] [CrossRef] [Medline]
    27. Fishbein DH, Ridenour TA, Stahl M, Sussman S. The full translational spectrum of prevention science: Facilitating the transfer of knowledge to practices and policies that prevent behavioral health problems. Transl Behav Med 2016 Mar;6(1):5-16 [FREE Full text] [CrossRef] [Medline]
    28. Royce WW. Managing the development of large software systems: Concepts and techniques. In: Proceedings of the 9th International Conference on Software Engineering. Los Alamitos, CA: IEEE Computer Society Press; 1987 Presented at: 9th International Conference on Software Engineering; March 30-April 2, 1987; Monterey, CA p. 328-338   URL: http://www-scf.usc.edu/~csci201/lectures/Lecture11/royce1970.pdf
    29. Green L. Toward cost-benefit evaluations of health education: Some concepts, methods, and examples. Health Educ Monogr 2016 Aug 27;2(1_suppl):34-64 [FREE Full text] [CrossRef]
    30. Floyd C. A systematic look at prototyping. In: Budde R, Kuhlenkamp K, Mathiassen L, Züllighoven H, editors. Approaches to Prototyping. Berlin, Germany: Springer-Verlag Berlin Heidelberg; 1984:1-18.
    31. Greenwald P, Cullen J. The new emphasis in cancer control. J Natl Cancer Inst 1985 Mar;74(3):543-551. [Medline]
    32. Flay BR. Efficacy and effectiveness trials (and other phases of research) in the development of health promotion programs. Prev Med 1986 Sep;15(5):451-474. [Medline]
    33. Rook P. Controlling software projects. Softw Eng J 1986 Jan;1(1):7-16. [CrossRef]
    34. Boehm BW. A spiral model of software development and enhancement. Comput 1988 May;21(5):61-72. [CrossRef]
    35. Hartson H, Hix D. Toward empirically derived methodologies and tools for human-computer interface development. Int J Man Mach Stud 1989 Oct;31(4):477-494. [CrossRef]
    36. Martin J. Rapid Application Development. Indianapolis, IN: Macmillan Coll Div; 1991.
    37. Onken LS, Blaine JD, Battjes RJ. Behavioral therapy research: A conceptualization of a process. In: Henderson SW, Santos AB, editors. Innovative Approaches for Difficult-to-Treat Populations. Washington, DC: American Psychiatric Press; 1997:477-485.
    38. Bartholomew LK, Parcel GS, Kok G. Intervention mapping: A process for developing theory- and evidence-based health education programs. Health Educ Behav 1998 Oct;25(5):545-563. [CrossRef] [Medline]
    39. Mayhew DJ. The Usability Engineering Lifecycle: A Practitioner's Handbook for User Interface Design (interactive Technologies). London, UK: Morgan Kaufmann Publishers; 1999.
    40. Beck K, Beedle M, Bennekum A, Cockburn A, Cunningham W, Fowler M, et al. agilemanifesto.org. 2001. Manifesto for Agile software development   URL: http://agilemanifesto.org/ [accessed 2019-09-06] [WebCite Cache]
    41. Kukafka R, Johnson SB, Linfante A, Allegrante JP. Grounding a new information technology implementation framework in behavioral science: A systematic analysis of the literature on IT use. J Biomed Inform 2003 Jun;36(3):218-227 [FREE Full text] [Medline]
    42. Collins LM, Murphy SA, Nair VN, Strecher VJ. A strategy for optimizing and evaluating behavioral interventions. Ann Behav Med 2005 Aug;30(1):65-73. [CrossRef] [Medline]
    43. Pagliari C. Design and evaluation in eHealth: Challenges and implications for an interdisciplinary field. J Med Internet Res 2007 May 27;9(2):e15 [FREE Full text] [CrossRef] [Medline]
    44. Boutron I, Moher D, Altman DG, Schulz KF, Ravaud P, CONSORT Group. Methods and processes of the CONSORT Group: Example of an extension for trials assessing nonpharmacologic treatments. Ann Intern Med 2008 Feb 19;148(4):W60-W66. [CrossRef] [Medline]
    45. Boutron I, Altman D, Moher D, Schulz K, Ravaud P, CONSORT NPT Group. CONSORT statement for randomized trials of nonpharmacologic treatments: A 2017 update and a CONSORT extension for nonpharmacologic trial abstracts. Ann Intern Med 2017 Jul 04;167(1):40-47. [CrossRef] [Medline]
    46. Cockburn A. Rochester Institute of Technology. 2008. Using both incremental and iterative development   URL: http:/​/www.​se.rit.edu/​~swen-256/​resources/​UsingBothIncrementalandIterativeDevelopment-AlistairCockburn.​pdf [accessed 2019-09-06] [WebCite Cache]
    47. Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M. Developing and evaluating complex interventions: The new Medical Research Council guidance. Int J Nurs Stud 2013 May;50(5):587-592. [CrossRef] [Medline]
    48. Catwell L, Sheikh A. Evaluating eHealth interventions: The need for continuous systemic evaluation. PLoS Med 2009 Aug;6(8):e1000126 [FREE Full text] [CrossRef] [Medline]
    49. van Gemert-Pijnen JE, Nijland N, van Limburg M, Ossebaard HC, Kelders SM, Eysenbach G, et al. A holistic framework to improve the uptake and impact of eHealth technologies. J Med Internet Res 2011 Dec 05;13(4):e111 [FREE Full text] [CrossRef] [Medline]
    50. Van Velsen L, Wentzel J, Van Gemert-Pijnen JE. Designing eHealth that matters via a multidisciplinary requirements development approach. JMIR Res Protoc 2013 Jun 24;2(1):e21 [FREE Full text] [CrossRef] [Medline]
    51. Michie S, van Stralen MM, West R. The behaviour change wheel: A new method for characterising and designing behaviour change interventions. Implement Sci 2011 Apr 23;6:42 [FREE Full text] [CrossRef] [Medline]
    52. Eysenbach G, CONSORT-EHEALTH Group. CONSORT-EHEALTH: Improving and standardizing evaluation reports of Web-based and mobile health interventions. J Med Internet Res 2011 Dec 31;13(4):e126 [FREE Full text] [CrossRef] [Medline]
    53. Eysenbach G. CONSORT-EHEALTH: Implementation of a checklist for authors and editors to improve reporting of Web-based and mobile randomized controlled trials. Stud Health Technol Inform 2013;192:657-661. [Medline]
    54. Peterson JC, Czajkowski S, Charlson ME, Link AR, Wells MT, Isen AM, et al. Translating basic behavioral and social science research to clinical application: The EVOLVE mixed-methods approach. J Consult Clin Psychol 2013 Apr;81(2):217-230 [FREE Full text] [CrossRef] [Medline]
    55. Riiser K, Løndal K, Ommundsen Y, Sundar T, Helseth S. Development and usability testing of an Internet intervention to increase physical activity in overweight adolescents. JMIR Res Protoc 2013 Jan 28;2(1):e7 [FREE Full text] [CrossRef] [Medline]
    56. van der Weegen S, Verwey R, Spreeuwenberg M, Tange H, van der Weijden T, de Witte L. The development of a mobile monitoring and feedback tool to stimulate physical activity of people with a chronic disease in primary care: A user-centered design. JMIR Mhealth Uhealth 2013 Jul 02;1(2):e8 [FREE Full text] [CrossRef] [Medline]
    57. Smits R, Bryant J, Sanson-Fisher R, Tzelepis F, Henskens F, Paul C, et al. Tailored and integrated Web-based tools for improving psychosocial outcomes of cancer patients: The DoTTI development framework. J Med Internet Res 2014 Mar 14;16(3):e76 [FREE Full text] [CrossRef] [Medline]
    58. Onken LS, Carroll KM, Shoham V, Cuthbert BN, Riddle M. Reenvisioning clinical science: Unifying the discipline to improve the public health. Clin Psychol Sci 2014 Jan 01;2(1):22-34 [FREE Full text] [CrossRef] [Medline]
    59. Kassam-Adams N, Marsac M, Kohser K, Kenardy J, March S, Winston F. A new method for assessing content validity in model-based creation and iteration of eHealth interventions. J Med Internet Res 2015 Apr 15;17(4):e95 [FREE Full text] [CrossRef] [Medline]
    60. Abroms L, Whittaker R, Free C, Mendel Van Alstyne J, Schindler-Ruwisch J. Developing and pretesting a text messaging program for health behavior change: Recommended steps. JMIR Mhealth Uhealth 2015 Dec 21;3(4):e107 [FREE Full text] [CrossRef] [Medline]
    61. Yardley L, Morrison L, Bradbury K, Muller I. The person-based approach to intervention development: Application to digital health-related behavior change interventions. J Med Internet Res 2015 Jan 30;17(1):e30 [FREE Full text] [CrossRef] [Medline]
    62. Czajkowski SM, Powell LH, Adler N, Naar-King S, Reynolds KD, Hunter CM, et al. From ideas to efficacy: The ORBIT model for developing behavioral treatments for chronic diseases. Health Psychol 2015 Oct;34(10):971-982 [FREE Full text] [CrossRef] [Medline]
    63. Nahum-Shani I, Hekler EB, Spruijt-Metz D. Building health behavior models to guide the development of just-in-time adaptive interventions: A pragmatic framework. Health Psychol 2015 Dec;34S:1209-1219 [FREE Full text] [CrossRef] [Medline]
    64. Salisbury C, Thomas C, O'Cathain A, Rogers A, Pope C, Yardley L, et al. TElehealth in CHronic disease: Mixed-methods study to develop the TECH conceptual model for intervention design and evaluation. BMJ Open 2015 Feb 06;5(2):e006448 [FREE Full text] [CrossRef] [Medline]
    65. Gustafson DHJ, Maus A, Judkins J, Dinauer S, Isham A, Johnson R, et al. Using the NIATx model to implement user-centered design of technology for older adults. JMIR Hum Factors 2016 Jan 14;3(1):e2 [FREE Full text] [CrossRef] [Medline]
    66. Mummah SA, Robinson TN, King AC, Gardner CD, Sutton S. IDEAS (Integrate, Design, Assess, and Share): A framework and toolkit of strategies for the development of more effective digital interventions to change health behavior. J Med Internet Res 2016 Dec 16;18(12):e317 [FREE Full text] [CrossRef] [Medline]
    67. Wilhide III CC, Peeples MM, Anthony Kouyaté RC. Evidence-based mHealth chronic disease mobile app intervention design: Development of a framework. JMIR Res Protoc 2016 Feb 16;5(1):e25 [FREE Full text] [CrossRef] [Medline]
    68. Harte R, Glynn L, Rodríguez-Molinero A, Baker P, Scharf T, Quinlan L, et al. A human-centered design methodology to enhance the usability, human factors, and user experience of connected health systems: A three-phase methodology. JMIR Hum Factors 2017 Mar 16;4(1):e8 [FREE Full text] [CrossRef] [Medline]
    69. Maar M, Yeates K, Perkins N, Boesch L, Hua-Stewart D, Liu P, et al. A framework for the study of complex mHealth interventions in diverse cultural settings. JMIR Mhealth Uhealth 2017 Apr 20;5(4):e47 [FREE Full text] [CrossRef] [Medline]
    70. Stephan L, Dytz Almeida E, Guimaraes R, Ley A, Mathias R, Assis M, et al. Processes and recommendations for creating mHealth apps for low-income populations. JMIR Mhealth Uhealth 2017 Apr 03;5(4):e41 [FREE Full text] [CrossRef] [Medline]
    71. Mohr D, Lyon A, Lattie E, Reddy M, Schueller S. Accelerating digital mental health research from early design and creation to successful implementation and sustainment. J Med Internet Res 2017 May 10;19(5):e153 [FREE Full text] [CrossRef] [Medline]
    72. Vilardaga R, Rizo J, Zeng E, Kientz J, Ries R, Otis C, et al. User-centered design of Learn to Quit, a smoking cessation smartphone app for people with serious mental illness. JMIR Serious Games 2018 Jan 16;6(1):e2 [FREE Full text] [CrossRef] [Medline]
    73. Davidson R, Randhawa G, Cash S. Identification of complex health interventions suitable for evaluation: Development and validation of the 8-step scoping framework. JMIR Res Protoc 2019 Mar 05;8(3):e10075 [FREE Full text] [CrossRef] [Medline]
    74. Wang Y, Fadhil A, Lange J, Reiterer H. Integrating taxonomies into theory-based digital health interventions for behavior change: A holistic framework. JMIR Res Protoc 2019 Jan 15;8(1):e8055 [FREE Full text] [CrossRef] [Medline]
    75. Bradway M, Carrion C, Vallespin B, Saadatfard O, Puigdomènech E, Espallargues M, et al. mHealth assessment: Conceptualization of a global framework. JMIR Mhealth Uhealth 2017 May 02;5(5):e60 [FREE Full text] [CrossRef] [Medline]
    76. Shanghai Ranking Consultancy. 2017. Academic ranking of world universities 2017   URL: http://www.shanghairanking.com/ARWU2017.html [accessed 2019-09-06] [WebCite Cache]
    77. Le Pen C, Priol G, Lilliu H. What criteria for pharmaceuticals reimbursement? An empirical analysis of the evaluation of "medical service rendered" by reimbursable drugs in France. Eur J Health Econ 2003;4(1):30-36. [CrossRef] [Medline]
    78. Hussain M, Al-Haiqi A, Zaidan AA, Zaidan BB, Kiah ML, Anuar NB, et al. The landscape of research on smartphone medical apps: Coherent taxonomy, motivations, open challenges and recommendations. Comput Methods Programs Biomed 2015 Dec;122(3):393-408. [CrossRef] [Medline]
    79. El Amrani L, Oude Engberink A, Ninot G, Hayot M, Carbonnel F. Connected health devices for health care in French general medicine practice: Cross-sectional study. JMIR Mhealth Uhealth 2017 Dec 21;5(12):e193 [FREE Full text] [CrossRef] [Medline]
    80. Yen P, Bakken S. Review of health information technology usability study methodologies. J Am Med Inform Assoc 2012;19(3):413-422 [FREE Full text] [CrossRef] [Medline]
    81. Michie S, West R, Campbell R, Brown J, Gainforth H. ABC of Behaviour Change Theories: An Essential Resource for Researchers, Policy Makers, and Practitioners. London, UK: Silverback Publishing; 2014.
    82. Davis R, Campbell R, Hildon Z, Hobbs L, Michie S. Theories of behaviour and behaviour change across the social and behavioural sciences: A scoping review. Health Psychol Rev 2015;9(3):323-344 [FREE Full text] [CrossRef] [Medline]
    83. Czajkowski SM, Lynch MR, Hall KL, Stipelman BA, Haverkos L, Perl H, et al. Transdisciplinary translational behavioral (TDTB) research: Opportunities, barriers, and innovations. Transl Behav Med 2016 Mar;6(1):32-43 [FREE Full text] [CrossRef] [Medline]
    84. Melchert TP. Leaving behind our preparadigmatic past: Professional psychology as a unified clinical science. Am Psychol 2016 Sep;71(6):486-496. [CrossRef] [Medline]
    85. Brunner J, Chuang E, Goldzweig C, Cain CL, Sugar C, Yano EM. User-centered design to improve clinical decision support in primary care. Int J Med Inform 2017 Aug;104:56-64 [FREE Full text] [CrossRef] [Medline]
    86. Matthew-Maich N, Harris L, Ploeg J, Markle-Reid M, Valaitis R, Ibrahim S, et al. Designing, implementing, and evaluating mobile health technologies for managing chronic conditions in older adults: A scoping review. JMIR Mhealth Uhealth 2016 Jun 09;4(2):e29 [FREE Full text] [CrossRef] [Medline]
    87. Epstein RM, Street RL. The values and value of patient-centered care. Ann Fam Med 2011;9(2):100-103 [FREE Full text] [CrossRef] [Medline]
    88. Jaspers MW. A comparison of usability methods for testing interactive health technologies: Methodological aspects and empirical evidence. Int J Med Inform 2009 May;78(5):340-353. [CrossRef] [Medline]
    89. Hood L, Heath JR, Phelps ME, Lin B. Systems biology and new technologies enable predictive and preventative medicine. Science 2004 Oct 22;306(5696):640-643. [CrossRef] [Medline]
    90. Bousquet J, Anto JM, Sterk PJ, Adcock IM, Chung KF, Roca J, et al. Systems medicine and integrated care to combat chronic noncommunicable diseases. Genome Med 2011 Jul 06;3(7):43 [FREE Full text] [CrossRef] [Medline]
    91. Balas EA, Boren SA. Managing clinical knowledge for health care improvement. Yearb Med Inform 2000(1):65-70. [Medline]
    92. Westfall JM, Mold J, Fagnan L. Practice-based research: "Blue Highways" on the NIH roadmap. JAMA 2007 Jan 24;297(4):403-406. [CrossRef] [Medline]
    93. Riley WT, Glasgow RE, Etheredge L, Abernethy AP. Rapid, responsive, relevant (R3) research: A call for a rapid learning health research enterprise. Clin Transl Med 2013 May 10;2(1):10 [FREE Full text] [CrossRef] [Medline]
    94. Bagadiya J. Buddha's Rest Stop. 2017 Jan 03. 151 amazing social media statistics you should know in 2017   URL: https://buddhasreststop.blogspot.com/2017/10/151-amazing-social-media-statistics-you.html [accessed 2017-05-12] [WebCite Cache]
    95. Neiger BL, Thackeray R, Burton SH, Giraud-Carrier CG, Fagen MC. Evaluating social media's capacity to develop engaged audiences in health promotion settings: Use of Twitter metrics as a case study. Health Promot Pract 2013 Mar;14(2):157-162. [CrossRef] [Medline]
    96. Mazzocchi F. Could Big Data be the end of theory in science? A few remarks on the epistemology of data-driven science. EMBO Rep 2015 Oct;16(10):1250-1255 [FREE Full text] [CrossRef] [Medline]
    97. Yildirim O, Gottwald M, Schüler P, Michel MC. Opportunities and challenges for drug development: Public-private partnerships, adaptive designs and big data. Front Pharmacol 2016;7:461 [FREE Full text] [CrossRef] [Medline]
    98. Tabak RG, Khoong EC, Chambers DA, Brownson RC. Bridging research and practice: Models for dissemination and implementation research. Am J Prev Med 2012 Sep;43(3):337-350 [FREE Full text] [CrossRef] [Medline]
    99. Matthews J, Win KT, Oinas-Kukkonen H, Freeman M. Persuasive technology in mobile applications promoting physical activity: A systematic review. J Med Syst 2016 Mar;40(3):72. [CrossRef] [Medline]
    100. Cambon L. Health smart devices and applications...towards a new model of prevention? Eur J Public Health 2017 Jun 01;27(3):390-391 [FREE Full text] [CrossRef]
    101. Moore GF, Audrey S, Barker M, Bond L, Bonell C, Hardeman W, et al. Process evaluation of complex interventions: Medical Research Council guidance. BMJ 2015 Mar 19;350:h1258 [FREE Full text] [CrossRef] [Medline]

    ‎
    ACTS: Accelerated Creation-To-Sustainment
    BIT: behavioral intervention technology
    CeHRes: Center for eHealth Research
    CEPS: Plateforme universitaire Collaborative d’Evaluation des programmes de Prévention et de Soins de support
    CONSORT: Consolidated Standards of Reporting Trials
    CONSORT-EHEALTH: Consolidated Standards of Reporting Trials of Electronic and Mobile HEalth Applications and onLine TeleHealth
    DoTTI: Design and develOpment, Testing early iterations, Testing for effectiveness, Integration and implementation
    DREAM-GLOBAL: Diagnosing hypeRtension—Engaging Action and Management in Getting LOwer Bp in Aboriginal and LMIC (lower- and middle-income countries)
    EVOLVE: Explore Values, Operationalize and Learn, and eValuate Efficacy
    FDA: US Food and Drug Administration
    IDEAS: Integrate, Design, Assess, and Share
    IT: information technology
    JITAI: just-in-time adaptive intervention
    JMIR: Journal of Medical Internet Research
    MOST: Multiphase Optimization STrategy
    MRC: Medical Research Council
    NHS: National Health Service
    NIATx: Network for the Improvement of Addiction Treatment
    NIDA: National Institute on Drug Abuse
    NIH: National Institutes of Health
    ORBIT: Obesity-Related Behavioral Intervention Trials
    PRECEDE-PROCEED: Predisposing, Reinforcing, and Enabling Constructs in Educational Diagnosis and Evaluation—Policy, Regulatory, and Organizational Constructs in Educational and Environmental Development
    SMS: short message service
    TECH: TElehealth in CHronic disease
    TUDER: Targeting, Understanding, Designing, Evaluating, and Refining
    WHO: World Health Organization

    Edited by G Eysenbach; submitted 03.02.19; peer-reviewed by S Bacon, P Rashidi, E Montague; comments to author 23.03.19; revised version received 08.06.19; accepted 02.08.19; published 16.10.19

    Copyright

    ©François Carbonnel, Gregory Ninot. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 16.10.2019.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.