Original Paper
Abstract
Background: Type 2 diabetes is 2 to 3 times more common among people with severe mental illness (SMI). Self-management is crucial, with additional challenges faced by people with SMI. Therefore, it is essential that any diabetes self-management program for people with SMI addresses the unique needs of people living with both conditions and the inequalities they experience within health care services.
Objective: We combined theory, empirical evidence, and co-design approaches to develop a type 2 diabetes self-management intervention for people with SMI.
Methods: The development process encompassed 4 steps: step 1 involved prioritizing the mechanisms of action (MoAs) and behavior change techniques (BCTs) for the intervention. Using findings from primary qualitative research and systematic reviews, we selected candidate MoAs to target in the intervention and candidate BCTs to use. Expert stakeholders then ranked these MoAs and BCTs using a 2-phase survey. The average scores were used to generate a prioritized list of MoAs and BCTs. During step 2, we presented the survey results to an expert consensus workshop to seek expert agreement with the definitive list of MoAs and BCTs for the intervention and identify potential modes of delivery. Step 3 involved the development of trigger films using the evidence from steps 1 and 2. We used animations to present the experiences of people with SMI managing diabetes. These films were used in step 4, where we used a stakeholder co-design approach. This involved a series of structured workshops, where the co-design activities were informed by theory and evidence.
Results: Upon the completion of the 4-step process, we developed the DIAMONDS (diabetes and mental illness, improving outcomes and self-management) intervention. It is a tailored self-management intervention based on the synthesis of the outputs from the co-design process. The intervention incorporates a digital app, a paper-based workbook, and one-to-one coaching designed to meet the needs of people with SMI and coexisting type 2 diabetes.
Conclusions: The intervention development work was underpinned by the MoA theoretical framework and incorporated systematic reviews, primary qualitative research, expert stakeholder surveys, and evidence generated during co-design workshops. The intervention will now be tested for feasibility before undergoing a definitive evaluation in a pragmatic randomized controlled trial.
doi:10.2196/43597
Keywords
Introduction
Health Inequalities
Long-term physical conditions (LTCs), also referred to as chronic diseases, develop earlier and are 2 to 3 times more common in people with severe mental illness (SMI; eg, schizophrenia, schizoaffective disorder, psychosis, and bipolar disorder) than in the general population [
- ]. Life expectancy is substantially reduced by 15 to 20 years among people with SMI [ , ].The rates of obesity [
], poor diet and nutrition [ ], and smoking [ , ] are higher and physical activity levels [ ] are lower among people with SMI, likely contributing to the higher prevalence of LTCs and premature mortality in this population. Type 2 diabetes is twice as prevalent in people with SMI, and those with SMI and coexisting type 2 diabetes are more likely to experience poorer clinical outcomes than those with type 2 diabetes alone [ - ]. Reducing the risk of poor clinical outcomes for people with type 2 diabetes is contingent on effective self-management [ - ]. Self-management of type 2 diabetes includes activities such as healthy eating, smoking cessation, stress management, physical activity, blood glucose monitoring, and adherence to medication as prescribed [ ]. In England, structured diabetes self-management education programs are recommended for those recently diagnosed with type 2 diabetes, and they provide people with an opportunity to develop their knowledge and skills for self-management [ , , ]. They are effective in improving self-management behaviors, clinical outcomes, and associated health care costs [ - ].However, people with SMI and type 2 diabetes rarely receive support for self-management [
]. People with SMI can experience symptoms such as avolition, social withdrawal, or hallucinations, as well as side effects from psychiatric medication, such as fatigue and excessive hunger [ ], making it difficult for them to effectively engage in self-management activities. Moreover, it is unclear whether existing diabetes self-management education programs are effective for people with SMI, as they are typically excluded from research on these programs [ ]. A Cochrane review of self-management interventions for people with diabetes and SMI included only 1 trial and concluded that there was insufficient evidence about the effectiveness of such interventions in this population [ ].Digital Exclusion
There is a growing emphasis within health care services on the use of digital technologies, including within services for supporting the self-management of LTCs, such as diabetes [
]. Digital exclusion will likely further exacerbate health inequality among people with SMI [ ]. Although digital exclusion for those with SMI is declining in the United Kingdom [ ], the proportion of people with SMI who use the internet or computers remains a minority [ ], and reasons for nonengagement with digital technology include the lack of access to devices or Wi-Fi, sociodemographic factors, the lack of independence, the lack of skills, and the symptoms of SMI [ ]. The COVID-19 pandemic further compounded this inequality, as health services responded to national restrictions by providing services remotely using digital technologies [ ]. Those without access to these technologies, including those without the knowledge or skills to use them, and those with symptoms that hinder the use of digital technology faced difficulties accessing health care services during the peak pandemic years [ ]. This shift to digital provision of health care services is likely to persist [ ], potentially reinforcing preexisting structural and symptom-driven digital exclusion among people with SMI and widening health inequalities among this population.Goal of This Study
Therefore, it is essential that any diabetes self-management program for people with SMI addresses the unique needs of people living with both conditions and the inequalities they experience within health care services. We aimed to co-design a self-management intervention for people with SMI and coexisting type 2 diabetes living in the community, incorporating digital technology and paper-based options and addressing the specific barriers that people living with coexisting SMI and type 2 diabetes experience when managing these conditions together.
Methods
Overview
We used an approach that integrated behavioral theory and empirical evidence with co-design methods to ensure that the intervention we developed would be effective for improving diabetes self-management in people with SMI, as well as be feasible to deliver and acceptable to both service users and those delivering the intervention. This study was conducted in the United Kingdom, and the approach to co-design was modeled on experience-based co-design, which is primarily a service improvement methodology increasingly used to develop complex interventions; it starts with understanding users’ experiences, needs, and preferences and involves working in partnership with users to design or improve services based on these understandings [
].An overview of the intervention development process is shown in
. This paper reports the development process for the intervention (stages 2-4), and separate papers report the findings of the systematic reviews [ , ] and qualitative research [ ] that formed the basis of stage 1.Theoretical Framework
The intervention development process was structured around understanding the mechanisms of action (MoAs) for the self-management behaviors of people with SMI and coexisting LTCs [
]. MoAs are theoretical constructs that are thought to explain the process through which specific behavior change techniques (BCTs) affect self-management [ ].Patient and Public Involvement
DIAMONDS Voice is a service user and carer group that has contributed to the DIAMONDS (diabetes and mental illness, improving outcomes and self-management) program since 2015. At the time of this co-design study, DIAMONDS Voice consisted of 8 people with SMI and 1 family member. DIAMONDS Voice contributed to the development of the co-design study protocol, attended DIAMONDS research team meetings, and supported the delivery of the study through input on expert consensus surveys, participation in workshops, involvement in synthesis sessions, and design of the intervention materials.
Program Development Grant
Preparatory Research: Underlying Principles and Candidate Intervention Components
The DIAMONDS program is underpinned by a previous work conducted as part of a National Institute for Health and Care Research Programme Development Grant (RP-DG-1214-10002). During this 12-month development period, we conducted a scoping review of the existing diabetes self-management and education programs for people with type 2 diabetes combined with a series of multidisciplinary workshops to explore how to adapt diabetes self-management for people with SMI and conducted 5 patient and public involvement (PPI) panels to build PPI capacity.
The scoping review identified >150 unique interventions for supporting the self-management of diabetes. Although differing in format, duration, and delivery, the interventions were broadly similar in content, and there was no leading candidate approach for adaptation for people with SMI. Many of the interventions from the scoping review were included in a previous systematic review of diabetes self-management interventions by Chrvala et al [
]. This review identified 118 interventions that included goal setting in at least 1 area of self-management and concluded that interventions with a combination of group and individual sessions and those with a mean contact time of >10 hours were more effective than those without these features. We used 2 intervention planning workshops alongside consultation with our PPI panel to obtain feedback on the suitability of different intervention types (eg, group or individual or combined) and to identify how self-management interventions may need to be adapted to meet the needs of people with SMI. The workshops were attended by the DIAMONDS research team and clinical professionals from primary care, diabetes services, and psychiatric services. A total of 19 people attended the first workshop, and 21 people attended the second workshop.In these workshops, we explored the specific barriers to and facilitators of self-management in people with SMI with experts in the field. Discussions around the educational content of programs concluded that any intervention should cover the 7 main areas of self-management included in the American Association of Diabetes Educators 7 Self-Care Behaviors (healthy coping, healthy eating, being active, taking medication, monitoring, problem-solving, and reducing risks) [
]. The workshops also specified a need for additional one-to-one support, which was not often provided in the existing diabetes self-management and education programs. There were mixed opinions about the extent to which we should use digital technologies and the potential acceptability and feasibility of this approach for people with SMI. The attendees of the workshops also discussed opportunities to train people working in mental health services to deliver a tailored intervention. Diabetes and mental health clinicians, including physicians, specialist nurses, and dietitians, agreed that this approach was feasible. However, they pointed out that although people working in mental health services have the skills to support people with SMI, they may know little about diabetes and would need specific training for this.PPI panel meetings confirmed that there were mixed opinions regarding the use of digital technologies to support health. The panel agreed that digital applications could form part of an intervention but should not be used to the exclusion of other platforms and intervention content. PPI meetings also identified that in-person support would need to be an essential aspect of any intervention to help reduce anxiety and address low motivation. There was consensus that both group and individual intervention sessions would be needed to support people with SMI and diabetes. There was agreement that group sessions need to be optional and stand-alone and not compulsory so that people could attend if able to.
Key Learning Underpinning the Co-design Process
Our preparatory research informed our decision to develop a bespoke self-management intervention for people with type 2 diabetes and SMI. The intervention would need to include both group and one-to-one support to ensure accessibility and engagement for people with SMI. The intervention should also incorporate a digital application, but there should be options available for participants who do not have access to digital technology or who struggle with using it, such as a participant workbook. In addition, the barriers to self-management experienced by people with SMI meant that the intervention would need to be adapted to individual needs. These design principles were taken forward and served as the foundation for a phased approach to intervention development using a co-design methodology.
Intervention Development Process
Stage 1: Primary and Secondary Analysis of Qualitative and Quantitative Evidence About the Determinants of Self-management Behaviors in SMI and LTCs
Stage 1 identified MoAs that were mapped against the determinants of self-management. This stage combined a secondary analysis of published qualitative and quantitative evidence about the enablers of, barriers to, and determinants (mapped against MoAs) of self-management for adults living in the community with SMI and LTCs [
, ] with an analysis of in-depth qualitative interviews with staff, informal caregivers, and service users in England about the lived experience of self-management of LTCs and SMI [ ]. Evidential links between all MoAs and the determinants of self-management behavior were cross tabulated. An example of this cross-tabulation mapping the MoA “attitude towards the behaviour” with the determinants identified from the primary qualitative research [ ] is shown in .Statement | Illustrative quote |
I take medication for my mental health by injection which I don’t like because it hurts but it keeps me well |
|
I get fed up with taking my medications because they cause side effects |
|
I enjoy doing some types of exercise but not others |
|
I drink alcoholic drinks because I enjoy it |
|
I don’t always monitor my blood glucose levels because it seems like too much hard work |
|
I have a great counsellor that I see every week to help me cope with my conditions |
|
I used to take part in relaxation sessions which were really good |
|
I take part in leisure activities and I like doing them |
|
aP: participant.
bI: interviewer.
Stage 2: Consensus and Prioritization Exercise With Stakeholders
Stage 2 involved a 2-phase survey and consultation with expert stakeholders (who participated in stage 1) to prioritize and select candidate BCTs based on established theoretical and empirical evidence about behavior change interventions. Expert stakeholders were first tasked with ranking (on a 4-point Likert scale) the MoAs identified in stage 1 in terms of the perceived strength of their association with the self-management of diabetes and modifiability. A total of 21 people responded to the electronic survey; the stakeholders included health care professionals working in primary care (n=1, 5%), diabetes services (n=5, 24%), chronic obstructive pulmonary disease care (n=4, 19%), and mental health services (n=2, 10%); clinical academics (n=3, 14%); carers (n=1, 5%); service users (n=4, 19%); and health care managers (n=1, 5%).
The DIAMONDS research team then mapped the average expert-rated rankings for MoA-BCT associations with the cross-tabulation mapping from stage 1 to generate a list of candidate MoAs and BCT links. The second phase of the survey was administered again to the expert stakeholders approximately 2 months after the completion of the first survey to judge each MoA and BCT link according to whether it was acceptable and practical using the APEASE (Affordability, Practicability, Effectiveness, Acceptability, Safety and Equity) criteria. The BCTs and MoAs that the stakeholders reached a consensus on were carried forward to inform the potential content in the co-design process.
For each MoA, between 4 and 11 BCT links were identified. Following the prioritization process, the DIAMONDS research team selected the BCTs that offered the most promise in terms of potential efficacy, acceptability, and deliverability using the results of both surveys, preparatory work, expert consultation, and PPI. The 22 BCTs that were included in the intervention are listed in
.BCTs | Linked MoAs |
Goal setting (behavior) | Intentions; goals; and memory, attention, and decision processes |
Problem-solving | Knowledge; skills; beliefs about capabilities; memory, attention, and decision processes; and behavioral regulation |
Action planning | Memory, attention, and decision processes; behavioral regulation; and behavioral cueing |
Graded tasks | Skills; beliefs about capabilities; and memory, attention, and decision processes |
Review behavior goals | Beliefs about capabilities, reinforcement, intentions, goals, and feedback processes |
Feedback on behavior | Reinforcement, motivation, and feedback processes |
Focus on past success | Beliefs about capabilities |
Prompts or cues | Memory, attention, and decision processes; environmental context and resources; behavioral regulation; and behavioral cueing |
Habit formation | Memory, attention, and decision processes; behavioral regulation; and behavioral cueing |
Information about health consequences | Knowledge, beliefs about consequences, intentions, and attitude toward behavior |
Information on how to perform the behavior | Knowledge, skills, and beliefs about capabilities |
Demonstration of the behavior | Knowledge, skills, and beliefs about capabilities |
Body changes | Beliefs about capabilities |
Verbal persuasion about capability | Beliefs about capabilities |
Self-monitoring of behavior | Behavioral regulation, feedback processes, and behavioral cueing |
Feedback on behavior | Reinforcement, motivation, and feedback processes |
Reduce negative emotions | Skills, emotion, and behavioral regulation |
Self-monitoring the outcome of behavior | Goals |
Monitoring of emotional consequences | Beliefs about consequences and emotion |
Information about emotional consequences | Knowledge, beliefs about consequences, and emotion |
Social support (unspecified) | Environmental context and resources |
Social support (practical) | Environmental context and resources |
Stage 3: Development of Trigger Films
Stage 3 happened in parallel with the first 2 stages and involved the creation of animated trigger films. Trigger films, also known as catalyst films, are films designed to generate discussion on a specific topic [
]. The films were based on key themes and illustrative points from our qualitative study [ ] and qualitative evidence synthesis [ ] about the experiences of people with coexisting SMI and LTCs, their informal caregivers, and health care professionals. These themes included the impact of SMI on self-management behaviors, interaction between SMI and LTCs, and barriers to and facilitators of self-management. During stage 3, we worked with an external collaborator, the digital design agency HMA (HMA Digital Marketing Ltd) [ ]. We used animations within the trigger films to integrate and illustrate various themes and points rapidly. Subsequently, we used these trigger films and the personas we developed in stage 4 as part of the co-design process. We developed 2 trigger films, one consisting of an overview of 3 characters who had SMI and LTCs and another providing different self-management scenarios for each of the 3 characters to highlight the difficulties people with SMI experience when trying to manage their health.Stage 4: Co-design Workshops With Staff, Service Users, and Carers
Stage 4 included an initial discovery day workshop that lasted 5.5 hours, followed by 5 co-design workshops with staff and service users over 8 months. The 5 co-design workshops took place face-to-face and lasted approximately 15 hours in total. The co-design workshops were facilitated by mHabitat (currently Thrive by Design Ltd [
]) and were attended by our digital design partner HMA [ ]. During the workshops, the DIAMONDS research team, digital design partner, service users, and health care professionals worked closely together to inform the development of the intervention.Co-design Participants
Workshop participants were recruited from three groups: (1) service users (ie, individuals with SMI and LTCs); (2) informal caregivers; and (3) health care professionals involved in the care of people with SMI, LTCs, or both.
We recruited participants from the existing DIAMONDS research cohorts (ie, people who had taken part in the program development work or related research and had given consent to be contacted again), DIAMONDS Voice, and 3 National Health Service (NHS) mental health trusts in Yorkshire, United Kingdom. The eligibility criteria for workshop participants are listed in
.All potential participants received an invitation pack about the study via post, including an invitation letter, a participant information sheet, and an expression of interest form that included consent to contact. The DIAMONDS research team contacted individuals who expressed an interest and explained the study to them. All participants signed a consent form at the beginning of the first discovery day for the duration of the co-design workshops. Overall, 24 people were recruited for stage 4 of the study. Of them, 8 (33%) people were recruited and provided consent but were unable to attend any of the workshops, 7 (88%) of whom were health care professionals, and 1 (12%) was a service user. One of the health care professionals attended the first discovery day workshop and then withdrew from the study. The number of attendees by group for each workshop is listed in
.Eligibility criteria for workshop participants.
- Service users
- People who have coexisting severe mental illness (SMI) and type 2 diabetes were eligible. People were excluded if they were in an acute psychiatric ward during the study recruitment period, if they lacked the capacity to participate, or if they had experienced a recent relapse of their SMI. People with a diagnosis of gestational diabetes only were also excluded.
- Informal caregivers
- People who provided informal support to people with coexisting SMI and type 2 diabetes, including family members and friends, were eligible.
- Health care professionals
- People who had professional experience supporting people with either SMI or type 2 diabetes were eligible.
Workshop and participant designation | Participant, n (%) | ||
Discovery day (n=9) | |||
Service users | 3 (33) | ||
Informal caregivers | 2 (22) | ||
Health care professionals | 4 (44) | ||
Workshop 1 (n=6) | |||
Service users | 3 (50) | ||
Informal caregiversa | 3 (50) | ||
Health care professionals | 0 (0) | ||
Workshop 2 (n=8) | |||
Service usersa | 4 (50) | ||
Informal caregivers | 2 (25) | ||
Health care professionals | 2 (25) | ||
Workshop 3 (n=10) | |||
Service usersa | 5 (50) | ||
Informal caregiversa | 3 (30) | ||
Health care professionals | 2 (20) | ||
Workshop 4 (n=9) | |||
Service usersa | 4 (44) | ||
Informal caregiversa | 3 (33) | ||
Health care professionals | 2 (22) | ||
Workshop 5 (n=5) | |||
Service users | 3 (60) | ||
Informal caregivers | 2 (40) | ||
Health care professionals | 0 (0) |
aIncluding a member of DIAMONDS Voice.
Discovery Day Workshop
The discovery day workshop was split between separate morning sessions for staff and service users and a joint afternoon session. To address the barriers to using a digital application to support self-management, the participants were asked to write down their frustrations surrounding digital technology on Post-it notes and place these on a “wall of frustration,” which included a poster illustrating participant quotes about experiences with digital technology from our qualitative research [
]. Following this initial exercise, the trigger films were shown separately to each group.The service user group was asked to complete a journey mapping activity, mapping how a typical day in their life compared with and related to the characters in the trigger films. Following journey mapping, the service users were invited to identify problematic moments they experienced during a typical day when trying to manage their health. During this activity, the service user participants expressed that their ability to engage in self-management behaviors was heavily compromised by their mental health symptoms. Furthermore, as people with SMI, the service users described how they often felt intimidated by going to public places, such as gyms, and that this acted as a barrier to physical activity. The service users also identified that deciding where to do food shopping and their ability to cook were important considerations that underpinned their ability to manage their physical health.
The staff group was asked to draw up a care pathway for service users with SMI and type 2 diabetes and to determine whether there were any points along this pathway where digital technology could be incorporated. Following these separate discovery sessions, service users and health care professionals were brought together for a joint discovery exercise. The health care professional group presented the pathway they had designed, whereas the service user group presented an outline of “a day in the life of” to highlight the self-management activities they engaged in each day. The service users highlighted the challenges they often experienced, such as access to health care services and the acceptability of the services provided. Health care professionals recommended that optimized care pathways for people with SMI should consider the inclusion of structured education about type 2 diabetes.
captures the perspectives of the workshop participants about the challenges of living with SMI and LTCs.In the joint final session on the discovery day, the 2 groups reviewed the outputs from the earlier sessions. Variable sleep patterns, taking medication, and eating were highlighted as issues that contributed to a person’s ability to engage in self-management. Mental health was identified as having a much greater impact on quality of life and health and, therefore, was typically prioritized over the self-management of type 2 diabetes.
Discovery Day Synthesis
Following the discovery day, an in-person synthesis session was held to develop paper prototype components for a self-management intervention that could be delivered within an app or a workbook with prioritized functions and placeholder content. This prototype was based on a synthesis of the information gained during the discovery day and the MoAs and BCTs identified in stages 1 and 2. The prototype was used as a framework for the following co-design workshops.
During the synthesis session, the research team, DIAMONDS Voice, and staff from HMA and mHabitat categorized the notes from the discovery day as follows: (1) content (eg, smoking cessation), (2) functionality (eg, any interaction with the intervention), (3) insights (into the behavior of people with SMI), and (4) risk (eg, clinical or data risk).
Next, the group collaboratively prioritized content in the following order:
- Motivation/daily routine (including sleep)
- Healthy eating
- Physical activity/exercise
- Stress management/emotion/mood
- Taking medication
For each of the prioritized functions, the workshop participants explored how that function could be delivered. For example, the development of a daily routine could be supported by self-monitoring, consisting of prompts within an app.
Co-design Workshops
A total of 5 co-design workshops were conducted after the discovery day and discovery day synthesis session. At least 1 member of the DIAMONDS research team attended each workshop to observe, take notes, and provide information about the existing evidence. Workshop content was developed iteratively based on a priori objectives, preceding synthesis sessions, and both verbal (in-person) and written inputs from the wider DIAMONDS team. Photographs of the workshop outputs were used as a basis for the synthesis sessions, during which the DIAMONDS team and DIAMONDS Voice iteratively tested and refined intervention prototypes, which included both the digital app and the manualized paper-based content.
Workshop 1: Testing of a Wireframe of the Prototype and Prioritization of the Target Health Behaviors
Bringing together discussions and learning from the previous steps, a paper prototype of an app entitled “Change One Thing” was presented to the participants. The prototype focused on supporting small lifestyle changes, hence the name “Change One Thing.” It was presented using a series of questions (eg, Do you sleep well?) and designed to offer daily prompts for a chosen activity (eg, for going to sleep at a regular time).
The participants were divided into 2 groups: a mix of service users and informal caregivers. One group was asked to provide feedback on the prototype, whereas the other group was asked to rank by priority 5 self-management behaviors that had been identified from stages 1 to 4, with number 1 being the highest priority (ie, the most important) and number 5 being the lowest priority (ie, the least important). Medication and managing sleep were the top priority self-management behaviors for most of the participants.
The participants felt that the app would help them remember to take their medication, and the inclusion of videos and the ability to track progress could motivate them to engage in self-management. The participants also discussed the use of one-to-one and group sessions to deliver the intervention. They agreed that one-to-one sessions would be necessary for engagement and help people work toward specific goals related to healthy eating, exercise, and sleep, whereas group sessions were considered to add social support.
Workshop 2: Ideation and Prototype Sense Checking
The participants were shown ideas for the “look and feel” of the intervention materials (app and workbook), the initial setup process for the app, and the process for goal setting and action planning. Discussion about the appearance, setup, and utility of the app was facilitated in relation to exemplar content about stress management and physical activity.
Participants reviewed 5 potential options for the look and feel of the “Change One Thing” app and workbook and identified preferred questions for supporting goal setting and action planning. Using the key priorities identified in workshop 1, suggested goals were provided for medication and sleep, with options for how these goals could be phrased and monitored. Additional content for the workbook and app was suggested for how people could add personalized changes to physical activity (eg, walking to the postbox) and how people could seek social support to manage stress. The participants placed an emphasis on person-centered content that considered personal ability and interests, with an intervention facilitator asking questions rather than making suggestions.
Workshop 3: Refined Prototype Sense Checking
The proposed components included 16 weekly one-to-one meetings, a digital app or a workbook entitled “Change One Thing,” and monthly group sessions. These components were informed by the preparatory research that was conducted during the program development grant, alongside the information and preferences provided during the discovery day and first 2 workshops and input from DIAMONDS Voice members.
The workshop participants were split into 2 groups: one group of health care professionals and one group of service users and informal caregivers. The health care professional group focused on the training and support needs of intervention facilitators. The proposed support included supervision, training on diabetes and mental health, goal setting, and personalization of care. Health care professionals also identified the need for processes to be set up around referrals and reporting for participants who may be at risk or who may become unwell while taking part in the intervention.
The service user group reviewed the specificity and emotional valence of the language and wording proposed for the initial goal-setting and action planning activities, which would be completed either in the “Change One Thing” app or workbook. The participants suggested alternative goal statements, which were not appropriate, as they could not be framed within action plans or contradicted the evidence base (eg, sleeping in on the weekend instead of having a regular sleep schedule). The service user and health care professional groups then reconvened and provided feedback on their activities.
Workshop 4: User Acceptance Testing and Evaluation
This workshop focused on the clinical safety of the intervention. Topics of concern addressed the problems associated with using the “Change One Thing” app and workbook, the one-to-one sessions, and the end of the intervention. The problems associated with using the app included concerns about the security of personal data, malfunctioning of the app, and phone charging. These could be best solved by ensuring that the workbook delivers the same content and function as the app. The participants also identified that after the final intervention sessions, people will need to be signposted to relevant support services. A photograph of this evaluative process, outlining the concerns and potential solutions associated with the first sessions, daily use of the app, and weekly one-to-one sessions, is shown in
.Workshop 5: Further User Acceptance Testing and Evaluation to Inform the Refinement of the Intervention
User testing and evaluation in workshop 4 were truncated because of a fire alarm; therefore, we repurposed workshop 5 to complete these processes. The participants watched a series of educational animations that had been specifically created for inclusion in the “Change One Thing” app and workbook (presented in the workbook as comic strips) and were asked to provide feedback about the acceptability, utility, and relevance of each video. The videos focused on sleep, medication taking, fluid intake, and physical activity and were based on the personas previously shown in the trigger films during the discovery day. After viewing the videos, the participants were asked a series of questions to review their understanding of the educational content. The participants demonstrated a good understanding of what the videos were trying to communicate and provided suggestions on how they could be improved.
Digital app user testing sessions were facilitated by the research team and ran in parallel with the review of the educational animations. The participants were asked to use the app and provide feedback on its functionality and content. Despite being smartphone users, several participants found it difficult to use the app. The problems that were encountered included repeated crashing or locking of the screen and difficulties using some of the features, such as setting the time or using sliders within the app. Therefore, it was recommended that service users be supported by the intervention facilitators to set up the app and enable accessibility options during the onboarding process.
Design of the Intervention Materials
The main outputs from the co-design process were the Change One Thing app and workbook and the DIAMONDS coach manual and training handbook. We hosted several collaborative meetings with Nifty Fox Creative (Nifty Fox Creative Ltd) [
] and DIAMONDS Voice to ensure that the layout, language, design, and organization of the workbook were accessible, engaging, fit for purpose, and matched the content of the app. We worked with the Leicester Diabetes Centre [ ] to design the coach manual and training handbook. The Change One Thing app was designed and developed during the co-design process by our digital design partner HMA [ ] and finalized by the app developer Pipe and Piper (Pipe & Piper Ltd) [ ].Ethics Approval
Ethics approval was obtained from the North West–Greater Manchester West Research Ethics Committee (reference: 19/NW/0356).
Results
Intervention Overview and Structure
The DIAMONDS intervention aims to support the self-management of type 2 diabetes for people with coexisting SMI by improving diabetes self-management knowledge and skills and enhancing the specific capabilities, opportunities, and motivations that influence diabetes self-management in this population. In particular, the intervention provides tailored and person-centered support for setting behavioral goals, action planning, problem-solving, and increasing physical activity and provides and facilitates peer support. The logic model for the DIAMONDS intervention is provided in
.The DIAMONDS intervention will be delivered by a trained facilitator called a “DIAMONDS Coach.” The participants will be offered weekly sessions either face-to-face, over the phone, or over video call with their DIAMONDS Coach for up to 16 weeks, in combination with the daily use of the DIAMONDS workbook or the DIAMONDS app called “Change One Thing.” They can also engage in optional in-person monthly group sessions facilitated by 2 DIAMONDS Coaches, with other participants receiving the intervention. We will establish the feasibility and acceptability of the intervention before evaluating its clinical effectiveness and cost-effectiveness in a randomized controlled trial [
]. An overview of the core intervention components is presented in .Delivering the BCTs: Goal Setting
As outlined in the logic model provided in
, the intervention aims to deliver several BCTs to support people in making lifestyle changes that will support their physical health. To demonstrate how these BCTs will be delivered using the Change One Thing app or the workbook, we will outline the BCT goal setting within the DIAMONDS intervention. In each one-to-one session, the DIAMONDS Coach will follow the subsequent steps to support the participant in setting a goal.Using the app or workbook, the DIAMONDS Coach will ask the participants what they would like to focus on during the following week. The app and workbook have been designed to focus on goals relating to sleep and taking medication, as these were identified during the development process as the main priorities for service users. However, the participants may want to focus on a different problem and, therefore, can identify their own goal if preferred. Examples of these choices in the app and workbook are shown in
and , respectively.Once the participant has identified what they want to focus on, the DIAMONDS Coach will support them in developing a goal. If the participant is using the Change One Thing app and chooses to focus on medication or sleep, the goal statement will be automatically generated. If the participant is using the participant workbook, they can choose from a list of goal statements, which are identical to those generated in the app, related to the chosen behavior—medication or sleep. If the participant has not identified medication or sleep as the behavior that they want to focus on, the DIAMONDS Coach will support them in generating a goal statement related to the other behavior they have identified (eg, healthy eating), ensuring that the goal is realistic and achievable.
The goal will be automatically recorded in the app once it has been generated by the participant. If the participant uses the workbook, then the participant or DIAMONDS Coach will record the goal in the goal-setting section of the workbook.
Discussion
Principal Findings
This paper describes how we integrated an evidence-based intervention development approach and a co-design methodology to develop a complex intervention for supporting people with SMI and type 2 diabetes in self-managing their health. Drawing on the philosophy that intervention development is best served by integrating theory, evidence, and person-centered approaches [
], we set out to combine behavioral theory with empirical evidence about the determinants of self-management behaviors and user-centered design. A consistent thread in our development work has been the use of the MoA framework to underpin our research on self-management behaviors in people with SMI and LTCs, combined with the use of the BCT taxonomy as a means of linking determinants to intervention components. This theoretical lens has offered a means to interpret systematic review evidence about the determinants of self-management behaviors in people with SMI and diabetes [ ]. Our understanding of the drivers and lived experiences of managing LTCs in the context of SMI was further extended by primary qualitative exploration and qualitative evidence synthesis [ ]. In this sense, we were able to ensure that the first phases of the co-design process were founded on a firm theoretical and evidential basis, leading to more targeted and person-centered co-design workshops.Person-centered approaches revolved around thinking about how end users would interact with and use the intervention. End users of the DIAMONDS intervention are not only service users with SMI and diabetes (and their informal caregivers) but also health professionals tasked with intervention delivery. The consistent involvement of end users throughout the co-design process was critical to managing tensions between developing an intervention that could be feasibly delivered for a clinical trial (with the primary goal of reducing glycated hemoglobin [HbA1c]) and developing an intervention that met the priorities of service users. For example, service users highlighted their desire to focus on mental health, whereas health care professionals felt that it was important to incorporate physical activity as a means of reducing HbA1c. Previous research has highlighted that the management of mental health is frequently the priority for people with SMI over the self-management of LTCs [
]. Trials of other bespoke self-management interventions for people with SMI have failed to demonstrate an effect on clinical biomarkers, despite improving other measures that may be of a higher priority to people with SMI, such as hospitalization [ ]. By co-designing an intervention that accommodates goals for both physical and mental health, for example, by framing physical activity as a method of improving mood, we have potentially offered a solution to overcome the disconnect between the priorities of service users and clinical imperative to reduce HbA1c.In addition, our person-centered approach was key to the decision-making about including a digital component in the final intervention specification. People with SMI are less likely to have access to digital technology [
] and face more barriers to its use when they do have physical access (eg, lacking necessary skills) [ ], but there is evidence that digital health technologies can feasibly support people with SMI [ ] and that access to the internet and smartphones is increasing in this population, particularly in younger age groups [ ]. To minimize digital exclusion, we, therefore, developed the “Change One Thing” app but built in user flexibility by ensuring that the same components are provided in the workbook.Although the effectiveness of this self-management intervention has yet to be demonstrated, our approach to co-designing embraced innovation processes likely to be critical to its future scalability. By involving different stakeholders from different professional roles in the design process, we recognized that digital interventions especially need to be co-created in the context of existing and future service delivery models and work practices [
]. In the English NHS, the establishment of integrated care systems offers opportunities to scale interventions such as the DIAMONDS intervention beyond the local level and reach the needs of populations [ ]. A key enabler here will be identifying a workforce able to support the onboarding and use of the digital content of this intervention. As part of the new job roles linked to the delivery of the NHS Long Term Plan, it is possible that health and well-being coaches could be trained to support the access to and use of the DIAMONDS intervention, including the Change One Thing app. Social support that can build trust about the safety and utility of the intervention among users is also a known enabler of scaling up digital interventions [ ]. Going forward, we will conduct a process evaluation within the definitive trial, exploring the barriers to and facilitators of the implementation of the DIAMONDS intervention. This process evaluation will consider the role of informal caregivers in the delivery of the intervention and whether they are crucial resources for wider scalability beyond the trial context. In these ways, our co-design study is embedded within a broader program of work that is necessarily anticipating how to address the barriers to the future scalability of the intervention.Conclusions
We adopted an inclusive and a participatory approach to co-designing a flexible and user-focused intervention for support the self-management of physical and mental health in people with SMI and diabetes. We integrated behavioral science theory, empirical evidence, and co-design approach to structure the intervention development process, informed by our linked work to model and evidence the barriers to and facilitators of self-management behaviors in this population. This approach shows the value of combining behavioral theory with critical insights from primary and secondary research to maximize the utility and success of co-design with diverse groups of stakeholders.
Acknowledgments
This paper reports the work undertaken as part of the DIAMONDS program, which is funded by the National Institute for Health Research under its Programme Grants for Applied Research (project number RP-PG-1016-20003). PAC and SG are partly funded by the UK Research and Innovation Closing the Gap Network+ (ES/S004459/1) and the National Institute for Health and Care Research Applied Research Collaboration Yorkshire and Humber.
The authors would like to acknowledge the work and contribution of the wider DIAMONDS research team, including members of our patient and public involvement panel DIAMONDS Voice.
Conflicts of Interest
DS is an expert adviser to the National Institute for Health and Care Excellence Centre for Guidelines; the views expressed are those of the authors and not those of National Institute for Health and Care Excellence. RIGH has received honoraria for speaker engagement, conference attendance, and advisory boards from Abbott, AstraZeneca, Boehringer Ingelheim, European Association for the Study of Diabetes, Eli Lilly, Encore, Janssen, Menarini, Napp Pharmaceuticals Limited, Novo Nordisk, Omniamed, Roche, and Sanofi.
Logic model.
DOCX File , 17 KBReferences
- Ruggeri M, Leese M, Thornicroft G, Bisoffi G, Tansella M. Definition and prevalence of severe and persistent mental illness. Br J Psychiatry 2000 Aug;177:149-155. [CrossRef] [Medline]
- De Hert M, Dekker J, Wood D, Kahl K, Holt R, Möller HJ. Cardiovascular disease and diabetes in people with severe mental illness position statement from the European Psychiatric Association (EPA), supported by the European Association for the Study of Diabetes (EASD) and the European Society of Cardiology (ESC). Eur Psychiatry 2009 Sep;24(6):412-424 [FREE Full text] [CrossRef] [Medline]
- Reilly S, Olier I, Planner C, Doran T, Reeves D, Ashcroft D, et al. Inequalities in physical comorbidity: a longitudinal comparative cohort study of people with severe mental illness in the UK. BMJ Open 2015 Dec 15;5(12):e009010 [FREE Full text] [CrossRef] [Medline]
- Brown S, Kim M, Mitchell C, Inskip H. Twenty-five year mortality of a community cohort with schizophrenia. Br J Psychiatry 2010 Feb 02;196(2):116-121 [FREE Full text] [CrossRef] [Medline]
- Osborn DP, Levy G, Nazareth I, Petersen I, Islam A, King MB. Relative risk of cardiovascular and cancer mortality in people with severe mental illness from the United Kingdom's General Practice Rsearch Database. Arch Gen Psychiatry 2007 Feb 01;64(2):242-249. [CrossRef] [Medline]
- Afzal M, Siddiqi N, Ahmad B, Afsheen N, Aslam F, Ali A, et al. Prevalence of overweight and obesity in people with severe mental illness: systematic review and meta-analysis. Front Endocrinol (Lausanne) 2021;12:769309 [FREE Full text] [CrossRef] [Medline]
- Dipasquale S, Pariante C, Dazzan P, Aguglia E, McGuire P, Mondelli V. The dietary pattern of patients with schizophrenia: a systematic review. J Psychiatr Res 2013 Feb;47(2):197-207 [FREE Full text] [CrossRef] [Medline]
- Gilbody S, Peckham E, Bailey D, Arundel C, Heron P, Crosland S, et al. Smoking cessation for people with severe mental illness (SCIMITAR+): a pragmatic randomised controlled trial. Lancet Psychiatry 2019 May;6(5):379-390 [FREE Full text] [CrossRef] [Medline]
- Szatkowski L, McNeill A. Diverging trends in smoking behaviors according to mental health status. Nicotine Tob Res 2015 Mar;17(3):356-360 [FREE Full text] [CrossRef] [Medline]
- Stubbs B, Firth J, Berry A, Schuch FB, Rosenbaum S, Gaughran F, et al. How much physical activity do people with schizophrenia engage in? A systematic review, comparative meta-analysis and meta-regression. Schizophr Res 2016 Oct;176(2-3):431-440. [CrossRef] [Medline]
- Vinogradova Y, Coupland C, Hippisley-Cox J, Whyte S, Penny C. Effects of severe mental illness on survival of people with diabetes. Br J Psychiatry 2010 Oct;197(4):272-277. [CrossRef] [Medline]
- Schoepf D, Potluri R, Uppal H, Natalwala A, Narendran P, Heun R. Type-2 diabetes mellitus in schizophrenia: increased prevalence and major risk factor of excess mortality in a naturalistic 7-year follow-up. Eur Psychiatry 2012 Jan;27(1):33-42. [CrossRef] [Medline]
- Wu C, Lai M, Gau S. Complications and mortality in patients with schizophrenia and diabetes: population-based cohort study. Br J Psychiatry 2015 Nov;207(5):450-457. [CrossRef] [Medline]
- Type 2 diabetes in adults: management. National Institute for Health and Care Excellence. 2015 Dec 2. URL: https://www.nice.org.uk/guidance/ng28 [accessed 2022-06-26]
- Diabetes UK. Improving supported self-management for people with diabetes. Diabetes UK. URL: http://www.diabetes.org.uk/Documents/Reports/Supported_self-management.pdf [accessed 2022-06-26]
- Action for Diabetes. NHS England. 2014 Jan 01. URL: https://www.england.nhs.uk/rightcare/wp-content/uploads/sites/40/2016/08/act-for-diabetes-31-01.pdf [accessed 2022-06-26]
- American Association of Diabetes Educators. An effective model of diabetes care and education: revising the AADE7 self-care behaviors. Diabetes Educ 2020 Apr;46(2):139-160. [CrossRef] [Medline]
- All-Party Parliamentary Group for Diabetes. Taking Control: Supporting people to self-manage their diabetes. All-Party Parliamentary Group for Diabetes. 2015 Mar. URL: https://jdrf.org.uk/wp-content/uploads/2015/10/APPG-Diabetes-Report-Education-Final-Report.pdf [accessed 2022-06-26]
- Newman S, Steed L, Mulligan K. Self-management interventions for chronic illness. Lancet 2004 Oct;364(9444):1523-1537. [CrossRef]
- Nobis S, Lehr D, Ebert DD, Baumeister H, Snoek F, Riper H, et al. Efficacy of a web-based intervention with mobile phone support in treating depressive symptoms in adults with type 1 and type 2 diabetes: a randomized controlled trial. Diabetes Care 2015 May;38(5):776-783 [FREE Full text] [CrossRef] [Medline]
- Norris SL, Engelgau MM, Narayan KM. Effectiveness of self-management training in type 2 diabetes: a systematic review of randomized controlled trials. Diabetes Care 2001 Mar;24(3):561-587. [CrossRef] [Medline]
- Smith R, Han L, Ali S, Prady S, Taylor J, Hughes T, et al. Glucose, cholesterol and blood pressure in type II diabetes: a longitudinal observational study comparing patients with and without severe mental illness. J Psychiatr Ment Health Nurs 2019 Nov;26(9-10):347-357 [FREE Full text] [CrossRef] [Medline]
- Carswell C, Brown J, Lister J, Ajjan R, Alderson S, Balogun-Katung A, DIAMONDS Research team. The lived experience of severe mental illness and long-term conditions: a qualitative exploration of service user, carer, and healthcare professional perspectives on self-managing co-existing mental and physical conditions. BMC Psychiatry 2022 Jul 19;22(1):479 [FREE Full text] [CrossRef] [Medline]
- Roberts SH, Bailey JE. An ethnographic study of the incentives and barriers to lifestyle interventions for people with severe mental illness. J Adv Nurs 2013 Nov 28;69(11):2514-2524. [CrossRef] [Medline]
- McBain H, Mulligan K, Haddad M, Flood C, Jones J, Simpson A. Self management interventions for type 2 diabetes in adult people with severe mental illness. Cochrane Database Syst Rev 2016 Apr 27;4(4):CD011361 [FREE Full text] [CrossRef] [Medline]
- Hadjiconstantinou M, Schreder S, Brough C, Northern A, Stribling B, Khunti K, et al. Using intervention mapping to develop a digital self-management program for people with type 2 diabetes: tutorial on MyDESMOND. J Med Internet Res 2020 May 11;22(5):e17316 [FREE Full text] [CrossRef] [Medline]
- Spanakis P, Peckham E, Mathers A, Shiers D, Gilbody S. The digital divide: amplifying health inequalities for people with severe mental illness in the time of COVID-19. Br J Psychiatry 2021 Oct 23;219(4):529-531 [FREE Full text] [CrossRef] [Medline]
- Robotham D, Satkunanathan S, Doughty L, Wykes T. Do we still have a digital divide in mental health? A five-year survey follow-up. J Med Internet Res 2016 Nov 22;18(11):e309 [FREE Full text] [CrossRef] [Medline]
- Tobitt S, Percival R. Switched on or switched off? A survey of mobile, computer and Internet use in a community mental health rehabilitation sample. J Ment Health 2019 Feb;28(1):4-10. [CrossRef] [Medline]
- The Lancet Digital Health. Digital tools for mental health in a crisis. Lancet Digital Health 2021 Apr;3(4):e204. [CrossRef]
- Litchfield I, Shukla D, Greenfield S. Impact of COVID-19 on the digital divide: a rapid review. BMJ Open 2021 Oct 12;11(10):e053440 [FREE Full text] [CrossRef] [Medline]
- Donetto S, Tsianakas V. Using Experience-based Co-design to Improve the Quality of Healthcare: Mapping Where We Are Now and Establishing Future Directions. London: King’s College London; 2014.
- Coventry PA, Young B, Balogun-Katang A, Taylor J, Brown JV, Kitchen C, et al. Determinants of physical health self-management behaviours in adults with serious mental illness: a systematic review. Front Psychiatry 2021 Aug 18;12:723962 [FREE Full text] [CrossRef] [Medline]
- Balogun-Katung A, Carswell C, Brown J, Coventry P, Ajjan R, Alderson S, DIAMONDS research team. Exploring the facilitators, barriers, and strategies for self-management in adults living with severe mental illness, with and without long-term conditions: a qualitative evidence synthesis. PLoS One 2021;16(10):e0258937 [FREE Full text] [CrossRef] [Medline]
- Michie S, Carey RN, Johnston M, Rothman AJ, de Bruin M, Kelly MP, et al. From theory-inspired to theory-based interventions: a protocol for developing and testing a methodology for linking behaviour change techniques to theoretical mechanisms of action. Ann Behav Med 2018 May 18;52(6):501-512 [FREE Full text] [CrossRef] [Medline]
- Michie S, Richardson M, Johnston M, Abraham C, Francis J, Hardeman W, et al. The behavior change technique taxonomy (v1) of 93 hierarchically clustered techniques: building an international consensus for the reporting of behavior change interventions. Ann Behav Med 2013 Aug;46(1):81-95 [FREE Full text] [CrossRef] [Medline]
- Chrvala CA, Sherr D, Lipman RD. Diabetes self-management education for adults with type 2 diabetes mellitus: a systematic review of the effect on glycemic control. Patient Educ Couns 2016 Jun;99(6):926-943 [FREE Full text] [CrossRef] [Medline]
- HMA. URL: https://www.hma.co.uk/ [accessed 2022-06-26]
- Thrive By Design. URL: https://www.thrivebydesign.org.uk/ [accessed 2022-06-26]
- Nifty Fox Creative. URL: https://www.niftyfoxcreative.com/ [accessed 2022-06-26]
- Leicester Diabetes Centre. URL: https://www.leicesterdiabetescentre.org.uk/ [accessed 2022-06-26]
- Pipe & Piper. URL: https://pipeandpiper.co.uk/ [accessed 2022-06-26]
- Brown JV, Ajjan R, Alderson S, Böhnke JR, Carswell C, Doherty P, et al. The DIAMONDS intervention to support self-management of type 2 diabetes in people with severe mental illness: study protocol for a single-group feasibility study. SSM Mental Health 2022 Dec;2:100086. [CrossRef]
- Band R, Bradbury K, Morton K, May C, Michie S, Mair FS, et al. Intervention planning for a digital intervention for self-management of hypertension: a theory-, evidence- and person-based approach. Implement Sci 2017 Feb 23;12(1):25 [FREE Full text] [CrossRef] [Medline]
- Osborn D, Burton A, Hunter R, Marston L, Atkins L, Barnes T, et al. Clinical and cost-effectiveness of an intervention for reducing cholesterol and cardiovascular risk for people with severe mental illness in English primary care: a cluster randomised controlled trial. Lancet Psychiatry 2018 Feb;5(2):145-154 [FREE Full text] [CrossRef] [Medline]
- Spanakis P, Wadman R, Walker L, Heron P, Mathers A, Baker J, et al. Measuring the digital divide among people with severe mental ill health using the essential digital skills framework. Perspect Public Health 2022 Aug 05:17579139221106399 [FREE Full text] [CrossRef] [Medline]
- Batra S, Baker R, Wang T, Forma F, DiBiasi F, Peters-Strickland T. Digital health technology for use in patients with serious mental illness: a systematic review of the literature. Med Devices (Auckl) 2017 Oct;10:237-251 [FREE Full text] [CrossRef] [Medline]
- Ennis L, Rose D, Denis M, Pandit N, Wykes T. Can't surf, won't surf: the digital divide in mental health. J Ment Health 2012 Aug;21(4):395-403 [FREE Full text] [CrossRef] [Medline]
- Cresswell KM, Bates DW, Sheikh A. Ten key considerations for the successful implementation and adoption of large-scale health information technology. J Am Med Inform Assoc 2013 Jun;20(e1):e9-13 [FREE Full text] [CrossRef] [Medline]
- Bell A. Briefing 55: Integrated Care Systems and mental health. Centre for Mental Health. 2020 Feb. URL: https://www.centreformentalhealth.org.uk/sites/default/files/2020-02/Briefing_55_Integrated_Care_Systems.pdf [accessed 2022-06-26]
- Schlieter H, Marsch LA, Whitehouse D, Otto L, Londral AR, Teepe GW, et al. Scale-up of digital innovations in health care: expert commentary on enablers and barriers. J Med Internet Res 2022 Mar 11;24(3):e24582 [FREE Full text] [CrossRef] [Medline]
Abbreviations
APEASE: Affordability, Practicability, Effectiveness, Acceptability, Safety and Equity |
BCT: behavior change technique |
DIAMONDS: diabetes and mental illness, improving outcomes and self-management |
HbA1c: glycated hemoglobin |
LTC: long-term condition |
MoA: mechanism of action |
NHS: National Health Service |
PPI: patient and public involvement |
SMI: severe mental illness |
Edited by T Leung; submitted 17.10.22; peer-reviewed by R Jansen, R Hawkes; comments to author 07.02.23; revised version received 20.02.23; accepted 14.03.23; published 12.05.23
Copyright©Claire Carswell, Peter A Coventry, Jennifer V E Brown, Sarah L Alderson, Keith Double, Simon Gilbody, Richard I G Holt, Rowena Jacobs, Jennie Lister, David Osborn, David Shiers, Najma Siddiqi, Johanna Taylor, Ian Kellar, DIAMONDS Research Group. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 12.05.2023.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.