This paper is in the following e-collection/theme issue:

Patient-Reported Outcome Measures (PROMs) (274) Diabetes Health Services and System Innovations (103) Web-based and Mobile Health Interventions (3092) Diabetes Self-Management (462) Insomnia and Sleep Hygiene (151) Sleep Monitoring, Sleep Quality, Sleep Disorders (167)

Published on in Vol 25 (2023)

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/44002, first published .
Psychosocial Outcomes Among Users and Nonusers of Open-Source Automated Insulin Delivery Systems: Multinational Survey of Adults With Type 1 Diabetes

Psychosocial Outcomes Among Users and Nonusers of Open-Source Automated Insulin Delivery Systems: Multinational Survey of Adults With Type 1 Diabetes

Psychosocial Outcomes Among Users and Nonusers of Open-Source Automated Insulin Delivery Systems: Multinational Survey of Adults With Type 1 Diabetes

Authors of this article:

Jasmine Schipp1, 2 Author Orcid Image ;   Christel Hendrieckx1, 3 Author Orcid Image ;   Katarina Braune4, 5, 6, 7 Author Orcid Image ;   Christine Knoll4, 6 Author Orcid Image ;   Shane O’Donnell8 Author Orcid Image ;   Hanne Ballhausen4, 6, 7 Author Orcid Image ;   Bryan Cleal9 Author Orcid Image ;   Mandy Wäldchen8 Author Orcid Image ;   Dana M Lewis10 Author Orcid Image ;   Katarzyna A Gajewska11, 12 Author Orcid Image ;   Timothy C Skinner1 Author Orcid Image ;   Jane Speight1, 3 Author Orcid Image
Article Authors Cited by Tweetations (3) Metrics

    Original Paper

    1The Australian Centre for Behavioural Research in Diabetes, Carlton, Australia

    2Section for Health Services Research, Institute of Public Health, University of Copenhagen, Copenhagen, Denmark

    3School of Psychology, Deakin University, Burwood, Australia

    4Department of Paediatric Endocrinology and Diabetes, Charité – Universitätsmedizin Berlin, Berlin, Germany

    5Institute of Medical Informatics, Charité – Universitätsmedizin Berlin, Berlin, Germany

    6Berlin Institute of Health, Berlin, Germany

    7Dedoc Labs GmbH, Berlin, Germany

    8School of Sociology & School of Medicine, University College Dublin, Dublin, Ireland

    9Diabetes Management Research, Steno Diabetes Center Copenhagen, Copenhagen, Denmark

    10#OpenAPS, Seattle, WA, United States

    11Diabetes Ireland, Dublin, Ireland

    12School of Public Health, University College Cork, Cork, Ireland

    Corresponding Author:

    Jasmine Schipp, PhD

    The Australian Centre for Behavioural Research in Diabetes

    Suite G01, 15-31 Pelham Street

    Carlton, 3053

    Australia

    Phone: 61 03 9244 6448

    Email: schippj@deakin.edu.au


    Background: Emerging research suggests that open-source automated insulin delivery (AID) may reduce diabetes burden and improve sleep quality and quality of life (QoL). However, the evidence is mostly qualitative or uses unvalidated, study-specific, single items. Validated person-reported outcome measures (PROMs) have demonstrated the benefits of other diabetes technologies. The relative lack of research investigating open-source AID using PROMs has been considered a missed opportunity.

    Objective: This study aimed to examine the psychosocial outcomes of adults with type 1 diabetes using and not using open-source AID systems using a comprehensive set of validated PROMs in a real-world, multinational, cross-sectional study.

    Methods: Adults with type 1 diabetes completed 8 validated measures of general emotional well-being (5-item World Health Organization Well-Being Index), sleep quality (Pittsburgh Sleep Quality Index), diabetes-specific QoL (modified DAWN Impact of Diabetes Profile), diabetes-specific positive well-being (4-item subscale of the 28-item Well-Being Questionnaire), diabetes treatment satisfaction (Diabetes Treatment Satisfaction Questionnaire), diabetes distress (20-item Problem Areas in Diabetes scale), fear of hypoglycemia (short form of the Hypoglycemia Fear Survey II), and a measure of the impact of COVID-19 on QoL. Independent groups 2-tailed t tests and Mann-Whitney U tests compared PROM scores between adults with type 1 diabetes using and not using open-source AID. An analysis of covariance was used to adjust for potentially confounding variables, including all sociodemographic and clinical characteristics that differed by use of open-source AID.

    Results: In total, 592 participants were eligible (attempting at least 1 questionnaire), including 451 using open-source AID (mean age 43, SD 13 years; n=189, 41.9% women) and 141 nonusers (mean age 40, SD 13 years; n=90, 63.8% women). Adults using open-source AID reported significantly better general emotional well-being and subjective sleep quality, as well as better diabetes-specific QoL, positive well-being, and treatment satisfaction. They also reported significantly less diabetes distress, fear of hypoglycemia, and perceived less impact of the COVID-19 pandemic on their QoL. All were medium-to-large effects (Cohen d=0.5-1.5). The differences between groups remained significant after adjusting for sociodemographic and clinical characteristics.

    Conclusions: Adults with type 1 diabetes using open-source AID report significantly better psychosocial outcomes than those not using these systems, after adjusting for sociodemographic and clinical characteristics. Using validated, quantitative measures, this real-world study corroborates the beneficial psychosocial outcomes described previously in qualitative studies or using unvalidated study-specific items.

    J Med Internet Res 2023;25:e44002

    doi:10.2196/44002

    Keywords

    artificial; diabetes mellitus; hypoglycaemia; pancreas; patient-reported outcome measures, surveys, and questionnaires; quality of life; sleep quality; type 1 


    Type 1 diabetes is a chronic auto-immune condition caused when the pancreas stops making the hormone insulin, which is essential to regulating blood glucose levels. Facing the daunting task of trying to imitate a functioning pancreas, people with type 1 diabetes must perform complex and relentless self-care activities designed to keep glucose levels in a safe, predefined target range to avoid both hyper- and hypoglycemia (by calculating and administering insulin doses matched to carbohydrate intake, balanced with physical activity and hormonal changes, among other factors affecting glycemic levels). This burden can cause considerable emotional distress and often negatively impact their quality of life (QoL) [1,2].

    Advancements in diabetes treatments and technologies can assist in these two main tasks: (1) insulin administration and (2) glucose monitoring (Figure 1 [3-9]). Insulin can either be administered through multiple daily injections with a “pen injector” device or through an insulin pump, which is a wearable electronic device that pumps insulin through a thin plastic tube (cannula) inserted under the skin. Glucose levels can be monitored with a drop of blood (commonly collected by pricking a finger) or through a sensor inserted under the skin to assess glucose in the interstitial fluid using a continuous glucose monitor (CGM).

    A recent review of randomized controlled trials (RCTs) and cohort studies investigating the impacts of diabetes technologies on psychosocial outcomes using validated person-reported outcome measures (PROMs) has shown that insulin pumps, CGMs, sensor-augmented pumps, and hybrid closed-loop technologies are often associated with considerable psychosocial benefits [10]. However, there appear to be fewer reported benefits in generic PROMs (eg, sleep quality) [10].

    While insulin pumps and CGMs can improve glycemic and psychosocial outcomes [10], they do not automate the task of insulin delivery. Such automation would potentially reduce the burden and further improve outcomes for the individual. The most advanced diabetes technologies have the potential to do so, with automated insulin adjustments and dosing calculated through an algorithm linking a CGM to an insulin pump. These are known as automated insulin delivery (AID) systems. In recent years, both commercial and open-source AID systems have been developed [11]. Before commercial AID systems became available to the public, open-source AID systems were built by people with diabetes (and the parents of children with diabetes) using free-to-access algorithms (Figure 1) [12]. Several studies have demonstrated improved glycemic outcomes, consistently showing greater time-in-range and improved hemoglobin A1c (HbA1c) [13].

    In contrast to commercial systems, there has been relatively little research into the psychosocial outcomes of those using open-source AID systems. Early surveys showed benefits for sleep and QoL but were limited by the use of study-specific (unvalidated) survey questions [12,14,15]. Evidence of similar benefits has also been derived from qualitative studies, including content analysis of data shared through Twitter (subsequently rebranded X), which also demonstrated reduced diabetes burden [16,17]. A total of 2 small studies (N=35 and N=15) using validated PROMs have demonstrated benefits for diabetes distress [14] and fear of hypoglycemia [18]. Recently, a larger cohort study (N=239) found significant improvements, from baseline to 6 months of open-source AID use, in diabetes distress, sleep quality, fear of hypoglycemia, and hypoglycemia confidence [19]. To date, there has been no large-scale investigation of psychosocial outcomes using validated PROMs comparing those using and not using open-source AID systems. This has been considered a “missed opportunity” [20].

    Thus, our aim was to compare the psychosocial outcomes of adults with type 1 diabetes using and not using open-source AID systems through the completion of validated PROMs in a multinational web-based survey.

    ‎
    Figure 1. Timeline depicting the development of type 1 diabetes management and technology. This timeline is based on publications on insulin administration [3-5], glucose monitoring [6,7], intermittently scanned continuous glucose monitors (CGMs) [8], and open-source automated insulin delivery (AID) systems [9].

    Overview

    The study design was cocreated by the “Outcomes of Patients’ Evidence with Novel, Do-it-Yourself Artificial Pancreas Technology Project Consortium” (OPEN) [21], which comprises an interdisciplinary team of clinicians, social scientists, data scientists, open-source AID developers, biomedical researchers, and public health researchers, many of them with lived experience in type 1 diabetes and in the use of open-source AID systems. This study was part of a larger, overall study conducted by OPEN. The overall survey was piloted with 8 volunteers from the diabetes online community. Minor edits were made based on their feedback.

    We recruited participants through diabetes online community platforms (eg, Diabetes Daily), including Facebook groups supporting the open-source AID community (eg, Looped and AndroidAPS users), the OPEN international health care professional network [22], and the OPEN project Twitter and Facebook accounts, website, and newsletter. The survey was available in English only and open for data collection for 11 weeks (September-November 2020).

    Participants were eligible for inclusion if they were adults (aged 18 years or older) with type 1 diabetes and attempted at least one PROM in the survey (eg, completed the first question). Adults with other types of diabetes were excluded from this analysis. Participants currently using an open-source AID system (a self-built automated insulin delivery system, either AndroidAPS, Loop, or OpenAPS) were included as the “user” group and self-reported their duration of use.

    Study Materials

    The survey included 8 validated generic, diabetes-specific, and hypoglycemia-specific PROMs. Brief descriptions, scoring details, and interpretations are presented in Table S1 in Multimedia Appendix 1. Briefly, the psychosocial outcomes and measures included:

    • General emotional well-being: 5-item World Health Organization Well-Being Index (WHO-5) [23]. This has been validated in adults with type 1 diabetes, has demonstrated reliability, and has been shown to have sensitivity and specificity for use as a screening tool for depressive symptoms among those with type 1 diabetes [24].
    • Subjective sleep quality: Pittsburgh Sleep Quality Index (PSQI) [25]. The PSQI has demonstrated reliability and validity [26] and has previously been used in studies of adults with type 1 diabetes [27,28].
    • Diabetes-specific quality of life: modified DAWN Impact of Diabetes Profile (DIDP) [29]. The 7-item DIDP has satisfactory psychometric properties and has been validated for use by adults with type 1 diabetes [29].
    • Diabetes-specific positive well-being: 4-item subscale of the 28-item Well-being Questionnaire (W-BQ28) [30]. The W-BQ28 is a valid and reliable measure of generic and diabetes-specific well-being [31].
    • Satisfaction with diabetes treatment: Diabetes Treatment Satisfaction Questionnaire (DTSQ) [32]. The DTSQ has satisfactory psychometric properties. It is reliable, has construct validity, demonstrates sensitivity to change, and has discriminatory power [32].
    • Diabetes-specific distress: 20-item Problem Areas in Diabetes (PAID) scale [33]. The PAID is psychometrically robust for assessing diabetes distress [34].
    • Fear of hypoglycemia: short form of the Hypoglycemia Fear Survey II (HFS-II SF) [35]. This matches the factor structure of the original instrument while maintaining reliability [36].
    • Impact of the COVID-19 pandemic on QoL: COVID-19 Impact on Quality of Life scale [37]. Adapted from the validated 7-item DIDP [29].

    Their selection was informed by our previous qualitative work [38] (summarized in Table S2 in Multimedia Appendix 1), previous surveys using unvalidated measures [12,15-17], and the lived experiences with open-source AID of the OPEN research team. At the time of measure selection, none of the studies using validated measures had been published. Where possible, we selected widely used measures to enable comparisons with published research. Overall participant burden and experience were also considerations, so we prioritized the selection of brief and, ideally, positively worded measures [39].

    To characterize the study sample, we collected self-reported demographic data (ie, gender, age, country of origin, ethnicity, employment, income, and education level) and clinical data (ie, diabetes duration, most recent self-reported HbA1c, type of diabetes management, and presence of complications).

    Statistical Analysis

    We performed all data processing and statistical analyses using SPSS Statistics (version 27; SPSS Inc). Missing data were dealt with according to guidance in the PROM scoring manuals (when available). When this guidance was not available, Cronbach α (and α if items were deleted) was used to guide whether missing data could be tolerated without loss of internal consistency reliability, enabling data to be imputed by using the mean response of the individual to that PROM. Total scores were calculated according to the developers’ guidance (Table S1 in Multimedia Appendix 1).

    We used chi-square tests and independent groups 2-tailed t tests to assess differences in self-reported demographic and clinical characteristics and PROM scores by use or nonuse of open-source AID systems. For PROM scores, a Bonferroni-adjusted α level of P<.005 per test was accepted as significant, accounting for the number of t tests conducted. As assumptions of normality were not met, we also performed nonparametric tests (the Mann-Whitney U test). The significance remained unchanged; therefore, the results of the parametric analyses are reported alone. To adjust the analysis for potentially confounding variables, we used analysis of covariance tests, including as covariates all sociodemographic and clinical characteristics that showed differences by use of open-source AID.

    Ethical Considerations

    Ethical approval was granted by the Life Sciences Human Research Ethics Committee, University College Dublin (LS-20-37) and the Ethics Committee of Charité–Universitätsmedizin Berlin (EA2/206/21). Potential participants had access to a participant information sheet and provided informed consent electronically (ticking a box) through REDCap (Research Electronic Data Capture; Vanderbilt University), a secure web application used to build and maintain web-based surveys and databases. The data have been deidentified. No financial compensation was provided to the participants.


    Figure 2 shows that, of the 787 people who responded to the survey advertisement, a total of 592 were eligible for inclusion in this analysis. Of these, a total of 451 individuals were using an open-source AID system (users) and 141 were not (nonusers).

    Sociodemographic characteristics are reported in Table 1, and clinical characteristics are reported in Table 2. In this study, a total of 47 countries were represented, across all 6 of the World Health Organization regions. Participants using open-source AID systems (compared with nonusers) were more likely to be men and have higher income levels (Table 1).

    Participants using open-source AID systems (compared with nonusers) had a longer duration of type 1 diabetes, lower self-reported HbA1c, and were less likely to have diabetes-related complications (Table 2).

    Participants in the nonuser group managed their diabetes in various ways (Table 2). Most (128/141, 90.7%) were using some form of advanced diabetes technology, including commercial AID, sensor-augmented pump therapy, insulin pump therapy, and CGM. Some participants were using multiple methods simultaneously or switching between methods; a total of 32.6% (46/141) of participants were using a sensor-augmented pump or commercial AID system (with 1 participant reporting using both a sensor-augmented pump and a commercial AID system) in combination with other self-management methods. A total of 58.2% (82/141) of participants were not using a sensor-augmented pump or commercial-AID system but were using either a CGM, an insulin pump, or both in combination with other self-management methods. A minority (8/141, 5.7%) were using multiple daily injections and manual blood glucose checks alone, or in addition to inhaled insulin or sodium-glucose cotransporter-2 inhibitors. A further 3.5% (5/141) of participants did not report their methods of managing diabetes.

    Table 3 shows between-group differences for PROM scale scores. Higher scores indicate greater emotional well-being, worse sleep quality, a greater negative impact of diabetes on QoL, greater diabetes-specific positive well-being, greater satisfaction with diabetes treatment, greater diabetes distress, greater fear of hypoglycemia, and a greater negative impact of the COVID-19 pandemic and restrictions on QoL. The number of participants who completed each questionnaire is reported in the “Completed questionnaires” column, with some questionnaires having lower completion rates due to attrition.

    Compared with nonusers, adults with type 1 diabetes using open-source AID systems had greater general emotional well-being, subjective sleep quality, diabetes-specific QoL, diabetes-specific positive well-being, and diabetes treatment satisfaction (all P<.001). They also had lower diabetes distress, fear of hypoglycemia, and perceived less impact of the COVID-19 pandemic on their QoL than nonusers (all P<.001). All effect sizes were medium or large.

    Where available, participants who fell above or below the cutoff scores for the PROMs were calculated. For the WHO-5, a total of 15.1% (68/451) of users and 35.5% (50/141) of nonusers reported a score <13, indicating likely depression. For the PSQI, a total of 32% (128/400) of users and 56.7% (63/111) of nonusers reported a score >5, indicating suboptimal sleep quality. For the PAID, a total of 14.1% (58/411) of users and 30.2% (38/126) of nonusers reported a score ≥40, indicating severe diabetes distress.

    When adjusted for between-group differences in gender, region of origin, income level, diabetes duration, and most recent self-reported HbA1c, all between-group differences in PROM scores remained significant (P<.005), indicating that the differences between groups were not due to these sociodemographic or clinical characteristics.

    ‎
    Figure 2. Diagram of participant inclusion and attrition. Adults with other types of diabetes (6 with type 2 diabetes, 21 with latent autoimmune diabetes, 6 with maturity-onset diabetes of the young, 1 with type 3c diabetes, and 1 participant with partial loss of pancreas) were excluded. AID: automated insulin delivery.
    Table 1. Participants’ self-reported demographic characteristics for users and nonusers of open-source automated insulin delivery (AID). Percentages may not add to 100 due to rounding.

    		Users (n=451)Nonusers (n=141)P value
    Gender (men)a, n (%)262 (58.1)50 (35.5)<.001
    Age (years)a, mean (SD)43.0 (12.7)40.4 (13.1).04
    Region of origina, n (%)<.001

    		African4 (0.9)0 (0)

    		Region of the Americas67 (15)38 (27.1)

    		South-East Asian3 (0.7)2 (1.4)

    		European348 (77.8)90 (64.3)

    		Eastern Mediterranean1 (0.2)4 (2.9)

    		Western Pacific24 (5.4)6 (4.3)
    Ethnicity, n (%).27
    White418 (92.7)123 (87.2)
    Education level, n (%).41

    		Incomplete high school8 (1.8)1 (0.7)

    		High school education48 (10.6)12 (8.5)

    		Undergraduate education186 (41.2)70 (49.6)

    		Postgraduate education194 (43)57 (40.4)
    Occupational status, n (%).68

    		Employed (full or part-time)351 (77.8)103 (73)

    		Unemployed15 (3.3)7 (5)

    		Retired35 (7.8)11 (7.8)

    		Student or apprentice33 (7.3)13 (9.2)

    		None of the above or rather not say17 (3.7)9 (4.9)
    Income levela (US $), n (%).002

    		<49,99996 (21.3)50 (35.5)

    		50,000-99,999140 (31)41 (29.1)

    		100,000- 199,999105 (23.3)20 (14.2)

    		>200,00027 (6)4 (2.8)

    aP<.05 for between-group differences by use (or not) of open-source AID.

    Table 2. Participants’ self-reported clinical characteristics for users and nonusers of open-source automated insulin delivery (AID). Percentages may not add to 100 due to rounding.

    		Users (n=451)Nonusers (n=141)P value
    Diabetes duration (years)a, mean (SD)26.7 (14.3)21.6 (12.7)<.001
    Most recent self-reported HbA1ca,b , mean (SD)<.001

    		%6.2 (0.9)6.8 (1.0)

    		mmol/mol44.7 (9.9)50.5 (10.6)
    Open-source AID type (users only)c, n (%)N/Ad

    		AndroidAPS298 (66.1)N/A

    		Loop141 (31.3)N/A

    		OpenAPS36 (8)N/A
    Open-source AID duration (years; users only), mean (SD)1.77 (1.0)N/AN/A
    Diabetes management (nonusers only)c, n (%)

    		Combined glucose monitor and insulin deliveryN/A


    		Commercial AID systemN/A23 (16.3)


    		Sensor-augmented pump therapyN/A24 (17)

    		Glucose monitoringN/A


    		Continuous glucose monitorN/A77 (54.6)


    		Manual blood glucose checksN/A36 (25.5)

    		Insulin deliveryN/A


    		Insulin pump (not sensor augmented)N/A43 (30.5)


    		Multiple daily injectionsN/A46 (32.6)


    		Inhaled insulinN/A1 (0.7)

    		Adjuvant diabetes medicationsN/A


    		SGLT2e inhibitorsN/A3 (2.1)
    Diabetes-related complicationsf; presence of at least 1a, n (%)105 (25.9)38 (31.4).004

    aP<.05 for between-group differences by use (or not) of open-source AID.

    bHbA1c: hemoglobin A1c.

    cPercentages do not add to 100% as some participants use multiple methods simultaneously or switch between methods.

    dN/A: not applicable.

    eSGLT2: sodium-glucose cotransporter-2.

    fComplications count based on experiences of the following: retinopathy, kidney disease, peripheral neuropathy, digestive complications, foot complications, diabetes-related complications related to sexual health, and cardiovascular disease.

    Table 3. Person-reported outcome measure (PROM) scores by use and nonuse of open-source automated insulin delivery.
    Psychosocial outcome (PROM score range)Users (n=451)Nonusers (n=141)Effect sizeP value


    		Mean (SD)Completed questionnaires, nMean (SD)Completed questionnaires, nCohen d
    Generic PROMs<.001

    		General emotional well-being (WHO-5a; scored 0-25)16.7 (4.5)45114.0 (4.8)141–0.6

    		Subjective sleep quality (PSQIb; scored 0-21)4.9 (2.8)4006.4 (3.0)1110.5
    Diabetes-specific PROMs<.001

    		Diabetes-specific quality of life (modified DIDPc; scored 1-7)4.3 (0.8)4264.8 (0.8)1290.6

    		Diabetes-specific positive well-being (4-item subscale of W-BQ28d; scored 0-12)8.7 (2.3)4196.6 (2.8)127–0.9

    		Diabetes treatment satisfaction (DTSQe; scored 0-36)31.3 (3.7)42224.5 (6.8)128–1.5

    		Diabetes distress (PAIDf; scored 0-100)19.4 (15.8)41132.8 (20.8)1260.8
    Hypoglycemia-specific PROMs<.001

    		Fear of hypoglycemia (HFS-II SFg):
    behavior subscale (scored 0-20)    		3.2 (3.1)4085.2 (4.3)1250.6

    		Fear of hypoglycemia (HFS-II SF):
    worry subscale (scored 0-24)    		5.1 (4.1)4098.4 (5.0)1250.7
    COVID-19–specific PROMs<.001

    		Impact of COVID-19 pandemic on quality of life (COVID-19 Impact on Quality of Life Scale; scored 1-7)4.6 (0.6)3944.9 (0.6)1130.5

    aWHO-5: 5-item World Health Organization Well-Being Index.

    bPSQI: Pittsburgh Sleep Quality Index.

    cDIDP: DAWN Impact of Diabetes Profile.

    dW-BQ28: 28-item Well-Being Questionnaire.

    eDTSQ: Diabetes Treatment Satisfaction Questionnaire.

    fPAID: Problem Areas in Diabetes.

    gHFS-II SF: short form of the Hypoglycemia Fear Survey II.


    Overview

    This study is the first to use a comprehensive set of validated PROMs to examine the psychosocial outcomes of adults with type 1 diabetes using open-source AID systems, compared with those not using such systems. Those using open-source AID systems report significantly greater general emotional well-being, subjective sleep quality, diabetes-specific QoL, diabetes-specific positive well-being, and diabetes treatment satisfaction compared with nonusers. They also report significantly less diabetes distress and fear of hypoglycemia, as well as less negative impact of the COVID-19 pandemic and its restrictions on their QoL, compared with nonusers.

    There were differences between those using and not using open-source AID systems in terms of gender, geographic region, income, diabetes duration, and most recent self-reported HbA1c. Importantly, the differences in PROM scores remained after adjusting for these differences, suggesting that these demographic and clinical differences do not explain the between-group differences in psychosocial outcomes. In the covariate analysis, the between-group difference in self-reported HbA1c did not explain the differences in psychosocial outcomes. This suggests that these user-reported benefits are independent of glycemic outcomes. It is important to understand and appreciate the impact that diabetes technologies can have on the experiences of people with type 1 diabetes [3], not just their glucose levels. These include benefits such as a reduction in the effort required to manage their diabetes, improvements in sleep quality, and increased flexibility (Table S2 in Multimedia Appendix 1).

    These results add to the existing qualitative and quantitative research (using unvalidated single items) supporting the beneficial psychosocial outcomes for users of open-source AID on sleep quality, diabetes distress, and QoL. This study is the first to use validated PROMS to assess general emotional well-being, diabetes-specific QoL, diabetes-specific positive well-being, and diabetes treatment satisfaction among open-source AID users and has demonstrated significant differences for all of these outcomes. This study corroborates the findings of 2 previous, but small (N≤35), open-source AID studies, which used validated PROMs to assess diabetes distress and fear of hypoglycemia, demonstrating reductions in both [14,18]. To date, 1 other study of open-source AID systems has used multiple validated PROMs in a large sample, showing significant improvements in sleep quality and significant reductions in diabetes distress and fear of hypoglycemia from baseline to 6 months following the initiation of an open-source AID system [19]. Those prospective findings are consistent with our cross-sectional analysis showing between-group differences. Furthermore, this study was multinational, across various health care systems and contexts, and found similar outcomes to their single-country (United States) study.

    These findings need to be corroborated with other designs, such as RCTs, which are able to minimize and account for the demographic and clinical differences between participant groups, as well as psychosocial outcomes, before the adoption of open-source AID systems. Such designs will add to the evidence regarding the psychosocial impacts of open-source AID systems. The CREATE RCT is the first worldwide study to randomize adults (n=49) to open-source AID versus sensor-augmented pump therapy [40]. The findings demonstrate open-source AID to be safe, acceptable, and effective, with those using open-source AID spending over 3 hours more per day with glucose in the target range than those using sensor-augmented pump therapy [41,42]. The psychosocial results are forthcoming, based on the use of 4 PROMs: the EuroQol-5 Dimension [43], DTSQ, Hypoglycemia Fear Survey, and PSQI [40]. A preliminary conference presentation indicated no significant differences between groups in terms of QoL, diabetes treatment satisfaction, or fear of hypoglycemia (for the adult participants). However, interview data show participants reported improved psychosocial outcomes. The authors state that multiple factors may have impacted the PROM findings, including low numbers of participants and control arm participants having funded access to CGMs [44]. Comparatively, this study has shown significant differences between those using open-source AID and nonusers in QoL, diabetes treatment satisfaction, and fear of hypoglycemia, plus additional apparent benefits. This study involved larger samples than the CREATE trial; we had the power to demonstrate differences where they occurred. Alternatively, it may be due to differences in the experiences between the participants in our real-world study and the participants in the RCT. Our participants built their systems themselves and engaged with online peer support, which may have added benefits beyond supporting the “build.” Due to the design of the RCT, participants were provided with the AID system (they did not have to build it themselves) and received support from the researchers and other participants in the study (which the authors note was not comparable to the experience of peer support in the open-source community). It is likely that these differences may have led to different psychosocial experiences.

    As highlighted by previous work, it is challenging to determine which PROMs to use among those using open-source AID systems [20]. There have also been concerns about whether existing measures, many of which were developed several decades ago, would be sensitive enough to the potential benefits of modern diabetes technologies [20]. In this study, we used 8 PROMs, all of which were sufficiently sensitive to demonstrate significant differences between open-source AID users and nonusers. The selection of these PROMs was informed by our earlier in-depth qualitative research [38] in addition to the lived experience of several of the authors. Several of the measures we selected have also been used in RCTs and cohort studies of advanced diabetes technologies, also demonstrating benefits [10]. This indicates the importance of being cognizant of what matters to adults with diabetes, how these issues are affected by living with diabetes and existing treatment regimens, and how new treatments and technologies can affect them [45]. Based on the results of this study, these PROMs could be used in future research on psychosocial outcomes among open-source AID users.

    The strengths of this study include the use of validated PROMs and the careful selection of these PROMs. This study has been able to provide an understanding of the psychosocial outcomes of a subsection of the open-source AID community, including multinational reach and a large sample relative to other psychosocial studies of open-source AID.

    However, there are also several limitations. Participants were recruited to this survey from several online communities, which may not be representative of those not involved in these communities. For example, both groups on average had relatively higher income levels and lower HbA1c levels compared with the general population of adults with type 1 diabetes. The study was also subject to self-selection bias, and recruiting through the OPEN networks may have led to additional bias. Further, although this study has compared 2 groups, neither was allocated at random. Finally, this study is limited by the cross-sectional design, which means we are unable to draw conclusions about causality. Despite adjusting for many known confounders of psychosocial outcomes, there may be unknown reasons why the open-source AID group reports better psychosocial outcomes.

    Conclusion

    In conclusion, this multinational study has responded to a call for research that uses validated measures to examine the psychosocial outcomes of adults with type 1 diabetes using open-source AID systems [13,20]. This real-world study shows that they report better psychosocial outcomes compared with nonusers, even when adjusted for sociodemographic and clinical characteristics. This study indicates that validated PROMs can quantify the sentiments previously expressed qualitatively (or using unvalidated single items) by the community [12,15-17]. Further research is needed to examine the reasons for these differences.

    Acknowledgments

    The authors would like to thank all the people with diabetes for sharing their experiences with us. We would also like to thank Professor Gonder-Frederick for permission to use the Hypoglycemia Fear Survey II short form, the Joslin Diabetes Center for permission to use the Problem Areas in Diabetes, and Health Psychology Research Ltd for permission to use the Diabetes Treatment Satisfaction Questionnaire and the 4-item diabetes-specific positive well-being subscale of the 28-item Well-Being Questionnaire. We are thankful for the helpful feedback we received when presenting this work as a poster at the 82nd American Diabetes Association Scientific Sessions. We would also like to thank Nicholas Riggall (Tidepool), Saira Khan-Gallo (Tidepool), Dr. Tom Crabtree (Department of Diabetes & Endocrinology, Nottingham University Hospitals National Health Service Foundation Trust), Dr. Martin de Bock (Department of Paediatrics, University of Otago), and Dr. Hamish Crocket (School of Health, University of Waikato) for sharing their insights and recommendations that improved our reporting of this work. This manuscript was supported by the “Outcomes of Patients’ Evidence with Novel, Do-it-Yourself Artificial Pancreas” (OPEN) team, and we are thankful for our colleagues’ combined efforts. Specifically, we would like to thank Timothée Froment, Saskia Wolf, Bastian Hauck, and Tebbe Ubben for their work on the OPEN survey.

    This project has received funding from the European Commission’s Horizon 2020 Research and Innovation Program under the Marie Skłodowska-Curie Action Research and Innovation Staff Exchange (RISE) grant agreement number 823902, the SPOKES Wellcome Trust Funded Translational Partnership program, and the Berlin Institute of Health (BIH) Digital Clinician Scientist program. J Schipp is funded by a joint PhD scholarship from Deakin University and the University of Copenhagen. J Speight and CH are supported by the core funding from the Australian Centre for Behavioral Research in Diabetes, derived from the collaboration between Diabetes Victoria and Deakin University.

    The authors attest that there was no use of generative artificial intelligence technology in the generation of text, figures, or other informational content in this manuscript.

    Data Availability

    The data sets generated or analyzed during this study are available from the corresponding author on reasonable request.

    Authors' Contributions

    J Schipp, CH, TCS, and J Speight selected measures in consultation with the OPEN team. The overall design of the OPEN survey was conducted by SOD, KB, HB, MW, and KAG. MW set up and maintained the REDCap (Research Electronic Data Capture) survey. J Schipp cleaned the data and conducted analyses with input and feedback from J Speight, CH, TCS, and CK. J Schipp prepared the first draft with input from J Speight, TCS, and CH. All authors reviewed the manuscript for critical content and approved the final version. J Schipp is the guarantor of this work and, as such, has full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

    Conflicts of Interest

    None declared.

    Multimedia Appendix 1

    Supplementary tables.

    DOCX File , 21 KB
    1. Dennick K, Sturt J, Speight J. What is diabetes distress and how can we measure it? A narrative review and conceptual model. J Diabetes Complications. 2017;31(5):898-911. [FREE Full text] [CrossRef] [Medline]
    2. Speight J, Holmes-Truscott E, Hendrieckx C, Skovlund S, Cooke D. Assessing the impact of diabetes on quality of life: what have the past 25 years taught us? Diabet Med. 2020;37(3):483-492. [FREE Full text] [CrossRef] [Medline]
    3. Kesavadev J, Saboo B, Krishna MB, Krishnan G. Evolution of insulin delivery devices: from syringes, pens, and pumps to DIY artificial pancreas. Diabetes Ther. 2020;11(6):1251-1269. [FREE Full text] [CrossRef] [Medline]
    4. Mathieu C, Martens PJ, Vangoitsenhoven R. One hundred years of insulin therapy. Nat Rev Endocrinol. 2021;17(12):715-725. [CrossRef] [Medline]
    5. Sims EK, Carr ALJ, Oram RA, DiMeglio LA, Evans-Molina C. 100 years of insulin: celebrating the past, present and future of diabetes therapy. Nat Med. 2021;27(7):1154-1164. [FREE Full text] [CrossRef] [Medline]
    6. Didyuk O, Econom N, Guardia A, Livingston K, Klueh U. Continuous glucose monitoring devices: past, present, and future focus on the history and evolution of technological innovation. J Diabetes Sci Technol. 2021;15(3):676-683. [FREE Full text] [CrossRef] [Medline]
    7. Hirsch I. Introduction: history of glucose monitoring. In: Role of Continuous Glucose Monitoring in Diabetes Treatment. Arlington. American Diabetes Association; 2018.
    8. Crabtree TSJ, Choudhary P, Kar P, Wilmot EG. Flash glucose monitoring: the story so far and the journey ahead. BMJ Innov. 2022;9(1):27-31. [FREE Full text] [CrossRef]
    9. Lewis DM. Automated Insulin Delivery: How Artificial Pancreas "Closed Loop" Systems Can Aid You in Living With Diabetes. US. Dana M Lewis; 2019.
    10. Speight J, Choudhary P, Wilmot EG, Hendrieckx C, Forde H, Cheung WY, et al. Impact of glycaemic technologies on quality of life and related outcomes in adults with type 1 diabetes: a narrative review. Diabet Med. 2023;40(1):e14944. [FREE Full text] [CrossRef] [Medline]
    11. Knoll C, Peacock S, Wäldchen M, Cooper D, Aulakh SK, Raile K, et al. Real-world evidence on clinical outcomes of people with type 1 diabetes using open-source and commercial automated insulin dosing systems: a systematic review. Diabet Med. 2022;39(5):e14741. [FREE Full text] [CrossRef] [Medline]
    12. Lewis D, Leibrand S, #OpenAPS Community. Real-world use of open source artificial pancreas systems. J Diabetes Sci Technol. 2016;10(6):1411. [FREE Full text] [CrossRef] [Medline]
    13. Jennings P, Hussain S. Do-it-yourself artificial pancreas systems: a review of the emerging evidence and insights for healthcare professionals. J Diabetes Sci Technol. 2020;14(5):868-877. [FREE Full text] [CrossRef] [Medline]
    14. Patel R, Crabtree TSJ, Taylor N, Langeland L, Gazis AT, Mendis B, et al. Safety and effectiveness of do-it-yourself artificial pancreas system compared with continuous subcutaneous insulin infusions in combination with free style libre in people with type 1 diabetes. Diabet Med. 2022;39(5):e14793. [CrossRef] [Medline]
    15. Hng TM, Burren D. Appearance of do-it-yourself closed-loop systems to manage type 1 diabetes. Intern Med J. 2018;48(11):1400-1404. [FREE Full text] [CrossRef] [Medline]
    16. Litchman ML, Lewis D, Kelly LA, Gee PM. Twitter analysis of #OpenAPS DIY artificial pancreas technology use suggests improved A1C and quality of life. J Diabetes Sci Technol. 2019;13(2):164-170. [FREE Full text] [CrossRef] [Medline]
    17. Braune K, Krug N, Knoll C, Ballhausen H, Thieffry A, Chen Y, et al. Emotional and physical health impact in children and adolescents and their caregivers using open-source automated insulin delivery: qualitative analysis of lived experiences. J Med Internet Res. 2022;24(7):e37120. [FREE Full text] [CrossRef] [Medline]
    18. Wu Z, Luo S, Zheng X, Bi Y, Xu W, Yan J, et al. Use of a do-it-yourself artificial pancreas system is associated with better glucose management and higher quality of life among adults with type 1 diabetes. Ther Adv Endocrinol Metab. 2020;11:2042018820950146. [FREE Full text] [CrossRef] [Medline]
    19. Wong JJ, Hood KK, Hanes SJ, Lal RA, Naranjo D. Psychosocial effects of the loop open-source automated insulin delivery system. J Diabetes Sci Technol. 2023;17(6):1440-1447. [CrossRef] [Medline]
    20. Vallis M, Holt RIG. User-driven open-source artificial pancreas systems and patient-reported outcomes: a missed opportunity? Diabet Med. 2022;39(5):e14797. [CrossRef] [Medline]
    21. O'Donnell S, Lewis D, Fernández MM, Wäldchen M, Cleal B, Skinner T, et al. Evidence on user-led innovation in diabetes technology (the open project): protocol for a mixed methods study. JMIR Res Protoc. 2019;8(11):e15368. [FREE Full text] [CrossRef] [Medline]
    22. Braune K, Lal RA, Petruželková L, Scheiner G, Winterdijk P, Schmidt S, et al. Open-source automated insulin delivery: international consensus statement and practical guidance for health-care professionals. Lancet Diabetes Endocrinol. 2022;10(1):58-74. [FREE Full text] [CrossRef] [Medline]
    23. Topp CW, Østergaard SD, Søndergaard S, Bech P. The WHO-5 well-being index: a systematic review of the literature. Psychother Psychosom. 2015;84(3):167-176. [FREE Full text] [CrossRef] [Medline]
    24. Halliday JA, Hendrieckx C, Busija L, Browne JL, Nefs G, Pouwer F, et al. Validation of the WHO-5 as a first-step screening instrument for depression in adults with diabetes: results from diabetes MILES—Australia. Diabetes Res Clin Pract. 2017;132:27-35. [CrossRef] [Medline]
    25. Buysse DJ, Reynolds CF, Monk TH, Berman SR, Kupfer DJ. The Pittsburgh sleep quality index: a new instrument for psychiatric practice and research. Psychiatry Res. 1989;28(2):193-213. [CrossRef] [Medline]
    26. Carpenter JS, Andrykowski MA. Psychometric evaluation of the Pittsburgh sleep quality index. J Psychosom Res. 1998;45(1):5-13. [FREE Full text] [CrossRef] [Medline]
    27. Al Hayek AA, Al Dawish MA. Assessing diabetes distress and sleep quality in young adults with type 1 diabetes using freestyle libre: a prospective cohort study. Diabetes Ther. 2020;11(7):1551-1562. [FREE Full text] [CrossRef] [Medline]
    28. Landau Z, Rachmiel M, Pinhas-Hamiel O, Boaz M, Bar-Dayan Y, Wainstein J, et al. Parental sleep quality and continuous glucose monitoring system use in children with type 1 diabetes. Acta Diabetol. 2014;51(3):499-503. [CrossRef] [Medline]
    29. Holmes-Truscott E, Skovlund SE, Hendrieckx C, Pouwer F, Peyrot M, Speight J. Assessing the perceived impact of diabetes on quality of life: psychometric validation of the DAWN2 impact of diabetes profile in the second diabetes MILES—Australia (MILES-2) survey. Diabetes Res Clin Pract. 2019;150:253-263. [CrossRef] [Medline]
    30. Speight J, Barendse S, Bradley C. The W-BQ 28: further development of the well-being questionnaire to include diabetes-specific as well as generic subscales and new stress subscales. Br Psychol Soc. 2000;8(1):21.
    31. Speight J, Khagram LA, Davies MJ. Generic and diabetes-specific well-being in the AT.LANTUS follow-on study: further psychometric validation of the W-BQ28 indicates its utility in research and clinical practice in type 2 diabetes in the UK. Diabet Med. 2012;29(9):e345-e353. [CrossRef] [Medline]
    32. Bradley C, editor. The diabetes treatment satisfaction questionnaire: DTSQ. In: Handbook of Psychology and Diabetes: A Guide to Psychological Measurement in Diabetes Research and Practice Chur. Switzerland. Harwood Academic Publishers; 1994;111.
    33. Polonsky WH, Anderson BJ, Lohrer PA, Welch G, Jacobson AM, Aponte JE, et al. Assessment of diabetes-related distress. Diabetes Care. 1995;18(6):754-760. [FREE Full text] [CrossRef] [Medline]
    34. Fenwick EK, Rees G, Holmes-Truscott E, Browne JL, Pouwer F, Speight J. What is the best measure for assessing diabetes distress? A comparison of the problem areas in diabetes and diabetes distress scale: results from diabetes MILES-Australia. J Health Psychol. 2018;23(5):667-680. [CrossRef] [Medline]
    35. Gonder-Frederick LA, Schmidt KM, Vajda KA, Greear ML, Singh H, Shepard JA, et al. Psychometric properties of the hypoglycemia fear survey-ii for adults with type 1 diabetes. Diabetes Care. 2011;34(4):801-806. [FREE Full text] [CrossRef] [Medline]
    36. Grabman J, Vajda Bailey K, Schmidt K, Cariou B, Vaur L, Madani S, et al. An empirically derived short form of the hypoglycaemia fear survey II. Diabet Med. 2017;34(4):500-504. [CrossRef] [Medline]
    37. Sacre JW, Holmes-Truscott E, Salim A, Anstey KJ, Drummond GR, Huxley RR, et al. Impact of the COVID-19 pandemic and lockdown restrictions on psychosocial and behavioural outcomes among Australian adults with type 2 diabetes: findings from the PREDICT cohort study. Diabet Med. 2021;38(9):e14611. [FREE Full text] [CrossRef] [Medline]
    38. Schipp J, Skinner T, Holloway E, Scibilia R, Langstrup H, Speight J, et al. How adults with type 1 diabetes are navigating the challenges of open-source artificial pancreas systems: a qualitative study. Diabetes Technol Ther. 2021;23(8):546-554. [CrossRef] [Medline]
    39. Schipp J, Holmes-Truscott E, Hendrieckx C, Coates E, Heller SR, Cooke D, et al. A qualitative investigation of the acceptability to adults with type 1 diabetes of five diabetes-specific quality of life measures. Presented at: Australasian Diabetes Congress; August 21-23, 2019, 2019; Sydney, Australia. [CrossRef]
    40. Burnside M, Lewis D, Crocket H, Wilson R, Williman J, Jefferies C, et al. CREATE (community derived automated insulin delivery) trial. Randomised parallel arm open label clinical trial comparing automated insulin delivery using a mobile controller (AnyDANA-loop) with an open-source algorithm with sensor augmented pump therapy in type 1 diabetes. J Diabetes Metab Disord. 2020;19(2):1615-1629. [FREE Full text] [CrossRef] [Medline]
    41. Burnside MJ, Lewis DM, Crocket HR, Meier RA, Williman JA, Sanders OJ, et al. Open-source automated insulin delivery in type 1 diabetes. N Engl J Med. 2022;387(10):869-881. [FREE Full text] [CrossRef] [Medline]
    42. Burnside MJ, Lewis DM, Crocket HR, Meier RA, Williman JA, Sanders OJ, et al. Extended use of an open-source automated insulin delivery system in children and adults with type 1 diabetes: the 24-week continuation phase following the CREATE randomized controlled trial. Diabetes Technol Ther. 2023;25(4):250-259. [CrossRef] [Medline]
    43. Herdman M, Gudex C, Lloyd A, Janssen MF, Kind P, Parkin D, et al. Development and preliminary testing of the new five-level version of EQ-5D (EQ-5D-5L). Qual Life Res. 2011;20(10):1727-1736. [FREE Full text] [CrossRef] [Medline]
    44. Crocket H, Ward H, Lewis D, Burnside M, Faherty A, Wheeler B, et al. Participant experiences of open source automated insulin delivery during the CREATE study: a 24-week randomised controlled trial. Presented at: Advanced Technologies & Treatments for Diabetes Conference; February 22-25, 2023, 2023; Berlin, Germany. [CrossRef]
    45. Speight J, Reaney MD, Barnard KD. Not all roads lead to Rome-a review of quality of life measurement in adults with diabetes. Diabet Med. 2009;26(4):315-327. [CrossRef] [Medline]

    ‎
    AID: automated insulin delivery
    CGM: continuous glucose monitor
    DIDP: DAWN Impact of Diabetes Profile
    DTSQ: Diabetes Treatment Satisfaction Questionnaire
    HbA1c: hemoglobin A1c
    HFS-II SF: short form of the Hypoglycemia Fear Survey II
    OPEN: Outcomes of Patients’ Evidence with Novel, Do-it-Yourself Artificial Pancreas Technology Project Consortium
    PAID: Problem Areas in Diabetes
    PROM: person-reported outcome measure
    PSQI: Pittsburgh Sleep Quality Index
    QoL: quality of life
    RCT: randomized controlled trial
    W-BQ28: 28-item Well-being Questionnaire
    WHO-5: 5-item World Health Organization Well-Being Index

    Edited by G Eysenbach, T de Azevedo Cardoso; submitted 02.11.22; peer-reviewed by A Morrison, R Meng; comments to author 16.02.23; revised version received 10.06.23; accepted 23.10.23; published 14.12.23.

    Copyright

    ©Jasmine Schipp, Christel Hendrieckx, Katarina Braune, Christine Knoll, Shane O’Donnell, Hanne Ballhausen, Bryan Cleal, Mandy Wäldchen, Dana M Lewis, Katarzyna A Gajewska, Timothy C Skinner, Jane Speight. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 14.12.2023.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.