This paper is in the following e-collection/theme issue:

Cost-Effectiveness and Economics (234) Registered Report (528) e-Mental Health and Cyberpsychology (1411) Depression and Mood Disorders; Suicide Prevention (1290) mHealth for Symptom and Disease Monitoring, Chronic Disease Management (1317) Borderline Personality Disorder (7) Behavior Change (655)

Published on in Vol 23, No 11 (2021): November

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/28874, first published .
Mobile Diary App Versus Paper-Based Diary Cards for Patients With Borderline Personality Disorder: Economic Evaluation

Mobile Diary App Versus Paper-Based Diary Cards for Patients With Borderline Personality Disorder: Economic Evaluation

Mobile Diary App Versus Paper-Based Diary Cards for Patients With Borderline Personality Disorder: Economic Evaluation

Authors of this article:

Sidsel Lund Laursen1 Author Orcid Image ;   Stig Helweg-Jørgensen2 Author Orcid Image ;   Astrid Langergaard1 Author Orcid Image ;   Jesper Søndergaard1 Author Orcid Image ;   Sabrina Storgaard Sørensen1 Author Orcid Image ;   Kim Mathiasen2, 3 Author Orcid Image ;   Mia Beck Lichtenstein2, 3 Author Orcid Image ;   Lars Holger Ehlers1 Author Orcid Image
Article Authors Cited by (7) Tweetations (4) Metrics

    Original Paper

    1Department of Clinical Medicine, Danish Center for Healthcare Improvements, Aalborg University, Aalborg Oest, Denmark

    2Research Unit for Telepsychiatry and E-mental Health, Mental Health Services in the Region of Southern Denmark, Odense, Denmark

    3Department of Clinical Research, University of Southern Denmark, Odense, Denmark

    Corresponding Author:

    Lars Holger Ehlers, BA, MSc, PhD

    Department of Clinical Medicine

    Danish Center for Healthcare Improvements

    Aalborg University

    Fredrik Bajers Vej 5

    Aalborg Oest, 9220

    Denmark

    Phone: 45 99408257

    Email: lehlers@dcm.aau.dk


    Background: The cost-effectiveness of using a mobile diary app as an adjunct in dialectical behavior therapy (DBT) in patients with borderline personality disorder is unknown.

    Objective: This study aims to perform an economic evaluation of a mobile diary app compared with paper-based diary cards in DBT treatment for patients with borderline personality disorder in a psychiatric outpatient facility.

    Methods: This study was conducted alongside a pragmatic, multicenter, randomized controlled trial. The participants were recruited at 5 Danish psychiatric outpatient facilities and were randomized to register the emotions, urges, and skills used in a mobile diary app or on paper-based diary cards. The participants in both groups received DBT delivered by the therapists. A cost-consequence analysis with a time horizon of 12 months was performed. Consequences included quality-adjusted life years (QALYs), depression severity, borderline severity, suicidal behavior, health care use, treatment compliance, and system usability. All relevant costs were included. Focus group interviews were conducted with patients, therapists, researchers, and industry representatives to discuss the potential advantages and disadvantages of using a mobile diary app.

    Results: A total of 78 participants were included in the analysis. An insignificantly higher number of participants in the paper group dropped out before the start of treatment (P=.07). Of those starting treatment, participants in the app group had an average of 37.1 (SE 27.55) more days of treatment and recorded an average of 3.16 (SE 5.10) more skills per week than participants in the paper group. Participants in both groups had a QALY gain and a decrease in depression severity, borderline severity, and suicidal behavior. Significant differences were found in favor of the paper group for both QALY gain (adjusted difference −0.054; SE 0.03) and reduction in depression severity (adjusted difference −1.11; SE 1.57). The between-group difference in total costs ranged from US $107.37 to US $322.10 per participant during the 12 months. The use of services in the health care sector was similar across both time points and groups (difference: psychiatric hospitalization <5 and <5; general practice −1.32; SE 3.68 and 2.02; SE 3.19). Overall, the patients showed high acceptability and considered the app as being easy to use. Therapists worried about potential negative influences on the therapist-patient interaction from new work tasks accompanying the introduction of the new technology but pointed at innovation potential from digital database registrations.

    Conclusions: This study suggests both positive and negative consequences of mobile diary apps as adjuncts to DBT compared with paper diary cards. More research is needed to draw conclusions regarding its cost-effectiveness.

    Trial Registration: ClinicalTrials.gov NCT03191565; http://clinicaltrials.gov/ct2/show/NCT03191565

    International Registered Report Identifier (IRRID): RR2-10.2196/17737

    J Med Internet Res 2021;23(11):e28874

    doi:10.2196/28874

    Keywords

    borderline personality disorder; dialectical behavior therapy; mobile app; psychotherapy; cost-consequence; mHealth; mobile phone 


    Background

    Borderline personality disorder (BPD) is estimated to affect between 0.7% and 4.8% of the general population in Scandinavia and accounts for approximately 15% of adult admissions in Danish psychiatric hospitals [1-3]. Patients diagnosed with BPD are often characterized as emotionally unstable, impulsive, and self-harming and have unstable relationships [4]. It has been shown that the mortality rate is 8 times higher in patients with BPD than in the general population [5]. Furthermore, BPD has large societal consequences, as one study has found that the total mean societal cost related to BPD is US $32,863.01 per patient per year [6].

    One of the most well-researched and clinically effective psychosocial treatments for patients with BPD is dialectical behavior therapy (DBT) [7-12]. Studies have shown DBT to be effective in reducing nonsuicidal self-injury and increasing quality-adjusted life years (QALYs), when compared with treatment as usual or client-centered therapy [13-16]. The DBT includes elements from cognitive-behavioral therapy and is centered on learning a set of predefined skills, such as mindfulness, tolerance of distress, regulation of emotions, and navigation of interpersonal situations [17,18]. Learning the DBT skills is done through self-monitoring using weekly paper-based diary cards filled out by the patient [19,20]. The use of paper-based diary cards, however, has limitations for both patients and therapists. Working with paper diaries is burdensome and may reduce patient engagement in treatment [21,22]. Retrieving and reviewing paper diaries to evaluate the patient’s progress over time is a time-consuming process for the therapist, and it is not possible for the therapist to follow the patient alongside self-treatment at home [20-22].

    New studies suggest that mobile diary apps can overcome some of these limitations and be more effective in reducing distress symptoms and suicidal and nonsuicidal self-harming behaviors compared with treatment with DBT alone [23-25]. Mobile apps have the potential to reduce barriers and increase engagement in treatment [21,22]. Mobile apps can automatically provide an overview of the patients' progress and enable therapists to monitor the present emotional stage of patients during treatment [21,24,26].

    Objective

    Economic evaluations of new health technology constitute important knowledge for decision-makers in health care as they estimate the value for money of the technology compared with alternative use of the scarce financial resources. To our knowledge, no economic evaluation has yet been conducted on the use of mobile apps as adjuncts to DBT in patients with BPD. Therefore, the aim of this economic evaluation is to investigate the costs and consequences of a mobile diary app compared with paper-based diary cards for patients with BPD, within the context of a psychiatric outpatient facility.


    Overview

    This economic evaluation was reported in line with the Consolidated Health Economic Evaluation Reporting Standards [27]. It was conducted alongside a pragmatic, multicenter, randomized controlled trial as part of the ENTER program [28]. The clinical study is described in detail in the study protocol [29] but will be summarized in the Methods section of this evaluation. Economic evaluation was carried out as a cost-consequence analysis, which is why costs and consequences are shown separately [30]. The cost-consequence analysis approach was chosen because of the possibility of investigating a broad range of consequences [30], such as intermediate outcomes and qualitative aspects. The time horizon was restricted to 12 months, and all relevant costs were included in the analysis. All costs were measured in Danish Krone (DKK) and subsequently converted to the US dollars (US $).

    Participants

    The participants were recruited from 5 Danish psychiatric outpatient facilities between June 2017 and December 2018. The participants were eligible for inclusion if they were ≥18 years of age, formally diagnosed by a psychiatric specialist with Emotionally Unstable Personality Disorder (F60.3 according to ICD-10 criteria) [31], and admitted for psychiatric outpatient treatment. To be included, participants also had to be either self-harming or to have had suicidal behavior within the last 3 years. Participants with comorbid disorders, such as depression, anxiety, and posttraumatic stress disorder, were allowed to participate. On the contrary, participants were excluded if they had no access to a working smartphone; an IQ <70; or a comorbid disorder such as substance abuse, bipolar disorder, or a disorder within the schizophrenic spectrum. Before entering the study, all participants signed an informed consent form.

    Interventions

    Participants were randomly assigned to either the mobile diary app group or to the paper-based diary card group using REDCap (Research Electronic Data Capture) [29]. In both groups, participants had to register entries about aspects such as emotional dysregulation, suicidal and self-harm thoughts, and skill use. The participants received the paper-based treatment registered daily on paper diary cards, whereas the participants using the mobile diary app were registered daily via the mDiary app (The Monsenso system). The mDiary app is a market solution containing psychoeducational material and visualizations of the participant’s data, which can be used in therapy sessions and for real time monitoring alongside treatment [32]. Participants in both groups received standard DBT (delivered as individual sessions, group sessions, and telephone coaching). The therapists instructed the participants on how to use the app or paper diary cards. Other treatments, such as medications, were allowed. Passive sensor data (eg, activity level and phone use) were collected from all participants through their smartphones. The treatment period was 40 weeks at 2 sites and 12 months at the other sites, and neither the participants nor the therapists were blinded to the group allocation.

    Consequences

    Health-Related Outcomes

    Health-related quality of life was based on the EuroQol 5-Dimensions 5-Levels (EQ-5D-5L) questionnaire [33]; depression severity from the Patient Health Questionnaire-9 [34]; borderline severity was assessed using the Zanarini Rating Scale for BPD [35]; and suicide behavior from the Suicide Behaviors Questionnaire [36]. Data were collected at baseline and at the 12-month follow-up. The participants' utility scores were calculated using the Danish weights for the EQ-5D-5L questionnaire [37]. QALY gain was estimated by assuming linear interpolation between utility scores at baseline and a 12-month follow-up, after which the area under the curve was calculated [38]. For depression severity, borderline severity, and suicide behavior, the difference from baseline to follow-up was calculated and compared between the groups.

    Health Care Sector

    In Denmark, the civil registration number makes it possible to link information from several registries to an individual [39]. Information on psychiatric hospitalizations and consultations with general practice was obtained at an individual level from the Primary National Health Insurance Service Register and the Danish National Patient Registry [40,41]. The specific purpose of the consultations in general practice was not available, which is why all consultations were included. The number of psychiatric hospitalizations and consultations with general practice was estimated 30 days before and after baseline and 12 months before and after baseline, respectively.

    Treatment Compliance and Skill Recordings

    It was continuously assessed whether the participants were still receiving treatment and whether they were recording their skills use. The recording of the skills used per week was estimated at an individual level as the total number of skills for each participant divided by the number of days in treatment, which was subsequently calculated for a week. A participating therapist estimated the time spent on recording in the 2 groups based on experience from the trial.

    The usability of the mobile diary app was assessed by patients and therapists using a system usability scale [42]. This assessment was conducted in focus group interviews alongside a feasibility study of the app.

    Future innovation potential was assessed by app developers, researchers, and other personnel related to the study. This assessment was conducted during a workshop to discuss the advantages, disadvantages, and development potential.

    Costs

    All relevant costs were included in the analysis and adjusted to the price level in 2019 using the Danish net price index [43]. The conversion rate from kr to US $ was kr 634.0 per US $100 from September 23, 2021 [44]. The costs were not discounted because of a time horizon of 12 months [45].

    The costs of health care services, patient costs, and municipality costs were used to adjust for baseline differences in the analysis. Health care services included primary care services, prescription medicine, inpatient and outpatient services, and emergency contacts to psychiatric or somatic hospitals. Patient costs included out-of-pocket medical expenses, including expenses for both medicine and primary care services. Municipality costs included nurse care, daily care, and domestic help at home. Information on each participant’s use of health care services and prices for public services was retrieved from the Danish Primary National Health Insurance Service Register, the Danish National Patient Registry, and the Danish National Prescription Registry [40,41,46]. Information regarding municipality costs was collected based on the Treatment Inventory of Costs in Psychiatric Patients questionnaire [47], which was administered to all participants at baseline. Baseline costs were calculated for resource use 12 months before the start of the study until the baseline date.

    Information on intervention costs was provided by the key personnel at a psychiatric outpatient facility. The intervention costs of the app were estimated as the capital and operating costs. Capital costs included the one-time cost of starting and establishing the mobile diary app program and the education of therapists on how to use the program’s software. Operating costs included costs, such as program licenses. Startup establishment and program license costs were estimated as a range due to price instability of new devices, especially if the device is new in the market [48,49]. The pricing was estimated in collaboration with the software company and public purchasers, with an expected discount if the software were to be implemented in Denmark. The license fee was allocated to an estimated number of patients per therapist. The costs of educating the therapists were estimated using the time allocated for training and the average effective hourly wages for the therapists. The wages were based on information from Statistics Denmark on the national average gross wages corresponding to the therapists’ occupation [50]. Capital costs were annuitized over 3 years, with a discount rate of 4% per year [51,52]. Subsequently, the costs were allocated to an estimated number of patients per therapist per year and were included in the analysis as annual costs. The relevant intervention costs in the paper group included the price of paper and the printing of diaries for 12 months. The choice between the app and paper was assumed to not influence the overhead costs. See Multimedia Appendix 1 [43,44,48-53] for further information on cost estimation.

    Statistical Analysis

    To account for missing data, multiple imputation was performed, which was applicable because the missing data were assumed to be missing at random [54,55]. The consequences of borderline severity and suicide behavior are shown as nonimputed estimates due to a high number of missing values at both baseline (17 and 20 missing values, respectively) and follow-up (20 and 17 missing values, respectively). The analyses were conducted using an intention-to-treat analysis. See Multimedia Appendix 2 [54] for a summary of the missing data and a full description of the imputation model.

    Data are reported either as the number of participants in each group and percentages, median, and IQR, or as means and SE. Incremental health-related outcomes were estimated as both unadjusted and adjusted outcomes based on the regression analysis. The health-related outcomes were adjusted for age, sex, relationship status, education level, baseline scores (utility score, depression score, suicide score, and borderline score), and baseline costs to control for baseline differences [38]. For consequences related to the health care sector and treatment compliance, P values were calculated using the Fisher exact test for categorical variables and Student t test (2-tailed) for continuous variables [56]. Statistical significance was set at a P value <.05. Statistical analyses were performed using the STATA (version 16.1, StataCorp).

    Ethics and Consent to Participate

    The trial was registered at ClinicalTrials.gov (identifier NCT03191565) and was performed in accordance with the Declaration of Helsinki and approved by the Danish Ethics Committee in the Region of Southern Denmark (Registration number S-20160085). Participants were recruited between June 2017 and December 2018. All participants signed an informed consent form before participating in the study.

    Data Sharing Statements

    The data sets generated and analyzed during this study are available from the corresponding author upon reasonable request.


    Overview

    In total, 79 participants were enrolled in the trial, of whom 78 were eligible for inclusion in the analysis. Participants in both groups were predominantly women, with a mean age of 29 years (Table 1).

    Table 1. Characteristics of the participants at baseline.
    Study populationMobile diary app (N=42)Paper diary cards (N=36)
    Age (years), median (IQR)26.4 (22.2-32.8)27.2 (22.3-34.6)
    Sex (female), n (%)>37 (≥88)>31 (≥86)
    Cohabiting status, n (%)

    		Living with somebody14 (33)18 (50)

    		Living alone28 (67)18 (50)
    Education, n (%)

    		Education <3 yearsa>37 (≥88)27 (75)

    		Education ≥3 yearsb<5 (≤12)9 (25)
    Employment, n (%)

    		In employment9 (21)9 (25)

    		Not in employment33 (79)27 (75)
    Costsc (US $), mean (SE)

    		Health care costs17,200.35 (3351.31)15,684.14 (2800.31)

    		Municipality costs7785.36 (5680.32)250.08 (140.15)

    		Patient costs303.66 (49.47)327.02 (48.09)

    		Total costs25,289.37 (6918.34)16,378.66 (2805.81)

    aParticipants with highest education of public school, high school, further education <3 years and apprenticeship.

    bParticipants with further education ≥3 years.

    cAnnual cost the year before the study start date.

    Consequences

    Participants in both groups experienced an increase in utility scores from baseline to the 12-month follow-up (Table 2). The adjusted QALY gain difference between the app group and the paper group was −0.054 (SE 0.03; P<.001), indicating a smaller QALY gain in the app group. The unadjusted QALY gain difference was close to the adjusted estimate. Both groups showed improvements in the EQ-5D domains, except for domain 1 (mobility) in the app group and domain 2 (self-care) in the paper group. The improvement in domain 5 (anxiety and depression) was comparable between the 2 groups.

    Table 2. Quantitative consequences of the mobile diary app and paper-based diary cards.
    Health-related outcomesMobile diary app (N=42)Paper diary cards (N=36)Unadjusted between-group differenceP valueAdjusted between-group differenceP value
    EQ-5Da levels, utility and QALYb, mean (SE)c

    		Baseline—d———


    		Domain 1- Mobility1.86 (0.15)1.89 (0.17)





    		Domain 2—Self-care1.52 (0.14)1.31 (0.10)





    		Domain 3—Usual activities2.86 (0.14)2.67 (0.15)





    		Domain 4—Pain or discomfort2.93 (0.16)2.58 (0.21)





    		Domain 5—Anxiety or depression3.17 (0.17)3.11 (0.15)





    		Utility score0.45 (0.05)0.51 (0.05)




    		Follow-up————


    		Domain 1-Mobility2.2 (0.22)1.57 (0.13)





    		Domain 2-Self-care1.37 (0.13)1.41 (0.14)





    		Domain 3-Usual activities2.56 (0.19)1.95 (0.17)





    		Domain 4-Pain or discomfort2.66 (0.17)2.30 (0.15)





    		Domain 5-Anxiety or depression2.41 (0.17)2.33 (0.18)





    		Utility score0.6 (0.06)0.74 (0.03)




    		QALY gain0.078 (0.03)0.115 (0.03)-0.037 (0.04).38-0.054 (0.03)<.001
    Borderline severity, mean (SE)e

    		Baseline18.6 (1.22)19.5 (1.08)—
    		——

    		Follow-up8.82 (1.16)8.83 (1.45)—
    		——

    		Change within the group9.85 (1.28)10.67 (1.39)−0.81 (1.93).67−0.67 (1.73).004
    Depression severity, mean (SE)c

    		Baseline17.52 (0.80)16.11 (0.88)————

    		Follow-up12.34 (1.03)10.47 (1.3)————

    		Change within the group5.18 (1.01)5.64 (1.38)−0.46 (1.67).78−1.11 (1.57).04
    Suicidal behavior, mean (SE)e

    		Baseline11.60 (0.75)11.96 (0.88)————

    		Follow-up8.86 (0.58)9.38 (0.62)————

    		Change within the group2.79 (0.59)2.76 (0.72)0.03 (0.93).980.47 (0.73) <.001

    aEQ-5D: EuroQol 5-Dimensions.

    bQALY: quality-adjusted life year.

    cImputed data set.

    dEmpty cells concern descriptive measures not relevant for statistical testing.

    eVariables with missing values.

    Participants in both groups had decreased borderline severity, depressive severity, and suicidal behavior during the 12-month follow-up. The adjusted difference in the change within the group in depression severity between the app group and the paper group was −1.11 (SE 1.57; P=.04), indicating that participants in the app group had a smaller decrease in depression severity. The unadjusted differences showed a similar result. In addition, participants in the app group had a smaller decrease in borderline severity but a greater decrease in suicidal behavior. These results should be interpreted with reservations due to a high number of missing values at both baseline and at follow-up.

    The use of health care sector services was similar across both the periods and the groups (difference: psychiatric hospitalization <5 and <5; general practice −1.32, SE 3.68 and 2.02, SE 3.19), indicating that registering in the app compared with on paper does not affect the resource use in the health care sector (Table 3).

    The paper group had an insignificantly higher number of participants dropping out before the start of treatment compared with the app group (P=.07). Of those starting treatment, participants in the app group were more persistent in adhering to treatment and had an average of 37.1 (SE 27.55) more days of treatment than the paper group. Participants in the app group registered 3.16 (SE 5.10) more skills per week compared with participants in the paper group. Therapists and researchers estimated that the app was time saving for the patient, approximately 1 minute on each diary entry. They further pointed out a potential utility gain for patients from the possibility of choosing according to preferences between registering in an app or on paper diary cards.

    Table 3. Quantitative consequences of the mobile diary app and paper-based diary cards.
    Health-related outcomesMobile diary app (n=42)Paper diary cards (n=36)Between-group differenceP value
    Health care sector contacts
    Participants hospitalized in a psychiatric hospital, n (%)

    		30 days before baseline<5 (≤12)<5 (≤14)≤5>.99

    		30 days after baseline<5 (≤12)<5 (≤14)≤5>.99
    General practice contacts, mean (SE)

    		12 months before baseline19.42 (2.4)20.75 (2.83)−1.32 (3.68).72

    		12 months after baseline19.02 (2.18)17 (2.32)2.02 (3.19).53
    Treatment compliance

    		Participants, who never start treatment, n (%)<5 (≤12)7 (19)≤−2.07

    		Number of treatment days for participants starting treatment, mean (SE)265.93 (18.64)228.83 (19.64)37.1 (27.55).18
    Skill recordings, mean (SE)

    		All participants


    		Skills recorded per patient647.93 (150.72)409.89 (173.99)238.04 (229).30

    		Participants starting treatment


    		Skills recorded per patient680.33 (156.56)508.83 (212.54)171.5 (257.9).51


    		Per week15.04 (3.04)11.88 (4.29)3.16 (5.10).54
    Time spent on recording

    		Time spent on completing the diary (minutes)45−1—a
    System Usability Scale, mean (SD)

    		Patients (n=16)81.2 (9.9)———

    		Therapists (n=23)61.6 (18.6)———

    aSystem usability scale is only reported for the mobile diary group.

    The System Usability Score was 81.2 (SD 9.9) in patients, but significantly lower at 61.6 (SD 18.6) in therapists (for further information see Marceau et al [22]). In general, patients were satisfied with the mobile solution, whereas the therapists expressed concern regarding the potential for a negative influence on the patient-therapist interaction due to new tasks related to the implementation of the new technology [22]. In addition, reduced flexibility in data collection due to the standard structure of the app was described as a potential disadvantage. Finally, numerous implementation issues were highlighted in the focus groups, including concerns relating to a potential lack of technical skills of both the patient and therapist, and security and storage of the collected data.

    Future potential for innovation was considered relevant as electronic data recording can be used to predict outcomes and benefits from changes in treatment strategy (Textbox 1). The mobile diary app could provide therapists with information on the patients alongside treatment, thereby making it possible to define cut-off values for intervening outside sessions. Program optimization and visualization of the entered data could further inform the patients about their progress, potentially increasing the patient’s self-insight and empowerment.

    Innovation potential of the mobile diary app and paper-based diary cards.

    Innovation potential

    • Real time recording makes it possible to track patients’ symptoms and possibly intervene outside sessions.
    • Electronic data from the app can potentially be used for predicting outcomes from treatment.
    Textbox 1. Innovation potential of the mobile diary app and paper-based diary cards.

    Costs

    The between-group difference in total costs during the 12-months ranged from US $107.38 to US $322.13 per participant (Table 4).

    Table 4. Intervention costs (US $) per participant at 12-month follow-up.
    Intervention costMobile diary app (US $)Paper diary cards (US $)Difference (US $)
    Startup costa9.41-28.4409.41-28.24
    License97.96-293.88097.96-293.88
    Education3.2703.27
    Paper and print03.263.26
    Total110.64-325.393.26107.38-322.13

    aAnnuitized over 3 years with a 4% discount rate.


    Principal Findings

    To our knowledge, this study is the first to evaluate the costs and consequences of a mobile diary app for patients with BPD. Our findings suggest that using a mobile app as an adjunct to DBT comes with a high acceptance of patients, and the potential for increased treatment adherence and future innovation from electronic database registrations. We also found, however, that the app was associated with a reduced improvement in patients’ depression and health-related quality of life scores within the first year of treatment. The app led to higher costs but was still a relatively inexpensive intervention.

    The economic evaluation was conducted using a cost-consequence approach rather than the traditional cost-utility (ie, cost-per-QALY gained) framework. The cost-consequence approach has been criticized by health economists for not being a normative analysis, as the decision-maker has to choose from her or his own opinion concerning the relative importance of the list of consequences [30]. As the mobile diary app was still in early development, we believe there is a need to investigate a broad range of potential consequences, including intermediate consequences. Indeed, health professionals should be encouraged to engage in the debate about how to evaluate internet-based interventions in mental health. Health economists have concluded that the economic evaluation of devices in health care, in general, represents a number of methodological challenges; in particular, the use of QALY as an outcome may be too narrow to capture all relevant benefits to patients and health care organizations [47]. Specifically, within mental health, the use of QALY and EQ-5D has also been found to be challenging with regard to instrument sensitivity [57].

    Participants in both the app and paper groups had a QALY gain, but the gain was significantly larger in the paper group. This result is difficult to explain as the difference in QALY gain between the groups was caused by a worsening in the mobility domain of the EQ-5D-5L among app users (Table 2). There is a risk that patients have interpreted the EQ-5D mobility domain differently, either as a question about physical restrictions or as limitations on mobility due to anxiety or depression. Nevertheless, the results suggest that using mobile apps as adjuncts to DBT might reduce the improvement in quality of life and depression scores within the first year.

    There are several limitations to this study, including the small sample size, short follow-up period, and missing data. The missing data on depression severity and utility domains were handled by performing multiple imputations and were assessed as not having an impact on the results [54]. There was a considerable amount of missing data for both borderline severity and suicide behavior, making it infeasible to run the imputation model for these outcomes. Therefore, borderline severity and suicidal behavior were presented as nonimputed data, potentially indicating that these results could be biased [55]. Previously, a moderate correlation has been shown between the EQ-5D utility scores and measures for borderline severity, supporting our findings of a similar improvement in QALY and borderline severity [58].

    Even though participants in the app group recorded more skills, this did not contribute to a larger QALY gain in the app group than in the paper group. One possible explanation is that a measurement of an effect like health-related quality of life might not capture the potential effects of the app [59], which is why including other effects is relevant. In this study, potential effects such as new and better information from the patients are highlighted, which is supported by the findings of Lauritsen [60], who concluded that the use of a mobile electronic diary can improve the quality and accuracy of data entered into the diary. A study by Derks et al [61] found that a mobile diary app together with a biosensor was effective in increasing self-awareness in patients with BPD. Similar results were found in this study, as key personnel stated that increased self-insight and empowerment might be a potential advantage linked to self-monitoring. Furthermore, both in this study and in prior studies [62,63], a mobile app has been proven to enhance treatment compliance. Thus, if this study had been conducted as a cost-utility, the aforementioned consequences would not have been included, meaning the conclusion might have been quite different.

    In this study, concerns regarding the app were primarily from therapists who worried that the app would negatively affect their interaction with the patient. In contrast, findings by Austin et al [64] suggests that a mobile app integrated into DBT was useful for patients with BPD in building an alliance with their therapist and that it promoted collaboration between therapist and patient. However, it is important to state that participants who enrolled in this study and the study by Austin et al [64] might have been advocates for technology since they knew the new and alternative treatment would be a mobile app, which could explain why patients were pleased with the app and why it was primarily therapists who had concerns. Thus, the impact of a mobile diary app on the therapist-patient relationship should be explored in future research.

    Conclusions

    In conclusion, this study suggests that using a mobile diary app as an adjunct to DBT has positive as well as negative consequences for patients and, on average, will lead to higher costs than paper-based diary cards. The use of a mobile diary app in the treatment of patients with BPD is still a relatively new field of research, and further investigation in this area is essential.

    Acknowledgments

    The authors would like to thank all the participants, both patients and therapists, at 5 different locations. Furthermore, the authors would like to acknowledge the staff at the Center for Telepsychiatry in the Region of Southern Denmark, University of Southern Denmark, and Monsenso for contributing to the mDiary program.

    SLL played a primary role in the economic evaluation. Contributions to the design of the trial and data collection were described in the protocol [29]. SLL, AL, JS, SSS, and LHE developed the analytic plan. SLL and AL analyzed the data. SLL, AL, JS, SSS, and LHE participated in data interpretation. SLL drafted the first version of the manuscript. All authors provided critical feedback, helped shape the analysis and writing of the manuscript, and approved the submission.

    This study is part of the ENTER project funded by Innovation Fund Denmark (grant number 5159-00002B). The Innovation Fund Denmark has not been involved in the design of the study or the subsequent handling of data. More information regarding the Innovation Fund Denmark can be found at the website [65].

    Conflicts of Interest

    The mDiary program was developed and owned by the private sector company Monsenso. SHJ has been employed at a public sector psychiatric ward, and he has no affiliation with Monsenso, nor does any of the other researchers.

    ‎
    Multimedia Appendix 1

    Detailed information on cost estimates in the economic analysis.

    DOC File , 86 KB
    ‎
    Multimedia Appendix 2

    A detailed description of the used approach for multiple imputation.

    DOC File , 43 KB
    1. Ekselius L, Tillfors M, Furmark T, Fredrikson M. Personality disorders in the general population: DSM-IV and ICD-10 defined prevalence as related to sociodemographic profile. Pers Individ Dif 2001 Jan 19;30(2):311-320. [CrossRef]
    2. Torgersen S, Kringlen E, Cramer V. The prevalence of personality disorders in a community sample. Arch Gen Psychiatry 2001 Jun;58(6):590-596. [CrossRef] [Medline]
    3. Pedersen L, Simonsen E. Incidence and prevalence rates of personality disorders in Denmark-a register study. Nord J Psychiatry 2014 Nov;68(8):543-548. [CrossRef] [Medline]
    4. National Collaborating Centre for Mental Health (UK). Borderline Personality Disorder: Treatment and Management. Leicester (UK): The British Psychological Society & The Royal College of Psychiatrists; 2009.
    5. Kjær JN, Biskin R, Vestergaard C, Munk-J Rgensen P. All-cause mortality of hospital-treated borderline personality disorder: a nationwide cohort study. J Pers Disord 2020 Dec;34(6):723-735. [CrossRef] [Medline]
    6. Wagner T, Fydrich T, Stiglmayr C, Marschall P, Salize H, Renneberg B, et al. Societal cost-of-illness in patients with borderline personality disorder one year before, during and after dialectical behavior therapy in routine outpatient care. Behav Res Ther 2014 Oct;61:12-22. [CrossRef] [Medline]
    7. Stoffers JM, Völlm BA, Rücker G, Timmer A, Huband N, Lieb K. Psychological therapies for people with borderline personality disorder. Cochrane Database Syst Rev 2012 Aug 15(8):CD005652. [CrossRef] [Medline]
    8. Rizvi SL, Steffel LM, Carson-Wong A. An overview of dialectical behavior therapy for professional psychologists. Prof Psychol Res Pr 2013;44(2):73-80. [CrossRef]
    9. Linehan MM, Comtois KA, Murray AM, Brown MZ, Gallop RJ, Heard HL, et al. Two-year randomized controlled trial and follow-up of dialectical behavior therapy vs therapy by experts for suicidal behaviors and borderline personality disorder. Arch Gen Psychiatry 2006 Jul;63(7):757-766. [CrossRef] [Medline]
    10. Linehan MM, Dimeff LA, Reynolds SK, Comtois KA, Welch SS, Heagerty P, et al. Dialectical behavior therapy versus comprehensive validation therapy plus 12-step for the treatment of opioid dependent women meeting criteria for borderline personality disorder. Drug Alcohol Depend 2002 Jun 01;67(1):13-26. [CrossRef] [Medline]
    11. McMain SF, Links PS, Gnam WH, Guimond T, Cardish RJ, Korman L, et al. A randomized trial of dialectical behavior therapy versus general psychiatric management for borderline personality disorder. Am J Psychiatry 2009 Dec;166(12):1365-1374. [CrossRef] [Medline]
    12. Verheul R, Van Den Bosch LM, Koeter MW, De Ridder MA, Stijnen T, Van Den Brink W. Dialectical behaviour therapy for women with borderline personality disorder: 12-month, randomised clinical trial in the Netherlands. Br J Psychiatry 2003 Feb;182:135-140. [CrossRef] [Medline]
    13. Brettschneider C, Riedel-Heller S, König HH. A systematic review of economic evaluations of treatments for borderline personality disorder. PLoS One 2014 Sep 29;9(9):e107748 [FREE Full text] [CrossRef] [Medline]
    14. Murphy A, Bourke J, Flynn D, Kells M, Joyce M. A cost-effectiveness analysis of dialectical behaviour therapy for treating individuals with borderline personality disorder in the community. Ir J Med Sci 2020 May;189(2):415-423. [CrossRef] [Medline]
    15. Priebe S, Bhatti N, Barnicot K, Bremner S, Gaglia A, Katsakou C, et al. Effectiveness and cost-effectiveness of dialectical behaviour therapy for self-harming patients with personality disorder: a pragmatic randomised controlled trial. Psychother Psychosom 2012;81(6):356-365 [FREE Full text] [CrossRef] [Medline]
    16. Swales M, Hibbs RA, Bryning L, Hastings RP. Health related quality of life for young people receiving dialectical behaviour therapy (DBT): a routine outcome-monitoring pilot. Springerplus 2016 Jul 20;5(1):1137 [FREE Full text] [CrossRef] [Medline]
    17. Chapman AL. Dialectical behavior therapy: current indications and unique elements. Psychiatry (Edgmont) 2006 Sep;3(9):62-68 [FREE Full text] [Medline]
    18. O'Connell B, Dowling M. Dialectical behaviour therapy (DBT) in the treatment of borderline personality disorder. J Psychiatr Ment Health Nurs 2014 Aug;21(6):518-525. [CrossRef] [Medline]
    19. Lindenboim N, Comtois KA, Linehan MM. Skills practice in dialectical behavior therapy for suicidal women meeting criteria for borderline personality disorder. Cogn Behav Pract 2007 May;14(2):147-156. [CrossRef]
    20. Linehan MM. DBT Skills Training Manual, Second Edition. New York: Guilford Publications; 2015.
    21. Helweg-Joergensen S, Schmidt T, Lichtenstein MB, Pedersen SS. Using a mobile diary app in the treatment of borderline personality disorder: mixed methods feasibility study. JMIR Form Res 2019 Sep 30;3(3):e12852 [FREE Full text] [CrossRef] [Medline]
    22. Marceau L, Link C, Smith L, Carolan S, Jamison R. In-clinic use of electronic pain diaries: barriers of implementation among pain physicians. J Pain Symptom Manage 2010 Sep;40(3):391-404 [FREE Full text] [CrossRef] [Medline]
    23. Rizvi SL, Hughes CD, Thomas MC. The DBT coach mobile application as an adjunct to treatment for suicidal and self-injuring individuals with borderline personality disorder: a preliminary evaluation and challenges to client utilization. Psychol Serv 2016 Nov;13(4):380-388. [CrossRef] [Medline]
    24. Rodante DE, Kaplan MI, Olivera Fedi R, Gagliesi P, Pascali A, Quintero PS, et al. CALMA, a mobile health application, as an accessory to therapy for reduction of suicidal and non-suicidal self-injured behaviors: a pilot cluster randomized controlled trial. Arch Suicide Res 2020 Oct 18:1-18. [CrossRef] [Medline]
    25. Schroeder J, Suh J, Wilks C, Czerwinski M, Munson S, Fogarty J, et al. Data-driven implications for translating evidence-based psychotherapies into technology-delivered interventions. Int Conf Pervasive Comput Technol Healthc 2020 May;2020:274-287 [FREE Full text] [CrossRef] [Medline]
    26. Lederer N, Grechenig T, Baranyi R. unCUT: bridging the gap from paper diary cards towards mobile electronic monitoring solutions in borderline and self-injury. In: Proceedings of the IEEE 3nd International Conference on Serious Games and Applications for Health (SeGAH).: IEEE; 2014 Presented at: IEEE 3nd International Conference on Serious Games and Applications for Health (SeGAH); May 14-16, 2014; Rio de Janeiro, Brazil. [CrossRef]
    27. Husereau D, Drummond M, Petrou S, Carswell C, Moher D, Greenberg D, ISPOR Health Economic Evaluation Publication Guidelines-CHEERS Good Reporting Practices Task Force. Consolidated health economic evaluation reporting standards (CHEERS) --explanation and elaboration: a report of the ISPOR health economic evaluation publication guidelines good reporting practices task force. Value Health 2013;16(2):231-250 [FREE Full text] [CrossRef] [Medline]
    28. The mDIARY Study. ENTER.   URL: http://www.ementalhealthresearch.dk/27-mdiary.html [accessed 2020-06-25]
    29. Helweg-Jørgensen S, Lichtenstein MB, Fruzzetti AE, Møller Dahl C, Pedersen SS. Daily self-monitoring of symptoms and skills learning in patients with borderline personality disorder through a mobile phone app: protocol for a pragmatic randomized controlled trial. JMIR Res Protoc 2020 May 25;9(5):e17737 [FREE Full text] [CrossRef] [Medline]
    30. Hartfiel N, Edwards R. Cost–consequence analysis of public health interventions. In: Applied Health Economics for Public Health Practice and Research. Oxford, United Kingdom: Oxford University Press; 2019.
    31. International statistical classification of diseases and related health problems 10th revision (ICD-10). World Health Organization. 2019.   URL: https://icd.who.int/browse10/2019/en#/F60.3 [accessed 2021-01-21]
    32. Monsenso. mDiary App. 2021.   URL: https://www.monsenso.com/mhealth-mental-health/individuals/ [accessed 2021-07-16]
    33. Janssen MF, Pickard AS, Golicki D, Gudex C, Niewada M, Scalone L, et al. Measurement properties of the EQ-5D-5L compared to the EQ-5D-3L across eight patient groups: a multi-country study. Qual Life Res 2013 Sep;22(7):1717-1727 [FREE Full text] [CrossRef] [Medline]
    34. Kroenke K, Spitzer RL, Williams JB. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med 2001 Sep;16(9):606-613 [FREE Full text] [CrossRef] [Medline]
    35. Zanarini MC, Vujanovic AA, Parachini EA, Boulanger JL, Frankenburg FR, Hennen J. Zanarini rating scale for borderline personality disorder (ZAN-BPD): a continuous measure of DSM-IV borderline psychopathology. J Pers Disord 2003 Jun;17(3):233-242. [CrossRef] [Medline]
    36. Osman A, Bagge CL, Gutierrez PM, Konick LC, Kopper BA, Barrios FX. The suicidal behaviors questionnaire-revised (SBQ-R): validation with clinical and nonclinical samples. Assessment 2001 Dec;8(4):443-454. [CrossRef] [Medline]
    37. Jensen C, Sørensen SS, Gudex C, Jensen M, Pedersen K, Ehlers L. The Danish EQ-5D-5L value set: a hybrid model using cTTO and DCE data. Appl Health Econ Health Policy 2021 Jul;19(4):579-591 [FREE Full text] [CrossRef] [Medline]
    38. Manca A, Hawkins N, Sculpher MJ. Estimating mean QALYs in trial-based cost-effectiveness analysis: the importance of controlling for baseline utility. Health Econ 2005 May;14(5):487-496. [CrossRef] [Medline]
    39. Schmidt M, Pedersen L, Sørensen HT. The Danish Civil Registration System as a tool in epidemiology. Eur J Epidemiol 2014 Aug;29(8):541-549. [CrossRef] [Medline]
    40. Andersen JS, Olivarius ND, Krasnik A. The Danish National Health Service Register. Scand J Public Health 2011 Jul;39(7 Suppl):34-37. [CrossRef] [Medline]
    41. Schmidt M, Schmidt SA, Sandegaard JL, Ehrenstein V, Pedersen L, Sørensen HT. The Danish National Patient Registry: a review of content, data quality, and research potential. Clin Epidemiol 2015 Nov 17;7:449-490 [FREE Full text] [CrossRef] [Medline]
    42. Lewis JR. The System Usability Scale: past, present, and future. Int J Hum Comput Interact 2018 Mar 30;34(7):577-590. [CrossRef]
    43. Net price index. Statistics Denmark. 2020.   URL: https://www.dst.dk/en/Statistik/dokumentation/documentationofstatistics/net-price-index [accessed 2020-11-04]
    44. Valutakurser [Exchange rates]. EUROinvestor. 2019.   URL: https://www.valutakurser.dk/ [accessed 2020-06-25]
    45. Drummond M, Sculpher M, Claxton K, Stoddart G, Torrance G. Methods for the Economic Evaluation of Health Care Programmes. 4th Edition. Oxford: Oxford University Press; Sep 2015.
    46. Kildemoes HW, Sørensen HT, Hallas J. The Danish National Prescription Registry. Scand J Public Health 2011 Jul;39(7 Suppl):38-41. [CrossRef] [Medline]
    47. Bouwmans C, De Jong K, Timman R, Zijlstra-Vlasveld M, Van der Feltz-Cornelis F, Tan SS, et al. Feasibility, reliability and validity of a questionnaire on healthcare consumption and productivity loss in patients with a psychiatric disorder (TiC-P). BMC Health Serv Res 2013 Jun 15;13:217 [FREE Full text] [CrossRef] [Medline]
    48. Kirisits A, Redekop WK. The economic evaluation of medical devices: challenges ahead. Appl Health Econ Health Policy 2013 Feb;11(1):15-26. [CrossRef] [Medline]
    49. Drummond M, Tarricone R, Torbica A. Economic evaluation of medical devices. Oxford Res Encycl Econ Financ 2018 Mar 28:A. [CrossRef]
    50. Statistikbanken - LONS20: Løn efter arbejdsfunktion, sektor, aflønningsform, lønmodtagergruppe, lønkomponenter og køn [Statistikbanken - LONS20: Wages estimated based on job function, sector, payment form, employment group, wage component and gender]. Statistics Denmark. 2018.   URL: https://www.statistikbanken.dk/statbank5a/default.asp?w=1536 [accessed 2020-06-10]
    51. Levetider [Service lives]. Agency for Modernisation - Ministry of Finance.   URL: https://oes.dk/oekonomi/oeav/regnskabsregler/generelle-bogfoeringsbestemmelser/levetider/ [accessed 2020-06-18]
    52. Den samfundsøkonomiske diskonteringsrente [The economic discount rate]. Ministry of Finance.   URL: https://fm.dk/media/10174/dokumentationsnotat-_-den-samfundsoekonomiske-diskonteringsrente.pdf [accessed 2021-09-17]
    53. Ehlers LH, Vestergaard AS. Costing in Health Economic Evaluation. Aalborg: Aalborg University Press; 2019:1-88.
    54. Faria R, Gomes M, Epstein D, White IR. A guide to handling missing data in cost-effectiveness analysis conducted within randomised controlled trials. Pharmacoeconomics 2014 Dec;32(12):1157-1170 [FREE Full text] [CrossRef] [Medline]
    55. Sterne JA, White IR, Carlin JB, Spratt M, Royston P, Kenward MG, et al. Multiple imputation for missing data in epidemiological and clinical research: potential and pitfalls. Br Med J 2009 Jun 29;338:b2393 [FREE Full text] [CrossRef] [Medline]
    56. Kirkwood B, Sterne J. Essential Medical Statistics. Malden, Mass: Wiley-Blackwell; 2003.
    57. Franklin M, Enrique A, Palacios J, Richards D. Psychometric assessment of EQ-5D-5L and ReQoL measures in patients with anxiety and depression: construct validity and responsiveness. Qual Life Res 2021 Sep;30(9):2633-2647 [FREE Full text] [CrossRef] [Medline]
    58. van Asselt AD, Dirksen C, Arntz A, Giesen-Bloo J, Severens J. The EQ-5D: a useful quality of life measure in borderline personality disorder? Eur Psychiatry 2009 Mar;24(2):79-85. [CrossRef] [Medline]
    59. Bergmo TS. Using QALYs in telehealth evaluations: a systematic review of methodology and transparency. BMC Health Serv Res 2014 Aug 03;14:332 [FREE Full text] [CrossRef] [Medline]
    60. Lauritsen K, Innocenti AD, Hendel L, Praest J, Lytje MF, Clemmensen-Rotne K, et al. Symptom recording in a randomised clinical trial: paper diaries vs. electronic or telephone data capture. Control Clin Trials 2004 Dec;25(6):585-597. [CrossRef] [Medline]
    61. Derks YP, Klaassen R, Westerhof GJ, Bohlmeijer ET, Noordzij ML. Development of an ambulatory biofeedback app to enhance emotional awareness in patients with borderline personality disorder: multicycle usability testing study. JMIR Mhealth Uhealth 2019 Oct 15;7(10):e13479 [FREE Full text] [CrossRef] [Medline]
    62. Carter MC, Burley VJ, Nykjaer C, Cade JE. Adherence to a smartphone application for weight loss compared to website and paper diary: pilot randomized controlled trial. J Med Internet Res 2013 Apr 15;15(4):e32 [FREE Full text] [CrossRef] [Medline]
    63. Abu-El-Noor N, Aljeesh Y, Bottcher B, Abu-El-Noor M. Impact of a mobile phone app on adherence to treatment regimens among hypertensive patients: a randomised clinical trial study. Eur J Cardiovasc Nurs 2021 Jun 29;20(5):428-435. [CrossRef] [Medline]
    64. Austin SF, Jansen JE, Petersen CJ, Jensen R, Simonsen E. Mobile app integration into dialectical behavior therapy for persons with borderline personality disorder: qualitative and quantitative study. JMIR Ment Health 2020 Jun 11;7(6):e14913 [FREE Full text] [CrossRef] [Medline]
    65. Innovation Fund Denmark.   URL: https://innovationsfonden.dk/en [accessed 2021-10-19]

    ‎
    BPD: borderline personality disorder
    DBT: dialectical behavior therapy
    EQ-5D-5L: EuroQol 5-Dimensions 5-Levels
    QALY: quality-adjusted life year

    Edited by R Kukafka; submitted 17.03.21; peer-reviewed by I Rodriguez, S Landes; comments to author 23.04.21; revised version received 17.06.21; accepted 27.07.21; published 11.11.21

    Copyright

    ©Sidsel Lund Laursen, Stig Helweg-Jørgensen, Astrid Langergaard, Jesper Søndergaard, Sabrina Storgaard Sørensen, Kim Mathiasen, Mia Beck Lichtenstein, Lars Holger Ehlers. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 11.11.2021.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.