Original Paper
Abstract
Background: Panic disorder is an anxiety disorder marked by severe fear of panic attacks in the absence of causes. Agoraphobia is a related anxiety disorder, which involves fear and avoidance of specific situations in which escape or help may be difficult. Both can be debilitating and impair well-being. One treatment option may be internet-based cognitive behavioral therapy (iCBT), which allows large-scale application and may overcome treatment barriers for some individuals.
Objective: This study aimed to evaluated the effectiveness of a novel online self-help intervention for panic disorder with or without agoraphobia. As our primary hypotheses, we expected the intervention to improve panic and agoraphobia symptoms and well-being. Our secondary hypotheses entailed improvements in daily functioning, mental health literacy, working ability, and health care use in the intervention group.
Methods: German-speaking patients (N=156) aged 18-65 years with internet access and a diagnosis of panic disorder with or without agoraphobia were recruited for this randomized controlled trial. The intervention group (n=82) received access to a 12-week online self-help program entailing psychoeducation, cognitive restructuring, exposure, and mindfulness elements. The control group (n=72) received care as usual during the study period and was offered the prospect of using the program after 12 weeks. The primary outcomes were assessed via the Panic and Agoraphobia Scale (PAS) and the WHO (World Health Organization)-5 Well-Being Index (WHO-5). Mixed effect models were computed using multivariate imputation by chained equation for the analysis of intervention effects.
Results: In the intervention group, participants completed on average 7.3 out of 12 (60.8%) modules, and 27 out of 82 (32.1%) participants finished the whole course. Changes in PAS revealed a significant effect in favor of the intervention group (t110.1=–2.22, Padj=.03) with a small to moderate effect size (d=–0.37, 95% CI –0.70 to –0.04). No significant effect was found for the second primary outcome WHO-5 (t149.8=1.35, Padj=.09) or the secondary outcomes. Improvements were observed in anxiety (t206.8=–4.12; P<.001; Cohen d=–0.60, 95% CI –0.089 to –0.32) and depression (t257.4=–3.20; P<.001; Cohen d=–0.41 95% CI –0.66 to –0.16). No negative effects were associated with the intervention (t125=–1.14, P=.26).
Conclusions: The presented online intervention can help reduce the core symptomatology of panic disorder and agoraphobia, as well as anxiety symptoms and associated depression. No effects were found for well-being and secondary outcomes. This may be due to higher illness burden in the intervention group and possibly the COVID pandemic, which caused unique challenges to patients suffering from panic disorder. Therefore, further research and intervention adaptations may be warranted to improve these outcomes.
Trial Registration: German Clinical Trials Register DRKS00023800; https://drks.de/search/en/trial/DRKS00023800
doi:10.2196/54062
Keywords
Introduction
Background
Panic disorder involves recurring sudden panic attacks and a constant fear of experiencing more episodes. These attacks come with physical symptoms like breathing issues, palpitations, sweating, and nausea, as well as psychological symptoms such as derealization and a fear of losing control or dying [,]. Agoraphobia, which affects 35%-65% of people with panic disorder [], involves excessive fear and avoidance of situations where escape may be difficult, like public transport, and help may not be readily available in case of a panic attack. Both panic disorder and agoraphobia are closely related and often seen as part of a continuum []. In earlier versions of the DSM (Diagnostic and Statistical Manual of Mental Disorders) (ie, the third and fourth editions [DSM-III and DSM IV]) agoraphobia was defined as a feature of panic disorder (ie, panic disorder with agoraphobia or panic disorder without agoraphobia), which is why some research is organized around this conceptualization.
Panic disorder, with or without agoraphobia, comes with functional impairments and reduced well-being, increasing the risk of other mental disorders [-]. It also imposes a significant economic burden, surpassing that of other anxiety, mood, or alcohol-related disorders []. These costs include hospital treatments, health care visits, and, most notably, absenteeism, which accounts for 60% of all expenses []. Thus, it is crucial to offer effective and timely treatment for panic disorder for both societal and individual well-being.
Regarding psychotherapeutic treatments, recently cognitive behavioral therapy (CBT) and short-term psychodynamic therapy have been identified as treatments of choice []. CBT targets fear and avoidance behavior by psychoeducation, exposure therapy, cognitive restructuring, mindfulness, and acceptance interventions leading to large effect sizes compared with a waitlist group (hedges g=0.96; []). Also, effects remain superior to treatment as usual for 6 months of follow-up []. Psychopharmacology yields small to moderate effect sizes in comparison to placebo and similar to CBT []. Although, compared with pharmacotherapy, CBT shows longer-term treatment effects, better cost-efficacy, and higher patient acceptance [].
An analysis of treatment barriers in anxiety disorders [] showed that 63 out of 77 (81.8%) patients with panic disorder contemplate treatment. Still, only 40 out of 59 (67.3%) sought help at least once in their life, and this number was even lower in patients with agoraphobia (21/56, 36.9%). Frequently reported barriers to help-seeking include self-reliance, presumed ineffectiveness, high waiting periods, or problems with the practitioner []. Further, negative attitudes and lack of knowledge and appropriate beliefs about mental health are associated with less help-seeking behavior [].
During treatment waiting periods, it is recommended to offer patients self-help programs based on CBT [], since technology-based treatment alternatives can help surpass the aforementioned barriers []. Most of these alternatives are based on CBT, since it is well suited for online intervention delivery due to its highly structured, directive, standardized nature and its focus on psychoeducation and homework []. Of particular note is the benefit of internet-based CBT (iCBT) to the healthcare system, as it is a cost-effective treatment alternative with similar efficacy to face-to-face CBT [,]. Also, iCBT can help to bridge the waiting periods for face-to-face psychotherapy, which on average lasts several months in Germany and has further increased since the beginning of the COVID-19 pandemic [,].
A systematic review and meta-analysis including 27 studies on iCBT for panic disorder [] showed high efficacy and effectiveness for reducing symptoms of panic disorder (hedges g=1.16) and agoraphobia (hedges g=0.91) compared with waitlist. Another meta-analysis of 13 RCTs for panic disorder and agoraphobia found no efficacy difference between unguided iCBT and face-to-face CBT in terms of panic and agoraphobia symptoms, comorbid depression and anxiety, as well as quality of life improvement [] indicating that unguided iCBT may be comparably effective to face-to-face CBT. However, in a network meta-analysis of 74 efficacy trials, unguided iCBT was not superior to care as usual for the treatment of panic with or without agoraphobia []. In the same analysis, both guided forms of iCBT and face-to-face CBT were superior to care as usual, though not superior to unguided iCBT.
Therefore, results regarding the efficacy and effectiveness of unguided iCBT for panic and agoraphobia are mixed, paradoxically indicating similar effects as face-to-face CBT while also indicating no improvement to care as usual. Also, little research can be found investigating broader effects on well-being, functioning, mental health literacy, working ability, and health care use. These seem particularly relevant to assess effects on cost-effectiveness.
Objectives
This study aimed to evaluate a 12-week online self-help program (Selfapy) for patients with panic disorder with or without agoraphobia within the German health care system. For the analysis, we had 2 objectives: first, we compared the intervention with care as usual, since the superiority for unguided self-help toward care as usual has been questioned; second, we assessed effectiveness for a range of broader outcomes with limited research data (eg, well-being and functioning).
Hypotheses
Based on previous research, the primary hypotheses expect superior improvement on panic and agoraphobia symptoms and well-being in the intervention group (IG). The secondary hypotheses expect superior improvement of daily functioning, work capacity, mental health literacy, and the efforts and burdens of patients for the health care sector. Finally, changes in comorbid anxiety and depression symptoms as well as negative effects of the program beyond symptomatology, are being explored as exploratory hypotheses. All outcomes were tested against the control group (CG) after 12 weeks.
Methods
Study Design
The study is reported according to the CONSORT-EHEALTH (Consolidated Standards of Reporting Trials of Electronic and Mobile Health Applications and Online Telehealth) eHealth guidelines (). The parallel group trial was preregistered beforehand [] and a study protocol was published []. Eligible patients were randomly assigned to the IG or CG in a 1:1 ratio. Patients in the IG could access the intervention immediately after randomization, while the CG could only access the intervention after a waiting period of 12 weeks. Interim and final evaluations occurred 6 (T2) and 12 (T3) weeks after the baseline assessment (T1).
Participants
Announcements for study participation were published in the whole of Germany via a university email newsletter, social media, and on flyers in clinics, pharmacies, and practices of medical doctors and psychotherapists. After an online prescreening, participants were invited to choose an appointment for a remote diagnostic interview assessing the inclusion and exclusion criteria. To this end, a structured diagnostic interview diagnostisches interview psychischer störungen-open access (DIPS-OA, [,]) was conducted with every participant via video calls. Regarding DSM-IV-TR (Diagnostic and Statistical Manual of Mental Disorders [Fourth Edition, Text Revision]) criteria, the DIPS-OA was found to have acceptable interrater (0.78) and retest (0.76) reliability for anxiety disorders []. Altogether 4361 people started the online screening, of whom 764 (17.5%) were deemed eligible to be scheduled for the diagnostic interview. The online screening consisted of short screening questionnaires of the inclusion and exclusion criteria.
Inclusion and Exclusion Criteria
Video calls were conducted with all individuals who checked the prescreening criteria to assess inclusion and exclusion criteria, during which eligibility was assessed via the DIPS-OA. Trained psychologists conducted all interviews under the supervision by a certified psychotherapist (CBT).
Eligible individuals were those who (1) were between 18 and 65 years of age, (2) had sufficient knowledge of the German language, (3) had uninterrupted internet access, (4) provided electronic informed consent to participate in the study, and (5) met the criteria for a diagnosis of panic disorder with or without an additional diagnosis of agoraphobia.
Individuals were excluded if they did not meet any of the inclusion criteria or met any of the following criteria: (1) past or current diagnosis of bipolar disorder, (2) past or current diagnosis of psychotic disorder, (3) current diagnosis of substance dependence, (4) current diagnosis of a severe major depressive episode, and (5) acute suicidality. The criteria were chosen because they could interfere with the successful implementation of the course.
Intervention
The online self-help program for the treatment of panic disorder with or without agoraphobia (Selfapy) was conducted as the intervention. The program is based on evidence-based methods and exercises derived from CBT (such as psychoeducation, cognitive restructuring, or exposure), as well as elements from Mindfulness-Based Therapy (eg, [,]). The online intervention consists of core modules, which include mandatory exercise content, and a subsequent set of optional specialization areas that are individually selectable (for a complete overview, []). Each module covers a specific topic, such as exposure, mindfulness, or problem-solving training. The modules contain informative texts, videos, audio, and interactive exercises and can be used via the web as well as on mobile devices.
Participants completed the online program independently and without support. They were monitored for suicidality in which case they were messaged by a psychologist. Active communication only occurred for safety reasons. The IG had immediate access to the 12-week self-help treatment and were advised to spend 15-20 minutes daily on it.
The CG
The CG received no treatment for 12 weeks but could seek other assistance beyond this trial, such as medication or therapy, to mimic routine care. All concurrent treatments were self-reported. The CG received the study intervention after study completion (=after 12 weeks), since 12 weeks is a common waiting time for psychotherapy in Germany [].
Outcomes
Measures were conducted at 3 different points in time: Before the start of the intervention (T1, baseline), after 6 weeks (T2, during treatment), and 12 weeks after the beginning of the intervention (T3, post treatment). At each measurement time point, the primary, secondary and exploratory outcomes were assessed. Assessment was conducted via an online assessment platform [].
Primary Outcome Measures
The change in panic and agoraphobia symptoms was evaluated using the Panic and Agoraphobia Scale (PAS; []). The PAS consists of 13 items that are rated on a 5-point Likert scale. There scale contains 5 subscales: panic attacks, agoraphobic avoidance, anticipatory avoidance, disability, and worries about health. In our data, we calculated McDonald ω [] as a reliability measure with ω=0.86 at T1. Well-being was assessed by the WHO (World Health Organization)-5 Well-Being Index (WHO-5; []). The WHO-5 contains items measuring positive mood, calmness, high energy levels, good rest, and interest in daily activities.
Secondary Outcome Measures
Functioning in daily life was measured by the Work and Social Adjustment Scale (WSAS; []) assessing professional and personal functioning. Reliability was good (ω=0.79) at T1.
Work capacity was measured with the iMTA Productivity Cost Questionnaire (iPCQ) to assess the amount of lost working hours in the last 4 weeks due to absenteeism or distress-related impaired work capabilities (iPCQ []).
Mental health literacy was measured with the Mental Health Literacy Scale (MHLS; []) with high reliability (ω=0.85 at T1).
The extent of therapy-related efforts and burdens of patients (Client Sociodemographic and Service Receipt Inventory [CSSRI] []) was collected on three subscales: CSSRI-partly inpatient to assess partly inpatient treatment, CSSRI-complementary to assess complementary services (eg, self-help groups), and CSSRI-ambulant to assess outpatient services (eg, psychotherapy treatment and medical treatment).
Additional Outcome Measures
Adverse treatment effects were assessed with the Negative Effects Questionnaire (NEQ; []). The NEQ contains 32 items and showed high reliability (ω=0.89 at T3). Also, general symptoms of anxiety were assessed using the Beck Anxiety Inventory (BAI; []) with ω=0.89 at T1. Depressive symptoms were collected with the Patient Health Questionnaire-9 (PHQ-9 []; ω=0.74) at T1.
Sample Size
The between-group effect size estimate was based on meta-analytic evidence for effect sizes in unguided online psychological interventions for anxiety disorders Cohen (d=0.45; eg, []). This effect was used as the basis for sample size determination. For the planned mixed model with 2 measurement time points with a general correlation structure [], a directed hypothesis, a group allocation of 1:1, a power of 0.80, and an α level of .025 after Bonferroni-Holm correction, a total of 156 patients (78 per group) were needed. The number of cases was calculated using the R-tool (Michael C. Donohue) longpower []. For the secondary outcomes, we calculated a minimal detectable effect size of Cohen d=–0.46 with 80% power and an α level of .0125 (Bonferroni-Holm adjustment) based on a post hoc power analysis of the WSAS with simr (Peter Green) [].
Randomization and Blinding
Randomization in 1:1 ratio without stratification or blocks was done automatically by a computer-generated code. Participants were automatically informed of their group via email. Data collection, evaluation, and statistical analysis were carried out blindly. A team member not involved in the analysis coded the group variable, and analysis scripts were prepared before knowing the actual data.
Statistical Analyses
The statistical analyses were conducted following the study protocol []. The analyses were performed with R (version 4.2.0; R Core Team) [].
Adhering to intention-to-treat principles, none of the enrolled participants were generally excluded. Missing values were replaced by multivariate imputation by chained equations (MICE; with n=5 imputations []) based on the control arm, using the variables age and gender as predictors in addition to the T1 outcome measurements. Four additional sensitivity analyses are reported in the online support material (OSM) in (OSM 1-4) for the primary outcomes: A completer analysis using only patient data with completed T1 and T3 measures, last-observation-carried-forward (LOCF), baseline-observation-carried-forward (BOCF), and a reference-based-multiple jump to reference imputation (J2R; []). In addition to these analyses, a “per-protocol” sample sensitivity analysis was defined for exploratory analyses, including all IG patients who completed at least 4 of the 12 modules.
The confirmatory analysis of the primary endpoints consisted of calculating a mixed model with 2 measurement time points and a general correlation structure []. A random effect for the participant was calculated (random intercept), and 3 fixed effects (group, time, and the interaction of these 2 effects). The 2 measurement times were nested within participants. The treatment effect was estimated as the fixed interaction effect after the final (T3) assessment. To assess the magnitude of the treatment effects, the fixed interaction effect of time and group was divided by the root of the summed variances of the random effects []. Effect sizes can be roughly interpreted according to Cohen d: effect sizes of 0.20 are considered small, 0.50 moderate, and 0.80 large [].
Secondary confirmatory outcomes were calculated only after success in the primary analysis to prevent alpha inflation, using the same mixed model with a random intercept for the participant. The CSSRI questionnaire was split into the 3 most relevant subscales (CSSRI-partially inpatient, CSSRI-outpatient, and CSSRI-complementary) that were also subject to Bonferroni-Holm adjustment by dividing by 3, 2, and 1, respectively. Due to highly skewed data, the iPCQ and the CSSRI partially inpatient scales were log10-transformed. The CSSRI-outpatient and CSSRI-complementary scales were dichotomized because of rare extreme outliers, in which case a transformation is not recommended []. For these dichotomized measures, the analysis was adapted to a mixed logistic regression model to stay as close as possible to the study protocol and odds ratios were calculated as the effect size.
The additional outcome measures, BAI and PHQ-9, were analyzed using the same model as the primary and secondary outcomes. Independent t tests and chi-square tests were used to estimate differences between groups in pretreatment sample characteristics. Also, t tests were used to identify differences in adverse effects in the NEQ at T3.
Data and Code
All data and analysis code have been made publicly available at the OSF repository and can be accessed at []. Materials about the content of the online intervention are reported in the study protocol [].
Ethical Considerations
This study was approved by the ethics committee at Heidelberg University in adherence to institutional guidelines (AZ Prüß 2021 1/1). All participants provided informed consent prior to their study participation and were informed that they could withdraw consent anytime. During the study, participant data were pseudonymized by replacing data identifiers (eg, participants’ name) with a pseudonymous code. After study completion, all data were anonymized by deleting the data identifiers. All participants received an allowance of 30€ after completing the questionnaires (T1 and T3). Suicidal thoughts were assessed at multiple time points (T1, T2, and T3) using a Likert scale. If participants rated their thoughts as 1 or higher (indicating they had thoughts of wanting to harm themselves) for the past 2 weeks, they were contacted by phone or email, and an emergency plan was established. For individuals contacted due to suicidality, their questionnaire participation was halted to prioritize immediate support. Suicidal incidents occurred 3 times, all within the CG.
Results
Participants Flow
Recruitment occurred from February 12, 2021, to March 21, 2022. A total of 764 clinical interviews were conducted, leading to 292 participants being excluded based on inclusion and exclusion criteria (
Figure 1). Further, 160 participants declined study participation and additional 156 participants joined another trial on generalized anxiety disorder []. The remaining 156 participants were randomized into the IG (n=82) and CG (n=74). Sociodemographic differences between the groups were not significant.
Deviations from the sample size occurred for some of the secondary and additional outcomes due to partial missingness.
More IG participants (30/81, 35.8%) had a current diagnosis of social phobia compared with CG (11/73, 15.1%, P=.005). However, more CG participants had a past diagnosis of social phobia (4/73, 5.5%) than IG (0/81, 0%, P=.048). For both current generalized anxiety disorder (31/81, 38.3% vs 17/73, 23.3%, P=.06) and current major depression (8/81, 9.8% vs 1/73, 1.4%, P=.08) there was a trend toward statistical significance for higher occurrence in the IG. No other diagnosis differences were observed (OSM 6 in ). Current psychopharmacology (P=.81) and psychotherapy (P=.34) also did not differ between groups. The CG had lower anxiety levels at baseline for the BAI (t153.8=2.30, P=.02). No group differences were found in primary or secondary outcomes.
Participants’ Characteristics
The sociodemographic characteristics of all participants are displayed in 
Table 1. Altogether, out of 194 participants, 59 (38.3%) were diagnosed with panic disorder, whereas 96 (61.7%) fulfilled the diagnostic criteria of panic disorder with additional agoraphobia. Regarding comorbidities, 47 (30.1%) participants also fulfilled the diagnostic criteria for generalized anxiety disorder, which was the most prominent comorbid disorder (OSM 6 in ). Comorbid depression rates were relatively low, with only 9 (5.8%) participants fulfilling the diagnostic criteria for major depressive disorder. It is important to note that the low comorbidity rate can be attributed to this study’s exclusion criteria, which specifically excluded individuals with severe depression.
| Characteristics | Treatment (n=82) | Control (n=74) | Total sample (N=156) | ||||
| Sex, n (%) | |||||||
| Female | 65 (79.27) | 56 (75.68) | 121 (77.56) | ||||
| Male | 16 (19.51) | 17 (22.97) | 33 (21.15) | ||||
| Nonbinary | 1 (1.22) | 1 (1.35) | 2 (1.29) | ||||
| Age in years (mean, SD) | 35.1 (11.5) | 35.0 (11.1) | 35.0 (11.3) | ||||
| Health care use, n (%) | |||||||
| Psychotherapy | 35 (42.68) | 26 (35.14) | 61 (39.10) | ||||
| Pharmacotherapy | 27 (32.93) | 23 (31.08) | 50 (32.05) | ||||
| Relationship statusa, n (%) | |||||||
| Married | 23 (27.71) | 26 (35.14) | 49 (31.21) | ||||
| Not living with partner | 3 (3.61) | 0 (0) | 3 (1.91) | ||||
| Single | 51 (61.45) | 45 (60.81) | 96 (61.15) | ||||
| Divorced | 5 (6.02) | 3 (4.05) | 8 (5.10) | ||||
| Widowed | 1 (1) | 0 (0) | 1 (0.64) | ||||
| Children, n (%) | |||||||
| Yes | 12 (14.63) | 19 (25.68) | 31 (19.87) | ||||
| No | 70 (85.37) | 55 (74.32) | 125 (80.13) | ||||
| Current diagnoses, n (%) | |||||||
| Panic disorder without agoraphobia | 27 (33.3) | 32 (43.8) | 59 (38.3) | ||||
| Panic disorder with agoraphobia | 54 (66.7) | 41 (56.2) | 96 (61.7) | ||||
| Generalized anxiety disorder | 31 (38.3) | 17 (23.3) | 48 (31.2) | ||||
| Major depressive disorder | 8 (9.8) | 1 (1.4) | 9 (5.8) | ||||
| Social phobia | 29 (35.8) | 4 (5.5) | 40 (26) | ||||
aMultiple options were possible.
Missing Data
Noncompletion rates were 16.0% (25/84) for the PAS (first primary outcome) and 18.6% (29/72) for the WHO-5 (second primary outcome) at T3. Therefore, 70 out of 82 (85.4%) participants in the IG and 61 out of 74 (82.4%) participants in the CG completed the PAS at posttreatment. Similarly, 68 out of 84 (82.3%) participants in the IG and 59 out of 72 (79.7%) participants in the CG completed the WHO-5 post treatment.
Logistic regression analyses showed that participant dropout for the WHO-5 was not associated with any of the following baseline variables: group allocation, sex, age, work capacity, medication intake, or the baseline values of any of the primary or secondary outcomes. For the PAS, dropout was not associated with group allocation, age, work capacity, psychotherapy, or the baseline values of any primary or secondary outcomes. However, female sex (P=.02) and medication intake (P=.05) were associated with reduced dropout for the PAS.
Since none of the baseline outcomes were associated with dropout, the empirical data supported missing at random instead of missing not at random. Therefore, as determined in the study protocol, MICE analysis was conducted as our primary analysis.
Adverse Events
During the trial, 3 participants from the CG reported suicidality and had to be excluded from the study. Trained psychologists immediately contacted them. They were excluded from further data collection, but all previous data were still used.
Adherence
Due to a technical problem, usage data were missing for 1 person in the IG. Based on data from 81 participants, the IG completed an average of 7.3 (SD 3.9) modules out of the total 12 modules, amounting to 7.3 out of 12 (60.8%) of all modules. Overall, 26 out of 84 (32.1%) participants completed the whole course, and 73 out of 84 (90.1%) participants completed the first 4 modules, which was chosen as a sensitivity analysis to assess a basic amount of engagement.
Primary Outcomes
For the primary outcome PAS (t110.1=–2.22, P=.03), we found a significant time×group interaction effect, but not for the primary outcome WHO-5 (t149.8=–1.35, P=.09), as shown in Table 2. Effect sizes were small to moderate for the PAS (d=–0.37, 95% CI –0.70 to –0.04) and small for the WHO-5 Cohen (d=0.22; 95% CI –0.10 to 0.53). Within-group T1-T3 effect sizes for PAS were moderate in the IG Cohen (d=–0.58, 95% CI –0.77 to –0.39) and small in the CG Cohen (d=–0.21, 95% CI –0.42 to 0.00). Within-group effects were small to moderate in the IG Cohen (d=0.30, 95% CI 0.06 to 0.53) and minimal in the CG Cohen (d=0.08, 95% CI –0.11 to 0.28) for WHO-5. All imputations and plots can be seen in OSM 1-4 in .
| Primary outcome | t test (df) | P value (1-sided) | Adjustment factor | Adjusted P value | Effect size d (95% CI) |
| PASa | –2.22 (110.1) | .01 | 2 | .03 | –0.37 (–0.70 to –0.04) |
| WHO-5b | 1.35 (149.8) | .09 | 1 | .09 | 0.22 (–0.10 to 0.53) |
aPAS: Panic and Agoraphobia Scale.
bWHO-5: WHO (World Health Organization)-5 Well-Being Index.
On average, severe levels of panic and agoraphobia were reported at baseline (mean 34.56, SD 8.31; scores range from 13 to 66; Table 3). Moreover, low well-being was reported at baseline (mean 2.87, SD=0.97; scores range from 0.0 to 4.2; Table 3).
| Imputation | T1 | T2 | T3 | ||||||||||||
| N | Mean (SD) | N | Mean (SD) | N | Mean (SD) | ||||||||||
| ITTa PASb | |||||||||||||||
| Treatment | 82 | 35.1 (7.93) | 66 | 32.3 (8.41) | 70 | 29.9 (7.90) | |||||||||
| Control | 74 | 33.9 (8.73) | 62 | 33.7 (7.84) | 61 | 32.7 (8.77) | |||||||||
| ITT WHO-5c | |||||||||||||||
| Treatment | 82 | 2.83 (0.94) | 64 | 3.01 (1.01) | 68 | 3.12 (1.06) | |||||||||
| Control | 74 | 2.92 (1.00) | 60 | 3.03 (1.02) | 59 | 2.99 (1.04) | |||||||||
aITT: intention-to-treat.
bPAS: Panic and Agoraphobia Scale.
cWHO-5: WHO (World Health Organization)-5 Well-Being Index.
Minimal Clinical Important Difference
The Reliable Change Index (RCI; []) was used to calculate reliable improvement or deterioration. Regarding the PAS, 33.6% of IG and 16.8% of CG patients improved reliably from T1 to T3. In contrast, 6.1% deteriorated in the IG and 9.2% in the CG. Therefore, a significant difference was found (P<.001).
For the WHO-5, improvement (28.3% vs 11.4%) favored the IG. However, deterioration (12.7% vs 11.4%) was stronger in the CG. A significant difference was identified (P=.002).
Sensitivity Analyses for Per-Protocol Sample (4 Completed Modules)
In addition, per-protocol sensitivity analyses were conducted for both primary outcomes, including only the 73 (90.1%) participants who completed at least the first 4 modules. Because no hypotheses were specified, the effects are calculated as 2-tailed tests without alpha adjustment. A significant group×time interaction (t252=–3.25, P=.001) was found for the PAS with a moderate effect size Cohen (d=–0.50; 95% CI –0.80 to –0.20). Regarding the WHO-5, no significant interaction was found (t252=1.80, P=.07).
Secondary Outcomes
None of the interaction effects of the secondary outcomes were significant after the Bonferroni-Holm adjustment (Table 4).
On average, impairment of daily functioning (WSAS) was moderate at baseline (mean 4.1, SD 1.7), while mental health literacy (MHLS) levels were very high (mean 4.4, SD 0.4). iPCQ scores and CSSRI outpatient treatment were log-transformed due to the right-skewed distribution (Table 5). Inpatient (CSSRI partly inpatient) and complementary (CSSRI complementary) treatment occurred for 15.3% and 12.1% of participants at baseline (Table 6).
| Outcome | Group×time | Effect size d (95% CI) | ||||
| t test (df) | P value (1-sided) | Adjustment factor | Adjusted P value | |||
| WSASa | –.72 (105.1) | .04 | 4 | .17 | –0.22 (–0.48 to 0.03) | |
| MHLSb | –0.76 (231.6) | .22 | 2 | .45 | –0.09 (–0.34 to 0.15) | |
| iPCQc | –1.37 (206.8) | .09 | 3 | .26 | –0.21 (–0.51 to 0.09) | |
| CSSRId outpatient | 0.48 (231.6) | 1.0 | 1×1e | 1.0 | 0.07 (–0.22 to 0.37) | |
| CSSRI partly inpatient | –0.56 (6.2) | .29 | 1×3e | .88 | 0.24f (0 to 30.16) | |
| CSSRI complementary | –0.30 (6.5) | .39 | 1×2e | .78 | 0.16f (0 to 28134) | |
aWSAS: Work and Social Adjustment Scale.
bMHLS: Mental Health Literacy Scale.
ciPCQ: iMTA Productivity Cost Questionnaire.
dCSSRI: Client Sociodemographic and Service Receipt Inventory.
eThe additional factors are due to the additional α adjustments of the CSSRI subscales.
fEffects are odds ratios.
| Outcome | T1 | T2 | T3 | |||||||||||
| N | Mean (SD) | N | Mean (SD) | N | Mean (SD) | |||||||||
| WSASa | ||||||||||||||
| Treatment | 82 | 4.16 (1.67) | 82 | 3.89 (1.72) | 82 | 3.55 (1.68) | ||||||||
| Control | 74 | 4.12 (1.73) | 74 | 3.87 (1.68) | 74 | 3.90 (1.86) | ||||||||
| MHLSb | ||||||||||||||
| Treatment | 82 | 4.38 (0.39) | 82 | 4.37 (0.40) | 82 | 4.35 (0.45) | ||||||||
| Control | 74 | 4.38 (0.37) | 74 | 4.36 (0.39) | 74 | 4.38 (0.42) | ||||||||
| iPCQc log | ||||||||||||||
| Treatment | 82 | 0.50 (0.67) | 82 | 0.45 (0.63) | 82 | 0.40 (0.65) | ||||||||
| Control | 74 | 0.44 (0.64) | 74 | 0.49 (0.68) | 74 | 0.48 (0.74) | ||||||||
| CSSRId outpatient log | ||||||||||||||
| Treatment | 82 | 1.25 (1.13) | 82 | 1.07 (1.12) | 82 | 0.95 (1.14) | ||||||||
| Control | 74 | 1.53 (1.06) | 74 | 1.19 (1.09) | 74 | 1.15 (1.11) | ||||||||
aWSAS: Work and Social Adjustment Scale.
bMHLS: Mental Health Literacy Scale.
ciPCQ: iMTA Productivity Cost Questionnaire.
dCSSRI: Client Sociodemographic and Service Receipt Inventory.
| Outcome | T1 cases, n (%) | T2 cases, n (%) | T3 cases, n (%) | |
| CSSRIa partly inpatient dichotomized | ||||
| Treatment | 82 (16.3) | 82 (18.3) | 82 (10.7) | |
| Control | 74 (18.6) | 74 (14.9) | 74 (16.8) | |
| CSSRI complementary dichotomized | ||||
| Treatment | 82 (15.9) | 82 (13.7) | 82 (9.3) | |
| Control | 74 (13) | 74 (9.5) | 74 (9.7) | |
aCSSRI: Client Sociodemographic and Service Receipt Inventory.
Additional Outcomes
For the BAI, a significant interaction was found at T3 (t206.8=–4.12, P<.001) with a moderate to large effect size Cohen (d=–0.60 95% CI –0.89 to –0.32). Within-group effects (T1-T3) were large in the treatment group Cohen (d=-0.82 95% CI –1.05 to –0.60) and small in the IG Cohen (d=–0.22 95% CI –0.39 to –0.05).
For the PHQ-9, also a significant interaction was found (t257.4=–3.20, P<.001) with a small to moderate effect size Cohen (d=–0.41 95% CI –0.66 to –0.16). Within-group effects (T1-T3) were small Cohen (d=–0.25 95% CI –0.42 to –0.09) in the IG and showed minimal to small deterioration in the CG Cohen (d=0.15 95% CI –0.03 to 0.34). Regarding adverse effects (NEQ), no difference was found between the groups (t=–1.14, P=.26; Table 7 and OSM 5 in ).
| Negative effects | Intervention group (N=82), n (%) |
| Unpleasant memories resurfaced | 38 (46.3) |
| I had more problems with my sleep | 15 (18.3) |
| I felt like I was under more stress | 17 (20.7) |
| I experienced more unpleasant feelings | 14 (17.1) |
| I felt more worried | 11 (13.4) |
| I experienced more anxiety | 11 (13.4) |
| I felt more dejected | 10 (12.2) |
| I stopped thinking help was possible | 6 (7.3) |
| I lost faith in myself | 6 (7.3) |
| I experienced more hopelessness | 5 (6.1) |
Discussion
Principal Findings
This study compared the effectiveness of an online-based self-help iCBT for patients with panic disorder with or without agoraphobia with the care as usual over 12 weeks. In this trial, a significantly greater reduction in panic and agoraphobia symptoms was found for patients using the online intervention. No effects were found regarding the well-being of the patients. Also, no effects were found regarding the secondary hypotheses, namely daily functioning, work capacity, mental health literacy, and health care-related burdens. In the additional outcomes, significantly greater reductions in anxiety symptoms and depression were identified in favor of the online intervention.
In our study, effect sizes for panic and agoraphobia symptoms were moderate within the groups Cohen (d=0.58) and small to moderate for the interaction effect Cohen (d=0.37). These interaction effects are smaller than the ones reported [] with g=1.04 for panic and g=0.64 for agoraphobia when the treatment length was between 5 to 12 weeks. However, a network meta-analysis did only find small Cohen (d=.21) and not significant effects for unguided iCBT in comparison to care as usual [], highlighting a substantial heterogeneity of results.
The small effect sizes in our study may be attributed to several factors. First, even though randomization occurred, the IG was more severely burdened than the CG with higher anxiety levels in the BAI, higher rates of social phobia (35.8% vs 15.1%), marginal higher rates of generalized anxiety disorder (38.4% vs 23.3%), and marginal higher rates of depression (9.8% vs 1.4%). These additional burdens mostly concern anxiety symptoms and may have diminished treatment effects in the IG. Second, the study was conducted from March 2021 to February 2022, which corresponds to the COVID-19 pandemic. Preliminary evidence suggests that the severity of panic disorder symptoms increased during the COVID-19 pandemic (Cohen d=0.85 during the first wave []). This could be plausible, since respiratory difficulties are common symptoms of panic and SARS-CoV-2, possibly leading to breath-related fear conditioning and associated hypervigilance, which could trigger and exacerbate panic symptoms [] in both the IG and CG. Third, there might have been insufficient treatment adherence in the IG, since on average only 60.8% of the modules were finished and only 32% of participants completed the whole course. Fourth, since treatment effectiveness did not improve in our sensitivity analysis including only patients who had finished at least 4 modules, the intervention may lack effectiveness without guidance. This would not be unexpected, as unguided iCBT for panic disorder was not superior to care as usual in a network meta-analysis [].
In summary, higher illness burden in the IG as well as insufficient adherence may have diminished the differential effects on panic and agoraphobia symptoms as well as well-being. However, without guidance, the intervention itself may not be sufficient to improve well-being in comparison to care as usual. Simultaneously, the COVID-19 pandemic might have lowered effects in both groups, but cannot explain weakened differential effect sizes.
Regarding the secondary hypotheses, several reasons for the lack of effects come to mind. First, this trial was not specifically powered for the secondary outcomes, especially due to the Bonferroni-Holm adjustment. The minimal effect size for 80% power and α=.0125 was calculated at Cohen d=0.46 based on a post-hoc power analysis of the WSAS, so that only moderate to large effects could be detected. Second, the iPCQ and CSSRI subscales had a strong floor effect [] due to the rare occurrence of additional treatments respective lost working hours at baseline. For the iPCQ, 45.8% (71/155) of participants reported zero lost work hours. For the CSSRI-partly inpatient subscale, 83.3% (120/143) reported no inpatient appointments, and for the CSSRI complementary subscale, 86.7% (124/143) reported no complementary treatments at baseline. Third, as discussed above, insufficient adherence to the intervention may also play a role. Fourth, it must also be considered that the program lacks differential effectiveness without guidance, as has been shown in the literature []. In conclusion, the lack of effects on the secondary outcomes could be due to various reasons, but it may also be due to lacking effectiveness without guidance by a therapist.
Implications for Future Research
In the current literature, most studies focus on panic and agoraphobia symptomatology, and there is little to no research on well-being and functioning. Yet, these measures are essential as they provide a more holistic perspective on recovery [,]. Further, as 60% of the financial burden is due to absenteeism [] and additional costs are resulting from treatment, work capacity, and health care use allow an estimation of the costs associated with panic disorder. This current study found only small and nonsignificant effects on well-being Cohen (d=0.22), functioning Cohen (d=0.22), and work capacity Cohen (d=0.21), which may be due to various reasons (eg, effects do not generalize to more distal outcomes, other problems come to the surface after some symptoms are less severe anymore, lack of guidance diminishes the patients’ ability to follow the program). Therefore, studies with higher statistical power are needed for reliable results. Also, modules specifically addressing these outcomes (eg, well-being and work-related problems) could be integrated into iCBT to broaden the effects. In addition, since insufficient adherence could also explain diminished effects in the IG, it may be worthwhile to edit the intervention to make it more engaging. It might also be necessary to integrate some form of guidance into the intervention to empower patients to follow the program more successfully.
There is some evidence for the cost-effectiveness of panic disorder prevention via face-to-face therapy []. Therefore, iCBT seems even more suitable for preventative measures, as it is readily available at disorder onset and cheaper than traditional CBT [].
Another important line of future research should focus on predictors of treatment effects. Although there are some preliminary findings for moderators (alliance [] and impairment []), further research is warranted to enable clinicians to design more effective treatment courses.
Limitations
The study had several limitations. First, a care as usual CG was chosen, to reflect routine care in Germany. However, this only allows conclusions in comparison to German routine care and provides no differential effect sizes to other interventions. Second, this study has some deviations from usual care in Germany, which were assessed via the PRECIS-2 (Providing Regional Climates for Impacts Studies) tool ([]; OSM 7 in ): Since many patients were recruited via university and social media and not from a mental health provider, this study may only partly reflect routine care. This is also indicated by the finding that our sample was predominantly female odds ratio (OR; OR 6.1), with a substantially higher percentage than in a representative sample from the US population (OR 2.0; []). This may not be too surprising since women show generally higher levels of help-seeking [], but it could also be due to the recruitment strategy, which is a deviation from the usual recruitment. Other deviations include the structured diagnostic interview and the 2 follow-up assessments, which may impede external validity. However, other aspects of the trial (eg, eligibility, delivery, and primary hypotheses) were designed with high external validity, altogether indicating a pragmatic (routine care) trial as assessed via PRECIS-2. Third, this analysis lacked follow-up data, which are essential to test the effects’ durability and assess effects with more extended time frames (eg, CSSRI, iPCQ).
However, this study provides evidence for the effectiveness of the online-based intervention in conditions comparable to routine care. The design allowed for a balance between high internal validity due to computer-generated randomization and blinding of the investigators and high external validity owing to inclusive exclusion criteria that only excluded comorbidities that would affect the use of the intervention while allowing for additional support such as psychotherapy and medication. Regarding the internal validity, randomization was mostly successful with similar levels of concurrent psychotherapy treatment and medication, as well as baseline severity (with the exception of the BAI). Further, this trial investigated vital outcomes such as well-being, everyday functioning, work capacity, and health care burdens, providing valuable insights into the intervention’s impact.
Acknowledgments
The authors thank all study participants for their participation and commitment. In addition, we want to thank Leonie Leila Hofmann for her administrative support and assistance with the clinical interviews.
No generative artificial intelligence was involved in the production of this manuscript.
The study was funded by a European Regional Development Fund awarded to Selfapy. The Professorship for Adult Clinical Psychology and Psychotherapy at Osnabrück University was asked by Selfapy to independently evaluate its product to be assessed to the best of our knowledge and following the standards of good scientific practice. Selfapy made suggestions for the selection of survey instruments in the study's design but had no influence on the data collection, analysis, and interpretation or production of the resulting publications.
The study will be funded by a European Regional Development Fund awarded to Selfapy GmbH.
Data Availability
The datasets generated and analyzed during this study are available in the OSF Repository [].
Disclaimer
Web-based self-help interventions can be successfully implemented in routine care and improve symptoms for patients with panic disorder with or without agoraphobia, though effects on more distal outcomes (such as functioning or well-being) have not been shown.
Authors' Contributions
Formal analysis handled by CL, TV, and SH. Funding acquisition conducted by JR. Investigation and software handled by CL. Methodology performed by CL, and JR. Project administration and supervision contributed by JR. Writing–original draft conducted CL, TV, and SH. Writing–review and editing managed by LP, CT, SH, SB, and JR
Conflicts of Interest
None declared.
References
- American Psychiatric Association. Diagnostic and statistical manual of mental disorders: DSM-5. 5th ed. Washington. American Psychiatric Publishing; 2013.
- Asmundson GJG, Taylor S, Smits JAJ. Panic disorder and agoraphobia: an overview and commentary on DSM-5 changes. Depress Anxiety. 2014;31(6):480-486. [CrossRef] [Medline]
- Wittchen HU, Gloster AT, Beesdo-Baum K, Fava GA, Craske MG. Agoraphobia: a review of the diagnostic classificatory position and criteria. Depress Anxiety. 2010;27(2):113-133. [CrossRef] [Medline]
- Thyer BA, Himle J, Curtis GC, Cameron OG, Nesse RM. A comparison of panic disorder and agoraphobia with panic attacks. Compr Psychiatry. 1985;26(2):208-214. [FREE Full text] [CrossRef] [Medline]
- Candilis PJ, McLean RY, Otto MW, Manfro GG, Worthington JJ, Penava SJ, et al. Quality of life in patients with panic disorder. J Nerv Ment Dis. 1999;187(7):429-434. [CrossRef] [Medline]
- Lecrubier Y. The impact of comorbidity on the treatment of panic disorder. J Clin Psychiatry. 1998;59 Suppl 8:11-15. [Medline]
- Simon NM, Otto MW, Korbly NB, Peters PM, Nicolaou DC, Pollack MH. Quality of life in social anxiety disorder compared with panic disorder and the general population. Psychiatr Serv. 2002;53(6):714-718. [CrossRef] [Medline]
- Tietbohl-Santos B, Chiamenti P, Librenza-Garcia D, Cassidy R, Zimerman A, Manfro GG, et al. Risk factors for suicidality in patients with panic disorder: a systematic review and meta-analysis. Neurosci Biobehav Rev. 2019;105:34-38. [CrossRef] [Medline]
- Batelaan N, Smit F, de Graaf R, van Balkom A, Vollebergh W, Beekman A. Economic costs of full-blown and subthreshold panic disorder. J Affect Disord. 2007;104(1-3):127-136. [CrossRef] [Medline]
- Brettschneider C, Bleibler F, Hiller TS, Konnopka A, Breitbart J, Margraf J, et al. Jena-PARADISE Study-Group. The allocation of resources in the care for patients with panic disorder in Germany: an excess cost analysis informing policy and science. Cost Eff Resour Alloc. 2019;17:9. [FREE Full text] [CrossRef] [Medline]
- Papola D, Ostuzzi G, Tedeschi F, Gastaldon C, Purgato M, Del Giovane C, et al. Comparative efficacy and acceptability of psychotherapies for panic disorder with or without agoraphobia: systematic review and network meta-analysis of randomised controlled trials. Br J Psychiatry. 2022;221(3):507-519. [FREE Full text] [CrossRef] [Medline]
- Cuijpers P, Cristea IA, Karyotaki E, Reijnders M, Huibers MJ. How effective are cognitive behavior therapies for major depression and anxiety disorders? A meta-analytic update of the evidence. World Psychiatry. 2016;15(3):245-258. [FREE Full text] [CrossRef] [Medline]
- van Dis EAM, van Veen SC, Hagenaars MA, Batelaan NM, Bockting CLH, van den Heuvel RM, et al. Long-term outcomes of cognitive behavioral therapy for anxiety-related disorders: a systematic review and meta-analysis. JAMA Psychiatry. 2020;77(3):265-273. [FREE Full text] [CrossRef] [Medline]
- Mitte K. A meta-analysis of the efficacy of psycho- and pharmacotherapy in panic disorder with and without agoraphobia. J Affect Disord. 2005;88(1):27-45. [CrossRef] [Medline]
- Otto MW, Deveney C. Cognitive-behavioral therapy and the treatment of panic disorder: efficacy and strategies. J Clin Psychiatry. 2005;66 Suppl 4:28-32. [Medline]
- Heinig I, Wittchen HU, Knappe S. Help-seeking behavior and treatment barriers in anxiety disorders: results from a representative German community survey. Community Ment Health J. 2021;57(8):1505-1517. [FREE Full text] [CrossRef] [Medline]
- Bonabi H, Müller M, Ajdacic-Gross V, Eisele J, Rodgers S, Seifritz E, et al. Mental health literacy, attitudes to help seeking, and perceived need as predictors of mental health service use: a longitudinal study. J Nerv Ment Dis. 2016;204(4):321-324. [FREE Full text] [CrossRef] [Medline]
- Bandelow B, Lichte T, Rudolf S, Wiltink J, Beutel ME. The diagnosis of and treatment recommendations for anxiety disorders. Dtsch Arztebl Int. 2014;111(27-28):473-480. [FREE Full text] [CrossRef] [Medline]
- Karow A. Digitales versorgungsgesetz in deutschland ? Chancen und herausforderungen für die behandlung von menschen mit psychischen erkrankungen. Psychother Psychosom Med Psychol. 2021;71(3-04):105-106. [CrossRef] [Medline]
- Berger T, Caspar F, Richardson R, Kneubühler B, Sutter D, Andersson G. Internet-based treatment of social phobia: a randomized controlled trial comparing unguided with two types of guided self-help. Behav Res Ther. 2011;49(3):158-169. [CrossRef] [Medline]
- Andrews G, Basu A, Cuijpers P, Craske M, McEvoy P, English C, et al. Computer therapy for the anxiety and depression disorders is effective, acceptable and practical health care: an updated meta-analysis. J Anxiety Disord. 2018;55:70-78. [FREE Full text] [CrossRef] [Medline]
- Bergström J, Andersson G, Ljótsson B, Rück C, Andréewitch S, Karlsson A, et al. Internet-versus group-administered cognitive behaviour therapy for panic disorder in a psychiatric setting: a randomised trial. BMC Psychiatry. 2010;10(1):54. [FREE Full text] [CrossRef] [Medline]
- Wartezeit für psychisch kranke Menschen weiterhin zu lang: BPtK zum G-BA-Gutachten zur Bedarfsplanung. Berlin. URL: https://www.bptk.de/wp-content/uploads/2019/01/2018-10-15_pm_gutachten_bedarfsplanung.pdf [accessed 2021-06-25]
- BPtK-Auswertung: Monatelange Wartezeiten bei Psychotherapeut*innen: Corona-Pandemie verschärft das Defizit an Behandlungsplätzen. URL: https://www.bptk.de/wp-content/uploads/2021/03/20210329_pm_bptk_monatelange-Wartezeiten.pdf [accessed 2021-06-25]
- Stech EP, Lim J, Upton EL, Newby JM. Internet-delivered cognitive behavioral therapy for panic disorder with or without agoraphobia: a systematic review and meta-analysis. Cogn Behav Ther. 2020;49(4):270-293. [CrossRef] [Medline]
- Polak M, Tanzer NK, Bauernhofer K, Andersson G. Disorder-specific internet-based cognitive-behavioral therapy in treating panic disorder, comorbid symptoms and improving quality of life: a meta-analytic evaluation of randomized controlled trials. Internet Interv. 2021;24:100364. [FREE Full text] [CrossRef] [Medline]
- Papola D, Ostuzzi G, Tedeschi F, Gastaldon C, Purgato M, Del Giovane C, et al. CBT treatment delivery formats for panic disorder: a systematic review and network meta-analysis of randomised controlled trials. Psychol. Med. 2022;53(3):614-624. [CrossRef]
- German clinical trials register. URL: https://drks.de/search/en/trial/DRKS00023800 [accessed 2024-08-21]
- Rubel J, Quest J, Pruessner L, Timm C, Hartmann S, Barnow S, et al. Evaluation of an online-based self-help intervention for patients with panic disorder - study protocol for a randomized controlled trial. Internet Interv. 2022;30:100584. [FREE Full text] [CrossRef] [Medline]
- Margraf J, Cwik JC, Pflug V, Schneider S. strukturierte klinische interviews zur erfassung psychischer störungen über die lebensspanne. Zeitschrift für Klinische Psychologie und Psychotherapie. 2017;46(3):176-186. [CrossRef]
- Margraf J, Cwik JC, Brachel RV, Suppiger A, Schneider S. DIPS Open Access 1.2: Diagnostisches Interview bei psychischen Störungen. Ruhr-Universität Bochum (RUB). URL: https://omp.ub.rub.de/index.php/RUB/catalog/book/172 [accessed 2021-02-01]
- Suppiger A, In-Albon T, Herren C, Bader K, Schneider S, Margraf J. Reliabilität des diagnostischen interviews bei psychischen störungen (DIPS für DSM-IV-TR) unter klinischen routinebedingungen. Verhaltenstherapie. 2008;18(4):237-244. [CrossRef]
- Hoyer J, Beesdo-Baum K, Becker ES. Ratgeber generalisierte Angststörung: Informationen für Betroffene und Angehörige. 2., aktualisierte Auflage. Göttingen. Hogrefe; 2016.
- Volz HD, Stieglitz RD. Generalisierte Angststörung: Krankheitsbild, Komorbiditäten, Psycho- und Pharmakotherapie. Stuttgart. Schattauer; 2010.
- SoSci Survey. URL: https://soscisurvey.com [accessed 2024-08-21]
- Bandelow B. Panic and Agoraphobia Scale (PAS): Manual. Seattle. Hogrefe & Huber Pub; 1999.
- Hayes AF, Coutts JJ. Use omega rather than cronbach’s alpha for estimating reliability. but. Communication Methods and Measures. 2020;14(1):1-24. [CrossRef]
- Topp CW, Østergaard SD, Søndergaard S, Bech P. The WHO-5 Well-Being Index: a systematic review of the literature. Psychother Psychosom. 2015;84(3):167-176. [FREE Full text] [CrossRef] [Medline]
- Mundt JC, Marks IM, Shear MK, Greist JM. The work and social adjustment scale: a simple measure of impairment in functioning. Br J Psychiatry. 2002;180(05):461-464. [CrossRef] [Medline]
- Bouwmans C, De Jong K, Timman R, Zijlstra-Vlasveld M, Van der Feltz-Cornelis C, Tan Swan S, et al. Feasibility, reliability and validity of a questionnaire on healthcare consumption and productivity loss in patients with a psychiatric disorder (TiC-P). BMC Health Serv Res. 2013;13:217. [CrossRef] [Medline]
- O'Connor M, Casey L. The mental health literacy scale (MHLS): a new scale-based measure of mental health literacy. Psychiatry Res. 2015;229(1-2):511-516. [CrossRef] [Medline]
- Chisholm D, Knapp MRJ, Knudsen HC, Amaddeo F, Gaite L, van Wijngaarden B. Client socio-demographic and service receipt inventory--European version: development of an instrument for international research. EPSILON study 5. European psychiatric services: inputs linked to outcome domains and needs. Br J Psychiatry Suppl. 2000;177(39):s28-s33. [CrossRef] [Medline]
- Rozental A, Kottorp A, Boettcher J, Andersson G, Carlbring P. Negative effects of psychological treatments: an exploratory factor analysis of the negative effects questionnaire for monitoring and reporting adverse and unwanted events. PLoS One. 2016;11(6):e0157503. [FREE Full text] [CrossRef] [Medline]
- Beck AT, Epstein N, Brown G, Steer RA. An inventory for measuring clinical anxiety: Psychometric properties. Journal of Consulting and Clinical Psychology. 1988;56(6):893-897. [CrossRef]
- Kroenke K, Spitzer RL, Williams JBW. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med. 2001;16(9):606-613. [FREE Full text] [CrossRef] [Medline]
- McCall HC, Hadjistavropoulos HD, Sundström CRF. Exploring the role of persuasive design in unguided internet-delivered cognitive behavioral therapy for depression and anxiety among adults: systematic review, meta-analysis, and meta-regression. J Med Internet Res. 2021;23(4):e26939. [FREE Full text] [CrossRef] [Medline]
- Lu K, Luo X, Chen PY. Sample size estimation for repeated measures analysis in randomized clinical trials with missing data. Int J Biostat. 2008;4(1):9. [FREE Full text] [CrossRef] [Medline]
- R foundation for statistical computing. longpower: Power and sample size calculations for longitudinal data. URL: https://cran.r-project.org/web/packages/longpower/longpower.pdf [accessed 2024-09-05]
- Green P, MacLeod CJ. simr: an R package for power analysis of generalised linear mixed models by simulation. Methods Ecol Evol. 2016;7(4):493-498. [CrossRef]
- R. Vienna, Australia: R Foundation for Statistical Computing. URL: https://www.bibsonomy.org/bibtex/7469ffee3b07f9167cf47e7555041ee7 [accessed 2022-10-21]
- Azur MJ, Stuart EA, Frangakis C, Leaf PJ. Multiple imputation by chained equations: what is it and how does it work? Int J Methods Psychiatr Res. 2011;20(1):40-49. [FREE Full text] [CrossRef] [Medline]
- Carpenter JR, Roger JH, Kenward MG. Analysis of longitudinal trials with protocol deviation: a framework for relevant, accessible assumptions, and inference via multiple imputation. J Biopharm Stat. 2013;23(6):1352-1371. [CrossRef] [Medline]
- Westfall J, Kenny DA, Judd CM. Statistical power and optimal design in experiments in which samples of participants respond to samples of stimuli. J Exp Psychol Gen. 2014;143(5):2020-2045. [CrossRef] [Medline]
- Cohen J. Hillsdale NJ, editor. Statistical Power Analysis for the Behavioral Sciences. 2nd Edition. United States. Lawrence Erlbaum Associates, Publishers; 1988.
- Lei Y, Carlson S, Yelland LN, Makrides M, Gibson R, Gajewski BJ. Comparison of dichotomized and distributional approaches in rare event clinical trial design: a fixed bayesian design. J Appl Stat. 2017;44(8):1466-1478. [FREE Full text] [CrossRef] [Medline]
- Lalk C. OSF Repository Panic Trial: Full Data. URL: https://osf.io/xmj28/ [accessed 2023-10-28]
- Rubel J, Väth T, Hanraths S, Pruessner L, Timm C, Hartmann S, et al. Evaluation of an online-based self-help program for patients with generalized anxiety disorder - a randomized controlled trial. Internet Interv. 2024;35:100716. [FREE Full text] [CrossRef] [Medline]
- Jacobson NS, Truax P. Clinical significance : a statistical approach to defining meaningful change in psychotherapy research. In: Kazdin AE, editor. Methodological issues and strategies in clinical research. 3rd print. Washington (D.C.). American Psychological Association; 1993.
- Langhammer T, Peters C, Ertle A, Hilbert K, Lueken U. Impact of COVID-19 pandemic related stressors on patients with anxiety disorders: a cross-sectional study. PLoS One. 2022;17(8):e0272215. [FREE Full text] [CrossRef] [Medline]
- Javelot H, Weiner L. Panic and pandemic: narrative review of the literature on the links and risks of panic disorder as a consequence of the SARS-CoV-2 pandemic. Encephale. 2021;47(1):38-42. [FREE Full text] [CrossRef] [Medline]
- Ho AD, Yu CC. Descriptive statistics for modern test score distributions: skewness, kurtosis, discreteness, and ceiling effects. Educ Psychol Meas. 2015;75(3):365-388. [FREE Full text] [CrossRef] [Medline]
- Gühne U, Pabst A, Löbner M, Breilmann J, Hasan A, Falkai P, et al. Employment status and desire for work in severe mental illness: results from an observational, cross-sectional study. Soc Psychiatry Psychiatr Epidemiol. 2021;56(9):1657-1667. [FREE Full text] [CrossRef] [Medline]
- Slade M. Mental illness and well-being: the central importance of positive psychology and recovery approaches. BMC Health Serv Res. 2010;10:26. [FREE Full text] [CrossRef] [Medline]
- Smit F, Willemse G, Meulenbeek P, Koopmanschap M, van Balkom A, Spinhoven P, et al. Preventing panic disorder: cost-effectiveness analysis alongside a pragmatic randomised trial. Cost Eff Resour Alloc. 2009;7(1):8. [FREE Full text] [CrossRef] [Medline]
- Zalaznik D, Strauss AY, Halaj A, Barzilay S, Fradkin I, Katz BA, et al. Patient alliance with the program predicts treatment outcomes whereas alliance with the therapist predicts adherence in internet-based therapy for panic disorder. Psychother Res. 2021;31(8):1022-1035. [CrossRef] [Medline]
- El Alaoui S, Hedman E, Ljótsson B, Bergström J, Andersson E, Rück C, et al. Predictors and moderators of internet- and group-based cognitive behaviour therapy for panic disorder. PLoS One. 2013;8(11):e79024. [CrossRef] [Medline]
- Loudon K, Treweek S, Sullivan F, Donnan P, Thorpe KE, Zwarenstein M. The PRECIS-2 tool: designing trials that are fit for purpose. BMJ. 2015;350:h2147. [FREE Full text] [CrossRef] [Medline]
- Kessler RC, Chiu WT, Jin R, Ruscio AM, Shear K, Walters EE. The epidemiology of panic attacks, panic disorder, and agoraphobia in the national comorbidity survey replication. Arch Gen Psychiatry. 2006;63(4):415-424. [FREE Full text] [CrossRef] [Medline]
- Kessler RC, Brown RL, Broman CL. Sex differences in psychiatric help-seeking: evidence from four large-scale surveys. Journal of Health and Social Behavior. 1981;22(1):49. [CrossRef]
Abbreviations
| BAI: Beck Anxiety Inventory |
| BOCF: baseline-observation-carried-forward |
| CBT: cognitive behavioral therapy |
| CG: control group |
| CONSORT-EHEALTH: Consolidated Standards of Reporting Trials of Electronic and Mobile Health Applications and Online Telehealth |
| CSSRI: Client Sociodemographic and Service Receipt Inventory |
| DIPS-OA: diagnostisches interview psychischer Störungen – open access |
| DSM: Diagnostic and Statistical Manual of Mental Disorders |
| DSM-IV-TR: Diagnostic and Statistical Manual of Mental Disorders (Fourth Edition, Text Revision) |
| iCBT: internet-based cognitive behavioral therapy |
| IG: intervention group |
| iPCQ: iMTA Productivity Cost Questionnaire |
| J2R: jump to reference imputation |
| LOCF: last-observation-carried-forward |
| MHLS: Mental Health Literacy Scale |
| MICE: multivariate imputation by chained Equations |
| NEQ: Negative Effects Questionnaire |
| OR: odds ratio |
| OSM: online support material |
| PAS: Panic and Agoraphobia Scale |
| PHQ-9: Patient Health Questionnaire-9 |
| PRECIS-2: Providing Regional Climates for Impacts Studies |
| RCI: Reliable Change Index |
| WSAS: Work and Social Adjustment Scale |
| WHO-5: World Health Organization–5 Well-Being Index |
Edited by T de Azevedo Cardoso; submitted 28.10.23; peer-reviewed by P Roberge; comments to author 02.08.24; revised version received 21.08.24; accepted 31.01.25; published 02.04.25.
Copyright©Christopher Lalk, Teresa Väth, Sofie Hanraths, Luise Pruessner, Christina Timm, Steffen Hartmann, Sven Barnow, Julian Rubel. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 02.04.2025.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research (ISSN 1438-8871), is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.