This paper is in the following e-collection/theme issue:

Web-based and Mobile Health Interventions (3297)

Published on in Vol 15, No 7 (2013): July

An Internet-Based Guided Self-Help Intervention for Panic Symptoms: Randomized Controlled Trial

An Internet-Based Guided Self-Help Intervention for Panic Symptoms: Randomized Controlled Trial

An Internet-Based Guided Self-Help Intervention for Panic Symptoms: Randomized Controlled Trial

Authors of this article:

Wouter van Ballegooijen1 ;   Heleen Riper1, 2 ;   Britt Klein3, 4, 5 ;   David Daniel Ebert2, 6 ;   Jeannet Kramer7 ;   Peter Meulenbeek8, 9 ;   Pim Cuijpers1
Article Authors Cited by (34) Tweetations (6) Metrics

    Original Paper

    1Department of Clinical Psychology and the EMGO Institute for Health and Care Research, VU University Amsterdam, Amsterdam, Netherlands

    2Innovation Incubator, Leuphana University, Lüneburg, Germany

    3DVC-Research Portfolio & the Faculty of Health Sciences, University of Ballarat, Ballarat, Australia

    4Centre of Mental Health Research, The Australian National University, Canberra, Australia

    5National eTherapy Centre, Faculty of Life and Social Sciences, Swinburne University, Melbourne, Australia

    6Philipps University Marburg, Marburg, Germany

    7Innovation Centre of Mental Health & Technology, Trimbos-institute, Utrecht, Netherlands

    8GGNet (Community Mental Health Centre), Warnsveld, Netherlands

    9Department of Psychology, Health and Technology, University of Twente, Enschede, Netherlands

    Corresponding Author:

    Wouter van Ballegooijen, PhD

    Department of Clinical Psychology and the EMGO Institute for Health and Care Research

    VU University Amsterdam

    Van der Boechorststraat 1

    Amsterdam, 1081 BT

    Netherlands

    Phone: 31 6 24112991

    Fax:31 20 598 8758

    Email: w.van.ballegooijen@vu.nl


    Background: Internet-based guided self-help is efficacious for panic disorder, but it is not known whether such treatment is effective for milder panic symptoms as well.

    Objective: To evaluate the effectiveness of Don’t Panic Online, an Internet-based self-help course for mild panic symptoms, which is based on cognitive behavioral principles and includes guidance by email.

    Methods: A pragmatic randomized controlled trial was conducted. Participants (N=126) were recruited from the general population and randomized to either the intervention group or to a waiting-list control group. Inclusion criteria were a Panic Disorder Severity Scale-Self Report (PDSS-SR) score between 5-15 and no suicide risk. Panic symptom severity was the primary outcome measure; secondary outcome measures were anxiety and depressive symptom severity. Measurements were conducted online and took place at baseline and 12 weeks after baseline (T1). At baseline, diagnoses were obtained by telephone interviews.

    Results: Analyses of covariance (intention-to-treat) showed no significant differences in panic symptom reduction between groups. Completers-only analyses revealed a moderate effect size in favor of the intervention group (Cohen’s d=0.73, P=.01). Only 27% of the intervention group finished lesson 4 or more (out of 6). Nonresponse at T1 was high for the total sample (42.1%). Diagnostic interviews showed that many participants suffered from comorbid depression and anxiety disorders.

    Conclusions: The Internet-based guided self-help course appears to be ineffective for individuals with panic symptoms. However, intervention completers did derive clinical benefits from the intervention.

    Trial Registration: Nederlands Trial Register: NTR1639; http://www.trialregister.nl/trialreg/admin/rctview.asp?TC=1639 (Archived by WebCite at http://www.webcitation.org/6ITZPozs9).

    J Med Internet Res 2013;15(7):e154

    doi:10.2196/jmir.2362

    Keywords

    Internet; self-help; panic disorder; anxiety disorders; patient adherence 


    Panic disorder (PD) with or without agoraphobia is a prevalent anxiety disorder associated with substantial loss of quality of life for the patient and considerable costs to society [1-4]. Subclinical PD, defined as panic symptoms that do not meet full Diagnostic and Statistical Manual of Mental Disorders (Fourth Edition; DSM-IV) criteria for PD, is just as prevalent [2,4]. Subclinical panic symptoms can develop into clinical PD and are also a predictor for the development of mental disorders other than PD, such as generalized anxiety disorder, social phobia, or major depressive disorder (MDD) [5].

    For treatment, PD can be effectively treated with psychological or drug therapy [6-8]. Research indicates that it is also possible to prevent or delay the onset of clinical PD in people with subclinical panic symptoms [9,10]. A recent study showed that a group intervention involving primarily cognitive behavioral therapy effectively reduced symptoms in subclinical cases of PD, as well as in relatively mild cases [10]. This group course, called Don’t Panic, could also be acceptable from a cost-effectiveness point of view [11].

    Internet-based guided self-help has shown to be an efficacious treatment of PD as well, with a large effect size (Hedge’s g=0.83) [12]. To date, all but 1 study [13] comparing Internet-based guided self-help for PD with a control condition have focused purely on groups with clinical PD, which commonly was also the primary diagnosis (eg, [14,15]). These studies excluded subclinical cases (eg, [14-16]). Recently, an Internet-based version of the group course Don’t Panic has been developed. This intervention, Don’t Panic Online, is an Internet-based self-help course with minimal guidance specifically for individuals with mild panic symptom severity. The aim was to provide an accessible, low-intensity, early intervention for panic symptoms.

    The current study is a pragmatic randomized controlled trial (RCT) of the effectiveness of Don’t Panic Online in reducing panic and anxiety symptoms among participants with subclinical and mild clinical PD. We postulate a difference in effect between Don’t Panic Online and a waiting-list control group.


    Design

    We conducted a pragmatic RCT with 2 arms: (1) Internet-based guided self-help, and (2) a waiting-list control group (see subsequent description). The Medical and Ethical Committee of VU University Medical Center approved the study protocol, which is described in greater detail elsewhere [17]. This paper was written in accordance with the CONSORT-EHEALTH checklist [18], and this trial has been registered in the Netherlands Trial Register (NTR1639). The Netherlands Trial Register is part of the Dutch Cochrane Centre.

    Study Population

    We included participants aged 18 and older, with subclinical PD or clinical PD with relatively mild symptom severity, who had access to the Internet. Any individuals who were at risk of suicide were excluded. Subclinical or mild PD was defined as having a score ranging from 5 to 15 on the Panic Disorder Severity Scale-Self Report (PDSS-SR) [19]. These cut-off points represent slight to moderate panic symptom severity [20]. No restrictions were imposed on the use of pharmacotherapy or psychotherapy.

    Sample Size

    Previous RCTs of Internet-based self-help interventions for panic symptoms showed large between-group effect sizes [12]. Our aim was to recruit participants with milder symptom severity than those who took part in these studies. Therefore, our sample was expected to show a smaller decrease in panic symptoms. Based on a moderate effect size (Cohen’s d [d]= 0.50), and using a 2-sided t test (alpha = .05, power 80%) to compare the PDSS-SR scores of the intervention group with those of the control group, we aimed to include 128 participants [21], with 64 in each group. Any missing values at posttreatment were imputed.

    Recruitment

    Participants were recruited from the general population. Most of those who applied for participation did so after reading about this study in the health section of an online newspaper. Additional online recruitment was conducted by means of a Facebook advertising campaign and by posting messages on panic-related or anxiety-related message boards. This was supplemented by offline recruitment by means of advertisements in national newspapers and articles in local newspapers. Interested individuals were directed to a study website, where they could find information about participation and a downloadable informed consent form. The application procedure involved printing and signing the informed consent form, then sending this to the research team (either as a physical document, by conventional mail, or as a scanned document attached to an email).

    Randomization and Procedure

    Consenting applicants were sent an email with a link to the online questionnaires. The baseline (T0) questionnaires included the screening questionnaires for inclusion. Any participants who reported severe panic symptoms or who were at risk of suicide were sent an automatic message advising them to contact their general practitioner and/or to visit a website for suicide prevention. This website [22] offers psychoeducation and a helpline by telephone or online chat [23]. Those participants who had completed T0 and who met the inclusion criteria were contacted within 2 weeks for a diagnostic interview by telephone. This interview was used to obtain a more detailed overview of the study sample, not for the purposes of inclusion or exclusion. After the interview, all participants were randomized to 1 of the 2 groups. Randomization was stratified for the presence or absence of agoraphobic symptoms (PDSS-SR item 4 score ≥2) and the use of antidepressants or sedatives. Randomization lists were generated automatically using a computer program. The T0 measurement can be considered to be double blind because the participants were not randomized until they had completed all of the questionnaires and the diagnostic interview. Blinding of the participants at posttreatment assessment (T1) was not possible because at that stage they were aware of the nature of the group to which they had been allocated. T1 was scheduled 12 weeks after the baseline assessment. Both T0 and T1 were self-reported and were conducted through the Internet. Any participants who had not completed T0 or T1 were sent up to 3 automated reminders by email at weekly intervals.

    Intervention

    Don’t Panic Online is a guided, Internet-based, individual, self-help course, based on cognitive behavioral therapy principles. The course consists of 6 sessions in which the participants learn to control their panic symptoms by applying various cognitive and behavioral techniques and skills. The course’s content is described in more detail elsewhere [17]. A typical lesson takes approximately 30 minutes and consists of an introduction, a discussion of the previous lesson’s homework, new theory, and homework for the following week. A track-and-trace system keeps a record of the dates on which participants log on and complete a lesson. The participants in the intervention group were coached by trained, Master’s-level clinical psychology students. Every week, these participants received an email from their coach, asking how they were doing and whether they were experiencing any difficulty in following the program. The coaches responded to questions about the course and the associated exercises. They also gave brief replies to questions about the participant’s mental health. The coaches were supervised by the first author. On average, the total time spent on each participant was 1 to 2 hours.

    Participants in the control group received access to Don’t Panic Online after completing the T1 measurement (12 weeks after T0). While waiting, they had access to an information website about the symptoms of panic and agoraphobia. This website included advice to contact a general practitioner in case the participant had further questions about panic symptoms and its treatment. All participants in the control group and the intervention group were free to seek any (additional) help they might require.

    Instruments

    The following variables were measured: demographic data, DSM-IV diagnosis, symptoms of anxiety and panic, depressive symptoms, and suicide risk. All variables were measured at both T0 and T1, except for demographic data, diagnosis, and suicide risk, which were only measured at T0.

    The T0 measurement started with demographic questions. These included age, gender, place of birth, marital status, education level, physical health, and previous mental health diagnoses.

    The Composite International Diagnostic Interview (CIDI) 12-month prevalence [24] was used to ascertain the presence or absence of PD, other anxiety disorders, and depression. A clinical diagnosis was made, not as an inclusion criterion, but to gain a more complete overview of the participants. The CIDI, which was developed by the World Health Organization, is an extensive, fully structured, diagnostic interview to assess DSM-IV Axis-I diagnoses [24]. The only subscales used were depression, PD, agoraphobia, generalized anxiety disorder, social phobia, and posttraumatic stress disorder. In this study, a trained interviewer administered the CIDI by telephone.

    The severity of current panic symptoms was measured using the PDSS-SR. The PDSS, which was originally designed as a face-to-face interview for both research and clinical practice [25], was adapted to be used in a patient self-report format [19]. The instrument contains 7 items that assess the severity of 7 dimensions of PD and its associated symptoms. The PDSS-SR generates a total score ranging from 0 to 28. The higher the score, the more severe the panic symptoms. The questionnaire has adequate psychometric properties when compared with the PDSS [19,26]. For the purposes of the current study, a score of less than 5 indicates that there are no clinically significant symptoms, whereas a score of more than 15 is interpreted as severe PD. Therefore, our study focused on the group with scores ranging from 5 to 15. According to the study by Furukawa et al [20], this score range identifies participants with mild to moderate panic symptoms but excludes those without panic symptoms as well as those with severe panic symptoms.

    Anxiety symptoms in general were measured using the Beck Anxiety Inventory (BAI) [27]. The BAI contains 21 short questions. Convergent and divergent validity is sufficient [28,29]. The score ranges from 0 to 63. A score of 30 or more is considered to correspond to severe anxiety symptoms.

    Depressive symptoms were measured using the Center for Epidemiologic Studies Depression scale (CES-D) [30]. The CES-D is a 20-item self-report questionnaire. The score of each individual item ranges from 0 to 3, whereas the total score ranges from 0 (no feelings of depression) to 60 (severe feelings of depression). Convergent validity of the online Dutch version for adults with other depressive measures is good [31]. With a cut-off score of 22 for MDD, it also has good predictive validity [31].

    Suicide risk and suicidal ideation were measured using the specific section of the Mini-International Neuropsychiatric Interview (MINI) [32,33]. The MINI suicide section consists of 6 items and classifies participants into categories ranging from no suicide risk to high suicide risk. Any individuals with a moderate to high suicide risk were excluded from this study. In the current study, these items were administered online and presented as self-report items.

    An indication of health care services usage during the past month was obtained using Part I of the Trimbos and Institute of Medical Technology Assessment Questionnaire on Costs Associated with Psychiatric Illness (TiC-P) [34].

    Finally, the T1 battery of online questionnaires included open questions concerning the participant’s subjective experience with Don’t Panic Online and reasons for not finishing the program. These questions were only administered to the intervention group.

    Analyses

    Firstly, means and standard deviations were calculated for age and symptom severity of panic, anxiety, and depression. Any differences in symptom severity between the intervention group and control group were expressed in terms of Cohen’s d (see subsequent description) to give an indication of the magnitude of the difference in question.

    Between-group effects at T1 were calculated using analyses of covariance (ANCOVA), controlling for pretreatment scores. Instead of F values, t values of parameter estimates are reported because only 2 groups are compared (where t2 = F). Effect sizes on continuous measures were expressed in terms of Cohen’s d, which was calculated by and dividing the mean difference between the 2 mean scores by the pooled standard deviation: (mean1–mean0)/SDPooled. Effect sizes of 0.8 can be assumed to be large, whereas effect sizes of 0.5 are moderate and effect sizes of 0.2 are small [21]. Because Cohen’s d does not take covariance into account, partial η2 is also reported in this paper. It cannot be estimated which level of partial η2 could be considered adequate because this effect size is dependent on several factors. Within-group effects were analyzed using paired-sample t tests and expressed in terms of Cohen’s d in which the correlation between T0 and T1 was taken into account by applying Morris and DeShon’s equation 8 [35]. Finally, the proportion of participants below the PDSS-SR cut-off points for clinical and subclinical PD was calculated for both T0 and T1. We used the cut-off points of 8 and 5, indicating clinical PD [25] and subclinical PD [20], respectively. All analyses were conducted for the full sample, for the subgroup completers, and for subgroups with and without the diagnosis of PD according to the CIDI. We maintained a 2-sided alpha of .05. For all analyses, SPSS version 17 (SPSS Inc, Chicago, IL, USA) was used.

    The data were analyzed in agreement with the intention-to-treat (ITT) principle. Missing data at T1 were imputed by multiple imputation, in which all variables except for nominal variables (ie, age, education level, clinical diagnoses, and symptom severity on all measures at T0 and T1) were included as predictors. Ten datasets were generated and analyses were performed using pooled data. Compared with single imputation methods, multiple imputation generates a more conservative estimate of the sample standard error [36] and overestimation of effect sizes and P values is unlikely. For the purpose of sensitivity analysis, P values and effect sizes were also estimated by running the Expectation Maximization (EM) algorithm [37] on the missing data.


    Sample

    Of 368 applicants who applied and sent in informed consent forms, 126 were included in the study. See Figure 1 for a flowchart and an overview of excluded applicants. The participants were primarily female (85/126, 67.5%), born in the Netherlands (115/126, 91.3%), with a mean age of 36.6 years (SD 11.4, range 18-67), and 50% had a bachelor’s degree or higher (Table 1). Diagnostic interviews showed that 97 (77.0%) of the included participants met the criteria for PD with or without agoraphobia. Other DSM-IV anxiety disorders and MDD were also prevalent (Table 1). Five participants (4.0%) did not meet the criteria for a diagnosis of a mood or anxiety disorder. The control group had slightly higher baseline scores than the intervention group (Table 1), but there were no to little further differences between the intervention group and control group. Details of health care services usage (eg, visits to the general practitioner) are presented in Table 2. Approximately half of the participants reported having consulted a general practitioner in the month immediately prior to the study, and one-third had seen a psychologist or psychiatrist.

    Study Dropout

    The posttreatment measurement was completed by 73 participants (57.9%). There was no significant difference between the measurements and characteristics of these 73 study completers and those of the 53 participants who were lost to follow-up. However, within the intervention group, study dropouts were less likely to have completed lessons 1 to 4 of the course (χ21 = 15.1, P<.001).

    Intervention Adherence

    Of the 63 participants in the intervention group, 60 (95%) started lesson 1, whereas 3 participants did not log in at all (Figure 1). Approximately half of the participants (31/63, 49%) completed lesson 2. Five participants (8%) finished all 6 modules of Don’t Panic Online, 4 (6%) of them within the given 3-month time frame. During the trial, 3 participants (5%) reported that they experienced difficulties accessing the website. Those participants in the intervention group who completed T1 but did not complete the intervention (n=30) were asked why they dropped out. The most frequently reported reasons involved time constraints (n=13), life events (n=5), and symptoms so severe that the individual was unable to follow the program (or parts thereof) or carry out the assignments (n=5; see Table 3).

    Intention-to-Treat Analyses

    After multiple imputation, ANCOVAs showed no significant difference in panic symptom severity at T1 between groups as measured by the PDSS-SR (t = –1.17, P=.25, partial η2 = .023, d=0.30; Table 4). The within-group difference of the intervention group was significant (t=3.06, P=.007, d=0.62), as was the within-group difference of the control group, albeit with a smaller effect size (t=2.26, P=.03, d=0.40). The mean BAI score did not differ between groups (t = –1.71, P=.09, partial η2 = .027, d=0.39; Table 4). Nor were there any differences between groups in terms of depressive symptoms, as measured by the CES-D (t = –1.56, P=.12, partial η2 = .034, d=0.39; Table 4).

    At T1, and with missing values imputed, 24 participants (38%) in the intervention group and 13 (20%) in the control group had PDSS-SR scores of less than 5 (ie, symptom free). This difference did not reach significance (χ2 = 5.7, P=.07). With regard to the cut-off point of 8 (the recommended cut-off for clinical diagnosis), 28 participants (44%) in the intervention group and 22 (35%) in the control group scored below 8 at T0. At T1, 38 participants in the intervention group (60%) and 33 participants in the control group (52%) scored below 8, a nonsignificant difference (χ2 = 1.3, P=.43).

    Sensitivity analyses with the EM algorithm gave slightly different results. There was no significant effect between groups on the primary outcome measure (PDSS-SR: t124 = –1.79, P=.08, partial η2 = .025, d=0.34), but the difference in BAI anxiety symptoms did reach significance (t124 = –2.33, P=.02) with a moderate effect size (d=0.46, partial η2 = .042). CES-D depressive symptoms also differed between groups (t124 = –2.69, P=.008) with a moderate effect size (d=0.47, partial η2 = .055).

    Table 1. Baseline characteristics of participants.
    CharacteristicsTotal sample N=126Intervention group n=63Control group n=63Difference at baseline (Cohen’s d)
    Demographics




    		Age, mean (SD)36.6 (11.4)36.7 (12.2)36.4 (10.7)

    		Female, n (%)85 (67.5)44 (69.8)41 (65.1)

    		Born in the Netherlands, n (%)115 (91.3)57 (90.5)58 (92.1)

    		Living alone, n (%)50 (39.7)23 (36.5)27 (42.9)

    		High education,a n (%)63 (50.0)30 (47.6)33 (52.4)

    		Physical health problems, n (%)9 (7.1)5 (7.9)4 (6.3)

    		Previously diagnosed with a mental disorder, n (%)47 (37.3)22 (34.9)25 (39.7)
    Diagnoses,bn (%)c




    		PD with agoraphobia61 (49.2)30 (47.6)31 (49.2)

    		PD without agoraphobia36 (29.0)17 (27.0)19 (30.2)

    		Agoraphobia without PD17 (13.7)10 (15.9)7 (11.1)

    		GAD11 (8.9)5 (7.9)6 (9.5)

    		Social phobia78 (62.9)39 (61.9)39 (61.9)

    		PTSD16 (12.9)4 (6.3)12 (19.0)

    		MDD53 (42.7)27 (42.9)26 (41.3)
    Symptom severity, mean (SD)




    		Panic (PDSS-SR)8.9 (3.0)8.8 (3.2)9.1 (2.8)0.12

    		Anxiety (BAI)24.9 (10.8)23.7 (10.2)26.0 (11.3)0.22

    		Depression (CES-D)20.8 (9.0)20.0 (9.1)21.6 (9.0)0.18

    aDefined as the equivalent of a bachelor’s degree or higher.

    bPercentages add up to more than 100% due to comorbid diagnoses.

    cMissing data of 2 participants (n=124).

    Table 2. Use of care in the past month.
    Care useT0, n (%)T1,a n (%)

    		Intervention group (n=63)Control group (n=63)Intervention group (n=16)Control group (n=39)
    Visited general practitioner27 (43%)31 (49%)2 (13%)12 (31%)
    Visited psychologist or psychiatrist23 (37%)17 (27%)5 (31%)14 (36%)
    Visited other professional health care giver18 (29%)25 (40%)3 (19%)14 (36%)
    Used antidepressants, sedatives, or sleeping pills20 (32%)23 (37%)7 (44%)13 (33%)

    aDifferences within groups and between groups did not reach significance.

    ‎
    Figure 1. Flow of participants through the study.
    View this figure
    Table 3. Reasons why participants did not finish Don’t Panic Online within 12 weeks (n=30).
    Reason for discontinuationna
    Time constraint (too busy or need more time)13
    Life events (pregnancy, loss, family issues)5
    Symptoms too severe to do assignments5
    Found other therapy4
    Content not applicable3
    Spontaneous recovery2
    Adverse effect1
    More guidance needed1
    Lack of structure1
    Lessons too slow1
    Not motivated1

    aNumbers do not add up to 30 because 2 participants did not give reasons and others gave several.

    Table 4. Differences between groups at T1, intention-to-treat (N=126).
    MeasureGroup, mean (SD)aBetween-groups effecta

    		Intervention group (n=63)Control group (n=63)Cohen’s d (95% CI)ANCOVAb




    		tcPPartial η2
    PDSS-SR5.8 (4.9)7.3 (4.9)0.30 (–0.91, 1.51)–1.17.25.023
    BAI17.0 (12.7)22.0 (12.7)0.39 (–2.74, 3.53)–1.71.09.027
    CES-D16.4 (12.3)21.1 (12.1)0.39 (–2.59, 3.42)–1.56.12.034

    aMissing data imputed by multiple imputation.

    bControlling for symptom severity at T0.

    cDegrees of freedom not provided due to multiple imputation.

    Completers-Only Analyses

    Those participants in the intervention group who had completed the first 4 lessons (or more) of the course (n=17) were included in the completers-only analyses. These completers cannot be all considered to have completed the intervention, but after 4 lessons, participants can be considered to have experienced most of the content of the intervention. Sixteen of the 17 participants in the intervention group who had completed the first 4 lessons also filled in T1 questionnaires. Accordingly, there were 16 completers in the intervention group. These 16 individuals did not significantly differ from the noncompleters in the intervention group at T0 in terms of age, education, clinical diagnosis, and symptom severity. Control group completers were those who filled in T1 (n=39).

    The ANCOVA showed significant differences between the intervention group completers and control group completers with regard to panic symptom severity at T1 (t53 = –2.60, P=.01, d=0.73; see Table 5), in favor of the intervention group. The intervention group was also characterized by a large within-group effect on panic symptoms (t15 = 4.92, P<.001, d=1.23). In the control group, within-group effects did not reach significance. ANCOVA also showed that BAI anxiety symptom severity differed significantly between groups (t53 = –2.37, P=.02, d=0.60, see Table 5), as did depressive symptom severity, as measured using the CES-D (t53 = –2.52, P=.02, d=.94).

    Ten (68%) of the intervention completers and 8 (21%) of the control group completers had a PDSS-SR score of less than 5 at T1, which is a significant difference (χ21 = 9.1, P=.003). In terms of the cut-off point for clinical diagnosis, 13 participants in the intervention group (81%) and 23 (59%) in the control group scored less than 8, but this difference did not reach statistical significance (χ21 = 2.5, P=.12).

    Lastly, health care service usage rates did not differ either within or between groups (see Table 2).

    Participants With Diagnosis of Panic Disorder Versus Those Without Diagnosis

    Neither ITT nor completers-only analyses showed differences on any outcome measure between participants with and without clinical PD.

    Table 5. Differences between groups at T1, completersa (n=55).
    MeasureGroup, mean (SD)Between-groups effect

    		Intervention group (n=16)Control group (n=39)Cohen’s d (95% CI)ANCOVAb




    		t53PPartial η2
    PDSS-SR4.6 (3.3)7.5 (4.2)0.73 (–0.60, 2.32)–2.60.01.115
    BAI15.6 (13.4)22.6 (11.2)0.60 (–2.93, 7.15)–2.37.02.098
    CES-D12.1 (8.5)21.6 (11.0)0.94 (–2.50, 5.10)–2.52.02.109

    aControl group completers are those who provided posttreatment data. Intervention group completers are those who provided posttreatment data and completed at least lesson 4.

    bControlling for symptom severity at T0.


    Overview

    This study showed that the Internet-based, guided, self-help intervention Don’t Panic Online was not effective in individuals with panic symptoms. Completers-only analyses did show moderate to large effect sizes between groups in favor of the intervention group. Adherence to the treatment was low. An analysis of the data using a less conservative imputation method revealed significant effects between groups in terms of the scores for general anxiety and depressive symptoms, but not for panic symptoms. Overall, the results show that Don’t Panic Online could be efficacious for intervention completers, but that it is not generally effective.

    Comparison With the Literature

    A meta-analysis revealed that the psychological treatment (offline and online) of full-blown PD is highly effective compared to a waiting-list control group, with a mean effect size of d=1.19 [8]. Samples in which more than 50% of the participants had comorbid disorders did not benefit as much, but they still showed a large effect size even when compared with pooled active and nonactive control groups (d=0.83) [8]. Self-help interventions have an average effect size of d=0.75, again when compared with pooled control groups [8]. The results of our completers-only analyses are in-line with these findings. Treatment adherence is not reported in this meta-analysis, only study dropout rates, which averaged 9.53% for intervention groups.

    For study design and intervention, our study is comparable with the trial of Meulenbeek et al [10]. That study found a moderate effect size of d=0.68 for the face-to-face group course Don’t Panic, an intervention with similar content to Don’t Panic Online. Treatment completion, defined as having followed at least 6 of the 8 sessions, was 75%. In that study, the participants had a relatively low baseline mean PDSS-SR score (7.2), which is similar to our study’s findings. Aside from panic symptoms, however, the sample differed from ours in a number of ways. Meulenbeek et al excluded participants with severe disorders other than PD, as well as those with social problems, and those who were receiving treatment for panic symptoms. In general, group interventions are no more effective than guided self-help interventions [38]. Possibly, any differences in outcome between the trials of Don’t Panic and Don’t Panic Online might be attributed to inclusion criteria.

    Previous studies that compared Internet-based guided self-help for panic symptoms with a control group showed an overall effect size of Hedge’s g=0.83 [12]. Similar to Don’t Panic Online, the interventions studied were based on cognitive behavioral therapy and were similar in length [14-16]. Compared with these studies, effect sizes in the current study were expected a priori to be lower. We included a less severe group, thereby ruling out large decreases in symptom severity. Accordingly, assuming that there was no deterioration in the control group, the difference between the intervention group and control group at T1 could not be as large. With regard to low treatment adherence, this was not found in previous studies and values ranged from 79% to 95% [12,15].

    There are several differences between our study and previous studies that may have had an impact on adherence. Firstly, all participants in our trial were free to use medication and find other treatment. Some may have found other help and decided to quit Don’t Panic Online. Secondly, our participants reported difficulties accessing the website. Thirdly, previous researchers had more telephone contact with their participants [14,15]. Our participants were also not interviewed after the treatment, whereas a scheduled interview after treatment may have led to better adherence [39]. Fourthly, the intervention we studied was not the same as the interventions of other studies. Perhaps Don’t Panic Online is not as effective or attractive as those examined in other studies. Lastly, our sample included a large proportion of participants with comorbid disorders, and possibly a proportion of participants who did not have PD as primary diagnosis. Perhaps an Internet-based intervention specifically for panic symptoms is less suited to this group. However, epidemiological data show that panic symptoms often coincide with psychiatric disorders other than PD [2,4]. Therefore, the participants of our study appear to be a representative sample of individuals with panic symptoms.

    In summary, both clinical effect and treatment adherence were lower in our study than in previous studies of Internet-based self-help interventions and the Don’t Panic group course. The differences in sample characteristics between our study and previous trials could indicate that Internet-based interventions for panic symptoms are efficacious, but they may not be effective for all individuals seeking help for panic symptoms.

    Limitations

    When interpreting our results, several limitations should be taken into account. One limitation of this study is nonresponse at the posttreatment measurement. For a large proportion of participants, it is unknown whether their panic symptoms increased, decreased, or remained stable. These missing values were estimated by multiple imputation. Although this can be considered a conservative imputation method, it is unlikely that the imputed values greatly underestimate the intervention effect. This is because many of the participants who did not respond at T1 also left the intervention after 1 or 2 sessions, and are, therefore, unlikely to have gained much benefit from it. A second limitation is that the intervention completers are small in number and may not be representative of the intervention group as a whole, even though there did not appear to be significant differences between completers and noncompleters. The comparison of this select group with the control group, for completers-only analyses, should be interpreted with caution. Thirdly, the control group could have had gained some benefit from the information website, which could have decreased the difference between T1 mean scores of the intervention group and control group. If that is the case, our study proved that Don’t Panic Online has, in general, no added value compared with an information website and our conclusion would remain the same. A fourth limitation is the lack of a follow-up measurement. It is not known whether the participants in either the intervention group or the control group showed any further improvement over the subsequent months to a year. Finally, all continuous measures were obtained by online self-report. The PDSS-SR could potentially yield lower mean scores than the PDSS interview [26], whereas online versions of questionnaires could potentially yield higher mean scores than pencil-and-paper versions [40,41]. These differences in psychometric properties limit the comparison of this study with other studies. However, this imposes no restrictions on comparisons between the intervention group and control group within our own study and, additionally, online and pencil-and-paper versions of panic questionnaires do appear to be equivalent [42,43].

    Implications and Future Research

    Although previous research indicates that Internet interventions can be an efficacious treatment of panic symptoms, our results may suggest that a linear program targeting specific symptoms is not always effective. As our study and others have shown, panic symptoms generally coincide with comorbid symptoms. Therefore, transdiagnostic and tailorable interventions could be a future direction of Internet-based treatment of panic. Internet-based transdiagnostic self-help programs, tailored to the anxiety and/or depressive symptoms of the participant, show promising results in terms of the treatment of panic and other common mental disorders [13,44,45]. Tailored interventions could be more effective for individuals with higher symptom severity and comorbidity rates than nontailored programs [46]. Tailoring might help to increase treatment adherence because participants would then only see those sections that are applicable to them. Given the results of our study, the further development of transdiagnostic and tailorable Internet interventions should be encouraged.

    Future research could focus on identifying those groups for whom Internet-based self-help interventions are effective, for example, by means of predictor and mediator analyses. Further research is also needed to investigate ways of boosting treatment adherence to Don’t Panic Online, of making it a feasible intervention for mild to moderate panic symptoms, and perhaps of modifying it to become more tailored and transdiagnostic in nature. This was the first study of Internet-based guided self-help for mild panic symptoms and our study needs to be replicated before we can draw any definitive conclusions. Lastly, although the efficacy of Internet-based guided self-help interventions has been established in several studies, it should be encouraged to conduct more pragmatic RCTs to examine the effectiveness.

    Acknowledgments

    This study is funded by the Trimbos-institute and VU University Amsterdam.

    Conflicts of Interest

    None declared.

    ‎

    Multimedia Appendix 1

    CONSORT-EHEALTH checklist V1.6.2 [18].

    PDF File (Adobe PDF File), 995KB
    1. Taylor CB. Panic disorder. BMJ 2006 Apr 22;332(7547):951-955 [FREE Full text] [CrossRef] [Medline]
    2. Batelaan N, De Graaf R, Van Balkom A, Vollebergh W, Beekman A. Thresholds for health and thresholds for illness: panic disorder versus subthreshold panic disorder. Psychol Med 2007 Feb;37(2):247-256. [CrossRef] [Medline]
    3. Batelaan N, Smit F, de Graaf R, van Balkom A, Vollebergh W, Beekman A. Economic costs of full-blown and subthreshold panic disorder. J Affect Disord 2007 Dec;104(1-3):127-136. [CrossRef] [Medline]
    4. Bystritsky A, Kerwin L, Niv N, Natoli JL, Abrahami N, Klap R, et al. Clinical and subthreshold panic disorder. Depress Anxiety 2010 Apr;27(4):381-389 [FREE Full text] [CrossRef] [Medline]
    5. Kinley DJ, Walker JR, Enns MW, Sareen J. Panic attacks as a risk for later psychopathology: results from a nationally representative survey. Depress Anxiety 2011 May;28(5):412-419. [CrossRef] [Medline]
    6. Furukawa TA, Watanabe N, Churchill R. Psychotherapy plus antidepressant for panic disorder with or without agoraphobia: systematic review. Br J Psychiatry 2006 Apr;188:305-312 [FREE Full text] [CrossRef] [Medline]
    7. Schmidt NB, Keough ME. Treatment of panic. Annu Rev Clin Psychol 2010;6:241-256. [CrossRef] [Medline]
    8. Sánchez-Meca J, Rosa-Alcázar AI, Marín-Martínez F, Gómez-Conesa A. Psychological treatment of panic disorder with or without agoraphobia: a meta-analysis. Clin Psychol Rev 2010 Feb;30(1):37-50. [CrossRef] [Medline]
    9. Gardenswartz CA, Craske MG. Prevention of panic disorder. Behavior Therapy 2001 Sep;32(4):725-737. [CrossRef]
    10. Meulenbeek P, Willemse G, Smit F, van Balkom A, Spinhoven P, Cuijpers P. Early intervention in panic: pragmatic randomised controlled trial. Br J Psychiatry 2010 Apr;196(4):326-331 [FREE Full text] [CrossRef] [Medline]
    11. Smit F, Willemse G, Meulenbeek P, Koopmanschap M, van Balkom A, Spinhoven P, et al. Preventing panic disorder: cost-effectiveness analysis alongside a pragmatic randomised trial. Cost Eff Resour Alloc 2009;7:8 [FREE Full text] [CrossRef] [Medline]
    12. Andrews G, Cuijpers P, Craske MG, McEvoy P, Titov N. Computer therapy for the anxiety and depressive disorders is effective, acceptable and practical health care: a meta-analysis. PLoS One 2010;5(10):e13196 [FREE Full text] [CrossRef] [Medline]
    13. Silfvernagel K, Carlbring P, Kabo J, Edström S, Eriksson J, Månson L, et al. Individually tailored internet-based treatment for young adults and adults with panic attacks: randomized controlled trial. J Med Internet Res 2012;14(3):e65 [FREE Full text] [CrossRef] [Medline]
    14. Carlbring P, Bohman S, Brunt S, Buhrman M, Westling BE, Ekselius L, et al. Remote treatment of panic disorder: a randomized trial of internet-based cognitive behavior therapy supplemented with telephone calls. Am J Psychiatry 2006 Dec;163(12):2119-2125. [CrossRef] [Medline]
    15. Klein B, Richards JC, Austin DW. Efficacy of internet therapy for panic disorder. J Behav Ther Exp Psychiatry 2006 Sep;37(3):213-238. [CrossRef] [Medline]
    16. Wims E, Titov N, Andrews G, Choi I. Clinician-assisted Internet-based treatment is effective for panic: A randomized controlled trial. Aust N Z J Psychiatry 2010 Jul;44(7):599-607. [CrossRef] [Medline]
    17. van Ballegooijen W, Riper H, van Straten A, Kramer J, Conijn B, Cuijpers P. The effects of an Internet based self-help course for reducing panic symptoms--Don't Panic Online: study protocol for a randomised controlled trial. Trials 2011;12:75 [FREE Full text] [CrossRef] [Medline]
    18. Eysenbach G, CONSORT-EHEALTH Group. CONSORT-EHEALTH: improving and standardizing evaluation reports of Web-based and mobile health interventions. J Med Internet Res 2011;13(4):e126 [FREE Full text] [CrossRef] [Medline]
    19. Houck PR, Spiegel DA, Shear MK, Rucci P. Reliability of the self-report version of the panic disorder severity scale. Depress Anxiety 2002;15(4):183-185. [CrossRef] [Medline]
    20. Furukawa TA, Katherine Shear M, Barlow DH, Gorman JM, Woods SW, Money R, et al. Evidence-based guidelines for interpretation of the Panic Disorder Severity Scale. Depress Anxiety 2009;26(10):922-929 [FREE Full text] [CrossRef] [Medline]
    21. Cohen J. Statistical Power Analysis for the Behavioral Sciences. Hillsdale, NJ: Lawrence Erlbaum Associates; 1987.
    22. 113 online.   URL: http://www.113online.nl/ [accessed 2013-07-23] [WebCite Cache]
    23. Mokkenstorm JK, Huisman A, Kerkhof AJ. [Suicide prevention via the internet and the telephone: 113Online]. Tijdschr Psychiatr 2012;54(4):341-348 [FREE Full text] [Medline]
    24. Robins LN, Wing J, Wittchen HU, Helzer JE, Babor TF, Burke J, et al. The Composite International Diagnostic Interview. An epidemiologic Instrument suitable for use in conjunction with different diagnostic systems and in different cultures. Arch Gen Psychiatry 1988 Dec;45(12):1069-1077. [Medline]
    25. Shear MK, Rucci P, Williams J, Frank E, Grochocinski V, Vander Bilt J, et al. Reliability and validity of the Panic Disorder Severity Scale: replication and extension. J Psychiatr Res 2001;35(5):293-296. [Medline]
    26. Wuyek LA, Antony MM, McCabe RE. Psychometric properties of the panic disorder severity scale: clinician-administered and self-report versions. Clin Psychol Psychother 2011;18(3):234-243. [CrossRef] [Medline]
    27. Beck AT, Epstein N, Brown G, Steer RA. An inventory for measuring clinical anxiety: psychometric properties. J Consult Clin Psychol 1988 Dec;56(6):893-897. [Medline]
    28. de Beurs E, Wilson KA, Chambless DL, Goldstein AJ, Feske U. Convergent and divergent validity of the Beck Anxiety Inventory for patients with panic disorder and agoraphobia. Depress Anxiety 1997;6(4):140-146. [Medline]
    29. Lykke J, Hesse M, Austin SF, Oestrich I. Validity of the BPRS, the BDI and the BAI in dual diagnosis patients. Addict Behav 2008 Feb;33(2):292-300. [CrossRef] [Medline]
    30. Radloff LS. The CES-D Scale: A self-report depression scale for research in the general population. Applied Psychological Measurement 1977 Jun 01;1(3):385-401. [CrossRef]
    31. Donker T, van Straten A, Marks I, Cuijpers P. Brief self-rated screening for depression on the Internet. J Affect Disord 2010 May;122(3):253-259. [CrossRef] [Medline]
    32. Lecrubier Y, Sheehan D, Weiller E, Amorim P, Bonora I, Harnett Sheehan K, et al. The Mini International Neuropsychiatric Interview (MINI). A short diagnostic structured interview: reliability and validity according to the CIDI. European Psychiatry 1997;12(5):224-231. [CrossRef]
    33. Sheehan DV, Lecrubier Y, Sheehan KH, Amorim P, Janavs J, Weiller E, et al. The Mini-International Neuropsychiatric Interview (M.I.N.I.): the development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. J Clin Psychiatry 1998;59 Suppl 20:22-33;quiz 34. [Medline]
    34. Hakkaart-van Roijen L, van Straten A, Donker M, Tiemens B. Handleiding Trimbos/iMTA questionnaire for Costs associated with Psychiatric illness (TiC-P). Rotterdam: Insituut voor Medische Technology Assessment, Erasmus MC; 2002.
    35. Morris SB, DeShon RP. Combining effect size estimates in meta-analysis with repeated measures and independent-groups designs. Psychol Methods 2002 Mar;7(1):105-125. [Medline]
    36. Donders AR, van der Heijden GJ, Stijnen T, Moons KG. Review: a gentle introduction to imputation of missing values. J Clin Epidemiol 2006 Oct;59(10):1087-1091. [CrossRef] [Medline]
    37. Little RJA, Rubin DB. Statistical Analysis with Missing Data. Hoboken, NJ: Wiley; 2002.
    38. Cuijpers P, Donker T, van Straten A, Li J, Andersson G. Is guided self-help as effective as face-to-face psychotherapy for depression and anxiety disorders? A systematic review and meta-analysis of comparative outcome studies. Psychol Med 2010 Dec;40(12):1943-1957. [CrossRef] [Medline]
    39. Nordin S, Carlbring P, Cuijpers P, Andersson G. Expanding the limits of bibliotherapy for panic disorder: randomized trial of self-help without support but with a clear deadline. Behav Ther 2010 Sep;41(3):267-276. [CrossRef] [Medline]
    40. Cuijpers P, Boluijt P, van Straten A. Screening of depression in adolescents through the Internet: sensitivity and specificity of two screening questionnaires. Eur Child Adolesc Psychiatry 2008 Feb;17(1):32-38. [CrossRef] [Medline]
    41. Donker T, van Straten A, Marks I, Cuijpers P. Quick and easy self-rating of Generalized Anxiety Disorder: validity of the Dutch web-based GAD-7, GAD-2 and GAD-SI. Psychiatry Res 2011 Jun 30;188(1):58-64. [CrossRef] [Medline]
    42. Austin DW, Carlbring P, Richards JC, Andersson G. Internet administration of three commonly used questionnaires in panic research: equivalence to paper administration in Australian and Swedish samples of people with panic disorder. International Journal of Testing 2006 Mar;6(1):25-39. [CrossRef]
    43. Carlbring P, Brunt S, Bohman S, Austin D, Richards J, Öst L, et al. Internet vs. paper and pencil administration of questionnaires commonly used in panic/agoraphobia research. Computers in Human Behavior 2007 May;23(3):1421-1434. [CrossRef]
    44. Carlbring P, Maurin L, Törngren C, Linna E, Eriksson T, Sparthan E, et al. Individually-tailored, Internet-based treatment for anxiety disorders: A randomized controlled trial. Behav Res Ther 2011 Jan;49(1):18-24. [CrossRef] [Medline]
    45. Titov N, Andrews G, Johnston L, Robinson E, Spence J. Transdiagnostic Internet treatment for anxiety disorders: A randomized controlled trial. Behav Res Ther 2010 Sep;48(9):890-899. [CrossRef] [Medline]
    46. Johansson R, Sjöberg E, Sjögren M, Johnsson E, Carlbring P, Andersson T, et al. Tailored vs. standardized internet-based cognitive behavior therapy for depression and comorbid symptoms: a randomized controlled trial. PLoS One 2012;7(5):e36905 [FREE Full text] [CrossRef] [Medline]

    ‎
    ANCOVA: analysis of covariance
    BAI: Beck Anxiety Inventory
    CIDI: Composite International Diagnostic Interview
    EM: Expectation Maximization
    ITT: intention-to-treat
    MDD: major depressive disorder
    MINI: Mini-International Neuropsychiatric Interview
    PD: panic disorder
    PDSS-SR: Panic Disorder Severity Scale-Self Report
    RCT: randomized controlled trial
    T0: baseline
    T1: posttreatment assessment (12 weeks after T0)

    Edited by G Eysenbach; submitted 24.10.12; peer-reviewed by J Hoffman; comments to author 13.11.12; revised version received 26.01.13; accepted 11.05.13; published 29.07.13

    Copyright

    ©Wouter van Ballegooijen, Heleen Riper, Britt Klein, David Daniel Ebert, Jeannet Kramer, Peter Meulenbeek, Pim Cuijpers. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 29.07.2013.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.