The second Randomised Evaluation of the Effectiveness, cost-effectiveness and Acceptability of Computerised Therapy (REEACT-2) trial: does the provision of telephone support enhance the effectiveness of computer-delivered cognitive behaviour therapy? A randomised controlled trial

Article history

The research reported in this issue of the journal was funded by the HTA programme as project number 06/43/504. The contractual start date was in November 2013. The draft report began editorial review in July 2015 and was accepted for publication in February 2016. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The HTA editors and publisher have tried to ensure the accuracy of the authors’ report and would like to thank the reviewers for their constructive comments on the draft document. However, they do not accept liability for damages or losses arising from material published in this report.

Declared competing interests of authors

David Richards reports grants from University of Exeter during the conduct of the study and is a member of the National Institute for Health Research Career Development Fellowship, Senior Research Fellowship and Transitional Research Fellowship Panel 2013 to the present.

Copyright statement

© Queen’s Printer and Controller of HMSO 2016. This work was produced by Brabyn et al. under the terms of a commissioning contract issued by the Secretary of State for Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science Park, Southampton SO16 7NS, UK.

Depression

Depression is the most common mental health disorder in community settings and is estimated to become the second largest cause of global disability by 2020. 1 It is one of the most common reasons for consulting with a general practitioner (GP) and its associated personal and economic burden is considerable. 2

Psychological therapy for depression

Although antidepressants remain an important treatment option, many patients and health-care professionals would like access to psychological therapy as an alternative or adjunct to drug therapy. 3 Cognitive behaviour therapy (CBT) has emerged as the leading evidence-supported form of brief psychological therapy for people with depression. 4,5 However, demand for CBT cannot be met from existing therapist resources. 6 One promising alternative to therapist-delivered CBT is the use of self-help interventions including the provision of therapy via a computer. 7 In recent years a number of interactive computer programs have been developed that enable CBT to be delivered by a computer. The National Institute for Health and Care Excellence (NICE) guidelines for depression recommend the provision of computerised CBT (cCBT) as an initial, lower-intensity treatment for depression as part of a ‘stepped care’ approach in primary care. 5 If effective, such programmes have the potential to expand the provision of psychological therapy in primary care and may represent an efficient and effective form of care for depression. 8

For those who decide to use (or commission the provision of) cCBT there are a number of interactive internet-based products, some commercially produced and others free to use. 7 In the first category, a number of commercial products have been marketed to bodies such as the NHS. Free-to-use products comprise a range of programs that have been developed by the public sector or by research institutes. These can be accessed at no direct purchase cost to health-care providers. An example of a free-to-use cCBT program is MoodGYM (National Institute for Mental Health Research, Australian National University, Canberra, ACT, Australia), which was developed in Australia and can be accessed by patients with depression, either directly or at the suggestion of their health-care provider.

Evidence for computerised cognitive behaviour therapy

Computerised CBT represents an alternative form of therapy delivery that has the potential to enhance access to psychological care. A number of systematic reviews have been conducted studying the effectiveness of cCBT. An overall beneficial effect of cCBT has been found within trials, although there is a high level of variability in effect size between studies. An early health technology assessment review by Kaltenthaler et al. ,6 published in 2006, noted preliminary evidence of clinical effectiveness and cost-effectiveness. The authors noted the existence of internet-based free-to-use packages, such as MoodGYM, which had been evaluated in randomised trials, but also noted that a major limitation of the existing literature was that the trials had been conducted by the package developers. 8 Since this review there have been few independent evaluations of cCBT packages and the randomised literature remains dominated by developer-led studies.

Later systematic reviews have also highlighted the potential for cCBT to be effective, but have also further demonstrated variable effect sizes and substantial between-study heterogeneity. 9,10 One important source of between-study heterogeneity is the level of support that is made available to people who are offered treatment with cCBT. cCBT involves replacing the therapist with a computer, and requires the person with depression to engage with a self-help computer-based technology. Research by Waller and Gilbody11 has shown that people with depression often do not engage with cCBT, and only a minority actually complete all of the planned sessions of the computer package. This observation is consistent with a broader body of research into the uptake and effectiveness across the range of self-help interventions for depression, including bibliotherapy (self-treatment using written materials). Research in the area of self-help treatments for depression has demonstrated that entirely self-guided materials (with no professional support) are likely to be less effective than self-help technologies for which there is a level of guidance and professional support (‘guided self-help’). Unsupported self-help treatment (including unsupported computer-delivered self-help) has been shown in systematic reviews to have minimal or relatively small effect sizes. In contrast, more intensively and professionally supported treatments have generally been found in efficacy trials to have moderate effect sizes claimed to be comparable to those achieved with face-to-face therapy. To our knowledge the comparative effectiveness of minimally supported cCBT versus more intensively supported cCBT has not been directly tested in large-scale, independently conducted, head-to-head effectiveness trials. Based on indirect estimates drawn from systematic reviews of trials of cCBT the effect is therefore potentially enhanced through the provision of professional support. The magnitude of benefit associated with supported cCBT in groups of patients is, on average, larger than that with unsupported/minimally supported cCBT [pooled effect size for professionally supported therapy, Cohen’s d = 0.61, 95% confidence interval (CI) 0.45 to 0.77 vs. unsupported therapy, Cohen’s d = 0.25, 95% CI 0.14 to 0.35]. 9

Existing Health Technology Assessment programme-funded research into computerised cognitive behaviour therapy for depression: the results of the REEACT trial and the need for further research on the effectiveness of supported computerised cognitive behaviour therapy

On the basis of a UK technology appraisal by Kaltenthaler et al. 6 and the identified need for independent (non-developer-led) research into cCBT, the Randomised Evaluation of the Effectiveness and Acceptability of Computerised Therapy (REEACT) trial was commissioned in 2008, recruited in 2010–12 and reported in 2015. 12 The design of the REEACT trial was to compare the clinical effectiveness and cost-effectiveness of commercially developed cCBT (Beating the Blues; Ultrasis, London, UK) versus free-to-use cCBT (MoodGYM) versus usual GP care. The trial was pragmatic in design and recruited 691 participants in UK primary care. The level of support that was offered to both cCBT packages was designed to replicate current practice in NHS primary care in which professional facilitation is not routinely offered. In view of the findings of systematic reviews, and evidence of lower uptake of packages in entirely self-directed/unsupported therapies, the REEACT trial included technical telephone support. Participants were proactively offered technical support and weekly encouragement to use the computer packages, but we purposely did not augment psychological therapy over the telephone. Telephone support in the REEACT trial did not involve explanations of CBT and did not involve a review of homework or between-session tasks. The cCBT was therefore a form of supported self-help, but was not one that was guided by a clinician. The REEACT trial is, at the time of writing, the largest publicly funded, independently conducted primary care trial of cCBT.

The main finding of the REEACT trial was that for the primary outcome of depression severity at 4 months, there was no significant benefit when participants were offered technically supported cCBT in addition to usual GP care. This negative finding was true for both a free-to-use package (MoodGYM) and commercially produced cCBT (Beating the Blues). The most likely explanatory mechanism of lack of effect was poor uptake and use of computer packages by trial participants. For both cCBT packages (MoodGYM and Beating the Blues) the median number of actual computer therapy sessions that were completed by participants was between 1 and 2. The conclusion of the REEACT trial was that technically supported cCBT was clinically ineffective when added to usual GP care, and that this treatment should not routinely be offered in this form to patients with depression.

Preliminary findings from a concurrent qualitative evaluation in REEACT, and anecdotal evidence from recruiting researchers, indicated that participants were demotivated as a consequence of depression and struggled to engage with computer sessions despite appreciating the offer of technical support. Participants expressed an interest in computer packages, but identified a preference for guidance. We postulated on the basis of these findings and on the basis of emerging trial-based evidence summarised in systematic reviews (e.g. Andersson and Cuijpers9) that cCBT might show an effect but only if offered alongside a greater level of facilitation and support and designed the REEACT-2 study to run alongside, but independently of, REEACT and test this hypothesis. The REEACT-2 trial represents a follow-on trial from the REEACT trial to answer this related question, and most of the fundamental aspects of trial design (primary care setting, recruitment process and inclusion criteria) are replicated in the two trials.

Research objectives

This was a fully randomised patient trial to examine the additional benefits of telephone facilitation alongside a free-to-use computer-delivered CBT package (MoodGYM). The comparator was a minimally supported mode of delivery of the same cCBT package that replicated the mode of delivery of cCBT as offered in primary care in the NHS. The REEACT-2 study included a concurrent economic evaluation to meet the following specific aims:

• to establish the clinical effectiveness of a telephone-facilitated cCBT package compared with minimally supported cCBT over a 1-year trial follow-up period

• to establish the cost-effectiveness of a telephone-facilitated cCBT package compared with minimally supported cCBT over a 1-year trial follow-up period.

Trial design

This study was as a multisite, pragmatic, open, two-arm, parallel-group randomised controlled trial. Participants were recruited from primary care through direct referral by their GP or a postal invitation from GP records. Participants were individually randomised to one of two arms:

1. minimally supported cCBT

2. telephone-facilitated cCBT.

Participants in both arms were given access to a free-to-use cCBT program (MoodGYM), an accompanying booklet, and a Freephone number for technical support, and continued with usual GP care. Participants in the telephone-facilitated cCBT arm were additionally allocated a telephone support worker (TSW) who provided a programme of weekly telephone calls. The programme is described in more detail in Intervention.

Participants were followed up over the course of 12 months, with data collected at 4 and 12 months post randomisation.

Approval

Ethics approval was granted by Bradford Research Ethics Committee (10/H1302/95) on 20 December 2010 and from relevant research and development committees. The trial was assigned the International Standard Randomised Controlled Trial Number (ISRCTN) ISRCTN55310481.

Trial sites

The study was conducted in four UK sites with well-established networks of practices from which to recruit. These were the Universities of York, Bristol, Sheffield and Manchester.

Participants

The study population included patients in primary care with depression or low mood as determined by a score of ≥ 10 on the Patient Health Questionnaire-9 (PHQ-9). 13 This cut-off point is known to detect clinical depression (major depression) in a UK primary care population with a sensitivity of 91.7% and a specificity of 78.3%.

The participants were recruited from a mix of rural and urban GP practices in and around Bristol, Avon, Somerset, Gloucestershire, Manchester, Sheffield and South Yorkshire, York, Humberside and East Yorkshire, Durham, Tyneside and Northumberland.

Participant recruitment

All recruiting researchers were given training in all aspects of the trial including trial recruitment, informed consent, adverse event reporting procedures, and risk assessment and reporting procedures. Each researcher was also given a study reference manual with full instructions on all the standard procedures. Participating GPs were provided with a GP manual with details of the trial processes, a GP information leaflet, adverse event reporting guidance, information about the interventions and contact details for the trial team.

Baseline assessment

Having consented to join the trial, participants completed a series of baseline questionnaires providing biographical, health status, health state utility and service use data. The participant was then randomly allocated according to the process outlined below (Figure 1).

FIGURE 1.

Flow of recruitment to the trial. HCP, health-care professional.

Intervention

In line with the template for intervention description and replication (TIDier)15 guidelines the intervention is described below using the prescribed checklist headings.

Statistical analysis methods

Analyses were performed using SAS version 9.3 (SAS Institute Inc., Cary, NC, USA) on a two-sided basis and using a 5% significance level.

Economic methods

The primary objective of the economic analysis was to assess the relative cost-effectiveness of telephone-facilitated cCBT compared with minimally supported cCBT. The economic analysis was conducted prospectively alongside the REEACT-2 trial.

Economic analysis: statistical methods

Cost-effectiveness analysis

Incremental estimates of total costs and QALYs were obtained through regression methods, adjusting for the following baseline characteristics: age, anxiety level, baseline depression severity, depression duration and sex. The costs and QALY estimates were also adjusted for baseline costs and baseline EQ-5D scores, respectively.

Incremental QALYs were estimated using ordinary least squares (OLS) regression, as this method has been recommended for the estimation of QALYs in economic evaluation. 33 The regression model applied for the incremental analysis of costs in the base case was a generalised linear model. 34 This type of model was preferred to an OLS model, as cost data tend to be heavily skewed and follow a non-normal distribution, which leads to violations of the OLS assumptions. For the analysis of costs, a gamma family distribution was selected. Selection of the family distribution was based on the modified Park’s test35 performed on each imputed data set and complete case data set. An identity link function was selected, thus assuming an additive effect of covariates on costs.

Cost-effectiveness was assessed by comparing the incremental costs and QALYs using standard decision rules. 36 An intervention that generates greater mean QALYs and lower mean costs can be considered dominant and, therefore, a cost-effective use of resources when compared with the alternative. When no dominance arises (i.e. one intervention is more costly and more effective than the other), the interventions can be compared by calculating the ratio between incremental costs and QALYs to establish the incremental cost-effectiveness ratio (ICER). When the ICER between two interventions is below the cost-effectiveness threshold that represents the rate at which health-care activities in the NHS (assumed to be cost-effective) generate health at the margin, then the more costly and more effective intervention is considered cost-effective. The cost-effectiveness of the interventions was assessed by comparing ICERs against a cost-effectiveness interval ranging from £20,000 to £30,000 per QALY, in line with NICE cost-effectiveness thresholds for the UK. 37

Uncertainty surrounding the decision was assessed using a probabilistic sensitivity analysis and presented through cost-effectiveness acceptability curves that graphically represent the probability of an intervention being cost-effective across a range of cost-effectiveness thresholds. The analysis was performed by simulating random draws of incremental mean costs and QALYs (n = 1000) from a multivariate normal distribution and estimating the proportion of those draws that corresponded to a cost-effective use of resources at cost-effectiveness threshold values ranging from £0 to £60,000 per additional QALY. In order to plot the cost-effectiveness acceptability curve, the variance–covariance matrices from the costs and QALYs regressions were extracted and the corresponding Cholesky decompositions calculated to parameterise multivariate normal distributions. 38 This approach is commonly used to ensure that parameters taken from a regression framework are appropriately correlated and not treated as independent when the probabilistic sensitivity analysis is performed (and cost-effectiveness acceptability curves are plotted), but it has the disadvantage of imposing normality on the sampling distribution.

Patient and public participation

Contributors with experience of depression were involved in the design of the trial and the writing of the protocol, and the chief executive of a user-led organisation was a collaborator and coapplicant. Two people with depression read the consent forms and questionnaires and commented on the experience of completing them. All research documentation was designed with patient and public input and the trial oversight committees included members with experience of mental health problems.

Protocol change 1: additional baseline questionnaires

Following the emergence of preliminary findings from the REEACT trial, measures to help identify possible psychosocial mechanisms that affect participants’ capacity or willingness to engage with cCBT were approved and added to the baseline CRF. They were to measure:

• need for affect, a measure of the need for emotionally stimulating experiences, which may underlie a preference for more emotionally engaging material or interaction23

• self-efficacy, a measure of the patient’s confidence that they can overcome typical difficulties to completing cCBT, which may underlie differences in motivation and ability to persist with the programme. 24

Protocol change 2: baseline client report form

On 29 June 2011, the Research Ethics Committee approved the addition of questions about participants’ current use of antidepressant medication. Following modification of the CRFs, participants were asked the following additional questions at baseline:

If you are currently taking any medication to help with your depression please give details:

(please include both prescription medicine and any you have bought yourself)

Name of medication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Dose . . . . . . . . . . . . . . . . . . . . . . .

Name of medication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Dose . . . . . . . . . . . . . . . . . . . . . . .

How long have you been taking the medicine(s)? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Protocol change 3: extension to the study date and sample size

In December 2012, the Research Ethics Committee approval was given to increase the trial sample size to 350 (175 participants per arm) to allow the detection of a smaller difference of 0.30 with 80% power at a level of significance of 0.05 (one sided) accounting for loss to follow-up of 20%.

Protocol change 4: primary care depression cohort

In general, studies of depression in primary care can follow the progress of participants for only a short time and little is known about the effectiveness of treatments in the longer term. Research Ethics Committee approval was granted on 14 November 2012 to invite participants to continue in the study for a further 9 years answering questions about their mood and their general health. This extra follow-up period is not funded as part of the REEACT-2 trial and is not reported here.

Randomisation and centre details

Four main UK sites were responsible for recruiting to the trial: the Universities of Manchester, Bristol, Sheffield and York. Table 2 shows the number of participants recruited to each arm by each site.

A total of 369 participants were randomised to the two-armed comparison of minimally supported cCBT with telephone-facilitated cCBT (n = 182 and n = 187, respectively). The first participant was randomised on 24 June 2011 and the last on 25 April 2013.

The rate of recruitment is shown in Figure 2. The rate had to be slowed after 3 months of rapid recruitment to accommodate the TSW workload. The flow of participants through the trial can be seen in the Consolidated Standards of Reporting Trials (CONSORT) flow chart (Figure 3).

FIGURE 2.

Cumulative recruitment graph for the REEACT-2 trial.

FIGURE 3.

The CONSORT flow diagram.

Baseline data

Primary outcome

Secondary analyses: depressed/not depressed (Patient Health Questionnaire-9) at month 12

Tables 12 and 13 show the results of modelling the odds of being depressed or not depressed at month 12, with a graphical representation in Figure 4. Again the analysis was carried out with all available data and then with the assumption that participants whose data were missing remained depressed as a sensitivity analysis.

TABLE 12 - Depression at 12 months as a discrete outcome

Effect	OR	95% CI	p-value
Telephone-facilitated cCBT vs. minimally supported cCBT	1.626	0.977 to 2.705	0.0615
Male vs. female	1.111	0.651 to 1.897	0.6996
Baseline PHQ-9 score	0.898	0.833 to 0.967	0.0046
Baseline GAD-7 score	0.965	0.904 to 1.031	0.2892
Age (years)	1.004	0.986 to 1.023	0.6764

TABLE 13 - Sensitivity analysis of depression at 12 months as a discrete outcome, assuming that those with missing data were still depressed

Effect	OR	95% CI	p-value
Telephone-facilitated cCBT vs. minimally supported cCBT	1.613	1.051 to 2.474	0.0286
Male vs. female	0.979	0.627 to 1.528	0.9249
Baseline PHQ-9 score	0.930	0.875 to 0.987	0.0177
Baseline GAD-7 score	0.941	0.891 to 0.994	0.0290
Age (years)	1.018	1.002 to 1.034	0.0263

FIGURE 4.

Mean and 95% CI of PHQ-9 score at each assessment.

These results show that telephone-facilitated cCBT was better than minimally supported cCBT for increasing the chances of no longer being depressed after 12 months, but this was not statistically significant (OR 1.63, 95% CI 0.98 to 2.71; p = 0.06). However, in the sensitivity analysis, assuming participants with a missing PHQ-9 score were still depressed, telephone-facilitated cCBT was significantly better (OR 1.61, 95% CI 1.05 to 2.47; p = 0.03).

The OR was 1.61 (95% CI 1.05 to 2.47; p = 0.028) for the main analysis and 1.63 (95% CI 0.98 to 2.71; p = 0.062) for the sensitivity analysis, indicating that adding telephone support to cCBT increased the odds of no longer being depressed by approximately 60%. The baseline PHQ-9 score was also a significant predictor, indicating that a higher baseline score was related to a reduced chance of not being depressed.

Generalised Anxiety Disorder Scale-7 items

The GAD-7 questionnaire was measured at baseline and at 4 and 12 months.

Secondary analyses: Generalised Anxiety Disorder Scale-7 items

The GAD-7 questionnaire was administered to assess self-reported anxiety at baseline and at 4 and 12 months. The mean score decreased in both groups during both follow-up periods.

Table 16 and Figure 5 show the mean GAD-7 scores for each group for the period for the trial. The scores in each group decreased over both time periods.

TABLE 16 - Generalised Anxiety Disorder Scale-7 items score

Time point	Trial arm		Total (n = 369)
	Minimally supported cCBT (n = 182)	Telephone-facilitated cCBT (n = 187)
Baseline
Mean (SD)	14.1 (4.4)	14.5 (4.4)	14.3 (4.4)
Median (range)	14.0 (4.0–21.0)	15.0 (3.0–21.0)	15.0 (3.0–21.0)
Missing	0	0	0
Month 4
Mean (SD)	8.6 (5.4)	7.5 (6.1)	8.1 (5.8)
Median (range)	8.0 (0.0–21.0)	6.0 (0.0–21.0)	7.0 (0.0–21.0)
Missing	65	59	124
Month 12
Mean (SD	8.2 (5.6)	7.0 (5.7)	7.6 (5.7)
Median (range)	7.0 (0.0–21.0)	5.0 (0.0–21.0)	6.0 (0.0–21.0)
Missing	59	56	115

FIGURE 5.

Mean and 95% CI of GAD-7 scores at each assessment.

Results are from a mixed model with repeated measures within participants to allow for within-participant correlation. An unstructured covariance matrix was used. The model analysed GAD-7 scores at months 4 and 12, and adjusted for age, sex, baseline PHQ-9 score, baseline GAD-7 score, time (as a categorical variable) and treatment.

Cohen’s d effect size was calculated as an estimate of mean difference/overall SD.

The results in Tables 17 and 18 show a statistically significant overall effect of treatment (p = 0.037), indicating that over the whole follow-up period the addition of telephone support to cCBT reduced the GAD-7 score by approximately –1.18 (95% CI –2.28 to –0.07). However, when comparing the treatments separately at month 4 and month 12, there was no statistically significant difference between them at either assessment. Baseline GAD-7 and PHQ-9 scores were significant predictors of follow-up GAD-7 score.

TABLE 17 - Repeated measures analysis of GAD-7 scores

Effect	Estimate	95% CI	F-value	p-value
Baseline PHQ-9 score	0.1992	0.0388 to 0.3596	5.98	0.0151
Baseline GAD-7 score	0.4338	0.2917 to 0.5759	36.12	< 0.0001
Age (years)	–0.0358	–0.0750 to 0.0034	3.24	0.0730
Sex	0.5970	–0.5812 to 1.7752	0.99	0.3194
Time	Overall effect		3.34	0.0690
Treatment	Overall effect		4.42	0.0365
Treatment × time interaction	Overall effect		0.02	0.8806

TABLE 18 - Mean differences in GAD-7 score

Effect	Cohen’s d effect size	Estimate	95% CI	t-value	p-value
Telephone-facilitated cCBT vs. minimally supported cCBT (month 4)	0.236	–1.2291	–2.4374 to 0.1425	1.85	0.0659
Telephone-facilitated cCBT vs. minimally supported cCBT (month 12)	0.166	–1.1269	–2.3122 to 0.1676	1.75	0.0819
Telephone-facilitated cCBT vs. minimally supported cCBT (over all assessments)		–1.1780	–2.2813 to –0.0747	2.10	0.0365
Month 4 vs. month 12 (over all treatments)		0.6203	–0.0487 to 1.2894	1.83	0.0690

Patient Health Questionnaire-15

The PHQ-15 was administered to assess somatoform symptoms at baseline and at 4 and 12 months, and descriptive statistics are presented in Table 19 and Figure 6.

FIGURE 6.

Mean and 95% CI of PHQ-15 score at each assessment.

Results are from a mixed model with repeated measures within participants to allow for within-participant correlation. An unstructured covariance matrix was used. The model analysed PHQ-15 scores at months 4 and 12, and adjusted for age, sex, baseline PHQ-15 score, baseline GAD-7 score, time (as a categorical variable) and treatment.

Cohen’s d effect size was calculated as an estimate of mean difference/overall SD.

These results (Tables 20 and 21) show a borderline statistically significant overall effect of treatment (p = 0.051), indicating that over the whole follow-up period the addition of telephone support to cCBT reduced the PHQ-15 score by approximately –1.11 (95% CI –1.75 to 0.002). When comparing the treatments separately at month 4 and month 12, there was no statistically significant difference between them at month 4 but the PHQ-15 score was significantly lower with the addition of telephone facilitation at month 12 (mean difference –1.24, 95% CI –2.27 to –0.21; p = 0.018).

Baseline PHQ-15 score was the only other significant predictor of follow-up PHQ-15 score (see Table 20).

European Quality of Life-5 Dimensions

The EQ-5D is summarised descriptively at each time point showing the number and percentage of participants with each type of response (Tables 22–26).

Computerised cognitive behaviour therapy usage

MoodGYM modules were reported as 25%, 50%, 75% or 100% complete. The percentage of participants in each intervention group who completed all or part of each module is shown in Table 27. The telephone-facilitation programme has increased completion rates for each module (Figure 7).

FIGURE 7.

MoodGYM usage session by session.

Medication to help with depression use

Participants were asked whether or not they were taking medication to help with depression. This included over-the-counter and prescribed medication. Their responses are reported in Table 28.

Early participants in the trial were not asked about current medication; data collection started at participant 1151, so numbers at baseline are smaller. The proportions of participants currently taking medication for depression were similar between the two groups at baseline.

Adverse events

There were a total of 10 serious adverse events (Table 29), none of which was thought to be related to the trial. All adverse events were reviewed by the Trial Steering Committee and the Data Monitoring and Ethics Committee.

Health-care resource use

Table 30 reports the descriptive statistics of health-care resource use by resource category during the trial follow-up.

Full resource-use data were available for all participants at baseline, and this decreased during follow-up to 67% at 4 months and 69% at 12 months, with complete data across the trial period available for 58.1% of participants from minimally supported cCBT (n = 98) and 59.4% from telephone-facilitated cCBT (n = 111).

General practitioners were visited by nearly all participants throughout the trial (95.08%) but were, on average, more frequently attended by those in the minimally supported cCBT group (mean 7.39 visits) than those in the telephone-facilitated cCBT group (mean 6.67 visits). Hospital inpatient services (both mental health and non-mental health related) were used infrequently across each trial arm for both mental health- and non-mental health-related services. Outpatient hospital attendance for non-mental health-related illnesses was common across both minimally supported cCBT (50%) and telephone-facilitated cCBT (44%). Inpatient mental health-related hospital attendances were low across both groups. Results should be interpreted cautiously as these are unadjusted estimates from the available case data set.

Costs

Table 31 reports the mean costs from baseline to 12 months by treatment group. Mean total costs for the 12-month follow-up were £1172 for minimally supported cCBT versus £1763 for telephone-facilitated cCBT. Total costs appeared broadly similar across both trial arms for mental health-related hospital costs, GP costs, primary care costs and medication costs. However, large disparities were seen in non-mental health-related hospital costs when average costs for telephone-facilitated cCBT (£1050.53) were high with large standard errors compared with the average minimally supported cCBT cost (£561.50). These hospital costs appear to be driven by single participants who had higher costs for ongoing treatments not related to depression. The resultant total costs for these individuals could be considered as outliers. The adjusted mean differences and corresponding 95% CIs obtained through regression analysis are reported in Cost-effectiveness analysis.

Health-related quality of life

Table 32 summarises the EQ-5D scores at baseline and at each follow-up period, and total QALYs over the 12-month follow-up. On average, patients in the telephone-facilitated cCBT group reported a slightly lower quality of life than the minimally supported cCBT group at baseline and 4 months, but better quality of life at 12 months.

Mean QALYs estimated through the EQ-5D over 12 months were 0.70 for minimally supported cCBT and 0.686 for telephone-facilitated cCBT. As with the individual EQ-5D scores, the overall QALY difference appears small based on the unadjusted means.

Cost-effectiveness analysis

Base-case cost-effectiveness results

Estimates of incremental mean costs and QALYs based on the regression analysis above were used in the cost-effectiveness analysis. Table 33 presents the fully incremental cost-effectiveness estimates and probability that each intervention is cost-effective at a threshold of £20,000 and £30,000 per QALY. Telephone-facilitated cCBT appears less costly (mean cost difference £3.42) and more effective (mean QALY difference 0.0026) than minimally supported cCBT. At a cost-effectiveness threshold of £20,000 and £30,000 per QALY, the probability that telephone-facilitated cCBT is a cost-effective intervention is 0.55.

TABLE 33 - Cost-effectiveness analysis summary table: base case

Comparing telephone-facilitated cCBT with minimally supported cCBT.

Adjusted for baseline costs, age, anxiety level, baseline depression severity, depression duration and sex.

Adjusted for baseline EQ-5D score, age, anxiety level, baseline depression severity, depression duration and sex.

Treatment	Incremental costsa,b	Incremental QALYsa,c	ICER	Probability cost-effective £20,000 per QALY	Probability cost-effective £30,000 per QALY
Minimally supported cCBT	0	0	Dominated	0.45	0.46
Telephone-facilitated cCBT	–£3.42	0.0026		0.55	0.55

Figure 8 presents the cost-effectiveness acceptability curves for each treatment. The curve indicates the probability that each intervention is the most cost-effective for a range of maximum amounts that the NHS may be willing to pay to gain an additional QALY. The curve illustrates that the probability that telephone-facilitated cCBT is cost-effective initially increases as the amount the NHS is assumed to be willing to pay for additional health gain rises, becoming fairly stable for higher cost-effectiveness thresholds (approximately ≥ £15,000 per QALY).

FIGURE 8.

Cost-effectiveness acceptability curves: base case.

As a sensitivity analysis, an alternative costing scenario was also considered, which included the same cost categories as the base-case analysis with the exception of all non-mental health-related hospital costs. The cost-effectiveness results are reported in Table 34 and Figure 9. In contrast to previous analyses, minimally supported cCBT is no longer dominated by telephone-facilitated cCBT. The ICER for the comparison of telephone-facilitated cCBT versus minimally supported cCBT is £3596 per QALY. Since this is below the lower bound of the currently recommended NICE cost-effectiveness threshold (£20,000 per QALY), minimally supported cCBT remains the cost-effective intervention in this comparison. At a cost-effectiveness threshold of £20,000 per QALY, the probability that telephone-facilitated cCBT is the cost-effective intervention is 0.55. As for the base-case, the probability of cost-effectiveness across thresholds becomes stable at > £15,000 per QALY, with probability of cost-effectiveness for either intervention remaining equal at £20,000 and £30,000 per QALY.

TABLE 34 - Cost-effectiveness analysis summary table: sensitivity analysis excluding non-mental health-related hospital costs

Comparing telephone-facilitated cCBT to minimally supported cCBT.

Adjusted for baseline costs, age, anxiety level, baseline depression severity, depression duration and sex.

Includes primary care (GP and other), and mental health-related hospital and medication costs.

Adjusted for baseline EQ-5D score, age, anxiety level, baseline depression severity, depression duration and sex.

Treatment	Incremental costsa,b,c	Incremental QALYsa,d	ICER	Probability cost-effective £20,000 per QALY	Probability cost-effective £30,000 per QALY
Minimally supported cCBT	0	0	–	0.45	0.45
Telephone-facilitated cCBT	£9.37	0.0026043	£3596.62	0.55	0.55

FIGURE 9.

Cost-effectiveness acceptability curves: sensitivity analysis excluding non-mental health-related hospital costs.

Summary

The within-trial results of the economic analysis suggest that telephone-facilitated cCBT appeared cost-effective compared with the minimally supported cCBT. In the base case, telephone-facilitated cCBT dominated minimally supported cCBT (i.e. lower mean costs and higher QALYs). Although no longer dominant, telephone-facilitated cCBT remained a cost-effective intervention in the sensitivity analyses (i.e. was well within conventional thresholds used to determine value for money in the NHS of £20,000–30,000 per QALY). However, the differences between the groups were relatively minor for both cost and QALY estimates in each of the separate analyses. Although telephone-facilitated cCBT consistently appeared the cost-effective treatment in the base-case and sensitivity analyses, the results from the probabilistic sensitivity analysis revealed high levels of uncertainty.

Trial-based estimates of the clinical effectiveness of telephone-facilitated computer-delivered cognitive behaviour therapy

The REEACT-2 trial found that when telephone facilitation was added to cCBT there were statistically significant benefits in the primary outcome of depression symptomatology and severity across the follow-up period, as measured by a commonly used tool for the identification of depression (PHQ-9). The benefit was most evident at 4 months and the magnitude of benefit was 1.9 points on the PHQ-9 scale, which equated to a small to moderate clinical effect size (Cohen’s d = 0.32). 40 By 12 months the between-group difference was attenuated and was no longer statistically significant. The odds of no longer being depressed (defined as a PHQ-9 score of < 10) at 4 months were increased twofold in the facilitated cCBT group compared with minimally supported cCBT group (OR 2.05, 95% CI 1.23 to 3.42).

Turning to the range of secondary outcomes that were collected in the REEACT-2 trial, there was evidence of statistically significant effects on the overall (including all time points) mean depression scores and anxiety scores (as measured by the GAD-7; between-group difference 1.1, 95% CI 0.1 to 2.3; p = 0.037). For somatoform complaints there was some evidence of a benefit, but the difference was not statistically significant (PHQ-15 between-group difference 1.1, 95% CI 0.0 to 1.8; p = 0.051).

When engagement with cCBT was monitored in the trial with reference to computer records, it was found that there was enhanced uptake and use of programmes. Telephone facilitation, therefore, had the anticipated effect of increasing engagement with computer-based technology. Nevertheless, very few participants (only 19%) completed all five treatment sessions. As was found in the REEACT trial, the use of computer sessions was quite low in the minimally supported treatment arm (with only 45% completing the first session) than in the telephone-facilitated group (65%).

In summary, the main finding is therefore that, for the primary outcome of depression severity and symptomatology, there was a clinically and statistically significant additional benefit across a range of psychological outcomes when participants were offered a telephone-facilitated form of computerised therapy in addition to usual GP care. This benefit was also seen in a range of secondary outcomes. Telephone facilitation resulted in additional clinical improvements when compared with minimally supported cCBT. The comparator arm represented an intervention that replicated cCBT as it is currently offered in routine NHS services, and the additional technology of telephone facilitation was therefore effective in improving clinical outcomes.

Summary of trial-based estimates of cost-effectiveness

The within-trial results of the economic analysis suggest that telephone facilitation resulted in increased quality of life (QALYs) and reduced health-care costs (i.e. it was dominant). In a more conservative sensitivity analysis, the scenario changed and telephone-facilitated cCBT was no longer dominant. However, the additional benefit in terms of QALYs was incurred at an acceptable ratio to costs. In the cost-effectiveness analysis, an ICER of £6933 per additional QALY for telephone facilitation was observed. The addition of telephone facilitation was likely to be cost-effective at a £20,000 per QALY threshold.

Although the cost-effectiveness conclusions appear sensitive to the choice costs, the magnitude of the differences between the two groups was relatively minor for both cost and QALY estimates in both the scenarios. Hence, minor differences in the assumptions can lead to different cost-effectiveness interpretations of either dominance or cost-effectiveness, and some caution should be exercised when interpreting these results. However, under no scenario was telephone facilitation cost-ineffective (> £30,000 per QALY) or dominated by minimally supported cCBT.

Discussion of main findings

The clinical results of the REEACT-2 trial are consistent with some increase in benefit that has been observed in systematic reviews of computer-mediated cCBT. 9 The addition of telephone facilitation based on a manualised support programme enhanced the effectiveness of cCBT. Minimally supported cCBT (such as is used in the NHS at present) was shown in the REEACT trial to be no more effective than usual GP care. In the REEACT-2 trial, a more intensive telephone facilitation was added to cCBT, and this resulted in statistically significant clinical benefit over and above usual GP care. This finding is important for those who deliver or commission psychological services in primary care. cCBT is a commonly advocated first-line low-intensity treatment option in UK primary care and the offer of this treatment without the provision of telephone facilitation is, on average, unlikely to be of benefit to patients. By adding a level of telephone facilitation that is structured and reinforces the content of CBT treatment sessions, engagement with the programme and clinical outcomes are, on average, improved. The magnitude of this benefit was small to moderate, and was broadly in line with other low-intensity psychological interventions for depression that are delivered in primary care. A systematic review and meta-analysis by Cuijpers et al. 41 has demonstrated that the magnitude of effect of primary care-based psychological interventions is, on average, small to moderate (pooled effect size, Cohen’s d = 0.31, 95% CI 0.17 to 0.45) and the results of the REEACT-2 trial are broadly in line with this body of research. 41 In comparison with other estimates of effect size obtained from developer-led trials, the magnitude of effect observed in REEACT-2 is smaller. It should be noted that REEACT-2 is a pragmatic trial, which recruited in primary care rather than specialist cCBT services and was conducted independently of product developers. We would therefore expect that the effect size would be smaller and more representative of the benefits expected under conditions of routine care.

To date there have been only limited cost-effectiveness data relating to psychological therapies generally and low-intensity therapies specifically. 39 In relation to cCBT there are very few economic evaluations, and those that do exist (e.g. Kaltenthaler et al. 6 and Proudfoot et al. 7) have been conducted alongside developer-led trials framed in specialist services and with larger clinical effect sizes than were observed in the REEACT and REEACT-2 trials. The REEACT and REEACT-2 trials represent the largest economic evaluations of cCBT to date, and directly examine the cost-effectiveness of cCBT from the perspective of the UK NHS. The results of the REEACT-2 trial-based economic evaluation indicate that telephone-supported cCBT is either dominant (cost saving and more effective) or cost-effective within an acceptable threshold of willingness to pay. These economic data will be of interest to decision-makers and those charged with the commissioning of services.

An important feature of the REEACT-2 trial is that we evaluated a free-to-use cCBT package that can be accessed by UK NHS patients at no direct cost. There are a number of cCBT products and packages that could be used in the NHS and that could have been trialled within the REEACT-2 trial. The rationale for choosing MoodGYM was threefold. First, there was evidence of effect from developer-led trials,8 suggesting that MoodGYM had the potential to be effective in NHS services. Second, we have shown in the REEACT trial that MoodGYM is not inferior to commercially developed cCBT products. 12 Third, the technology can be accessed at no direct cost to patients in the UK NHS and it had received cautious support in earlier technology appraisals. 6 The results of the REEACT-2 trial are therefore of relevance to decision-makers as investment or commissioning of the use of cCBT will not require purchase of commercially developed products.

There were limitations to the REEACT-2 trial. The first limitation is that we did not obtain a standardised diagnosis of depression at the point of entry to the trial. Instead, we chose a more pragmatic method that judged the presence of symptomatic depression in a more pragmatic way, according to scores on a depression severity scale (the PHQ-9). We also noted that coexisting anxiety and somatoform complaints were common. Although this might be a more heterogeneous population than that seen in efficacy studies, it could be argued that the participants in the REEACT-2 trial were more representative of people with common mental disorders seen in primary care. The level of severity of depression observed with a PHQ-9 cut-off point of ≥ 10 is in line with moderate depression and previous research has shown that this cut-off point is sensitive and specific in identifying people with clinically significant depression. The second limitation is that the level of use of cCBT was quite low and was lower than that observed in developer-led trials. It might be argued that higher levels of uptake and use might have produced larger and more clinically significant effect sizes. However, we have noted that higher levels of uptake have generally been observed in developer-led trials or studies for which there was intensive one-to-one input from a psychological therapist with the therapist present at the time of delivery of the cCBT intervention. 7 We would argue that the REEACT-2 trial represented an evaluation of a feasible model of cCBT and low-intensity support that might more readily be delivered at scale within NHS primary care psychological therapy services. The third limitation is the greater than planned level of loss to follow-up that was observed in the REEACT-2 trial. We observed that 27% of participants were lost to follow-up at 4 months and 26% of participants were lost to follow-up at 12 months. There was some evidence of differential attrition, with levels of attrition between 3% and 5% higher in the minimally supported cCBT group. The levels of loss to follow-up were, however, broadly in line with other primary care-based studies of psychological interventions (e.g. King et al. 42).

Conclusion

Computerised CBT forms a core component of stepped psychological care in the UK primary care and other health systems. We have previously found in the REEACT trial that minimally supported cCBT is clinically ineffective and cost-ineffective and confers no additional benefit over usual GP care for depression. In the REEACT-2 trial, a level of telephone-delivered facilitation was added to cCBT and modest but statistically significant improvements were found in the primary outcome of the presence of depression and other psychological symptoms. Telephone-facilitated cCBT was also cost-effective.

Contributions of authors

Sally Brabyn (Research Fellow) was the trial manager.

Ricardo Araya (Professor of Global Mental Health), Michael Barkham (Professor of Clinical Psychology), Peter Bower (Professor of Health Services Research), Cindy Cooper (Professor of Health Services Research and Clinical Trials), David Kessler (Senior lecturer in Primary Care and General Practitioner), Karina Lovell (Professor of Mental Health), Stephen Palmer (Professor of Health Economics), David Richards (Professor of Mental Health Services) and Simon Gilbody (Professor of Psychological Medicine and Health Services Research) were applicants and contributed to the original protocol and study design.

Ana Duarte (Research Fellow), Richard Mattock (PhD Student) and Simon Walker (Research Fellow) conducted the economic analysis.

Sarah Knowles (Research Fellow), Debbie Tallon (Trial Manager) and David White (Study Co-ordinator) were site trial co-ordinators, and collected and managed data at their sites.

Sally Brabyn and Karina Lovell trained and managed the TSWs.

The report writing team consisted of Sally Brabyn, Ana Duarte, Elizabeth Littlewood (Research Fellow), Richard Mattock, Stephen Palmer, Jodi Pervin (Trial Support Officer), Simon Walker, Gillian Worthy (Statistician) and Simon Gilbody.

Stephen Palmer oversaw the design and conduct of the economic analysis.

Gillian Worthy designed and conducted the clinical analysis.

Simon Gilbody was the chief investigator and chaired the Trial Management Group.

Contributions of collaborators

Helen Lester, Nicola Lidbetter, Mark Sculpher and David Torgerson were coapplicants and contributed to the study design. David Torgerson, Helen Lester and Nicola Lidbetter were members of the Trial Management Group.

Trial Steering Committee members

Professor Joe Reilly (Independent Chairperson), Professor of Mental Health, Durham University.

Professor Mike Slade, Professor of Health Services Research, Head of Section for Recovery, Health Service and Population Research Department, Institute of Psychiatry, King’s College London.

Professor Shirley Reynolds, Professor of Evidence-Based Psychological Therapies, University of Reading.

Professor Robbie Foy, Professor of Primary Care, Leeds Institute of Health Sciences, University of Leeds.

Dr Lina Gega, Reader in Mental Health, Northumbria University.

Mr Nick Seccombe (service user representative), informatics and governance lead, Self Help Services, Manchester. Anxiety UK, cCBT self-help services.

Data Monitoring and Ethics Committee members

Professor Mike Lucock (Independent Chairperson), Professor of Clinical Psychology, University of Huddersfield, and Associate Director of Research, South West Yorkshire Partnership NHS.

Professor Chris Roberts, Professor in Biostatistics, Centre for Biostatistics, University of Manchester.

Dr Paul Blenkiron, Consultant Psychiatrist and Honorary Senior Clinical Lecturer, Hull-York Medical School.

Publication

Gilbody S, Brabyn S, Lovell K, Kessler D, Devlin T, Smith L, et al. REEACT-2: a large scale pragmatic trial of telephone-supported computerised cognitive behaviour therapy. Br J Psychiatry 2016; in press.

Data sharing statement

Available data can be obtained from the data management group through the corresponding author.

Disclaimers

This report presents independent research funded by the National Institute for Health Research (NIHR). The views and opinions expressed by authors in this publication are those of the authors and do not necessarily reflect those of the NHS, the NIHR, NETSCC, the HTA programme or the Department of Health. If there are verbatim quotations included in this publication the views and opinions expressed by the interviewees are those of the interviewees and do not necessarily reflect those of the authors, those of the NHS, the NIHR, NETSCC, the HTA programme or the Department of Health.