A randomised controlled trial of computerised cognitive behaviour therapy for the treatment of depression in primary care: The Randomised Evaluation of the Effectiveness and Acceptability of Computerised Therapy (REEACT) trial

Article history

The research reported in this issue of the journal was funded by the HTA programme as project number 06/43/05. The contractual start date was in May 2009. The draft report began editorial review in January 2014 and was accepted for publication in September 2014. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The HTA editors and publisher have tried to ensure the accuracy of the authors’ report and would like to thank the reviewers for their constructive comments on the draft document. However, they do not accept liability for damages or losses arising from material published in this report.

Declared competing interests of authors

Simon Gilbody is a Health Technology Assessment Clinical Evaluation and Trials Board Member. Michael Barkham is a developer of the Clinical Outcomes in Routine Evaluation – Outcome Measure, which was used as a secondary outcome measure in the trial. Peter Bower reports personal fees from paid consultancy for the British Association of Counselling and Psychotherapy, outside the submitted work. Karina Lovell is a Non-Executive Director for Manchester Mental Health and Social Care Trust and is paid a salary.

Copyright statement

© Queen’s Printer and Controller of HMSO 2015. This work was produced by Littlewood et al. under the terms of a commissioning contract issued by the Secretary of State for Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science Park, Southampton SO16 7NS, UK.

Depression

Depression is the most common mental health disorder in community settings and is estimated to become the second largest cause of global disability by 2020. 1 It is one of the most common reasons for consulting a general practitioner (GP), and is associated with significant personal and economic burden. 2

Psychological therapy for depression

Antidepressant medication is an important treatment option for depression; however, many patients and health-care professionals would like to access psychological therapy as an alternative or adjunct to medication. 3 A leading evidence-supported form of brief psychological therapy for people with depression is cognitive behaviour therapy (CBT). 4,5 However, given that patient demand for CBT cannot be met from existing therapist resources,6 there is a need to increase patient access to psychological therapy. One potential way of achieving this might be the provision of CBT delivered via computer. 7 In recent years, a number of interactive programs have been developed which enable CBT to be delivered by computer. The provision of computerised CBT (cCBT) is now recommended in the National Institute for Health and Care Excellence (NICE) guidelines as an initial lower-intensity treatment for depression as part of a ‘stepped care’ approach in primary care. 5 cCBT, if shown to be effective, has the potential to expand the provision of psychological therapy in primary care and, as such, may represent an efficient and effective form of care for depression. 8

There are a number of interactive internet-based products that are available for those who decide to use (or commission the provision of) cCBT. Some of these cCBT products are commercially produced whereas others are free to use. 7 A number of commercial products have been marketed to bodies such as the NHS and have also been made available for patients to purchase directly. The free-to-use products have been developed by the public sector or by research institutes and can be accessed at no direct purchase cost to patients or health-care providers, although there may be costs associated with their support and use in the NHS.

Evidence for computerised cognitive behaviour therapy

Computerised CBT represents an alternative form of therapy delivery that has the potential to enhance access to psychological care. Existing research into cCBT has been summarised by Kaltenthaler and colleagues in their 2006 Health Technology Assessment (HTA) review of clinical effectiveness and cost-effectiveness. 6 With respect to depression, three commercially produced computerised packages available to the NHS were considered: Beating the Blues^® (Ultrasis, London, UK), COPE and Overcoming Depression. Of these, only one, Beating the Blues, had been evaluated in a randomised controlled trial (RCT); the program was shown to be effective at reducing symptoms of depression. 7 However, this research was conducted by those who owned and held the intellectual copyright to Beating the Blues. Among internet-based free-to-use packages, only one, MoodGYM (National Institute for Mental Health Research, Australian National University, Canberra, Australia), has been evaluated in a randomised trial, which was also conducted by the package developers. 8 MoodGYM was found to be effective at reducing depressive symptomatology. 8 The overall conclusion of the HTA review was that ‘the efficacy but not effectiveness of Beating the Blues had been established in comparison with treatment as usual’. 6

Recommendations for further research on the effectiveness of computerised cognitive behaviour therapy

At the time of the commissioning and design of the REEACT (Randomised Evaluation of the Effectiveness and Acceptability of Computerised Therapy) trial, several caveats applied and specific recommendations for further research were made that are important with respect to the present trial.

• Computerised CBT had been shown to be effective in the short term, but trials rarely examined the longer-term impact on depression compared with usual care. Trials with longer periods of follow-up are needed.

• The cost-effectiveness of computerised packages is as yet unknown. More importantly, the cost-effectiveness from the perspective of the UK NHS has not been sufficiently established and the longer-term cost-effectiveness beyond the brief time horizon of existing trials is essentially unknown. This is important, as commercial packages (such as Beating the Blues) will need to be purchased by the NHS.

• Existing trials are based on populations who have been referred to specialist cCBT services and are necessarily comfortable with information technology and willing to be offered computerised therapy as a treatment option. Computer-delivered CBT uses a computer rather than a trained CBT therapist, and the acceptability of the replacement of the therapist with a machine interface is largely unknown. Patients with depression might show a strong preference for or against computer therapy, and this might, in turn, be related to uptake and effectiveness. The acceptability and effectiveness of cCBT among patients who are representative of people treated for depression in NHS primary care services has not yet been established. There is, therefore, a need for pragmatic evaluations of cCBT based in UK primary care that examine real-world effectiveness and the issue of patient preference.

• There are no trials of free-to-use cCBT packages versus pay-to-use cCBT packages. This is important, as the effectiveness of free-to-use cCBT would need to be comparable to pay-to-use CBT if it were to be a viable alternative within a stepped care pathway. 9

• Evaluations of all the commercially available and free-to-use packages of cCBT have been conducted by researchers responsible for their development. Although this does not invalidate the results, it does raise concerns that a truly independent evaluation of the clinical effectiveness and cost-effectiveness of cCBT is needed to inform NHS decision-making. In their 2006 HTA report, Kaltenthaler and colleagues make this a core research recommendation and state that ‘Research needs to be carried out by independent researchers. It should be carried out by those who are not associated with commercial or product gains’. 6

The present trial was designed to address these recommendations. A subsequent meta-analysis has demonstrated that cCBT can be effective for depression, but there remains a need for longer-term pragmatic studies and evaluations by researchers other than the product developers. 10

The REEACT study represents a pragmatic evaluation of cCBT in a trial that is adequately controlled and has appropriate statistical power.

Research objectives

This was a fully randomised patient preference trial of usual GP care for depression versus the addition of one of two cCBT packages to usual GP care. The REEACT study included a concurrent economic and qualitative evaluation to meet the following specific aims:

• to establish the clinical effectiveness and cost-effectiveness of the addition of cCBT to usual GP care compared with usual GP care alone over a 2-year trial follow-up period

• to establish the acceptability (to patients and health professionals) of cCBT

• to establish the differential clinical effectiveness and cost-effectiveness of a free-to-use computerised package, in comparison with a commercial pay-to-use cCBT package over a 2-year and longer-term time horizon.

Some of this text has been reproduced from Gilbody et al. 11 © BMJ 2015. This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/.

Trial design

The REEACT trial was a pragmatic, multicentre, open, three-armed parallel RCT with simple randomisation. The design included a fully randomised patient preference approach. 12 Participants with depression [defined as a score of ≥ 10 on the Patient Health Questionnaire-9 (PHQ-9) depression severity instrument]13 were randomised (1 : 1 : 1) to receive:

• a commercial pay-to-use cCBT program (Beating the Blues) plus usual GP care or

• a free-to-use cCBT program (MoodGYM) plus usual GP care or

• usual GP care alone.

Approvals obtained

The Leeds (East) Research Ethics Committee (REC) approved the study on 10 July 2008 and approved the substantial amendment to revise the trial design following advice from the funder on 22 September 2008 (see Chapter 3 for details of this substantial amendment). The details of the REC and Research and Development Department approvals are provided in Appendix 1. The trial was registered as ISRCTN91947481; EudraCT number 2007-007645-12; and UK Clinical Research Network Study Portfolio (UKCRN) identification number 4115.

Trial sites

The trial was conducted in nine UK sites. Five sites were involved from the commencement of the trial, and the remaining four sites were recruited throughout the duration of the trial. Details of the study sites are provided in Appendix 2.

Participant eligibility

People with depression were eligible to take part in the trial. Both prevalent and incident cases of depression were included.

Recruitment into the trial

All researchers participating in the study received training in all aspects of the trial including trial recruitment, eligibility criteria, trial protocol, adverse event reporting procedures, participant risk assessment and reporting procedures, trial database and trial documentation. Each researcher received a researcher manual detailing these procedures in order to standardise the running of the study across researchers and trial sites. All researchers also undertook Good Clinical Practice training.

Potential participants were referred to the trial through GP practices. All GP practices were provided with a GP practice manual. This provided details of key trial contacts and site information, information about cCBT and the trial intervention cCBT programs, a GP information sheet (see Appendix 3), and procedures and forms for recruiting participants via the various recruitment routes (described in Recruitment routes) and for reporting serious adverse events (SAEs). The manual also contained copies of trial documentation including the trial protocol, participant information sheet and patient consent form (see Appendix 4). GP practices were also provided with participant study information packs containing a study invitation/cover letter, a participant information sheet and a permission form for release of personal details (see Appendix 4).

In line with recommendations from a HTA-funded primary care depression trial by Peveler and colleagues,16 GP practices were reimbursed for the additional time involved in recruitment of patients to the trial via service support costs. An additional GP practice incentive was the use of the Quality and Outcomes Framework (QOF). 17 As the PHQ-9 was a validated assessment measure incentivised for the Depression 2 indicator in the QOF during the trial’s recruitment period, all GPs were provided with participants’ PHQ-9 scores at recruitment.

Baseline assessment

After written informed consent (parts 1 and 2) had been obtained, baseline data were collected on participants prior to randomisation using several self-report baseline questionnaires. The following data were collected.

Randomisation

Participants were randomised 1 : 1 : 1 between the three trial arms: cCBT (Beating the Blues) plus usual GP care; cCBT (MoodGYM) plus usual GP care; and usual GP care alone. In view of the large number of participants to be recruited to the study, stratification by depression severity was not required. Randomisation was simple and, therefore, not restricted in any way. Simple randomisation reduces the risk of any possible subversion associated with restricted randomisation methods and has been shown to produce equally precise results28 as stratified randomisation, which improves treatment precision in trials with fewer than 50 participants. However, simple randomisation can lead to unequal group sizes by chance alone. To maintain allocation concealment, the generation of the randomisation sequence and subsequent treatment allocation were performed by an independent, secure, remote telephone or web-based randomisation service. The computerised randomisation sequence was checked periodically during the trial following standard operating procedures. Owing to the nature of the trial design and the intervention, it was not possible to conceal treatment allocation from either the participant or the REEACT researcher.

Sample size

We needed to know if cCBT represented a clinically effective addition to usual GP care and whether or not the free-to-use computerised program (MoodGYM) represented a less effective (non-inferior) choice of therapy for patients compared with the commercial pay-to-use computerised program (Beating the Blues). We therefore powered our trial both to capture any benefit of cCBT over usual GP care alone and to test the non-inferiority of free-to-use cCBT. We based our sample size calculation on the usual care arm of our own primary care trial of collaborative care for depression, where the proportion of patients responding to usual care was in the region of 0.6;29 a response rate similar to that found in a UK HTA trial of antidepressants in primary care16 and a US pragmatic depression trial. 30 We regarded a response rate of not more than 0.15 below this rate as being acceptable, given the additional care options that are available to patients who do not initially respond to cCBT within a stepped care framework. Our original sample size calculation of 600 participants (200 participants in each of the three arms) gave us in excess of 80% power to detect non-inferiority using the percentage success in both groups as 60%, with a non-inferiority margin of 15% and allowing for 25% attrition. However, the sample size was re-estimated as, despite better levels of recruitment than anticipated, levels of attrition were slightly higher than originally expected (see Chapter 3). In order to retain similar levels of power to detect non-inferiority between the free-to-use and commercial cCBT packages with 5% probability, while allowing for 35% attrition, 690 participants were recruited (230 participants in each of the three arms). The trial also retained 80% power to detect a difference of 15% between the usual GP care arm and either of the two cCBT arms using a conventional power analysis (α = 0.05; two-sided significance).

Trial interventions

Participants were randomised to receive a commercial pay-to-use cCBT program (Beating the Blues) plus usual GP care, a free-to-use cCBT program (MoodGYM) plus usual GP care, or usual GP care alone.

Participant follow-up

Participants were followed up at three time points following randomisation: 4 months, 12 months and 24 months. During the first 6 months of the trial follow-up period, participants completed their follow-up questionnaires with a REEACT researcher during face-to-face interviews or telephone interviews, according to participant preference. In order to improve follow-up rates, participants were later given the additional option of completing and returning follow-up questionnaires in the post (see Chapter 3). The participant information sheet was revised to reflect this additional method of follow-up completion, and participants already recruited to the trial were advised of this option and received a copy of the revised participant information sheet at their next scheduled follow-up. As a further means of improving our retention rates, participants received an unconditional £5 voucher with their follow-up as an acknowledgement for their taking part in the study and completing the questionnaires. The £5 voucher either accompanied their appointment letter for face-to-face or telephone interviews, or was sent along with postal questionnaires.

Trial completion

Participants were deemed to have exited the trial when:

• The participant had completed the 24-month follow-up.

• The participant wished to exit the trial fully.

• The participant’s GP withdrew him or her from the trial.

• The participant died.

Instead of withdrawing fully from the trial, participants had the option of:

• withdrawing only from receiving the trial intervention (applicable only to participants randomised to receive either of the two cCBT programs).

Participants who elected to withdraw from the trial intervention (cCBT programs) and follow-ups were deemed to be full withdrawals (trial exit). REEACT researchers were able to indicate any change in the patient’s level of participation by registering this information on the trial database, which immediately notified the co-ordinating centre (University of York) of this change. REEACT researchers notified the patient’s GP when a participant elected to withdraw fully from the trial.

Measurement and verification of primary outcome

The primary outcome measure was depression severity and symptomatology as measured by the PHQ-913 questionnaire at 4 months. Participants completed the primary outcome measure during face-to-face interviews or telephone interviews with REEACT researchers, or as a postal questionnaire, according to their preference.

The PHQ-9 is a nine-item self-report questionnaire which records the core symptoms of depression based on the Diagnostic and Statistical Manual of Mental Disorders diagnostic criteria for major depressive disorder. 32 Each item is rated on a scale of 0 to 3 based on the frequency of depressive symptoms (0 = ‘not at all’ to 3 = ‘nearly every day’). Scores range from 0 to 27, with higher scores indicating a greater degree of depression severity. It has the added advantage that it can be reliably administered over the telephone. 33

There are extensive US and non-US validation and sensitivity to change data using the PHQ-9. It has been validated in a UK primary care population14 and has become the instrument of choice in the management of depression in UK primary care, including monitoring symptom change,34,35 and in the fulfilment of QOF routine depression measurement. 36 It has also been shown to be sensitive to change over time. 37,38

Measurement and verification of secondary outcomes

A variety of disease-specific and generic outcome measures were administered to ascertain clinical effectiveness. Cost-effectiveness was determined by obtaining measures of HRQoL and resource utilisation using self-report questionnaires, along with objective data collection from participants’ GP medical records.

Outcome measures completed by participants were administered at each follow-up – 4 months, 12 months and 24 months – and were completed during face-to-face interviews or telephone interviews with REEACT researchers, or as a postal questionnaire, according to participant preference. Objective data from participants’ GP medical records were obtained after participants had completed their 24-month follow-up.

The following data were collected.

Adverse events

An adverse event was defined as ‘any undesirable clinical occurrence in a subject, whether it is considered to be caused by or related to treatment or not’.

Adverse events were identified during participant recruitment and follow-up by REEACT researchers or at any point during the participant’s involvement in the trial by the participant’s GP. In addition, participants completed a Health Events Questionnaire at each follow-up (see Appendix 7). This asked participants if they had experienced any health problems since their last follow-up, with particular reference to any problems or events that may be related to their depression. Participants were asked to describe their health problems, including when the problem or event happened. Adverse events were also identified via the adapted CSRI questionnaire, which was administered at each follow-up and asked participants about their use of inpatient hospital services during the previous 6 months.

Adverse events were categorised as serious or non-serious. A serious adverse event (SAE) was defined as an event that resulted in death, was life-threatening, required inpatient hospitalisation or prolonging of existing hospitalisation, resulted in persistent or significant disability or incapacity, resulted in a congenital anomaly or birth defect or was deemed medically significant by the reporter.

Adverse events that were felt to be serious were reported to the trial co-ordinating team either by REEACT researchers or by the participant’s GP. The reporter was asked to complete a ‘serious adverse event/reaction form’ (see Appendix 7) for all those adverse events he or she judged to be serious, indicating why, in the reporter’s opinion, the event was considered to be serious and the relationship of the adverse event to treatment. Reporters were asked to report any SAEs to the trial co-ordinating team within 24 hours of the event occurring, or within 24 hours of becoming aware of the event. All adverse events judged to be serious were reviewed by at least two members of the Trial Management Group. SAEs felt to be related to treatment in some way were reviewed by the Data and Ethics Monitoring Committee (DMEC) and the Trial Steering Committee (TSC). SAEs that were felt to be unrelated to treatment were reviewed by the DMEC and TSC at the next scheduled meeting.

The trial co-ordinating team regularly reviewed participant outputs reported on the Health Events Questionnaire. A SAE/reaction form was completed for any reported health problems/events considered to be serious and/or related to treatment. Health problems/events reported by participants that were felt to be non-serious by the trial co-ordinating team were reviewed by the DMEC and the TSC at the next scheduled meeting.

Statistical analysis of the REEACT clinical effectiveness data

All analyses were conducted on an intention-to-treat basis, including all participants in the groups to which they were randomised. For superiority comparisons, two-sided significance tests at the 5% significance level were used. Analyses were conducted in Stata version 13 (StataCorp LP, College Station, TX, USA).

Economic analysis of the REEACT cost-effectiveness data

Qualitative study of the acceptability to patients and health professionals of computerised therapy

A concurrent qualitative study was undertaken to examine the acceptability and experiences of patients and health professionals in relation to computerised therapy.

Patient and public participation

The REEACT trial received patient and public participation input at the design, conduct, analysis and interpretation stages. We received input from those with lived experience of depression in the design of all research materials. The REEACT trial received oversight via committee membership from members with lived experience of common mental health problems and a representative of a user-led self-help organisation. A user-led organisation (Anxiety UK and Self-Help Services via its Chief Executive) acted as co-applicant and collaborator on the REEACT trial.

Some of this text has been reproduced from Gilbody et al. 11 © BMJ 2015. This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/.

Research objectives/trial design

In the original protocol the primary research objective was to establish the clinical effectiveness and cost-effectiveness of two cCBT programs – Beating the Blues and MoodGYM – versus antidepressant medication over a 2-year follow-up period. All participants were to receive usual GP care, with the key comparison being in the selection of initial treatment for depression, where the comparison would be between antidepressants and a non-drug alternative in the form of cCBT. Participants who were randomised to either of the two cCBT programs would receive computerised therapy rather than medication in the first instance, with the addition of drug treatment to cCBT if the participant’s GP felt that the patient was failing to respond to computerised therapy. Participants randomised to the control group – usual GP care alone – would be prescribed antidepressants by their GP. Although no specific patient-level recommendations were to be made about which medication to prescribe or the dosage, GPs were asked to follow the NICE recommendations on the prescription of antidepressants15 and the British National Formulary dosage recommendations. 55

Following ethical approval of our original protocol (10 July 2008), we received notification from the Medicines and Healthcare products Regulatory Agency that our design qualified our trial as a clinical trial of an investigational medicinal product. We sought advice from our funders, who advised against using this trial design and recommended that all participants receive ‘care as usual’ and that participants’ medication use should not in any way be altered by participating in the trial.

In light of this advice, the basic design of the trial was revised. The revised design maintained the original structure of a three-armed trial but with slight changes to the content of each arm. The control arm was revised to be usual GP care. The two intervention arms were amended to include the addition of usual GP care to one of two cCBT programs (usual GP care plus Beating the Blues and usual GP care plus MoodGYM). The revised trial design now meant that all participants would receive usual GP care. Usual GP care might include antidepressant medication but also might not. As a result, the original exclusion criterion that participants already in receipt of antidepressants would be ineligible to participate in the trial was removed. This change in trial design meant our primary research objective was now to establish the clinical effectiveness and cost-effectiveness of the addition of cCBT to usual GP care over a 2-year follow-up period.

On the basis that all participants would be receiving care as usual, the trial was no longer classified as a clinical trial of an investigational medicinal product but as a non-interventional trial. We received ethical approval (22 September 2008) for this revised trial design before the study commenced.

Outcome measures

Telephone support calls

During early trial meetings (and prior to the first participant being randomised) we decided to introduce regular telephone support calls to those participants who were randomised to use either one of the two cCBT programs in order to ensure uptake and to be in line with good clinical practice. These telephone calls were undertaken by telephone support staff who would deal with any technical problems the participants might experience and would try to encourage them to engage with the cCBT programs. The telephone calls were not intended to offer any form of psychotherapy. This protocol change was implemented following information that cCBT interventions are usually implemented in practice with this kind of support, and so this change allowed us to reflect current practice and maintain the pragmatic design of the trial.

In addition, the protocol was adapted to include supervision of the telephone support staff. Telephone support staff were required to work to specific and standardised instructions on what should be covered during each telephone call to ensure the delivery of ‘support’ rather than ‘therapy’ and to provide the telephone support staff with the skills needed to respond to the common questions that would inevitably arise during the telephone calls. Therefore, as a means of ensuring standardisation and quality control, we gained ethical approval to record the telephone calls. All participants were asked for their informed consent to record each telephone call and participants were not excluded from the trial if they withheld their consent to the recordings. All telephone support staff were aware that telephone calls were being recorded where participant consent was given. Regular supervision with the chief investigator or other senior study co-investigators was provided to the telephone support staff.

These protocol changes were implemented before the first participant was randomised.

Exclusion criteria

It was decided to add two additional exclusion criteria when considering potential trial participants. It was felt by the study investigators that patients who have a primary diagnosis of alcohol or drug abuse and patients who are not able to read and write in English would be highly unlikely to benefit from cCBT. This protocol change was implemented before the first participant was randomised.

Trial referrals

In our original protocol, direct referrals to the trial could be made only by a patient’s GP. Practice nurses could identify potential participants with depression but would need to advise the patient to make an appointment with his or her GP and notify the GP of the patient’s potential eligibility for the trial. However, during the first 3 months of recruitment we received numerous requests from those health professionals attached to a GP practice (primary care-based mental health workers and practice-attached nurses) participating in REEACT to allow them to directly refer patients to the trial. Following discussions with GPs, practice-attached nurses and members of the primary care mental health team, we decided it was reasonable to allow the latter two groups of staff to check participant eligibility and refer directly to the trial, with the understanding that the referrer would need to inform the patient’s GP of the trial referral. The REEACT researcher would then need to inform both the referrer and the patient’s GP of the outcome of the referral. This method of direct referral was permitted only following discussions with and agreement from individual GP practices and all practice staff involved. This recruitment method was available from November 2009.

Pre-screening for trial eligibility

Problems with the process of REEACT researchers confirming trial participant eligibility led us to change our protocol. During the first 4 months of trial recruitment, we were finding that approximately 40% of participants who had provided their permission for us to contact them had a depression score below the inclusion threshold (PHQ-9 ≥ 10) when assessed for eligibility during face-to-face visits. This meant that, because of the large geographical areas covered by each centre, researchers were travelling long distances to see ineligible patients. In addition, there seemed to be a level of disappointment among otherwise willing participants when they were told they were ineligible to take part in the trial following a face-to-face meeting.

We therefore extended the eligibility protocol to allow REEACT researchers to administer the PHQ-9 over the telephone to pre-screen for baseline eligibility before arranging a face-to-face meeting. The PHQ-9 has been shown to be reliable when administered over the telephone. 33 Those participants who scored ≥ 10 on the PHQ-9 during the pre-screen would receive a face-to-face meeting where the PHQ-9 eligibility check would be repeated to confirm eligibility at the actual point of entry into the trial. Those participants who proved to be ineligible following the pre-screen were advised that their symptoms did not match the symptoms under investigation in the trial and were advised to continue to see their GP as normal. This process of pre-screening for trial eligibility was implemented from November 2009.

Postal questionnaires

During the first 6 months of the follow-up period it became apparent that our current retention rate of approximately 65% at 4-month follow-up – the point at which our primary outcome measure was collected – was falling below our target retention rate of 75%. REEACT researchers were finding it difficult to contact participants to arrange follow-up interviews. It was felt that this might in part be a result of participants having returned to their normal daily activities following their recruitment into the trial, as they might find it more difficult to spare the additional time required to be interviewed.

In an attempt to improve our retention rates, we gained ethical approval to give participants the option of completing and returning follow-up questionnaires by post, in the hope that this method might make it easier for participants to fit completing the questionnaires into their daily lives. The participant information sheet was revised to reflect the additional option of postal questionnaire completion. Participants were still provided with the option of completing their follow-up through a face-to-face or telephone interview as described in the original participant information sheet. Participants already in the trial were given the option of postal questionnaire completion at their next follow-up and received a copy of the updated participant information sheet. The suicide protocol was also revised to describe the procedure to follow if a potential suicide risk was identified on return of a postal questionnaire. Participants recruited to the trial following implementation of this protocol change were asked to provide their consent to us informing their GP about potential risk if this was deemed necessary. The addition of postal questionnaires as a completion method was implemented in August 2010.

Qualitative study

We revised our plans for evaluating the acceptability of cCBT from those detailed in our original protocol. The original protocol stated that the qualitative data would be collected during 10 focus groups each with 7–10 participants. However, because of pragmatic difficulties in arranging group sessions across the four recruitment centres, our qualitative researcher was concerned that participants might be excluded from the qualitative study if they were unable to find convenient times to attend the focus groups. We also considered that individual interviews might be more appropriate given the individual nature of the intervention and might better enable us to explore how the individual patients experienced the independent computerised therapy. Therefore, in order to increase the accessibility of the study, and in the light of considerations about the nature of the individual experience, we decided instead to collect the qualitative data through individual semistructured interviews, which could be arranged at times and locations to suit each participant.

We also revised our patient-sampling frame. In the original protocol, we planned to sample participants based on expressed preference for cCBT prior to randomisation; on randomisation to Beating the Blues, usual GP care, or MoodGYM; and on whether cCBT was accessed at home or from a central location. However, data collected from participants during the first 8 months of the follow-up period indicated that many participants were struggling to complete the cCBT programs and that the majority of participants were accessing the cCBT programs at home. Treatment completion is a significant issue in patient acceptability and we felt it was important to explore reasons for discontinuing or continuing treatment. As a result, the patient sampling frame was changed so that participants would be sampled based on their expressed preference for cCBT prior to randomisation and on whether they completed the course of cCBT or not. This allowed us to identify both positive and negative experiences of the treatment. We planned to recruit between 30 and 40 participants, with the group reflecting a roughly equal balance of preference and completion.

In the original protocol we were not able to explore the views of participants who did not think computerised therapy was acceptable and who did not wish to be randomised into one of the trial groups. Non-acceptability – refusal to begin treatment – is a noted problem in the literature on cCBT81 and it is likely to be especially relevant in routine primary care, where not all individuals are likely to be willing to try cCBT. 82 We therefore revised our original protocol to allow us to explore reasons for treatment refusal in an attempt to understand perceived barriers to using cCBT and to identify how treatment uptake can be improved. We thus amended the original protocol. Participants who spontaneously expressed an unwillingness to try cCBT as the reason why they did not wish to join the trial would be asked if they would consider being interviewed. Those participants who agreed would be sent an information sheet before being contacted again. We aimed to recruit between 8 and 10 participants in this group.

Finally, in the light of the finding that the majority of participants did not access the cCBT programs in their GP practices as previously expected, we revised the interview topic guides and information sheet to be used in the planned qualitative interviews with health professionals. However, data would still be collected on professionals’ opinions of the feasibility of providing cCBT on site and their perceptions of the acceptability of home and central access for patients.

These protocol changes were implemented from September 2010.

Sample size

Early in the trial we noticed lower than expected follow-up rates at 4-month follow-up (the point at which our primary outcome was measured). This led us to increase our recruitment target from 600 (200 participants in each of the three groups) to 690 participants (230 participants in each of the three groups). In our original protocol we had allowed for a loss to follow-up of 25% to retain 80% power to detect superiority between the usual GP care group and either of the two cCBT groups and non-inferiority between commercial (Beating the Blues) and free-to-use (MoodGYM) cCBT.

To calculate the most effective sample size we modelled the effects of follow-up rates on power. We subsequently gained ethical approval to over-recruit by 30 participants in each arm (an overall increase of 90 participants), which allowed for a loss to follow-up of 35% while retaining 80% power to detect superiority between the usual GP care group and either of the two cCBT groups, and non-inferiority between commercial and free-to-use cCBT. This protocol change was implemented from September 2010.

Questionnaire response rates

As previously mentioned, we experienced lower than expected follow-up rates. As a way of improving our follow-up response rate, we received ethical approval to send a £5 voucher as a token ‘thank you’ payment to participants at each follow-up time point. This token payment was not mentioned in the participant information sheet so that the risk of this being interpreted as a financial incentive to take part in, or continue with, the study was minimised. Receipt of a £5 voucher at follow-up was not conditional on participants completing their follow-up questionnaires. For follow-ups completed during face-to-face visits or telephone interviews, participants received a £5 voucher with their appointment letter. Follow-up questionnaires that were posted out to participants for them to complete and return were accompanied by a £5 voucher. This approach was based on the findings of a systematic review investigating ways of increasing response rates to postal questionnaires. 83 This protocol change was implemented from February 2011.

Long-term follow-up

Given that this study allowed us to assemble one of the largest groups of people with depression in UK primary care, we took the opportunity to investigate the impact of depression and treatment over the longer term. We therefore gained ethics approval to develop a primary care depression cohort of trial participants over a follow-up period of 10 years, with additional follow-ups taking place at 3, 5, 7 and 10 years. Participants were provided with information about these additional follow-ups and those interested were asked to provide their consent to take part. Although the development of a primary care depression cohort represented an extension of our original research objectives, it was not an objective for which we received funding. Therefore, although the primary care depression cohort is referred to in the protocol, it will not be discussed within this report.

Trial recruitment

Four main sites across the UK co-ordinated recruitment to the study. These were the University of Bristol, the University of Manchester, the University of Sheffield and the University of York. In addition, the University of York site was responsible for co-ordinating recruitment at an additional five sites (Durham and Darlington, Gateshead, Hull, North of Tyne and South of Tyne). Table 3 shows the number of participants recruited at each site. Participants were recruited through GP practices, with each site recruiting between 4 and 21 GP practices. Eighty-three out of 100 participating GP practices recruited at least one trial participant (recruitment of participants per GP practice ranged from 1 to 56). Recruitment was staggered, with GP practices participating at different time points during the course of the study. Further details of recruiting sites and GP practices are presented in Appendix 12.

Recruitment of participants began in August 2009 and ended in March 2011. In total, 1273 individuals were screened as potential participants and, of these, 691 (54.3%) were randomised. The overall rate of recruitment is shown in Figure 1. Figure 2 shows the Consolidated Standards of Reporting Trials (CONSORT) flow chart of participants through the trial. The follow-up period began in December 2009 and ended in April 2013.

FIGURE 1.

Trial recruitment rate.

FIGURE 2.

The REEACT trial CONSORT diagram. Reproduced from Gilbody et al. 11 © BMJ 2015. This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/.

Baseline data

Patient Health Questionnaire-9

To be included in the study participants had to score 10 or more on the PHQ-9 at baseline. A cut-off point of 10 has been shown to detect clinical depression in UK primary care. 14 A cut-off point of 10 was used to categorise participants as depressed or not depressed at 4 months, 12 months and 24 months (PHQ-9 ≥ 10 = depressed, PHQ-9 < 10 = not depressed). The descriptive statistics for the PHQ-9 by group across all time points are presented in Table 8 and displayed in Figure 3.

FIGURE 3.

Bar chart of the proportion of patients who are depressed by group. (a) Beating the Blues; (b) MoodGYM; and (c) usual GP care alone.

As previously stated, to be eligible for the study participants had to score 10 or above on the PHQ-9 at baseline. The mean PHQ-9 score at baseline was 16.65 (SD 4.25) and randomisation resulted in balance across the three groups. The mean PHQ-9 score at 4 months was 9.71 (SD 6.35), at 12 months was 7.99 (SD 5.98) and at 24 months was 8.33 (SD 6.19). If we dichotomise the PHQ-9 using a cut-off point of 10, the number of participants depressed at 4 months was 250 (47.53%), at 12 months was 170 (35.12%) and at 24 months was 176 (38.18%).

Primary outcome

At 4 months, 83 of the 165 (50.30%) participants in the Beating the Blues group, 78 of the 179 (43.58%) participants in the usual GP care group and 89 of the 182 (48.90%) participants in the MoodGYM group scored 10 or more on the PHQ-9. The logistic regression results at 4 months highlighted that there was little or no evidence of a difference between Beating the Blues and usual GP care alone (OR 1.19, 95% CI 0.75 to 1.88) or between MoodGYM and usual GP care alone at 4 months (OR 0.98, 95% CI 0.62 to 1.56; Table 9).

The non-inferiority comparison between MoodGYM and Beating the Blues found the OR at 4 months was 0.91 (90% CI 0.62 to 1.34, p = 0.69). The upper limit of the 90% CI for the OR was 1.34, thus satisfying statistical criteria for non-inferiority of MoodGYM compared with Beating the Blues.

Secondary outcomes

Patient Health Questionnaire-9: results from the logistic regression at 12 and 24 months

At 12 months, 54 of the 153 (35.29%) participants in the Beating the Blues group, 66 of the 166 (39.76%) participants in the usual GP care group and 50 of the 165 (30.30%) participants in the MoodGYM group scored 10 or more on the PHQ-9. At 24 months, 60 of the 143 (41.96%) participants in the Beating the Blues group, 61 of the 158 (38.61%) participants in the usual GP care group and 55 of the 160 (34.38%) participants in the MoodGYM group scored 10 or more on the PHQ-9. The logistic regression results revealed that there was little or no evidence of a difference between Beating the Blues and usual GP care across any of the time points (12 months OR 0.77, 95% CI 0.47 to 1.26; 24 months OR 1.00, 95% CI 0.60 to 1.68; Table 9). There was some evidence of a difference between MoodGYM and usual GP care at 12 months (OR 0.56, 95% CI 0.34 to 0.93) but this was no longer evident at 24 months (OR 0.68, 95% CI 0.41 to 1.15; Table 9). For the non-inferiority comparison between MoodGYM and Beating the Blues at 12 months, the OR was 0.77 (90% CI 0.50 to 1.18, p= 0.31) (Table 9 and Figure 4) and at 24 months was 0.72 (90% CI 0.47 to 1.11, p = 0.21) (Table 9 and Figure 4). At both time points, the upper limit of the 90% CI for the OR satisfied statistical criteria for non-inferiority of MoodGYM compared with Beating the Blues. For the non-inferiority comparison between MoodGYM and Beating the Blues at 24 months, the OR was 0.72 (90% CI 0.47 to 1.11, p = 0.21) (Table 9 and Figure 4). The upper limit of the 90% CI for the OR was 1.11, thus satisfying statistical criteria for non-inferiority of MoodGYM and Beating the Blues.

FIGURE 4.

Mean PHQ-9 score over time by trial arm comparison. Means are predicted means and 95% CIs, estimated from the mixed model and adjusted for gender, age, baseline PHQ-9 score, duration of depression and level of anxiety as fixed effects. The model also included an interaction between month and treatment. Raw unadjusted means are presented for baseline. (a) Beating the Blues vs. usual GP care; and (b) MoodGYM vs. usual GP care. Reproduced from Gilbody et al. 11 © BMJ 2015. This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/.

Patient Health Questionnaire-9 dichotomised: results from the mixed model

We also fitted a repeated measures multilevel logistic regression model to the PHQ-9 scores, and Table 10 shows the results of this analysis. The values at 4, 12 and 24 months were the outcome measures, and the baseline PHQ-9 score, age, gender, depression duration, level of anxiety, treatment group and time were included as fixed effects; an interaction between treatment and time was also included in the model. Different covariance patterns were assessed for the repeated measurements within participants and, as the results were similar, the model with the smallest AIC was chosen for each comparison (independent for all comparisons). For the Beating the Blues versus usual GP care and MoodGYM versus usual GP care comparison there was no evidence of an overall difference between the two groups (p = 0.96 and p = 0.11, respectively). The results are similar to those from the logistic regression models at each individual time-point presented in Table 9. However, the results did change for the non-inferiority comparison between MoodGYM and Beating the Blues at 4 months: the OR was 0.94 (90% CI 0.57 to 1.55, p = 0.84). The upper limit of the 90% CI for the OR was 1.55, thus no longer satisfying statistical criteria for non-inferiority of MoodGYM and Beating the Blues (Figure 5).

FIGURE 5.

Results from the logistic regression and mixed models for the non-inferiority comparisons between MoodGYM and Beating the Blues.

TABLE 10 - Result of mixed model for the dichotomised PHQ-911

Odds are presented as ‘odds of being depressed’; multilevel model adjusted for gender, age, baseline depression severity, depression duration and level of anxiety as fixed effects. The model also included an interaction between month and treatment.

90% CI for the non-inferiority comparison.

Comparison	OR	95% CI		p-value
		Lower	Upper
4 months
Beating the Blues vs. usual GP care alone	1.27	0.70	2.28	0.43
MoodGYM vs. usual GP care alone	1.13	0.61	2.10	0.70
MoodGYM vs. Beating the Blues	0.94	0.57a	1.55a	0.84
12 months
Beating the Blues vs. usual GP care alone	0.66	0.32	1.34	0.24
MoodGYM vs. usual GP care alone	0.44	0.22	0.88	0.02
MoodGYM vs. Beating the Blues	0.73	0.40a	1.32a	0.39
24 months
Beating the Blues vs. usual GP care alone	1.16	0.44	3.05	0.77
MoodGYM vs. usual GP care alone	0.62	0.30	1.29	0.20
MoodGYM vs. Beating the Blues	0.60	0.29a	1.26a	0.26

Clinical Outcomes in Routine Evaluation – Outcome Measure

The CORE-OM questionnaire was measured at baseline and at 4, 12 and 24 months. Descriptive statistics of the CORE-OM scores are presented in Table 15. The mean CORE-OM scores decreased over time across all trial arms but increased slightly at 24 months in the two cCBT groups.

TABLE 15 - Clinical Outcomes in Routine Evaluation – Outcome Measure summary by duration of follow-up and treatment group

Characteristic	Beating the Blues	MoodGYM	Usual GP care alone
Baseline	n = 209	n = 242	n = 237
Mean (SD)	19.23 (5.22)	18.65 (5.75)	18.63 (5.59)
Median (minimum to maximum)	19.12 (7.94 to 34.12)	18.97 (4.41 to 32.94)	18.53 (4.12 to 34.41)
4 months	n = 155	n = 170	n = 167
Mean (SD)	13.81 (7.19)	12.85 (7.37)	13.03 (8.28)
Median (minimum to maximum)	13.82 (0.88 to 33.24)	11.91 (0.59 to 30.00)	12.94 (0.29 to 34.41)
12 months	n = 137	n = 155	n = 150
Mean (SD)	11.07 (7.42)	10.10 (6.72)	11.75 (7.99)
Median (minimum to maximum)	10.59 (0.29 to 31.18)	9.12 (0 to 31.47)	10.58 (0 to 31.47)
24 months	n = 124	n = 143	n = 139
Mean (SD)	12.76 (7.68)	10.57 (6.97)	11.48 (8.05)
Median (minimum to maximum)	11.76 (0.29 to 32.65)	9.41 (0 to 30.59)	10.29 (0 to 33.24)

We fitted a repeated-measures multilevel linear regression model to the CORE-OM scores. The values at 4, 12 and 24 months were the outcome measures, and the baseline value, treatment group and time were included as fixed effects; an interaction between treatment and time was also included in the model. Different covariance patterns were assessed for the repeated measurements within participants and, as the results were similar, the model with the smallest AIC was chosen for each comparison (independent for both comparisons). For the Beating the Blues versus usual GP care comparison there was little or no evidence of an overall difference between the two groups (p = 0.50) and the mean CORE-OM score (over all follow-up assessments) was 11.74 (95% CI 10.27 to 13.21) for Beating the Blues and 12.18 (95% CI 10.64 to 13.71) for usual GP care. For the MoodGYM versus usual GP care comparison there was some evidence of an overall difference between the two groups (p = 0.05) and the mean CORE-OM score (over all follow-up assessments) was 11.98 (95% CI 10.47 to 13.50) for MoodGYM and 13.14 (95% CI 11.55 to 14.73) for usual GP care. Table 16 and Figure 6 display the results of the mixed model for each time point individually.

TABLE 16 - Results of the linear mixed model for CORE-OM11

Multilevel model adjusted for gender, age, baseline CORE-OM score, duration of depression and level of anxiety as fixed effects. The model also included an interaction between time point (month) and treatment.

Comparison	Intervention, mean (95% CI)	Usual GP care, mean (95% CI)	Difference (95% CI)	p-value
4 months
Beating the Blues vs. usual GP care alone	12.87 (11.31 to 14.43)	12.75 (11.14 to 14.36)	0.12 (– 1.33 to 1.57)	0.87
MoodGYM vs. usual GP care alone	13.61 (12.01 to 15.20)	13.69 (12.02 to 15.36)	– 0.09 (– 1.44 to 1.27)	0.90
12 months
Beating the Blues vs. usual GP care alone	10.51 (8.91 to 12.11)	12.00 (10.34 to 13.66)	– 1.49 (– 3.04 to 0.06)	0.06
MoodGYM vs. usual GP care alone	10.87 (9.25 to 12.48)	12.98 (11.28 to 14.68)	– 2.12 (– 3.54 to – 0.69)	0.004
24 months
Beating the Blues vs. usual GP care alone	11.84 (10.13 to 13.55)	11.78 (10.05 to 13.51)	0.06 (– 1.68 to 1.80)	0.95
MoodGYM vs. usual GP care alone	11.48 (9.80 to 13.16)	12.74 (11.00 to 14.49)	– 1.27 (– 2.80 to 0.27)	0.11

FIGURE 6.

Mean CORE-OM scores over time by trial arm comparison. (a) Beating the Blues vs. usual GP care; and (b) MoodGYM vs. usual GP care.

Short Form questionnaire-36 items Health Survey version 2

The SF-36v2 questionnaire was used to assess self-reported HRQoL at baseline and at 4, 12 and 24 months. Descriptive statistics of the domain scores and the PCS scores and MCS scores are presented in Tables 17 and 18. Only the PCS scores and MCS scores have been analysed, all other components are presented descriptively. Figures 7 and 8 show the mean MCS scores and PCS scores (and 95% CI) by trial arm, over time. This shows that, for the PCS, there was little difference between the treatment groups at any time and no clear pattern of improvements over time. For the MCS, the mean scores for Beating the Blues increased between 4 and 12 months and then decreased slightly at 24 months, but this score was still higher than at 4 months. For MoodGYM the scores increased between 4 and 12 months and then remained constant, whereas for usual GP care scores remained similar over all time points.

TABLE 17 - Short Form questionnaire-36 items Health Survey version 2 summary scores by duration of follow-up and treatment group

Characteristic	Beating the Blues		MoodGYM		Usual GP care alone
	n	Mean score (SD)	n	Mean score (SD)	n	Mean score (SD)
Physical functioning
Baseline	210	48.39 (9.90)	242	49.25 (9.32)	239	49.55 (9.78)
4 months	155	48.06 (10.65)	170	49.53 (10.02)	167	49.86 (9.41)
12 months	137	48.51 (10.77)	156	49.53 (10.34)	151	49.69 (10.27)
24 months	123	47.23 (11.77)	143	47.61 (12.13)	139	49.78 (9.87)
Role-physical
Baseline	210	43.17 (11.94)	242	42.85 (12.67)	239	45.24 (12.35)
4 months	155	44.37 (11.31)	170	45.56 (11.95)	167	46.35 (11.59)
12 months	137	45.82 (11.38)	156	45.91 (11.75)	151	45.45 (11.92)
24 months	124	44.75 (12.84)	143	44.97 (12.41)	139	46.19 (11.11)
Bodily pain
Baseline	210	43.29 (10.95)	242	43.89 (10.70)	239	44.48 (10.52)
4 months	155	45.00 (11.94)	170	46.77 (11.67)	167	47.14 (11.41)
12 months	137	46.94 (12.05)	156	46.92 (12.49)	151	47.84 (11.26)
24 months	123	45.45 (12.95)	142	46.30 (13.02)	137	47.81 (12.04)
General health
Baseline	210	37.84 (10.43)	242	39.49 (11.08)	239	39.47 (10.20)
4 months	155	39.63 (11.74)	169	41.93 (11.55)	167	41.48 (11.65)
12 months	136	42.39 (12.53)	156	44.04 (12.07)	150	43.27 (11.18)
24 months	124	40.17 (13.18)	141	43.93 (12.75)	139	42.80 (11.41)
Vitality
Baseline	210	32.01 (8.10)	242	32.39 (7.70)	239	33.61 (8.05)
4 months	155	38.60 (10.63)	170	40.55 (11.49)	166	40.36 (10.92)
12 months	137	41.77 (11.24)	156	42.58 (11.67)	151	41.51 (11.23)
24 months	124	40.13 (12.05)	143	42.91 (11.81)	139	41.51 (11.57)
Social functioning
Baseline	209	30.39 (10.47)	242	29.47 (10.83)	238	31.05 (10.00)
4 months	155	36.37 (12.68)	170	38.18 (12.11)	167	39.21 (11.91)
12 months	136	41.69 (13.22)	156	41.43 (12.64)	151	40.96 (12.29)
24 months	123	38.71 (13.53)	143	41.02 (13.25)	137	41.32 (12.23)
Role-emotional
Baseline	210	28.34 (11.26)	242	28.57 (12.12)	239	29.79 (11.07)
4 months	155	36.49 (13.12)	170	37.31 (13.13)	167	38.10 (13.06)
12 months	137	43.00 (12.06)	156	42.55 (12.75)	151	40.02 (14.02)
24 months	124	39.08 (13.88)	143	41.82 (12.47)	139	40.86 (13.18)
Mental health
Baseline	210	28.73 (7.98)	242	28.11 (9.01)	239	29.05 (8.78)
4 months	155	36.15 (11.31)	170	38.15 (12.45)	166	38.25 (13.19)
12 months	137	41.11 (12.05)	156	42.67 (11.88)	151	40.17 (13.18)
24 months	124	38.55 (12.81)	143	42.19 (12.16)	139	40.32 (13.08)

TABLE 18 - Short Form questionnaire-36 items Health Survey version 2 PCS scores and MCS scores by duration of follow-up and treatment group

SF-36v2 higher score is better in health.

Characteristic	Beating the Blues	MoodGYM	Usual GP care alone
Baseline	n = 209	n = 242	n = 238
PCS score, mean (SD)	50.31 (11.55)	51.20 (10.08)	51.94 (10.40)
MCS score, mean (SD)	22.81 (9.96)	22.22 (10.55)	23.51 (9.64)
4 months	n = 155	n = 169	n = 166
PCS score, mean (SD)	48.45 (11.72)	50.08 (11.22)	50.29 (10.39)
MCS score, mean (SD)	32.85 (13.27)	34.50 (14.20)	34.79 (14.01)
12 months	n = 135	n = 156	n = 150
PCS score, mean (SD)	48.24 (12.46)	48.80 (11.91)	49.80 (10.88)
MCS score, mean (SD)	39.56 (12.59)	40.16 (13.05)	37.28 (14.93)
24 months	n = 122	n = 140	n = 137
PCS score, mean (SD)	47.45 (13.19)	47.96 (12.99)	49.92 (10.70)
MCS score, mean (SD)	36.11 (13.95)	40.36 (13.04)	37.79 (14.07)

FIGURE 7.

Short Form questionnaire-36 items Health Survey version 2 MCS scores over time trial arm comparison. (a) Beating the Blues vs. usual GP care; and (b) MoodGYM vs. usual GP care.

FIGURE 8.

Mean SF-36v2 PCS scores over time trial arm comparison. (a) Beating the Blues vs. usual GP care; and (b) MoodGYM vs. usual GP care.

We fitted a repeated measures multilevel regression model to the PCS scores and MCS scores. The values at 4, 12 and 24 months were the outcome measures, and the baseline value, treatment group and time were included as fixed effects. An interaction between treatment and time was also included in the model. Different covariance patterns were assessed for the repeated measurements within participants and, as the results were similar, the model with the smallest AIC was chosen for each component score and comparison (independent for Beating the Blues vs. usual GP care alone and unstructured for MoodGYM vs. usual GP care alone for both component scores). For the Beating the Blues versus usual GP care comparison, there was little or no evidence of an overall difference between the two groups for the MCS (p = 0.40) or for the PCS (p = 0.22), and the mean SF-36v2 MCS score (over all follow-up assessments) was 38.52 (95% CI 35.77 to 41.27) for Beating the Blues and 37.50 (95% CI 34.62 to 40.38) for usual GP care and for the PCS was 48.66 (95% CI 47.05 to 50.28) for Beating the Blues and 49.55 (95% CI 47.86 to 51.23) for usual GP care. For the MoodGYM versus usual GP care comparison, there was evidence of an overall difference between the two groups for the MCS (p = 0.009) but little or no evidence for the PCS (p = 0.14). The mean SF-36v2 MCS score (over all follow-up assessments) was 38.98 (95% CI 36.03 to 41.92) for MoodGYM and 35.99 (95% CI 32.88 to 39.11) for usual GP care and for the PCS was 48.87 (95% CI 47.11 to 50.63) for MoodGYM and 49.91 (95% CI 48.05 to 51.76) for usual GP care. Tables 19 and 20 and Figures 7 and 8 display the results of the mixed model for each time point individually.

TABLE 19 - Results of the Linear Mixed Model for the SF-36v2 for MCS11

Adjusted for gender, age, baseline depression severity, depression duration and level of anxiety.

Comparison	Intervention, mean (95% CI)	Usual GP care, mean (95% CI)	Difference (95% CI)	p-value
4 months
Beating the Blues vs. usual GP care alone	35.23 (32.28 to 38.17)	36.33 (33.29 to 39.37)	– 1.10 (– 3.85 to 1.64)	0.43
MoodGYM vs. usual GP care alone	35.07 (31.90 to 38.23)	34.87 (31.54 to 38.20)	0.20 (– 2.60 to 2.99)	0.89
12 months
Beating the Blues vs. usual GP care alone	41.58 (38.55 to 44.60)	38.25 (35.11 to 41.39)	3.32 (0.37 to 6.28)	0.03
MoodGYM vs. usual GP care alone	40.91 (37.77 to 44.05)	36.66 (33.33 to 40.00)	4.25 (1.47 to 7.02)	0.003
24 months
Beating the Blues vs. usual GP care alone	38.76 (35.52 to 41.99)	37.92 (34.62 to 41.22)	0.84 (–2.49 to 4.17)	0.62
MoodGYM vs. usual GP care alone	40.95 (37.68 to 44.23)	36.44 (33.03 to 39.86)	4.51 (1.51 to 7.51)	0.003

TABLE 20 - Result of the linear mixed model for the SF-36v2 for PCS11

Adjusted for gender, age, baseline depression severity, depression duration and level of anxiety.

Comparison	Intervention, mean (95% CI)	Usual GP care, mean (95% CI)	Difference (95% CI)	p-value
4 months
Beating the Blues vs. usual GP care alone	49.18 (47.44 to 50.92)	49.85 (48.05 to 51.64)	– 0.67 (– 2.33 to 0.99)	0.43
MoodGYM vs. usual GP care alone	49.86 (47.97 to 51.75)	50.21 (48.23 to 52.19)	– 0.35 (– 2.04 to 1.33)	0.68
12 months
Beating the Blues vs. usual GP care alone	48.61 (46.80 to 50.43)	49.73 (47.86 to 51.61)	– 1.12 (– 2.95 to 0.71)	0.23
MoodGYM vs. usual GP care alone	48.85 (46.96 to 50.73)	50.06 (48.07 to 52.05)	– 1.21 (– 2.91 to 0.49)	0.16
24 months
Beating the Blues vs. usual GP care alone	48.19 (46.17 to 50.22)	49.06 (47.02 to 51.10)	– 0.87 (– 3.06 to 1.32)	0.44
MoodGYM vs. usual GP care alone	47.89 (45.82 to 49.97)	49.45 (47.30 to 51.60)	– 1.56 (– 3.61 to 0.50)	0.14

Summary

There was little or no evidence of statistically significant effects of Beating the Blues on the proportion of participants who are depressed across all time points when compared with usual GP care alone. There was some evidence of a statistically significant effect of MoodGYM on the proportion of participants who are depressed when compared with usual GP care at 12 months but this was no longer evident at 24 months and was not present at 4 months. The free-to-use cCBT program MoodGYM was found to be non-inferior to the commercial pay-to-use cCBT program Beating the Blues across all time points. There was no evidence of statistically significant effects of the interactions between patient preference and allocated treatment across any of the comparisons.

There was no evidence of statistically significant effects on the overall (including all time points) mean depression scores, CORE-OM scores and MCS and PCS quality of life scores of Beating the Blues when compared with usual GP care alone; however, there was evidence of a statistically significant effect of Beating the Blues on MCS scores at 12 months. There was no evidence of statistically significant effects on overall (including all time points) mean depression scores and PCS quality of life scores of MoodGYM when compared with usual GP care alone; however, there was evidence of a statistically significant effect of MoodGYM on the mean depression scores at 12 months when compared with usual GP care. There was evidence of statistically significant effects on the overall (including all time points) mean CORE-OM scores and MCS quality of life scores of MoodGYM when compared with usual GP care alone, with highly statistically significant effects of MoodGYM on CORE-OM scores at 12 months and MCS quality of life scores at 12 and 24 months when compared with usual GP care alone.

Participant engagement with the cCBT programs was low for both Beating the Blues and MoodGYM, with the majority of participants not completing the series of sessions. Participants were offered and received similar levels of technical telephone support across Beating the Blues and MoodGYM programs.

Some of this text has been reproduced from Gilbody et al. 11 © BMJ 2015. This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/.

Health-care resource use

Descriptive statistics (mean, SD, median and IQR) of health-care resource use by resource category per time period during trial follow-up, and aggregated for the full duration of the trial, are shown in Tables 23–25. The unit costs applied to resource use items to calculate total costs are also presented within the tables. The results presented are based on the available case data set, as multiple imputation was performed for total costs as opposed to individual resource use items.

Regarding the completeness of data, resource use data were available for 83.9% (n = 580) of all trial participants during the 2 years of follow-up. In general, differences between treatment groups in resource use appeared small, and resource use estimates appeared considerably variable with large SDs. Regarding the dispersion of the results, median estimates were smaller than means for all resource categories, treatment groups and follow-up periods. Medians were equal to zero for all treatment groups and resource use categories, except GP and nurse visits in both follow-up periods. This suggests that the distributions were skewed to the right.

Costs

Mean costs from baseline to 12 months and from 12 months to 24 months by treatment arm are shown in Table 26. The results are presented by type of services, namely primary care, hospital and other community services. Costs of depression-related medication and cCBT are also shown. Table 27 reports undiscounted and discounted costs aggregated for the full 24-month duration of the trial. All costs results were estimated from the multiple imputed data sets.

Primary care services represented the largest share of health-care expenditure for all treatment groups in both trial follow-up periods, constituting over 50% of total costs for all groups. The second largest category of costs was hospital services, which varied from 25% to 35% across the groups.

Total (discounted) costs for the 24 months were £1186 for Beating the Blues plus usual GP care, £1121 for usual GP care alone and £1098 for MoodGYM plus usual GP care group. There is limited value in interpreting differences in mean costs across time periods and for different treatment groups at this level, as the results shown in Table 27 are unadjusted for baseline costs and participant covariates. The adjusted mean differences for each cCBT treatment versus usual GP care alone and corresponding 95% CIs, obtained through regression analysis, are reported within the cost-effectiveness results section (see Cost-effectiveness analysis).

Health-related quality of life

Table 28 shows the extent to which EQ-5D questionnaires were completed at each time point during trial follow-up. Table 29 shows the corresponding information for SF-6D scores.

There was a steady decrease in completed EQ-5D questionnaires throughout trial follow-up. The rate of questionnaire return was slightly higher for the Beating the Blues plus usual GP care treatment group but, in general, it was similar for all treatment groups.

The pattern of response to SF-6D is similar to that of EQ-5D, with the response rate decreasing over time. Tables 30 and 31 summarise the unadjusted HRQoL scores and QALY estimates obtained from EQ-5D and SF-6D, respectively, from the multiple imputed data sets.

Mean QALYs estimated through EQ-5D over 24 months (discounted at 3.5% rate for QALYs accrued in the last 12 months of follow-up) were 1.3325 for Beating the Blues plus usual GP care, 1.3888 for usual GP care alone and 1.3564 for MoodGYM plus usual GP care. Mean QALYs estimated through SF-6D over 24 months (discounted at 3.5% rate for QALYs accrued in the last 12 months of follow-up) were 1.2464 for Beating the Blues plus usual GP care, 1.2856 for usual GP care alone and 1.2810 for MoodGYM plus usual GP care.

As for total costs, HRQoL scores and QALYs estimates shown here are unadjusted means, and, therefore, there is limited value in interpreting differences between treatments. Mean differences for each cCBT treatment versus usual GP care alone and corresponding 95% CIs, adjusted for patient covariates by regression analysis, are shown in the following section.

Cost-effectiveness analysis

Table 32 shows the results for the differences in mean costs and QALYs adjusted for patient covariates, between each cCBT treatment group and usual GP care alone. In general, differences in costs and QALYs between both cCBT groups and usual GP care alone were small with wide CIs and were not statistically significant at a 5% significance level.

Results for the SUR models, where incremental QALYs and costs are estimated simultaneously, are presented in Appendix 13. The estimates from the SUR model are very similar to those estimated for the base-case analysis. The results lack statistical significance (5% level of significance) and CIs on both models are wide (and overlap between models).

Base-case cost-effectiveness results

Table 33 presents the fully incremental cost-effectiveness estimates and probability that each intervention is cost-effective at a threshold of £20,000 per QALY.

TABLE 33 - Cost-effectiveness analysis summary table: base case

Compared with usual GP care alone.

ICER on south-west quadrant of the cost-effectiveness plane.

Treatment group	Δ costsa (£)	Δ QALYsa	ICERa (£ per QALY)	Probability cost-effective at £20,000/QALY
Beating the Blues	104.24	– 0.0435	Dominated	0.038
MoodGYM	– 106.07	– 0.0153	6933b	0.417
Usual GP care alone	–	–	–	0.545

Beating the Blues plus usual GP care is dominated by usual GP care alone, with higher mean costs and lower QALYs. As both mean costs and QALYs were lower for MoodGYM plus usual GP care than for usual GP care alone, the ICER estimated falls within the south-west quadrant of the cost-effectiveness plane. In this quadrant, the interpretation of the ICER is estimated using the difference in costs and QALYs between the lower-cost intervention (in this case, MoodGYM) and the higher-cost intervention (usual GP care alone). Consequently, the ICER of £6933 per additional QALY actually represents the ICER of usual GP care alone versus MoodGYM. As this falls within the £20,000 per QALY threshold, the conclusion is that usual GP care alone is cost-effective when compared with MoodGYM plus usual GP care. Consequently, these main results do not provide evidence supporting the cost-effectiveness of either Beating the Blues or MoodGYM compared with usual GP care alone.

Table 33 also reports probabilities of cost-effectiveness for each of the treatments under comparison at a cost-effectiveness threshold of £20,000 per QALY. The corresponding CEACs, showing the probability of each treatment being cost-effective across a range of cost-effectiveness thresholds, are shown in Figure 9. At a £20,000 per QALY threshold, usual GP care alone is the treatment most likely to be cost-effective (0.545 probability of cost-effectiveness), followed by MoodGYM plus usual GP care (0.417 probability of cost-effectiveness) and Beating the Blues plus usual GP care (0.038 probability of cost-effectiveness).

FIGURE 9.

Base case cost-effectiveness acceptability curve.

Figures 10 and 11 present the incremental cost and effectiveness estimates of the comparator (Beating the Blues or MoodGYM) compared with usual GP care alone taken from the probabilistic sensitivity analysis. Each point represents the incremental costs and effectiveness given one particular realisation of uncertainty. The £20,000 per QALY line represents the cost-effectiveness threshold. All points south-east of this line represent realisations of uncertainty when the comparator (Beating the Blues or MoodGYM) is cost-effective compared with usual GP care alone at a cost-effectiveness threshold of £20,000 per QALY.

FIGURE 10.

Distribution of incremental costs and QALYs of Beating the Blues vs. usual GP care alone.

FIGURE 11.

Distribution of incremental costs and QALYs of MoodGYM vs. usual GP care alone.

Extrapolation

No strong evidence of difference between the treatments arms in terms of clinical effectiveness, HRQoL or costs was found in the trial. Given the lack of difference between arms, there is no basis for inferring that any differences might occur in the future and, therefore, that conclusions might be altered if extrapolation was conducted. Thus, no extrapolation of the trial results has been conducted.

Evidence synthesis

External evidence was used to facilitate the inclusion of additional trial evidence on the effectiveness of cCBT and also to incorporate additional comparators which were not included within the REEACT study. This was undertaken as an exploratory analysis to determine the robustness of the results to external evidence and to ascertain the generalisability of the findings to a broader decision problem which included additional comparators. Regression analyses were used to determine the impact of changes in depression, measured by change in PHQ-9 from baseline, rather than treatment arm on costs, excluding intervention costs, and QALYs measured in the trial. The coefficients on the change in PHQ-9 from the regression equations estimate the impact of changes in PHQ-9 on costs and QALYs for participants within the trial. This approach allowed external evidence on the changes in PHQ-9 to be related to expected changes in total costs and QALYs for a range of different comparators.

As this analysis was exploratory in nature, we did not undertake a formal systematic search of individual RCTs. Instead we restricted our approach to including external evidence published in recent meta-analyses of internet-based and face-to-face psychological therapies for adult patients with depression. Two meta-analyses were identified, one on the effectiveness of internet-based treatments for depression in adults,10 in which there was some evidence of effect for therapist-guided cCBT, and another on the effectiveness of brief face-to-face psychological therapies for patients with depression managed in primary care settings. 79 These reported the effect of the interventions compared with usual care in terms of a SMD. These SMDs can be converted back onto the change in PHQ-9 (ΔPHQ-9) scale by multiplying them by the SD in ΔPHQ-9 observed in the trial, so as to estimate the incremental impact of the two interventions on costs and QALYs in the trial population compared with usual GP care. These analyses were then combined with estimates of the intervention-specific costs to estimate the cost-effectiveness of cCBT and other psychological therapies compared with usual GP care alone in participants from the REEACT study.

As shown in Tables 42 and 43, a one-point reduction in PHQ-9 from baseline to 12 months resulted in a 0.02927 increase in QALYs over the trial period and a £7.07 reduction in costs over the trial period. For cCBT, the meta-analysis10 showed supported cCBT to be the most effective, with a SMD of 0.61, which, when converted onto the change in PHQ-9 scale, resulted in a reduction of 4.32 on the PHQ-9 score. Unsupported cCBT had a SMD of 0.25, which resulted in a reduction of PHQ-9 score of 1.771, whereas all cCBT had a SMD of 0.41, which resulted in a reduction of PHQ-9 score of 2.905. Costs for cCBT treatment were based on the REEACT trial, assuming that 50% of participants would use Beating the Blues and 50% MoodGYM and that supported cCBT would involve 1 hour’s support from a community support worker per patient.

Other psychological therapies, which included face-to-face CBT, had a SMD of 0.3179 which resulted in a reduction in PHQ-9 score of 2.196. Treatment costs for other psychological therapies were based on the cost of face-to-face CBT from a previous economic evaluation. 84

Table 44 presents the results of the evidence synthesis analysis. As can be seen from the table, all treatments, when compared with usual care, resulted in increased costs but also increased QALYs. Supported cCBT was the most effective option and extendedly dominated or dominated all the other cCBT and other psychological treatment options. Supported cCBT compared with usual care generated an ICER of £126 per QALY and a probability of being cost-effective at a £20,000 per QALY cost-effectiveness threshold of 0.9747.

The evidence synthesis analysis results suggested that, based on external evidence of effectiveness and the impact of changes in PHQ-9 from baseline to 12 months on costs and QALYs (observed in the REEACT trial), therapist-guided cCBT is the most cost-effective treatment option compared with usual GP care alone, unsupported cCBT and other psychological therapies. These findings are based on exploratory analyses and, as such, the findings should be considered tentative, as they are not based on directly randomised evidence. Furthermore, although we sought to include evidence from published meta-analyses, we did not attempt to critically appraise or quality assess their findings.

Summary

The within-trial results of the economic analysis suggest that neither Beating the Blues nor MoodGYM appeared cost-effective compared with usual GP care alone. In the base-case results, Beating the Blues was dominated by usual GP care alone (higher mean costs and lower QALYs), and mean costs and QALYs were lower for MoodGYM than for usual GP care alone, resulting in an ICER of £6933 per additional QALY for usual GP care alone versus MoodGYM. Usual GP care alone, compared with either cCBT intervention, was also the cost-effective intervention in the majority of scenario analyses, and it was the intervention most likely to be cost-effective at a £20,000 per QALY threshold (probabilities ranging across scenarios from 0.545 and 0.619). The exceptions to this were scenarios where SF-6D scores were used to estimate QALYs. In these scenarios, MoodGYM dominated usual GP care alone (lower mean costs and higher QALYs) and was the intervention most likely to be cost-effective at a £20,000 per QALY threshold (probabilities ranging across scenarios from 0.662 to 0.756).

Although the cost-effectiveness conclusions appear sensitive to the choice of whether EQ-5D or SF-6D is used to estimate them, the magnitude of the differences between all three groups was relatively low (and non-significant) for both cost and QALY estimates across all the scenarios. Hence, minor differences in the assumptions can lead to different cost-effectiveness interpretations because of relatively small impacts on the mean incremental estimates of costs and QALYs and, therefore, some caution should be exercised when interpreting these results. Results for the patient subgroup analysis were broadly consistent between the overall population and patient preference subgroups. There was little evidence of interaction effects between treatment allocation and patient preference.

Patient interviews

A total of 320 trial participants were sent an invitation to take part in the qualitative study. Of these, 39 (12%) participants responded to the invitation letter and were contacted by telephone to discuss participating in the study. A total of 36 participants were subsequently recruited to the study from across the four main trial sites, sampling for expressed preference for cCBT prior to randomisation and self-reported engagement with the cCBT program. Owing to a slowing in overall trial recruitment, no participants were recruited who indicated that their primary reason for refusal to participate in the main trial prior to randomisation was concern over using cCBT. All recruited participants received a participant information sheet about the qualitative study and completed a consent form. Individual semistructured patient interviews were conducted face to face in participants’ homes, except for three patient interviews with participants recruited via the University of Sheffield, which were conducted by telephone. Patient interviews were conducted by three qualitative researchers from the Universities of Manchester and Bristol (SK, HT, EA). Participants were interviewed between January and October 2011, at which point data saturation was judged complete.

Main themes from patient interviews

We did not observe any specific differences in themes between the two cCBT programs (Beating the Blues and MoodGYM). Differences in patient experience did not appear to be due to any specific features of a particular program, but related to the broader experience of therapy delivered by computer.

Three key themes emerged from the data: acceptability; engagement and adherence; and absence of support. Although participants varied in their reported acceptability, common issues relating to adherence and support were reported across the whole sample.

Patient acceptability: perceived benefits and barriers

We observed in the data that participants could be classified as ‘positive’, ‘negative’ or ‘ambivalent’ based on their perceptions of cCBT, referring to the fact that the participants’ perceptions could be clearly and consistently grouped under the headings of positive, negative or ambivalent and particularly noting how opinions on the same aspect of the program (e.g. its flexibility) could be seen to vary across these dimensions. Examples of this, presented in the classifications, are shown in Figure 12. These classifications were found to conceptualise differences usefully and were exhaustive (all participants could be clearly defined within the groupings). It is not typical to assign qualitative data into formal groups in this manner, but the categories are useful in the context of this pragmatic trial to understand experience and uptake of the health technology. Seventeen participants were categorised as ‘ambivalent’ (Beating the Blues, n = 7; MoodGYM, n = 10 ), 10 were categorised as ‘negative’ (Beating the Blues, n = 2; MoodGYM, n = 8) and nine as ‘positive’ (Beating the Blues, n = 4 ; MoodGYM, n = 5). These classifications were reached through consensus among the qualitative team, based on the rereading of the transcripts and the identification of consistent differences that could be classified as positive, negative or ambivalent. However, it is notable that these categorisations did not map clearly onto expressed prior preference for cCBT. There was no consistent relationship between prior expressed preference at baseline and whether the participant reported being positive or negative after they had personal experience of the programs. Although numbers are too small to draw any definitive conclusions, this may indicate that patients are not able to anticipate if computerised therapy would be appropriate for them based on initial preferences alone.

FIGURE 12.

Subthemes and extracts relating to perceived benefits and barriers. (a) Flexibility and autonomy; and (b) relational and connectedness. Reproduced from Knowles et al. 85 © BMJ Open 2015. This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/.

Subthemes and extracts relating to perceived barriers and benefits are shown in Figure 12, which illustrates both the consistency within classifications and also the contrast across the subthemes dependent on classification. Specifically, the four subthemes presented (‘flexibility, ‘autonomy’, ‘relational’, ‘connectedness’) illustrate how the same aspects of cCBT could be perceived both positively and negatively, depending on individual participant experience and preference. The subthemes are further illustrated with quotations within Figure 12.

• ‘Flexibility’ refers to the unstructured, flexible nature of the treatment, which was perceived as advantageous by positive participants, who appreciated the greater accessibility, but as a barrier by negative participants because of the greater responsibility placed on participants to organise and adhere to sessions themselves. Ambivalent participants recognised the potential benefits of this flexibility but reported that additional support or monitoring was necessary to encourage them to engage.

• ‘Autonomy’ refers to how the independence of cCBT was experienced, with negative participants feeling overburdened by the absence of support, in contrast to positive participants, who felt empowered to take control of their recovery. Ambivalent participants occupied a ‘middle ground’, in which they were willing to engage with the program independently but felt that barriers within the content prevented them from doing so.

• ‘Relational’ refers to whether the absence of another person, such as a therapist, was perceived positively (owing to the comfort offered by anonymity) or negatively (with participants struggling without the relational or empathetic input of a therapist). Again, ambivalent participants had a more moderate response, with increased personalisation being felt to be necessary for them to relate to the characters in the program and relate the content to their own lives.

• ‘Connectedness’ refers to the degree of connection that participants experienced with the program, with negative participants feeling that cCBT further isolated them and emphasised their loneliness, in contrast to positive participants, who felt that the program was ‘always there’ for them. Ambivalent participants again seemed instead frustrated by limitations of the content of these particular programs, similar to the ‘Relational’ theme where being able to identify with the material was perceived as important – in this case, the potential for connection with the content was hindered by inappropriate or overly generic material.

The analysis demonstrated that although some participants clearly embraced or rejected computerised therapy, as illustrated in Figure 12, the majority of participants (17 of the sample classed as ‘ambivalent’) fell between these extremes. For these ‘ambivalent’ participants, cCBT was not rejected outright as with the ‘negative’ participants or fully embraced as with the ‘positive’ participants but was considered to require greater external support and greater personalisation of material to enable participants to engage fully with the programs. Barriers relating specifically to experience of technology, such as problems in accessing the program or discomfort using computers, were not observed. This may reflect either that patient populations are increasingly familiar with using technologies, or that only those patients comfortable with technology were willing to enter the trial.

Patient engagement and adherence

All participants reported some difficulties with engaging with the programs, but, consistent with their reported attitudes, there was a trend for ‘positive’ participants to report greater adherence, and three of the four participants who reported completing the programs were in this group. ‘Negative’ participants reported lower levels of engagement than the rest of the sample, suggesting that participant attitudes are associated with behavioural engagement to some extent. On self-reported engagement, positive participants reported using the program an average of 9.1 times in total, ambivalent participants reported using the program 6.7 times in total and negative participants reported an average of 3.2 uses in total.

Participants also reported difficulties maintaining engagement when their depression was more severe and perceived the programs to be inappropriate for patients experiencing severe episodes.

I was feeling so miserable that all those questions did was actually just hold the mirror and show me how miserable I was and, so, I just turned the program off . . . And that was, for me, the first, kind of, 2 months really, 3 months, because I was just feeling so down.

DS025. Reproduced from Knowles et al. 85 © BMJ Open 2015. This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/

If you’re mildly depressed, or if you’ve turned the corner, then I think that’s when it’s appropriate. But I think if you were deeply depressed, and still struggling, then it would be much harder . . . I think you probably would fail and that would make you feel worse. Because the last thing you need is another failure when you’re feeling really down.

DS017. Reproduced from Knowles et al. 85 © BMJ Open 2015. This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/

Despite the provision of regular technical telephone support calls, participants also indicated that a greater level of monitoring or support would have encouraged engagement, which is reflected in the third theme.

Perceived limitations of support

All participants perceived the programs to be unsupported, suggesting that the technical support and encouragement offered via telephone support calls within the trial may not meet patient expectations regarding level or type of support that should be provided. However, although ‘negative’ participants appeared to disengage with the programs deliberately, both ‘positive’ and ‘ambivalent’ participants expressed a desire for greater monitoring and follow-up, providing greater motivational support to help them engage with and complete the programs.

Just someone being aware and sort of either saying ‘oh that’s great, you did another unit! Hey that’s good, keep it up!’ Or ‘you haven’t done a unit for 4 weeks. Only takes an hour, how about it?’.

DSB0906

If I had somebody ringing me up every week, then I would have been more likely to have stuck with it.

DS018

Participants reported that GPs did not tend to monitor their engagement with the programs, and suggested that the support needed to sustain engagement was unlikely to be provided in primary care because of lack of time in consultations and interruptions to continuity of care:

I think if they’d got the time you maybe would, but I think it’s indicative of the world we live in and the system that we live with. The good doctors are always in demand and you go back for your repeat prescription and you see who you see and there are maybe three of them that I would see, but you don’t see the one that suggested this therapy in the first place. So no, I’ve never talked about it to anybody.

DS029. Reproduced from Knowles et al. 85 © BMJ Open 2015. This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/

I wouldn’t have expected a GP to call me. It would have been nice, the family GP practitioner in the countryside but we live in a city and they’ve got what 10,000 patients on their book. So yeah I understand.

DS08. Reproduced from Knowles et al. 85 © BMJ Open 2015. This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/

Health professional interviews

A total of 70 GP practices that had recruited participants to the main trial were sent invitations to participate in the qualitative study. Fourteen health professionals responded to the invitation and were contacted to discuss the study in more detail. A total of 11 GPs were subsequently recruited to the study from 10 GP practices, across six out of the nine trial sites. No expressions of interest were received from practice managers. All recruited GPs received a participant information sheet about the qualitative study and completed a consent form. Individual semistructured interviews were conducted with GPs by telephone by a qualitative researcher (SK) from the University of Manchester. GPs were interviewed between September 2012 and February 2013.

Comparison of the patient and health professional interviews

A core finding from the patient interview transcripts concerned the individual differences in experience of computerised therapy, which related to different preferences around privacy, flexibility and independent working. The GPs tended to consider demographic or pragmatic factors in their referrals such as computer illiteracy or inability to access therapy because of work demands. GPs may be conflating perceptions of ability or education with perceptions of the patient as willing or able to engage with self-help therapies, or assuming certain demographic groups will be more suitable, which contrasts with the findings of the patient qualitative study that did not show consistent differences in age and gender. In fact, the most positive attitudes were reported by female participants, contrary to stereotypes about preference for cCBT among males.

Matching appropriate patients to cCBT treatment may need to be focused more on the individual needs and preferences of the patients, although the lack of congruence between patients’ expressed prior preference and their experience as positive or negative demonstrates the complexity of doing this. This may indicate a need for greater GP follow-up in the early stages of use to identify more rapidly those patients who are now struggling with the intervention, which would be consistent with the principles of stepped care around self-correction9 when patients do not respond to minimal interventions.

Both patients and health professionals reported a lack of follow-up on use of the cCBT program in primary care. GPs suggested that notifications from within the cCBT program about completion and assessment would be helpful to them but did not perceive themselves to have a role in supporting engagement with the program itself. However, patients’ desire for support was specifically related to a higher level of ongoing support while completing the cCBT program, although the patients interviewed also questioned whether or not it was feasible that this could be provided in primary care consultations. This suggests that support for engagement with cCBT in primary care may need to be provided by other health professionals or provided as supplemental remote or face-to-face support as part of the cCBT packages. However, as noted previously, providing earlier feedback to GPs themselves around initial engagement with the cCBT program may be necessary to help them follow the self-correcting principles of stepped care and enable alternative referrals for patients who are unable to complete the computerised intervention.

The majority of GPs were sceptical that computerised therapy could be provided in their practices, demonstrating that in routine primary care it would be unlikely that cCBT would be provided to patients on site. Given that those patients who reported positive experiences with cCBT valued the anonymity and flexibility of access from home, this suggests that attempting to implement cCBT services within general practice sites may lead to barriers for patient engagement as well as creating difficulties for GPs. It is possible that those patients who struggled with the flexibility of the home-accessed treatment would appreciate the greater structure offered by on-site access, but given the reported barriers to implementing this, it may again be the case that more remote support or motivational support built into the intervention would be most feasible and effective.

Limitations of the qualitative study with patients and health professionals

The patient interview sample was predominantly female and all participants were from a white background. This indicates that the sample is representative of the wider trial population (being consistent with the trial population profile); but, nevertheless, further research is necessary to determine whether similar experiences are reported by more diverse patient groups. The finding that technical aptitude was not commonly reported as a prominent issue may be because potential participants with low computer literacy declined to take part in the trial. Interviewing participants who decline should be attempted in future research on computerised interventions to explore if such self-selection occurs.

The GP interview sample was predominantly male, although there did not appear to be any consistent differences between male and female interviewees’ reports. We did not collect data on other factors that may have influenced attitudes, such as numbers of years in practice and specialist training in mental health, although, given the small sample size, it would have been difficult to analyse subgroups within the sample. The reports from GPs regarding follow-up and support may not be representative of how they would respond to patients receiving the intervention who were not part of a larger trial and should be interpreted with caution.

Implications of findings from qualitative study

The combined interviews suggest several approaches for improving engagement and acceptability of cCBT in primary care:

• identifying factors that impact on patient engagement, specifically personal preferences around working alone or with the support of health professionals

• improving patient experience and engagement through greater personalisation and provision of motivational support which could involve minimal health professional contact remotely or face to face

• encouraging early follow-up in primary care to identify which patients find the intervention unsuitable.

Some text has been reproduced from Knowles et al. 85 © BMJ Open 2015. This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/.

Trial-based estimates of clinical effectiveness for computer-delivered cognitive behaviour therapy

There was little or no evidence of statistically significant effects of Beating the Blues on the proportion of participants who were depressed across all points of follow-up when compared with usual GP care alone. For the free-to-use MoodGYM package, there was no evidence of effect in the primary outcome on the proportion of participants who were depressed at 4 months. There was some evidence of a statistically significant effect of MoodGYM on the proportion of participants who were depressed when compared with usual GP care at 12 months, but this was no longer evident at 24 months. When comparing MoodGYM and Beating the Blues, the free-to-use cCBT program MoodGYM was found to be non-inferior to the commercial pay-to-use cCBT program Beating the Blues across all time points. It is worth noting that, although the target sample size of 230 participants randomised to each trial arm was not achieved in the Beating the Blues arm (n = 210), the actual number of participants required for the analysis (n = 150 per arm) was achieved across all trial arms (Beating the Blues, n = 165; usual GP care alone, n = 179; and MoodGYM, n = 182).

Turning to the broader range of secondary outcomes which were collected in the REEACT trial, there was no evidence of statistically significant effects on the overall (including all time points) mean depression scores, CORE-OM scores and MCS and PCS quality of life scores of Beating the Blues when compared with usual GP care alone; however, there was evidence of a statistically significant effect of Beating the Blues on the MCS scores at 12 months. There was no evidence of statistically significant effects on the overall (including all time points) mean depression scores and PCS quality of life scores of MoodGYM when compared with usual GP care alone; however, there was evidence of a statistically significant effect of MoodGYM on the mean depression scores at 12 months when compared with usual GP care alone. When all time points were considered simultaneously in a mixed model, there was no clinical benefit for either MoodGYM or Beating the Blues when compared with usual GP care.

The clinical significance of the small but statistically significant clinical benefit of MoodGYM at 12 months was not clear. This finding of no benefit at 4 months followed by a small benefit at 12 months for MoodGYM was unexpected and was difficult to interpret. The primary outcome at 4 months is the point where any clinical benefit would most readily have been expected, and this finding is not routinely observed in trial-based evaluations of brief psychological intervention. Based on existing research and systematic reviews, the greatest amount of benefit would have been expected in the short term and any short-term benefits would have been expected to have attenuated by 12 months. 88,89

When baseline preference for treatment allocation was measured prior to randomisation, we found that there was a strong preference in favour of cCBT. This baseline preference did not impact on clinical outcomes within the trial. Computer-delivered CBT was equally ineffective for those with and those without a strong preference for treatment.

When the process of engagement with cCBT was monitored in the trial with reference to computer records it was found that there was low uptake and use of programs. Although the large majority of REEACT participants logged on and used the programs at least once (modal number of sessions = 1 for both programs), further usage was lower than that which has been reported in developer-led trials. Very few participants completed all treatment sessions, and the most common number of sessions for either computer program was just one session. This low usage was apparent despite the provision of regular (weekly) technical telephone support calls.

In summary, the main finding is, therefore, that for the primary outcome of depression severity at 4 months there was no significant additional benefit when participants were offered a technically supported form of computerised therapy in addition to usual GP care. This finding was true for both a free-to-use package (MoodGYM) and commercially produced cCBT (Beating the Blues).

Summary of trial- and model-based estimates of cost-effectiveness

The within-trial results of the economic analysis suggest that neither Beating the Blues nor MoodGYM appeared cost-effective compared with usual GP care alone. In the base-case results, Beating the Blues was dominated by usual GP care alone (with higher mean costs and lower QALYs associated with Beating the Blues), and mean costs and QALYs were lower for MoodGYM versus usual GP care alone, resulting in an ICER of £6933 per additional QALY for usual GP care alone versus MoodGYM. Usual GP care alone compared with either cCBT intervention was also the cost-effective intervention in the majority of scenario analyses, and it was the intervention most likely to be cost-effective at a £20,000 per QALY threshold. The exceptions to this were scenarios where SF-6D scores were used to estimate QALYs. In these scenarios, MoodGYM dominated usual GP care alone (lower mean costs and higher QALYs) and was the intervention most likely to be cost-effective at a £20,000 per QALY threshold.

Although the cost-effectiveness conclusions appear sensitive to the choice of whether EQ-5D or SF-6D is used to estimate them, the magnitude of the differences between all three groups were relatively minor (and non-significant) for both cost and QALY estimates across all the scenarios. Hence, minor differences in the assumptions can lead to different cost-effectiveness interpretations because of relatively small impacts on the mean incremental estimates of costs and QALYs, and some caution should be exercised when interpreting these results.

Results for the patient subgroup analyses were broadly consistent between the overall population and both patient preference subgroups. There was little evidence of interaction effects between treatment allocation and patient preference.

The evidence synthesis analysis results suggested that, based on external evidence of effectiveness and the impact of changes in PHQ-9 from baseline to 12 months on costs and QALYs (observed in the REEACT trial), therapist-guided cCBT may in fact be the most cost-effective treatment option compared with usual GP care alone and less-supported cCBT. These findings are tentative and are not based on directly randomised evidence.

Summary of main findings from qualitative examination of acceptability and use of computerised cognitive behaviour therapy

The qualitative patient interviews demonstrated a level of individual difference in experience of cCBT, with a ‘negative’ subset who consider therapy by computer unacceptable because of the absence of interpersonal therapeutic support, but another subset preferring cCBT to traditional therapy because of the greater accessibility, flexibility and anonymity. However, the majority of participants had a more mixed reaction. While they recognised the potential benefits of cCBT, they struggled with the content and delivery of the intervention, particularly with the lack of personalisation of content and the limited support and unstructured delivery of the program. Poor adherence and engagement were typical and were attributed to severity of depression and absence of follow-up support to encourage adherence. Even ‘positive’ participants reported that greater monitoring and follow-up than was provided in weekly supportive telephone calls would be needed to help sustain their engagement. Although the data suggested that addition of more intensive monitoring or supportive follow-up could improve engagement and completion, respondents did not think that this higher level of follow-up could feasibly be provided by GPs in primary care.

When GPs were interviewed, they perceived the attraction of cCBT to be its accessibility and flexibility, in terms of being the quickest treatment available to access, particularly to certain patients such as those who work long hours. GPs were sceptical that cCBT could ever be provided on site and perceived home access to be most appropriate and also preferable for patients. The GPs suggested that cCBT would be inappropriate for patients with more severe depression and most appropriate for patients high in computer literacy who were open to self-help interventions (as opposed to preferring to receive treatment from others).

Matching appropriate patients to cCBT treatment may need to be focused more on the individual needs and preferences of the patients, although the lack of congruence between patients’ expressed prior preference and their experience as positive or negative demonstrates the complexity of doing this. This may indicate a need for greater GP follow-up in the early stages of use, to more rapidly identify those patients who are now struggling with the intervention, which would be consistent with the principles of stepped care around self-correction9 when patients do not respond to minimal interventions.

The qualitative evaluation also found that support for engagement with cCBT in primary care may need to be provided by health professionals (other than GPs), or provided as supplemental remote or face-to-face support as part of the cCBT packages. Earlier feedback to GPs themselves around initial engagement with the cCBT program may also be necessary to help them follow the self-correcting principles of stepped care and enable alternative referrals for patients who are unable to complete the computerised intervention.

That the majority of GPs were sceptical that computerised therapy could be provided in their own practices demonstrates that in routine primary care it would be impractical and unlikely that cCBT would be provided to patients on site. It is possible that those patients who struggled with the flexibility of the home-accessed treatment would appreciate the greater structure offered by on-site access but, given the reported barriers to implementing this, it may again be the case that more remote support or motivational support built into the intervention would be most feasible and effective.

The combined interviews suggest several approaches for improving engagement and acceptability of cCBT in primary care.

Discussion of main findings

The REEACT trial offered an independent and pragmatic evaluation of cCBT. The clinical results of the trial were shown to be inconsistent with a number of developer-led trials, which have been summarised in systematic reviews90,91 and UK technology appraisals. 6 In addition to the independence of the REEACT trial from trials conducted by the developers of cCBT, there are a number of differences in the trial design of REEACT that are important to acknowledge.

The first difference in design is that the REEACT trial was purposely conducted entirely within primary care, which is the setting in which most cases of depression are managed. This focus on a primary care setting is in contrast with a number of trials which have recruited either from specialist (secondary care) referral centres or from centres that have developed specialist clinics requiring participants to be supervised in their use of the computer package.

The second design difference is the level of support that was offered alongside cCBT. In the REEACT trial, participants were offered regular (weekly) technical support and general encouragement to use the cCBT programs but were purposely not offered structured psychological support over the telephone by a mental health professional. Furthermore, the technical telephone support provided in the REEACT trial did not involve detailed explanations of CBT and did not involve detailed review of homework or between-session tasks. The chosen level of technical and motivational support was, therefore, less intense than other trials where, for example, computer usage has been supervised by therapists on a 1 : 1 basis92 or where a health-care professional has met with the user and has therefore been physically present to ensure the user interacts successfully with the computer. 7 Trials where there has been an intensive level of support have generally demonstrated a greater level of effect than less supported programs. 91 Nevertheless, the pragmatic nature of the REEACT trial meant that the level of support that was offered is one that is at least as intensive as the support offered in many NHS care settings and is in line with (or for MoodGYM exceeds) the level of recommended support. As such, the REEACT trial represents a realistic evaluation of an intervention that replicates the use of cCBT in routine primary care settings. The aim of conducting a pragmatic effectiveness trial of cCBT was, therefore, achieved.

Another important finding in this trial is that there was relatively low uptake of cCBT, despite the provision of technical telephone support. Other lower intensity interventions for depression have similarly shown that dropout and failure to engage are common when such interventions are offered in primary care. 79 The REEACT findings therefore serve to add to this existing evidence. However, the finding of the low uptake of cCBT in REEACT contrasts with other developer-led trials that report good levels of uptake and engagement. 79 Here we note again that REEACT was a pragmatic trial and one which replicated a feasible and representative primary care intervention within the context of a RCT. 7,93 The REEACT trial is a realistic evaluation of computer-delivered CBT, as it is currently offered in the NHS. Within the context of trials, there has been limited investigation of the reasons for poor engagement and barriers to the use of cCBT in routine care, and a concurrent qualitative evaluation in the REEACT trial explored reasons for poor engagement in depth. It was clear that participants with significant levels of depression were not generally motivated to engage in weekly unguided cCBT sessions. The technology was also not one which was sufficiently attractive to ensure that participants returned to further sessions after their initial session.

Conclusion

Computerised CBT forms a core component of stepped psychological care in the UK primary care and other health systems. The overall conclusion of the REEACT trial is that technically supported cCBT conferred modest or no benefit over usual GP care. This finding would therefore suggest that the routine promotion and commissioning of cCBT be reconsidered in the light of the results of this trial. A firm conclusion of the trial is that a commercially developed cCBT product conferred no benefit over a free-to-use product. This finding is particularly important for those who commission services and purchase commercial products on behalf of publicly funded health services. The REEACT trial results suggest that the routine use and purchase of cCBT is likely to be an ineffective low-intensity form of treatment for people with depression and, as such, an inefficient use of finite health-care resources. There are other treatments for depression that might be considered instead of cCBT, which includes telephone-guided self-help, bibliotherapy (unguided or standalone self-help) and therapist-delivered CBT. 86 We would recommend that more large-scale pragmatic trials of these low-intensity forms of guided self-help are conducted within routine NHS primary care services.

Implications for health care

• In this trial for patients with moderate or severe depression powered to detect non-inferiority, technically supported cCBT in addition to usual GP care was no more effective than usual GP care alone. Practice recommendations such as those offered by NICE and IAPT stepped models of care might usefully be re-examined in the light of these findings.

• We consider that, where cCBT continues to be offered within the portfolio of low intensity psychological treatment, there should be early follow-up in primary care to identify patients for whom the intervention may be unsuitable.

• Commissioners of services should take note of our findings that commercially produced products may add little benefit to usual GP care alone.

• We found no substantial difference in outcomes between the commercially produced product (Beating the Blues), when offered in addition to usual GP care, and the free-to-use product (MoodGYM), which is clearly less costly for the NHS.

• Free-to-use products such as MoodGYM could be offered in response to patient choice. However, our overall finding of the relative lack of benefit of these programs in addition to usual GP care should also be taken into account in this context.

Recommendations for research

• There is a clinical and economic need for brief, low-intensity self-help interventions for depression. The finding that computer-based therapies had low uptake and were not well regarded underlines both the need to further understand barriers to uptake of such therapies and the need for trials of alternative forms of guided self-help or brief psychological intervention.

• Although technically supported cCBT is largely ineffective and inefficient, a more highly supported form of cCBT with therapeutic advice, guidance and psychological support may in fact improve outcomes at an acceptable level of cost. Further trials of more intensively supported computer-delivered guided self-help are, therefore, required to test this hypothesis.

• Large-scale pragmatic trials of treatments such as bibliotherapy or telephone-based psychological interventions are, therefore, needed.

• All such studies should be framed in primary care and conducted by researchers other than product developers.

• In the longer term, if computers are to be used to deliver psychological treatment with minimal therapist input, then there needs to be improved patient experience and engagement through greater personalisation of treatment packages. This requires further research and innovation at the human–computer interface.

Some of this text has been reproduced from Gilbody et al. 11 © BMJ 2015. This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/.

Collaborators

The REEACT collaborators (current and past) are Phil Andersen (Trial Support Worker, Health Services Research), Professor Ricardo Araya (Professor, Psychiatry), Professor Michael Barkham (Professor, Clinical Psychology), Professor Peter Bower (Professor, Primary Care), Sally Brabyn (Research Fellow, Health Services Research), Gwen Brierley (Study Co-ordinator, Health Services Research), Dr Cindy Cooper (Director, Sheffield Clinical Trials Research Unit, Health Services Research), Dr Ana Duarte (Research Fellow, Health Economics), Professor Linda Gask (Professor, Primary Care Mental Health), Professor Simon Gilbody (Professor, Health Services Research), Dr Catherine Hewitt (Deputy Director, York Trials Unit, Statistics), Dr David Kessler (Consultant Senior Lecturer and GP, Primary Care), Dr Sarah Knowles (Research Fellow, Primary Care), Professor Helen Lester (Professor, Primary Care), Nicola Lidbetter (Chief Executive of Anxiety UK and Self-Help Services), Dr Elizabeth Littlewood (Research Fellow, Health Services Research), Professor Karina Lovell (Professor, Mental Health), Professor Stephen Palmer (Professor, Health Economics), Usman Muhammad (Research Fellow, Statistics), Professor Glenys Parry (Professor, Psychological Therapies), Professor David A Richards (Professor, Mental Health), Rachel Richardson (Senior Research Fellow, Health Services Research), Professor Mark Sculpher (Professor, Health Economics), Debbie Tallon (Trial Co-ordinator, Primary Care Mental Health), Dr Puvan Tharmanathan (Senior Research Fellow, Health Services Research), Professor David Torgerson (Director, York Trials Unit, Clinical Trials), Dr Simon Walker (Research Fellow, Health Economics), David White (Trial Co-ordinator, Clinical Trials).

Contributions of authors

Elizabeth Littlewood, Gwen Brierley, Rachel Richardson and Puvan Tharmanathan were the trial managers (Rachel Richardson between 2009 and 2010, Gwen Brierley between 2010 and 2011, Puvan Tharmanathan in 2011 and Elizabeth Littlewood between 2011 and 2013), monitored the whole trial and contributed to the design of protocol revisions and aquisition of data.

Ana Duarte and Simon Walker conducted the economic analysis.

Catherine Hewitt oversaw the conduct of the clinical analysis.

Sarah Knowles, Peter Bower, Helen Lester and Karina Lovell conducted the qualitative work and analysis.

Stephen Palmer designed and oversaw the conduct of the economic analysis.

Phil Andersen and Gwen Brierley provided technical telephone support to trial participants.

Stephen Palmer, Ricardo Araya, Michael Barkham, Peter Bower, Cindy Cooper, Linda Gask, David Kessler, Helen Lester, Karina Lovell, Glenys Parry, David A Richards, Rachel Richardson and Simon Gilbody were applicants on the HTA application and contributed to the study design.

Sarah Knowles, Sally Brabyn, Debbie Tallon and David White were the site trial co-ordinators and undertook data collection in their sites.

Catherine Hewitt and Usman Muhammad designed and conducted the clinical analysis.

Gwen Brierley was the trial support officer between 2009 and 2010.

Simon Gilbody and Rachel Richardson wrote the original protocol.

Simon Gilbody was the chief investigator, chaired the Trial Management Group and edited and approved the final draft of the report.

The report writing team consisted of Phil Andersen, Sally Brabyn, Ana Duarte, Simon Gilbody, Catherine Hewitt, Sarah Knowles, Elizabeth Littlewood, Stephen Palmer and Simon Walker, who drafted the report.

Contributions of collaborators

Nicola Lidbetter, Mark Sculpher and David Torgerson were co-applicants on the HTA application and contributed to the study design.

Mark Sculpher codesigned the economic analysis.

Statement of independence of researchers

The developers of the MoodGYM (National Institute for Mental Health Research, Australian National University) and Beating the Blues (Ultrasis) cCBT programs used in the study had no role in the design of REEACT or in the collection, analysis and interpretation of data.

Trial Steering Committee members

Professor Joe Reilly (Independent Chair), Professor of Mental Health, Durham University.

Professor Mike Slade (Professor of Health Services Research), Head of Section for Recovery, Health Service and Population Research Department, Institute of Psychiatry, King’s College London.

Professor Shirley Reynolds (Professor of Evidence-Based Psychological Therapies), University of Reading.

Professor Robbie Foy (Professor of Primary Care), Leeds Institute of Health Sciences, University of Leeds.

Dr Lina Gega, Clinical Lead, Northumberland Talking Therapies and Honorary Lecturer at Norwich Medical School.

Mr Nic Seccombe (service-user representative), Informatics and Governance Lead, Self Help Services, Manchester; Anxiety UK, cCBT self-help services.

Data Monitoring and Ethics Committee members

Professor Mike Lucock (Independent Chair), Professor of Clinical Psychology, University of Huddersfield, and Associate Director of Research, South West Yorkshire Partnership NHS.

Professor Chris Roberts (Professor in Biostatistics, Centre for Biostatistics), University of Manchester.

Dr Paul Blenkiron (Consultant Psychiatrist and Honorary Senior Clinical Lecturer), Hull York Medical School.

Publication

Brierley G, Richardson R, Torgerson D. Using short information leaflets as recruitment tools did not improve recruitment: a randomised controlled trial. J Clin Epidemiol 2012;65:147–54.

Gilbody S, Littlewood E, Hewitt C, Brierley G, Tharmanathan P, Araya R, et al. Computerised cognitive behaviour therapy (cCBT) as treatment for depression in primary care (REEACT trial): large scale pragmatic randomised controlled trial. BMJ 2015;351:h5627.

Knowles SE, Lovell K, Bower P, Gilbody S, Littlewood E, Lester H. Patient experience of computerised therapy for depression in primary care. BMJ Open 2015;5:e008581.

Disclaimers

This report presents independent research funded by the National Institute for Health Research (NIHR). The views and opinions expressed by authors in this publication are those of the authors and do not necessarily reflect those of the NHS, the NIHR, NETSCC, the HTA programme or the Department of Health. If there are verbatim quotations included in this publication the views and opinions expressed by the interviewees are those of the interviewees and do not necessarily reflect those of the authors, those of the NHS, the NIHR, NETSCC, the HTA programme or the Department of Health.

Scenario 1

Scenario 2

Scenario 3

Scenario 4

Scenario 5