Cost and Outcome of BehaviouRal Activation (COBRA): a randomised controlled trial of behavioural activation versus cognitive behavioural therapy for depression

Article history

The research reported in this issue of the journal was funded by the HTA programme as project number 10/50/14. The contractual start date was in April 2012. The draft report began editorial review in October 2016 and was accepted for publication in March 2017. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The HTA editors and publisher have tried to ensure the accuracy of the authors’ report and would like to thank the reviewers for their constructive comments on the draft document. However, they do not accept liability for damages or losses arising from material published in this report.

Declared competing interests of authors

All authors report grants from the National Institute for Health Research (NIHR) during the course of the study. David A Richards reports grants from the European Science Foundation. David A Richards and Rod S Taylor have received funding support from NIHR Collaborations for Leadership in Applied Health Research and Care. David A Richards reports NIHR Clinical Development and Senior Clinical Fellowship and Senior Investigator Panel memberships. Rod S Taylor reports membership of NIHR Health Technology Assessment (HTA) programme themed call, NIHR HTA Efficient Study Designs Board and NIHR Health Services and Delivery Research Commissioning Boards. Simon Gilbody reports membership of the NIHR HTA Evidence Synthesis Board and NIHR HTA Efficient Study Designs Board. Willem Kuyken reports fees from Guilford Press for book royalties and Collaborative Case Conceptualisation.

Disclaimer

This report contains transcripts of interviews conducted in the course of the research and contains language that may offend some readers.

Copyright statement

© Queen’s Printer and Controller of HMSO 2017. This work was produced by Richards et al. under the terms of a commissioning contract issued by the Secretary of State for Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science Park, Southampton SO16 7NS, UK.

Scientific background and review of current literature

Clinical depression is one of the most common and debilitating of the psychiatric disorders. It accounts for the greatest burden of disease among all mental health problems, and is the second largest cause of global disability. 3 Lifetime prevalence has been estimated at 16.2% and rates of comorbidity and risk for suicide are high. 4–6 Depression is often recurrent, and without treatment many cases become chronic, lasting > 2 years in one-third of individuals. Over three-quarters of all people who recover from one episode will go on to have at least one more. 7 In the UK, depression and anxiety are estimated to cost the economy £17B in lost output and direct health-care costs annually, with a £9B impact on the Exchequer through benefit payments and lost tax receipts. 8 Globally, the economic impact of depression on aggregate economic output is predicted to be US$5.36 trillion between 2011 and 2030. 9

Reducing these substantial costs is a key objective for low-, medium- and high-income countries alike. Antidepressant medication (ADM) and cognitive–behavioural therapy (CBT) are the two treatments with most evidence of effectiveness, both of which are recommended by the National Institute for Health and Care Excellence (NICE). 10 Problems with ADM include side effects, poor patient adherence and relapse risk on ADM discontinuation. Service user organisations and policy think tanks advocate greater availability of psychological therapies, which many people prefer. 11 CBT, which is of similar efficacy to ADM,12 has several advantages: (1) it reflects the desire of many service users for non-pharmacological treatment; (2) it has no physical side effects; and (3) it modifies the illness trajectory in that benefits continue after the end of treatment, preventing recurrence. 13 However, CBT has several disadvantages: (1) its complexity makes it difficult to learn to implement in a competent fashion; (2) its efficacy is dependent on the skill of the individual practitioner; (3) patients are required to learn quite high-level skills; and (4) the high cost of training and employing sufficient therapists limits access to CBT.

As a consequence of the disadvantages above, many people do not receive adequate treatment, and, even when treatment is given, many respond only partially or not at all. 14 Despite the recent government initiative in England – ‘Improving Access to Psychological Therapies’ (IAPT; URL: www.iapt.nhs.uk/) – no more than 15% of people with depression will receive NHS-delivered CBT, and only 50% receiving CBT will recover. 15 It is therefore important to continue to test promising new treatments, especially if there are indications that such treatments reduce the risk of symptom return, are applicable to a wide range of depressed people including those with severe disease, are easy to implement in clinical practice and are therefore potentially more accessible,16 and are a cost-effective use of resources.

Globally, health services require effective, easily implemented and cost-effective psychological treatments for depression that can be delivered by less specialist health workers in order to close a treatment gap that can be as much as 80–90% in some low-income countries. 17 The English NHS, in order to meet public and professional expectations, requires a simple, equivalently effective, easily implemented psychological treatment for depression which can be delivered by less specialist (albeit appropriately competent) junior mental health workers (MHWs) to treat many more people with depression in a more cost-effective manner.

Rationale for the research

Behavioural activation (BA) is a psychological treatment based on behavioural theory that alleviates depression by focusing directly on changing behaviour. 18–20 This theory states that depression is maintained by avoidance of normal activities. As people withdraw and disrupt their basic routines, they become isolated from positive reinforcement opportunities in their environment. The combination of increased negative reinforcement with reduced positive reinforcement results in a cycle of depressed mood, decreased activity and avoidance which maintains depression. 19 BA systematically disrupts this cycle, initiating action in the presence of negative mood, when people’s natural tendency is to withdraw or avoid. 21,22 Although CBT incorporates some behavioural elements, these focus on increasing rewarding activity and initiating behavioural experiments to test specific beliefs. In contrast, BA targets avoidance from a contextual, functional approach not found in CBT (i.e. BA focuses on understanding the function of behaviour and replacing it accordingly). BA also explicitly prioritises the treatment of negatively reinforced avoidance and rumination. Furthermore, the BA rationale is easier to understand and operationalise for both patients and MHWs than CBT, which also focuses on increasing activity, but primarily on changing maladaptive beliefs. 23 Moreover, there is some evidence that CBT is less effective when delivered by less competent therapists. 12,24

In the UK, CBT is delivered by professionally qualified senior MHWs (mainly clinical psychology, nursing, occupational therapy, social work or counselling), who have obtained a further 1-year, full-time postgraduate qualification in CBT. Their training is long and expensive and their employment grade is costly compared with junior MHWs, who deliver much of the routine mental health care in the UK. The relative simplicity of BA treatment may make it easier and cheaper to train junior MHWs in its application than CBT, the argument of ‘parsimony’ first advanced by one of the early proponents of this approach, Neil Jacobson. 19 However, this is appropriate only if BA delivered in this way is as effective as, and more cost-effective than, CBT.

Limitations of previous trials

A number of systematic reviews have attempted to address the more general question of BA effectiveness compared with CBT. 10,25–28 All have commented on the relatively poor quality of component studies. We conducted a meta-analysis of randomised controlled trials (RCTs) of BA,25 and identified 12 studies with a total of 476 patients. At the primary end point we found no difference between the groups on depression symptom level [Hedges’ g = 0.102, 95% confidence interval (CI) −0.122 to 0.326; I² = 29%; p = 0.372] (Figure 1). At follow-up we found no difference between the groups on depression symptom level (Hedges’ g = 0.395, 95% CI −0.032 to 0.822; I² = 61%; p = 0.070).

FIGURE 1.

Meta-analysis of pre-COBRA (Cost and Outcome of BehaviouRal Activation) trial BA vs. CBT primary outcome point, all included studies. AT, automatic thoughts; LS, low severity.

In a subsequent update of this review26 we found no additional trials comparing BA with CBT. Many of the trials included in our review were of limited methodological quality, all were underpowered for comparing treatments, and most did not utilise diagnostic interviews for trial inclusion. Treatments in many cases did not conform to modern clinical protocols for BA. Long-term outcomes were rarely reported, with average follow-up only to 4 months. These results have been replicated in two recent Cochrane reviews of behavioural therapies,27,28 which concluded that there was only low- to moderate-quality evidence that behavioural therapies and other psychological therapies were equally effective and called for ‘Studies recruiting larger samples with improved reporting of design and fidelity to treatment to improve the quality of the evidence’. 27

Most significantly, NICE10 reviewed the same evidence and regarded only a small subset of three trials31,38,40 as of sufficient quality to be able to contribute evidence of effect (Figure 2). In those studies no difference was found between BA and CBT at primary end point (Hedges’ g = 0.139, 95% CI –0.4.00 to 0.122; I² = 1%; p = 0.296) or at follow-up (Hedges’ g = 0.135, 95% CI –0.456 to 0.186; I² = 0%; p = 0.409).

FIGURE 2.

Meta-analysis of pre-COBRA (Cost and Outcome of BehaviouRal Activation) trial BA vs. CBT, subset of high-quality included studies in NICE review. AT, automatic thoughts; LS, low severity.

The conclusion of the NICE Guideline Development Group was that that the evidence base for BA was not ‘sufficiently robust’ for it to be recommended as an alternative to CBT. It was suggested that BA could be an option for clinicians, but the limited evidence base should be considered when making this treatment choice. 10 Consequently, NICE made a clear research recommendation ‘to establish whether behavioural activation is an effective alternative to CBT’ using a study which is ‘large enough to determine the presence or absence of clinically important effects using a non-inferiority design’ (p. 256). 10

Pilot work preceding this trial

In order to test uncertainties around our main objectives, we piloted BA in a Phase II RCT to examine whether or not MHWs without previous specialist training in psychological therapy can effectively treat depressed people using BA. 41 We compared BA against usual care. Relatively junior NHS MHWs (‘band 5’ – equivalent to a basic grade, qualified mental health nurse) with no previous formal training or experience in psychotherapy delivered BA. These workers received 5 days’ training in BA and, subsequently, 1 hour of clinical supervision, fortnightly, from a clinical nurse consultant or trained psychotherapist. Intention-to-treat (ITT) analyses indicated a difference in favour of BA of −15.79 points (n = 47, 95% CI −24.55 to −7.02 points) on depression (as measured via the Beck Depression Inventory-II), an effect size of −1.15 standard deviation (SD) units (95% CI −0.45 to −1.85 units). We also found a quality-adjusted life-year (QALY) difference in favour of BA of 0.20 points (95% CI 0.01 to 0.39 points; p = 0.042), incremental cost-effectiveness ratio (ICER) of £5756 per QALY and a 97% probability that BA is cost-effective at a threshold value of £20,000. 41

Conclusion

From our literature reviews and pilot work we concluded that BA was a potentially viable treatment for depression when delivered by junior MHWs, but that, as NICE had suggested, a non-inferiority trial of BA versus CBT was required to test whether or not BA was non-inferior to CBT and if BA could be a potentially cost-effective alternative to CBT for depression. We now report the results of this randomised trial to determine if BA is non-inferior to CBT in the treatment of patients with depression. This report is divided into chapters detailing the methods and results of our primary clinical effectiveness and cost-effectiveness questions followed by a chapter for our process evaluation. We conclude with a discussion chapter summarising our results and considering their implications for the treatment of depression in the UK and internationally.

Research objectives

1. To assess the clinical effectiveness of BA compared with CBT for depressed adults in terms of depression treatment response at 12 and 18 months.

2. To assess the cost-effectiveness of BA compared with CBT in terms of QALYs at 18 months.

In addition, we undertook a secondary process evaluation to investigate the moderating, mediating and procedural factors in BA and CBT that influence outcome, the methods for which are covered in Chapter 4.

Study design

We undertook a research assessor-blinded, multicentre, two-arm, non-inferiority, patient-level RCT for people with depression, to test the effectiveness of a psychological intervention for depression, BA, against the current gold standard, evidence-based treatment, CBT. We included clinical, economic and process evaluations. The rationale for a non-inferiority trial is that we needed to establish whether or not the clinical effectiveness of BA is not substantially inferior to CBT. Therefore, we powered our trial on the basis of clinical non-inferiority, and analysed our data accordingly. 42,43

Patient and public involvement

We involved patient and public involvement (PPI) representatives at all stages of the project. A PPI advisor (NR) was a full member of the Trial Management Group (TMG). He attended all meetings of the TMG and advised on patient-facing materials, including ethics materials and participant therapeutic manuals, and on the conduct of the trial including project management, questionnaire development, data collection and project dissemination. There was a PPI representative on the Trial Steering Committee (TSC) from a depression consumer advocacy group who provided important checks and balances as part of the independent TSC oversight of the trial.

All sites had excellent local PPI mechanisms. We followed national good practice guidance for researchers on public involvement in research and the paying of PPI representatives actively involved in research. 44 We also worked with our PPI representatives to ensure that our dissemination strategies were inclusive and accessible to other people who use services. In addition, the trial was co-ordinated from the University of Exeter’s Medical School. The Medical School operates within a culture of PPI – guided by published theories of participation, empowerment and engagement – through the National Institute for Health Research (NIHR) Collaborations in Leadership in Applied Health Research and Care for the Peninsula Public Involvement Group.

Setting and participants

We recruited participants over a 20-month period from September 2012 to April 2014. Potential participants were identified by clinical studies officers (CSOs) or practice staff from the electronic case records of primary care and psychological therapy services in Devon, Durham and Leeds, indicating that the person had been identified as currently depressed at least once during the previous 2 months. Practices or services contacted patients to seek permission for researcher contact. The research team interviewed those who responded, provided detailed information on the study, took informed consent and assessed people for eligibility.

Inclusion and exclusion criteria

Randomisation, concealment of allocation and blinding

Following interview, participants were allocated in a 1 : 1 ratio to either the BA or CBT arm stratified according to their symptom severity on the Patient Health Questionnaire-9 (PHQ-9)46 (PHQ-9 of < 19 vs. ≥ 19 points), ADM use (currently using ADM or not) and recruitment site. The registered Peninsula Clinical Trials Unit (PenCTU) allocated participants remotely after the researchers had collected and entered baseline data into a computer database to ensure researcher blinding and allocation concealment. Investigators were not informed of participants’ allocations. The computer-based system allocated the first 20 participants to each arm on a truly random basis. For subsequent participants, allocation was minimised to maximise the likelihood of balance in stratification variables across the two study arms. Concealment was ensured by the use of a password-protected trial website and retaining a stochastic element to the minimisation algorithm. The computer-based allocation and website were set up and maintained by PenCTU, independent of the trial. The participant’s details were then sent to the relevant MHW to alert them to contact this person and begin treatment. The general practitioner (GP) was then informed of their patient’s involvement in the study.

In this type of trial, in which interventions are complex and clearly different from each other, it is not possible to blind participants or clinicians, so our procedures focused on helping to keep research workers blind to participant allocation and by protecting the study against assessment interpretation bias through the use of self-report measures. All research measures were applied to both groups, and researchers were instructed to maintain blindedness by reminding participants at follow-up of the need not to discuss their treatment with the researcher. We recorded instances where researchers were unblinded by patients disclosing their treatment during interviews.

Sample size calculation

We estimated the non-inferiority margin for the primary outcome (PHQ-9) using two potential approaches with reference to (1) the effect size of historical trials comparing BA versus control; and (2) the published minimum clinically important difference for the primary outcome (PHQ-9) of 2.59 to 5.00 points. 47 Based on our meta-analysis, BA was superior to control in depression score by a mean of 0.7 SD units (95% CI 0.39 to 1.00 SD units) or 3.8 PHQ-9 points (95% CI 2.1 to 5.4 PHQ-9 points) (assuming a SD of 5.4 from Lowe et al. ). 47 It has been proposed that non-inferiority margins be taken as ≈0.5 × mean control effect size (i.e. 0.5 × 3.8 = 1.90 points) or as the lower 95% limit of the control effect size (i.e. 2.1 points). 48,49 To ensure the adequacy of this trial to test non-inferiority between BA and CBT, we therefore examined a number of potential scenarios taking into account the potential uncertainty in the non-inferiority margin for the primary outcome.

We selected a conservative non-inferiority margin of 1.90 points and power of 90%. As a consequence, we needed to recruit a total of 440 participants to detect a between-group non-inferiority margin of 1.90 points in PHQ-9 at one-sided 2.5% alpha, allowing for 20% attrition caused by dropouts and protocol violators. Furthermore, although previous trials of CBT have shown little or no effects of clustering in outcome by therapists, even when delivering group CBT,50,51 if we were to assume a small therapist clustering effect (i.e. intracluster correlation coefficient of 0.01), this sample size would still have 80% power for a non-inferiority margin of 1.90 points on the PHQ-9 at one-sided 2.5% alpha, allowing for 20% attrition.

Our sample size was inflated by 20% for participant dropout to take account of participants who might exit the trial and refuse follow-up assessment, although our experience in running large primary care trials of depression treatment is that attrition rates would be less than this. Therefore, we planned to recruit 440 participants to the trial, 220 per arm. A summary of the sample size calculations is provided in Table 1.

Recruitment

Randomised controlled trials are vulnerable to selection bias and threats to external validity if there are systematic differences in behaviour between referring clinicians. We minimised this potential bias by recruiting participants through searching general practice records and referral logs from primary care to local depression and anxiety treatment services rather than by direct referral from GPs. We identified suitable participants by examining electronic case records for all patients in each general practice or treatment service. The search was conducted by practice staff or Clinical Research Network CSOs, identifying people with at least one identification code for depression recorded against their name in the last 2 months. In primary care practices we searched for the codes most widely used by GPs to classify participants as depressed. The list of potentially suitable participants was reviewed by GPs to identify any patients whom had known exclusion criteria. The remaining patients were written to, inviting them to take part in the study. For patients already referred to local psychological therapies services, we contacted those on the waiting list. Letters were sent with a short participant information sheet, stamped and addressed envelope and a ‘Permission for Researcher to Contact’ form to allow a researcher to contact them. If potential participants did not return the form, they were contacted by telephone by practice staff or practice-based Clinical Research Network CSOs to check that they had received the letter and asking them if they wished to participate in the Cost and Outcome of BehaviouRal Activation (COBRA) trial. Potential participants identified were interviewed by researchers on the telephone to confirm the presence of depressive symptoms and to explain the trial fully. If positive on the screen, potentially eligible participants were interviewed face to face by researchers to confirm eligibility, take consent, conduct a diagnostic interview and collect baseline measures. Eligible, fully informed and consenting participants were then entered into the study and randomised.

From our experience of previous trials, we calculated that 37% of potential participants interviewed at baseline would be likely to decline participation, would not meet our inclusion criteria or would meet one of the exclusion criteria. Therefore, we were required to interview 700 potential participants in order to induct our planned sample size of 440 eligible participants into the trial. Following random allocation of 440 participants, a maximum of 20% attrition would lead to our target sample size of 366 participants.

In order to identify 700 people for baseline interview, we planned to contact around 3400 potential participants through letter and/or telephone to inform them of the trial and offer them the chance to participate. In order to do so, we needed to identify 5300 potential participants from a sensitive coded search of practice case note records, as our existing data predicted that 1900 (approximately 36%) of these would be excluded by GPs against known trial exclusion criteria. Identifying 5300 potential participants was expected to generate at least 700 positive replies.

For an average-size practice of 7000 registered patients, we expected that searches would be likely to identify around 37 potentially eligible participants per search. Four searches per practice would, therefore, identify 148 potential participants per practice. Consequently, we planned to recruit at least 36 practices (12 per site) to identify sufficient potential participants to meet our target number of 5300.

Trial interventions

We developed our BA and CBT intervention protocols in line with (a) published treatment protocols,19,21,22,40,53,54 including that developed for BA and CBT in our trials;41,55 (b) advice from national and international collaborators (Martell, Dimidjian, Hollon); and (c) NICE recommendations10 for duration, and frequency, of BA and CBT. To recognise realities of real-world clinical presentations, our protocols included behavioural and cognitive strategies for managing comorbidity, particularly anxiety, where this is present in addition to depression. Therapists were able to provide participants with a maximum of 20 sessions over 16 weeks with the option of four additional booster sessions. 10

Outcomes

An overview of our data collection timings is presented in Table 2.

Baseline information

We collected demographic data at baseline through a purposely designed form. We recorded data on gender, age, ethnic origin, education level, employment, marital status, number of children, presence or absence of ADM treatment, previous history and age at onset of depression, duration of any ADM treatment and presence of any comorbid anxiety disorder.

Clinical data

We conducted follow-up assessments at 6, 12 and 18 months post-baseline assessment. Our primary outcome was self-reported depression severity and symptomatology, as measured by the PHQ-9,46 at 12 months. The PHQ-9 is a nine-item questionnaire that records the core symptoms of depression with established excellent specificity and sensitivity characteristics in a UK population. 57 Our secondary outcomes were DSM-IV MDD status and number of depression-free days between follow-ups, assessed by the SCID,45 anxiety assessed by the Generalised Anxiety Disorder-7 (GAD-7) questionnaire58 and health-related quality of life assessed by Short Form questionnaire-36 items (SF-36). 59 We also assessed the relative proportions of participants meeting criteria for ‘recovery’ (proportions of participants with PHQ-9 scores of ≤ 9 points) and ‘response’ (50% reduction in scores from baseline) on the PHQ-9. We also recorded the presence of DSM-IV anxiety disorders at baseline and follow-ups.

Economic data

We took the UK NHS and Personal Social Services (PSS) perspective consistent with the UK NICE reference case. 60 A broader societal perspective was explored in sensitivity analysis to capture the effects of productivity loss as a result of time off work due to illness, as depression is known to impact on an individual’s ability to work and can result in substantial losses in the workplace. 61 In addition, the use of complementary therapies was included in a further sensitivity analysis following advice from the clinical team that such therapies are commonly used by adults with depression. Narrower perspectives of intervention and mental health care were also examined in sensitivity analyses.

We collected participants’ use of BA and CBT from clinical records, with additional resource information (e.g. training, supervision and other non-face-to-face activities) collected from therapists and trainers. We measured all other health and social care services used, including medication prescription using the adult service use schedule (AD-SUS), designed on the basis of previous evidence of service use in depressed populations. 62 We measured productivity losses using the absenteeism and presenteeism questions of the World Health Organization’s Health and Work Performance Questionnaire. 63 The AD-SUS and the Health and Work Performance Questionnaire were completed by patients in interviews with a research assessor at baseline and at the 6-, 12- and 18-month follow-ups. At baseline, participants were asked to report service use over the previous 6 months. At all follow-up points, participants were asked to report service use since the last interview, to capture service use for the entire period from baseline to follow-up, even if participants had missed intermediate interviews.

We measured effectiveness for the economic evaluation in terms of QALYs calculated using the EuroQol-5 Dimensions, three-level version (EQ-5D-3L), a non-disease-specific measure for describing and valuing health-related quality of life, at baseline and at the 6-, 12- and 18-month follow-ups. 64 The EQ-5D-3L consists of five dimensions in the domains of mobility, self-care, usual activities, pain/discomfort and anxiety/depression, each scored on three levels (no problems, some problems or extreme problems), and classifies individuals into one of 243 health states. Health states are converted into a single summary index utility score by applying weights to each level in each dimension derived from the valuation of EQ-5D-3L health states in adult general population samples. 65 QALYs were calculated as the area under the curve defined by the utility values at baseline and each follow-up. It was assumed that changes in the utility score over time followed a linear path. 66

Process data

In addition to information on age at depression onset and number of previous episodes collected using the SCID,45 we also collected data on changes in specific behaviour [as assessed via the Behavioural Activation for Depression Scale (BADS)],67 changes in beliefs [as assessed via the Dysfunctional Attitudes Scale (DAS)],68 rumination [as assessed via the Ruminative Response Scale (RRS)],69 hedonic tone [as assessed via the Snaith–Hamilton Pleasure Scale (SHAPS)],70 acceptability of BA and CBT for participants and clinicians (assessed with qualitative methods), and per protocol (PP) treatment adherence (from therapist case records). We collected qualitative data via semistructured interviews to access participants’, BA MHWs’ and CBT therapists’ accounts of the mechanisms and impacts of treatment. Interviews focused on acceptability, views of the role of cognitive and behavioural change strategies and broader impacts of treatment in participants’ lives.

Intervention fidelity

We assessed the quality of, and adherence to, BA and CBT clinical protocols using audiotapes and written records of therapy sessions, which MHWs and therapists were instructed to take, with participant permission, for each clinical session. A random sample of tapes, stratified by therapist, therapy session and intervention, were sent to independent experts in both treatments for competency rating using the Cognitive Therapy Scale-Revised (CTS-R)71 (range 0–78, competency cut-off score = 36) for CBT and the Quality of Behavioural Activation Scale72 (range 0–84, competency cut-off score = 42) for BA. Independent rating of recorded therapy sessions was undertaken by the Oxford Cognitive Therapy Centre (Oxford Health NHS Foundation Trust) and Dr Christopher Martell of the University of Wisconsin–Milwaukee, for CBT and BA, respectively.

Therapy ratings were used for several different purposes:

1. to assess each therapist’s competency, at the start of the trial, to deliver the interventions by rating the first two therapy sessions undertaken

2. to monitor whether or not levels of competence/adherence were maintained throughout the trial.

We asked all therapists in both treatment groups to record their in-session activity by completing specially designed therapy record sheets. These sheets included a list of therapeutic techniques specific to each type of therapy, with a tick box against each element. Therapists indicated which of the techniques they had used in each session.

Safety and adverse events

For adverse events (AEs), we recorded deaths from whatever cause, and all self-harm and suicide attempts. The independent Data Monitoring Committee (DMC) reviewed all AEs and made relevant trial conduct recommendations as a consequence.

Data analysis

We analysed and report primary and secondary outcomes in accordance with Consolidated Standards of Reporting Trials (CONSORT) guidelines for non-inferiority and equivalence trials. 43 All analyses were carried out using an a priori statistical analysis plan prepared in the first 6 months of the trial and agreed with the TMG, TSC and the DMC.

Equivalence of baseline characteristics and outcomes in the two groups were assessed descriptively. As differences between randomised groups at baseline could have occurred by chance, no formal significance testing was conducted. We also undertook a descriptive analysis of the baseline patient characteristics according to the recruitment method (recruitment from psychological therapies waiting list vs. GP case note review).

We undertook both ITT and PP analyses. PP analysis provides some protection for any theoretical increase in the risk of type I error (erroneously concluding non-inferiority). 73 The ITT population was defined as all randomised patients in the groups to which they were allocated with observed outcome data at follow-up. The PP population was predefined by the TMG as those patients who met the ITT definition and received eight or more treatment sessions for both groups. Although the CONSORT guidelines recommend a PP approach (i.e. analysis according to actual treatment received) as the conservative non-inferiority analysis option, given the potential biases of both PP and ITT analyses,43 we took the approach of the European Medicines Agency, that security of inference depends on both PP and ITT analyses demonstrating non-inferiority of the primary outcome. 74 We, therefore, checked for non-inferiority in both the PP and ITT populations. In order to check the security of inference of non-inferiority, sensitivity analysis for the primary outcome was undertaken for the ITT imputed population and PP analyses based on different definitions of adherence/protocol adherence.

The TMG predefined our PP population. We also conducted sensitivity analyses using different definitions of PP adherence. We included varying proportions of PP participants in these sensitivity analyses populations, depending on how much of each therapy they had received, ranging from 40% to 100% of planned therapy sessions. Our analysis plan specified that, if non-inferiority was consistently shown by these analyses, we would proceed to assess superiority of CBT compared with BA (i.e. the CI lower bound lies above 0). If we found that conclusions were inconsistent across analyses, we planned to revert back to primacy of the PP analysis to confirm or refute the non-inferiority hypothesis.

Our primary analysis compared observed primary and secondary outcomes between BA and CBT groups at 12 months after randomisation using linear regression models that adjusted for baseline outcome values and stratification/minimisation variables (symptom severity, site, ADM use). Although we initially planned to include therapist as a random-effects variable in our models, given the low levels of observed clustering we took a parsimonious approach and fitted our models without inclusion of therapist. We also checked that there was no difference in inference with and without the inclusion of a random-effects therapist term.

We estimated that the one-sided 97.5% CI for the between-group difference and non-inferiority of BA compared with CBT was accepted (in a 0.025 level test) if the lower bound of the 97.5% CI lay within the non-inferiority margin of –1.90 points in PHQ-9 score. We checked for non-equivalence of the primary outcome at all follow-up points using the same approach. We extended the primary analysis models to fit interaction terms to explore possible differences in treatment effect in baseline symptom severity and ADM usage.

We undertook secondary analyses to compare groups at follow-up across 6, 12 and 18 months using a mixed-effects, repeated measures regression approach. We also ran sensitivity analyses for both primary and secondary analyses to assess the likely impact of missing data using multiple imputation models. We also calculated the relative proportions of participants meeting criteria for ‘recovery’ (proportions of participants with PHQ-9 scores of ≤ 9 points) and ‘response’ (50% reduction in the PHQ-9 scores from baseline). Between-group differences are presented for continuous outcomes as CBT versus BA (i.e. CBT minus BA) and for binary outcomes as BA relative to CBT (i.e. BA divided by CBT).

No interim inferential analyses were undertaken. However, the DMC requested (October 2013) a check of the statistical power of the trial for the PP analyses. This calculation used the assumptions of our original power calculation and was based on the observed level of attrition of the primary outcome at 12 months and the proportion of patients who fulfilled the PP definition (eight or more treatment sessions; as of January 2014). All analyses were undertaken using Stata v.14 (StataCorp LP, College Station, TX, USA).

Economic analysis

Although studies designed to test equivalence of effects are considered to be a legitimate situation in which a cost minimisation analysis (where costs alone are compared given equal outcomes) may be appropriate,75 the same may not be true for non-inferiority designs. Even in situations where equivalence or non-inferiority are demonstrated, exploration of the joint distribution of costs and effects in a cost-effectiveness analysis is recommended to represent uncertainty75 and to help interpret the economic results. 76 For these reasons, we prespecified that we would undertake a cost-effectiveness analysis irrespective of whether or not non-inferiority in the primary clinical outcome was demonstrated. We assessed cost-effectiveness in terms of QALYs using the net benefit approach. 77 We explored Bosmans et al. ’s methods76 for economic evaluations alongside equivalence or non-inferiority trials, which requires specification of non-inferiority margins for both costs and effects. However, as our prespecified method of economic evaluation was cost–utility analysis, using QALYs, rather than cost-effectiveness analysis, using the PHQ-9, which was the measure on which the hypothesis of non-inferiority is based, no non-inferiority margin for economic effects was specified. In addition, there is a general lack of guidance on how to define an economically unimportant difference in costs with which to estimate an appropriate non-inferiority margin for costs.

We compared the costs and cost-effectiveness of BA and CBT at the final, 18-month, follow-up to capture the economic impact of events, such as relapse with unit costs from the 2013–14 financial year. 78,79 We discounted costs and QALYs in year 2 at 3.5%. 60 We used complete-case analysis, with missing data explored in a sensitivity analysis using multiple imputation by chained equations (MICE). Our primary analysis took the NHS/PSS perspective preferred by NICE. 80 The impact of productivity losses as a result of time off work, known to be a substantial cost in depression,81 were explored in sensitivity analysis. In addition, narrower cost perspectives were tested (e.g. an intervention perspective and a mental health-care perspective), to ensure that the NHS/PSS perspective had not captured irrelevant costs that may hide the true impact of BA and CBT on service use.

For each participant, a unit cost was applied to each item of service use reported to calculate the total cost for the duration of the trial. All unit costs are summarised in Table 3.

We estimated intervention costs using the bottom-up costing approach set out by the Personal Social Services Research Unit at the University of Kent. 82 We based BA MHW costs on NHS AfC salary band 5 (salary range: £21,909–28,462; US$31,662–41,130; €27,726–35,993) and NHS AfC band 7 (salary range: £31,383–41,373; US$45,350–59,786; €39,738–52,388) for CBT therapists, including employer’s National Insurance and pension contributions plus capital, administrative and managerial costs. 79 We calculated a cost per hour using standard working time assumptions,79 weighted to account for time spent on non-patient facing activities, which was estimated based on the results of a survey of trial therapists.

We costed hospital services using unit costs from the NHS Reference Costs 2013–14. 78 Unit costs for NHS primary care and social care services were taken from nationally applicable published sources. 79 Costs for complementary services were taken from the NHS Choices website. 83 The costs of medications were calculated based on averages listed in the British National Formulary84 for the generic drug and using daily dose information collected using the AD-SUS.

We valued productivity losses as a result of time off work due to illness using the human capital approach, which involves multiplying the individual’s salary by reported days off work due to illness. 85

We report differences in use of services between randomised groups descriptively as the mean by group and as a percentage of the group who had at least one contact. We tested for differences in mean costs per participant between groups using standard parametric t-tests, with the results confirmed using bias-corrected, non-parametric bootstrapping. 86 This is the recommended approach, despite the skewed nature of cost data, as it allows inferences to be made about the arithmetic mean. 87

We explored cost-effectiveness using ICERs – the difference in mean cost divided by the difference in mean effect88 – and cost-effectiveness planes constructed to show the probability that BA is more or less effective and more or less costly than CBT. As ICERs are calculated from four sample means and are therefore subject to statistical uncertainty, the planes were generated using 1000 bootstrapped resamples from regression models of total cost and outcome by treatment group. These were then used to calculate the probability that each of the treatments is the optimal choice, for different values a decision-maker is willing to pay for a unit improvement in outcome (the ceiling ratio, λ). Cost-effectiveness acceptability curves (CEACs) are presented by plotting these probabilities for a range of possible values of λ to explore the uncertainty that exists around estimates of mean costs and effects, and to show the probability that BA is cost-effective compared with CBT. 89

All analyses were controlled for the following covariates: site, ADM use, symptom severity and baseline measurement of the variables of interest. Additionally, data have been truncated to exclude influential outliers (i.e. cases with total costs in the 99th percentile that make a significant difference to the results). 90 Between-group differences for costs are presented for continuous outcomes as BA versus CBT (i.e. BA minus CBT).

We carried out a number of independent sensitivity analyses to test assumptions made in the analysis:

1. the impact of including the use of complementary therapies

2. the impact of including productivity losses as a result of time off work as a result of illness

3. the impact of missing data, considered using MICE

4. the impact of taking an intervention perspective

5. the impact of taking a mental health-care perspective.

Process data analysis

A full description of the methods and analyses of our process data, including qualitative data, is presented in Chapter 4. Based on recent reviews,91 exploratory analyses examined baseline variables that might moderate outcome at multiple time points (6, 12 and 18 months) across the two treatments, including depression severity, age at depression onset, number of previous episodes, and baseline levels of cognitive and behavioural dysfunction, using the approach set out by Kraemer et al. 92 Although the power to detect moderate subgroup interactions was low, we were primarily interested in exploring the possibility of large interactions that could inform subsequent clinical decision-making regarding treatment allocation.

Mediational analyses investigated the hypothesised mechanisms of change (for BA, changes in specific behaviour such as reduced avoidance and rumination, learned capacity to apply behavioural principles to modify the environment; for CBT, changes in beliefs and underlying information processing style), pretreatment to mid-treatment, mid-treatment to post treatment across the trial arms using approaches to testing mediation that allow multiple mediators in one model. 92 The effects of the mediators on outcome at 12 and 18 months were modelled. This approach to examining mediation ensures that changes in putative mediators temporally precede changes in the primary outcome and allow baseline to post-treatment change in symptoms to be statistically controlled, necessary to rule out reverse causality. 93

Qualitative data analysis

Participant and therapist interviews were analysed using a framework approach94 combining deductive themes from the topic guides and inductive themes emerging from the data. Some interviews were coded independently to assess the reliability of coding and meetings were held to discuss and refine emerging themes. 95 Transcripts were examined thematically across the whole data set, as well as in the context of each interview, using constant comparison techniques. 96 Data were indexed and sorted using the identified themes and subthemes, and were summarised in framework matrices. 94 In keeping with the framework approach, we interrogated the data, searching for comparisons and contradictions and keeping interpretive notes. Alternative explanations or negative cases were identified, discussed and a consensus reached. 95

Ethics issues

We conducted the trial in such a way as to protect the human rights and dignity of the participants as reflected in the Helsinki Declaration. 97 Participants did not receive any financial inducement to participate. The study received National Research Ethics Committee (REC) approval from the South West REC in the UK (reference number 12/SW/0029). Local REC and NHS research and development approvals were also given for each recruitment site. To conform with data protection and freedom of information acts, all data have been stored securely and anonymised wherever possible. No published material will contain patient-identifiable information.

Obtaining informed consent from participants

We determined informed consent by a two-phase consent process. Participants received a study information sheet in the post and a form seeking their permission to be contacted by a member of the research team, not at this stage to give consent to trial participation. The information leaflets were produced using current guidelines for researchers on writing information sheets and consent forms98 and informed by our consumer/lived-experience user representatives. Participants who wished to partake in the trial returned their initial written consent to be contacted form to the site research team. Full informed written consent was obtained through an interview by a researcher where the information sheet was fully explained and where the opportunity to ask questions was given. The opportunity to withdraw from the trial was also fully explained. Researchers seeking consent were fully trained and supervised by the chief investigator and site leads. Communication and recording systems were set up to enable the trial team to monitor and act on participants’ wishes to withdraw from the trial.

Anticipated risks and benefits

All participants received usual GP care and, therefore, no treatment was withheld from participants in this trial. Both arms were active psychological treatments with previously demonstrated efficacy and no known iatrogenic effects. This trial may have in fact benefited individual participants, as CBT is not generally available for the majority of people with depression. By participating in this trial, participants also received an intensive level of monitoring such that any participants with worsening symptoms or who were at suicidal risk were identified and directed to appropriate care. We recorded all instances of AEs as detailed earlier.

Informing participants of anticipated risks and benefits

Participant information leaflets provided potential participants with information about the possible benefits and known risks of taking part in the trial. Participants were given the opportunity to discuss this issue with their GP or the trial manager prior to consenting. The trial manager would have informed the participant if new information came to light that may have affected the participant’s willingness to participate in the trial.

Management of suicide risk

Inherent in the nature of the population under scrutiny is the risk of suicide. We followed good clinical practice in monitoring for suicide risk during all research and clinical encounters with trial participants, developed for our previous trials. 55,99 Where any risk to participants attributable to expressed thoughts of suicide were encountered, we reported these directly to the GP (with the participant’s expressed permission), or if an acute risk was present we sought advice from the GP immediately and followed locally established suicide risk management plans. Systems were put into place to ensure that the chief investigator, trial manager and researchers were informed if there were any risks to the participants’ safety.

Trial Steering Committee and Data Monitoring Committee

A TSC was set up and included an independent chairperson, an academic GP and at least two other independent members, along with the lead investigator and some other study collaborators. The TSC met at least once a year. The DMC was set up and was composed of an independent mental health professional, statistician and clinician. The role of the DMC was to review serious AEs thought to be treatment related and look at outcome data regularly during data collection.

Execution dates

The preparatory period started in March 2012. Recruitment ran from September 2012 to April 2014. Follow-up lasted 18 months after randomisation and was completed by October 2015. Data analysis and reporting were completed 12 months after this (September 2016). The entire study period lasted 54 months (March 2012 to September 2016).

Participant flow and retention

Between 26 September 2012 and 3 April 2014, we recruited 440 participants, randomly allocating 221 participants (50%) to the BA group and 219 participants (50%) to the CBT group. We recruited to time and target and the majority of participants were recruited from primary care (87%). Progress over the course of recruitment and achievement of the target is shown in Figure 3. Participant recruitment and retention is shown for both the ITT and PP analyses in the trial CONSORT diagram (Figure 4). There were no protocol deviations.

FIGURE 3.

Trial recruitment. Solid line shows target recruitment and dashed line shows cumulative actual recruitment.

FIGURE 4.

The trial CONSORT flow diagram. (a) 6-month follow-up, (b) 12-month follow-up and (c) 18-month follow-up.

Baseline characteristics of participants

Patient- and trial-level characteristics at baseline were well balanced between groups (Table 4). We also found no evidence of a difference in patient characteristics between recruitment methods (Table 5). The PHQ-9 primary outcome at baseline was negatively skewed, with a high proportion of participants scoring towards the upper end of the distribution (Figure 5), and scores were similar between groups [BA, 17.7 PHQ-9 points (SD 4.8 PHQ-9 points); CBT, 17.4 PHQ-9 points (SD 4.8 PHQ-9 points)] (Table 6).

FIGURE 5.

Distribution of primary outcome (PHQ-9 score) at baseline.

Delivery and receipt of the interventions

Ten MHWs provided BA [median 22 participants each (interquartile range 19–25 participants each)] and 12 therapists provided CBT [median 21 participants each (interquartile range 13–23 participants each)]. MHWs had a mean of 18 months’ mental health experience (SD 11 months’ mental health experience) and CBT therapists had a mean of 22 months’ experience (SD 24 months’ experience) post CBT qualification. We removed one CBT therapist from the trial in the early stages who did not meet acceptable competency.

Participants received a mean of 11.5 BA sessions (SD 7.8 sessions) or 12.5 CBT sessions (SD 7.8 sessions). Three hundred and five participants (69%) completed the PP number of at least eight sessions [BA 147 (67%) participants, mean 16.1 sessions (SD 5.3 sessions); CBT 158 (72%) participants, mean 16.4 sessions (SD 5.4 sessions)]. Participants completing fewer than eight sessions {135 participants (31%) [BA 74 participants (33%) and CBT 61 participants (28%)] completed a mean of 2.5 BA sessions (SD 1.9 sessions) or 2.6 CBT sessions (SD 2.1 sessions)}.

Primary outcome: Patient Health Questionnaire-9 at 12 months

We present primary and secondary outcomes at 12 months in Table 6. We found no evidence of inferiority of PHQ-9 score at 12 months in either the ITT [CBT 8.4 PHQ-9 points (SD 7.5 PHQ-9 points), BA 8.4 PHQ-9 points (SD 7.0 PHQ-9 points); mean difference 0.1 PHQ-9 points, 95% CI –1.3 to 1.5 PHQ-9 points; p = 0.89] or PP [CBT 7.9 PHQ-9 points (SD 7.3 PHQ-9 points), BA 7.8 PHQ-9 points (SD 6.5 PHQ-9 points); mean difference 0.0 PHQ-9 points, 95% CI –1.5 to 1.6 PHQ-9 points; p = 0.99] populations. The non-inferiority of BA to CBT was accepted for both the ITT and PP populations, as the lower bound of the 95% CI (one-sided 97.5% CI) of the between-group mean difference lies within the non-inferiority margin of –1.9 PHQ-9 points (Figure 6). We ruled out superiority of CBT to BA as the lower bound of the 95% CI includes zero for the ITT and PP populations. The inference of non-inferiority was robust to sensitivity analysis across different PP definitions (Table 7). Our predefined stratification variable subgroup analyses (Table 8) showed no ITT or PP between-group difference by depression severity, ADM or site.

FIGURE 6.

Mean difference and two-sided 95% CI for the primary outcome of PHQ-9 at 12 months and non-inferiority margin.

Response and recovery at 12 months

Between 61% and 70% of ITT and PP participants in both groups met criteria for recovery or response, with no difference in the proportions of patients in each group who recovered or responded (see Table 6).

Primary and secondary outcomes at all follow-up points

Tables 9 and 10 show the repeated measures comparison of primary and secondary outcomes across time points for both ITT and PP populations. For the primary outcome, there was no evidence of difference between the CBT and BA groups across observed or imputed outcomes over the period of the trial, as indicated by a non-significant time by treatment effect interaction. Although there was some weak evidence (p = 0.06) of a higher number of depression-free days at follow-up with BA than CBT in the ITT analyses, this difference was not apparent in the PP analysis (p = 0.11).

Primary and secondary outcomes and clustering by therapist

There was evidence of a small, negligible clustering of primary and secondary outcome scores at follow-up across therapists overall and within BA and CBT groups (intracluster correlation coefficient of ≤ 0.04).

Missing data

Of the 440 participants recruited, 76 participants (17%) had missing primary outcome data at 12 months’ follow-up. The proportion of missing PHQ-9 data was higher in the BA than in the CBT group [46 (21%) vs. 30 (14%), ITT relative risk 1.15, 95% CI 1.0 to 1.32; p = 0.044]. Drop out from PP treatment was not significantly different (PP relative risk 1.13, 95% CI 0.98 to 1.30; p = 0.84). Imputation of data for primary and secondary outcomes at 12 months showed that, in accordance with the observed data analysis, no difference existed between groups (see Table 6), supporting our conclusion of non-inferiority. The odds of missing PHQ-9 data were higher for patients with increased baseline severity of depression (PHQ-9 ≥ 19, odds ratio 1.6, 95% CI 1.0 to 2.6; p = 0.05), and increasing age (in years) was associated with lower odds of missing PHQ-9 data (odds ratio 0.97, 95% CI 0.96 to 0.99). We found no evidence of an association between missingness and any other baseline characteristic.

Blinding

Outcome assessors reported having been unmasked for 16 (4%) participants [five (2%) in the BA group and 11 (5%) in the CBT group] as a result of participants informing assessors of their treatment allocation.

Safety and adverse events

Two (1%) non-trial-related deaths [one (1%) multidrug toxicity death in the BA group and one (1%) cancer-related death in the CBT group] and 15 depression-related, but not treatment-related, serious AEs (three in the BA group and 12 in the CBT group) occurred in three (2%) participants in the BA group [two (1%) patients who overdosed and one (1%) who self-harmed] and eight (4%) participants in the CBT group [seven (4%) who overdosed and one (1%) who self-harmed].

Intervention quality

Mental health workers and therapists met acceptable competency standards above our set thresholds, as judged by our independently rated tapes: mean Quality of Behavioural Activation Scale score for BA competence was 55 (SD 7.5) and mean CTS-R for CBT competence was 37.9 (SD 10.9).

Results of economic evaluation

Cost-effectiveness

As observed costs were lower and QALY outcomes better in the BA group than in the CBT group, this generated an ICER (the additional cost of one intervention compared with another divided by the additional effects) of –£6865, suggesting that BA dominates CBT (i.e. BA is both cheaper and more effective). The scatterplot of the bootstrapped cost and effectiveness pairs for BA versus CBT (Figure 7) illustrates dominance of BA over CBT, with the point estimate and two-thirds of the scatter points (66%) falling in the south-east quadrant of the cost-effectiveness plane, where BA replications are cheaper and more effective than CBT ones.

FIGURE 7.

Scatterplot showing the bootstrapped mean differences in costs and effects of BA compared with CBT. NE, north-east; NW, north-west; SE, south-east; SW, south-west.

The CEAC showing the probability of BA being cost-effective compared with CBT does not fall < 75% and is closer to 80% at the standard NICE-preferred willingness-to-pay levels of £20,000–30,000 per QALY (Figure 8).

FIGURE 8.

Cost-effectiveness acceptability curve showing the probability that BA is cost-effective compared with CBT for different values of willingness to pay per QALY.

Sensitivity analyses

Inclusion of the costs of complementary therapies and productivity losses (Table 15) increased the overall mean difference in total cost between the two groups, making BA significantly less costly than CBT in both cases (p = 0.039 with the inclusion of complementary therapies and p = 0.003 with the inclusion of productivity losses). Narrower perspectives, focusing only on the cost of the interventions and the mental health service perspective, had the same effect (p < 0.0001 in both cases). Figure 9 shows the CEACs for the inclusion of complementary therapies and productivity losses and Figure 10 shows the CEACs for the two narrower cost perspectives. In all cases, BA continues to have a higher probability of being cost-effective compared with CBT at the NICE threshold of £20,000–30,000 per QALY.

TABLE 15 - Results of the sensitivity analyses of economic data

Adjusted for site, ADM use, symptom severity and baseline covariates.

Analysis type	Trial arm				Mean differencea (BA – CBT)	95% CIa	p-valuea
	BA		CBT
	n	Mean (SD)	n	Mean (SD)
Impact on total costs (£)
Main analysis	159	2596.62 (1846.72)	168	3250.74 (3040.99)	–343.24	–857.62 to 171.13	0.190
Inclusion of complementary therapies	158	2729.54 (2604.00)	168	3367.90 (3530.36)	–535.60	–1045.28 to –25.92	0.039
Inclusion of productivity losses	152	3305.91 (3410.20)	166	4937.92 (6367.92)	–1726.37	–2870.80 to –581.93	0.003
Intervention perspective	159	992.73 (60.54)	168	1255.03 (88.16)	–262.30	–381.40 to –143.19	< 0.0001
Mental health-care perspective	159	914.71 (67.86)	168	1253.85 (99.97)	–339.14	–472.64 to –205.64	< 0.0001
Imputation of missing data	221	1841.67 (287.97)	219	2282.40 (423.94)	–440.73	–1007.70 to 126.26	0.127
Impact on QALYs
Main analysis	152	0.984 (0.422)	157	0.935 (0.433)	0.050	–0.145 to 0.046	0.308
Imputation of missing data	221	1.224 (0.043)	219	1.198 (0.061)	0.026	–0.058 to 0.109	0.546

FIGURE 9.

Cost-effectiveness acceptability curve showing the probability that BA is cost-effective compared with CBT for different values of willingness to pay for a QALY, including costs of complementary therapies and productivity losses.

FIGURE 10.

Cost-effectiveness acceptability curve showing the probability that BA is cost-effective compared with CBT for different values of willingness to pay for a QALY, from intervention and mental health-care perspectives.

Imputation of missing data increased the difference in total cost (£440.00 vs. £343.00 in the main analysis), but reduced the difference in QALYs (0.026 vs. 0.050 in the main analysis), increasing the ICER from –£6865.00 to –£16,951.00. Figure 11 shows the CEAC for the missing data analysis, which again supports the likelihood that BA is cost-effective compared with CBT.

FIGURE 11.

Cost-effectiveness acceptability curve showing the probability that BA is cost-effective compared with CBT for different values of willingness to pay for a QALY, including imputed missing data.

Introduction

Alongside the main clinical and economic evaluation, we undertook a process evaluation to investigate the implementation of the intervention, moderators of outcome and possible mechanisms of effect. This chapter consists of a description of our quantitative and qualitative methods, analyses and results.

Objectives

• To investigate factors that may moderate the effect of treatment allocation on outcome (PHQ-9 score), and mechanistic and procedural factors that may mediate the effect of treatment on outcome.

• To obtain a more in-depth understanding of the ongoing mechanisms and impact of treatment from participants and therapists including participants’ views of the role of cognitive and behavioural change strategies and the broader impacts of treatment on participants’ lives.

Quantitative process study

Results

Moderation analyses

The results of the analyses investigating interactions between treatment effects and selected covariates are shown in Table 17. Across all three follow-up times, and in both the ITT and PP populations, there were no statistically significant interactions between treatment group and the specified covariates. However, in the PP population, at both 12 and 18 months’ follow-up, there was weak evidence (p < 0.1) for an interaction between treatment group and baseline PHQ-9 score. This interaction was in the direction of participants with higher PHQ-9 scores at baseline, and receiving BA, having lower PHQ-9 scores at follow-up than with participants who received CBT. This treatment moderation effect of baseline PHQ-9 score is shown graphically in Figure 12 for the 12-month follow-up. Weak evidence was found for a moderating effect of baseline total RRS score on the treatment effect for PHQ-9 at 12 months’ follow-up, also in the PP population. However, this effect was not observed at any other follow-up time points.

TABLE 17 - Interactions between treatment allocation and baseline covariates

All models include one interaction term between treatment group and the specified potential moderator, and adjustment for baseline PHQ-9 score, ADM use at baseline and trial site.

Reference category for treatment preference: no preference. Interaction effects: CBT/received preferred treatment; CBT/did not receive preferred treatment.

Interaction terma	PHQ-9: time point
	6-month follow-up				12-month follow-up				18-month follow-up
	ITT population		PP population		ITT population		PP population		ITT population		PP population
	Mean difference (95% CI), n	Global p-value	Mean difference (95% CI), n	Global p-value	Mean difference (95% CI), n	Global p-value	Mean difference (95% CI), n	Global p-value	Mean difference (95% CI), n	Global p-value	Mean difference (95% CI), n	Global p-value
CBT/baseline PHQ-9 score	0.22 (–0.06 to 0.51), 380	0.128	0.21 (–0.11 to 0.53), 284	0.206	0.18 (–0.11 to 0.48), 364	0.226	0.28 (–0.05 to 0.61), 286	0.092	0.17 (–0.13 to 0.46), 356	0.274	0.29 (–0.03 to 0.62), 284	0.080
CBT/baseline BADS total score	0.01 (–0.07 to 0.08), 350	0.880	0.00 (–0.08 to 0.08), 263	0.983	0.01 (–0.08 to 0.06), 338	0.766	–0.01 (–0.09 to 0.07), 265	0.858	0.00 (–0.07 to 0.07), 331	0.990	0.00 (–0.08 to 0.08), 263	0.978
CBT/baseline DAS score	–0.02 (–0.05 to 0.02), 375	0.313	–0.01 (–0.05 to 0.03), 282	0.704	0.01 (–0.02 to 0.05), 361	0.508	0.02 (–0.02 to 0.07), 284	0.249	–0.02 (–0.06 to 0.01), 354	0.193	–0.02 (–0.06 to 0.02), 283	0.423
CBT/baseline RRS total score	0.07 (–0.06 to 0.21), 379	0.278	0.01 (–0.14 to 0.16), 284	0.907	0.08 (–0.06 to 0.22), 364	0.249	0.13 (–0.02 to 0.29), 286	0.097	0.08 (–0.06 to 0.22), 356	0.247	0.09 (–0.07 to 0.25), 284	0.268
CBT/age at onset of depression	–0.01 (–0.11 to 0.08), 371	0.774	0.01 (–0.09 to 0.11), 278	0.909	–0.03 (–0.13 to 0.07), 357	0.572	–0.07 (–0.18 to 0.04), 281	0.194	–0.02 (–0.12 to 0.07), 349	0.625	–0.05 (–0.15 to 0.06), 279	0.373
CBT/number of past episodes of depression	0.13 (–0.04 to 0.30), 333	0.141	0.09 (–0.11 to 0.29), 245	0.367	–0.02 (–0.21 to 0.17), 319	0.827	–0.09 (–0.31 to 0.14), 248	0.447	–0.05 (–0.22 to 0.13), 312	0.605	–0.07 (–0.27 to 0.13), 246	0.489
Treatment preferenceb	0.54 (–3.35 to 4.43), 378; 0.09 (–3.28 to 3.46)	0.963	1.70 (–2.35 to 5.74), 283; –0.57 (–4.28 to 3.13)	0.623	0.42 (–3.59 to 4.43), 363; 0.23 (–3.27 to 3.72)	0.976	1.35 (–2.93 to 5.63), 285; –1.28 (–5.16 to 2.60)	0.589	0.39 (–3.57 to 4.36), 353; 0.14 (–3.43 to 3.70)	0.981	0.62 (–3.71 to 4.95), 283; –1.26 (–5.21 to 2.69)	0.747

FIGURE 12.

Margins plot of interaction, with 95% CIs, between the effect of treatment group and baseline PHQ-9 on PHQ-9 at 12 months’ follow-up using the PP observed data sample only (with adjustment for site and baseline antidepressant use). a, Linear predictor of PHQ-9 at 12 months’ follow-up.

Mediation analyses

Table 18 shows the unadjusted scores for PHQ-9 and all mediator variables at all time points at which the variable was recorded. Figures 13–17 show the treatment group mean scores and Figures 18–22 (see Appendix 2) show the individual participant scores, for all time points at which each variable was recorded. For PHQ-9, the greatest reduction in mean scores occurred between baseline and 6 months’ follow-up, with a smaller reduction between 6 and 12 months’ follow-up. Mean scores were stable between the 12- and 18-month follow-ups. On looking at the individual participant trajectories for PHQ-9 between the follow-up times, there was wide variation among participants with regard to their personal PHQ-9 trajectory. Some participants had an evident reduction in PHQ-9 score between baseline and 6 months’ follow-up, which may be sustained in some cases but increased after the 6-month follow-up in others. A similar wide variation in individual participant trajectories was observed across all process mediator variables. No significant between-group difference across time points was found for any of the process mediators (p-values for interaction between group and time point were > 0.1 for all process mediators across all available time points). There was no significant difference between the treatment groups (CBT vs. BA) for percentage of sessions attended (mean difference 4.4 sessions attended, 95% CI –1.7 to 10.5 sessions attended). For basic fidelity (on a scale of 0 to 100), there was evidence that the fidelity for CBT was lower than for BA (mean difference –12.1, 95% CI –17.6 to –6.7), although this may be artefactual because of the different numbers of topics for the two therapies. Among participants receiving BA, 158 out of 220 (71%) completed all six of the core topics, compared with only 70 out of 218 (32%) participants who received CBT completing all seven of the core topics, and only 132 out of 218 (61%) completed at least six out of seven core topics. For overall fidelity, there was no significant difference between the groups (mean difference –4.9, 95% CI –11.0 to 1.1), although again this may be arbitrary as a result of the weighting system used to weight core and mandatory topics.

TABLE 18 - Unadjusted scores for PHQ-9 and mediator variables for mediation population

DNC, data not collected; DNR, data not reported.

Treatment fidelity outcomes measured across the full follow-up period, reported under 12 months’ follow-up for convenience.

Outcome variable	Baseline		PM1		PM2		6-month follow-up		12-month follow-up		18-month follow-up
	CBT	BA	CBT	BA	CBT	BA	CBT	BA	CBT	BA	CBT	BA
PHQ-9: mean (SD), n	17.39 (4.85), 214	17.69 (4.80), 219	DNC	DNC	DNC	DNC	9.41 (7.04), 190	9.72 (6.88), 183	8.20 (7.32), 184	8.34 (6.97), 173	8.39 (7.12), 176	8.32 (7.13), 174
BADS total: mean (SD), n	88.30 (19.98), 195	87.65 (21.32), 206	76.35 (24.46), 92	75.69 (27.10), 91	63.74 (27.76), 78	67.15 (27.56), 67	55.88 (29.65), 146	56.74 (30.18), 144	DNC	DNC	DNC	DNC
BADS activation: mean (SD), n	29.57 (7.21), 206	29.90 (7.67), 214	26.73 (7.86), 104	26.35 (7.97), 100	22.82 (8.28), 85	22.78 (9.48), 78	21.34 (9.24), 155	22.09 (9.90), 151	DNC	DNC	DNC	DNC
BADS avoidance: mean (SD), n	26.77 (9.47), 209	25.86 (9.31), 215	23.85 (10.54), 105	23.35 (10.03), 106	19.66 (10.14), 85	20.19 (9.39), 79	15.96 (11.25), 155	17.37 (10.72), 159	DNC	DNC	DNC	DNC
BADS work impairment: mean (SD), n	15.78 (6.24), 212	16.09 (6.43), 219	14.23 (6.38), 105	14.15 (6.97), 105	11.56 (7.20), 85	11.30 (6.07), 81	10.09 (7.08), 157	9.98 (6.99), 160	DNC	DNC	DNC	DNC
BADS social impairment: mean (SD), n	15.81 (7.23), 205	16.13 (7.77), 211	11.91 (7.53), 101	13.45 (8.39), 101	10.25 (8.14), 84	11.28 (8.27), 74	8.63 (8.02), 156	8.75 (8.06), 154	DNC	DNC	DNC	DNC
DAS: mean (SD), n	154.23 (38.35), 211	152.02 (38.67), 217	158.36 (40.04), 109	158.72 (40.65), 107	150.71 (41.98), 88	152.70 (41.38), 84	134.66 (41.95), 159	135.97 (42.15), 160	DNC	DNC	DNC	DNC
RRS total: mean (SD), n	60.28 (10.05), 213	59.61 (11.18), 219	58.29 (11.59), 109	58.18 (12.42), 111	52.22 (12.28), 88	53.45 (13.53), 85	48.27 (14.48), 159	49.95 (14.24), 160	DNC	DNC	DNC	DNC
RRS reflection: mean (SD), n	12.30 (3.12), 213	12.05 (3.41), 219	12.06 (2.99), 109	11.86 (3.35), 111	11.03 (3.00), 88	11.04 (3.59), 85	10.38 (3.44), 159	10.58 (3.37), 160	DNC	DNC	DNC	DNC
SHAPS: mean (SD), n	21.11 (2.37), 212	20.87 (2.43), 217	DNC	DNC	DNC	DNC	21.35 (3.16), 161	21.39 (3.02), 161	21.08 (2.31), 163	20.59 (2.75), 147	DNR	DNR
Proportion (%) of sessions attended:a mean (SD), n									52 (33), 214	48 (33), 219
Basic treatment fidelity (scale 0–100):a mean (SD), n									73 (31), 213	86 (28), 218
Overall treatment fidelity:a mean (SD), n									68 (31), 213	73 (33), 218

FIGURE 13.

Mean PHQ-9 scores with 95% CI at baseline and at 6, 12 and 18 months’ follow-up, by treatment group for the mediation population.

FIGURE 14.

Mean BADS total score with 95% CI at baseline, PM1 (session 4), PM2 (session 7) and at 6 months’ follow-up, by treatment group for the mediation population.

FIGURE 15.

Dysfunctional Attitudes Scale total score at baseline, PM1 (session 4), PM2 (session 7) and at 6 months’ follow-up, by treatment group for the mediation population.

FIGURE 16.

Ruminative Response Scale total score at baseline, PM1 (session 4), PM2 (session 7) and at 6 months’ follow-up, by treatment group for the mediation population.

FIGURE 17.

Snaith–Hamilton Pleasure Scale total score at baseline, and at 6 and 12 months’ follow-up, by treatment group for the mediation population.

Using the SEM approach, at the 6-month PHQ-9 follow-up, when considered individually, each mediator measured at PM1 mediated only a small proportion of the total effect of treatment (Table 19). None of the mediators in the single mediator models or in the multiple mediator models was found to have a statistically significant indirect effect of treatment allocation acting via that mediator.

TABLE 19 - Results of the SEM analyses for mediation of treatment effect on PHQ-9 at 6 months’ follow-up

Proportions of total effect mediated are not required to sum to 100%.

Mediator	Model
	Single mediator			Including all mediators
	Indirect effect of treatment allocation acting through mediator (95% CI), n	Total effect of treatment allocation (95% CI)	Proportion of total effect mediated by indirect effect (%)	Indirect effect of treatment allocation acting through mediator (95% CI)	Proportion of total effect mediated by indirect effect (%)a
Mediators measured at PM1 (therapy session 4)
				Total effect of treatment allocation –1.84 (95% CI –3.79 to 0.04), n = 158
BADS total	0.18 (–0.34 to 0.91), 163	–1.52 (–3.43 to 0.34)	–12	Not included	Not included
BADS activation	0.12 (–0.51 to 0.75), 186	–1.52 (–3.45 to 0.32)	–8	0.13 (–0.11 to 0.81)	0.13 (–0.11 to 0.81)
BADS avoidance	0.11 (–0.17 to 0.71), 195	–1.34 (–3.14 to 0.51)	–8	–0.01 (–0.38 to 0.18)	–0.01 (–0.38 to 0.18)
BADS work impairment	0.10 (–0.45 to 0.69), 198	–1.45 (–3.24 to 0.42)	–7	0.05 (–0.10 to 0.65)	0.05 (–0.10 to 0.65)
BADS social impairment	–0.06 (–0.50 to 0.32), 184	–1.54 (–3.43 to 0.18)	4	0.04 (–0.12 to 0.52)	0.04 (–0.12 to 0.52)
DAS	0.13 (–0.26 to 0.71), 202	–1.73 (–3.46 to 0.16)	–8	0.18 (–0.12 to 0.84)	0.18 (–0.12 to 0.84)
RRS total	0.13 (–0.31 to 0.63), 206	–1.64 (–3.40 to 0.18)	–8	0.06 (–0.14 to 0.64)	0.06 (–0.14 to 0.64)
RRS reflection	0.04 (–0.09 to 0.39), 206	–1.60 (–3.36 to 0.21)	–2	Not included	Not included
Mediators measured at PM2 (therapy session 7)
				Total effect of treatment allocation –2.63 (95% CI –4.87 to –0.32), n = 126
BADS total	–0.23 (–1.45 to 0.89), 129	–2.52 (–4.94 to –0.35)	9	Not included	Not included
BADS activation	0.21 (–0.57 to 1.15), 153	–1.68 (–3.67 to 0.37)	–13	–0.01 (–0.47 to 0.19)	1
BADS avoidance	–0.05 (–0.84 to 0.72), 154	–2.26 (–4.42 to –0.12)	2	–0.01 (–0.48 to 0.30)	0
BADS work impairment	0.34 (–0.64 to 1.38), 160	–2.01 (–4.05 to –0.01)	–17	0.04 (–0.35 to 0.80)	–1
BADS social impairment	–0.29 (–1.29 to 0.56), 147	–2.33 (–4.46 to –0.23)	13	–0.19 (–1.19 to 0.19)	7
DAS	0.23 (–0.54 to 0.93), 166	–1.95 (–3.95 to 0.20)	–12	0.10 (–0.16 to 0.84)	–4
RRS total	–0.02 (–0.77 to 0.68), 167	–2.00 (–4.07 to –0.02)	1	–0.04 (–0.59 to 0.24)	2
RRS reflection	0.11 (–0.42 to 0.71), 167	–2.04 (–4.06 to –0.02)	–5	Not included	Not included

At the 12-month PHQ-9 follow-up, with mediators measured at PM1, again the BADS variables appeared to be the strongest individual mediators, with social impairment mediating 21% of the total effect of treatment, and the total BADS score mediating –18% (Table 20).

TABLE 20 - Results of the SEM analyses for mediation of treatment effect on PHQ-9 at 12 months’ follow-up

Proportions of total effect mediated are not required to sum to 100%.

Treatment fidelity mediators included in model with mediators measured at 6 months’ follow-up.

Mediator	Model
	Single mediator			Including all mediators
	Indirect effect of treatment allocation acting through mediator (95% CI), n	Total effect of treatment allocation (95% CI)	Proportion of total effect mediated by indirect effect (%)	Indirect effect of treatment allocation acting through mediator (95% CI)	Proportion of total effect mediated by indirect effect (%)a
Mediators measured at PM1
				Total effect of treatment allocation –0.58 (95% CI –2.61 to 1.59), n = 154
BADS total	0.09 (–0.78 to 0.81), 158	–0.49 (–2.55 to 1.41)	–18	Not included	Not included
BADS activation	0.01 (–0.69 to 0.61), 181	–0.79 (–2.74 to 1.09)	–1	0.08 (–0.27 to 0.69)	–14
BADS avoidance	0.02 (–0.47 to 0.49), 189	–0.82 (–2.76 to 1.14)	–3	0.01 (–0.20 to 0.45)	–3
BADS work impairment	0.03 (–0.69 to 0.61), 192	–0.56 (–2.51 to 1.35)	–5	0.08 (–0.26 to 0.79)	–15
BADS social impairment	–0.16 (–0.80 to 0.23), 179	–0.78 (–2.71 to 1.18)	21	0.00 (–0.24 to 0.24)	0
DAS	0.08 (–0.15 to 0.61), 198	–0.93 (–2.84 to 0.92)	–9	–0.18 (–0.90 to 0.13)	32
RRS total	0.09 (–0.41 to 0.66), 201	–0.98 (–2.88 to 0.83)	–9	0.10 (–0.09 to 0.73)	–17
RRS reflection	0.08 (–0.17 to 0.54), 201	–0.95 (–2.84 to 0.87)	–9	Not included	Not included
Mediators measured at PM2
				Total effect of treatment allocation –0.84 (95% CI –3.23 to 1.58), n = 125
BADS total	–0.29 (–1.40 to 0.64), 128	–0.53 (–2.98 to 1.74)	55	Not included	Not included
BADS activation	–0.09 (–0.73 to 0.84), 150	–0.27 (–2.29 to 1.69)	–33	–0.04 (–0.72 to 0.14)	5
BADS avoidance	–0.10 (–0.74 to 0.48), 150	–0.37 (–2.36 to 1.66)	26	0.00 (–0.38 to 0.52)	–1
BADS work impairment	0.14 (–0.76 to 1.10), 156	–0.47 (–2.58 to 1.67)	–31	–0.03 (–0.91 to 0.78)	4
BADS social impairment	–0.33 (–1.35 to 0.43), 145	–0.58 (–2.78 to 1.52)	57	–0.20 (–1.51 to 0.15)	23
DAS	0.11 (–0.42 to 0.67), 162	–0.57 (–2.58 to 1.49)	–19	–0.04 (–0.73 to 0.21)	5
RRS total	0.01 (–0.67 to 0.71), 163	–0.68 (–2.71 to 1.32)	–2	–0.03 (–0.63 to 0.43)	4
RRS reflection	0.20 (–0.23 to 0.90), 163	–0.72 (–2.73 to 1.28)	–28	Not included	Not included
Mediators measured at 6 months’ follow-up
				Total effect of treatment allocation 0.30 (–1.37 to 1.95), n = 250
BADS total	0.09 (–0.77 to 1.05), 258	0.27 (–1.33 to 1.93)	33	Not included	Not included
BADS activation	–0.05 (–0.82 to 0.68), 281	–0.22 (–1.73 to 1.31)	25	0.00 (–0.29 to 0.35)	2
BADS avoidance	–0.40 (–1.13 to 0.37), 285	–0.06 (–1.52 to 1.42)	720	–0.12 (–0.64 to 0.10)	–39
BADS work impairment	0.24 (–0.46 to 1.04), 294	–0.26 (–1.82 to 1.17)	–94	0.02 (–0.13 to 0.39)	6
BADS social impairment	0.26 (–0.52 to 1.09), 282	–0.06 (–1.58 to 1.53)	–463	0.06 (–0.11 to 0.52)	19
DAS	0.08 (–0.80 to 0.61), 296	–0.26 (–1.72 to 1.26)	33	–0.02 (–0.36 to 0.11)	–5
RRS total	–0.31 (–0.99 to 0.41), 297	–0.30 (–1.82 to 1.16)	105	–0.11 (–0.59 to 0.19)	–38
RRS reflection	–0.14 (–0.68 to 0.40), 297	–0.30 (–1.81 to 1.17)	47	Not included	Not included
SHAPS	0.02 (–0.04 to 0.22), 298	–0.09 (–1.70 to 1.41)	–20	0.05 (–0.03 to 0.33)	17
Treatment fidelity mediatorsb
Proportion of sessions attended	–0.04 (–0.28 to 0.04), 350	–0.14 (–1.49 to 1.30)	29	0.01 (–0.12 to 0.24)	2
Basic treatment fidelity	0.33 (–0.03 to 0.83), 349	–0.15 (–1.55 to 1.27)	–216	0.07 (–0.79 to 0.76)	23
Overall treatment fidelity	0.18 (0.01 to 0.54), 349	–0.15 (–1.55 to 1.27)	–117	0.05 (–0.43 to 0.72)	17

For the overall treatment fidelity, the indirect effect of treatment group acting via this mediator was statistically significant (–0.18, 95% CI –0.54 to –0.01), mediating –117% of the total treatment effect (see Table 20). In the multiple mediator model, however, none of the mediators was found to have a significant indirect effect. BADS avoidance, BADS social impairment, total RRS score, SHAPS, basic treatment fidelity and overall treatment fidelity were found to mediate at least 10% in magnitude of the total treatment effect.

At 18 months’ follow-up, the indirect effect of treatment via basic treatment fidelity in the individual mediator model was statistically significant (observed effect 0.61, 95% CI 0.23 to 1.17) and associated with a proportion of total effect mediated of 610% (Table 21). In the multiple mediators model, with mediator variables measured at 6 months’ follow-up, this effect was not statistically significant, but was associated with a proportion of total effect mediated of 77%. Similarly, the indirect effect via overall treatment quality was statistically significant (observed effect 0.24, 95% CI 0.03 to 0.65) in the individual mediator model, but this significant effect was not seen in the multiple mediators model.

TABLE 21 - Results of the SEM analyses for mediation of treatment effect on PHQ-9 at 18 months’ follow-up

Treatment fidelity mediators included in model with mediators measured at 6 months’ follow-up.

Mediator	Model
	Single mediator			Including all mediators
	Indirect effect of treatment allocation acting through mediator (95% CI), n	Total effect of treatment allocation (95% CI)	Proportion of total effect mediated by indirect effect (%)	Indirect effect of treatment allocation acting through mediator (95% CI)	Proportion of total effect mediated by indirect effect (%)
Mediators measured at 6 months’ follow-up
				Total effect of treatment allocation –0.52 (95% CI –2.16 to 1.13), n = 248
BADS total	0.06 (–0.76 to 0.96), 255	0.64 (–0.91 to 2.27)	9	Not included	Not included
BADS activation	–0.10 (–0.81 to 0.60), 278	0.14 (–1.42 to 1.64)	–75	0.00 (–0.15 to 0.22)	1
BADS avoidance	–0.38 (–1.05 to 0.25), 283	0.19 (–1.32 to 1.69)	–196	0.02 (–0.13 to 0.37)	3
BADS work impairment	0.17 (–0.52 to 0.90), 291	0.03 (–1.47 to 1.49)	653	0.11 (–0.07 to 0.64)	21
BADS social impairment	0.25 (–0.47 to 1.11), 278	0.56 (–0.88 to 2.14)	45	0.12 (–0.29 to 0.71)	23
DAS	–0.02 (–0.63 to 0.64), 294	0.03 (–1.44 to 1.52)	–76	0.00 (–0.23 to 0.21)	0
RRS total	–0.31 (–0.93 to 0.32), 294	0.02 (–1.43 to 1.49)	–1600	–0.06 (–0.47 to 0.08)	–12
RRS reflection	–0.09 (–0.51 to 0.35), 294	0.03 (–1.44 to 1.50)	–357	Not included	Not included
SHAPS	0.00 (–0.11 to 0.08), 293	0.01 (–1.50 to 1.48)	–9	0.00 (–0.11 to 0.13)	0
Treatment fidelity mediatorsa
Proportion of sessions attended	–0.04 (–0.28 to 0.04), 343	0.12 (–1.29 to 1.53)	–36	0.04 (–0.08 to 0.45)	7
Basic treatment fidelity	0.61 (0.23 to 1.17), 342	0.10 (–1.28 to 1.54)	610	0.40 (–0.37 to 1.19)	77
Overall treatment fidelity	0.24 (0.03 to 0.65), 342	0.10 (–1.28 to 1.54)	237	–0.05 (–0.67 to 0.46)	–9
Mediators measured at 12 months’ follow-up
SHAPS	0.04 (–0.10 to 0.29), 291	0.34 (–1.16 to 1.96)	11	Not included	Not included

Across the three PHQ-9 time points, the effects of the mediators varied considerably, with the proportions of total effect mediated increasing at later time points, possibly as a result of the mediator variables having a chance to stabilise at later measurement times, and also the PHQ-9 scores becoming more stable on progressing through the follow-up period. The only mediator that was statistically significantly different between the groups was basic treatment fidelity, which was a significant mediator of treatment allocation at 18 months’ follow-up. Using individual regression models, both basic and overall treatment fidelity were associated with the 12-month PHQ-9 score, with poorer fidelity being associated with higher PHQ-9 scores (p-values 0.020 and 0.023, respectively). Similarly, both basic and overall treatment fidelity were significantly associated with the 18-month PHQ-9 follow-up, again with poorer fidelity being associated with higher PHQ-9 scores (p-values 0.001 and 0.020, respectively), the basic fidelity remaining significant when included in a model with overall fidelity (p-value 0.007). No evidence was found for a significant interaction between treatment group and basic or overall fidelity, with regard to either 12 or 18 months’ PHQ-9 follow-up.

Qualitative process study

Results

Summary of findings

We found that BA for depression is not inferior to CBT in terms of reduction of depression symptoms and is cost-effective compared with CBT against commonly applied decision-maker willingness-to-pay thresholds. We observed our results using both ITT and PP analyses, using a conservative non-inferiority margin. Our economic outcomes were driven by the lower costs of the MHWs who delivered BA, compared with the more experienced psychological therapists who routinely deliver CBT. Our study results, therefore, substantiate the hypothesis that BA is as effective as CBT and that its simplicity renders BA suitable for delivery by junior MHWs with no professional training in psychological therapies. 19

Our process data found that, despite being challenging at times, BA and CBT were acceptable and feasible for participants, MHWs and therapists, and effected changes in people’s specific symptoms and in their lives more broadly. Despite experiencing initial difficulties that could be detected by some participants, with sufficient training, experience and supervision, junior MHWs could feel confident in delivering BA effectively. We found weak evidence for an interaction between treatment and baseline PHQ-9 score on PHQ-9 at 12 and 18 months’ follow-up, indicating that BA may be a better choice of treatment for patients with higher baseline PHQ-9 scores.

Strengths and limitations

The COBRA trial is the largest trial of BA to date and is one of the largest trials of psychological treatments for depression. We followed up participants for 18 months and our economic analysis is one of few in this field. Therapists and MHWs working in three different routine UK care settings delivered treatment, providing evidence of potential generalisability. We assessed therapy quality using independent raters and ensured that treatment in both arms was delivered to the standard recommended in guidelines. Given the nature of the intervention and comparator we could not mask patients or the MHWs or therapists who were delivering the interventions to treatment allocation, but we used self-reported outcome measures and robust outcome assessor-masking procedures to reduce researcher unmasking to < 5%.

At 21% and 14% for BA and CBT, respectively at 12 months, our levels of attrition and outcome loss to follow-up were low at 12 and 18 months, similar to other trials in this area, but are still a limitation. Although we found a between-group difference in attrition for the ITT analyses, this was not the case for the PP analyses, suggesting that any differential attrition was an artefact of the trial and not of the treatments. Furthermore, our between-group inferences were robust to data imputation. Although participants in the PP population attended similar numbers of sessions to those in other CBT trials, 35% of participants overall chose to not attend a minimal number of sessions, a problem well known to routine psychological therapies services.

This pragmatic trial carried out in routine environments means we were unable to quantify or control for the contribution of ADM on outcomes. However, most participants who were taking medication had been doing so for a considerable time before entering the trial, making it unlikely that our results were driven by pharmacological treatment.

In terms of competency ratings, the ratings of our random sample of therapy tapes showed that both CBT therapists and BA MHWs were, on average, performing above competency thresholds. In terms of CBT, our mean sessional competency ratings (37.9) were very similar to the means reported in the CoBalT trial,55 another significant and large recent UK trial of CBT – 38.8, demonstrating that our therapists were achieving competency levels consistent with other similar pragmatic effectiveness studies. However, on this random sample of tapes some therapists were scoring below the threshold of competence for that specific session as assessed by our external raters, although we would stress that all therapists had exceeded the competence threshold at the end of their COBRA-specific protocol training course.

We note that our CBT therapists had received 1 year of postgraduate training in CBT, had passed similar competency tests in order to qualify from these courses, had received an additional specialist training in the COBRA protocol and were supervised by CBT experts. Most importantly, they were NHS employees engaged in routine treatment for patients with depression in the UK NHS IAPT services; indeed they were working for the NHS alongside their trial duties. Finally, the competency ratings from the CBT arm in our trial are consistent with other research findings in the field. 12,40 Therefore, although it may be possible to train CBT therapists to achieve higher competency ratings, we suggest that these levels of competence are those actually seen in routine clinical practice in IAPT services in the UK, results achieved following substantial investment in therapists’ clinical training, and appear unrelated to clinical outcomes.

Furthermore, although the measures of competence for the BA MHWs and the CBT therapists are not directly comparable, we note that the MHWs scored, on average, further above the competency threshold than the CBT therapists on the relevant competency measure. This provides some, albeit weak and indirect, supporting evidence for the proposition that it might be easier to train people to be competent BA workers than CBT therapists, although this is moderated by the difficulties with the unknown psychometric properties of the BA competence measure in particular and we did not set out to test this proposal directly.

The period between baseline and the 6-month follow-up was the period in which the main treatment effect occurred; therefore, it is during this period that any mediation effect would be of greatest interest. However, there may be difficulties in assessing the effect of potential mediators during this period because of the variation in such mediators at the times when recorded during the first 6 months of treatment. Such variation may be attributable to differences in the pace of therapy in terms of how quickly the patients completed their sessions and engaged with the topics relevant to their therapy. At later stages of follow-up, with regard to both the outcome and the mediator variables, it may be the case that the mediator variables had become more stable and therefore better able to facilitate mediation analysis.

The wide variation among patients in their trajectories across time points within the follow-up period, with regard to PHQ-9 and each mediator variable, may also be an impediment to distinguishing possible mediation effects. However, little difference was noted between the CBT and BA groups in terms of the mediating psychological variables, possibly indicating little practical difference in how participants responded to the two forms of therapy. Owing to differences in the structure of therapy, the BA group scored more highly for measures of treatment fidelity, although the proportion of sessions attended was similar across the two treatment groups.

Considering differences in the pattern of mediation across outcome follow-up times, at the earlier follow-up time of 6 months, the indirect effects of the mediators appeared to be accounting for only small proportions of the total treatment effect. This may reflect lack of stability in the early phases of the trial, while many participants were still undergoing therapy, with regard to both PHQ-9 and mediator scores.

In terms of moderators and mediators, COBRA was a large study that incorporated the collection of several psychological variables that were potential mediators of treatment effect; these variables were collected at several time points during the follow-up period of the trial. Also, the primary outcome, PHQ-9, was collected at three time points (at 6, 12 and 18 months’ follow-up). The collection of data for several psychological mediators at multiple time points has facilitated the comparison of how the two treatment groups progressed over the study period, at the group level and at the individual participant level. However, the large number of comparisons performed as a result of this brings about concerns regarding multiple testing and the possibility of finding statistically significant results by chance. With regard to the mediation analyses, only three statistically significant results were found when including a single mediator in each model, one relating to PHQ-9 measured at the 12-month follow-up and two at the 18-month follow-up. These results are consistent with the expectation of significant results occurring by chance. Therefore, these results should be viewed with caution.

Similarly, of the 42 inferential tests for moderation across the ITT and PP populations, and the three follow-up time points, none were found to be statistically significant at a p-value of < 0.05, although three moderation analyses yielded a p-value of < 0.1. In view of the low power to detect interaction effects, and in view of the number of tests performed, these possible interaction effects should also be viewed with caution.

The COBRA trial was the largest trial to date comparing CBT and BA and enabled an in-depth qualitative analysis of patients’ and therapists’/MHWs’ experiences of these treatments alongside the clinical effectiveness, cost-effectiveness and quantitative process analyses. To date, there have been very few qualitative studies of this. Our purposive sampling method for our participant interviews ensured diversity in our sample and we successfully interviewed participants from each recruitment site, both therapies, some who completed the full course/ended treatment early and some who were still depressed/no longer depressed, as well as both male and female participants across a range of ages. Similarly, we interviewed therapists/MHWs from both treatments, from all three recruitment sites and with a range of backgrounds and experience. However, the generalisability of our findings are limited to participants who were both eligible and willing to participate in the COBRA trial, and participants who declined to take part in the qualitative study may also have had different views to those who agreed to be interviewed. Likewise, qualitative interviews were only completed with 13 of the 22 therapists/MHWs involved in the trial, and those who did not respond to the invitation to be interviewed may have had different experiences to those who did respond.

All interviews were carried out over the telephone as a result of the long-distance nature of cross-site interviewing. This maintained researcher blindness for participant quantitative follow-ups and was crucial for the integrity of the COBRA trial. Although some researchers have proposed that telephone interviewing may be less effective than face to face,109 evidence suggests that telephone interviews yield the same number and quality of data as those conducted face to face,110 and some argue that telephone interviewing may even be preferable when participants are discussing sensitive topics. 111 Finally, although we aimed to interview participants as soon as possible after completion of therapy, in practice this was difficult because of delays in obtaining information about therapy end dates, delays in the ability to recruit participants quickly after this time point, and an overall difficulty recruiting participants who had fewer than eight sessions of therapy, resulting in an extension of our qualitative recruitment period. A small number of participants commented that at the time of their interview they found it difficult to remember specific aspects of their treatment, and it is possible that participants who were interviewed soon after therapy completion may have had different reflections on their experiences from those interviewed some time later.

We recruited participants from primary care, rather than specialist settings. Our results are, therefore, applicable to the great majority of patients with depression in these settings. We excluded people who had other major psychiatric diagnoses, people who are most likely to be found in specialist mental health services. We did so because, notwithstanding their depression, people with diagnoses such as addictive disorders, bipolar disorder or psychosis require specialist treatment for these disorders as their primary psychiatric input. We excluded people with these conditions on the understanding that psychological treatment for their depression would not be the first line of treatment offered. Therefore, for this population, our results may not be generalisable.

Implications for health care

Our findings could have substantial implications for the scalability of psychological treatment for depression in the UK and internationally,17 given the greater availability and ease with which a BA workforce could be trained than could a CBT workforce. For many years, CBT has been the foremost psychological therapy recommended by therapists, researchers and policy-makers. Our results challenge this dominance. Although more work needs to be done than has been undertaken so far to find ways to effectively treat the 20–30% of participants whose depression was unchanged by BA or CBT, our findings suggest that BA should be a front-line treatment for depression, with significant potential to improve reach and access to psychological therapy globally.

Our results in both groups compare favourably with a meta-analysis112 of the effects of CBT compared with second-generation ADM. This analysis found no difference between ADM and CBT on rates of remission between 12 and 16 weeks post randomisation (ADM, 40.7%; CBT, 47.9%; risk ratio 0.98, 95% CI 0.73 to 1.32). For comparison purposes, our ITT recovery rates (synonymous with remission in the meta-analysis) at 18 months post randomisation were 66% for BA and 59% for CBT. Equally, in the meta-analysis referred to above, response rates were 44.2% for ADM and 45.5% for CBT (risk ratio 0.91, 95% CI 0.77 to 1.07); for comparison, 61% of our participants receiving BA and 60% of those receiving CBT had responded at 18 months. Thus, the proportion of participants in both treatment arms who experienced long-term positive clinical outcomes was higher in our study than in this recent meta-analysis.

Our cost-effectiveness analyses show the high probability that BA is cost-effective and affordable compared with CBT at standard willingness-to-pay thresholds. Our most striking finding is that BA leads to comparable clinical outcomes for patients with depression, but at a financial saving to clinical providers of 21% compared with the cost of provision of CBT, with no compensatory use of other health-care services by patients.

Driving these savings is the fact that BA can be delivered by inexperienced MHWs with no professional training in psychological therapies, with no lesser effect than that of more highly trained and experienced psychological therapists giving patients CBT. Although training of MHWs is only one of many obstacles to successful dissemination, our findings suggest that health services globally could reduce the need for costly professional training and infrastructure, reduce waiting times and increase access to psychological therapies. 17 Our findings have substantial implications given the increasing global pressure for cost containment across health systems in high-income countries, and the need to develop accessible, scalable interventions in low- and medium-income countries. Such countries might choose to investigate the training and employment of junior workers over expensive groups of psychological professionals. Our results, therefore, offer hope to many societies, cultures and communities worldwide, rich and poor, struggling with the effect of depression on the health of their people and economies.

The results of the moderation analysis did not provide any strong evidence for differential treatment effect across subgroups of participants, although there was some weak evidence for a stronger beneficial effect of BA among participants who were more severely depressed at baseline. This finding is consistent with a previous study,40 and may indicate that, clinically, BA would be the preferred choice of therapy for more severely depressed patients.

The mediation analyses indicated that the quality of therapy, with regard to coverage of core topics, is the strongest mediator of treatment effect. Hence, this may provide guidance to therapists that they should concentrate on including the core topics within their therapy sessions. The mediation effects of the psychological mediators appear to be weaker than those of the mediators related to therapy fidelity, and clear findings with regard to the mediation effects of the psychological mediators were not observed.

Our qualitative findings suggest that, despite BA being non-inferior to CBT in terms of clinical outcomes, junior MHWs delivering BA could feel somewhat overwhelmed by the new skills they were learning and felt an initial lack of confidence in their ability to deliver therapy effectively. This lack of confidence was unique to BA, and was able to be detected by some patients. This is unsurprising given the new role that these workers were being asked to undertake, unlike CBT therapists, who had already been trained and were experienced in delivering CBT. BA delivered by therapeutically naive junior MHWs, therefore, requires good-quality training, time for therapists to build confidence through practice and ongoing supervision from experienced BA practitioners. Indeed, therapists/MHWs in both groups described training and supervision positively, and the model of support provided in the COBRA trial appears to be effective and well regarded by those delivering therapy. In particular, having a week of in-depth training, group supervision with peers and playing out of audio-clips from therapy during supervision were elements of the COBRA trial support model that were regarded as helpful, and should be considered for future implementation of BA and CBT for depression.

For CBT therapists, the COBRA trial treatment manual was generally considered to have been useful, although there was also anxiety about the lack of flexibility permitted. When training therapists to follow a therapeutic manual such as those used in the COBRA trial, therapists should not be too flexible in their clinical practice at the expense of overall treatment fidelity, since our quantitative process analysis demonstrated that fidelity predicts clinical outcomes. In addition, as outcomes for CBT in the COBRA trial were superior to those seen in IAPT services,113 where therapists are not trained to follow a particular depression treatment manual, consideration should be given to whether or not it would be beneficial to routinely use such a treatment manual in IAPT.

Relapse prevention was an element of both treatments that was valued highly by therapists/MHWs and patients. This is important because research has shown that, even when patients make significant clinical improvements during therapy, they expect themselves to remain susceptible to depression and continue to implement techniques learnt in therapy as a way of managing what they consider to be a chronic condition. 114 Previous work has also suggested that the long-term effects of CBT are attributable to patients learning skills that they can continue to implement after therapy has ended. 115,116 Relapse prevention is therefore a crucial component of both therapy protocols, which should continue to be implemented.

Implications for future research

Despite the encouraging nature of both our overall treatment outcomes and the cost-effectiveness and non-inferiority of BA compared with CBT, there remain three issues that should be priorities for research. The first is how to engage the considerable number of people with depression who either do not start, or rapidly drop out of, psychological treatment. Neither treatment in our trial performed better than the other in terms of this engagement and retention. Multiple contextual, specific and common therapeutic factors may contribute to the hypothesis that some patients find psychological therapy either generically undesirable, impractical or specifically inappropriate to their needs. It is unlikely that there is one simple solution – for example, psychological treatments with a different theoretical orientation – that will overcome these diverse factors impeding patient engagement. Nonetheless, research into ways in which patients can be engaged more fully could be considered.

Second, even for the PP population, around 30% of people did not experience a clinically relevant change in their depression symptoms, whether they received BA or CBT. It remains a possibility that matching patients to treatments by specific patient-level moderator variables, as has been hypothesised by some researchers,117 would lead to better outcomes. Unfortunately, in terms of depression severity, although a phenomenon previously observed,40 our study has only provided weak evidence to support the hypothesis that BA has a stronger therapeutic effect for patients who are more severely depressed at the onset of treatment. In terms of potentially enhancing the mediating effect of a range of psychological variables, despite previous studies of CBT55 reporting the mediating effect of changes in dysfunctional attitudes and metacognitive awareness, we were unable to demonstrate any substantial effect of psychological mediators on outcome. We are unable to recommend, therefore, that therapists enhance their therapeutic focus on these areas with any guarantee of improving outcomes for patients. Our only substantive finding that fidelity to a clinical protocol mediates outcome in both BA and CBT merely emphasises the importance for MHWs and therapists of following an evidence-based clinical treatment closely in their work. Much more prospective research using precision medicine approaches such as the Personalised Advantage Index117 to test the potential of matching treatments to individual patient characteristics seems to be warranted.

Third, Kanter and Puspitasari118 have noted that, ‘now that we have support for BA as a treatment that is clinically effective and cost-effective, we can shift our efforts to focus on what is necessary to produce sustainable large-scale BA implementation across diverse geographical and cultural settings’. 118 The central rationale of BA, now supported by our COBRA trial results, is that it is also a simple treatment suitable for widespread dissemination beyond high-income countries. Knowledge for the sustainable dissemination and implementation of BA as an effective health technology to low- and medium-income countries remains suboptimal. Although there have been a number of studies in low- and medium-income countries, including India, Iran and Iraq,119–121 the outcome data are equivocal as these studies either do not use an optimum clinical protocol tested in RCTs, are underpowered, incorporate BA as one part of a multicomponent complex intervention, or focus on very tightly defined populations. Nonetheless, these international studies do provide some evidence for the cross-cultural face validity of BA. 118 We now need to identify the low- and medium-income countries workforce best able and available to deliver this simple treatment. We need to determine if the face validity of BA can go some way to overcoming mental health stigma and people’s reluctance to engage in treatment in different cultures. We need to engage with organisations and cultures unused to the concept of evidence-based mental health therapies. Finally, we need to apply our scientific methods in low and middle income countries contexts, not least our health economic methods, as these data will drive recommendations to implement BA widely or not.

In summary, future research should focus on strategies to improve the initial engagement of depressed people with psychological therapies, should examine ways to personalise and optimise the allocation of a range of evidence-based treatments, and should take an implementation science approach to dissemination and reach outside the high-income countries in which BA in particular has been developed and tested.

Contributions of authors

Professor David A Richards (Professor of Mental Health Services Research) was chief investigator, designed the study, was responsible for its conduct, and contributed to the writing and editing of the report.

Dr Shelley Rhodes (Trial Manager) was responsible for study management and data collection, and contributed to the writing and editing of the report.

Dr David Ekers (Nurse Consultant Primary Care Mental Health/Senior Visiting Research Fellow) designed the study, was responsible for its conduct, and contributed to the writing and editing of the report.

Dr Dean McMillan (Senior Lecturer) designed the study, was responsible for its conduct, and contributed to the writing and editing of the report.

Professor Rod S Taylor (Professor of Health Services Research) designed the study, was responsible for its conduct, undertook data analysis, and contributed to the writing and editing of the report.

Professor Sarah Byford (Professor of Health Economics) designed the study, was responsible for its conduct, undertook data analysis, and contributed to the writing and editing of the report.

Dr Barbara Barrett (Senior Lecturer, Health Services and Population Research) undertook data analysis, and contributed to the writing and editing of the report.

Katie Finning (Associate Research Fellow) was responsible for study management and data collection, undertook data analysis, and contributed to the writing and editing of the report.

Poushali Ganguli (Research Associate, Health Services and Population Research) undertook data analysis, and contributed to the writing and editing of the report.

Dr Fiona Warren (Lecturer in Medical Statistics) undertook data analysis, and contributed to the writing and editing of the report.

Dr Paul Farrand (Associate Professor) designed the study, was responsible for its conduct, and contributed to the writing and editing of the report.

Professor Simon Gilbody (Director of the Mental Health and Addictions Research Group) designed the study, was responsible for its conduct, and contributed to the writing and editing of the report.

Professor Willem Kuyken (Professor of Clinical Psychology) designed the study, was responsible for its conduct, and contributed to the writing and editing of the report.

Dr Heather O’Mahen (Senior Lecturer in Clinical Psychology) designed the study, was responsible for its conduct, and contributed to the writing and editing of the report.

Professor Ed Watkins (Professor of Experimental and Applied Clinical Psychology) designed the study, was responsible for its conduct, and contributed to the writing and editing of the report.

Dr Kim Wright (Senior Lecturer) designed the study, was responsible for its conduct, and contributed to the writing and editing of the report.

Nigel Reed (PPI representative) provided expert advice on PPI, and contributed to the writing and editing of the report.

Emily Fletcher (Trial Manager) was responsible for study management and data collection, and contributed to the writing and editing of the report.

Professor Steven D Hollon (Gertrude Conaway Vanderbilt Professor of Psychology) provided expert advice on clinical aspects of cognitive–behavioural therapy, and contributed to the writing and editing of the report.

Dr Lucy Moore (Research Fellow) undertook data analysis, and contributed to the writing and editing of the report.

Amy Backhouse (Associate Research Fellow) contributed to data collection, and contributed to the writing and editing of the report.

Claire Farrow (Associate Research Fellow) contributed to data collection, and contributed to the writing and editing of the report.

Julie Garry (Associate Research Fellow) contributed to data collection, and contributed to the writing and editing of the report.

Deborah Kemp (Associate Research Fellow) contributed to data collection, and contributed to the writing and editing of the report.

Faye Plummer (Associate Research Fellow) contributed to data collection, and contributed to the writing and editing of the report.

Faith Warner (Associate Research Fellow) contributed to data collection, and contributed to the writing and editing of the report.

Rebecca Woodhouse (Associate Research Fellow) contributed to data collection, and contributed to the writing and editing of the report.

All authors had full access to all of the data (including statistical reports and tables) in the study and can take responsibility for the integrity of the data and the accuracy of the data analysis.

Publications

Rhodes S, Richards DA, Ekers D, McMillan D, Byford S, Farrand PA, et al. Cost and outcome of behavioural activation versus cognitive behaviour therapy for depression (COBRA): study protocol for a randomised controlled trial. Trials 2014;15:29.

Richards DA, Ekers D, McMillan D, Taylor RS, Byford S, Warren FC, et al. Cost and outcome of behavioural activation versus cognitive–behavioural therapy for depression (COBRA): a randomised, controlled, non-inferiority trial. Lancet 2016;388:871–80.

Finning K, Richards DA, Moore L, Ekers D, McMillan D, Farrand PA, et al. Cost and outcome of behavioural activation versus cognitive–behavioural therapy for depression (COBRA): a qualitative process evaluation. BMJ Open 2017;7:e014161.

Data sharing statement

The authors confirm that all data underlying the findings are fully available without restriction. The authors have made the clinical and economic data set available through the University of Exeter’s Institutional Repository – Open Research Exeter (see https://ore.exeter.ac.uk). Access to these data is permitted but controlled through requests made via the repository to the chief investigator (Professor Richards: d.a.richards@exeter.ac.uk). Although use is permitted, this will be on the basis that the source of the data is acknowledged (including the funder) and it includes reference to the data set ‘handle’.

Disclaimers

This report presents independent research funded by the National Institute for Health Research (NIHR). The views and opinions expressed by authors in this publication are those of the authors and do not necessarily reflect those of the NHS, the NIHR, NETSCC, the HTA programme or the Department of Health. If there are verbatim quotations included in this publication the views and opinions expressed by the interviewees are those of the interviewees and do not necessarily reflect those of the authors, those of the NHS, the NIHR, NETSCC, the HTA programme or the Department of Health.