Radically open dialectical behaviour therapy for refractory depression: the RefraMED RCT

Treatment resistance

Major depressive disorder is a disabling condition that causes substantial impairment in psychosocial functioning and quality of life, especially in the workplace, at home and with friends. 1 By 2030, depression will be the leading cause of disability in developed countries and the second leading cause of disability worldwide. 2

Only 30–40% of individuals treated with antidepressant medication (ADM) achieve full remission and only half of individuals respond to psychological treatment. 3 About one-third of patients do not respond to ADM4 and, of those non-responding patients, fewer than one-third benefit from adding, or switching to, cognitive therapy. 5,6 Approximately 50% of patients who are diagnosed with major depressive disorder experience recurrent or chronic illness needing long-term treatment. 7 Those with chronic depression are least likely to respond to currently available treatments. 8,9 Thus, treatment resistance is a common outcome for individuals with major depression and reported prevalence rates range from 15% to 60%, depending on how treatment-resistant depression (TRD) is defined. 7,10–12

Defining refractory depression

Definitions of TRD and refractory depression vary across studies; for example, Berlim and Turecki3 found > 10 different definitions of TRD. In practice, chronic depression and TRD often overlap, with many patients meeting both definitions. In the present study, we define ‘refractory depression’ as comprising both TRD, that is, depression that does not respond to adequate intervention, and chronic depression, that is, depression lasting > 2 years.

In a recent systematic review of TRD, Negt et al. 9 found that, compared with acute or episodic depression, chronic depression is associated with earlier age at onset, higher rates of adverse childhood experiences, more comorbid disorders and poorer social adjustment.

The costs of refractory depression

The economic burden of depression is substantial, especially for refractory depression. The total cost of adult depression across England in 2007 was > £7.5B, including £1.6B for patient care (treatment and social care) and £5.8B for loss of earnings. 13 Many of these costs are a result of refractory depression; for example, Crown et al. 7 found that depression-related costs for treatment-resistant inpatients were 19 times greater than those for patients not meeting criteria for treatment resistance and the costs for treatment-resistant outpatients were 2.5 times greater than for those not meeting the criteria. Similarly, in a US-based study, Ivanova et al. 14 reported that, after adjusting for differences in baseline comorbidities, risk-adjusted costs for employees with TRD were US$11,600 more than for depressed control employees who did not have TRD.

Psychological intervention research for refractory depression

Most studies investigating psychological interventions for depression have focused on acute or episodic depression rather than refractory depression.

For acute or episodic depression, cognitive–behavioural therapy (CBT), behavioural activation, interpersonal psychotherapy (IPT) and short-term psychodynamic psychotherapy have the strongest empirical support. 9 Meta-analyses of psychotherapies for non-chronic depression typically find effects (Cohen’s d) relative to waiting list and minimal treatment control groups ranging from a d of 0.70 to a d of 0.90, whereas placebo control groups result in smaller effect sizes, averaging a d of 0.36. 15 A recent systematic review found weak evidence that behavioural therapies and other psychological therapies for acute depression are equally effective. 16

The evidence for these treatments for chronic depression is much more limited. 9 Cuijpers et al. 15 conducted a meta-analysis of studies investigating psychological treatments for chronic depression or dysthymia. It included 16 studies published between 1966 and 2009, of which only four studied chronic major depression; the remainder studied dysthymia, double depression (both major depressive disorder and dysthymia), or other categories of chronic depression or dysthymia. Of the psychological treatments investigated, seven were CBT and six were IPT, in addition to eight other treatments, including the Cognitive Behavioural Analysis System of Psychotherapy (CBASP), problem-solving therapy, cognitive-interpersonal group psychotherapy for chronic depression and supportive therapy. The authors reported a small but significant effect size (Cohen’s d) with a d of 0.23 for psychotherapy on depression symptom scores when compared with control groups. McPherson et al. 17 identified 12 studies investigating psychotherapeutic treatment for TRD, but only four were controlled trials and none recruited > 25 participants;17 the effect sizes ranged from a d of 1.23 to a d of 3.10.

One of the reasons that there are fewer trials that investigate psychotherapy for refractory depression is because most established treatments have been developed for less severe forms of depression. However, recently, treatments and studies have been developed specifically for refractory depression. The most investigated treatment to date is CBASP, a behavioural analytic therapy focusing on behaviour and its consequences. 18 McCullough18 hypothesises that chronically depressed individuals have deficits in social problem-solving and interpersonal communication, arising from early adverse events (AEs). Although the treatment was originally developed for individual therapy, CBASP has been modified for group formats. 19,20 A recent meta-analysis of six randomised trials,19,21–25 studying 1510 patients, found a small effect size for CBASP, with a d of 0.34. 9 Remission rates ranged from 19% to 57% in the CBASP groups (median 35%), compared with 6–50% in the control groups (median 25%).

Another recently developed treatment for chronic depression is the Relief of Chronic Or Resistant Depression (Re-ChORD) programme, an intensive, time-limited outpatient programme lasting 4 months. The major components of the Re-ChORD programme are medication management, group-based IPT and group occupational therapy. 26 The Re-ChORD study was a parallel-group, open randomised trial comparing the Re-ChORD programme with treatment as usual (TAU). This study was somewhat underpowered because (1) the authors chose a binary primary outcome [clinical remission, defined as a Hamilton Rating Scale for Depression (HRSD) score of ≤ 7 points after 4 months] and (2) they recruited only 64 patients. As a result, the difference in the proportion of patients remitting after 4 months (36% in the Re-ChORD programme group vs. 13% in the TAU group) was only on the borderline of statistical significance.

The Tavistock Adult Depression Study27 was a small pragmatic randomised trial evaluating long-term psychoanalytic psychotherapy (LTPP) as an adjunct to TAU for 129 patients with long-standing major depression who had failed to respond to at least two different treatments. Patients received either TAU alone or LTPP plus TAU and were assessed at 6-monthly intervals until 42 months, including the 18 months of treatment. Complete remission (i.e. a HRSD score of ≤ 8 points) was infrequent and never significantly different between groups. Partial remission (i.e. a HRSD score of ≤ 12 points) was significantly more likely in the LTPP plus TAU group than in the TAU-alone group (at 24 months, 39% vs. 19%; at 30 months, 35% vs. 12%; and at 42 months, 30% vs. 44%). Similarly, the difference between the group means became significant only at 24 months, when the LTPP plus TAU group started to show significantly larger decreases in HRSD scores than those shown in the TAU-alone group, with moderate effect sizes.

The CoBalT (Cognitive Behavioural Therapy as an adjunct to pharmacotherapy for primary care patients with treatment-resistant depression: a randomised controlled trial)28 recruited 213 primary care patients with TRD to evaluate the clinical effectiveness of CBT as an adjunct to TAU, compared with TAU alone. The trial found that 28% of patients in the CBT group and 15% of those in the TAU group achieved remission [i.e. a Beck Depression Inventory^®-II (BDI^®-II) score of < 10 points]. The difference between groups in improving BDI-II scores had an effect size with a d of 0.53. A recently published follow-up study29 reported that this effect size still had a d of 0.45 at around 40 months after the end of treatment (BDI-II scores at 6 months: CBT, 18.9 points and TAU, 24.5 points; BDI-II scores at 40 months: CBT, 19.2 points and TAU, 23.4 points). The CoBalT team concluded that CBT as an adjunct to usual care that includes ADM is clinically effective and cost-effective over the long term for individuals whose depression has not responded to pharmacotherapy.

In summary, immediately after treatment, CBASP had a small beneficial effect on depression,9 whereas CBT and the Re-ChORD programme had moderate beneficial effects on depression. 26,28 Moreover, emerging evidence suggests that longer and more complex treatments may be needed for treatment-resistant and chronic forms of depression than for more acute forms of depression. 30

Major paradigm shift: refractory depression reconceptualised as personality dysfunction

A major premise of the Refractory depression: Mechanisms and Efficacy of RO DBT (RefraMED) trial is that when psychopathology is long-standing and does not respond to efficacious first-line treatments, then broad-based personality dimensions and overlearned perceptual and regulatory biases are preventing change. Biotemperamental deficits, combined with damaging family environmental influences, are hypothesised to severely handicap openness and flexible responding, resulting in habitual overcontrol or undercontrol of socioemotional behaviour. This distinction shares features with the well-established division between internalising and externalising disorders. 31 Major depressive disorder exemplifies this perspective of limited openness and flexible responding, resulting in an overcontrolled coping style. This is further supported by studies reporting that 40–60% of unipolar depressed patients meet the criteria for comorbid personality disorder (PD),32–34 the most common being paranoid, avoidant and obsessive–compulsive PDs, all of which are overcontrolled PDs that do not respond well to treatment. 8,32,35 However, most trials exclude the most severely unwell individuals, for example those with PDs who are known to respond less favourably to existing treatments for acute depression, notably CBT. 8

Previous studies have reported that adults with chronic depression are characterised by overcontrolled traits, including rigid internalised expectations, excessive control of spontaneous emotion, greater self-criticism, inordinate fears of making mistakes and impaired autonomy. 34 Excessive self-control has been linked to social isolation, aloof interpersonal functioning, maladaptive perfectionism, disingenuous emotional expression and mental health problems such as anorexia nervosa, obsessive–compulsive PD and chronic depression. 36–40 However, current treatments neglect the potential mediating role of personality and maladaptive emotional overcontrol in refractory depression.

Radically open dialectical behaviour therapy

A novel transdiagnostic psychotherapy,41,42 RO DBT was designed specifically to address maladaptive emotional overcontrol and associated loneliness. The neurobiosocial theory underlying RO DBT proposes that individuals presenting with problems of overcontrol are biotemperamentally predisposed to exhibit heightened threat sensitivity, diminished reward sensitivity and heightened capacities for self-control and detailed-focused processing. 41,42 These biotemperamental biases can be strengthened by family, cultural and environmental histories that value performance and self-control and by avoiding risk and masking emotions. A person with heightened threat sensitivity is more likely to scan new or unfamiliar circumstances for potential harm rather than reward: their sympathetic nervous system – responsible for defensive-arousal and flight–fight responses – is activated while the ventral vagal complex of their parasympathetic nervous system withdraws. 43 Unfortunately, as a result of withdrawal of the parasympathetic nervous system ventral vagal complex, facial expressions become frozen and the ability to express oneself flexibly is lost; in other words, prosocial co-operative social signalling becomes impaired. 44 People who are inexpressive or who show expressions that do not match their inner experiences are often perceived as inauthentic or untrustworthy by others, which may lead to social ostracism and emotional loneliness. 41,45,46 Consequently, RO DBT emphasises the importance of targeting social-signalling deficits and uniquely links the communicative and facilitative functions of emotional expressions to the formation of close social bonds and uses micromimicry as a means to understand and increase emphatic behaviours. 41,42,47 The skills that are taught in RO DBT focus on relaxing rigid inhibitory self-control, activating the social safety system, expressing context-appropriate emotions and vulnerable self-disclosure, practising self-enquiry to learn from new experiences and critical feedback, and reducing maladaptive social comparisons related to envy and bitterness.

Earlier versions of RO DBT (RO DBT-E) have shown promise in two small randomised trials48,49 for patients with refractory depression and comorbid PD. The first study49 explored the feasibility of a group intervention for TRD in chronically depressed older adults aged > 60 years. The treatment comprised 28 weeks attending a skills training group and weekly 30-minute telephone contact with an individual therapist, followed by 3 months in which telephone contact was made every 2 weeks and 3 months in which it was made every 3 weeks. The 17 participants who received RO DBT-E showed significantly greater improvements than the 17 control participants in self-rated and interviewer-rated depression (d = 0.71). Post-treatment interviewer ratings showed that 71% of RO DBT-E recipients met criteria for remission [i.e. a Beck Depression Inventory^® (BDI) score of ≤ 9 points or a HRSD score of ≤ 7 points], but only 47% of control participants did so. After 6 months, the corresponding percentages were 75% and 31%, respectively. In addition, RO DBT-E recipients improved significantly in adaptive coping and dependency, whereas control participants did not.

The second trial48 compared 24 weeks of both individual and group RO DBT-E plus ADM in 21 adults aged > 55 years who had PD and depression with ADM alone in 14 similar adults. To be included, participants had to demonstrate TRD prospectively by having a poor response to an 8-week course of researcher-controlled ADM. The RO DBT-E recipients showed significantly greater decreases in interpersonal sensitivity and aggression than control participants. At the end of the skills group, 71% of RO DBT-E recipients were in remission (i.e. had a HRSD score of < 10 points), compared with only 50% of control participants. Both groups of participants showed significant reductions in clinician-rated depression, with a between-group d of 0.85.

Efficacy

The primary objective was to estimate the efficacy of RO DBT for refractory depression in addition to TAU, compared with TAU alone, over the course of 18 months. This objective can be achieved in two ways: first, as the effect of being randomly allocated to RO DBT plus TAU, rather than TAU alone; and, second, as the effect of exposure to specific ‘doses’ of RO DBT, in which exposure was measured by adherence to RO DBT treatment protocols and zero exposure corresponded to TAU. For both methods, the primary outcome was depressive symptoms measured using HRSD score at 12 months (i.e. 5 months after the end of treatment). To interpret this outcome, the resulting rates of remission and measures of other symptoms were used, including suicidal ideation or behaviour and global functioning.

Conventional analysis ‘by treatment allocated’ was used to fulfil the first of these approaches. Although this analysis provided an unbiased pragmatic estimate of the effectiveness of RO DBT in clinical practice, this is not identical to ‘efficacy’ under ideal conditions because participants varied in their adherence to the recommended course of treatment, and, therefore, there was heterogeneity in exposure to RO DBT. To account for this, estimations of the causal effect of exposure to RO DBT were sought. In doing so, this allowed for the fact that attendance at therapy sessions occurred after randomisation and, thus, could be subject to confounding.

Cost-effectiveness

Analysis ‘by treatment allocated’ was used to estimate the relative cost–utility of RO DBT plus TAU in comparison with TAU alone over 12 months. We calculated cost per quality-adjusted life-year (QALY) using the EuroQoL Questionnaire-5 Dimensions, three-level version (EQ-5D-3L). 50 We also explored cost-effectiveness using HRSD score. The economic perspective was that of the NHS and personal social services, as preferred by the National Institute for Health and Care Excellence (NICE). 51 The addition of productivity losses was explored, resulting from time off work because of illness, in the sensitivity analysis.

Mechanisms

We extended the efficacy analysis, which addressed questions relating to the effect of allocation and exposure to treatment, to ask questions about how RO DBT might have been efficacious. The approach included both RO DBT-specific and transtheoretical concepts, and recognised that important elements of psychotherapy might be common to many treatments.

The primary aim was to estimate the causal effects of the pathways through which RO DBT is hypothesised to reduce depressive symptoms (as measured via the HRSD score) for patients in the trial. The chief challenge in doing this was the potential presence of unobserved confounding. Bias from unobserved confounding potentially arises because, unlike the treatment itself, patients are not randomly allocated to the number of sessions that they attend, the strength of the therapeutic alliances that they develop with their therapists or the degree to which they acquire new skills through engaging with the treatment. Patients make decisions to engage with treatment in a non-random manner that may be related to their underlying mental health. For example, suppose that patients with certain characteristics choose to attend as many sessions as possible, form strong alliances with their therapists and develop the skills needed for RO DBT; if these same characteristics are also associated with a milder form of refractory depression, then the statistical estimates of these pathways obtained using standard pathway models52 will overstate the causal effect of each pathway because of uncontrolled confounding bias.

There are two main objectives of this analysis:

1. to specify a series of pathway models to decompose the overall effect of RO DBT into that resulting from the pathways through which the treatment is hypothesised to work

2. to estimate the decomposition of this treatment effect using:

   • standard adjustments for confounding variables

   • instrumental variables (IVs) to adjust for possible unobserved confounding.

The analyses focused on four specific pathways between allocation and outcome: treatment exposure, therapeutic alliance, skill acquisition and expectancy. To estimate the causal effect of mediators, the trial manipulated selected mediators in a random fashion. We also measured variables that represented sources of variation in mediators that were unlikely to be contaminated by selection effects. In other words, IVs were identified, both experimental and observational, to facilitate causal analyses.

In addition to these primary pathways, several potential modifiers of treatment outcomes were measured. Based on this study and other research showing links between refractory depression and PD,32 temperamental risk aversion and reward insensitivity53–55 and childhood adversity,56 we assessed potential moderators of treatment response by measuring the following at baseline:

1. PD diagnosis

2. invalidating childhood experiences

3. reward sensitivity or risk aversion.

These variables will be used to conduct a complementary analysis of repeated measurements of outcomes and mediators using longitudinal models. This analysis will address theoretically driven questions about the ordering of changes in key variables.

Temporal patterns and precedents of change

In conjunction with the causal analyses of the therapeutic alliance, the study wanted to examine patterns of change and cross-lagged effects among depressive symptoms, alliance ratings and skills learned in RO DBT. In addition, whether or not there are differential rates of change in positive versus negative affect and whether or not different patterns of temporal ordering exist for positive affect and negative affect will be investigated. Previous research has highlighted the distinct and important role of positive affect in adaptive coping. 57 Positive affect is thought to broaden an individual’s attentional focus and behavioural repertoire and, as a consequence, build social, intellectual and physical resources. 58 A change in positive affect is expected to be more rapid and more closely associated with factors common to psychosocial interventions (e.g. expectancy and alliance) than for change in negative affect and to precede improvements in coping strategy. Furthermore, ecological momentary assessment will enable questions to be answered in relation to the variability of affect in treated versus untreated patients. Daily variability in affect is expected to rise early in treatment for RO DBT participants (as a consequence of the difficult work clients undertake with therapists), but to decline relative to the TAU group patients by 7 months. Although these longitudinal analyses estimate temporal ordering or so-called ‘Granger causality’ rather than true causality, they complement the primary mediation analyses by providing a richer picture of patterns of change in response to treatment.

Adjusted sample size targets

Recruitment rates in all centres were limited by available assessor time. The consequent difficulty in meeting target recruitment rates and a reduced therapist capacity in one centre forced there to be a re-evaluation of the target sample size. In consultation with the Data Monitoring and Ethics Committee (DMEC), we agreed to recalculate the sample size using less conservative assumptions and, consequently, reduced the target number of patients randomised to 240.

Primary randomisation and allocation ratio

Participants were allocated between treatments through an adaptive randomisation algorithm administered by the Swansea Trials Unit. This system of randomisation maintained the balance across groups of the three stratifying variables stochastically, rather than deterministically, chosen as potential outcome moderators to minimise the risk of subversion:60

1. early onset of depression (before or after 21 years of age)

2. depression severity at baseline (a HRSD score of < or ≥ 25 points)

3. PD [meets the Structured Clinical Interview for DSM-IV Axis II (SCID-II) criteria for cluster A or cluster C or not].

Participants were allocated to RO DBT or TAU using a final allocation ratio of 15 : 8 in favour of RO DBT.

Secondary randomisation

Within the RO DBT group, participants were further randomly allocated to their therapist and the three IVs listed below. An adaptive randomisation method was used to allocate patients between therapists so as to use as many as feasible of the treatment slots at each centre. We also randomly allocated all RO DBT participants to one of eight combinations of contextual non-therapeutic features (i.e. IVs) within a 2 × 2 × 2 factorial design:

1. the participants received individual RO DBT in either a standard therapy room or an enhanced therapy room that potentially improves both the strength of the alliance with the therapist and attendance at treatment sessions

2. after every session the participants either did or did not have the opportunity to provide written feedback to their therapist, which potentially improves the strength of the alliance with the therapist61

3. the participants received compensation for questionnaire completion either personally (via their therapists) or impersonally (via mail), potentially increasing attendance at treatment sessions.

The allocation sequence was determined dynamically using a database independently administered at the Swansea Trials Unit. The database incorporated a study-specific version of a published dynamic randomisation allocation method62 and the resulting allocations were then sent by e-mail to the trial manager for further dissemination to participants and study therapists.

Inclusion criteria

Participants were considered eligible if they were aged ≥ 18 years, had a HRSD63 score of at least 15 points (note that a HRSD score of 14 points is regularly used as the upper limit for partial remission), had a current diagnosis of major depressive disorder [as assessed with the Structured Clinical Interview for DSM-IV Axis I Disorders (SCID-I)64] and had refractory depression, which was defined as two or more previous episodes of depression or having experienced their current episode for ≥ 2 years. Finally, participants had to have taken an adequate dose of ADM for at least 6 weeks without symptom relief during their current episode.

Exclusion criteria

We excluded those participants who met criteria for dramatic–erratic PD (cluster B), bipolar depression, psychosis or a primary diagnosis of substance dependence or substance abuse disorder. Patients had to have an intelligence quotient of > 70 and speak English well enough to participate in the treatment and the study. Finally, patients who were receiving standard DBT at the time of recruitment were not eligible to take part in the current study because standard DBT aims to treat borderline PD, which was one of the exclusion criteria.

Discontinuations

Participants were allowed to withdraw from the treatment or the study at any time without their regular care being affected. Treatment withdrawal was defined as participants being unable or unwilling to attend RO DBT sessions; on such occasions, we encouraged participants to provide follow-up data. Study withdrawals were defined as participants being unable or unwilling to attend follow-up sessions; on such occasions, we permitted participants to continue with RO DBT if they had been allocated to that treatment arm. However, if participants withdrew consent to participate in the study, we did not seek additional data.

Recruitment and eligibility screening

Participants were recruited through mental health professionals, such as psychologists, psychiatrists and mental health nurses, through general practitioners (GPs) and via advertisements in secondary and primary care clinics (e.g. waiting rooms or GP surgeries) and in the community (e.g. libraries). We carried out database searches for potential participants in the primary and secondary care services, although GPs and secondary care clinicians could also refer patients directly to the study. In addition, patients could self-refer after seeing posters, leaflets or the website.

In the primary care settings, database searches were conducted by the GP practice staff; medical records were checked for eligibility using the inclusion and exclusion criteria and the resulting lists were screened by GPs. The GPs and practice staff then signed pre-prepared letters describing the study and inviting patients to contact their local RefraMED clinical studies officer if they were considering participating in the trial. Patients who called to express an interest provided oral consent to be screened for eligibility.

In secondary care, database searches were carried out by the local RefraMED clinical studies officers. The resulting lists were screened by the responsible clinicians before letters were sent out. Patients received letters describing the study and inviting them to opt out or consider participating in the trial. Unless patients opted out, the clinical studies officer tried to contact them by telephone to discuss the study and, with their oral consent, screened them for eligibility.

Recruitment started at the Dorset site in March 2012 and at the other two sites in September 2012, and ended at all sites in March 2015. The last follow-up interviews were completed in May 2016.

Eligibility interview

In-person appointments were made with trained assessors for the potential participants who were eligible at the telephone screening stage and willing to attend the study assessment. The participants signed a formal consent form on arrival. The team of trained assessors screened them for eligibility and conducted baseline assessments of the outcome measures set out in Table 1 for those who were found to be eligible for study inclusion. After the interview, participants received further questionnaires for home completion.

Assessment training and inter-rater reliability

Assessors were trained to competence on the Structured Clinical Interviews for DSM-IV (SCID) and HRSD by a clinical psychologist who was experienced in administering the SCID and HRSD in clinical trials. Ongoing queries regarding the SCID and HRSD were discussed at weekly consensus meetings and inter-rates reliability on the HRSD was re-assessed at regular (9-month) intervals to ensure ongoing adherence with the HRSD.

Following the recommended training sequence of SCID, all assessors read through the SCID and HRSD and watched the SCID training digital versatile discs (DVDs). Assessors watched key sections of the SCID training tapes of interest for this study (i.e. mood episodes, anxiety disorders and DSM-IV Axis II) together with the trainer, in order to ensure depth of diagnostic and assessment learning. In-group learning included role-plays of both the SCID and HRSD with assessors and with the trainer, followed by questions and discussion. Assessors subsequently conducted role-plays of the SCID and HRSD with a colleague and compared their ratings with the expert ratings from the training DVDs. Assessors then either audiotaped an assessment and had this assessment reviewed by the trainer or shadowed an experienced assessor and completed an initial joint assessment.

We included an embedded ‘reliability study’ in which all assessors coded audiotaped interviews that had been taped and coded by another member of the team. Krippendorff’s alpha65 was used, as it can assess reliability between multiple raters and it handles nominal data and missing data.

Blinding

The assessors were blind to participant allocation. All participants were informed of allocation by the trial manager or administrator. Follow-up assessments were carried out at different locations from treatment locations to ensure that the research assistants carrying out the follow-ups remained blind to treatment allocations. Precautionary strategies included consideration of locations for follow-up assessments, participants being reminded by the assessors not to talk about allocation before the assessment and, after the initial assessment, assessors not looking at clinical notes. If an allocation was revealed, then reblinding occurred by using another rater for the subsequent follow-up. If the blinding was broken during an assessment session, then these ratings were used. Unblinded ratings were used for 17 7-month follow-ups, 12 12-month follow-ups and zero 18-month follow-ups.

Reimbursement

Participants in both groups were reimbursed by cheque for their time spent completing research questionnaires and attending the follow-up interviews. The total amount of reimbursement ranged from £0 to £150, depending on whether or not the participants were still in the study and the number of follow-up assessments they attended. Participants received £30 after they had been in the study for 3 months, £40 for attending the first follow-up interview after 7 months, another £50 for attending the follow-up interview after 12 months and a final £30 for attending the follow-up interview after 18 months.

Settings and locations of data collection

Participants were recruited, assessed and treated at three different NHS trusts across the UK: Dorset HealthCare University NHS Foundation Trust, with locations in Bournemouth and Poole (Dorset); Southern Health NHS Foundation Trust, with locations mainly in Winchester, Eastleigh and Southampton (Hampshire); and Betsi Cadwaladr University Health Board, with locations in Bangor and Rhyl (North Wales). All three sites routinely offer outpatient psychological treatment.

To ensure accuracy, completeness and reliability, trained assessors collected outcome data during interviews in person or by telephone following standard operating procedures for data collection and transferred the data to the trial office in Southampton. In addition, monthly paper questionnaires were posted directly to participants with a postage paid envelope, and text or e-mail reminders were sent after 10 days if the questionnaires had not been received by the trial office.

Electronic data capture system

All paper-based data were entered twice onto Signalbox (version 2.3.7; B Whalley, Plymouth, UK),66 a validated electronic data capture system. All duplicate replies were then checked by an experienced data manager and, in the case of discrepancies between the two replies, the correct reply was selected for further processing. This system also collected data from automated telephone calls used to collect certain measures (described in Measures) and prompted therapists to enter their online treatment notes through weekly electronic reminders.

Reporting of adverse events and study termination

Site principal investigators were responsible for monitoring and reporting serious adverse events (SAEs). These were immediately reported to the chief investigator. The trial office reported these events annually to the DMEC or immediately in the case of a suspected unexpected serious adverse reaction (SUSAR). The trial procedure for evaluating and reporting AEs and SAEs can be found in Appendix 2.

Treatment as usual

All participants received TAU, consisting (primarily) of prescribed ADM or psychotherapy (with the exception of standard DBT). At each follow-up assessment, participants were asked to report the type of ADM, their adherence to it and the type and amount of psychotherapy that they had accessed in the months since their previous assessment (or in the 6 months preceding their baseline assessment). In addition, we did not restrict access to appropriate mental health care during follow-up. ‘Service contact’ may have served different functions, namely for the purpose of (1) assessment, (2) psychological treatment or (3) routine clinical monitoring of needs and risk. TAU differed across sites.

In Dorset, the practice of many services was to assess patients (with services typically offering three assessments) to formulate their difficulties and determine their needs, before putting their name on a waiting list for appropriate treatment. During the waiting period, the patient may have received input from a Community Mental Health Team (CMHT) to review their needs, risk and medication. Patients who had been referred to CMHTs received variable treatment, which may have included CBT, schema-focused cognitive therapy, standard DBT-based skills groups or cognitive–analytic therapy (CAT), all of which delivered was by psychologists, occupational therapists or nurses. Alternatively, the patients were case managed by CMHT staff, which did not include a therapeutic model as such. In many services the wait to receive treatment was long, typically up to 1 year. Some patients were seen only by their GP and TAU consisted of ADM and whatever support their GP offered in the administration and monitoring of ADM.

In North Wales, patients referred to mental health services were assessed via a single-point-of-access system that occurred within hours for urgent cases and within 28 days for routine cases. Typically, all of these patients were already in receipt of ADM. Patients who were assessed as having mild depression were seen in primary care mental health services, in which the interventions offered included basic behavioural activation, support and monitoring of ADM and occasionally CBT. Others may also have been referred to third-sector counselling services and, in some areas, access to mindfulness-based cognitive therapy groups was available. Patients with moderate to severe problems were seen in secondary care by the CMHTs and would typically have a generic case manager and may also have been seen by the psychiatrist or a clinical psychologist. In most areas, the clinical psychologist would offer a formulation-based psychological intervention; the models offered would include CBT, dialectical behavior therapy (DBT)-informed, schema-focused cognitive therapy, acceptance and commitment therapy and CAT. Depending on their geographical area, patients may also have received CBT from a CBT therapist. Waiting lists for therapy were long, ranging from 3 months to 2 years, depending on the area, and the approximate average wait was 12 months.

In Hampshire, some GP practices and all universities provide a counselling service. There is also an active voluntary sector, which provides support, information, counselling and group work. Patients could self-refer or be referred to Increasing Access to Psychological Therapies (IAPT) services. IAPT offers a stepped approach to treatment for anxiety and depression with telephone sessions, group sessions, couple work or up to 20 individual sessions of CBT or IPT. Patients with more severe and enduring difficulties were referred to their CMHT by their GP. The patients were typically seen within a few weeks for an assessment of their needs. They may also have received regular monitoring and case-management by CMHT staff, including problem-solving, reviewing of risk and medication advice or being offered a range of treatment options, for example, group work, such as mindfulness or standard DBT-based skills groups. If the patient required more intensive input, then a home treatment team or admission to an inpatient acute unit was available. In addition to these options, patients could have been referred for specific psychological therapy. The waiting time would typically be up to 4 months from referral, and therapy options included group and individual work using CBT, DBT-informed, schema-focused cognitive therapy, CAT or acceptance and commitment therapy.

Radically open dialectical behaviour therapy

Radically open dialectical behaviour therapy is a transdiagnostic treatment designed to address a spectrum of difficult-to-treat disorders, including chronic depression. RO DBT significantly differs from other treatment approaches, most notably by linking the communicative functions of emotional expression to the formation of close social bonds and by skills targeting social signalling and changing neurophysiological arousal.

The experimental intervention was fully manualised and comprised 29 weekly individual therapy sessions lasting 50–60 minutes and 27 weekly skills training classes lasting 2.5 hours, including 15-minute breaks. 41,42,69 In addition, RO DBT therapists met weekly in person for a consultation team meeting that lasted 1.5–2.5 hours and by telephone when needed. The RO DBT treatment commenced as soon as possible after the participant had been notified of their treatment allocation. Although RO DBT participants received ADM as prescribed, we strongly discouraged them from seeking additional psychotherapy during RO DBT. The RO DBT treatment developer and study chief investigator (Thomas R Lynch) did not contribute to treatment delivery.

The RO DBT lesson plan used in this trial is presented in Appendix 3. The research manual comprised a combination of newly developed RO DBT lessons, which are now part of a fully comprehensive RO DBT skills manual,69 and standard DBT. 70

Radically open dialectical behaviour therapy therapist training, supervision and adherence

Twenty-three therapists (male, n = 2, female, n = 21; age range from 32 to 61 years), who were trained in RO DBT, delivered the treatment across the three sites (Dorset, n = 8 therapists; Hampshire, n = 10 therapists; and North Wales, n = 5 therapists). All therapists received a minimum of 10 days of specific training in RO DBT from the treatment developer and chief investigator of the study (Thomas R Lynch).

Primary outcomes: depression symptoms

The primary outcome was the difference in depressive symptoms at 12 months between patients receiving RO DBT and TAU, as measured using the 17-item HRSD. 63 The HRSD was assessed at four time points (baseline and at 7, 12 and 18 months after randomisation) by trained assessors.

Secondary outcomes

Secondary outcomes at the same four time points included remission status and suicidal ideation and behaviour. We also examined clinically significant change post hoc using reliable change indices. 73

Economic measures

We collected resource-use data using a version of the Adult Service Use Schedule (AD-SUS), which is designed for depression populations. The AD-SUS collects service use and associated data and has been successful in a range of adult mental health population trials. 80 Completed in interview with participants, the questionnaire records use of hospital- and community-based health and personal social care services for all causes. For medications, we asked participants to report prescribed antidepressants, antipsychotics, sleeping tablets and painkillers. To avoid unblinding researchers, participants reported their use of all talking therapies on a separate self-report questionnaire. We collected information about time off work as a result of illness alongside the AD-SUS, using the productivity questions from the World Health Organization (WHO) Health and Work Performance Questionnaire. 81 We asked participants to complete the AD-SUS and WHO’s Health and Work Performance Questionnaire at baseline to cover the previous 6 months and at the 7-, 12- and 18-month interviews to cover the time since the previous interview, thus covering the full period from baseline to final follow-up. We extracted information on RO DBT-related service use, including the number of individual sessions and group sessions attended by each participant, from therapy records.

We estimated QALYs from EQ-5D-3L scores at baseline and at 7, 12 and 18 months. The EQ-5D-3L is a non-disease-specific measure for describing and valuing health-related quality of life. 50 The measure includes a rating of health in five domains (mobility, self-care, usual activities, pain/discomfort and anxiety/depression) and a rating of own health using a visual analogue scale (a thermometer scaled from 0 to 100). The health states described in the EQ-5D-3L were assigned a utility weight or score using responses from a representative sample of adults in the UK. 82 These weights were applied to the time between interviews and QALYs calculated using the area under the curve approach. 83 The EQ-5D-3L has been validated in economic evaluations for common mental health disorders. 84

Moderating variables

At baseline we collected the following data on potential moderating variables (see Table 1):

• Basic demographics, including age, sex and marital status.

• Balanced Inventory of Desirable Responding-Short Form85 – this is a 16-item questionnaire that is answered using a seven-point Likert scale ranging from 1 = totally disagree to 7 = totally agree; the scale assesses two dimensions: (1) self-deceptive enhancement and (2) impression management.

• Ego-Undercontrol and Ego-Resiliency Scales86 – the Ego-Undercontrol Scale has 37 questions and the Ego-Resiliency Scale has 14 questions; both questionnaires are answered using a four-point Likert scale ranging from 1 = disagree very strongly to 4 = agree very strongly.

• Invalidating Childhood Experiences Scale87 – this scale asks 14 questions about negative childhood experiences (≤ 18 years) in relation to each parent. Each item is rated on a Likert-type scale (1 = never; 5 = all the time). The levels of perceived invalidation by mothers and fathers are given by the mean score for the 14 items for each parent. Higher scores reflect a greater perception of invalidation by that parent. Four additional questions include descriptions of three types of invalidating environment (perfect, chaotic or typical) and one description of a validating environment. The items were rated on a five-point scale (‘not like my family’ to ‘like my family all of the time’). Higher mean scores indicate greater levels of either the validating environment or the three types of invalidating environment.

• Personal Need for Structure Scale88 – this 11-item scale comprises two subscales: desire for structure and response to lack of structure. Questions are answered using a six-point Likert scale ranging from 1 = strongly disagree to 6 = strongly agree.

• Urgency Premeditation Perseverance Sensation Seeking Scale89 – this is a 45-item questionnaire consisting of four subscales: (negative) urgency, (lack of) premeditation, (lack of) perseverance and sensation-seeking. Questions are answered using a four-point Likert scale ranging from 1 = agree strongly to 4 = disagree strongly.

• SCID-I and SCID-II64,90 – these are semistructured diagnostic interviews for identifying DSM-IV Axes I and II mental health and PDs, respectively.

Both participants and therapists completed the following:

• Cambridge Prospective Memory Test91– this is a laboratory measure of prospective memory that comprises a total of six prospective memory tasks, three cued by time and three cued by events. Participants were asked to work on some distractor tasks, such as word-finder puzzles or a general knowledge quiz, for a 20-minute period while they also had to remember to perform the prospective memory tasks. For each task, between 0 and 6 points are awarded depending on whether or not the tasks were completed and to what degree they required prompting from the assessor, with higher scores reflecting better prospective memory performance.

• Conscientiousness Subscale [of the NEO (neuroticism, extraversion and openness) Five-Factor Inventory]92 – this subscale comprises 12 items scored using a five-point Likert scale ranging from 1 = strongly disagree to 5 = strongly agree.

• The Frost Multidimensional Perfectionism Scale93 – 35 questions are answered using a five-point Likert scale ranging from 1 = strongly disagree to 5 = strongly agree. We computed separate scores for adaptive and maladaptive perfectionism. 94–96

• Schwartz Values Scale97 – 57 statements are rated on a nine-point Likert scale indicating the degree to which each statement is ‘a guiding principle’ in the participant’s life. The items reflect 10 universal value types: security, power, achievement, hedonism, stimulation, self-direction, universalism, benevolence, conformity and tradition.

In addition, therapists completed:

• Measure of Parenting Style Scale98 – this scale measures perceived parenting style across three domains: indifference, abuse and overcontrol. Fifteen items ask about the behaviour of the participant’s parents towards them during the first 16 years of their life for each parent separately. The items are rated from 0 = not true at all to 3 = extremely true.

Mediating variables

Data on potential mediators were collected at 0, 3, 7, 12 and 18 months using the following questionnaires (see Table 1):

• Acceptance and Action Questionnaire-II (AAQ-II)99 – this questionnaire is a 7-item scale measuring psychological inflexibility. Questions are answered using a seven-point Likert scale ranging from 1 = never true to 7 = always true.

• Ambivalence over Emotional expression Questionnaire100 – this is a 28-item questionnaire measuring conflict over emotional expression. It is answered using a five-point Likert scale ranging from 1 = never to 5 = very often.

• The Dialectical Behaviour Therapy Ways of Coping Checklist (DBT-WCCL)101 – this checklist is based on the earlier Revised Ways of Coping Checklist and is an inventory of emotional coping skills taught in standard DBT. It comprises 59 items rated on a three-point Likert scale ranging from 0 = never used to 3 = regularly used. The scale has three subscales: DBT skills use, general dysfunctional coping and blaming others.

• Inventory of Interpersonal Problems–Personality Disorders102 – this 25-item dimensional PD measure is answered using a four-point Likert scale ranging from 0 = not at all to 4 = extremely. Five subscales can be calculated: interpersonal sensitivity, interpersonal ambivalence, aggression, need for social approval and lack of sociability.

• The 3-item Social Support Questionnaire (SSQ)103 – this questionnaire asks participants to list the initials of people who provide support to them in three different manners and participants are asked to rate their degree of satisfaction for each manner of support received using a Likert scale ranging from 1 = very dissatisfied to 6 = very satisfied.

• White Bear Suppression Inventory104 – this 15-item questionnaire measures suppression and avoidance of unwanted thoughts using a five-point Likert scale ranging from 1 = strongly disagree to 5 = strongly agree.

Temporal sequencing measures

Participants were asked to complete the following measures at baseline, then once a month for the first 12 months and again at 18 months:

• The Patient Health Questionnaire-9105 – this is a 9-item psychometrically valid assessment of depressive symptoms using a three-point Likert scale ranging from 0 = not at all to 3 = nearly every day.

• The Positive and Negative Affect Scale (PANAS)106 – the 20-item PANAS provides independent estimates of positive and negative affect and is rated from 1 = very slightly/not at all to 5 = extremely.

• The Emotional Approach Coping scale (EAC)107 – this scale comprises eight items and has two subscales: emotional processing and emotional expression. Items are rated from 1 = I usually don’t do this at all to 4 = I usually do this a lot.

• The Credibility/Expectancy Questionnaire108 – this questionnaire indexes positive expectancies for treatment and comprises six items with various scales. The first four items aim to measure credibility and the last two items aim to measure expectancy.

In addition to the listed questionnaires, we also used an automated telephone system to measure mood using the 10-item PANAS Short-Form109 and coping skills using the six items from the DBT-WCCL101 once a week over the first 6 months that participants were in the trial (ecological momentary assessments).

Therapeutic alliance

To measure therapeutic alliance, patients allocated to RO DBT were asked to complete the patient version of the California Psychotherapy Alliance Scales110 during the first 4 weeks of treatment and then once a month until the end of treatment. Therapists were asked to complete the therapist version at the same time points.

Therapist characteristics

To investigate whether or not therapist characteristics, and their relationship with patient characteristics, influence outcome, we asked them to complete measures of conscientiousness (the conscientiousness scale from the NEO Five-Factor Inventory),92 perfectionism (Frost Multidimensional Perfectionism Scale),93 personal values (Schwartz Values Scale)97 and attachment experiences (Measure of Parenting Style Scale). 98 More detailed descriptions of the scales are available in Moderating variables.

We also measured therapists’ capacity to generate strong alliances before the trial by asking several patients from the caseload of each therapist to complete the California Therapeutic Alliance Scale. 110 This measure was used as an IV for strength of therapeutic alliance.

Client Satisfaction Questionnaire

To assess the acceptability of the treatment and the study as a whole, we asked the first 50 participants to complete the Client Satisfaction Questionnaire-8. 111

Data processing and reduction

We stored study data in several databases, which were subsequently linked for analysis using the five-digit participant code (in which the first digit indicates study site) provided with each randomisation request. In parallel to the randomisation database, the main study database stored questionnaire responses and notes, and further databases stored additional information on therapists and therapist sessions.

Data validation included confirmation of the consistency of information in various databases, and thorough checks on the fidelity of double-entry data, with inconsistencies checked against original paper forms. Following such checks and the resolution of all identified inconsistencies, documented audits of questionnaire responses across a range of questionnaires and time points were undertaken, in each case using a random selection of participants representing all study sites and both study arms.

Study data were exported from the main study database and subsequently processed by a series of scripts (primarily written in Stata^®, version 14; StataCorp LP, College Station, TX, USA) that renamed variables, created labels for variables and variable values, created codes for reasons for missing data, identified canonical responses for double-entry data, resolved time points to specific months, defined pairings of participant and therapist and scored questionnaires, using agreed rules on the treatment of missing data. Each script generated an interim file for any further checking required. The final stage dropped all intermediate variables and individual responses before merging the reduced data set with randomisation and therapist information to create a final data set for use in the analyses.

Primary and secondary outcome analyses: continuous variables

The primary outcome, HRSD score, and all continuous secondary outcomes were analysed by treatment allocated, modelling differences between groups at baseline and then at 7, 12 and 18 months after randomisation. Covariates in the primary analyses included treatment site (Dorset, Hampshire, North Wales), baseline HRSD score, baseline PD status (presence or absence) and early-onset depression (yes or no). A three-level mixed-effects model was used to account for clustering of data by patient and therapist. Notably, this model does not assume that all therapists are equally effective. 112 These mixed models are efficient and unbiased in the case that data are missing at random, that is, missing at random conditional on the covariates included in the model. As suitable auxiliary data were not available, missing scale responses were not imputed (e.g. via multiple imputation). However, missing questionnaire items were imputed for rows when fewer than 10% of items were missing in a given scale. This item imputation was performed by linear regression using the other scale items as covariates.

For each outcome, the main effects of time and treatment allocation and the interaction of time and treatment were tested for. Planned contrasts were then made for each outcome at 12 months. Secondary analyses computed differences between treatment groups at months 7 and 18. Details of the analyses are available in Box 1.

BOX 1 - Details of analyses

Both primary and secondary outcomes were modelled using mixed-effects models using the Lmer package in R (version 3.3.3; The R Foundation for Statistical Computing, Vienna, Austria). 113 Models for primary and secondary outcomes were specified in the form:

In these models [dv].baseline indicates baseline score for the dependent variable, and pd.baseline denotes a binary predictor for the presence of any PD at baseline. The month, patient and therapist variables are factors. Type III tests of fixed effects were computed using Satterthwaite correction114 for degrees of freedom, and between-group contrasts at each time point were computed using the contrast function within the emmeans package. As the sample of therapists was small, null-hypothesis tests on the random effects in the models were not performed, but the proportion of variance attributable to therapists (see Chapter 3, Variations in therapist performance) was reported.

Outcome analyses: binary or categorical outcomes

Economic evaluation

Process variables

The exposure, alliance and skills mediators will all be treated as ‘process variables’ in the model. 130 A process variable takes a fixed value (usually zero) for everyone receiving TAU. As such, in the model, there is no mediation pathway for TAU patients, only a direct effect of TAU on HRSD scores.

This is despite the fact that a measure, such as the DBT-WCCL score, is not truly a process variable because it is measured for patients in both the RO DBT and TAU groups. However, the skills that are relevant for TAU are different from those for RO DBT and, more generally, the other hypothesised pathways through which TAU affects HRSD scores are different from those for RO DBT. Hence, in line with the primary focus of this research on estimating the hypothesised pathways through which RO DBT works, no attempt to decompose the direct effect of TAU is made.

Direct effects of allocation

One important assumption of ‘by treatment allocated’ analyses is that of zero treatment value for participants allocated to a control condition. Specifically, inferences made about treatments in RCTs assume that the outcomes of participants who are allocated to a control condition do not differ as a function of their allocation, that is, these participants do no worse than if they had not been recruited to any trial and had been followed up covertly. Nonetheless, both research participation alone and reactions to the experimental procedures (e.g. to allocation itself, rather than the intervention under study) may bias treatment estimates. 143 Even when a trial offers bona fide alternative treatments, participants often express strong preferences for arms within RCTs and these preferences can, in and of themselves, influence trial outcomes. 144 When trials, such as the RefraMED trial, use TAU as a control then it is possible – perhaps likely – that participants will be disappointed if they are not allocated to the active intervention.

Cook and Campell145 describe both ‘resentful demoralisation’ and ‘compensatory rivalry’ as potential sources of post-randomisation bias in which the allocation to treatment group cannot be fully concealed. Resentful demoralisation is the deleterious effect that randomisation to a control condition may have on disappointed participants, for example if they give up other attempts to change behaviour on hearing of their allocation. Compensatory rivalry denotes the reverse effect, for example if participants redouble efforts to find alternative sources of help if they are not allocated to their desired treatment. However, despite both the likelihood that participants may actively respond to treatment allocation and the concomitant potential for bias in unblinded intervention studies, few trials have engaged with these potential problems. 143

In the RefraMED trial, it was explicitly sought to measure changes in patients’ attitudes caused by allocation to treatment. This direct effect was operationalised as the change between participants’ hypothetical expectancies before allocation (e.g. ‘what will happen if you are assigned to RO DBT?’) and actual expectancies after allocation (e.g. ‘what will happen now you have been assigned to RO DBT [TAU]?’), when this change is different for participants assigned to RO DBT and TAU. These analyses are exploratory but are important because they seek to check the validity of the exclusion restriction for treatment allocation, upon which intention-to-treat and causal analyses depend.

Recruitment conversion rates

Of the 2678 patients referred to the study, 9% (n = 250) were eventually randomised. Conversion rates for referral sources into randomisations were 3% for primary care (55/1703), 20% for secondary care (170/849), 19% for self-referrals (19/101) and 24% for other sources (6/25).

Study withdrawals, treatment withdrawals and losses to follow-up

Table 20 in Appendix 1 presents a breakdown of reasons for study withdrawals (participants who indicated that they no longer wanted to provide outcome data) or losses to follow-up (participants who were uncontactable despite repeated attempts) per group for each assessment time point, as well as the number of participants allocated to RO DBT who withdrew from treatment (defined as having attended fewer than three sessions).

A total of 56 participants (22%) withdrew or were otherwise lost to follow-up during the study: 34 in the RO DBT group (21%) and 22 in the TAU group (25%). Unfortunately, only one of the participants withdrawing from treatment agreed to provide outcome data at follow-up interview and, thus, stayed in the study. Hence, there was no significant difference between treatment groups in the proportion of participants who did not contribute to analysis (χ² = 0.712, degrees of freedom = 1; p = 0.399).

Of those participants allocated to RO DBT who withdrew from the study (n = 34), 10 participants did not attend any sessions and four withdrew from treatment after having attended fewer than three sessions and, thus, were considered treatment withdrawals.

The most common reasons for withdrawal for participants allocated to TAU was their dissatisfaction with having been allocated to TAU (n = 9), whereas for those allocated to RO DBT the most common reason for withdrawal was their being unable to commit to the study because of life, work or family commitments (n = 10).

Demographics

Table 5 presents baseline demographic data for the RO DBT and TAU groups. Of the entire sample, 164 participants were female (64%) and 193 participants (77%) were aged between 35 and 65 years. The vast majority of participants (n = 222; 93%) were of white British descent and 124 (50%) reported being in a relationship. With regard to highest education level, 113 participants (46%) had higher education experience. As can be seen from Table 5, demographics were comparable across groups.

Table 21 in Appendix 1 shows the mean ages of the participants per group and per site; as can be seen, the age distributions per site per group did not differ greatly. The average age in the TAU group was 48 years (range 20–69 years) and in the RO DBT group was 47 years (range 21–72 years).

Furler et al. 147 note that demographic information reported by clinical trials typically fails to include data on participants’ socioeconomic status but that this can be important for the interpretation and generalisation of trial results. We used participants’ postcodes to look up values from publicly available geographically based indices of deprivation in England and Wales. For England, we used the English Indices of Deprivation for 2014148 and for Wales, the revised Welsh Index for Multiple Deprivation for 2014. 149 Data were available for 224 out of the 250 participants. Figure 8 in Appendix 4 presents these data split by treatment group; we found no evidence that the treatment groups differed on these indices. Decile scores are not directly comparable between England and Wales. 150 However, overall, the Welsh sample was close to the Welsh national average on the Welsh Index of Multiple Deprivation [mean decile 5.5, standard deviation (SD) 2.5]. The two English sites were slightly more deprived relative to other areas [the mean decile for the English Indices of Deprivation for Dorset was 5.8 (SD 2.4) and for Hampshire was 7.4 (SD 2.5)].

Clinical backgrounds

Table 6 presents clinical background data for the RO DBT and TAU groups. More than one-third of the participants (38%) reported that they had experienced their first depressive episode before the age of 16 years. Eighty-five per cent of participants were classed as having chronic rather than recurrent depression at the time of interview, and of those that could recall the number of previous episodes that they had experienced, one-third (34%) reported no previous episodes, one-third (35%) reported six or more and the remainder of participants reported between one and five previous episodes. The vast majority of participants (83%) had previously received psychotherapy.

High comorbidity rates were present in the sample: only 3.6% of participants did not report any comorbid disorders; the remaining 96.4% were experiencing an Axis-I, Axis-II or a combination of both types of mental health disorder. Eighty-six per cent reported at least one comorbid Axis-I disorder and 78% reported at least one comorbid Axis-II disorder. The two most common Axis-I disorders were social phobia (80%) and specific phobia (63%), although other anxiety disorders were also very common in this sample. The two most common PDs were avoidant PD (54%) and obsessive–compulsive PD (47%).

As can be seen in Table 6, two participants were assessed as having bipolar I disorder. Although established bipolar disorder was an exclusion criterion, these participant’s notes indicated that both were experiencing a moderately severe depressive episode at the time of study inclusion (without rapid cycling and without seasonal pattern). They were, thus, deemed eligible, recruited and included in the analyses.

Outcome of adaptive randomisation procedures

The adaptive randomisation procedure appears to have been effective in balancing the clinical backgrounds of the participants across groups (see Table 6). Treatment sites were also comparable in the clinical characteristics of participants (see Table 22 in Appendix 1).

Patients’ treatment attendance

Number of participants analysed

All analyses were ‘by treatment allocated’. Figure 9 in Appendix 4 presents the proportion of data available for analyses at baseline and each of the main follow-up points. For the primary outcome, the HRSD score, data were available for all participants at baseline, for 73% of participants at month 7 and for 76% of patients at month 12. Fewer data points (67%) were available at month 18 by design, because data collection ended when all participants had completed the 12-month follow-up, but not all had completed the 18-month follow-up.

Primary outcome: depression symptoms measured by the Hamilton Rating Scale for Depression

We did not find a significant interaction between treatment allocation and time. Planned between-group contrasts revealed a statistically significant difference between RO DBT and TAU at month 7, but not at months 12 or 18 (Table 8). Standardised differences between groups were large at month 7 (d = 1.03). At 12 months, the effect size matched that used in the power calculations (d = 0.41) but was slightly reduced at month 18 (d = 0.32). Model-adjusted predictions for each treatment group are presented in Figure 3. These predictions suggest that treatment gains for RO DBT participants were maintained between month 7 and month 18, but that TAU participants also experienced reductions in symptoms between months 7 and 12. The descriptive statistics of the HRSD scores are presented in Table 9 and the model-adjusted means in Table 10.

FIGURE 3.

Model-adjusted predictions for each treatment group based on HRSD score. (a) Adjusted predictions; (b) standardised mean differences between groups; and (c) change from baseline. SMD, standardised mean difference.

Other continuous outcomes

We also ran mixed-effect models and computed contrasts at months 7, 12 and 18 for psychological inflexibility (as measured using the AAQ-II), emotional approach coping (as measured using the EAC), impaired functioning (as measured using the LIFE-RIFT), suicide ideation (as measured using the MSSI) and social support (as measured using the SSQ). Table 8 presents the planned contrasts between TAU and RO DBT at 7, 12 and 18 months for all of the outcomes.

Table 9 presents the descriptive statistics for all variables, whereas Table 10 presents the adjusted means for these models. We found significant advantages for the RO DBT patients on EAC and AAQ-II scores at months 7, 12 and 18. AAQ-II scores for RO DBT patients again decreased consistently, with medium to large differences between TAU and RO DBT groups (see Figure 12 in Appendix 4). For EAC, the RO DBT group mean score increased consistently and the standardised difference grew from small to moderate/large by 18 months (see Figure 13 in Appendix 4).

No significant advantage was found for RO DBT on LIFE-RIFT, MSSI or SSQ scores at any follow-up point. Both treatment groups exhibited substantial reductions in LIFE-RIFT (impaired functioning) scores from baseline (see Figure 14 in Appendix 4). In both treatment groups, the mean MSSI scores at baseline were below eight, indicating low levels of suicidal ideation, and this remained low throughout the trial (see Figure 15 in Appendix 4). Perceived social support increased over time in the RO DBT group but not the TAU group, although this difference did not reach significance (see Figure 16 in Appendix 4).

Remission rates and reliable change

Remission rates calculated by the primary criteria, and by other criteria reported in the literature, are given in Table 11 and presented in Figure 17 in Appendix 4. There were substantial differences in absolute rates of remission depending on the criteria adopted, and also whether or not we calculated remission based on model-adjusted predictions or raw scores. This is likely because the mixed models tend to ‘shrink’ extreme values towards the cluster and grand mean. 151 Depending on the criterion used (from the many described in the literature), the effect of this shrinkage is more or less marked. For the purposes of comparison between groups, it was believed that the model-estimated remission rates are preferable, because they adjust for differences in severity at baseline in the same way as the model-based inferences reported as the primary outcome.

This variation in remission rates as a result of choice of methodology and criterion highlights the substantial disadvantage of dichotomising or trichotomising continuous outcome data; although remission provides a simple and tangible metric of the performance of a treatment, this may come at the cost of oversimplification. Nonetheless, to aid comparison with other trials, remission rates were reported based on a variety of criteria in Table 11 and Figure 17 in Appendix 4. Using the primary criterion for remission, which was based on both the HRSD and LIFE-RIFT scores, more RO DBT participants achieved full or at least partial remission at months 7, 12 and 18. At the primary follow-up (month 12), and using model-adjusted data, this equated to around eight additional RO DBT participants achieving full remission for every 100 treated, and so had a number needed to treat to achieve an additional fully remitted case of 12.5. For partial remission, an additional 14 participants benefited from RO DBT, with a number needed to treat of 7.1. Using the secondary criterion of ‘worthwhile’ change from the participant perspective, which equates to a 17.5% reduction in symptoms from baseline,76 it was found that an additional 21 participants per 100 benefited from RO DBT.

Clinical predictions for new patients

From the perspective of clinicians and patients, it is relevant to consider the likely range of outcomes that an individual patient might experience, given different treatment choices. For example, patients might wish to know what benefit they are likely to obtain in choosing a new treatment, compared with continuing in TAU. By simulation from the primary outcome model, fit by Hamiltonian Monte Carlo, predictions were made for new patients who would be treated with either TAU or RO DBT. These predictions show good convergence, with group means from models fit via maximum likelihood (see Figure 3), especially at months 7 and 12. At month 18, the predictions from this Bayesian model are more pessimistic than the equivalent model fit via maximum likelihood, with median predictions at 18 months of 18.12 and 19.17 points for RO DBT and TAU, respectively, which compares with 13.96 (RO DBT) and 16.09 points (TAU) for the equivalent maximum-likelihood model. These discrepancies are believed to be primarily caused by the larger number of missing data points at month 18 (part of the study design) and the different assumptions embodied by the priors in the Bayesian model, which influence how data are pooled across time periods. These predictions are also presented in Table 26 in Appendix 1.

Based on these predictions, we computed the difference between outcomes if patients had been treated with RO DBT rather than TAU and, thus, the expected additional benefit of being treated with RO DBT for a new patient. Inspecting the distribution of this predicted additional benefit allows patients to form an expectation for what their outcome might be should they accept RO DBT treatment over TAU. Figure 4 presents expected changes in symptoms for the 5^th through to the 95th centiles of patients graphically, and Table 27 in Appendix 1 presents the same figures in a table. The 5^th centile represents the additional benefit of RO DBT in the ‘best-case scenario’ – only 5% of patients would experience a benefit this great. Conversely, the 95th centile represents the expected change in symptoms for the least-fortunate 5% of patients. The 50th centile represents the expected change for the average patient. Both Figure 4 and Table 27 in Appendix 1 demonstrate how variable prognosis is from the patients’ perspective. When choosing RO DBT rather than TAU, the models show that patients should expect to make additional improvements in symptoms, but the intervals on this prediction are wide.

FIGURE 4.

The expected benefits for new patients accepting RO DBT rather than TAU at 7, 12 and 18 months.

Table 12 and Figure 18 in Appendix 4 represent these predicted values as the percentage of participants who achieve ‘worthwhile’ change, using Button’s criteria76 of a 17.5% change from baseline; in the sample, the mean HRSD value at baseline was 23 points, and so this required, on average, a 4-point change on the HRSD. From this we can say that 50 out of 100 patients would experience a worthwhile benefit from accepting RO DBT, rather than TAU, at the end of treatment. At month 12, despite the improvements in the TAU group over this period, 32 out of 100 patients would still obtain a worthwhile reduction in symptoms from accepting RO DBT. Note that in Table 12 (and see Figure 4) the ‘worsening’ of symptoms does not refer to an iatrogenic effect of RO DBT but rather reflects the fact that predictions for new patients must take into account the natural variation in depressive symptoms; failure to account for this natural variation in symptoms would lead to predictions for prognosis that were unjustifiably confident. 152

Resource use

Resource use over the follow-up period is summarised in Table 13. Participants who enrolled in the RO DBT group attended an average of 22.8 individual RO DBT sessions (median 26 sessions) and 19.3 group RO DBT sessions (median 22.5 sessions). In addition, the participants attended an average of 3.4 sessions of other types of talking therapy. Participants in the TAU group attended an average of 9.1 sessions of various talking therapies. The use of all other health and social care services, including medications, tended to be lower in the RO DBT group or broadly similar between the two groups. Days reported off work were also lower in the RO DBT group than in the TAU group.

Costs

Table 14 summarises health and social care costs from the NHS and personal social services perspective over the 6 months prior to trial entry and over the 12- and 18-month follow-up periods. Disaggregated costs are based on complete-case data, as data imputation was conducted at the aggregate level. The costs of all health and social care services used over the 6 months prior to trial entry were lower in the RO DBT group than in the TAU, group, but the difference was not significant (mean cost difference –£1029, bootstrapped 95% CI –£2465 to £407; p = 0.160). Excluding the cost of RO DBT, the cost of all health and social care services used was significantly lower over the 12-month follow-up period (adjusted mean cost difference –£909, bootstrapped 95% CI –£1799 to –£19; p = 0.045) and was also lower at the 18-month follow-up period, although not significantly so (adjusted mean cost difference of –£901, bootstrapped 95% CI –£2755 to £952; p = 0.340). The RO DBT intervention cost per person was approximately £5000, including the cost of both individual and group sessions, resulting in total costs that were significantly higher in the RO DBT group than TAU over both the 12-month follow-up period (adjusted mean cost difference £4566, bootstrapped 95% CI £3691 to £5440; p < 0.001) and the 18-month follow-up period (adjusted mean cost difference £4463, bootstrapped 95% CI £2915 to £6011; p < 0.001).

Outcomes

The mean EQ-5D-3L scores and QALYs are presented in Table 15, together with unadjusted differences and differences after imputation for missing data. The EQ-5D-3L scores improved in both groups over the 18 months’ follow-up. Over the 7- and 12-month follow-up periods, the EQ-5D-3L scores and QALYs were slightly higher in the RO DBT group than in the TAU group, but at 18 months’ follow-up the scores were slightly higher in the TAU group. Differences were small and non-significant at all follow-up points.

Cost-effectiveness

The ICERs, based on the bootstrapped adjusted differences in imputed costs and QALYs with missing data imputed, are as follows:

• at 7 months, the ICER was £707,600 (difference in costs, £7076; difference in QALYs, 0.010)

• at 12 months, the ICER was £220,250 (difference in costs, £7048; difference in QALYs, 0.032)

• at 18 months, the ICER was £320,000 (difference in costs, £7360; difference in QALYs, 0.023).

The base-case 12-month additional cost of £7048 was associated with a difference of 0.032 QALYs, yielding an ICER of £220,250 per QALY; this ICER is well above the NICE upper willingness-to-pay threshold of £30,000 per QALY. ICERs were well above the maximum willingness-to-pay threshold at all time points.

The cost-effectiveness plane for RO DBT versus TAU from the perspective of NHS and personal social services at the primary 12-month end point is shown in Figure 5. This illustrates the scatterplot of 30,000 bootstrapped cost and effectiveness pairs for RO DBT versus TAU. All scatter points lie above the x-axis, illustrating that total health and social care costs are higher for the RO DBT group than the TAU group in all cases. The majority of scatter points fall in the north-eastern quadrant in which RO DBT is more effective than TAU but also more costly. Uncertainty around the ICER was explored in a cost-effectiveness acceptability curve, which is shown in Figure 6. This graph illustrates that RO DBT in its current format has zero probability of being cost-effective compared with TAU at the NICE willingness-to-pay threshold of £30,000.

FIGURE 5.

A scatterplot of differences in costs vs. differences in QALYs for RO DBT vs. TAU at 12 months’ follow-up, taking the NHS/personal social services perspective.

FIGURE 6.

The cost-effectiveness acceptability curve for QALYs showing the probability that RO DBT is cost-effective compared with TAU at 12 months’ follow-up, taking the NHS/personal social services perspective.

Sensitivity analyses

The primary economic analysis (also known as the ‘base case’) was based on a cost–utility analysis conducted at the 12-month follow-up point, from the perspective of NHS and personal social services with imputation of missing data. Table 16 summarises how variation in methods and assumptions affected the ICERs. In sensitivity analyses 1–4 (complete-case analysis, inclusion of productivity losses, reduction in the cost of RO DBT group sessions and analysis at the 18-month follow-up point), with all other factors being equal to the base case, RO DBT remained cost-ineffective compared with TAU, with costs per QALY all well above the NICE willingness-to-pay threshold of £30,000 per QALY. At 18 months’ follow-up (scenario 4), the TAU group achieved more QALYs at a lower cost, thereby dominating RO DBT.

In the final scenario, a cost-effectiveness analysis was undertaken using the primary clinical outcome as the measure of effect (the HRSD score), with all other factors being equal to the base-case analysis (the 12-month follow-up point), from the perspective of the NHS and personal social services, with imputation of missing data. This analysis yielded an ICER of £7048 per unit improvement on the HRSD. Figure 19 in Appendix 4 illustrates the scatterplot of 30,000 bootstrapped cost and effectiveness pairs for RO DBT versus TAU based on differences in HRSD score using the £1000 per HRSD unit threshold proposed by Romeo et al. 153 The results were very similar to the cost–utility analysis, with all replications falling above the threshold of willingness to pay and the cost-effectiveness acceptability curve indicating a zero probability that RO DBT was cost-effective compared with TAU (Figure 20, Appendix 4).

Single-mediator model

The results obtained using the single-mediator model for each mediator variable in turn are shown in Table 17. Results were unadjusted for baseline predictors and adjusted for the same baseline predictors (as used elsewhere): the HRSD score prior to randomisation, site, age at depression onset and PD status.

Focusing on those results obtained without using the IVs (i.e. assuming there is no unobserved confounding), it can be seen that the indirect effects mediated by skills (DBT-WCCL) and alliance (California Psychotherapy Alliance Scales) are significant in the sense of having 95% CIs that exclude zero. However, the width of these CIs is wide, and it is clear that the model struggled to accurately decompose the total effect using the information available from this small data set.

Moving to the results obtained using the IVs (i.e. assuming that there is unobserved confounding), it can be seen that this situation deteriorates further. The CIs of the various effects have increased in size compared with those obtained without using the IVs. Although all the decomposed effects, bar those for exposure, are significantly different from zero, the estimates are too large to be plausible and the precision is poor. This could be because of collinearity between the predicted value of exposure and the direct effect.

The reason for this is not because the average direct and indirect effects are non-identified, but because these effects are poorly estimated from the sample data. Running the weak-IV diagnostics using the ivregress post-estimation command, the three IVs for the second part of the mediator in Equation 2 give Cragg–Donald statistics no larger than one. The critical values provided by the command tell us that we can expect the bias of the estimator in both cases to be considerably greater than 30% (to expect a bias of < 10%, it would have needed a Cragg–Donald statistic of at least 9.08), and the actual size of the significance test (nominally at the 5% level) is actually much greater than 25% (the critical value for a size < 10% is 22.30), which indicates a high chance of obtaining a false positive.

All-mediator model

The results (unadjusted and then adjusted) for the joint model are shown in Table 18. Note that the total mediated effect through exposure does not equal the indirect effect through exposure in Table 17. This is because the sample size for the all-mediator model is smaller than that for the single-mediator model for exposure.

If the IVs are not used and the assumption of no unobserved confounding is reasonable, these results suggest that the pathway from RO DBT through exposure to skills has a protective effect on patients’ mental health. However, the results obtained using IVs are again implausible and inaccurate. This demonstrates, again, that the IVs are too weak for us to be able to decompose the pathway effects as had been hoped.

Radically open dialectical behaviour therapy was considered helpful

The vast majority of patients who attended RO DBT found it helpful, with only 5 out of 81 RO DBT participants stating that they did not think it was helpful or that it made their depression worse:

I find it hard to imagine what life would have been like without this intervention. The group sessions and the skills file have been so useful. I refer to the work every day. I have enjoyed embracing the skills/learning. The CD [compact disc]/loving kindness and other structures have helped shape my problem solving. Through RO DBT I have learnt to cope with myself without harsh judgement. I think I now like myself more and more.

Patient 64

My experience of it has exceeded my expectations. It has allowed me to eradicate a way of thinking and behaving that was detrimental to my mental health. Challenging at times but as it progressed it all made sense and became intensely rewarding.

Patient 81

I don’t feel it has done much; it has just made me more aware of what makes my life stressful. A lot of time and effort went into this for no reward or change.

Patient 120

Suggestion 1: increase treatment duration or provide follow-up sessions

The most common suggestions for improvement were to increase the number of treatment sessions, allow participants to go through the skills training twice or offer several follow-up sessions at the end of treatment:

I received therapy for 6 months and felt that was the ideal length for me. For some people this may be too short and would benefit from a longer time. It would be ideal to have gone back after 4–6 weeks to see a therapist to see how you are getting on. Be given something (support material) at the end when you left.

Patient 7

Very positive and helpful but would have been better if it was longer as many things came to the surface near the end of the therapy which I didn’t have a chance to deal with.

Patient 122

Therapy could be improved by giving patients who have completed a check-up session every 3 to 4 months with a therapist for a period of up to 1 year. When the final session took place it felt quite final. A follow-up would certainly be of some reassurance.

Patient 117

Suggestion 2: increase skills group duration or change format slightly

Although two people found that the duration of the skills groups was too long, at least nine people felt that the sessions were too short and another five felt that less time should be spent on going through individual homework and instead focus on the new skills to be learned:

I felt as though some of the group sessions were rushed. There is a lot to take in and it sometimes feels that some parts of the course needed a little more time to study and digest.

Patient 26

Less time could be spent going over the homework each week in the group therapy so there is more time to spend on learning and understanding the skills. Our skills were often taught in a rushed way because we had run out of time in the group session.

Patient 27

Suggestion 3: simplify language and homework

Although the vast majority reported that the homework was very beneficial to them, 7 out of 81 RO DBT participants said that they found the homework difficult or that the terms used were hard to understand:

I think that the jargon and mental health vocabulary could be improved. The pressure to name things in a certain way I feel is not necessary to get the best out of the therapy.

Patient 99

I found the homework worrying i.e. had to make sure I did it. I did not always understand what was being put across which worried me. The ‘terms’ were very off putting. I felt I had to keep asking for explanations. I probably inhibited some helpful features of the experience.

Patient 2

Suggestion 4: develop application or other supporting materials

At least seven people suggested that additional supporting materials, such as mobile phone applications (apps), a forum or CDs, should be offered to assist with learning skills:

The creation of a phone app so I have to hand, on the go, a reminder of the skills.

Patient 16

An app or apps so that skills could be used as reminders/aid on mobiles. Or PDFs [portable document format files] for some purpose.

Patient 86

Simplified version. CD to listen to as sometimes sitting down to read something is hard.

Patient 111

Open group structure

Finally, four people commented on the open group structure, saying that they would have preferred it to be a closed group with participants all starting at the same time:

Joining the group at different times was difficult. I think all should attend + start at the same time.

Patient 14

The only other thing I found hard was people starting therapy at different times. It’s nerve racking when everyone seems to know about skills not covered already and at the other end it was really difficult to watch friends leave.

Patient 40

Depressive symptoms

The finding that depressive symptoms at the end of treatment were significantly fewer after RO DBT than after TAU reinforces findings from another trial of RO DBT-E, which reported effect sizes with a d of 0.49 and 0. 34 after the end of group sessions and individual treatments, respectively. 49 That a very large effect was found at the end of treatment in this trial is especially important, as the TAU participants were free to access any available treatment, including psychotherapy. Furthermore, the effect size is much larger than those reported by previous trials of psychotherapy for refractory depression, which were either small (CBASP)9 or medium sized (CBT, the Re-ChORD programme). 26,28

More than half of the TAU participants received some form of psychotherapy and were not solely reliant on ADM. Table 13 shows that between 7 and 12 months, 44% of TAU participants accessed ‘talking therapy’ at least once, in comparison with < 5% of RO DBT participants. In contrast, control groups received only ADM in most other studies, including both RO DBT-E pilot studies48,49 and several CBASP studies. 21–23 A recent meta-analysis suggested that TAU in the UK ‘may be considered to be of a high standard encompassing pharmacological and psychological treatment’ (p. 158). 154 The subgroup analyses of Watts et al. 154 comparing the effects of CBT with different types of TAU (collaboration between primary and secondary care, multiple service providers, psychotherapy alone, minimal contact and others that were not well described) revealed that the effect size of CBT relative to TAU depended on the nature of that TAU. The smallest differences were for comparisons between CBT and primary–secondary care collaboration and the largest were for comparison with ‘minimal contact’. Surprisingly, there was a larger difference between CBT and TAU including psychotherapy than between CBT and TAU managed in primary care. One possible explanation is that primary care-managed TAU often includes combined pharmacological and psychological therapy, which is close to current best practice in the UK. 154

Personality matters

Radically open dialectical behaviour therapy is the first treatment specifically to target maladaptive personality traits in the treatment of refractory depression related to overcontrol. This study suggests that, when psychopathology is long-standing and does not respond to efficacious first-line treatments, overlearned patterns of coping – either overcontrolled or undercontrolled – are interfering with attempts to change. Importantly, RO DBT does not seek to rid clients of their overcontrolled personality style, instead it focuses on changing maladaptive coping linked to overcontrol, such as rigid rule-governed behaviour, inhibited emotional expression and aloof or distant relationships, which are hypothesised to exacerbate emotional loneliness and depression.

Surprisingly, this study does not suggest that the effect of RO DBT depends on the presence of comorbid PD. As 78% of the sample showed at least one overcontrolled PD, it is possible that this reflects the homogeneity of the sample. A meta-analysis of 34 studies concluded that the occurrence of a PD in people with depression is about twice as likely to be associated with a poor response than no PD. 155 This review also found that this result did not differ across treatment types such as electroconvulsive therapy, drugs, psychotherapy or drugs and psychotherapy. A later study by Fournier et al. 156 compared the response rates of depressed patients with and without PD who were randomised to either ADM or cognitive therapy. People with PD responded better to ADM (44%) than to cognitive therapy (66%), whereas those without PD responded better to cognitive therapy (70%) than ADM (49%). Finally, a recent large study with 1249 participants within the UK IAPT initiative showed that higher scores on a screening measure for PD were associated with poorer outcomes on measures of depression, anxiety and social functioning and reduced recovery rates at the end of treatment. 157

Remission rates

As a secondary outcome, we compared remission rates between groups at each time point. Using the primary criteria for full or partial remission, which combined scores on both the HRSD and the LIFE-RIFT, low rates of full remission were found in both groups: at 12 months, 8% of RO DBT participants and no TAU participants achieved full remission. The rates of partial remission were higher, that is, 36% for RO DBT participants and 22% for TAU participants at month 12. For both these criteria and several others, there were higher rates of remission for participants treated by RO DBT than by TAU.

Compared with the two earlier trials of RO DBT-E for refractory depression, full remission rates are much lower in both RO DBT and TAU groups. In Lynch et al. ’s first study,49 71% of RO DBT-E recipients met criteria for remission compared with 47% of controls, and after 6 months the corresponding percentages were 75% and 31%. However, this earlier study may have included potentially less severe patients; for example, the mean age at first depression in the RefraMED trial was much lower, at 23 years versus 36 years. Lynch et al. ’s second trial48 recruited patients with PD and comorbid depression and reported remission rates of 71% for RO DBT-E and 50% for control participants. However, the authors defined remission as a HRSD score of ≤ 10 points, whereas the present study used a more conservative method and defined full remission as a HRSD score of ≤ 7 points plus a LIFE-RIFT score of ≤ 12 points. Lynch, the treatment developer, conducted both pilot trials at Duke University, a prestigious academic research centre, led treatment teams and personally provided treatment for study patients. In the present study, in contrast, treatment occurred in non-academic NHS clinics and Lynch did not participate in the delivery of treatment or directly lead clinical teams.

Other psychotherapy trials for refractory depression report a wide range of remission rates, varying between 9.4% and 57%. However, remission rates are not helpful in comparing studies because there is wide variation in the definitions of full and partial remission, the methods used to assess that participants remitted, the duration of follow-up and the study samples. For example, the rates of comorbid Axis-I and Axis-II disorders and the age at depression onset, both of which are factors that can influence treatment response, vary by study. Furthermore, because remission rates dichotomise continuous scores, these factors may also interact in an unexpected fashion. Therefore, to aid interpretation of the findings, we simulated the likely outcomes for new patients choosing between TAU and RO DBT: this suggests that 32 in every 100 patients would make ‘worthwhile’ gains after 12 months from choosing RO DBT rather than TAU.

Suicide ideation

Despite the moderate to severe depression scores and high comorbidity in the sample, participants reported medium to low suicidal ideation throughout the trial. The odds ratio of a suicide attempt is particularly high among individuals with a history of three or more mental health disorders (odds ratio 19.7)158 and refractory depression is one of the strongest predictors of suicidal acts. 159 As 44% of the participants had three or more comorbid Axis-I disorders, the number of suicide attempts (7) in the sample is remarkably low.

Instrumental variables analysis

The aims of this analysis were to divide the effect of RO DBT between the pathways through which it is hypothesised to work and to estimate the contribution of each pathway by adjusting for confounding variables and IVs.

We found that, if there was no unobserved confounding, single-mediator models would suggest that the pathway through skills or through alliance had a positive effect on participants’ HRSD scores at 7 months. The all-mediators model suggests that skills played the main role, although further exploration of measures of alliance taken early in treatment may aid understanding of the inter-relationships between these variables.

If it is assumed that there is unobserved confounding then the IVs must be incorporated into the analysis. However, no conclusions can be drawn from the IV analysis because the results are severely affected by ‘weak’ IV bias and so are highly misleading. Although effects could have been identified in a larger study or if the IVs had been far more strongly correlated with the mediator variable, the results highlight the challenge of applying IV methods in this context rather than any specific issue with the design. In particular, we identify a tension between the desire to introduce manipulations that are effective in altering levels of mediators of interest and the desire of clinicians and treatment developers to provide the best possible care and to test new interventions at full dose. Future studies may need to be bolder and, rather than randomising to additional interventions, which have only weak effects on mediators, subdivide the intervention itself. This would randomise participants between variants of the treatment with more or less exposure to core elements and provide greater certainty that these IVs would be strong enough to mitigate bias from unobserved confounding.

In summary, using the IVs incorporated into the design of the study resulted in statistically significant but implausible and imprecise estimates. Standard diagnostics for IVs indicated that these IVs were weak and that the resulting estimates are likely to be biased and unreliable. Thus, there is, as yet, no proof of principle for IV mediation analysis.

Psychological flexibility

One of the proposed mechanisms of RO DBT is that a stable sense of emotional well-being engenders openness, flexibility and social connectedness. Thus, RO DBT aims to help rigid and rule-governed patients learn psychological flexibility. Indeed, the RO DBT skill most frequently emphasised in training is ‘flexible mind’. The results show that those participants who received RO DBT had significantly better psychological flexibility (AAQ-II) scores than those in TAU and this effect was present at all three follow-up time points, indicating that those who received RO DBT continued to experience the benefits of treatment in this area.

Emotional approach coping and expressiveness

Radically open dialectical behaviour therapy targets masked and constrained social signalling as the primary source of emotional loneliness, isolation and misery. Therefore, a core aim of treatment is to reinforce context-appropriate disclosure and uninhibited expression of emotion. The results indicate that those participants who received RO DBT scored higher on emotional approach coping and expressiveness (as measured via the EAC scale) than those who received TAU. The difference between groups was nearly significant at month 7 and increased at months 12 and 18.

Generalisability of findings

Although the RefraMED trial has much in common with clinical effectiveness studies (e.g. few exclusion criteria), it is best viewed as a hybrid efficacy trial of a promising treatment. For example, centralised control of assessments and use of IVs characterise efficacy trials, whereas cost-effectiveness analyses and exclusion of the treatment developer from providing or supervising therapy characterise effectiveness trials.

A real strength of the study was the inclusion of patients with depression that is difficult to treat, including chronic and recurrent presentations, PDs and high levels of suicidal ideation. This inclusion strategy considerably increases the generalisability of the findings. As mental health practitioners treat numerous patients with multiple mental health needs and PDs, transdiagnostic treatments, such as RO DBT, are needed to meet the demands of clinical practice.

Therapist variation

The random allocation of therapists to participants is both a scientific strength and a practical limitation. Allocating therapists to participants at random (rather than allocating the most difficult participants to the most skilled therapists, as in clinical practice) enhanced the observed performance of the best therapists while reducing that of the least skilled. The study found that 12% of the variability in the primary outcome was attributable to variation between therapists, with a difference of 2.6 points on the HRSD scale (equivalent to an effect size with a d of 0.43) between the best and weakest therapists. This substantial variation between therapists has decreased the statistical power of the study. Our power analysis assumed that for less variability in the primary outcome would be attributable to the variation between therapists; combined with more loss to follow-up than expected, this reduced the power of the study for the primary outcome at month 12.

Sample size and design considerations

As most studies in this field use the end of treatment as their primary end point, rather than a predefined time as used in study, it is difficult to compare the primary outcome with other studies that have investigated treatments for refractory depression. For example, the Re-ChORD trial did not collect sufficient follow-up data to investigate post-treatment effects26 and of the six studies19,21–25 included in the CBASP meta-analysis,9 only one study reported follow-up data 12 months after therapy. 22 Similar to the findings 12 months after the end of therapy (i.e. at month 18), the Research Evaluating the Value of Augmenting Medication With Psychotherapy (REVAMP) study22 reported no significant differences between treatment groups, although both groups showed significant changes over time for depressive symptom scores. The Tavistock Adult Depression Study27 assessed patients at 6-monthly intervals during 18 months of treatment and 2 years’ follow-up. The study reported that the intervention group started to show significantly larger decreases in HRSD scores than those shown by the TAU group at 6 months after therapy, with medium effect sizes. The CoBAlT study28 showed that BDI scores after 6 and 12 months were significantly lower in the intervention group than in the control group.

Despite the best efforts to achieve a sample size of 276 participants, only 250 were randomised and only 190 were analysable at the primary end point of 12 months. Although the desired effect size was achieved at month 12 (with a d of 0.41) and a spectacular effect size at month 7 (with a d of 1.03), the shortage of power was disappointing.

Weak instrumental variables

The choice of IVs for the RefraMED trial was necessarily speculative. To our knowledge, the RefraMED trial is the first mental health trial to examine the potential advantage of adding IVs. The results of the mechanisms analysis using IVs are consistent with those obtained using ‘weak’ IVs, which lead to biased and misleading conclusions. Nevertheless, the work has highlighted an important consideration for future researchers in this area: stronger IVs are needed if studies wish to subdivide the direct and indirect effects of treatment.

Recruitment and membership

The Mental Health Research Network and INVOLVE helped us recruit service users: two for the TSC and two for the TMG. Of these four service users, one attended all meetings and two attended one or two meetings, but we lost contact with one, who did not attend any meetings. These service users actively engaged with the development of patient information leaflets by providing detailed feedback on the readability and clarity, the management of the study by attending trial management meetings and the dissemination of the study by providing text for and feedback on the report, in particular the plain English summary and Chapter 2, Patient and public involvement.

Effect of patient and public involvement

First, the Mental Health Research Network provided anonymous feedback on the wording of the patient information leaflets, thus improving the intelligibility and accessibility of the materials. Second, input from the service users who were members of the TSC and, in particular, the TMG, informed the recruitment and dissemination strategies; this improved how the study communicated about the treatment and the findings. Third, during the trial participants were asked for feedback about the treatment; this will inform the further development of RO DBT training and delivery.

Improving cost-effectiveness

This study concluded that RO DBT in its current form is not cost-effective on the basis of NICE criteria. Hence, research should address how RO DBT can be refined to achieve the same gains but at a lower cost and for a longer period of time. There are many ways in which the delivery of RO DBT could be varied to reduce costs and it is important that future work addresses the clinical effectiveness and cost-effectiveness of these amended treatment protocols. For example, treatment might be tapered from weekly to fortnightly or monthly attendance after an initial intensive period. Another important avenue of future research will be the development and evaluation of a mobile- or web-based RO DBT support programme for patients. Several participants in this study indicated that this would support them both during the active treatment phase and afterwards.

Furthermore, RO DBT skills groups alone – without individual treatment sessions – may achieve important clinical outcomes. A recent non-randomised trial of overcontrolled personality dysfunction in treatment-resistant adults161 found that those who had learned RO DBT skills in 18 3-hour skill classes over 9 weeks showed significantly greater improvements in severity of psychological symptoms (large effect size), social safeness (medium effect size) and effective use of coping skills (large effect size) than those receiving TAU while on the waiting list for RO DBT. More thorough investigation of the most effective RO DBT skills, including exploratory analyses of the RefraMED data set, will be critical in developing a shorter version of the skills programme for use in NHS settings with limited funding.

Skill consolidation after end of treatment

It is also important to ensure that patients who have completed RO DBT continue to benefit from the skills that they have learned. One possible way of achieving this would be to offer RO DBT maintenance treatment. Feedback from the participants shows that they would welcome a maintenance programme (see Chapter 3, Patient perspectives). These ‘graduate groups’ could be led by clinicians or peers, as a recent meta-analytic review reported that peer-led skills groups show promise in reducing depressive symptoms. 168 The Dorset research site is currently developing a peer-led approach. Graduate programmes designed to consolidate skills learned in therapist-led group sessions have been trialled: a recent pilot demonstrated that patients with predominantly depressive symptoms who attended a weekly skills group for 9 months experienced significantly improved mood and a greater number of patients were able to return to work. 169 The authors reported a gradual reduction in attendance with patients, initially with weekly participation but reducing their attendance thereafter.

Longer follow-up periods

This was the first multisite trial of RO DBT for refractory depression and the largest to date. It demonstrated that RO DBT is much better than TAU at improving depressive symptoms over the 7 months of active treatment and that the benefits of RO DBT are maintained over the 12 months following the end of treatment, possibly owing to maintained skills in patients. However, we need a better understanding of whether or not and how these gains are maintained over a longer period of time, especially because of the recurrent nature of refractory depression and the associated risk of suicide: the recurrence of major depression increases the risk of suicide attempts by seven times in the 2 years after discharge from hospital. 159 It is not yet known whether or not RO DBT and control patients differ in relapse risk long after treatment. Furthermore, although RO DBT is expensive in its current form, the cost of other health and social care services were significantly higher in the control group than in the RO DBT arm. It is possible that if these cost-savings continue over a long time, RO DBT may become more cost-effective than TAU. Future studies using a longer follow-up period would shed light on these issues.

Transdiagnostic studies

Finally, although RO DBT has predominantly been investigated in adults with depression or eating disorders, it is clinically being used much more widely, including for adolescents and populations with anxiety or autism spectrum disorders. Future studies should aim to test the clinical effectiveness of RO DBT in these other overcontrolled populations, when possible using a transdiagnostic approach.

Seriousness

Any untoward medical occurrence will be deemed serious if it:

• results in death

• is life-threatening

• requires hospitalisation, accident and emergency (A&E) department visit or prolongation of existing hospitalisation

• results in persistent or significant disability or incapacity

• results in a congenital anomaly or birth defect.

Attempted suicide and acts of self-harm that are life-threatening or that require hospital inpatient care will also be considered serious in this study.

All serious events not considered to have a causal relationship with the treatment will be classified and reported as SAEs. All serious events that are considered to have a causal relationship with the treatment will be classified and reported as SARs.

Causality

Causality is the degree to which an untoward medical occurrence can be attributed to the intervention and can be classed as unknown, definitely not related, remotely related, possibly related, probably related or definitely related (see below for definitions). Only untoward medical occurrences that are considered to be possibly, probably or definitely related to the intervention will be reported as having a causal relationship.

If the untoward medical occurrence is not considered to have a causal relationship with the treatment at the time of the event (i.e. it is not believed to be a consequence of DBT) this will be classified as an AE. However, if it is considered to have a causal relationship with DBT at the time of the event it will be classified as an adverse reaction.

The investigator is responsible for defining, in his/her best judgement, the relationship of the AE/SAE to the study behavioural interventions. The degree of certainty for which the AE/SAE is attributed to the study interventions or alternative causes (e.g. natural history of the underlying disease, concomitant therapies) should be determined by how well the experience can be understood in terms of one or more of the following:

• Exposure – is there evidence that the subject was actually exposed to the behavioural interventions?

• Timing of the study behavioural interventions – did the AE/SAE follow in a reasonable temporal sequence from administration of the behavioural intervention?

• Alternative explanations for the AE such as concomitant medications, concurrent illness, non-medicinal therapies, diagnostic tests, procedures or other confounding findings.

• Response to discontinuation of the study behavioural interventions.

Terms and definitions to be used in assessing the behavioural intervention relationship to the AE/SAE are:

• unknown – use this category only if the cause of the AE/SAE is not possible to determine

• definitely not related – the subject did not receive the behavioural intervention, the temporal sequence of the AE/SAE onset relative to administration of the behavioural intervention is not reasonable, or there is another obvious cause of the AE/SAE

• remotely related – there is evidence of exposure to the behavioural intervention or there is another more likely cause of the AE/SAE

• possibly related – there is evidence of exposure to the behavioural interventions, the temporal sequence of the AE/SAE onset relative to administration of the behavioural interventions is reasonable, but the AE/SAE could have been because of another equally likely cause

• probably related – there is evidence of exposure to the behavioural interventions, the temporal sequence of the AE/SAE onset relative to administration of the behavioural interventions is reasonable, and the AE/SAE is more likely explained by the behavioural interventions than by any other cause

• definitely related – there is evidence of exposure to the behavioural interventions, the temporal sequence of the AE/SAE onset relative to administration of the behavioural interventions is reasonable, the AE/SAE is more likely explained by the behavioural interventions than by any other cause, and the AE/SAE shows a pattern consistent with previous knowledge of the behavioural interventions or behavioural interventions class.

Expectedness

An untoward medical occurrence will be considered to be ‘unexpected’ if its nature and severity are not consistent with the information in the summary of product characteristics for that treatment. If an AE is (1) considered to be related to the behavioural intervention, (2) is serious and (3) is unexpected, then it will be classed as a SUSAR.

Reporting bias

In order to obtain potentially useful data to more fully understand the ‘reporting bias’, we will also report the source of the AE and SAE data, specifically whether the information was obtained from the patient, therapist, assessor, non-study health-care provider or patient record. In addition, assessors will be trained to assess factors related to self-disclosure, in particular (1) has this event happened before and, if so, (2) is this the first time the patient has disclosed this to someone?

Adverse events that we expect may differ between conditions because of the ‘reporting bias’:

• self-reported suicidal behaviours

• self-reported non-suicidal self-injury.

Serious adverse events that may differ between conditions because of the ‘reporting bias’:

• self-reported hospitalisations, A&E visits.

Responsibility for reporting

The reporting requirements differ depending on the causality, seriousness and expectedness of the medical occurrence, as summarised in the flow diagram of safety reporting (Figure 7).

FIGURE 7.

Flow chart of safety reporting. CI, chief investigator.

Serious adverse event and serious adverse reaction

If an AE occurs that is anticipated or no more severe than expected, it will be summarised in the progress report reported to the DMEC, TSC, sponsor and the Efficacy and Mechanism Evaluation programme at 6 monthly intervals.

Suspected unexpected serious adverse reaction

If the event is considered to be a SUSAR, the DMEC, sponsor and Research Ethics Committee will be notified immediately (Table 29).

All AEs will be recorded on the trial database, evaluated by the principal investigator or other designated person responsible for the clinical aspect of the trial at each site and included in the annual safety report to the Medical Research and Ethics Committee (MREC) and the Medicines and Healthcare products Regulatory Agency.

However, for all events considered to be serious additional reporting is required. The trial co-ordinating centre must be informed within 24 hours of the site principal investigator’s knowledge of the event. A ‘Serious Adverse Event Form’ must be completed and sent to the trial coordinating centre as soon as possible. The trial manager will immediately contact the clinical and methodological chief investigators who have been delegated the responsibility of reporting any SAEs on behalf of the sponsor (University of Southampton). A decision must then be made by the investigator with the clinical responsibility for aspects of the patient’s care that are relevant to the trial whether or not an AE is related to DBT. The investigator should decide the degree to which the event is caused by the intervention. If a decision cannot be made, the investigator must contact the co-ordinating centre. Advice will be sought from the clinical chief investigator and other clinicians may be asked if a decision cannot be reached.

If it is decided that the AE is a SAR, then it must be determined whether or not the reaction is expected. If it is unexpected and thus considered a SUSAR, then the SAE form will be completed and reported to the DMEC and MREC. Serious but expected reactions or non-SARs will be recorded in the database and reported in the usual way.

For fatal or life-threatening SUSARs the DMEC and MREC will be notified as soon as possible but no later than 7 calendar days after the sponsor has first knowledge of the event. In each case, relevant follow-up information will be sought and a report completed as soon as possible. The follow-up information will be sent to the DMEC and the MREC within an additional 8 calendar days.

Non-fatal and non-life-threatening SUSARs will be reported to the DMEC and the MREC as soon as possible but no later than 15 calendar days after the sponsor has first knowledge of the event. Further relevant follow-up information will be given as soon as possible.