Multisystemic therapy compared with management as usual for adolescents at risk of offending: the START II RCT

Article history

The research reported in this issue of the journal was funded by the HS&DR programme or one of its preceding programmes as project number 12/136/70. The contractual start date was in January 2014. The final report began editorial review in October 2018 and was accepted for publication in April 2019. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The HS&DR editors and production house have tried to ensure the accuracy of the authors’ report and would like to thank the reviewers for their constructive comments on the final report document. However, they do not accept liability for damages or losses arising from material published in this report.

Declared competing interests of authors

All authors report grants from the Department for Children, Schools and Families (now the Department for Education), grants from the Department of Health and Social Care, non-financial support from the Youth Justice Board for England and Wales, grants and non-financial support from the National Institute for Health Research (NIHR) Clinical Research Network, during the conduct of the study. David Cottrell reports being the chairperson of the NIHR Clinician Scientist Fellowship panel during the conduct of the study. Ian M Goodyer reports personal fees from Lundbeck (Copenhagen, Denmark), outside the submitted work, and was a member of Efficacy and Mechanism Evaluation Prioritisation Group during the conduct of the study.

Copyright statement

© Queen’s Printer and Controller of HMSO 2020. This work was produced by Fonagy et al. under the terms of a commissioning contract issued by the Secretary of State for Health and Social Care. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science Park, Southampton SO16 7NS, UK.

2020 Queen’s Printer and Controller of HMSO

Objectives

The second phase of the START trial extended the follow-up period to 4 years for secondary outcomes and to 5 years for the primary outcome. The aim was to evaluate the medium- to long-term effectiveness and cost-effectiveness of MST compared with MAU in the UK. Extending the follow-up period enables stakeholders to make informed, evidence-based decisions about the role of MST in treating CD.

The primary objective was to compare MST with MAU in terms of the proportion of young people who had criminal convictions by 5 years post intervention. This was evaluated using data from the PNC, a centralised database of offending data across the UK. Secondary outcomes included measures of psychiatric problems and outcomes in areas where CD can result in poor outcomes: educational attainment, work adjustment, social relationships, pregnancy and physical health.

In addition, the cost-effectiveness of MST compared with MAU was explored at the 4-year follow-up in terms of quality-adjusted life-years (QALYs) and offending. Two qualitative substudies were also designed for a subsample of the families to better understand the subjective experiences of young people and their parents or carers and to better characterise the life circumstances and quality of young people’s lives post intervention. Qualitative Study 1 was carried out approximately 18 months post randomisation. Qualitative Study 2 took place around the 48-month follow-up visit.

Trial design

The trial was a pragmatic, individually randomised, multicentre superiority study. Allocation was carried out by minimisation, controlling for the number of past convictions, gender and age at onset of criminal behaviour. Treatment centre was also included in the minimisation stratification to control for differences between centres.

Figure 1 illustrates participant referral flow. Participants were referred to the trial by youth offending teams, children’s services, educational services, child and adolescent mental health services (CAMHS) and other referrers. Referred participants were screened for eligibility by a multiagency panel at each site. Those who met the inclusion criteria were considered eligible, and were sent a plain-English letter explaining the study and consent procedures and inviting them to take part. After a brief period (1–2 weeks), the families were contacted by a member of the MST team to arrange an explanatory meeting in person. This meeting was used to explain to the family what participation in the trial entailed and give them the opportunity to ask questions, but also to identify any potential exclusion criteria (e.g. risk of injury to worker, severe substance dependence). Unless the family decided at this stage to not take part, a member of the research team contacted them telephone, no longer than 3 days later, to arrange a second meeting. The second meeting entailed reviewing and completing consent forms, and completing pre-randomisation questionnaires and measures of the secondary outcomes in the study. Both information sheets and consent forms were written with input from families who had previously completed MST, and ensured to be appropriate and accessible in terms of language and age. Following the second meeting, if the family agreed to take part and no contraindications were identified, a member of the research team contacted the MST site to obtain allocation (1 : 1 ratio).

FIGURE 1.

Referral flow.

Assessments for secondary outcomes were conducted at baseline, and at 6, 12, 18, 24, 36 and 48 months post randomisation. Measures were typically completed in the family home. A member of the research team telephoned in advance to arrange the visit at a convenient time for the family. The questionnaire pack required approximately 2 hours to complete for both the young person and the parent or carer. After each visit, the family received £25 to thank them for their time.

Research assistants were carefully supervised on an ongoing daily basis by the trial co-ordinator, and had additional supervision meetings once every 2 weeks with the project co-ordinator, who had substantial experience in clinical and research activities with young people, including in MST.

The study protocol was approved by the London South-East Research Ethics Committee (reference number 09/H1102/55). The trial was overseen by the Trial Steering Committee and the Data Monitoring and Ethics Committee. Both committees were independently chaired and reviewed trial progress throughout the study. Progress reports were submitted to the funder every 6 months. One serious adverse event was recorded: a participant in the MAU group had died. This participant had been seen by the research team at the 12-month follow-up, and news of their death was reported to the research team at 36 months. The sponsor and the ethics committee reviewed the case and agreed that the death was not related to the intervention or the trial.

Multisystemic therapy

Multisystemic therapy is a manualised, licensed programme for young people exhibiting antisocial behaviour and their families. The programme uses a social-ecological approach with the aim of simultaneously affecting the multiple systems around the young person, including home life, school, peer relationships and the wider community. The techniques used in MST draw from cognitive–behavioural therapy, behavioural therapy, strategic family therapy and structural family therapy.

The MST therapist works intensively with the family, focusing primarily on the young person’s caregiver(s) to improve their parenting skills and the parent–child relationship, improve communication problems, foster support from social networks, encourage the young person’s participation in education and reduce any associations with delinquent peers. The intervention lasts between 3 and 5 months, depending on the family’s needs. Average treatment duration in this trial was 139 days. The therapists have a small caseload (four to six cases), meet with the family three times a week in the family home, and are available to be contacted 24 hours a day. The content of the programme is individualised to the specific needs of each family.

All nine trial sites were licensed by MST services to deliver the programme. Each MST team consists of at least three specially trained clinicians supervised by an accredited MST supervisor. Each team is offered weekly 1-hour conference calls for consultation with a staff member of MST services. Clinical teams additionally benefit from input and collaboration with local mental health-care professionals with postgraduate qualifications in the field of psychology, counselling or social work.

Treatment fidelity is maintained through (1) manualised weekly group supervision with a MST expert from MST services, (2) a well-developed quality assurance system and twice-yearly implementation reviews, and (3) the Therapist Adherence Measure – Revised (TAM-R). The TAM-R assessment is evaluated using a rating scale completed by a research assistant by speaking with the parents or carers. These assessments were carried out by an independent research group to maintain research separation and prevent possible researcher bias. Results from the first phase of the trial (baseline to 18 months) found good levels of treatment adherence, with an average TAM-R score of 0.698 (standard error 0.012, range 0.610–0.806) across all sites; the cut-off point for satisfactory adherence is 0.61.

After completing the MST programme, participants in the MST group received MAU.

Management as usual

Because this was a pragmatic trial involving several collaborating services across nine different sites, it was never possible to prespecify MAU content. As a result, MAU consisted of the standard care offered to young people and families who met eligibility criteria for the trial. There was a significant amount of variability in the MAU group in the type of care offered, the duration and intensity of care, and practitioners’ theoretical approaches. However, it was expected that all treatment approaches were designed in line with current community practice and treatment guidelines issued by the Social Care Institute for Excellence and NICE. Families in the MAU group were offered services specific to their particular needs (e.g. intervention for substance misuse or support for re-engaging with education). A range of professionals was involved in MAU care, including social workers, probation officers and specialist therapists.

Service use was monitored using a version of the Child and Adolescent Service Use Schedule (CA-SUS) designed specifically for the trial. The CA-SUS recorded contact with all health, social care and criminal justice sector services, including the number and duration of contacts. Results suggest that MAU programmes were multicomponent, no less resource-intensive than MST, and consistent with the complex needs and difficulties of this population. The use of MAU services in both the MST and the MAU groups, including mean number of contacts, total duration of all contacts and percentage of young people making use of each service, can be found in Table 1. At 12 months’ follow-up, the MST group had slightly briefer contacts with all services combined [t(484) = 2.03; p = 0.0429], but these differences were no longer observed at 18 months.

Changes to trial outcomes

Educational attainment (number of absences and exclusions) was initially intended to be collected from the National Pupil Database. However, it became apparent during the first phase of the trial that the data available from the database are collected over rigid time periods tied to school term dates. As a result, the data could not be reliably matched to the data collection points for the participants in the study, and therefore were not collected during the second phase. Data on whether participants were in full- or part-time education or employment were still collected as part of the Adult Self-Report measure and the Work and Relationships questionnaire.

The trial initially included an intention to carry out a characterisation of the MST and MAU services using the Children and Young People – Resources, Evaluation and Systems Schedule (CYPRESS) measure. This was unfortunately not possible during the second phase of the trial. Most of the MST sites had closed by 24-month follow-up and it was not possible to contact the practitioners associated with the MAU services delivered in the first phase of the trial, who had frequently left their posts or were no longer delivering the relevant service. As a result, a meaningful CYPRESS evaluation was not possible. However, we include a report on the CYPRESS evaluation carried out during the first phase of the trial in Appendix 6.

The Child Attachment Interview was trialled, but was determined to be not acceptable to the young people within the sample, and was therefore discontinued in favour of retaining the collaboration of the participants.

Sample size

Annual recruitment into MST was estimated at 140 families per year. Of these, approximately 30% were estimated to meet the criteria and agree to randomisation. By this calculation, each site was estimated to be able to recruit and treat about 70 families over 1.5 years, for a total of 700 participants (350 in each group). On the assumption that 30% of the MAU group would have out-of-home placements, this sample size would give 86% power to detect a 10% difference in out-of-home placements (a reduction from 30% to 20%).

Randomisation

Participants who met the criteria and screening procedures were randomised on a 1 : 1 basis by an assistant from University College London’s Trials Unit who was independent of the trial team.

Randomisation was initiated by the research assistant using a secure telephone randomisation service that ensured both allocation and concealment. A computer-generated algorithm incorporating a random element generated the allocation using the following stratification factors: treatment centre, gender, age (11–14, 15–17 years), age at onset of severe conduct problems (2–11, ≥ 11 years) and number of past convictions (≤ 2/≥ 3). These strata were selected because previous research has shown that younger age at onset and more previous convictions are associated with poorer prognosis. Minimisation ensured an even distribution of participants across both treatment groups.

Investigators and RAs were blind to treatment allocation. Allocation data were kept physically inaccessible to investigators and RAs to avoid leakage of the information. All coding, data entry and data cleaning were conducted by research team members blind to allocation. Treatment fidelity was assessed by a geographically separate research group who did not have access to outcomes.

Patient and public involvement

Each MST site had its own dedicated patient and public involvement group, which met regularly to advocate for patients and liaise with the clinical team. These groups were independent of the trial and provided an extra layer of oversight to service delivery. The trial itself had patient involvement at both the planning and the delivery stages. During study design, a meeting was organised by Music and Change UK, a mental health charity that advocates for young people. The purpose of the meeting was to receive feedback from young advisors on the study methodology and content. During the course of the study, a Trial Steering Committee met regularly to discuss the study progress and resolve arising issues. The committee included a patient-public liaison member: this was a parent of a young person with similar mental health and behavioural problems to those of the young people who took part in the study. The parent was able to advise the committee and the research team on conducting the research in a mindful way and keeping family perspectives in mind.

Missing data

All analyses were performed with statistical methods that handle missing outcome data that are missing at random. That is, the probability of an observation being missing depends only on observed variables that are accounted for in the model.

As an additional analysis, we used multiple imputation on a data set with all baseline and outcome variables. Twenty complete data sets were imputed using predictive mean matching. Pooled analyses were performed using Rubin’s rules. 43 This analysis should be treated as exploratory, as it will be using postbaseline data in the imputation, meaning that comparisons between groups cannot be treated as causal.

Primary outcome

The primary outcome was analysed using a mixed-effects logistic regression model with a fixed effect for treatment group allocation. The model was adjusted for number of convictions prior to randomisation, gender, age at onset of criminal behaviour (early or late) as fixed effects, and site as a random effect. The model was fitted in R (The R Foundation for Statistical Computing, Vienna, Austria) using the glmer function in the library lme4.

The primary analysis estimated the effect of group (i.e. the OR between MAU and MST) from the model, along with the CI and p-value, calculated from the Wald test, for whether or not the difference is significant.

Secondary outcomes

Secondary outcomes were analysed according to whether they were time-to-event, count data or continuous data.

The time-to-event outcome (time to first offence) was analysed using a Cox proportional hazards model adjusted for the same fixed effects as the primary outcome.

The count data outcomes (number of offences, and delinquency volume and variety outcomes) were analysed using Poisson mixed models adjusted for the same fixed and random effects as the primary outcome. An additional baseline effect was included for the delinquency outcomes representing the measurement at the baseline.

The continuous outcomes (all questionnaire outcomes) were analysed with a linear mixed-effects model adjusted for the same fixed effects as the primary outcome. In addition, a random intercept and slope for each individual were included.

For both the count and the continuous outcomes, separate treatment effects for each follow-up time were included. The estimated treatment effect for each time, together with the CI and p-value, was extracted directly from the model.

For young person outcomes that are measured only in participants aged ≥ 18 years, we treated observations not taken because of age as missing.

Subgroup analyses

The primary outcome and time to first offence outcome were tested for subgroup effects, with the following moderators considered:

1. gender

2. age

3. onset of CD (≤ 11 years vs. > 11 years old)

4. baseline Inventory of Callous-Unemotional Traits (ICUT) score*

5. baseline peer delinquency score*

6. baseline Antisocial Beliefs and Attitudes Scale score*

7. previous offence or no previous offence at baseline

8. diagnosis of CD or attention deficit hyperactivity disorder at baseline

9. diagnosis of CD, depression or anxiety at baseline

10. referral path.

Variables marked with * were treated as continuous variables, and the others were treated as binary.

For the primary analysis results, two-sided p-values < 0.05 were considered significant. For secondary outcomes, there was a considerable multiple testing burden. We applied a procedure called Benjamini–Hochberg44 to control the false-discovery rate at 5%. The false-discovery rate is the proportion of rejected null hypotheses that are in fact true (i.e. the proportion of positive findings that are in fact false positives). Thus, we do not control the probability of making any type I error, but control the proportion of significant results that are type I errors to be < 5%. This procedure was applied separately to the non-imputed and imputed analyses.

Resource use and costs

Resource use was recorded in interviews with young people and their families using CA-SUS. 46,47 For this medium- to long-term follow-up of the original trial, the CA-SUS was completed at 30 months, for the period since the end of the trial (24 months), and at the 36-, 42- and 48-month follow-ups, covering service use for the period since the previous assessment of these data.

Resource use data were collected in the following domains:

• delivery of the MST intervention

• use of accommodation services – foster care, residential care, staffed accommodation

• use of education services – mainstream school, specialist school, residential school, hospital school, pupil referral unit, home tuition, further education

• use of NHS secondary care services – inpatient stays (mental health and all medical specialties), outpatient appointments (mental health and all medical specialties), accident and emergency attendances (including use of ambulance services)

• use of community-based services – counsellor, family therapist, social worker, family support worker, education welfare officer, mentor, advice service

• use of prescribed psychotropic medication – antidepressants, medication for attention deficit hyperactivity disorder, benzodiazepines, medication for sleep disturbance, antipsychotics, antiepileptics

• criminal justice system resource use – police custody, youth custody, probation officer, youth offending team, solicitor, court appearance. 17

Nationally applicable unit costs were applied to each item of service use reported in the CA-SUS to calculate the total costs for each trial participant. Unit costs for education services were taken from national statistics of school income and expenditure for local authority maintained schools in England for 2011–12 and 2012–13. 48 Unit costs for hospital services were taken from the national schedule of NHS Reference Costs 2012 to 2013. 49 Costs contained in the 2013 Unit Costs of Health and Social Care were used to calculate costs of accommodation and of community-based health, social and voluntary sector services. 50 The cost of medication was calculated based on averages listed in the British National Formulary51 for the generic drug and using daily dose information collected using the CA-SUS. Unit costs for the criminal justice sector were taken from the 2013 Unit Costs in Criminal Justice52 and reports from the Home Office on the cost of criminal justice. 53

The cost of the MST intervention was calculated using a standard microcosting approach54 and utilised data on salaries of therapists as well as employer on-costs (National Insurance and superannuation), and appropriate managerial, administrative and capital overheads and conditions of service. 50 Costs of MST training, provision of MST supervision and the MST licence were also incorporated into the total cost of the intervention. 17 A detailed costing schema of the MST intervention has been published previously. 17 All unit costs are reported in UK pounds sterling for the 2012–13 financial year and inflated where necessary. Costs and outcomes were discounted at a rate of 3.5% annually, as recommended by NICE. 45

Effectiveness

The primary outcome of the economic evaluation was QALYs, calculated from health states derived from the EuroQol-5 Dimensions, three-level version (EQ-5D-3L), measure of health-related quality-of-life instrument and using the area under the curve approach. 55 The EQ-5D-3L was completed at 30, 36, 42 and 48 months’ follow-ups.

Economic analysis

Service use was reported descriptively using means and standard deviations (SDs). No statistical comparisons were made in order to avoid problems with multiple significance testing.

Differences in costs and outcomes between trial groups were compared using random-effects linear regression. The total costs, as well as the costs per sector, over the 48-month follow-up period in each trial group were summarised using the mean and SD for two data sets: the observed data and the multiply imputed data. Analyses compared mean costs, despite the skewed nature of cost data, to enable inferences to be made about the arithmetic mean, which is the most meaningful summary statistic for cost. 56 The validity of the results was confirmed by examining CIs from bias-corrected accelerated non-parametric bootstrap. Differences in QALYs were compared using the mean and SD for three data sets: observed data, mean imputation of baseline data and multiply imputed data. Differences in proportion with criminal convictions were compared using the mean and SD for two data sets: observed data and multiply imputed data. In all cases, multiple imputation followed the approach described in Missing data.

Cost-effectiveness of MST compared with MAU at 48 months’ follow-up was assessed by calculating incremental cost-effectiveness ratios (ICERs): the additional cost of one intervention compared with another divided by the additional effect. 17,57 ICERs are calculated from four sample means and are therefore subject to statistical uncertainty. Cost-effectiveness planes were generated using 1000 bootstrapped resamples from regression models of total cost and outcome by trial group. These were then used to calculate the probability that MST is the optimal choice for different values a decision-maker is willing to pay for a unit improvement in outcome (the ceiling ratio, λ). Cost-effectiveness acceptability curves are presented by plotting these probabilities for a range of possible values of λ to explore the uncertainty that exists around estimates of mean costs and effects, and to show the probability that MST is cost-effective compared with MAU. 17,58

In line with the clinical analyses, all economic analyses were controlled for treatment centre, number of previous convictions, gender and age at onset of criminal behaviour, plus the baseline measurement of the variables of interest.

As a result of an administrative error, the EQ-5D-5L was excluded from the outcome packs in the early stages of the original study, which resulted in extensive missing EQ-5D-3L data [68% (n = 462) at baseline, 49% (n = 336) at 6 months, 37% (n = 251) at 12 months, and 33% (n = 223) at 18 months], and economic analyses using QALYs were abandoned in the original economic evaluation. 17 Given that the data were missing completely at random, we have attempted to impute these missing values to generate a complete sample for economic analyses in the present study. The impact of missing data is explored in sensitivity analyses by employing different means of imputing these data, including multiple imputation, mean imputation and identification of an appropriate imputation value from the current literature.

Recruitment

Families taking part in the START trial were invited to take part in an optional qualitative interview about their experiences of MST following the completion of the 18-month follow-up period (i.e. approximately 1 year after finishing MST). Families who had completed their 18-month follow-up measures were approached to take part in the study. In total, 14 young people and 13 of their parents or carers agreed to be interviewed. Of the five families who did not take part, two initially agreed but an appointment could not be scheduled successfully, and three more did not respond to the letter and could not be reached by telephone.

Following the initial letter and information sheet inviting families to take part, families were contacted by telephone to arrange for the interview to take place. All interviews took place in the family home. Informed written consent procedures were completed immediately prior to the interview.

Participants

Of the young people who took part in the study, eight (57%) were male. The mean participant age at the time of interview was 16 years. Six were white British (43%), five were of mixed ethnicity (36%), two were black British (14%) and one participant was Asian (7%).

Procedure

Parents or carers and young people were interviewed separately using semistructured interview schedules. Questions were open-ended, with suggested prompts and follow-up questions. The interview schedules covered topics including participants’ experience of MST, changes (or lack thereof) during and after MST, and the ways in which participants believed MST helped (or did not help) them. The purpose of the interview and the interview schedule was to encourage participants to describe their experiences with MST from their own perspective and in their own words. Interviews were audio-recorded and lasted for an average of around 45 minutes for young people and 70 minutes for parents or carers.

Analysis

Audio-recordings of the qualitative interviews were transcribed verbatim. Transcripts were analysed using a framework analysis. 59 The method is a systematic analysis designed to facilitate analysis of large numbers of qualitative data. A thematic coding framework is developed first, which is then used to code occurrences of each thematic category within the entire data set. The process is dynamic, with both the coding framework and the coding patterns continuously revised throughout.

Recruitment

The purpose of the second qualitative study was to evaluate the long-term impact of MST as young people began their transition into adulthood and how this affected their maturity levels, their relationships and their outlook on the future. Families who took part in the START trial were invited to take part in this study 4 years after baseline evaluation. In total, 32 young people agreed to take part in the study; 16 had been allocated to the MST group, and 16 had been allocated to the MAU group.

Participants

Participants were matched on the basis of age, gender and region where possible. Sixteen participants were male, with a mean age of 18.3 years (SD 1.25 years, range 17–21 years); the other 16 participants were female, with a mean age of 17.7 years (SD 1.01 years, range 16–19 years). Participants in the MST group were found to be significantly younger (mean age 17.4 years, SD 0.81 years) than those who received MAU [mean age 18.6 years, SD 1.21 years, t(30) 3.09; p < 0.01].

Procedure

Participants were approached following the completion of their 48-month follow-up measures. All interviews were conducted in the participants’ homes. The semistructured interviews lasted an average of 57 minutes (range 19−117 minutes). The interview schedule focused on recent events in the young person’s life (e.g. ‘how have things been going over the last month or so?’), on important life relationships (e.g. ‘how would you describe your relationship with your parents?’) and on the future (‘how do you feel about your future?’). MST-related questions began with a prompt (e.g. ‘do you remember working with [insert name] during [insert rough date]?’) and continued with a general question (‘what can you tell me about that?’).

Analysis

Interviews were audio-recorded, transcribed verbatim and anonymised. Each transcript was analysed individually using interpretative phenomenological analysis. 60 A key organising construct emerged from this process: considering participant material in terms of how far it demonstrated a participant being forward-looking and hopeful in outlook (‘mature’) or static and frustrated (‘stuck’). Material illustrating this construct within each of four life domains (family relationships, peer relationships, child–adult transition and work-related experiences) was recorded in a table for each of the first 16 participants. Each entry in the tables was codified in terms of presence or lack of maturity and this coding system was then applied to the other 16 cases. The aim was to see whether or not there was a difference in the degree of maturity displayed between MST and MAU groups. The researcher ratings were then checked against an independent rater, blinded to condition.

Participant flow

Although 684 participants were initially randomised, one participant asked to be entirely removed from the trial during the second phase (i.e. from the 24-month follow-up point onwards). As a result, the total number of participants in the second phase was 683.

Figure 2 illustrates participant flow at each follow-up point. Table 3 outlines the different reasons why data were not collected. Every effort was made to ensure that data were collected. The research team offered flexible appointments to the families in order to accommodate other commitments, in terms of both time and location. Communication was carried out by telephone calls, text messages, e-mails and letters, according to family preference or to minimise the loss of contact. Other professionals involved with the families also helped with engagement and sometimes facilitated data collection meetings. In Table 3, ‘dropped out’ refers to a participant who explicitly requested to no longer participate in the trial and to not be contacted. Data for each time point had a set time window within which they could be collected; if it was not possible to collect the data within that window, they were classed as ‘overdue’.

FIGURE 2.

Participant flow.

Follow-up measures took place between August 2011 (earliest 18-month follow-up point) and September 2016 (latest 48-month follow-up point). Offending data were additionally collected up to September 2017.

Baseline data

Tables 4 and 5 outline the demographic and diagnostic characteristics of the young people in the sample at baseline. Appendix 2, Table 21, contains data on the routine care received by young people in each group at baseline.

Outcomes and estimation

Secondary offending outcomes

No group differences were found in median time to first offence; median value was 36 months post baseline for the MST group, and 48 months for the MAU group (HR 1.03, 95% CI 0.84 to 1.26; p = 0.78). There was a significant interaction of baseline peer delinquency scores (p = 0.015), but not of any of the other prespecified subgroups (p > 0.05). Figure 3 illustrates the proportion of non-offenders over the follow-up period.

FIGURE 3.

Kaplan–Meier plot of time to first offence. Reprinted from The Lancet Psychiatry, volume 7, Fonagy et al. , Multisystemic therapy versus management as usual in the treatment of adolescent antisocial behaviour (START): 5-year follow-up of a pragmatic, randomised controlled, superiority trial, pp. 420–30,19 copyright (2020), with permission from Elsevier.

Reprinted from The Lancet Psychiatry, volume 7, Fonagy et al. , Multisystemic therapy versus management as usual in the treatment of adolescent antisocial behaviour (START): 5-year follow-up of a pragmatic, randomised controlled, superiority trial, pp. 420–30,19 copyright (2020), with permission from Elsevier.

There were significantly more overall offences in the MST group than in the MAU group at 24, 36 and 48 months’ follow-up. The mean number of offences in the MST group at 24 months was 0.75 compared with 0.41 in the MAU group (adjusted mean difference 0.37 offences, 95% CI 0.05 to 0.69 offences; p = 0.024). At 36 months, young people in the MST group had an average of 0.45 offences, compared with 0.37 in the MAU group (adjusted mean difference 0.38, 95% CI 0 to 0.69; p = 0.048). At 48 months, those in the MST group had an average of 0.39 offences compared with 0.38 in the MAU group (adjusted mean difference 0.38 offences, 95% CI 0.03 to 0.72 offences, p = 0.033). However, these differences were no longer present at the 60-month follow-up. There were also no significant group differences when the numbers of violent and non-violent offences were analysed separately. Tables 7–9 contain a summary of secondary offending outcomes.

TABLE 7 - Mean number of offences (violent and non-violent) in year periods

Follow-up time point	Group, mean number		Difference (95% CI)	p-value
	MST	MAU
24 months	0.75	0.41	0.35 (0.03 to 0.67)	0.031
36 months	0.45	0.37	0.33 (–0.02 to 0.67)	0.063
48 months	0.39	0.38	0.35 (0 to 0.69)	0.049
60 months	0.30	0.28	–0.08 (–0.46 to 0.31)	0.700

TABLE 8 - Mean number of offences (violent) in year periods

Follow-up time point	Group, mean number		Difference (95% CI)	p-value
	MST	MAU
24 months	0.22	0.12	0.15 (–0.26 to 0.57)	0.78
36 months	0.12	0.11	–0.01 (–0.49 to 0.47)	0.96
48 months	0.11	0.14	0.11 (–0.37 to 0.58)	0.66
60 months	0.08	0.08	–0.29 (–0.85 to 0.28)	0.32

TABLE 9 - Mean number of offences (non-violent) in year periods

Follow-up time point	Group, mean number		Difference (95% CI)	p-value
	MST	MAU
24 months	0.32	0.18	0.34 (–0.03 to 0.71)	0.075
36 months	0.20	0.17	0.34 (–0.08 to 0.75)	0.110
48 months	0.17	0.15	0.38 (–0.04 to 0.8)	0.078
60 months	0.10	0.11	–0.15 (–0.67 to 0.37)	0.560

Economic evaluation

Cost-effectiveness analysis using multiply imputed data

Multiply imputed total costs and outcomes per participant at 48 months are summarised in Table 18. Costs are higher in the MST group (adjusted mean difference £5629.48), and outcomes, both QALYs and proportion with convictions, are poorer in the MST group (adjusted mean difference –0.025 and 0.03, respectively).

TABLE 18 - Incremental cost-effectiveness

Total cost/outcomes	Group, mean		Adjusted mean difference
	MST (n = 342)	MAU (n = 341)
Total cost	62579.80	55983.17	5629.48
QALYs	3.328	3.596	–0.025
Proportion with convictions	0.53	0.51	0.03

Figure 4 shows the scatterplot of bootstrapped mean differences in costs and QALYs at 48 months and indicates that MST is dominated by MAU. The y-axis indicates the difference in costs between trial groups (MST – MAU), whereas the x-axis indicates the difference in QALYs (MST – MAU). Points lying above the x-axis indicate that MST is more costly than MAU, whereas points lying below the x-axis indicate that MST is less costly than MAU. Points lying to the right of the y-axis indicate that MST is more effective than MAU, whereas points lying to the left of the y-axis indicate that MST is less effective than MAU. The majority of scatter points lie in the north-west quadrant (51%), where MST is less effective and more costly than MAU. Very few points (7%) lie in the south-east quadrant, where MST is more effective and less costly than MAU. The remaining scatter points lie in the north-east (24%; MST more effective and more costly) and south-west (18%; MST less effective and less costly) quadrants.

FIGURE 4.

Bootstrapped mean differences in costs and effects of MST compared with MAU at the 48-month follow-up. Reprinted from The Lancet Psychiatry, volume 7, Fonagy et al. , Multisystemic therapy versus management as usual in the treatment of adolescent antisocial behaviour (START): 5-year follow-up of a pragmatic, randomised controlled, superiority trial, pp. 420–30,19 copyright (2020), with permission from Elsevier.

Reprinted from The Lancet Psychiatry, volume 7, Fonagy et al. , Multisystemic therapy versus management as usual in the treatment of adolescent antisocial behaviour (START): 5-year follow-up of a pragmatic, randomised controlled, superiority trial, pp. 420–30,19 copyright (2020), with permission from Elsevier.

The cost-effectiveness acceptability curve using QALYs as the outcome (Figure 5) shows the probability that MST is cost-effective compared with MAU, and suggests that MST is not cost-effective compared with MAU at 48 months at the NICE willingness-to-pay threshold of £20,000 per QALY, as the probability of this is < 50%. The probability of MST being cost-effective compared with MAU ranges from 19% to 25% for a willingness-to-pay threshold ranging from £0 to £250,000 per QALY. Cost-effectiveness analyses were repeated for mean imputation of QALYs, and the results remained the same, with no evidence of MST being cost-effective compared with MAU for the full range of values of willingness to pay for QALYs.

FIGURE 5.

Cost-effectiveness acceptability curve showing the probability that MST is cost-effective compared with MAU for different willingness-to-pay thresholds for a unit improvement in QALYs at 48 months’ follow-up. Reprinted from The Lancet Psychiatry, volume 7, Fonagy et al. , Multisystemic therapy versus management as usual in the treatment of adolescent antisocial behaviour (START): 5-year follow-up of a pragmatic, randomised controlled, superiority trial, pp. 420–30,19 copyright (2020), with permission from Elsevier.

Reprinted from The Lancet Psychiatry, volume 7, Fonagy et al. , Multisystemic therapy versus management as usual in the treatment of adolescent antisocial behaviour (START): 5-year follow-up of a pragmatic, randomised controlled, superiority trial, pp. 420–30,19 copyright (2020), with permission from Elsevier.

Figure 6 shows the scatterplot of bootstrapped mean differences in costs and proportion of sample with convictions. The majority of scatter points lie in the north-west quadrant (62%), where MST is less effective and more costly than MAU. Very few points (6%) lie in the south-east quadrant, where MST is more effective and less costly than MAU. The remaining scatter points lie in the north-east (13%; MST more effective and more costly) and south-west (18%; MST less effective and less costly) quadrants.

FIGURE 6.

Bootstrapped mean differences in costs and effects (proportion with convictions) of MST compared with MAU at 48 months’ follow-up. Reprinted from The Lancet Psychiatry, volume 7, Fonagy et al. , Multisystemic therapy versus management as usual in the treatment of adolescent antisocial behaviour (START): 5-year follow-up of a pragmatic, randomised controlled, superiority trial, pp. 420–30,19 copyright (2020), with permission from Elsevier.

Reprinted from The Lancet Psychiatry, volume 7, Fonagy et al. , Multisystemic therapy versus management as usual in the treatment of adolescent antisocial behaviour (START): 5-year follow-up of a pragmatic, randomised controlled, superiority trial, pp. 420–30,19 copyright (2020), with permission from Elsevier.

The cost-effectiveness acceptability curve using criminal convictions as the outcome (Figure 7) suggests that the probability of MST being cost-effective compared with MAU remains < 25% for a range of willingness-to-pay thresholds for a percentage point reduction in criminal convictions.

FIGURE 7.

Cost-effectiveness acceptability curve showing the probability that MST is cost-effective compared with MAU for different willingness-to-pay thresholds for a percentage point reduction in criminal convictions at 48 months’ follow-up. Reprinted from The Lancet Psychiatry, volume 7, Fonagy et al. , Multisystemic therapy versus management as usual in the treatment of adolescent antisocial behaviour (START): 5-year follow-up of a pragmatic, randomised controlled, superiority trial, pp. 420–30,19 copyright (2020), with permission from Elsevier.

Reprinted from The Lancet Psychiatry, volume 7, Fonagy et al. , Multisystemic therapy versus management as usual in the treatment of adolescent antisocial behaviour (START): 5-year follow-up of a pragmatic, randomised controlled, superiority trial, pp. 420–30,19 copyright (2020), with permission from Elsevier.

Qualitative study 1

The analysis generated 15 themes that were grouped under four domains; these are summarised in Table 19. The first domain related to how families described their change trajectory (or lack thereof) after taking part in MST. The second and third domains related to how families attributed initial and sustained change to aspects of MST. The fourth and final domain related to families’ descriptions of factors unrelated to MST that affected change.

Qualitative study 2

The coding revealed good inter-rater agreement for male data (92.7% agreement, κ = 0.73) and for female data (87.5% agreement, κ = 0.76). We examined evidence of an effect of therapy group (MST or MAU) on maturity score (mature vs. stuck). Among male participants, a Mann–Whitney U-test that indicated significantly greater maturity was associated with having received MST (median = 1) instead of MAU (median = 0.25), U = 12; p = 023. Among female participants, differences in maturity were not significantly associated with receiving MST (median = 1) relative to receiving MAU (median = 0.88), U = 21; p = 16. Illustrative examples of how maturity was apparent are presented below. This is summarised below.

Results summary

The overall results of the second phase of the START trial indicated that there were few differences between MST and MAU. There were no differences in the primary outcome between the treatment arms; the proportion of young people with a criminal conviction in the 5-year follow-up to randomisation was not significantly different between MST and MAU. The predicted moderators were also found not to be significant, with the exception of the baseline peer delinquency score. Analysis of the secondary offending outcomes suggests that young people in the MST group had a greater mean number of offences over the follow-up period, but these differences disappeared by 60 months. Almost none of the secondary end points, based on self-report data from young people and their families, showed any significant differences between the groups (with the exception being parents’ perceptions of inconsistent discipline at 24 months only). The economic evaluation suffered from partially missing baseline data for the measure of health status, but this affected only one aspect of the analysis. The results suggest that service use was similar in the MST and MAU groups, with the MST group reporting a somewhat greater number of police contacts. Service costs for MST were found to be higher than those for MAU and unlikely to be more cost-effective. The first qualitative study, conducted around 18 months after randomisation, suggested that families had different trajectories following the MST intervention, with some continuing to improve and others struggling. Parents or carers and young people spoke about the different factors to which they attributed change, including the specific techniques and approach of MST, as well as external factors. The second qualitative study was conducted later to address the transition from childhood to adulthood. In male (but not in female) participants, receiving MST was associated with significantly higher maturity levels. Participants spoke about different aspects of the transition to adulthood, including work and relationships.

Interpretation

The results of the second phase of the START trial, as reported here, continued follow-up to 48 months for secondary outcomes and to 60 months for primary outcomes. The outcomes of the first phase17 did not find any evidence that MST was superior to MAU in reducing out-of-home placements or criminal convictions. The secondary outcomes in the first phase suggested that antisocial behaviour, as based on parents’ or carers’ reports, improved more rapidly in the MAU condition, although these group differences were not sustained in the long term (i.e. at 18 months’ follow-up).

The results of the 60-month follow-up are consistent with those reported in the first phase; there is no evidence of MST being superior to MAU in reducing criminal convictions among this population. Although young people in the MST group had more criminal convictions on average at the 24-, 36- and 48-month follow-up points, these differences were not sustained to the 60-month follow-up.

Secondary outcomes reported by young people, including antisocial behaviour, psychiatric problems, and emotional and physical well-being, did not yield any significant group differences, which suggests that MST did not benefit young people in terms of these domains beyond the provisions of MAU. The results also indicate that MST did not contribute to improved employment or education rates, or to differences in pregnancy rates.

Significant group differences in parent-rated perceptions of inconsistent discipline were found at 24 months, but none of the outcomes consistently favoured one group over the other in the long term. This is in contrast to the outcomes of the first phase of the trial, which suggested that, according to parents or carers, the intervention brought about more rapid change to young people’s antisocial behaviour in the MST group than in the MAU group. Data from the present longer-term follow-up suggest that there is no consistent pattern to the differences in behavioural change beyond 18 months.

The results from the economic analysis, in line with the results from the first phase of the trial, do not indicate any economic benefit for MST, with evidence instead to suggest there is a higher probability of MAU being cost-effective as a result of lower costs and slightly better outcomes in the MAU group. Although EQ-5D-3L scores were very similar in the two groups at all time points, it is interesting to note that scores fell over time, suggesting a reduction in health-related quality of life over time for the sample as a whole. It is hard to know exactly what is driving this reduction, but it may relate to the transition from young person to adulthood, as young people move away from home, their families and the services they are familiar with, and move out of education and into employment or unemployment.

It is unclear why the results of the present trial are at odds with previous US-based research that found MST to be beneficial compared with MAU. 11–14 One possibility is the difference between the juvenile criminal justice systems of the USA and the UK. The criminal justice system in the USA has a stronger punitive emphasis than that in the UK, where the emphasis is on rehabilitation. 63 The MST approach may have more common ground with MAU services in the UK than with those in the USA; previous studies have found larger effects when MST was compared with non-multimodal treatment. 64 Alternatively, young people in the USA may have more of an incentive to engage with MST.

In addition, it is also possible that MAU was able to effectively address the wide range of individual needs that contributed to the young person’s antisocial behaviour problems. MST is essentially a social learning-based model that strongly targets parenting skills and contingent interactions between young people and their parents or carers to reduce behavioural problems. The evidence base for MST suggests that it is less effective when antisocial behaviour is accompanied by comorbid conditions, especially serious emotional disturbance. In this respect, parental qualitative interviews from an earlier small randomised controlled trial evaluation of MST in the UK revealed that, in some cases, parents or carers believed that MST was not a good match for their children’s emotional needs (e.g. when their children had symptoms of depression and post-traumatic stress disorder as a result of maltreatment).

Young people in our sample were not typically referred to court, and we are likely to have randomised a more diverse group of antisocial young people in our trial than in previous MST evaluations. MST may not have been able to better address the diverse needs of our sample of antisocial young people than MAU. It is noteworthy that all young people in the trial were subject to multiagency review and treatment planning prior to randomisation in an effort to define their mental health needs and to provide a credible MAU based on these.

Although this is a potential explanation for the absence of group differences, it is curious that the results of our qualitative study suggest that, at least for some of our participants, MST conferred a benefit, especially males, where as MAU did not. Young people who were interviewed from the MST group showed greater evidence of having become more mature post intervention than those in MAU group.

How do we explain these findings? First, as can occur with a mixed-methods design, the qualitative group threw up inductively something that we did not quantitatively measure – the multifaceted construct of maturity – so we are unable to look at the whole sample in relation to this construct. Second, and speculatively, it may be that the maturity and well-being of young people benefited from the coherent delivery of an integrated intervention by one professional (MST) in a way that is not captured by symptom-reduction measures. For example, from the professional viewpoint, MST is an intervention designed to reduce antisocial behaviour by targeting parenting skills and family interactions. It may be that, for young people and parents or carers, receiving MST was seen as a chance to turn the young person’s life around. In this respect, the intense involvement of a committed professional, coming to their house numerous times a week, may have validated young people and sent a message about parental investment and concern that was not captured in the quantitative measures.

This idea is consistent with the qualitative findings, suggesting that the intervention helped young people better understand the impact of their behaviour on their parents’ or carers’ lives, and that, when young people were involved in sessions, the therapist allowed them to imagine a better future. These data suggest growth in the young person and changes that may be thought of from the vantage point of ‘maturity’. These changes were perhaps driven by renewed investment, concern and commitment on the part of parents or carers that was not skill based. In turn, the young person felt validated, and began to work on changing their lives in ways commensurate with their own goals, which was aided by the modelling and support of the MST therapist.

The implication is that a by-product of MST is growth towards greater maturity, which is related to the young person’s intrinsic motivation and goals (as opposed to reducing antisocial behaviour), which may reflect more extrinsic goals. Although speculative, the qualitative findings open up potentially exciting and beneficial avenues for future MST trials.

Generalisability

The present outcomes represent what is to our knowledge the largest trial, with longest follow-up, of MST in the UK to date. Although the recruitment criteria limited the population sample, the results do not support MST as superior to MAU for young people with CD for reducing offending rates, or for psychological, emotional or physical well-being.

The sample for this study may be broadly considered to be representative of the general UK population. Young people were recruited into the trial using a careful and thorough process, with 87% of the eligible families approached agreeing to take part. It is somewhat less clear how representative the sample is of the population of young people with antisocial behaviour problems, although the two groups were matched on a range of demographic parameters, and moderator analyses did not yield significant differences in outcomes.

It is possible that the outcome of the trial would have been different had it been compared with different MAU services, or had young people with more severe behavioural difficulties been included, but further research would be required to evaluate this.

Limitations

There are a number of limitations to this study. These include some failures arising from practical barriers or human error, others stemming from the pragmatic nature of the trial, and still others that can be attributed to the inherent nature of conducting randomised controlled trials with human participants. We outline these below.

The study failed to deliver on three elements of the intended data collection: objective education data, the CYPRESS evaluation and the proportion of missing EQ-5D-3L data at baseline. It was intended that objective education data (attendance, suspensions and exclusions) would be collected from the National Pupil Database. Although it was possible to obtain the data, it would not have been possible to match them to the study’s follow-up points, as they are collected and stored based on school term dates. Efforts to obtain this information from schools directly were also unsuccessful. The absence of comprehensive data on school adjustment is disappointing, but unfortunately consistent with other randomised controlled trials of MST, none of which were able to deliver education data post intervention. On a similar note, although we were able to obtain objective offending data from the PNC, we understand that it is not a perfect repository of information. The first issue is that the information it contains is supplied by police staff, and therefore is prone to human error and delays. 65 Second, not all committed crimes will be detected by police,66 although we supplemented the offending data in our sample with self-reported data. The barriers that come with database reliance should be taken into consideration in subsequent evaluations that have outcomes based in education or offending.

The CYPRESS measure is designed to define service characteristics; this was another measure for which we were not able to collect data. At this stage of the follow-up period, most of the MST sites that had taken part in the intervention were no longer being funded and had closed, with the clinical staff moving on to other posts. This was, of course, not possible to anticipate in the planning stages. However, the CYPRESS evaluations that were carried out during the first phase of the trial still give us some meaningful insight into the service delivery frameworks of both MST and MAU.

The economic evaluation was limited by a substantial number of missing EQ-5D-3L data as a result of an administrative error that was identified and rectified during the trial. Health-related quality of life was recorded for some time points, and a number of different approaches were taken to impute these missing data to make best use of the available information. The method selected made no difference to the results reported, and similar results were found when outcomes were measured in terms of offending, thus giving greater confidence in the validity of the results. Perhaps more importantly, the EQ-5D-3L may not have been the most appropriate measure for this particular population. Although NICE recommends the use of the EQ-5D-3L in all health economic evaluations to allow comparison across groups and interventions, the current population is defined less by their health status and more by their behaviour. As a result, a health-related measure of quality of life may have not have been sensitive enough to changes in the broader quality of these young people’s lives. In terms of the cost data, in common with most clinical trials, use of services was self-reported by young people and their parents or carers, which will inevitably have limited accuracy. However, there is no reason to assume any bias in accuracy between the two groups, and the pattern of costs and effects remained consistent over a long period of time, with little difference in service use or costs between the two groups, and little difference in outcomes, giving us greater confidence in the conclusion that there is no evidence to suggest that MST is cost-effective compared with MAU.

It is possible that selective attrition had an impact on the results of the study. Although the percentage of participants who were followed up was substantial for a study of this size and duration, dropouts may have been self-selecting. As a result, the findings represent a sample of participants who chose to continue taking part in the study. However, data for the primary outcome were objective and were collected from a national database, which minimised the self-selection bias. It is also worth noting that some measures were not collected despite participants’ willingness, because in some cases parents had lost touch with their children and therefore could not meaningfully report on their behaviour or well-being. Certain measures were also designed for young people aged ≥ 18 year, which reduced the pool of available participants.

Another limitation is the lack of clarity about whether or not the services received by those in the MAU group were representative of reality. The services provided to those in the MAU group were deliberately not standardised; the aim of the trial was to contrast MST services with whatever provision was typically awarded to young people with CD. Because services made proactive bids to participate in the trial, it is possible that the MAU services associated with the trial were of a higher standard or entailed a higher level of involvement than may typically be expected. However, these findings do suggest that MST is not superior to MAU when MAU is delivered to a high standard.

Finally, there are certain methodological limitations that necessarily affect all randomised controlled trials evaluating psychological intervention. First, unlike pharmacological interventions, it is impossible for a psychological intervention to be double-blind. The research assistants who collected the data were blinded to treatment condition, but it would not have been possible for the service users to be blinded to their own treatment. Second, some psychosocial problems can be self-correcting over time, in which case these effects would be indistinguishable from the effects of the intervention. Although this is true, the projected long-term outcomes for young people with CD are generally poor, and evidence from this and other trials evaluating organised treatment approaches suggests that interventions of this type can improve young people’s mental health and well-being and have a positive effect on family functioning. Finally, as discussed earlier, it was not possible to maintain perfect separation between MST and MAU delivery, as practitioners involved in MST may have influenced the teams delivering MAU. These are all methodological problems that can, and probably did, influence this trial, and therefore should be considered when interpreting the outcomes.

Implications for practice

The outcomes of the trial suggest that MST is well accepted and tolerated by service users. Families who engaged with MST spoke positively about the impact it had on them. This was particularly true for parents, who noticed improvements in young people’s behaviour before the young person noticed it themselves, and reported improvements in their own mental health and their perceived family functioning. 17 This is also evident in the results from the first qualitative study, in which young people and parents or carers spoke positively about the intervention and its effects on their well-being and their family. This was also corroborated by the relatively few dropouts from the intervention; most families completed the 4-month programme. This is something that young people and parents or carers may want to consider when deciding which treatment pathway is right for them.

Nevertheless, we were unable to demonstrate an objective superiority of MST to MAU in terms of reducing criminal behaviour at the 5-year follow-up. This is a finding that may be especially pertinent to professionals who need to make decisions about service commission or referrals. Although MST does lead to positive outcomes for young people and their families, other alternatives may be more suitable, depending on service availability and considerations around funding. The economic analysis does not suggest that MST is more cost-effective, and the service costs are higher than those for MAU. This finding may also factor in decisions made by commissioners or clinicians. The outcomes of the second qualitative study also suggest that MST was helpful for changing young people’s attitudes to and capacity for employment, and their outlook on their future work prospects. These findings may be beneficial for programmes focused on workforce development, which may want to incorporate aspects of MST principles into work with young people with CD.

The research team is aware that the teams that implemented MST could not have maintained a perfect separation between the delivery of MST and MAU. As a result, some aspects of MST delivery may have influenced the delivery of MAU. Monitoring this was not an aim of the trial, so we are unable to comment on the extent and outcomes of this officially, but it was something that we noticed informally and should be a consideration both for service delivery and for similar future trials.

In broader terms of treating young people with CD, the present study contributes to a wealth of existing literature that suggests that organised treatment holds substantial benefits for young people who experience these problems, as well as for their families. Young people with behavioural problems often receive inadequate support, if they receive support at all. This can have serious, long-term consequences for the young person’s well-being. Both service users and practitioners need to be aware of the value and benefits of organised interventions, which includes MST, the services that constituted the MAU group, and other types of programmes and therapeutic approaches available to young people with these types of problems. Moreover, the findings from the second phase of the trial, as described here, suggest that the positive outcomes observed in the first phase (up to 18 months’ follow-up) are maintained in the long term.

Future research

To our knowledge, the present study has yielded the largest data set related to young people with CD to date. There is significant potential to expand the scope of these findings beyond what has been reported here. It is the intention of the research team to carry out multiple follow-up analyses before making the data set publicly accessible via an open access policy. Initial follow-up analyses will include causal modelling of the outcomes in order to identify predictors of outcomes, moderators and/or mediators. This will be done using sophisticated data analysis approaches, including multilevel modelling and bi-factor analysis. The results of one of these analyses, focusing on using a bi-factor model for distinguishing between general and specific therapeutic changes, have already been published. 67

Future research may focus on identifying individuals who particularly benefited from MST. Earlier in this report we have discussed the potential of MST being more beneficial to young people whose CD problems are on the more severe end of the spectrum; however, this has not yet been tested empirically. Identifying specific moderators could help to deliver a more personalised treatment approach as well as interventions that are better tailored to the specific needs of young people with behavioural problems. This is particularly in the light of the fact that none of the moderators used in this study, despite being selected on the basis of previous research, yielded a significant result (with the exception of baseline peer delinquency). Assumptions can be tested in small-scale studies, and then evaluated in larger samples.

As discussed in Limitations, this trial had its share of barriers and drawbacks. Future trials with a similar design should take these into account at the planning stage and make efforts to mitigate or eliminate the effects. This may mean taking into account the limitations of education and criminal-offending databases and either consulting multiple sources (where possible) or supplementing with self-reported data. We have also noted that delivering MST at a specific site may have influenced MAU delivery. Although a solution to this was not possible in this case because of the limited number of sites that were funded for MST, other trials may consider a cluster randomised trial instead, which could help reduce this.

Finally, there are broader questions about the structure of mental health disorders in adolescents, and their diagnosis and treatment, which future research must ultimately address. The research team hopes to combine the data set resulting from this study with the work done on the RESTART (Researching cognitive and brain functioning in Typically-developing and At-Risk Teens) trial to begin formulating research to address these overarching issues.

Protocol

The protocol can be accessed at www.ncbi.nlm.nih.gov/pubmed/23962220.

Thank you

The trial team would like to thank the research assistants, administrative assistants and interns who assisted with data collection and management over the course of the START trial: Anna Abramowski, Gemma Antoniak, Ifkar Arifin, Chloe Austerberry, Helen Baldwin, Jakki Birtwistle, Isobel Brunton, Matthew Bury, Angeline Dharmaindra, Jessie Greisbach, Karolina Kaminska, Ulrike Klossek, Nikki Knight, Nicolas Lorenzini, Pauline Martin, Christina Maurice, Melissa McRae, Lauren Mountain, Jennifer O’Connell, Rebecca Packer, Lauren Pae, Emma Percy, Rebecca Ramsden, Alix Smith, Samantha Tipping, Kasturi Torchia, David Turgoose, Kate Watchorn and April Wiltshire.

Thank you to Hannah Guzinska and Dominic Conroy for carrying out qualitative study 1 and 2, respectively. Thank you to Cressida Gaffney for her work on CYPRESS during the first phase of the trial.

Thank you to Nicole Hickey (Imperial College London), who prepared the youth offending data from the PNC records.

Thank you to Eric Taylor and the members of the Trial Steering Committee that he chaired, and to Philip Graham and the members of the Data Management and Ethics Committee that he chaired, for their counsel.

Thank you to Cathy James (National Multisystemic Therapy Programme Lead) and Kathryn Harney (Associate Director of Research, Greater Manchester Mental Health NHS Foundation Trust) for their support, co-operation and guidance.

Thank you to the young people, parents and carers who took the time and effort to make this research happen, and without whom it would not have been possible.

Contributions of authors

Professor Peter Fonagy conducted the design, delivery, data analysis and write-up of the research.

Dr Stephen Butler advised on the day-to-day running of the trial and liaised with the clinical sites.

Professor David Cottrell advised on the effectiveness and management of the interventions in CAMHS.

Professor Stephen Scott advised on running randomised controlled trials with antisocial young people.

Professor Stephen Pilling developed the trial method, including the assessment of therapist competence, treatment fidelity and parallel process evaluation.

Professor Ivan Eisler advised on the implementation of systemic interventions.

Dr Peter Fuggle was the senior co-investigator responsible for the clinical validity of the research.

Dr Abdullah Kraam provided the psychiatric coverage for the trial and the establishment of diagnostic categories.

Professor Sarah Byford conducted the economic evaluation of the intervention.

Professor James Wason carried out the statistical data analysis.

Professor Jonathan A Smith was responsible for the qualitative research.

Dr Alisa Anokhina was the trial co-ordinator responsible for trial closure, preparation of data for analysis, and consolidation of the final report.

Ms Rachel Ellison was the trial co-ordinator responsible for the day-to-day running of the trial and management of the research assistants.

Ms Elizabeth Simes was the trial co-ordinator responsible for the day-to-day running of the trial and management of the research assistants.

Ms Poushali Ganguli conducted the economic evaluation of the intervention.

Dr Elizabeth Allison advised on the conceptualisation and application of the design and the interpretation of data, and supervised the research staff.

Professor Ian M Goodyer was the senior co-investigator with joint responsibility for design, delivery, analysis and write-up of the research.

Publications

Fonagy P, Butler S, Cottrell D, Scott S, Pilling S, Eisler I, et al. Multisystemic therapy versus management as usual in the treatment of adolescent antisocial behaviour (START): a pragmatic, randomised controlled, superiority trial. Lancet Psychiatry 2018;5:119–33.

Fonagy P, Butler S, Cottrell D, Scott S, Pilling S, Eiser I, et al. Multisystemic therapy versus management as usual in the treatment of adolescent antisocial behaviour (START): 5-year follow-up of a pragmatic, randomised controlled, superiority trial. Lancet Psychiatry 2020;7:420–30.

Data-sharing statement

Requests for data access should be addressed to the corresponding author.

Patient data

This work uses data provided by patients and collected by the NHS as part of their care and support. Using patient data is vital to improve health and care for everyone. There is huge potential to make better use of information from people’s patient records, to understand more about disease, develop new treatments, monitor safety, and plan NHS services. Patient data should be kept safe and secure, to protect everyone’s privacy, and it’s important that there are safeguards to make sure that it is stored and used responsibly. Everyone should be able to find out about how patient data are used. #datasaveslives You can find out more about the background to this citation here: https://understandingpatientdata.org.uk/data-citation.

Disclaimers

This report presents independent research funded by the National Institute for Health Research (NIHR). The views and opinions expressed by authors in this publication are those of the authors and do not necessarily reflect those of the NHS, the NIHR, NETSCC, the HS&DR programme or the Department of Health and Social Care. If there are verbatim quotations included in this publication the views and opinions expressed by the interviewees are those of the interviewees and do not necessarily reflect those of the authors, those of the NHS, the NIHR, NETSCC, the HS&DR programme or the Department of Health and Social Care.

Background

Between November 2011 and April 2012, members of your service took part in the CYPRESS procedure as part of the START randomised controlled trial. The intention here was to develop a measure of service quality that allows assessment of the service delivery aspects of complex interventions for children and young people with conduct and related psychosocial problems, and to characterise MST and MAU in the START trial along these dimensions. The measure is intended to have utility across a range of interventions for young people in general, developed as it has been to capture the service delivery frameworks of both MST and usual services.

CYPRESS was developed by Stephen Butler, Stephen Pilling and Cressida Gaffney on the basis of a review of existing measures in the field, as well as research into the central aspects of service delivery that support complex interventions, as an interview-based measure of the service delivery frameworks supporting complex interventions. The opinion of a further two experienced clinicians was called on in the development of the measure; they reviewed an early draft and their considerations were incorporated into the design of CYPRESS. CYPRESS was piloted with two non-START trial teams, and subsequently administered to 17 teams (9 MST and 8 MAU) taking part in the START trial. The results of these interviews were used to compare the service delivery elements supporting MST and MAU, and to characterise the MAU services in the trial.

The measure

CYPRESS comprises three sections, aimed at capturing key issues pertaining to service delivery level factors that characterise a service. These are as follows: service characteristics (scored out of 30), team operation (scored out of 30) and delivery of interventions (scored out of 40). Below are the item headings and one sample item for each of the three sections.

Characterisation of management as usual

Characterisation and comparison of the different MAU services in the START trial is an interesting part of this endeavour, offering a perspective on the relatively unexplored usual services, seen here in the context of comparison with MST. Given the restrictions on the reporting of outcome data until the trial is completed, the characterisation of MAU will be the main findings reported here.

The teams constituting MAU represented the following categories:

• child and adolescent mental health services

• youth offending team

• family support charity project

• specialist local authority child safeguarding service

• specialist local authority family intervention team.

Some key observations on the nature of the MAU services are summarised below.

Conclusions

Analysis found that median scores on the CYPRESS measure across the three subsections and total score showed significant differences between MST and MAU teams. The MST teams consistently scored higher on average and demonstrated narrower ranges of scores across the three subsections and total score. However, individual MAU teams in several cases scored higher than MST teams. A case comparison explored a number of ways in which the highest- and lowest-scoring MAU teams were different from one another in terms of the service delivery framework in which they work. It should be noted that the findings outlined above represent both data that emerged directly from the CYPRESS procedure and understandings formed by the raters as a result of interviewing a number of teams and synthesising the information gathered.

We concluded that CYPRESS is a viable means of distinguishing between services that provide complex interventions for children and young people. It offers the chance to characterise services on the basis of the service delivery frameworks that support them rather than by the specific features of interventions, and as such can be used across a range of different services. The potential for bias in development, administration of CYPRESS and analysis of scores is noted, and all findings are considered in view of these, yet overall the CYPRESS procedure indicates that there is potential for further development and use of this means of measuring and characterising services. Further development and testing of CYPRESS is recommended.

Interview schedules for parent and young person

Analysis stage 2: developing the coding frame

Analysis stage 4: chart development

Analysis stage 5: mapping and interpretation

Interview schedule

Thanks for agreeing to meet with me and talk a bit about how things are going for you now and about your experiences in receiving help. As I mentioned on the telephone, I would like to speak with you about how things are going with your friends, your relationships with your parents, how you feel about the future, things like that. I really want to hear what you think, including good things and bad things, so there are no wrong or right answers to the questions that I will ask. It’s however you see it really.

1. I would like to start by asking you how things have been going over the last month or so? (Prompt: any major activities, things of particular interest, usual routines?)

2. Please could you tell me about your friends? (Prompt: people you hang out with; what are they like, what do you do together?)

3. What do you think makes a good friend?

4. Have you experienced a good friend doing something wrong? (Prompt: if so, how did you feel?)

5. How would you describe your relationship with your parents? (Prompt: what is good about it/not so good about it?)

6. Can you tell me about a good time with your parents?

7. Now tell me about a bad time.

8. How do you feel about your life at present? (Prompt: good things/not so good things?)

9. What do you think it means to become an adult? (Prompt: what is the main difference between being a child and being an adult?)

10. How much control or influence do you think you have over how that future will unfold? (Prompt: up to you to make the most of it vs. see what happens; any abilities you can draw on?)

11. Have you been getting support from any professionals over the last couple of years? (Prompt: if so, who from and what has that help been like?)

12. Do you remember the MST person who saw you about 4 years ago? What do you remember about that? (Prompt: good things/less good things?) (**Only asked of MST group.)

13. Do you think there is help available as you go through the process of becoming an adult? (Prompts: parents, peers, professionals, others?)

14. Looking ahead, how do you feel about your future? (Prompts: exciting vs. worrying, job prospects?)