Brief education supported psychological treatment for adolescent borderline personality disorder: the BEST feasibility RCT

Objective 1

To refine the BEST intervention to maximise the likelihood that it can be successfully implemented within the context of educational settings (schools and colleges).

Objective 2

To assess the feasibility of evaluating the clinical effectiveness and cost-effectiveness of the refined BEST intervention in a future RCT.

The factors we planned to consider in assessing feasibility were:

1. our ability to recruit participants to time and target

2. our ability to retain participants in the trial post randomisation

3. the acceptability and suitability of the proposed outcome measures and feasibility of identifying all costs and resource use that are relevant to the intervention

4. the ability of staff to deliver the intervention with fidelity to the model

5. the degree of contamination of the control arm (i.e. the extent to which participants randomised to the control arm received elements of the trial intervention)

6. the acceptability of the intervention from the perspective of staff and young people.

Stage 1: intervention refinement

The first stage of the study was intended to allow for refinement of the intervention and study processes to maximise the likelihood of their successful implementation. This preliminary stage of the study comprised two components: (1) a rapid evidence synthesis of barriers to and facilitators of the implementation of indicated mental health interventions within schools and colleges, and (2) a pilot of the intervention within three educational settings (one school and two colleges). Learning from the evidence synthesis and findings from the pilot were combined to enable us to finalise the intervention manual and resources, refine the practitioner training workshop and amend study procedures in preparation for the feasibility RCT.

Stage 2: feasibility randomised controlled trial

This second stage of the study comprised a feasibility RCT in which eligible young people were randomised in a 1 : 1 ratio to receive either BEST plus TAU or TAU alone. Participants were assessed pre-randomisation and followed up at 12 and 24 weeks using a battery of validated self-report and interviewer-rated measures. The feasibility RCT was accompanied by a detailed mixed-methods process evaluation exploring the acceptability of the intervention, fidelity of implementation across educational settings and contamination of the control arm.

Design

A rapid evidence synthesis is a type of systematic review in which components of the systematic review process are simplified, omitted or made more efficient to produce the information required for a specific purpose within a limited time period. 37 To ensure completeness and quality, we retained core features of the systematic review process, including publication of the protocol, a comprehensive literature search, and duplicate study selection and data extraction. The key simplification we made to the review process was the omission of formal assessment of the scientific quality of the included studies. This was considered less relevant to the aims of the review because the transferability of findings related to barriers to and facilitators of implementation of an intervention is not necessarily dependent on the validity of the study design used to examine its effectiveness, risk of bias or other factors commonly assessed as part of determining study quality. In addition, we limited the scope of the review by including only English-language publications and studies conducted in high-income countries.

The protocol was registered with the PROSPERO registry prior to implementation of the search strategy (CRD42018102830).

Search strategy

We searched eight electronic databases [EMBASE, MEDLINE, PsycInfo^®, Cumulative Index to Nursing and Allied Health Literature (CINAHL), British Nursing Index, Applied Social Sciences Index and Abstracts (ASSIA), Education Resources Information Center (ERIC) and British Education Index] from inception to 15 November 2018. To identify potentially eligible articles missed by the electronic search, we hand-searched a list of records retrieved as part of a recent related systematic review27 and contacted key experts in the field.

Eligibility criteria

Studies were included if they met the following criteria: (1) the study had an interventional design; (2) participants were aged 10–19 years at the time of recruitment; (3) all participants were presenting with elevated mental health symptoms or psychological distress; (4) the intervention studied was a psychological intervention (i.e. based on psychological theory as evidenced in a manual or other supporting material) designed to reduce symptoms of an emotional disorder; (5) the intervention studied was delivered wholly or partly within an institution whose primary function was education; (6) the study was conducted in a high-income country (as defined by the World Bank38); and (7) the report included information on barriers to and/or facilitators of the implementation of the intervention.

We included studies with any interventional design, that is any study that involved the implementation of an intervention. Purely observational studies of interventions that were already part of routine practice were excluded. Studies of universal interventions delivered to all pupils were outside the scope of this review. Studies of integrated indicated–universal approaches were eligible for inclusion only if separate findings were reported on the implementation of the indicated component. The focus of this review was on the implementation of interventions in schools and (sixth form/further education) colleges; therefore, studies of interventions delivered within universities or other higher education institutions were excluded.

Study selection

Study selection was carried out with the aid of Covidence systematic review software (Melbourne, VIC, Australia). After duplicate records were removed, the titles and abstracts of all articles identified by the literature search were independently reviewed by two reviewers. All disagreements between reviewers were discussed as a team and consensus decisions were reached. The full texts of articles deemed potentially relevant were obtained and assessed for eligibility by two researchers. Reviewers assessed eligibility independently and all disagreements regarding eligibility or conflicts in criteria for exclusion recorded were discussed by the two reviewers concerned and, if consensus not reached, resolved by a third reviewer. A flow diagram of the selection process was maintained as per Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidance. 39

Data extraction

Data were independently extracted by two reviewers and cross-checked to ensure accuracy. Contextual information was recorded using a piloted data extraction spreadsheet. The following information regarding the study sample was recorded: lower and upper age, gender (percentage female) and criterion for elevated mental health symptoms or psychological distress. In addition, the following information about the intervention was recorded: name, brief description, planned contact hours, whether parents or carers were involved and whether the intervention was delivered by staff members internal to the school or external facilitators. Included articles were imported into NVivo software version 12 (QSR International, Warrington, UK) where barriers and facilitators were synthesised as described below. Post hoc, we extracted details of the participant identification and referral processes employed in each study.

Data synthesis

A thematic synthesis40 of factors reported to affect intervention implementation was conducted with the aid of NVivo (version 12) qualitative data analysis software. Included papers were imported into NVivo and sections of the text describing barriers to or facilitators of implementation, including both quoted original data and author interpretation, were coded using an inductively developed coding structure. Coding of all included studies was completed by Briony Gee and independently by one other member of the review team. Discrepancies in coding were discussed by the two reviewers concerned and consensus interpretations were researched in all cases.

Codes generated inductively were first organised into descriptive themes that aimed to summarise the barriers and facilitators reported, staying close to the primary studies. The next stage of the analysis involved developing analytic themes by structuring and interpreting the descriptive themes according to the selected theoretical framework. This stage of the analysis aimed to ‘go beyond’ describing the findings of the included studies to generate new understandings of the factors influencing successful implementation of indicated school-based mental health interventions. A suitable framework was selected a posteriori with the aid of Nilsen’s taxonomy of implementation science theories, models and frameworks. 41 The theoretical framework was selected only after the generation of initial descriptive themes to ensure that, as far as possible, the analytic themes developed were data driven rather than reflecting the review team’s prior assumptions.

Durlak and Dupre’s ecological framework for effective implementation42 is premised on the view that a multilevel ecological perspective is necessary for understanding successful implementation. It is a determinant framework41 that aims to understand influences on implementation outcomes by specifying individual, organisational and community-level factors that act as implementation barriers and enablers. This framework was deemed to be an appropriate organising concept for the current review because schools and colleges are dynamic and complex social organisations, and thus the implementation of new practices within them is influenced by factors on multiple interacting levels. 43

A sensitivity analysis was conducted in which only studies of interventions found to be effective were retained. Studies in which there was no evidence that the intervention was effective in improving the primary outcome or that did not report a group-based statistical analysis of intervention effectiveness were removed from the thematic synthesis to explore whether or not implementation issues differed by study outcome.

Characteristics of included studies

Our electronic searches returned 2559 unique study records. In addition, 10 studies were identified through hand-searching and correspondence with experts. The study selection process is illustrated in Figure 1.

FIGURE 1.

The PRISMA diagram illustrating study selection process.

We identified 50 unique papers that met the inclusion criteria (Table 1).

Included studies were published over a 20-year period between 1998 and 2018. Most included studies were of indicated interventions for young people with symptoms of depression (n = 17), anxiety (n = 16), post-traumatic stress disorder (n = 5) or either depression or anxiety symptoms (n = 3).

The majority of studies were of interventions described as cognitive behavioural therapy (CBT) or CBT based (n = 35). Half the included studies (n = 25) were of interventions delivered by an external facilitator, 11 were delivered by an internal school-based staff member and 10 by both internal and external personnel. The remaining studies (n = 4) did not report whether those delivering the intervention were internal or external to the school.

Although studies of interventions delivered within sixth form and further education colleges were eligible for inclusion, no such studies were identified. As all included studies were of interventions delivered within schools, the results of the thematic synthesis below are specific to school-based interventions.

Thematic synthesis

Eleven analytic themes were developed (Figure 2): two related to intervention characteristics (acceptability, practicality); three related to organisational capacity, that is practices, processes and culture of the structures through which the intervention is implemented (relationships between intervention facilitators and school staff, support of school leadership and school environment); two related to training and technical assistance (quality of training and ongoing supervision, and suitability of the intervention manual and other materials); two provider characteristics (ability of staff to deliver the intervention successfully and ability of staff to identify eligible students); and two community-level factors (stigma and mental health literacy, and priorities of health and education systems).

FIGURE 2.

Diagram of factors reported to influence the successful implementation of school-based indicated interventions for adolescents with symptoms of an emotional disorder.

The number of the included studies that contributed to each analytic theme is given in brackets next to the name of each theme below. Quotations from primary papers to be presented alongside the findings were selected based on how clearly they exemplified the themes.

Intervention characteristics

Organisational capacity

Training and technical assistance

Provider characteristics

Community factors

Sensitivity analysis

Themes remained broadly similar when studies in which there was no evidence of effect on the primary outcome or that did not report statistical analysis of intervention effectiveness were removed. There was a change of more than 5% in the percentage of included studies that contributed to two of the themes: practicality was reported as affecting the implementation of fewer of the interventions found to be effective than the complete set of included interventions, and quality of training and ongoing supervision was reported as a facilitator of implementation by only the subset of studies of interventions found to be effective.

Limitations

Although studies of interventions delivered within sixth form and further education colleges were eligible for inclusion, no such studies were identified. Therefore, we are unable to reach any conclusions about how to deliver mental health support in such colleges. In the UK, colleges educate and train more than two million people each year, and over two-thirds of all 16- to 18-year-olds are enrolled at a college. 102 There are substantial differences between schools and colleges that are likely to have an impact on the implementation of mental health interventions. For instance, colleges tend to be less formal environments than schools with less structured timetables and greater student independence. Therefore, there is a need for further research on the delivery of mental health interventions within this context to inform UK policy.

The scope of the current review was limited to studies conducted in high-income countries. This was necessary to facilitate meaningful synthesis as the factors affecting the implementation of interventions in low-resource contexts are likely to differ in important ways from the factors that affect the implementation of similar interventions in contexts in which greater resources are available. However, there are promising school-based mental health interventions delivered in low- and middle-income countries17 and understanding the factors that have an impact on the implementation of these interventions in these contexts is undoubtedly important.

The sensitivity analysis conducted post hoc was intended to provide an indication of whether or not the implementation barriers and facilitators reported differed according to the effectiveness of the interventions concerned. The results of this analysis indicate that the inclusion of studies of interventions not found to be effective did not have a substantial impact on the themes identified. However, there are several factors that complicate the interpretation of this analysis, including the use of inconsistent definitions of effectiveness across studies and the lack of systematic measurement and reporting of barriers and facilitators. Therefore, it is not possible to draw conclusions regarding whether or not implementation barriers and facilitators actually differed or to infer that the presence or absence of a particular factor is linked to effectiveness.

The findings of this review must be interpreted with some caution owing to the quality of the evidence regarding implementation synthesised. Although we did not formally assess the quality of included studies, because this would not necessarily relate to the quality of the information on implementation, we noted that most coded sections of the text describing barriers to or facilitators of implementation were from author interpretation rather than objectively collected process data. Implementation is a topic that has received relatively scant attention in comparison with effectiveness and, for this reason, it was rarely a primary focus of eligible studies. As a result, implementation factors were often captured informally and therefore the data lacked richness. Future research should employ formal process evaluation and implementation science designs. It has been argued that one of the most critical issues in mental health services research is the gap between what is known about effective treatment and what is provided in routine care. 35 If this gap is to be bridged, it is important that researchers give increased attention to factors affecting implementation and design studies accordingly, incorporating process evaluation and implementation science approaches.

Implications

The findings of this review have important implications for those with a role in planning and implementing school-based mental health initiatives (Box 1). Recent UK policy proposals23 include the creation of new MHSTs based within schools and colleges and the introduction of designated senior leads for mental health in each setting. MHSTs will offer direct support to young people experiencing mild to moderate mental health difficulties, supervised by NHS mental health professionals. There is the potential for this model to offer an effective solution to the tension between fidelity and sustainability highlighted by this review; learning from the current evidence will be important to realising this potential.

The findings of this review indicate the need to ensure that the curriculum for the workforce who will be trained to work as part of the new MHSTs is designed with input from young people and education professionals. This will help to ensure that the interventions this new workforce are trained to deliver are acceptable to young people and can practicably be delivered in educational settings. Interventions are more likely to be implemented successfully if they are well structured, manualised and delivered by staff who receive high-quality training and supervision.

Designated senior leads for mental health will be well placed to encourage genuine and committed ‘buy-in’ from all aspects of the system, including senior leaders, governors, teaching staff and parents/carers. However, changing whole-school culture is no small task. It will be important that leads are appropriately supported to fulfil this role. This might include the creation of forums for designated senior leads to share good practice, and the co-production of a school and college mental health charter to support cultural change.

There is a danger that the creation of new school-based services will add further complexity to a system that is already fragmented and which in turn could lead to the creation of more treatment silos. 103 We must avoid this and instead use these developments as an opportunity for greater joint working and system alignment.

Background

In Norfolk, following Future in Mind recommendations,22 child and adolescent mental health service (CAMHS) transformation and the development of local transformation plans (LTPs) encouraged services to develop innovative practices. Norfolk LTP priority areas included responding to the rise in reported self-harm and suicidal acts in young people and improving access to mental health support. It was recognised that mental health services for children and young people must be designed around their needs. Therefore, system leaders have been working to develop strong relationships between NHS mental health services and educational settings to facilitate improved collaboration.

The Norfolk Youth Service is a pragmatic, assertive and ‘youth-friendly’ service for young people aged 14–25 years that transcends traditional service boundaries between CAMHS and adult services. The service was developed in 2012 in collaboration with young people and partnership agencies and is based on an engaging and inclusive ethos. The service is recovery oriented and evidence based and aims to satisfy recent policy commitments to extend mental health provision for young people to those up to the age of 25 years. The challenges of engaging and supporting young people with early symptoms of BPD has been a key area of work for the Norfolk Youth Service. This work has been ongoing since the development of the youth service and has resulted in several innovative approaches to working with early symptoms of BPD in adolescence. This work has included collaboration with, and learning from, specialist services worldwide, including the Orygen (Orygen Youth Health, Parkville, VIC, Australia) service.

Part of the work of the Norfolk Youth Service has been to produce and deliver a treatment package for young people with BPD symptoms that distils fundamental elements of evidence-based interventions for adolescent BPD into a brief (three to six sessions) practicable format. This package was designed to be delivered within secondary mental health services. However, because of long waiting times and high thresholds for treatment as a result of increasing demand for mental health support, the team recognised that many young people with BPD symptoms did not access secondary services until they had reached crisis point. Therefore, the need to adapt the treatment package to enable it to be delivered outside specialist services to facilitate earlier intervention was identified.

Theoretical underpinnings

The BEST intervention was developed to address issues apparent in the delivery of evidence-based interventions for adolescent BPD symptoms, including difficulties of access to specialist services that provide such treatments, problems engaging young people in treatment, early treatment drop-out and lack of resources available to deliver lengthy interventions. Therefore, the intervention takes account of issues specific to engaging with an adolescent population and providing interventions tailored to this group.

The development of the intervention utilised knowledge from attachment theory, which identifies how patterns of relating are established in the context of early attachment relationships. In-depth psychological therapies that aim to identify unhelpful patterns of relating to others and work to establish new healthier patterns of relating (such as MBT-A or CAT) have been demonstrated as effective with young people. 1 However, as previously mentioned, there are also many difficulties in engaging young people in this form of treatment early enough and for a sustained period.

The BEST intervention draws from relational elements of attachment theory to support the young person to identify unhelpful patterns of relating and to work towards the development of more helpful strategies. This is supported through enhancing the relationship with an identified member of staff at the young person’s school or college, thus nurturing relationships that are already established and part of the young person’s everyday life, eliminating the need for additional specialist support from sources external to the young person’s current support network.

Drawing from MBT-A,2 the BEST intervention recognises that adolescents with BPD symptoms are those most vulnerable to mentalisation failure. Mentalisation refers to the ability to make sense of the subjective states and mental processes of self and others. It is known that a decrease in the ability to mentalise leads to an increase in emotional arousal. The initial phase of MBT-A involves formulation and crisis planning. The BEST intervention mirrors this phase and aims to develop a shared understanding of the presenting difficulties, identify difficulties of mentalisation and develop a crisis plan for managing periods of distress. The staff training element of the intervention aims to increase staff’s ability to mentalise during incidents of distress or conflict, thus supporting the young person to restore their own ability to mentalise. A recent systematic review of MBT interventions for children and families confirmed that these interventions support mentalising and reflective functioning skills. 104

The BEST intervention also incorporates elements from DBT-A. 4 This approach aims to support young people to achieve behavioural control and stabilisation through promoting the understanding of symptoms, the development of positive coping strategies and crisis planning. The BEST intervention makes use of resources for developing positive self-care coping strategies delivered within DBT-A.

Furthermore, the BEST intervention draws on theory from developmental psychology and neurodevelopmental research. Findings from neurodevelopmental research5,6 have informed current understanding of changes in emotional regulation and social cognition during adolescence. The intervention uses this evidence as the basis for educating professionals and young people about the difficulties they are experiencing, as well as to inform the structure and content of sessions.

Outline of prototype intervention

The BEST intervention is a brief, manualised treatment package designed to be co-delivered by a mental health professional working together with a member of staff from the young person’s school or college. Therefore, the intervention addresses several challenges faced by young people experiencing BPD symptoms and those working to support the young people. The approach is designed to:

• tackle the confusion and anxiety experienced by the young person experiencing BPD symptoms by providing psychoeducation and strategies for the self-management of symptoms

• contain the anxiety often experienced by educational staff supporting a young person with BPD symptoms by increasing their understanding of these symptoms and empowering them with tools to offer effective support

• respond to evidence of the need for treatment strategies to focus on early intervention by providing support that, while drawing from existing evidence-based interventions, is delivered in a format that can be implemented consistently by staff without specialist training.

The BEST treatment package is delivered over up to six sessions lasting approximately 1 hour each, over a treatment window of 8 weeks. The sessions cover three manualised components delivered over the six sessions, supported by a resource pack (see www.journalslibrary.nihr.ac.uk/programmes/hsdr/170931/#/documentation; accessed 6 April 2022).

The first component of the intervention focuses on education about emotional instability: how it relates to early features of BPD, why it can happen and what helps with managing it. This component also looks in detail at typical early features of BPD and allows the young person to reflect on which of these symptoms are relevant to them and the ways in which they are affected by them. The key message in this component is one of education to reduce confusion and anxiety about distressing symptoms. This component is delivered by working through a psychoeducation leaflet about emotional instability and early features of BPD. This leaflet can then be taken away by the young person to be discussed further at the following session, with their reflections.

The second component of the intervention incorporates co-development of a maintenance cycle to help the young person understand what factors are maintaining the current difficulties, and thus identify areas for change. Feedback received from service users has indicated that they do not want to receive interventions that give the impression of ‘box ticking’ and that are not tailored to their unique individual experience. The formulation is used to validate the experience of the individual and provide a framework for the intervention, increasing its meaning and purpose for the young person. This individualised approach aims to encourage engagement and motivation.

The third component builds on areas for change identified in the development of the maintenance cycle. This incorporates the co-development of a crisis plan to support with managing periods of distress. Crisis plans will focus on the development and use of self-care strategies to support emotional regulation. This component introduces self-care distress tolerance strategies, including techniques for sensory self-soothing, grounding and distraction. Introduction of these strategies will be supported by completion of worksheets, which the young person can take away to support ongoing practice. Crisis plans will also incorporate the development of appropriate pathways for accessing additional support when needed to support the young person with managing their distress in a helpful way and to develop positive help-seeking behaviours.

The content of each of the six sessions is summarised in Table 2.

Co-delivery

Every session is co-delivered by a trained member of pastoral staff from the young person’s school or college (referred to during the project and in this report as education practitioners) and a clinician working as part of local NHS-funded mental health services (referred to as mental health practitioners) working together. The role of the mental health practitioner is to support the education practitioner in maintaining adherence to the intervention and in monitoring and managing risk issues. The education practitioner was an ongoing point of contact for the young person between treatment sessions.

Co-delivery of the BEST intervention allows treatment to be delivered within a setting that is accessible to the young person and removes the need to access specialist services. As well as the benefits of a setting known to the young person, co-delivery also means that the intervention utilises an ongoing relationship that the young person has with school staff. Schools and college staff currently feel inadequately supported to meet the needs of pupils with mental health problems. 28 Leaving school and college staff to provide support to pupils experiencing BPD symptoms without professional advice is likely to result in suboptimal outcomes for young people and may build resistance to later interventions for those who go on to meet the thresholds for specialist CAMHS. Co-delivery of the intervention was intended to contain anxiety experienced by educational staff by reducing the confusion and anxiety that often surrounds young people with BPD who self-harm, increasing their understanding and empowering them with tools, knowledge and skills to offer effective support.

All staff (both education and mental health practitioners) attended a 1-day training workshop prior to delivering the intervention with young people. The purpose of the workshop was to prepare practitioners to co-deliver sessions and, in the case of education practitioners, provide ongoing support to participants from their institution. The 6-hour workshop introduced relevant theory, covered the practicalities of delivering sessions and equipped practitioners with skills to enhance their ability to mentalise during incidents of distress or conflict.

Supervision

Supervision for both education and mental health practitioners delivering the intervention was provided by supervisors trained in the delivery of MBT-A who had extensive experience of working with young people with early features of BPD within a CAMHS setting. Supervision was used to support the use of a mentalising approach within sessions and to support adherence to the treatment package. Supervision was provided in group format at least fortnightly during the intervention phase. At the end of each session, the co-facilitators rated their adherence to the intervention using the fidelity checklist and completed session notes, which were reviewed by the supervisor.

Design

The prototype BEST intervention was piloted across three educational settings. This phase of the study was non-randomised; all referred participants who met the study eligibility criteria were offered the intervention. Process evaluation methods were used to identify potential barriers to successful delivery, enabling the team to refine the intervention and study procedures in preparation for the next phase of the study.

Setting

Three educational settings, two further education colleges and one secondary school, were selected to take part in the pilot phase of the study. Settings were selected from among those whose staff had expressed an interest in participating during the PPI work undertaken to inform protocol development. In selecting settings to participate in the pilot, we aimed to maximise variation in type of setting (school or college), geographical area (urban or rural) and levels of deprivation among the community served.

Participants

We aimed to recruit a convenience sample of two young people from each of the three pilot settings. Young people who met the following criteria were eligible to participate.

Recruitment procedure

Potentially eligible students were identified through liaison with school and college pastoral staff teams. Pastoral staff members were asked to consider the young people in their school and college and approach those that they believed might meet the study’s eligibility criteria to ask whether or not they would be interested in taking part.

Verbal consent (of the young person and, for those aged under 16 years, their parent or carer) to be contacted by the research team and participate in the screening was recorded using an expression of interest form. Young people (and their parent or carer if they were under 16 years of age) who expressed an interest were sent a copy of the participant information sheets and contacted by a research assistant who explained the study in more detail and answered any questions. If, following this conversation, the young person remained interested in participating (and with the verbal consent of their parent or carer if under 16 years of age), then the researcher arranged to meet with the young person at a convenient venue (e.g. their home address or the school/college) to complete the screening process.

The screening process involved the young person being asked to complete two self-report measures, the BPFSC and the self-harm subscale of the RTSHI-A, and being asked by the research assistant about any treatment for mental health problems they were currently receiving or have received in the past. If this screen indicated that the young person would likely meet the study’s eligibility criteria, then written informed consent in the case of participants aged ≥ 16 years, or written informed assent and parental consent in the case of participants aged under 16 years was then sought. This included consent for the video-/audio-recording of sessions and use of both written quotations and anonymised audio-/video-clips of session recordings.

Staff participants

Staff at each of the three settings were asked to identify one or more members of staff willing to be trained to co-deliver the intervention to participants from their school or college. Mental health practitioners were recruited to participate in the pilot through an e-mail sent to all local CAMHS/Youth Service teams inviting expressions of interest. Those practitioners invited to take part from among the expressions of interest received were selected to maximise variation in professional background and banding. This was to enable us to begin to explore the impact of prior training and level of experience on the ability to deliver the BEST intervention as planned.

All potential staff participants were provided with a copy of the staff participant information sheet and had the opportunity to have their questions answered before being asked to complete a consent form to document their informed consent to participate in the study.

Data collection

Demographic data were obtained from participants for the purpose of describing the sample. Site profile questionnaires were completed by a member of staff for each setting to capture information on the composition and training of the pastoral team, the level of demand for mental health support, and the protocols and procedures for supporting pupils with mental health needs.

After the screening measures had been administered and informed consent obtained, the research assistant completed the remaining baseline assessment measures with the participant. These were the Difficulties in Emotion Regulation Scale (DERS),105 the Childhood and Adolescent Social Support Scale (CASSS),106 the Time Use Survey (TUS),107 the Childhood Interview for Borderline Personality Disorder (CI-BPD)108 and the psychosis, alcohol and substance abuse modules of the Kiddie Schedule for Affective Disorders and Schizophrenia (K-SADS). 109 The purpose of administering these measures was to enable eligibility to be established and to allow for any issues related to the acceptability or practicality of the assessment battery to be identified at an early stage.

With the participants’ permission, treatment sessions were either audio-recorded or video-recorded (according to the preference of the participant) to facilitate the process evaluation. Following completion of the pilot cases, focus groups and individual interviews were conducted to gather data on staff participants’ experience of delivering the intervention and their ideas about how the intervention and research processes should be modified to facilitate successful implementation in the next stage of the research.

Data analysis

School and college characteristics

Three educational settings acted as sites for the pilot phase: they will be referred to in this report as site A, site B and site C.

Site A is a large further education college, and the main campus is located close to the city centre. The college had approximately 4000 students enrolled on its courses for young people aged 16–19 years, as well as a large number of adult learners and apprentices. The college’s overall Ofsted (Office for Standards in Education, Children’s Services and Skills, London, UK) rating at the time of the pilot was ‘Good’.

Site B is a large further education college with provision spread over two campuses, both located in coastal towns. The college had approximately 2500 enrolled on its courses for young people aged 16–19 years, in addition to a smaller number of adult learners and apprentices. The college serves coastal and rural communities with high levels of deprivation. At the time of the pilot the college had an overall Ofsted rating of ‘Requires Improvement’.

Site C is a secondary school for children aged 11–16 years located in a small market town, serving the town and surrounding villages. The school had approximately 1200 students enrolled. At the time of the pilot the school had recently opened as an academy; however, the preceding school of the same name received an overall rating of ‘Good’ at its last inspection.

Responses to the site profile questions completed by staff at each site are summarised in Table 3.

Recruitment

Recruitment of young people began following confirmation of local capacity and capability on 2 January 2019. The first participant was consented on 21 January 2019. Five participants were recruited to the pilot. We planned to recruit two participants from each of the pilot sites: this target was met at sites A and B, but we were unable to identify a second eligible participant from site C within the pilot’s recruitment window. Two young people referred by site C did not meet the eligibility criteria: one scored > 34 on the BPFCS and one was 13 years old but was in school Year 8.

Participant characteristics

The characteristics of the five young people who consented to participate in the pilot phase are summarised in Table 4.

Staff characteristics

Eleven staff members were consented to participate in the pilot and were trained to deliver the intervention. Five staff participants were school or college staff members: two from site A employed as mental health advisors, two from site B employed as safeguarding co-ordinators and one from site C employed as a school counsellor. Six staff participants were mental health professionals, employed in a range of clinical roles with Agenda for Change (AfC) bandings ranging from band 4 to band 8a. The characteristics of the staff members allocated to deliver the intervention to the participants recruited are presented in Table 5.

Implementation of the prototype BEST intervention

Of the five young people recruited, three (P001, P003 and P005) completed the full intervention, one (P004) completed only the first session before disengaging and one (P002) was unable to receive any sessions because they were excluded from the college. No reason for disengagement was given by P004; however, during the session he attended, he expressed scepticism about whether or not he would find the intervention helpful based on his prior experiences of mental health support.

For those who completed the intervention, two received six sessions and one (P003) received seven sessions because the content of session 3 was delivered over two sessions. The mean duration of sessions was 64 minutes; however, session 3 (which focused on psychoeducation regarding the features of emotional instability/symptoms of BPD) took significantly longer to deliver than other sessions (mean duration 87 minutes). Owing to constraints imposed by the school or college timetable, it was generally not possible to extend the length of sessions significantly beyond 1 hour. Therefore, the content of session 3 was completed at the beginning of the next session (P001), divided over two sessions (P003) or left unfinished (P005).

Each session was observed to follow a similar sequence of activities, in line with the structure of the intervention manual. Sessions began with setting an agenda and introducing the topic of the session and ended with rating the session and agreeing a task for the young person to complete over the coming week. Most of each session was spent on activities involving (1) conveying information to the young person, (2) modelling a skill or demonstrating the application of information conveyed and (3) applying the information or skills introduced to the young person’s own experiences.

An example of the completed activity analysis coding completed for all session recordings is included as Figure 3. In this figure, the first row of each block shows the recording timestamp corresponding with each column. The second row denotes the nature and duration of each activity occurring and the colour indicates the individual who led the activity: light blue indicates that the activity was led by the education practitioner, orange that the activity was led by the mental health practitioner, purple that the activity was jointly led by both practitioners and dark blue that the activity was led by the young person. Rows 3–5 represent the conversational turns taken by practitioners and participants (coloured cells) and other actions such as writing and referring to the manual (represented in the figure by letters explained in the key provided). Silences of more than 10 seconds, when these occurred, were presented in the final row (as in for P005).

FIGURE 3.

Sample activity analysis coding. To facilitate comparison, coding relating to the same activity from across cases is presented in a stacked arrangement, therefore discontinuations in the timestamps are present, for example for P001 between 29:30 and 40:00.

Completion of this coding allowed for broad patterns of delivery to be compared across cases and interactional patterns of interest to be identified for further analysis using conversation analysis techniques. For instance, in Figure 3 we see two patterns of delivery of the component of the intervention involving formulation of mentalisation failure. Across all three sessions, we see the activity being introduced and initially led by the education practitioner. The person coded as leading the activity was not necessarily the person who spoke most; for example, for P001, the explanation of mentalisation failure was led by the education practitioner. The activity commenced prior to the 26 minute timestamp, and, although at this point it is the mental health practitioner who is speaking, with contributions from the EP at 27 minutes, the EP continues to lead the activity, seen with utterances that demonstrate steering the activity (e.g. ‘so we’re just reading down on the er on the worksheet’).

As the activity progresses, we start to observe differing patterns of contributions from the young people. In the sessions with P001 and P003 we see consistent verbal contributions are made by the young people throughout the session. In addition, in the session with P003 we observe the young person taking the lead in the formulation activity by telling the story of an incident in their life when their ability to mentalise broke down. In contrast, the session with P005 is primarily characterised by frequent silences throughout the activity.

We, therefore, transcribed the relevant sections of these sessions to enable us to further examine how differing practices and patterns of delivery might lead to differences in the extent of the young person’s active participation and engagement in the session activities. Through this process, we identified patterns of delivery that appeared to be more successful in enabling the session objectives to be met to inform the development of the manual and content of practitioner training.

Acceptability of prototype intervention and pilot research procedures

Changes made to the intervention

Changes made to study procedures

We had initially planned to involve six to eight schools in the feasibility trial. However, our experience of recruitment in the pilot indicated that we may need to involve more schools and colleges to meet our recruitment targets. We therefore planned to increase the number of schools and colleges invited to participate from 6–8 to 12–16 settings. In response to the findings of the evidence synthesis, we approached all schools and colleges we invited to participate in the feasibility RCT via a member of the senior leadership team to ensure support for the study at an appropriately senior level within each setting.

Furthermore, to assist referrers in identifying young people potentially eligible to participate, we produced a brief pre-screening questionnaire for referrers to use to help them decide whether or not a potential referral was likely to be appropriate. Once a referral was received, we conducted the screening, informed consent and assessment process as original planned.

In line with the findings of the evidence synthesis regarding the logistical constraints imposed by the school environment, intervention piloting indicated that it was unlikely to be possible to complete the intervention within the 8-week time window originally proposed owing to school holidays and staff and student absences. Therefore, we increased the time frame for intervention delivery to 12 weeks and postponed the post-intervention follow-up to 12 weeks from randomisation (the final follow-up assessment remained at 24 weeks).

In line with feedback received, we updated the fidelity checklist to include further guidance on how to rate each item and added a space for practitioners to provide a brief justification of their rating. Furthermore, a section was added to allow staff delivering the intervention to record a session summary to be shared with the supervisor, including a prompt to note any risk issues and record actions taken to address any issues identified. To assess the extent of staff learning and development following training as a BEST practitioner as part of the process evaluation, we intended to administer an additional questionnaire assessing staff knowledge, skills and confidence pre training and after having co-delivered the intervention.

Inclusion criteria

• Aged 13–18 years (school years 9–13).

• Enrolled at a participating school/college.

• Score of ≥ 34 on the BPFSC. 112

• History of repeated self-harm assessed using the self-harm subscale of the RTSHI-A113 (has intentionally harmed him/herself more than once).

• Able to provide written informed consent or, for those aged < 16 years, written informed assent and parent/carer consent.

Exclusion criteria

• Currently receiving inpatient treatment or a specific psychological intervention.

• Moderate/severe learning disability.

• Current psychotic disorder (those with subthreshold psychotic symptoms were not to be excluded) or substance dependence (current substance abuse was not an exclusion criterion) requiring care planned treatment.

Potentially eligible young people were identified through liaison between the research team and staff from participating schools and colleges or local mental health services. Referrers were asked to consider the young people they worked with and approach those they believed might meet the study’s eligibility criteria to ask if they would be interested in finding out more about the study. Potential referrers were provided with an optional pre-screening questionnaire to complete with young people who expressed an interest to help determine whether or not a referral to the trial would be appropriate. The agreement of the young person (and a parent/carer for those aged < 16 years) to be contacted by the research team and participate in the screening was recorded using an expression of interest form.

Following a referral being made, young people (and their parent/carer if aged < 16 years) were contacted by a member of the research team who explained the study, answered any questions and provided copies of the approved study information sheets. If, following this, the young person remained interested in participating, the researcher arranged to meet with the young person at either their school/college or their home address to screen him or her for eligibility.

The screening process involved completing the BPFSC and the self-harm subscale of the RTSHI-A, and confirming any treatment for mental health problems they had received in the past or were currently receiving. If this screen indicated that the young person was likely to meet the study’s eligibility criteria, written informed consent (in the case of participants aged ≥ 16 years) or written informed assent and parental consent (for participants aged < 16 years) was sought.

After informed consent has been obtained, participants were asked to complete the remaining baseline assessment measures. Only young people who met all eligibility criteria outlined above were randomised. In recognition of their time and commitment, participants were given a £10 shopping voucher for each research assessment they completed (up to £30 in total across the three assessment points).

Outcome measures

To assess the suitability and acceptability of the proposed outcome measures for a definitive RCT, participants were assessed pre randomisation (baseline) and at 12 and 24 weeks post randomisation with the following measures: BPFSC112 DERS,105 RTSHI-A (self-harm subscale),113 CASSS106 and TUS. 107 Data on school/college attendance and exclusions were also requested from the school or college with the participant’s consent. We also sought consent from participants (and their parent/carer if aged < 16 years) to access the information recorded about them on the National Pupil Database (NPD)116 to assess the feasibility of utilising these data in future research.

To enable us to describe the sample in terms of the number of eligible participants who meet research diagnostic criteria for BPD, we administered the CI-BPD108 at baseline only. We also administered the psychosis and substance abuse modules of the K-SADS109 at baseline only , to assist in determining whether or not a potential participant met the eligibility criteria. The scheule of enrolment, interventions and assessments in accordance with SPIRIT (Standard Protocol Items: Recommendations for Interventional Trials) guidelines is shown in Table 6.

Health economic data

We recorded all resources required to implement the intervention, including providing training and supervision, staff time to deliver the intervention, consumables and materials required, and any other necessary expenditure. A modified version of the Client Service Receipt Inventory (CSRI)117 was administered at each assessment time point (baseline, 12 weeks and 24 weeks) to evaluate whether or not this measure was an appropriate tool for assessing use of health, pastoral and social care services in this group.

The CSRI was modified in accordance with discussions with the chief investigator to ensure that the instrument was relevant to individuals with BPD and to minimise the burden on individuals (see the Journals Library web page at www.journalslibrary.nihr.ac.uk/programmes/hsdr/170931/#/; accessed 6 April 2022). The CSRI collected data on several potential health and social care items of resource use. These included contacts in a person’s school or college, medicines, inpatient stays, outpatient and A&E visits, community-based health and social care contacts, overnight stays, and any health-care use by the participant’s family that was potentially connected to the participant’s BPD. The EuroQol-5 Dimensions, five-level version (EQ-5D-5L), was administered at all research protocol data collection time points to allow for its usefulness and likely sensitivity to adolescents with BPD symptoms to be assessed. Both the CSRI and the EQ-5D-5L were collected alongside other instruments using face-to-face interviews at baseline and face-to-face or telephone interviews at follow-up.

Process evaluation data

The process evaluation employed an ethnographic methodology118,119 to investigate how the wider context of schools and colleges shaped delivery of the intervention. A mixed-methods approach to analysing data was carried out, including site profile questionnaires; video-/audio-recordings of intervention sessions; staff log sheets to record contacts between participants and trained BEST practitioners; observational field notes of training workshops; workshop feedback forms; questionnaires assessing staff attitudes, skills and knowledge regarding work with young people with BPD symptoms; interviews with young people; and focus groups with staff (education and mental health practitioners) delivering the intervention.

All 45 available video-/audio-recordings were collected to describe intervention delivery and to assess implementation fidelity according to the study protocol. All young people participating in the RCT (from across both arms) were invited to take part in semistructured interviews. Young people who had consented to take part in a qualitative interview were approached about being interviewed by the research assistant by telephone following the 24-week assessment. Eleven interviews were conducted to understand participants’ perspectives on the acceptability of participating in the study. In the intervention arm this included eliciting their views on different intervention sessions and co-delivery with the educational and mental health practitioner; how and whether the intervention functioned to improve their understanding and skills for coping with mental health difficulties; perceived benefits or harms of taking part; completing outcome questionnaires; and asking for suggestions for improving intervention delivery. Interviews with young people allocated to the usual-care arm focused on forms of support obtained during the trial and views on completing questionnaires.

Six focus groups with a total of 19 staff members (nine education practitioners and 10 mental health practitioners) and one individual interview were conducted to obtain practitioner perspectives on delivering the intervention, contextual barriers to and facilitators of successful delivery, intervention sessions, potential contamination in the provision of usual support and additional work required to support the delivery of BEST, and to ask for suggestions for improvement.

At the conclusion of the study, a stakeholder feedback event was held to share key study findings and gather stakeholder views on the likely sustainability of intervention implementation and the future of the research programme.

Statistical analysis

We initially intended that recruitment and retention rates would be estimated along with 95% confidence intervals and, if appropriate, time until dropout would be estimated using a reverse Kaplan–Meier curve. Analysis was based on the intention-to-treat principle, analysing each randomised patient in the arm they were allocated to regardless of compliance. Owing to the small sample size, formal hypothesis testing was not planned; instead, we intended that the analysis would focus on estimation. We planned that for the primary outcome measure, the BPFSC, the mean difference would be estimated using a linear model along with corresponding 95% confidence intervals. The rate of completion of each outcome measure would be given and, if appropriate, multiple imputation would be undertaken. A similar approach was planned for the secondary outcome measures.

The above plan was amended, prior to data analysis, owing to the early closure of the study with reduced sample size. Hence, the study team felt that the estimation plan above was too ambitious given the limited sample size, and that the analysis should report on the summary statistics of the relevant outcomes along with the completion rate and retention rate at each time point. Analysis was undertaken in Stata^® (StataCorp LP, College Station, TX, USA).

Health economic analysis

As this was a feasibility study, it was not our intention to demonstrate the definitive cost-effectiveness of the intervention as part of this trial. This is because the study was not powered to demonstrate either clinical effectiveness or cost-effectiveness. This was further compounded by the need to stop the study early. However, we collected information to inform the economic evaluation alongside any future definitive trial. This will yield useful information, such as the likely cost of the intervention and key components of resource use. It will also inform the design of a future study by informing the best design of health economic data collection instruments. The focus in this evaluation was on three key pieces of information necessary to conduct an economic evaluation in any future study: resources required to provide the intervention, health and social care service use and health-related quality of life.

First, we collected data on the resources required to provide the intervention in this study. This included the costs associated with providing the two workshops required to train those providing the intervention. Resources used would relate to the time of those providing training and of those receiving it. We did not record time spent developing the training workshops. For the cost of providing intervention sessions, we recorded the number of sessions received by participants, the AfC band of the mental health practitioner providing the session and estimates of duration of sessions obtained from the pilot work. We were also able to estimate distance travelled to provide the intervention. We did not have a record of the type or grade of school/college staff so estimates of type staff were made based on those observed in the pilot study. The final element costed related to supervision received by those providing the intervention. Details were kept of the number of supervision sessions provided, the AfC band of those providing the supervision and the distance travelled to provide this supervision. Resources required to provide the intervention were combined with appropriate unit cost data to provide an estimate of the cost of providing the BEST intervention. Unit costs were obtained from a published source and relate to the cost year 2018/19. 120 As these costs were derived from a small sample, they were not reflective of costs applying to a large trial or to routine clinical practice. For this reason, we will conduct scenario analyses, which estimate the cost of provision if any assumptions about how the service is provided and who provides it are changed.

Second, we looked at health and social care resources used by the two study groups. This was carried out in three time periods: the 12 weeks prior to randomisation; the first 12 weeks of follow-up, when the intervention would be provided; and the final 12 weeks of follow-up. Owing to the small sample size, a formal costing of the service use by individuals was not undertaken; instead, analysis will focus on a description of the type of services used by individuals. In addition, we assessed the performance of the CSRI in terms of completeness. A key question would be whether the CSRI could be used in future studies or alternative methods of collecting resource use data would be needed.

Finally, we used the EQ-5D-5L121 to investigate health-related quality of life, to test whether or not this might be a suitable instrument to use in this population. The EQ-5D-5L was scored using the ‘crosswalk values’. 122 Again, this was limited by the small sample size in the study, particularly in follow-up periods. However, it was possible to look at response characteristics and correlations with other measures for the baseline sample.

Process evaluation analysis

Linguistic ethnography provides analytical tools for systematically analysing talk and behaviour within context. In process evaluations it can be used to search for how the implementation of the intervention ‘disrupts’ the complex system into which it is being introduced,123 exposing wider social forces structuring intervention delivery at the point of delivery: relationships that are otherwise hidden from view. Observational methods including using interactional data are central to a linguistic ethnography approach, and are often supported by interview data to triangulate perspectives with delivery in practice.

All recordings of individual sessions were independently rated against the fidelity checklist by members of the study team and subject to qualitative activity analysis using the same approach employed in the pilot study (see Chapter 3, Pilot methods) to identify patterns of successful delivery; interactional difficulties between staff and pupils in negotiating the content of the treatment sessions; and analysis of how the theories of adolescent development, attachment and mentalisation that underpin the BEST intervention are enacted within sessions.

Site profile questionnaires were completed by a member of staff from each school and college to enable us to identify characteristics of the participating settings that may have an impact on delivery. Practitioners were asked to complete log sheets of their contacts with participants to provide an audit trail of activities surrounding treatment sessions that either facilitate or constrain delivery and to allow us to assess the extent to which control participants may be receiving elements of the intervention.

All interviews and focus groups were transcribed verbatim and thematically analysed with the aid of NVivo software. The approach to the thematic analysis was informed by a critical realist stance, which holds that, although reality exists independently of our observation of it, our understanding of reality is filtered through our own past experiences and our particular social, cultural and historical position. Therefore, each individual occupies a unique vantage point from which they experience and interpret the world. Therefore, we sought to understand each participant’s individual experience of participation in BEST through close engagement with their language use.

The thematic analysis took an inductive approach, generating codes and themes based on the words used by participants themselves rather than an a priori framework. Coding was initially undertaken independently by authors Briony Gee and Jamie Murdoch, who then met to discuss the codes generated and develop provisional themes. Following this, Briony Gee revisited the data to ensure that each theme had sufficient empirical support, removing and collapsing themes where necessary, before defining each theme and naming it using a short quotation from the data set. Further verbatim quotations to be included in the presentation of the analysis were selected based on their suitability as stand-alone illustrations of each theme.

Although neither Briony Gee nor Jamie Murdoch was involved in the initial development of the BEST intervention, both played a role in its refinement in preparation for the feasibility RCT. Therefore, reflection on the impact that our investment in the success of the intervention had on our interpretation of participant experiences was an important part of the analytical process.

The analysis of qualitative data was iterative, moving between data collection and data analysis to test emerging theories, working laterally across data types. Care was taken to identify and follow up deviant cases that did not fit into emerging theories. By setting the analysis of intervention delivery within an understanding of the wider context of school and college environments, we were able to make the transition from the identification of routines and patterns of the use of BEST in specific schools and colleges to theoretical explanations of how different structural relations and mechanisms of the intervention organised moments of delivery, which then affected specific outcomes.

In drawing case comparisons across settings, we developed hypotheses about why the intervention was linked to outcomes, which we can test in a future definitive trial. This led us to identify factors that were plausibly and/or consistently related to successful or unsuccessful delivery of the components of the intervention. Emerging theories and the relationship of the data to the conceptual literature underpinning the intervention were discussed and refined at team meetings throughout the research.

Progression criteria

The primary output of the research was to be the design of a subsequent definitive trial. We planned to assess the outcomes of the feasibility trial against the following criteria to make recommendations regarding progression:

• Recruitment rate is within 70% of the target.

• At least 70% of those randomised to receive the intervention attended three or more treatment sessions within the 12-week treatment window.

• Follow-up assessments were completed by at least 75% of participants at 12 weeks and 70% of participants at 24 weeks.

• Contamination of the control arm was sufficiently limited for individual randomisation to be justified (informed by process evaluation findings).

The design of subsequent research will be informed by the results of the feasibility study, the views of participants in the stakeholder feedback event and Medical Research Council guidelines124 on developing and evaluating complex interventions. If the above progression criteria are met and a definitive RCT is judged to be a suitable and acceptable methodology, we intend to progress to a multisite, assessor-blind, superiority RCT of the effectiveness and cost-effectiveness of the intervention plus TAU in comparison with TAU alone. The proposed primary outcome measure is severity of borderline personality features measured using the BPFSC. 112 The primary analysis of effectiveness would be a multilevel comparison of BPFSC total score at 24 weeks, with treatment arm as the main effect, school as a random intercept and baseline BPFSC total score as a covariate. A definitive cost-effectiveness analysis would be conducted, informed by the results of the preliminary economic analysis conducted alongside the feasibility trial.

If one or more of the above progression criteria are met, we intend to discuss with the TSC whether or not the trial design proposed above is likely to be feasible given steps to overcome specific issues encountered during the feasibility trial. If it is not, alternative study designs will be considered; for instance, a cluster trial or stepped wedge design could be more suitable if the findings of the feasibility trial suggest that contamination of the control arm would threaten the validity of an individually randomised definitive trial. In the case that the results of the feasibility study suggest that substantial changes to the protocol are required prior to progression, we would incorporate an internal pilot with stop–go criteria within any future definitive trial.

Recruitment of schools and college

Schools and colleges were approached about taking part via the research team’s existing contacts or through Educate Norfolk (Norfolk, UK), a local network of secondary school and college leaders. We presented an overview of the study at an Educate Norfolk forum and invited expressions of interest from school and college leaders in attendance.

Staff at 16 of the 21 schools and colleges that received details of the study and requirements for being involved gave their agreement for their school or college to take part. Reasons given for declining participation included that the study would place too great a burden on already stretched school staff members (n = 1) and an upcoming merger creating uncertainty about the structure of pastoral provision for the coming academic year (n = 1). Staff in other settings did not give a reason for declining.

Of the 16 schools and colleges that had staff who were willing to be involved, 12 went on to become sites for the feasibility trial. Staff at one school who had initially agreed to participate were unable to identify a member of staff able to attend one of the training workshops. Staff at a further three schools had confirmed their willingness to participate in the study and identified staff members to be trained to deliver the intervention just prior to the suspension of the trial.

School and college characteristics

An overview of the characteristics of the 12 schools and colleges that were sites is provided in Table 7.

Staff participants

In total, 30 school and college staff members and 21 mental health practitioners gave consent to take part in the feasibility trial, seven of whom had also participated in the pilot phase of the study. However, a proportion of these had been recruited and trained shortly before the suspension of the study as a result of the COVID-19 pandemic and so did not have the opportunity to deliver the intervention.

Practitioners who delivered the intervention to participants in the feasibility trial attended one of two training workshops. Workshop 1 was a full-day workshop attended by most trainees [15 school/college staff members and 15 mental health professionals (8 × band 4, 2 × band 5, 4 × band 6, 1 × band 7). Workshop 2 was a half-day workshop offered primarily as a refresher for those who had already attended training during the pilot phase [two school/college staff members and three mental health professionals (2 × band 4 and 1 × band 7)]. For those practitioners trained during the pilot phase who opted not to repeat the training workshop, a member of the team met with each practitioner individually to review the content of the intervention and provide updated training to reflect changes to the manual since the first phase.

Staff were asked to complete a questionnaire regarding their attitude, skills and knowledge in relation to working with young people experiencing BPD symptoms prior to the training workshop. Responses are summarised in Table 8.

Participant flow

The flow of referrals and participants through the study is illustrated in Figure 4.

FIGURE 4.

Consolidated Standards of Reporting Trials diagram for feasibility trial.

Recruitment

We recruited and randomised 32 eligible participants prior to the suspension of the study. The overall rate of recruitment was slower than anticipated: on average four participants per month versus a target recruitment rate of 6.7 participants per month. The pattern of referrals suggests that staff at sites where more staff were trained as BEST practitioners may have been more able to refer suitable participants than sites where fewer staff were trained as BEST practitioners. Site J, at which four staff members were trained as BEST practitioners (the maximum number trained at other participating sites was two), referred considerably more students than any other setting: 16 students, of whom 12 were eligible.

This slower than anticipated recruitment rate was largely the result of very limited recruitment during school holidays. We had anticipated that we would receive a substantial proportion of our referrals from mental health teams. However, only five referrals (8%) were received from mental health services, with the majority received directly from schools and colleges. Consequently, we received few referrals at the start of the recruitment period, which coincided with the school summer break, and a pause in referrals over the Christmas break.

We encouraged referrals from mental health services through attendance at team meetings, distribution of promotional materials and contact with wait list co-ordinators and the single point of access (the central triaging function). A key barrier to recruitment via mental health teams was that the school or college attended was not routinely enquired about and recorded in young people’s medical records. Therefore, it was not possible for teams to easily identify service users who attended the schools and colleges participating in the trial. Potential referrers from within mental health services also reported that many of the young people on their caseload presenting with BPD features were nor currently engaging in education owing to the severity of their difficulties.

A further issue encountered was with the original inclusion criterion regarding self-harm, which required that self-harm had occurred within the past month. The rationale for including self-harm as an inclusion criterion in addition to BPD symptoms was to increase the predictive specificity for BPD. We also felt that including self-harm as an inclusion criterion may make it easier for school and college staff to identify potentially eligible young people because self-harm might be more visible/readily understood than BPD symptoms more generally.

However, the 1-month time frame selected as the index period for self-harm was not evidence based and was perceived by referrers as unduly restrictive. During the first 2 months of the recruitment period, we observed that help-seeking young people with BPD symptoms were not being referred or were excluded at the screening stage (the first five young people referred were excluded on this basis). We were also concerned about the potential for harm as a result of young people learning that they were not eligible to participate as a result of not having self-harmed recently enough. This criterion was therefore amended to include young people with a history of repeated self-harm regardless of the time period during which this had occurred. This substantial amendment received confirmation of favourable ethics opinion from the Research Ethics Committee on 17 October 2019. The number of young people recruited who would have met the original criterion is reported below.

The recruitment rate was also affected by the suspension of the study from mid-March, meaning that fewer participants were recruited in March that we would otherwise have anticipated. Planned screening assessments with seven participants who had been referred had to be cancelled as a result of the pandemic. The early closure of the study also affected the total number of recruits because the recruitment period was cut short.

From October, when the first participant was recruited, the average recruitment rate was 5.4 participants per month. To reach 70% of the target sample size a recruitment rate of 4.6 participants per month across the initially planned recruitment period would have been sufficient. The rate of recruitment is plotted alongside the projected recruitment rate and the rate of recruitment that would have enabled us to meet our progression criterion related to recruitment in Figure 5.

FIGURE 5.

Accrual curve for feasibility trail.

Retention

Only those participants who reached the 12-week time point prior to the suspension of the trial were followed up, because those randomised later did not have the opportunity to receive their allocated intervention (whether BEST plus TAU or TAU alone) owing to school closures. Of the 19 participants who had reached the 12-week follow-up before the suspension of the trial, 17 (89.5%) were followed up at 12 weeks and 14 (73.7%) at 24-weeks. Follow-up was more challenging at 24 weeks, as all assessments were completed remotely at a time when schools and colleges were closed to most students. As a result, schools and colleges were unable to assist in contacting participants and arranging appointments as they had at the 12-week time point.

Acceptability of recruitment and randomisation

The acceptability of research procedures, including recruitment and randomisation, was explored with practitioners and participants as part of focus groups and interviews.

Rates of completion

Outcome measures were generally well completed. Of those successfully followed up at 12 weeks, 16 out of 17 (94.1%) completed all outcome measures, and of those successfully followed up at 24 weeks, 12 out of 14 (85.7%) completed all outcome measures. A study research assistant facilitated all assessments, and so they were able to check self-report measures for missing items and prompt the participant to complete those accidently missed. As a result, there were very few individual missing items within completed measures; missing item scores were present for only one participant who declined to complete the items missed.

Data on participants’ school or college attendance and exclusions were requested but complete data were received for only 26.3% of those followed up. However, this was requested at the end of the follow-up period when schools and colleges were closed to most pupils during the COVID-19 lockdown. We also sought consent to access the information recorded about participants on the NPD via an optional item on the consent form: 84.2% of participants agreed to these data being accessed by the research team.

Descriptive statistics for outcome measures collected

Descriptive statistics summarising the outcome measures collected are presented in Table 11.

Given that the objective of the study was to assess the feasibility of a future RCT, this study was not powered to detect any significant changes in outcomes. However, we were interested in any indication of whether or not the outcome measures would be likely to be able to detect any change as a result of the intervention in a future trial. Mean changes from baseline by allocated arm are presented for all continuous measures in Table 12, and data on change in self-harm in the past month is summarised in Table 13.

Acceptability of outcome assessments

Participants were asked about their experiences of completing research assessments as part of qualitative interviews. Most young people fed back that they found the process of completing the research assessments acceptable and did not recommend any changes to the outcome measures included in the assessments. Two young people commented that they had found completing the follow-up assessments beneficial as it had offered a chance to review how they were feeling and track their progress.

One young person fed back that the TUS did not feel relevant when she was asked to complete it during the lockdown, as it asked about activities that were not permitted at the time. Another participant commented that he found some questionnaires repetitive.

Only one participant told us that she had experienced discomfort completing the outcome measures, explaining that she had felt annoyed by how personal some of the questions were:

I knew you wouldn’t have to answer some of the questions if you didn’t want to but it’s just the fact that some questions were actually kind of so personal . . . they were kind of annoying.

Young person

When asked if there were any particular questions she found too personal, she replied, ‘the ones like the harming yourself ones and stuff like that’. She explained that she answered these questions, despite understanding that she did not have to answer any questions she was uncomfortable with, because she felt that she could trust the research team owing to their having the trust of the school:

Like it wasn’t way too personal for me to not answer but it was kind of personal but the fact that I knew I could probably trust you . . . ‘cos the school trust you.

Young person

Fidelity of BEST intervention delivery

We encountered some practical and technical difficulties with recording equipment and file transfer, which meant that not all sessions could be reviewed. However, the majority of completed BEST sessions, 45 sessions with 10 participants (including two sessions with a participant who was not followed up owing to not reaching the 12-week time point before the study was halted because of the pandemic), were successfully recorded and available for analysis.

Fidelity of treatment as usual (limiting contamination)

Compliance with the completion of practitioner contact logs was not high enough to offer useful information regarding contamination. However, findings from staff focus groups and interviews with participants allocated to the TAU arm did not suggest that any contamination occurred. Practitioner and participant accounts suggest that the support provided by school and college staff to those in the TAU arm was clearly distinct from the BEST intervention; support was typically provided on an ad hoc basis and centred around active listening, problem-solving and social activities. Participant reports suggest that TAU was less structured and less focused on emotions than the BEST intervention. For instance, one TAU participant reported that the support he was provided at school involved taking part in a ‘nurture’ group. When asked what happened in these group sessions, he replied, ‘we did an escape room and played Monopoly [Hasbro, Inc., Pawtucket, RI, USA]’. When asked whether he found these activities helpful, he replied, ‘yes ’cos I like Monopoly’.

Most TAU participants said that the support they received in school after consenting to participate in BEST was no different from the support they had received in school previously. However, one TAU participant said that she was offered more regular meetings with a member of the school pastoral team than had been the case before her BEST participation:

. . . support like in the past it was just like one thing like one thing and then not really caught up on, but this was like a more scheduled kind of thing that happened more than once.

Young person

However, when asked about the content of these sessions she replied, ‘we’d just catch-up on things and like asking questions [about my week]’. She also responded ‘no’ to a follow-up question about whether the sessions had involved using any worksheets or resources, suggesting that the support offered did not contain any BEST-specific components. When asked about what she liked about the support she received, the participant replied, ‘I liked that it was a familiar a familiar adult . . . I liked that it got me out of a couple of lessons’.

Although there was no evidence of BEST-specific components being used with TAU participants, some practitioners reported that they had used, or had intended to use, resources or strategies from BEST in their wider practice:

. . . the mentalisation section in particular is something that I want to do more with other students to get them to think about yes why other people might be behaving as they are and what might be going through their thought process.

Education practitioner

I have to be honest I have been using some skills from you guys with my current clinical practice because we also do like the relapse management plans but sometimes we have like difficulty and use different grounding skills that was quite helpful.

Mental health practitioner

Furthermore, some education practitioners reported that the experience of participating in BEST had changed how they viewed behaviours that may be symptoms of emotional instability (discussed further in Proposed mechanisms of change and factors affecting successful delivery). Therefore, although there does not appear to have been an issue with direct contamination of the control arm, it is possible that the implementation of BEST may indirectly have an impact on TAU over time through affecting wider practice.

Estimated costs of providing the BEST intervention

The most important cost in providing this intervention was staff time. We did not have detailed information on the staff type and grade of participating school/college staff. However, staff type indicated in the pilot study had a mean annual salary similar to AfC band 4 NHS staff. Therefore, the cost per hour for AfC band 4 was used in the cost estimates presented here. We recorded the AfC grades for mental health practitioners co-providing the intervention sessions, providing supervision or facilitating the training workshop. Unit costs for these grades were obtained from a published source,120 and these relate to the cost year 2018/19. These unit costs include salary and on-costs (costs on top of salary e.g. employer pension contributions) as well as allowances for overheads. However, if any activities occurred outside the sessions or supervision, these would not be included in the costs reported here. Travel time, however, was included.

Training to provide the BEST intervention was provided in two workshops. Workshop 1 was a full-day event provided to 15 school/college staff members and 15 mental health professionals, delivered by four facilitators. A second workshop was provided primarily as a refresher for those who had already attended training during the pilot phase. Attendees at these workshops were recorded along with those facilitating. The cost of the workshop was calculated based on staff costs for all staff at the workshop, whether attending or facilitating.

There would have been some consumables (i.e. training materials, catering) but details of these costs were not included. Estimates for workshop costs are presented in Table 16. In terms of the feasibility study and estimating the cost of the intervention, these costs would be divided by the number of study participants who received the intervention. In this feasibility study this was nine participants, meaning that training would represent a very high proportion of the costs of the intervention. The training sessions were based on the original aim of recruiting 30 individuals and we have additionally estimated a cost per person based on this number; it is this estimate that will be used in the final per-person estimate of the BEST intervention. However, if this was rolled out to a larger study or to clinical practice then each person trained would likely provide sessions for a larger number of young persons and therefore the cost of training as a proportion of the cost the intervention is likely to be lower than the estimates presented here.

We recorded the number of sessions received by participants in the BEST plus TAU arm. Each session was provided by one school/college member of staff and a mental health professional. We assumed that the average duration of sessions observed for the recorded sessions would be the same as the feasibility phase, hence assumed a duration of 50 minutes. We also recorded the AfC band of the mental health professional. In addition, we noted the distance from the school/college to the base of the mental health professional. This enabled an estimate of mileage costs (based on a cost per mile of £0.40) and travel time. An additional time of 10 minutes was estimated for parking and other activities. Again, school/college staff were costed using AfC band 4. 120 As part of the intervention, staff received ongoing supervision. Again, numbers of supervision sessions, the grade of staff providing the supervision and the distance from the supervisor’s base to the school/college were recorded. An average duration of supervision of 60 minutes was assumed. However, in one case supervision was provided by e-mail. In the absence of data on the time commitment involved, a value of 15 minutes was used for all practitioners involved with the supervision for writing and reading these e-mails. The cost of supervision included the time of those providing it as well as those receiving it.

Estimates of the cost of providing the intervention are given in Table 17. This is based on the nine individuals receiving the intervention. However, the cost per participant of training received is based on the cost of training the 30 participants who were originally planned to receive the intervention, as discussed above. This results in a total cost per participant receiving the BEST intervention of approximately £1000.

There are some lessons from this costing exercise that could inform a future study. First, as carried out in this feasibility study, both training and supervision represent significant costs. Careful consideration should be given as to how these should be provided in a future study and whether or not these would be feasible and achievable in routine clinical practice. Second, the amount of information requested from individual practitioners was limited, such as details related to time spent in non-face-to-face activities. More details here could have affected cost estimates. However, there are likely to be limits to the amount of information that can be completed. A future larger-scale study is likely to be informed by more detailed surveys of a limited number of practitioners rather than taking information from all practitioners.

Resource use by participants

Question 1 of the CSRI related to individuals seen in the participant’s school or college in the last 3 months. The use of these resources can be seen in Table 18. This table shows the total number of participants at each time point in each arm who reported use of these services, as well as the total number of contacts reported. At baseline the type of contact that was most reported was welfare officer, with 8 out of 14 and 14 out of 18 participants reporting having used services provided by this contact in the TAU and BEST plus TAU arms, respectively. Service use appears highest in the BEST plus TAU arm, even at baseline. This illustrates the usefulness of having a baseline measure of resource use in any future study, as it allows for the analysis of costs in the follow-up period controlling for baseline costs. This table also shows total contacts and total contacts per person. Total contacts per person are slightly higher in the 12 weeks following randomisation than at baseline. Contacts per person are lowest in the final 12 weeks. Some individuals report quite high use of some services, for example one participant at the 24-week questionnaire reports 65 contacts with a school counsellor. The CSRI appeared to be well completed for this question with no reported missing data.

Questions 2 and 3 of the CSRI relate to the use of medicines. Question 2 relates to the use of specified medicines. These are fluoxetine, sertraline, melatonin, quetiapine, risperidone, mirtazapine, lorazepam and promethazine. Details of medicine use are given in Table 19. At baseline, 4 out of 18 individuals in the BEST plus TAU arm reported use of at least one of these medicines, with no reported use in the TAU-only group. At follow-up there was some reported use of these medicines in both groups. Again, these questions appeared to be well completed, with only one use of fluoxetine and one use of sertraline at the 12-week questionnaire potentially reported as missing. Question 3 relates to the use of any other medicines. This is reported in the last row of Table 19. Drugs here would include those not related to BPD, such as asthma medicines and antibiotics.

Melatonin, mirtazapine and lorazepam were included in the CSRI but no use was indicated. At the 12-week follow-up, one individual (TAU arm) appeared to have a missing value for fluoxetine and 1 for sertraline.

Other health and community services are given in Table 20. There are some reported instances of inpatient stays, the majority being of short duration. However, one individual at the 24-week follow-up in the TAU arm reported a stay of 35 days for mental health reasons. There is also some reported use of CAMHS and A&E services. Total contacts per person for ‘other hospital services’ appear to show no clear pattern. Total contacts per person for ‘other services’ appear to be higher in the BEST plus TAU arm than in the TAU-only arm in the 3 months prior to baseline. Contacts for these services appear to decrease over time, being lowest at the 24-week time point. This may have been influenced by the introduction of lockdown in the UK.

Question 7 of the CSRI asks about overnight stays in a children’s home, with a foster carer or in other residential placement. One individual reported a stay of 14 days in the ‘other residential placement’ category. Finally, question 8 refers to any service use by the participant’s family that resulted from their behaviour or mental health. The BEST plus TAU arm had six contacts at baseline and four at the 12-week follow-up. There was also one contact for the TAU arm at the 12-week follow-up. Types of services contacted included a general practitioner (GP), family support, counselling and a social worker.

In general, the CSRI appears to have been completed well. Responses seem complete with little evidence of missing data, and feedback from the research assistants regarding the process of administering it was generally positive. However, question 8, which refers to any service use by the participant’s family that resulted from the participant’s behaviour or mental health, was experienced as challenging to ask in a sensitive manner. Given that the resource use here was comparatively low, this could be omitted from the modified version of the measure in a future study.

Performance of the EuroQol-5 Dimensions, five-level version

An important aim of the health economics assessment in the feasibility study was to inform the choice of health economics outcome measure in any future study. Hence, we wanted to evaluate the performance of the EQ-5D-5L in this feasibility study. To have a score for the EQ-5D-5L there needs to be a response for all five questions. In general, the completion rate for the EQ-5D-5L was very good. All 32 participants had a complete EQ-5D-5L instrument at baseline. At the 12-week follow-up, one person in the BEST plus TAU group did not complete the instrument and one person in this group did not give an answer to one of the five questions. Hence, there were two missing EQ-5D-5L scores at this time point. One person in the TAU group at the 24-week follow-up point did not complete the instrument. Therefore, we had EQ-5D-5L scores for 59 out of the 62 possible observations over the three time points. As well as the EQ-5D-5L instrument, there is also an associated visual analogue scale (VAS) in which participants rate their health that day on a scale between 100 (the best health you can imagine) and 0 (the worst health you can imagine). In total, there were 60 complete observations for this instrument over the three time points.

The responses for the EQ-5D-5L are shown in Figures 6 (baseline only) and 7 (all completed responses). For EQ-5D-5L responses for each dimension, possible responses vary from 1 (best case) to 5 (worst case). For both Figures 6 and 7, the majority of responses for the mobility and self-care dimensions show no or mild problems. In marked contrast, the anxiety and depression dimension shows respondents reporting responses across the whole range of possible responses, with few reporting no problems. The dimensions of ‘usual activities’ and ‘pain’ also had a wide range of responses. As three out of the five dimensions showed marked differences from full health, it offers encouragement that the EQ-5D-5L may be responsive to BPD and hence may show differences as a result of changes in BPD symptoms.

FIGURE 6.

Baseline EQ-5D-5L responses.

FIGURE 7.

The EQ-5D-5L responses for all time points.

The EQ-5D-5L and EuroQol-5 Dimensions, visual analogue scale (EQ-5D-VAS) scores are shown in Table 21. For the EQ-5D-5L index score a value of one represents a health state considered equivalent to ‘full health’, a value of zero represents a health state of equative value to being dead. Negative scores are possible. Owing to the small sample size, we limited the analysis to a descriptive comparison between groups. One thing that immediately stands out is the low mean scores reported in this sample. Means of 0.59 and 0.67 at baseline indicate very large changes from a score of 1, which would be full health.

The EQ-5D-5L appears responsive in this sample as there are large changes from full health and it appears to be showing a range of responses in three out of the five dimensions. However, it would also be beneficial if the EQ-5D-5L was correlated with other, BPD-specific, scores. For this reason, Pearson correlations were performed against two other measures, the BPFSC and the DERS. For the 32 responses at baseline, there was a –0.720 and –0.595 correlation for EQ-5D-5L with the BPFSC and the DERS, respectively. Both were significant at the 0.01 level. When this was expanded to include all 59 observations at the three time points for which there were values for all three instruments, the correlations were –0.596 and –0.622 for the BPFSC and DERS, respectively. Again, both were significant at the 0.01 level. These results are also encouraging for the potential usefulness of the EQ-5D-5L in a future large study for individuals with BPD. The EQ-5D-5L is likely to be used in such a study as it enables the estimation of quality-adjusted life-years for use in economic evaluations.

Adequacy of training and supervision

Overall acceptability

Views of content and format of sessions

Experiences of co-delivery

Mechanisms of change

Taking together the perceptions of participants and practitioners expressed in the interviews and focus groups, and the processes observed to have occurred within sessions, we theorise that the mechanisms of change may operate on three interacting levels: the individual student, the education practitioner and the wider school or college. For the individual participant, the intervention was perceived to foster better self-understanding, enhance emotional and social literacy, and improve coping skills. For some participants, this appeared to have had a positive impact on their ability to regulate their emotions and their relationships with others. For the education practitioners involved in co-delivery, the perceived benefits of the intervention included increased confidence and skills in supporting young people experiencing emotional instability, equipping them to provide ongoing support to participants and informing their wider practice. Practitioners also perceived a potential benefit for the whole school or college, through enacting a change in attitudes towards behaviour that may be symptomatic of emotional instability. These proposed mechanisms are summarised in Figure 8 and discussed in more detail below.

FIGURE 8.

Possible mechanisms of change of the BEST intervention.

Potential barriers to and facilitators of successful delivery

We identified a number of factors that might help or hinder the effective activation of the proposed mechanisms discussed above.

Stage 1: intervention refinement

To inform the refinement of the prototype intervention, we carried out a rapid evidence synthesis to bring together the available evidence on what factors influence the successful implementation of indicated psychological interventions within schools and colleges. Fifty studies were identified for inclusion in the synthesis. All studies were of school-based interventions, highlighting a gap in the literature for studies involving implementing mental health support within further education and sixth form colleges.

A thematic synthesis of the factors reported by included studies to have an impact on implementation generated 11 analytical themes. These themes encompassed factors on multiple interacting levels, including intervention characteristics, organisational capacity, training and technical assistance, provider characteristics and community-level factors.

The findings of the evidence synthesis highlighted several important issues that contributed to the refinement of the BEST intervention and influenced the conduct of the feasibility trial. These included: (1) the need to consider how to maximise the ability of staff to identify and refer students likely to benefit from the intervention; (2) the importance of the intervention manual and supporting materials, as well as the quality of training and ongoing supervision, for the ability of staff to deliver interventions with fidelity; (3) the need to identify intervention champions at a senior level within settings to promote buy-in from other staff members; and (4) the inevitability of logistical challenges associated with the constraints of the school calendar, routines and environment.

Alongside the evidence synthesis, we piloted the prototype intervention with five participants from three education settings. Of the five young people recruited, three completed the full intervention, one completed only the first session before disengaging and one was unable to receive any sessions owing to being excluded from the setting. Analysis of 19 recorded treatment sessions suggested that the sequence of activities occurring during sessions closely mirrored the structure of the intervention manual. Both education practitioners and mental health practitioners were seen to play an active role in co-delivery of the intervention, although mental health practitioners were more frequently coded as leading activities. Analysis of the link between differing practices employed to deliver intervention activities and participant responses within sessions enabled us to generate hypotheses to be explored further in the second stage.

Practitioner feedback gathered through interviews and a focus group indicated that the intervention was generally positively received, and the co-delivery model was valued despite some logistical challenges. However, practitioners suggested several ways in which the training and supervision, content and format of treatment sessions, and research procedures could be improved.

Findings from the evidence synthesis and pilot were combined to enable us to finalise the intervention manual and resources, refine the practitioner training and amend study procedures in preparation for the feasibility RCT. Key refinements made to the intervention manual were to add further examples and replace technical language to improve comprehension, incorporating information originally provided as an appendix within the body of the manual, and reformatting one of the sessions to ensure that it was possible to deliver within the time frame of a typical school or college period. Key changes made to the training and supervision provided were providing staff with additional pre-training materials and enhancing the practitioner handbook, clarifying the co-delivery model and division of roles within the training workshop, and reducing the frequency of supervision sessions. In addition, several amendments were made to the planned study procedures. These included increasing the number of schools and colleges invited to participate and approaching all potential sites via the senior leadership team in the first instance, introducing a pre-screening questionnaire to assist staff to identify potentially eligible pupils, and increasing the intervention delivery window to allow for missed sessions owing to school holidays, staff and student absence, etc.

Stage 2: feasibility randomised controlled trial

Following the intervention refinement phase, we carried out a feasibility RCT to assess the feasibility of conducting a future trial of the clinical effectiveness and cost-effectiveness of the refined BEST intervention. The feasibility trial was disrupted because of the COVID-19 pandemic and resultant closure of schools and colleges. Consequently, the study was concluded early, reducing the window for recruitment and the number of data we were able to collect. However, the available data were sufficient to answer key research questions regarding the feasibility and acceptability of the intervention, and provide useful information to inform the most appropriate design for a future definitive trial.

Our findings on recruitment, retention and contamination are summarised below in relation to the criteria for progression to a definitive RCT, as set out in the study protocol (see https://journalslibrary.nihr.ac.uk/programmes/hsdr/170931/#/documentation; accessed 6 April 2022). Further factors that we planned to consider in determining the feasibility of evaluating the clinical effectiveness and cost-effectiveness of the refined BEST intervention in a future RCT were the acceptability and suitability of the proposed outcome measures, the ability of practitioners to deliver the intervention with fidelity to the model, and the acceptability of the intervention from the perspective of staff and young people.

The acceptability of the proposed outcome measures appears to have been satisfactory. Measures were generally well completed, and the qualitative feedback suggests that most young people did not find the experience of completing the measures unduly burdensome. However, one young person recounted experiencing some discomfort related to the personal nature of certain questions, particularly surrounding the topic of self-harm. This underlines the need to remain alert to the potential for distress and the importance of appropriate protocols to ensure that this is managed sensitively and in a way that minimises risk and maximises support.

Although the trial was not powered to detect any significant changes in outcomes in line with its aim, mean changes from baseline for continuous outcome measures suggest that they are sensitive to change and, if the intervention is effective, would be able to detect differences in a future trial. However, the BPFSC data, together with practitioner feedback regarding the framing of the intervention, promoted discussion with stakeholders regarding whether or not the severity of BPD symptoms would be the most appropriate primary outcome for a future effectiveness trial. This is discussed further in Stakeholder feedback event.

The health economic measures also appeared to perform adequately. The modified CSRI was well completed and generally acceptable, although we suggest that the final question concerning parental resource use could be omitted in a future study. Similarly, the EQ-5D appears to have performed acceptably in this population and so would be a good candidate for the health economic outcome measure in any future trial of the BEST intervention. Although the cost-effectiveness of the intervention would need to be established in a future trial, the estimated costs of delivering the intervention appear favourable compared with existing interventions for BPD. For instance, the cost of providing a DBT-A programme has been estimated at €10,511 (£9229). 125

Analysis of session recordings suggests that the ability of practitioners to deliver the intervention with fidelity to the model was very good. Of a sample of 31 session recordings rated for fidelity by intervention supervisors, 93.5% were rated as adherent to the manual. Unlike in the pilot phase, the majority of mental health practitioners who co-delivered the intervention in the feasibility RCT were AfC band 4 and band 5 practitioners without core professional training, such as assistant psychologists and children’s well-being practitioners. The high levels of fidelity we observed suggests that mental health practitioners without core professional training would be an appropriate workforce to train and to co-deliver the intervention in future research.

Acceptability of the intervention also appears to have been high. Excluding those who were unable to be offered the intervention as a result of the pandemic, 90% of those allocated to the intervention arm attended at least three sessions and the majority completed all six. Interview and focus group data indicate that the intervention was valued and seen to offer positive benefits both by practitioners and by the young people who received it.

The intervention was perceived to bring about positive benefits at three interacting levels. For the individual participant, the intervention was perceived to foster better self-understanding, enhance emotional and social literacy and improve coping skills. For some participants, this appears to have had a positive impact on their ability to regulate their emotions and on their relationships with others. For the education practitioners involved in co-delivery, the perceived benefits of the intervention included increased confidence and skills in supporting young people experiencing emotional instability, equipping them to provide ongoing support to participants and informing their wider practice. Practitioners also perceived a potential benefit for the whole school or college, through enacting a change in attitudes towards behaviour that may be symptomatic of emotional instability.

Factors identified as potentially facilitating or creating barriers to the activation of the hypothesised mechanisms of change spanned interacting contextual levels. In accordance with the socioecological model developed through the evidence synthesis, these findings underlined the likely impact of structural and environmental factors, interacting with intervention characteristics, in determining whether or not the intervention could be successfully implemented and sustained.

Qualitative analysis of audio- and video-recorded intervention sessions showed a close correspondence between the structure and delivery of sessions and the intervention manual. Analysis of interviews and focus groups revealed how both young people and staff found the intervention acceptable, appreciated the clear structure of sessions, and importantly, reported specific ways in which the intervention components functioned to improve young people’s capacity to mentalise. For example, it was reported that use of the emotions wheel improved how participants recognised and managed their emotions outside intervention sessions. These findings clearly enhanced our understanding of the quantitative finding that 93.5% of sessions were rated as adherent. They also offered important points of triangulations with the findings from Table 14 that showed that the relative contribution of both staff and young people was evenly distributed across session activities, indicating a high level of engagement. Taken together, we were able to conclude that the BEST intervention was not only feasible and acceptable to deliver in schools, but also possible to deliver with a high level of implementation and theoretical fidelity.

Progression criteria

Below we summarise our findings in relation to the prespecified criteria for progression to a multisite RCT of the effectiveness and cost-effectiveness of the BEST intervention plus TAU in comparison to TAU alone.

• Recruitment rate was within 70% of the target.

  Recruitment was slower than anticipated, largely owing to very limited recruitment during school holidays as nearly all referrals were received via schools and colleges as opposed to mental health services. However, prior to the suspension of recruitment owing to COVID-19, we had been confident that the total number of participants recruited by the end of the recruitment period would exceed 70% of the target. From October, when the first participant was recruited, the average rate of recruitment was 5.4 participants per month. The initially projected recruitment rate that would have enabled us to reach 100% of the target sample size within the recruitment period was 6.6 participants per month. To reach 70% of the target a recruitment rate of 4.6 per month across the recruitment period would have been sufficient. Furthermore, additional schools and colleges had agreed to participate in the project and there were a good number of pending referrals at the point of the COVID-19 lockdown, which we anticipated would have resulted in an increase in recruitment.

• At least 70% of those randomised to receive the intervention attended three or more treatment sessions within the 12-week treatment window.

  Of the 10 participants randomised to the BEST arm who had come to the end of the 12-week treatment window prior to suspension of intervention delivery, nine (90%) attended three or more treatment sessions.

• Follow-up assessments were completed by at least 75% of participants at 12 weeks and 70% of participants at 24 weeks.

  Of the 19 participants who had reached the 12-week follow-up before the suspension of the trial, 17 (89.5%) were followed up at 12 weeks and 14 (73.7%) at 24-weeks. All 24-week follow-up assessments were completed remotely at a time when schools and colleges were closed to most pupils because of COVID-19. As a result, staff were unable to assist in contacting participants and arranging appointments as they had at the 12-week time point.

• Contamination of the control arm can be sufficiently limited for individual randomisation to be justified.

  The data available to assess contamination were limited by poor compliance with completion of practitioner contact logs. However, findings from staff focus groups and interviews did not give any indication that direct contamination of the control arm through provision of the intervention or its components to those allocated to receive TAU had occurred. Although there was no evidence of the BEST intervention being used with TAU participants, some practitioners reported that they had, or intended to, use resources or strategies from BEST in their wider practice. Furthermore, some education practitioners reported that the experience of participating in BEST had changed how they viewed behaviours that may be symptoms of emotional instability. Therefore, it is possible that the implementation of BEST within schools and colleges may indirectly affect TAU over time through having an impact on wider practice.

Stakeholder feedback event

Following the conclusion of the feasibility RCT, a stakeholder feedback event was held to share key study findings and gather stakeholder views on the likely sustainability of intervention implementation and the future of the research programme. The online event was attended by stakeholders (n = 12) from across education and mental health services, all of whom had been involved in or supported the study in some capacity. The event began with a presentation of the study findings and opportunity for attendees to ask questions. The presentation of findings was followed by a group discussion on the topic of whether or not to continue to research the BEST intervention with a view to future widespread implementation. Last, break-out rooms were then used to facilitate small group discussions of the following questions:

• What changes to the BEST intervention/approach would be needed for it to be sustainably implemented by schools and colleges?

• What changes to the eligibility criteria or method of identifying young people who may benefit from the intervention might be needed?

• What would be the most important outcomes to focus on in future research on the effectiveness of the BEST intervention?

The discussions were recorded with the consent of those in attendance, to enable stakeholder views to be captured and points of consensus and disagreement to be noted. Following the event, a recording of the presentation of findings was shared with those unable to attend and two further stakeholders provided written feedback on the topics discussed.

There was consensus among participating stakeholders that the programme of research is worth taking forward. The intervention was viewed as having value as a means of bringing the mental health and education sectors together, as well as in enabling schools and colleges to meet the needs of a group of students that they would otherwise feel ill equipped to support. It was raised that schools and colleges had seen a notable rise in the demand for mental health support in the wake of the COVID-19 pandemic; it was suggested that the intervention could potentially play a role in meeting this increased demand.

There was some debate as to whether or not the co-delivery approach would prove sustainable in the long term. However, most attendees agreed that co-delivery was a core component of the intervention and should be preserved. It was suggested that the sustainability of intervention delivery could be maximised by expanding the pool of staff members trained to deliver it. One attendee expressed that if there was evidence that provision of the BEST intervention would, in time, lead to decreased utilisation of specialist mental health services, this would justify diverting resources to its delivery.

The increased use of technology to facilitate intervention delivery was discussed. All were in favour of the creation of an electronic version of the manual. It was also suggested by some attendees that the mental health practitioner could participate in sessions via video call rather than in-person to reduce staff time required to deliver the intervention. However, stakeholders disagreed over whether or not the intervention could be successfully delivered in this format.

This is an issue that requires further exploration given the context created by the COVID-19 pandemic. To futureproof the intervention against the possibility of ongoing disruption, it would clearly be desirable to have the flexibility to accommodate remote delivery when needed. Therefore, this would be a pertinent issue to consider in future research, alongside exploring how trial procedures may need to be carried out in more innovative ways, including remote consent, randomisation and delivery processes.

Although they saw value in a targeted intervention, most stakeholders supported the widening of the inclusion criteria to encompass a broader range of young people who might benefit from intervention to improve their emotional regulation. However, it was raised that this would require a reframing of the intervention to avoid inappropriate pathologising. The prospect of allowing self-referrals or using school-/college-wide screening as a means of identifying young people to receive the intervention was not supported; it was felt that identifying potential recipients in this manner would risk overwhelming already stretched services.

Stakeholders suggested a range of outcomes that they felt would be important indicators of the success of the intervention: these included the young person having improved coping skills, relationships, behaviour and educational engagement, leading to better educational outcomes. It was also suggested that the confidence of school and college staff members in supporting students would be a valuable outcome to capture. There was consensus across groups that improved emotion regulation would be the most appropriate primary outcome for a future trial of the effectiveness of the BEST intervention.