Notes
Article history
The research reported in this issue of the journal was funded by the HTA programme as project number 08/53/06. The contractual start date was in February 2010. The draft report began editorial review in November 2014 and was accepted for publication in December 2015. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The HTA editors and publisher have tried to ensure the accuracy of the authors’ report and would like to thank the reviewers for their constructive comments on the draft document. However, they do not accept liability for damages or losses arising from material published in this report.
Declared competing interests of authors
Hywel Williams is Deputy Director of the National Institute for Health Research (NIHR) Programme and chairperson of the Health Technology Assessment (HTA) Commissioning Board. From 1 January 2016 he became Programme Director for the HTA programme. The NIHR HTA programme funded this study; however, Professor Williams was not involved in that funding decision.
Disclaimer
This report contains transcripts of interviews conducted in the course of the research and contains language that may offend some readers.
Permissions
Copyright statement
© Queen’s Printer and Controller of HMSO 2016. This work was produced by McMurran et al. under the terms of a commissioning contract issued by the Secretary of State for Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science Park, Southampton SO16 7NS, UK.
Chapter 1 Introduction
Aims
The study aimed to determine if psychoeducation with problem-solving (PEPS) therapy in addition to usual treatment for people with personality disorder (PD) results in improved social functioning 72 weeks after randomisation (approximately 12 months after the end of treatment), compared with treatment as usual alone.
In addition, we intended to:
-
assess the costs and cost-effectiveness of PEPS therapy in addition to usual treatment compared with usual treatment alone
-
examine the effects on scheduled and unscheduled use of services
-
examine the effect on mood, as measured by the Hospital Anxiety and Depression Scale (HADS)
-
evaluate referrers’ perceived effects of the intervention using the Global Assessment of Functioning (GAF)
-
evaluate participants’ perceived effects of the intervention in relation to the severity of the self-identified three most important problems
-
examine the process of change by testing the hypotheses:
-
that psychoeducation improves the therapeutic relationship
-
that social problem-solving therapy improves social problem-solving skills
-
-
conduct a qualitative investigation of the receipt of PEPS therapy in practice to identify the views of service users.
Scientific background
Personality disorder
Personality disorder is one of the most common mental health problems, and it is associated with substantial health-care costs. 1–3 Compared with those with no PD, people with PD show higher rates of premature mortality,4 greater engagement in health-compromising behaviours such as substance abuse,1 greater levels of general health problems5 and more use of health-care services. 6 PD is also associated with financial difficulties and problems maintaining jobs,5 marital dissatisfaction and intimate partner violence,7 crime8 and poor quality of life. 9 These matters make a strong case for treating people with PD.
Despite this, there is relatively little reliable evidence on the effectiveness of treatments for PD. Systematic reviews of all psychosocial treatments for PD identified only 27 randomised controlled trials (RCTs) published up to 200610,11 and only a few trials have been published since then. 12,13 The majority of studies are underpowered and most related to one specific PD – borderline PD. Chambless and Hollon’s14 criterion for a treatment to be considered effective is that there should be at least two independent, well-conducted RCTs or single-case experiments. Only one therapy meets this criterion; dialectical behaviour therapy is more effective than usual treatment for reducing suicide attempts, service use and borderline symptomatology in borderline PD, although positive effects decay over time. 12 However, other cognitive and behavioural interventions are supported by single RCTs.
Many treatments for PD are intensive and of long duration, which limits the number of services that can be provided. Hence, the great majority of people who may benefit from psychological treatments do not receive them. Testing the clinical effectiveness and cost-effectiveness of shorter interventions is important if more people with PD are to be treated. Additionally, interventions that can be used with any PD have the potential to alleviate the burden on clinical services that can be created if specific types of PD need to be identified for treatment allocation, and if groups of service users with specific PDs need to be gathered for treatment to commence. In treating groups of people with mixed PDs, the treatment target necessarily needs to be a problem common to all.
One core feature of all PDs is the experience of problems with social and interpersonal functioning. 15–17 One relatively brief skills-training approach to improving problem-solving that has been evaluated specifically with people with borderline PD is the Systems Training for Emotional Predictability and Problem Solving (STEPPS) programme. 18 STEPPS is a 20-week group treatment programme for individuals with PD and others within that person’s system, such as family members, partners, friends and health-care professionals. STEPPS focuses on psychoeducation, emotion management training and behaviour management training. One RCT19 found that borderline symptoms were significantly more improved in those in a STEPPS group (n = 65) than in those who received treatment as usual (n = 59); however, this difference was no longer apparent at the 1-year follow-up. Improvements in secondary outcomes (including GAF, negative affectivity, depression, impulsiveness) and symptoms were also significantly greater for the STEPPS group post treatment but not at the 1-year follow-up. In another RCT,20 those who completed STEPPS (n = 33) were compared with those who received treatment as usual (n = 33), with no significant group differences in PD or other psychological symptoms at the 6-month follow-up, although the STEPPS group showed a significantly greater improvement in overall quality of life. There are clearly improvements to be made to problem-solving approaches for people with PD in terms of enhancing and sustaining outcomes and, as mentioned, offering a problem-solving approach to people with PDs other than borderline is a more practical prospect. Social problem-solving therapy is one viable and empirically supported alternative.
Social problem-solving therapy
Good social problem-solving is one component of social competence. 21,22 Social problem-solving is defined as ‘the self-directed cognitive-affective-behavioural process by which an individual attempts to identify or discover solutions to specific problems encountered in everyday living’23 (p.11). There is abundant evidence of an association between social problem-solving deficits and problems related to PD. 24–27 People with PD report less desirable scores on all scales of the Social Problem-Solving Inventory – Revised: Short Version (SPSI-R)28 compared with a sample of mature students,22 and people high on borderline traits show poorer social problem-solving abilities than those with lower borderline traits, particularly when experiencing negative emotions. 29 This information suggests that social problem-solving therapy may benefit people with PD.
Problem-solving therapy is suited to people with PD because the focus is on improving social functioning and reducing personal distress, which are considered to be of paramount importance in the treatment of PD. 30 The aim in problem-solving therapy is to help people to recognise their strengths and limitations, and to work with these to learn new skills that will enable them to cope more effectively with life’s problems. Problem-solving therapy works to decrease the person’s negative problem orientation and develop positive orientation, without which therapy is unlikely to be effective. 31
Engaging people with PD in treatment is a major challenge. 32,33 The social problem-solving approach enhances engagement by offering an accessible framework for change, supporting people in the experience of successful problem-solving and encouraging independence rather than reliance on therapy. Furthermore, the preliminary psychoeducation component of PEPS therapy aims to educate, build rapport and motivate people for problem-solving therapy. 34 PDs and their impact are discussed in a collaborative dialogue and problems that may be worked on in subsequent group sessions are identified. Psychoeducation has shown good effects with people with borderline PD, showing significantly greater declines in general impulsivity and the storminess of close relationships over 12 weeks than in those who did not receive psychoeducation. 35
Meta-analyses of problem-solving therapy outcome studies document its effectiveness for people with a wide range of mental health problems. 31,36,37 Detained PD offenders were identified as performing poorly in all aspects of social problem-solving compared with offenders with no PD and non-offenders. 38,39 A pilot study of a psychoeducational intervention aimed at clarifying the offenders’ PD diagnosis and identifying associated problems led to an increase in patients’ knowledge about PD and improved the therapeutic alliance. 34 A brief problem-solving therapy was evaluated with this client group, finding that social problem-solving abilities improved post therapy and that this improvement was sustained at a 6-month follow-up. 40 A social problem-solving intervention has also been evaluated with women in a secure setting, with improvements after treatment in risk and health,41 and with people at high risk of suicide, showing reduced suicidal ideation over those receiving treatment as usual. 42
Psychoeducation with problem-solving therapy
A combined PEPS therapy was evaluated with community adults with PD in a Phase 2 exploratory trial. 43 Overall, this sample had the lowest social problem-solving scores in comparison with mature students, prisoners and PD offenders. 44 At the end of treatment, compared with a wait-list control group, those treated with PEPS therapy showed better social functioning, as measured by the Social Functioning Questionnaire (SFQ). 45 Analyses were conducted to examine the hypothesised mechanism of change, namely that improved social problem-solving leads to improved social functioning. 46 These analyses indicated that all aspects of social problem-solving improved over the course of PEPS therapy and, that controlling for baseline level of social functioning, the most important predictor of improvement in social functioning was a reduction in negative problem orientation (i.e. people felt less threatened by problems and more confident in their ability to solve problems). This exploratory study has been identified as important in four ways. 47,48 First, the intervention was brief and, hence, is likely to be more acceptable to patients and services. Second, PEPS therapy was delivered in clinical settings, hence its likely effectiveness in everyday practice was indicated. Third, PEPS therapy was offered to people with any PD or combination of PDs, so it was inclusive rather than exclusive. Finally, non-specialist staff delivered PEPS therapy; hence it would be possible to deliver it relatively cheaply. In addition, a Delphi study of patients’ views of PEPS therapy indicated that it was perceived as acceptable and useful. 49
Overall, PEPS therapy has the potential to contribute to the Department of Health’s agenda that PD should no longer be a diagnosis that excludes people from services. 50 It is an intervention in which staff can easily be trained and, thus, has the potential to make a significant contribution to building workforce capacity. 51 Here, we present the results of an adequately powered, multisite RCT.
Chapter 2 Methods
Design
The PEPS trial was a two-arm, parallel-group, pragmatic randomised controlled superiority trial comparing PEPS therapy plus treatment as usual with treatment as usual alone. Participants were individually randomised at a ratio of 1 : 1, and stratified by sex and centre.
An economic analysis was conducted alongside the trial to determine the costs and cost-effectiveness of PEPS therapy compared with treatment as usual (see Chapter 5). In addition, a qualitative component sought to explore participants’ experiences of PEPS therapy and treatment as usual (see Chapter 6).
Study setting and participants
Study participants were recruited from three NHS trusts providing mental health services in central and north-west London, South Wales and the North-East of England.
We recruited participants from mental health services including community mental health teams (CMHTs), crisis resolution teams, primary care liaison teams, psychology services and on discharge from inpatient care.
Eligibility criteria
At the point of randomisation participants were required to have one or more PD, including a PD not otherwise specified, identified through the International Personality Disorder Examination (IPDE) completed as part of the study-specific procedures at the screening visit. In addition, eligible participants were aged ≥ 18 years, living in the community (including residential or supported care settings) and proficient in spoken English and had capacity to provide informed consent.
Exclusion criteria were a primary diagnosis of major functional psychosis, insufficient degree of literacy, comprehension or attention to be able to engage in trial therapy and assessments, engagement in a specific programme of psychological treatment for PD or likely to start such treatment during the trial period and participation in any other trial.
Identification of participants
Participants were identified by their mental health team. The initial approach about the study was made by a member of the potential participant’s mental health team, who sought verbal agreement from the potential participant to meet with the research team to discuss the study. Referral to the research team was made according to local procedures at each site.
All potential participants referred to the research team were recorded on the Participant Screening and Enrolment Log, whether or not they were enrolled in the trial.
Recruitment
Potential participants providing verbal agreement were referred to the research team who assessed eligibility according to the available clinical information, and invited potentially eligible participants to consider taking part in the trial. Potential participants were provided with written and verbal information about the trial and were given a minimum of 24 hours to consider whether or not to participate.
All participants joining the study provided written, informed consent. Participants were free to withdraw from the study at any time without giving a reason. If participants declined to complete follow-up assessments when originally approached, the researcher sought verbal consent to contact them again at a later stage to see if they were willing to complete the assessments.
Recruitment strategy
The identification and recruitment of participants was actively managed at each site to reduce potential delays and group waiting times. Problem-solving group start dates were determined in advance, allowing a time-limited baseline and randomisation period to be specified, based on recommended minimum and maximum waiting times before commencement of treatment and between the individual and group components of PEPS therapy. The specifications were that psychoeducation should be completed a maximum of 4 weeks before the problem-solving group started. The maximum wait between randomisation and the group starting should be 10 weeks and the minimum should be 5 weeks. This enabled completion of the individual treatment sessions and first follow-up prior to the start of the problem-solving group.
Within each recruitment phase there was an approximate 5-week period within which baseline assessments and randomisation were completed for participants in a particular recruitment phase. Randomisation was completed as soon as possible after baseline assessments and in all cases this should have been done within 1 week.
A minimum starting group size of six was recommended. During the randomisation period, local teams aimed to randomise a minimum of 12 participants to ensure an adequate minimum starting group size. It was recommended that starting group sizes should generally be no more than 10 participants. However, local teams could use discretion in determining the appropriate group starting size according to local circumstances, current waiting times and recruitment rates.
Screening
To confirm eligibility for the trial the following screening measures were undertaken before randomisation:
-
The presence of PD was confirmed using the IPDE. 52 The IPDE is a 99-item, semistructured interview that allows both diagnostic and dimensional scores to be extracted for each PD according to either Diagnostic and Statistical Manual of Mental Disorders (DSM)53 or International Classification of Diseases54 criteria. DSM criteria were used in this trial. Each item is scored as the behaviour or trait being absent or normal (score 0), exaggerated or accentuated (score 1), or at the criterion level or pathological (score 2). Diagnostic scores were calculated in accordance with the scoring manual. A minimum of one ‘probable’ score on any diagnostic category including PD not otherwise specified was required to be eligible for the trial.
-
Adequate literacy was required to engage in trial therapies and assessments. In the majority of cases this was assessed by the investigator or authorised designee in conjunction with the participant’s usual-care team. Adequate literacy was determined in discussion with the participant and their clinical team, based on the ability to participate in the trial therapy and assessments. The Basic Skills Agency’s, Fast Track 20 Questions55 was available as an additional screening measure to aid assessment of literacy if required, but was not used. Study recommendations were that a score of ≥ 3 on the literacy component of the Fast Track 20 Questions indicated that additional consideration may be required, but did not prohibit further involvement in the trial. The final decision about inclusion or exclusion was made by the therapist in consultation with the referrer, the client and, if necessary, the site coinvestigator or site clinical supervisor.
Interventions
This was a two-arm trial comparing PEPS therapy in addition to treatment as usual with treatment as usual only.
Psychoeducation with problem-solving therapy
Psychoeducation with problem-solving therapy is a complex cognitive–behavioural intervention that integrates individual and group therapies. There are two distinct components – individual psychoeducation and group problem-solving therapy – with optional individual support sessions.
Psychoeducation
Psychoeducation consists of up to four sessions delivered by a mental health worker trained to administer the procedure. The number of sessions depends on the duration of sessions and the speed at which the participant can comfortably work through the session content. Although the guidance is to work in 1-hour sessions, some participants prefer to have longer and less frequent sessions to maintain the flow of the content.
The sessions are conducted as a one-to-one collaborative dialogue and are designed to fulfil both general and specific functions. In general, the aims are to build rapport with participants and enhance their motivation for the subsequent problem-solving therapy. This is done specifically by asking participants their views on how their personality leads to problems in interpersonal relationships and social functioning, introducing them to and discussing their PD diagnoses, and explaining how therapy can help people ameliorate their problems.
Interviewers follow a set procedure described in a facilitator’s manual (see Appendix 1). Participants are first asked about their understanding of personality and any personality-related problems that they experience in a brief interview consisting of six questions:
-
What does the word ‘personality’ mean to you?
-
Do you think your personality causes you problems? In what way?
-
Do you think your personality causes problems for other people? In what way?
-
Would you like to change the way you handle problems?
-
Some people are diagnosed as having a PD. Do you know what a PD is?
-
Have you ever been told you might have a PD?
Information on personality and PD is then provided, following an information sheet explaining the concept of personality in terms of it being the way people typically think, feel and behave, and PD being personality styles that persistently cause difficulties and distress. The suggestion that problem-solving therapy can help ameliorate problems is then introduced. Participants are asked to complete a checklist of what problems they experience in relation to their PD. The interviewer completes a checklist that takes the individual through their PD diagnoses, as identified prerandomisation using the IPDE,52 which is a structured clinical assessment. The interviewer and the participant discuss information about the individual’s personality problems from both perspectives. Participants are then guided to identify specific problems that they want to change, and prioritise those to be addressed in the subsequent problem-solving therapy sessions. The interviewer summarises the progress made in psychoeducation and logs the problems to be addressed in problem-solving therapy on a summary pro forma. This summary is used to convey the information to the problem-solving therapy facilitators. The content of psychoeducation is also summarised in a personalised booklet (see Appendix 2) that the participant is given to keep.
Problem-solving therapy
Problem-solving therapy is a 12-session manualised (see Appendix 3) group intervention designed to teach people a strategy for solving interpersonal problems. Problem-solving therapy is delivered by two mental health professionals trained to administer the therapy. The recommended starting group size was between 6 and 10 participants, but local sites were advised to use discretion so that when trial recruitment was slow, groups could start without undue delay; actual group sizes were between 5 and 12 participants. Sessions lasted approximately 2 hours, divided into 75 minutes of problem-solving work, a 15-minute break and 30 minutes of problem-solving work.
In each session, one participant worked through an actual problem that was identified in collaboration with one of the group facilitators prior to the group session. The problem selected could be an emotional or interpersonal problem, rather than a practical problem, and would be one that was current and important but not excessively distressing or unsuited to sharing in a time-limited group. Participants were then guided to learn the steps of the problem-solving process, based on the work of D’Zurilla and Nezu:56,57
-
orientation – identifying negative feelings and using these as a cue for initiating the problem-solving process
-
problem definition – defining their problem clearly and accurately, breaking down large problems into smaller, more manageable ones
-
goal-setting – setting specific goals for change
-
generating alternatives – generating solution options
-
decision-making – considering the consequences of each option to themself and others in both the short and the long term
-
action-planning – selecting potentially effective options and organising these into a means-end action plan.
Participants were then expected to implement the action plan and were offered optional fortnightly individual support sessions throughout the 12-week problem-solving group therapy to help with implementation. Progress with the action plan was reviewed in the next group session.
The problem-solving process is translated into colloquial questions, which are shown in Table 1, along with the formal stages of the process and the skills learned in each stage.
| Question | Stage | Skills |
|---|---|---|
| Feeling bad? | Orientation | Recognition and understanding of feelings countering impulsivity |
| What’s my problem? | Problem definition | Information gathering Assessing the quality and relevance of information Breaking down large problems |
| What do I want? | Goal-setting | Identification of needs Setting targets |
| What are my options? | Alternatives | Creative thinking |
| What’s my plan? | Decision-making | Challenging dysfunctional beliefs Challenging antisocial attitudes Anticipation of outcomes Forward planning |
| How did I do? | Evaluation | Recognise and reward success Recognise and address obstacles |
Throughout this process, attention was paid to improving optimism and hope for change, which is identified as of equal importance to problem-solving skills. This was done by helping participants experience success in problem-solving through guiding them through the problem-solving process, giving them support in their efforts to solve problems, identifying their strengths and highlighting problem-solving successes.
The process of problem-solving through addressing the key questions was recorded on a flip chart as the session progressed. The flip chart could be written by a group member or one of the facilitators, depending on the abilities of group members. This material was then transcribed to A4 sheets, which were given to the participant for his or her records, and a copy retained for the facilitators’ records. Individual support sessions of 1-hour duration were offered fortnightly to help the individual carry out problem-solving action plans. Additionally, participants were encouraged to work through problems independently outside sessions in order to generalise the new skills. A worksheet was provided to assist with independent working.
Problem-solving therapy was provided in mixed- or single-sex groups, depending on the stage of the trial (described in Changes to the intervention during the trial), the number and suitability of referrals received, and participant preference. Participants allocated to PEPS therapy were expected to attend every session, and regular attendance was encouraged in accordance with normal clinical practice. A record of attendance at sessions was maintained for all participants. Participants were not withdrawn from trial therapy for reasons of poor attendance. Owing to variable group attendance rates, a prespecified minimum attendance at group treatments was defined for participants to be considered to have received therapy per protocol. The agreed hypothesis was that attending ≥ 6 of the maximum 12 group sessions of problem-solving therapy would be associated with improved outcomes on the SFQ.
Changes to the intervention during the trial
Within the trial, problem-solving groups were originally intended to be single sex. This was to ensure consistency with the pilot study and in response to preferences expressed by service user representatives advising on the design of the study during protocol development. However, the requirement for single-sex groups was found to cause delays while awaiting the accrual of sufficient participants to form a group. This was a particular issue for male participants because fewer men were referred to the study.
After consulting with the Trial Steering Committee (TSC), the study team took the decision to allow problem-solving therapy to be offered in mixed-sex groups. The reasons for this were:
-
Mixed-sex groups are routine practice in community-based clinical services already offering PEPS therapy.
-
Mixed-sex groups may help to reduce waiting times and delays between recruitment and randomisation.
-
Mixed-sex groups can provide clinical benefits (e.g. helping participants to address issues with relating to people of the opposite sex).
An amendment was submitted to the Research Ethics Committee to introduce this change in August 2011, approximately halfway through the recruitment period. Following implementation of this amendment, allocation to mixed- or single-sex groups was made in accordance with usual clinical practice, incorporating participant preference where possible.
Treatment as usual
Usual treatment was provided by participants’ usual-care teams in accordance with normal clinical practice. No restrictions were placed on access to other treatments during the trial period, although engagement in a specific programme of psychological treatment for PD was an exclusion criterion applied at the point of enrolment.
The original protocol included a standardised form of treatment as usual as the control. Shortly after the start of the recruitment period, it became apparent that there was substantial variability in the level and type of care provided to people with PD at each of the participating sites. Many potential participants were being assessed by mental health services and discharged without treatment. To exclude these people would seriously compromise recruitment to the trial. For this reason, the study team could not impose a standardised form of treatment as usual on the referring clinical services. The study team felt that this issue was likely to become more pressing as NHS cuts at the time caused CMHTs to reduce services. As a result, it was agreed that the trial should compare PEPS therapy with treatment as usual in whatever form that took, and the planned requirement for a standardised form of treatment as usual was removed from the protocol in August 2010. The standardised form of treatment as usual, outlined in the original protocol, was recommended as a minimum standard of care but this was not imposed on clinical services referring participants to the trial.
Treatment fidelity
Manualised assessment and treatments
The IPDE schedule, psychoeducation and problem-solving therapy are all comprehensively manualised.
Training and supervision
Therapists were qualified mental health nurses or psychology graduates with clinical experience. All IPDE assessors attended training in administering and scoring the structured interview from a qualified and highly experienced clinician and researcher. Each lead clinician who delivered psychoeducation and problem-solving therapy was trained to conduct the intervention. Problem-solving therapy groups were facilitated by two facilitators. Most cofacilitators also attended training in delivery of the intervention; however, on the rare occasions that this was not possible, groups could be cofacilitated by a facilitator who had not completed the training, provided that they were fully briefed by the lead therapist and were aware of the limitations of their involvement. A minimum of one fully trained and assessed facilitator was present at every group session.
Psychoeducation training was delivered after IPDE training and consisted of informing therapists of the rationale for psychoeducation, explaining the delivery mode of an educational dialogue and familiarising therapists with the materials and their sequence of delivery. Problem-solving therapy training consisted of 3 days in which groups of participants were given the theory, outcome evidence and role-play practice.
In each case (IPDE, psychoeducation and problem-solving), training followed an existing training protocol. Training was conducted centrally by experienced clinicians and researchers. After training, regular supervision was provided, both centrally and locally.
Competence checks
Audiotapes of treatment delivery were scrutinised by the trainers to ensure that each therapist was adhering to the treatment specification. Competence checklists were constructed for this assessment (see Appendices 4–6). These specified the key activities for conducting the IPDE and delivering psychoeducation and problem-solving sessions according to the intended treatment model. Cut-off scores for competence were agreed in advance and therapists were assessed for competence in delivering the treatment. None of the therapists failed to meet the competence criteria on any of the measures. Having assessed the therapists as competent to deliver the treatment according to the model and the protocol, no further checks were made. This was considered to reflect actual clinical practice in which staff are trained in a procedure and, if they meet the standards set by the trainers, they commence practice and the quality of their continued practice is monitored through supervision.
Fidelity checks
Treatment fidelity was assessed in three ways:
-
Measuring adherence to protocol implementation (e.g. frequency and duration of treatment sessions).
-
Assessing adherence to therapy, as specified in the treatment manual.
-
Clinical supervision.
Adherence to psychoeducation was self-rated by the therapist after the end of all psychoeducation sessions, using a standard protocol (see Appendix 7). Adherence to problem-solving group sessions was rated by experienced clinicians, based on a sample of audiorecorded sessions.
Outcome measures
Primary outcome
The primary outcome was social functioning as measured by the SFQ. 45 This is an 8-item self-report scale, on which each item is scored from 0 to 3. The total SFQ score ranges from 0 to 24. A reduction (i.e. an improvement) of ≥ 2 points on the SFQ at the 72-week follow-up was the specified clinically significant change.
Secondary effectiveness outcomes
-
Scheduled and unscheduled health-service use collected through a retrospective review of mental health service and general practitioner (GP) records.
-
Mood, measured by the HADS,58 a 14-item self-report questionnaire with scores in the range of 0 to 42, and on which higher scores are less desirable.
-
The referring clinician’s judgement of the participant’s overall level of psychosocial functioning assessed by the GAF. 53
-
The client’s assessment of severity, on a scale from not at all distressing (0) to very distressing (10), of the three problems they considered most important (three main problems).
Process measures
The following measures were intended as measures of the processes of change during PEPS therapy:
-
Therapeutic relationship was assessed using the Working Alliance Inventory – Short Revised (WAI-SR),59 a 12-item scale rated by both client and therapist to assess agreement on the tasks of therapy, agreement on the goals of therapy, and the bond between client and therapist, with a range of scores between 12 (poor) and 48 (good).
-
Social problem-solving abilities were measured by the SPSI-R,28 a 25-item self-report questionnaire that measures problem-solving orientations and styles, with five items in each of the five subscales: positive problem orientation, negative problem orientation, rational problem-solving, impulsivity/careless style and avoidance style. Effective social problem-solving is indicated by higher scores on positive problem orientation, rational problem-solving and the SPSI-R, and lower scores on negative problem orientation, impulsivity/careless style and avoidance style. A total social problem-solving score ranges from 0 to 25, in which a higher score is more desirable.
Health economic outcomes
Participants’ views and experiences
Qualitative semistructured interviews were completed with participants allocated to PEPS therapy after psychoeducation and after problem-solving therapy to seek participants’ views on treatment. Further interviews were completed with all participants after the final follow-up to seek participants’ views on the experiences of PEPS therapy and usual treatment.
Safety and tolerability measures
Adverse events occurring in trial participants were recorded and monitored. For the purposes of this trial, a recordable adverse event was defined as any of the following:
-
death for any reason
-
inpatient hospitalisation for any reason
-
any other serious, unexpected adverse event.
Adverse events were reported for all participants from consent to trial completion or early withdrawal from trial follow-up. If a participant withdrew from treatment but agreed to remain in trial follow-up, data collection, including adverse event reporting, continued in accordance with the protocol.
Premature withdrawal from the trial therapies or follow-up was reported, with reasons for withdrawal documented when these were given.
Statistical methods
Statistical analysis
The analysis and reporting of the trial was in accordance with Consolidated Standards of Reporting Trials (CONSORT) guidelines. 62–65 Analyses were detailed in a statistical analysis plan, which was finalised prior to completion of data collection and database lock. All analyses were conducted using Stata version 13 (StataCorp LP, College Station, TX, USA).
Preliminary analyses
Descriptive statistics of demographic and clinical measures were used to examine the balance between the randomised arms at baseline.
Primary analysis
The primary analysis compared the mean SFQ score between PEPS and usual treatment at the 72-week postrandomisation follow-up, adjusted for baseline SFQ score and stratification variables (centre and sex), and implemented using maximum likelihood-based generalised linear modelling. The primary analysis compared individuals as randomised, regardless of treatment actually received or if 72-week follow-up SFQ data were observed (intention-to-treat principle). The effect is presented as an adjusted difference in means, 95% confidence intervals (CIs) and p-value for the comparison.
Imputation of missing primary outcome data
The pattern of missing data was investigated by examining variables recorded at baseline that were associated with ‘missingness’ of SFQ score at the 72-week follow-up. Multiple imputation and analysis of multiple imputed data sets were conducted using ‘mi’ procedures in Stata. The imputation model contained site, age, sex, ethnicity, social status, PD category (simple or complex), SFQ at baseline and 24 weeks, baseline EQ-5D health state score, baseline HADS score, baseline SPSI-R score and baseline three main problems score, and 20 data sets were imputed.
Missing item data
For all outcomes that are a scale comprising a number of items, the following procedure was undertaken when > 0% and ≤ 15% of items were missing:
-
Step 1: calculate the scale mean for each participant (denoted by m1 for those with > 0% and ≤ 15% of items missing).
-
Step 2: calculate the mean of scale means for participants with complete scale data only (denoted by M1).
-
Step 3: calculate each item mean for all participants with observed data for that item (denoted by S1).
-
Step 4: for each item, calculate M1 – S1 (denoted by d).
-
Step 5: impute missing item data using m1 – d.
When > 15% of items were missing, the total scale score was regarded as missing and imputed using multiple imputation.
Clustering in psychoeducation with problem-solving arm
In this trial there were two potential sources of clustering in the PEPS arm only: by therapist in the first treatment phase and by the problem-solving therapy group in the second treatment phase. Data for the former were not available for some participants, or else treatment in the first phase was delivered by a single therapist per centre. Furthermore, any clustering effect was expected to be dominated by the latter. Therefore, we obtained clustered sandwich estimates of variance by specifying the ‘cluster’ option in all regression models, which relaxes the assumption that all observations are independent.
Sensitivity analyses
We conducted the following sensitivity analyses of the primary outcome:
-
Repeated the primary analysis with additional adjustment for any variables displaying marked imbalance between the arms at baseline.
-
Repeated the primary analysis restricted to those participants with observed primary outcome data at 72 weeks.
-
To examine treatment efficacy, complier average causal effect (CACE) estimates66 were calculated using instrumental variable regression methods for those participants in the PEPS arm who received the intervention in line with the treatment protocol. The definition of treatment received as per protocol was having completed psychoeducation according to the therapist assessment and attended a minimum of six of the group problem-solving sessions.
Subgroup analyses
Although no subgroup analyses were specified a priori, we conducted two exploratory subgroup analyses by including appropriate interaction terms in the regression model for the primary outcome. We investigated whether or not there was any evidence of differential effects of treatment on SFQ score at 72 weeks according to (1) study site (central and north-west London, South Wales and North-East England); (2) PD category (simple, complex); and (3) borderline PD diagnosis at baseline.
Secondary outcomes
Analysis of secondary outcomes was conducted using a similar approach as for the primary outcome, except that missing data were not imputed, and choice of regression model and presentation of the estimated between-group effect was dependent on outcome type (continuous, binary, ordinal, rate). We used proportional odds logistic regression for ordinal data and we checked the goodness-of-fit assumption for the Poisson regression analysis of count data using the Pearson test. Descriptive data are presented for each time point, but formal comparisons were only conducted for 72-week data. The exception to this is the SPSI-R total score, for which a repeated measures analysis was conducted including data at both the 24- and 72-weeks follow-up to examine whether any treatment effects were sustained or emerged later. This was tested formally with an interaction term between treatment group and time in the model, and in the absence of any evidence of a time effect, a repeated measures analysis generates an average effect size over the duration of follow-up.
Between-group differences in health-care service use and adverse events were estimated using binomial/Poisson regression modelling and allow for multiple events per individual.
Interim analysis
No formal interim analyses for effectiveness were planned or undertaken, however, unblinded data were periodically reviewed by the Data Monitoring and Ethics Committee (DMEC) during routine meetings.
Sample size
The sample size calculation for the study was based on the primary hypothesis that those randomised to PEPS therapy in addition to usual treatment would have improved social functioning at 72 weeks after randomisation compared with those randomised to usual treatment only. We powered the trial to detect a difference of 2 points on the SFQ score (standardised effect size of 0.44). This is agreed to be a clinically significant and important difference. 67 We based our sample size estimate on a conservative (i.e. largest) estimate of standard deviation (SD) of 4.53 points.
To detect a difference in mean SFQ score of 2 points with a two-sided significance level of 1% and power of 80%, with equal allocation to two arms, would require 120 patients in each arm of the trial. In anticipation of a 30% loss to follow-up at 72 weeks after randomisation, we planned to randomise 340 participants (170 in each arm).
Randomisation
Following recruitment and completion of screening and baseline assessments, participants were randomly allocated to receive PEPS therapy in addition to usual treatment or usual treatment alone at a ratio of 1 : 1.
Randomisation was based on a computer-generated pseudo-random code using random permuted blocks of randomly varying size, created by the Nottingham Clinical Trials Unit (NCTU), in accordance with its standard operating procedure and held on a secure server. The randomisation was stratified by recruiting centre and sex. The sequence of treatment allocations was concealed until recruitment, data collection and all other trial-related assessments were complete.
The investigator, or an authorised designee, accessed the treatment allocation for each participant by means of a remote, internet-based randomisation system developed and maintained by the NCTU. Allocation was therefore fully concealed from recruiting staff.
Study procedures
Preparatory phase
Site initiation visits were completed prior to the start of recruitment to ensure that all site staff were trained in the protocol and study-specific procedures.
Visit schedule
The duration of follow-up was 72 weeks post randomisation. The study schedule is shown in Figure 1.
FIGURE 1.
Schedule of visits. TAU, treatment as usual.
Data collection
Follow-up visits were completed in person or by telephone. To improve response rates at the final follow-up, the SFQ was posted to participants who could not be contacted by another means.
The majority of data were collected through the use of standardised, self-report assessment measures completed by participants during scheduled follow-up visits. Research staff involved in data collection were provided with guidance on the principles of standardised assessment and on the specific measures employed in the trial. Assessments were self-completed by participants or read aloud to participants by the researcher if required. In this case, questions were read out verbatim and were not reworded. No test feedback was given to participants.
The Service Use Record Check was completed by the research assistants after the final assessment measures had been collected. Service use data were collected from GP and mental health records retrospectively for the duration of the trial, according to a data collection manual that outlined procedures for accessing GP records, procedures for dealing with incomplete or inconsistent data, and definitions of key terms; standardised data collection forms were used (see Appendix 8).
Blinding
In pragmatic trials of this type, as in usual clinical practice, it is not possible to blind participants or clinicians to whether they are in the intervention or control arm of the trial; therefore, participants, mental health workers delivering the interventions and participants’ usual-care teams were aware of the treatment allocation. Most of the outcome data were obtained from self-report questionnaires from participants who were not blind to treatment allocation. However, outcome measures were administered by research assistants blinded to treatment allocation in order to reduce assessment bias as far as possible. Data analysts remained blinded to allocation during the study by having access to only aggregate data and no access to data that could reveal treatment arm, such as course attendance. Final analyses were conducted using treatment labels A/B, with allocation decodes released only after completion of analyses. Data that could reveal allocation were analysed following release of allocation decodes.
At the start of each follow-up, participants were reminded of the importance of not disclosing their treatment allocation to the research assistant using a suggested unblinding script (see Appendix 9). If the research assistant was inadvertently unblinded to treatment allocation before completing the final follow-up, a record of the incident of unblinding was made. Researchers also reported whether or not they were aware of the treatment allocation at the time of completing the primary end-point assessments. Owing to changes in personnel over the course of the trial, in some cases, end-point assessments were conducted by researchers who were not unblinded. A record was made of the blinding status of the researcher conducting the final follow-up data collection.
Payments to participants
Participants reaching the final follow-up were offered a non-contingent voucher payment in recognition of their contribution to the trial. Contact with the participant at the final follow-up was sufficient for provision of the voucher (i.e. payment was not contingent on completion of the final follow-up assessments). This voucher payment was introduced in an amendment in April 2013, approximately halfway through final follow-up completion.
Reimbursement of travel expenses incurred in relation to attendance at research appointments was offered, and travel expenses incurred by participants in conjunction with the treatments provided in the trial were paid in accordance with normal clinical practice at the local sites.
Patient and public involvement
Two service users were involved in the protocol development and in the preparation of the participant information sheet and consent form. Service user representatives on the Trial Management Group (TMG), TSC and DMEC contributed to the management and oversight of the trial.
Research governance
The study was conducted in accordance with the principles of good clinical practice and the Research Governance Framework for Health and Social Care. 68
Ethics
Ethical approval for the study, including amendments, was given by the South Wales Research Ethics Committee (reference number 09/WSE03/48).
The final approved protocol was version 6.0, dated 3 April 2013. The original approved protocol was version 1.0, dated 18 September 2009. For a summary of amendments implemented during the trial see Appendix 10.
Oversight
A number of committees were assembled to ensure the proper management and conduct of the trial, and to uphold the safety and well-being of participants. The general purpose, responsibilities and structures of the committees were described in the protocol, with separate charters developed for the independent oversight committees.
The TMG comprised members of the study team and met regularly throughout the trial to oversee the day-to-day management of the trial. The TMG met approximately once a month for the duration of the trial, with meetings held face to face and by teleconference for those unable to attend in person. The TMG reviewed recruitment and data completion rates, as well as identifying and addressing any issues arising during the course of the trial.
Independent oversight of trial conduct was provided by the TSC and DMEC.
The independent TSC monitored, reviewed and supervised the progress of the trial. The TSC also monitored pooled data to consider safety and efficacy indications, and considered reports from the DMEC.
An independent DMEC was established, with access to unblinded data, to provide independent reviews and recommendations to the TSC regarding continuation of the study in light of potential treatment effect. The DMEC was advisory to the TSC. During routine conduct of the trial, the DMEC was the only group with access to unblinded data. The DMEC reviewed unblinded data at routine meetings held during the course of the trial. The data that were presented included listings of reported adverse events and reported hospitalisations collected from the CSRI.
For the schedule of meetings of the DMEC and TSC see Appendix 11.
Safety monitoring
Local procedures were implemented at each site to ensure adverse events were recognised and reported, including asking participants about adverse events during each contact and asking the participant’s clinical team to inform the site principal investigator if an adverse event was identified. The participant’s responsible clinician was also contacted by letter to request information on adverse events throughout the trial. In addition, in the event of loss to follow-up, the participant’s clinical team and/or GP were contacted to alert the responsible clinician to the participant’s loss to follow-up and to request information on any unreported adverse events to ensure that safety data remained accurate and up to date. All adverse events were reported to the trial co-ordinating centre within 24 hours of the study team becoming aware of them.
Adverse event reports were reviewed on receipt at the co-ordinating centre, and were assessed for relatedness and expectedness by the chief investigator in accordance with the National Research Ethics Service guidance on adverse event reporting in trials that do not include medicines. To guide this assessment, the adverse event form collected information on all possible and suspected causes identified from the available clinical information, including clinical notes and participant self-report. A categorical assessment of ‘relatedness to the trial’ was also made by the person reporting the event.
Adverse events were also classified by the person reporting them according to whether or not there were indications of ‘psychological antecedents’. Events that were deemed to have psychological antecedents were defined as mental health-related events. Mental health-related events were further categorised as follows:
-
self-harm, including drug or alcohol overdose
-
deterioration in mental health
-
suicidal ideation
-
suicide or attempted suicide
-
planned/respite hospital admission
-
other (specify).
The primary classification only was recorded. Adverse events were classified by the person reporting the event on the basis of the information available at the time (e.g. through participant self-report or clinical notes). For example, attempted suicide was recorded when this was the reason given by the participant, and it does not necessarily relate to the severity of harm caused or evidence of clear intent (i.e. events recorded as ‘attempted suicide’ are not necessarily life-threatening).
All adverse events were routinely reported to the Research Ethics Committee, DMEC and TSC as part of the regular reporting requirements. In addition, serious adverse events that were deemed to be both related to administration of any of the trial procedures and that were not identified as expected occurrences were subject to expedited reporting to the Research Ethics Committee, as required by the National Research Ethics Service guidance for studies that are not clinical trials of investigational medicinal products in the UK.
Chapter 3 Early stopping of recruitment and delivery of the trial intervention
The decision to stop recruitment and trial therapy
On 29 October 2012, following the fourth meeting of the DMEC, the DMEC chairperson wrote to the TSC notifying the TSC of a safety alert in the PEPS trial based on an untoward pattern of serious adverse events. The DMEC had done some investigations but had been unable to satisfy themselves of whether this finding might or might not have been treatment related. The DMEC recommended further investigations to clarify the safety and tolerability of the treatment and advised that randomisation to the trial be suspended until this had been further investigated. The DMEC did not recommend stopping the treatment of people currently in the trial, as they considered that the potential risks of harm in discontinuing treatment were not justified at that stage. Following subsequent correspondence and discussions between the two committees, during which the TSC reviewed the recommendations of the DMEC and the unblinded data on which they were based, the TSC agreed that a safety concern could not be ruled out, but made different recommendations to those advocated by the DMEC. The TSC communicated the following decisions to the chief investigator and trial co-ordinating centre in a letter dated 6 November 2012:
-
No further patients should be randomised into the PEPS trial.
-
Patients who were currently in treatment in the trial should no longer receive trial treatment within the parameters of the trial treatment protocol.
-
To fulfil the duty of care to patients who have completed the treatment phase of the trial the trial team should consider how to inform patients of the possibility of harm.
-
Trial data should continue to be collected and all patients followed up as per protocol.
A number of clinical concerns were raised by the PEPS trial team between 10 and 12 November 2012, and the study team felt that a challenge to the decision to stop trial treatment was warranted. In the absence of precedent in this situation, the following process was proposed by the trial co-ordinating centre and agreed by all parties, including the funder:
-
A joint meeting of TSC and trial team would be held.
-
The TSC would explain their decisions and rationale behind them.
-
The study team would present their concerns.
-
Chaired discussion between all parties.
-
The TSC would convene a closed meeting to consider the challenge and agree its response.
These discussions took place at a face-to-face meeting held on 15 November 2012. The meeting was hosted by the NCTU and chaired by a senior member of the NCTU, who was not otherwise involved in the trial. The closed TSC meeting was held on 19 November 2012. Immediately following this meeting, the TSC wrote to the study team to confirm the unanimous decision of the TSC to uphold its original decision. Following this confirmation, the study team took immediate steps to action the implementation plan that had been previously agreed during the meeting held on 15 November 2012 and in subsequent correspondence.
Stopping recruitment and trial therapy
On receipt of the notification of the initial decision from the TSC on 6 November 2012, the following interim actions were taken immediately by the chief investigator and trial co-ordinating centre:
-
The online randomisation system was suspended on 6 November 2012 to ensure that no further participants would be randomised.
-
The TMG, other coinvestigators, the trial funder and the trial sponsor were notified immediately.
-
A holding statement, approved by the TSC and sent on their behalf, was circulated to recruiting sites on 8 November 2012. Site principal investigators were asked to disseminate the holding statement to site staff and to stop any further recruitment of participants with immediate effect on 8 November 2012.
Delivery of the trial therapy was suspended where it was possible to do so without undue disruption (e.g. postponing appointments). At this stage, steps were not taken to permanently discontinue treatment or inform participants until the implications had been fully considered and a clinically appropriate action plan, with support and alternative treatment arrangements, was in place.
A brief search and consultation failed to find any previous examples of cessation of recruitment and treatment because of a safety alert in a trial of this type. Therefore, the process of stopping recruitment and delivery of trial therapy meant that additional, specific procedures were developed in collaboration between the TMG, TSC and NCTU with reference to clinical guidelines and accepted good practice. Regular communication was maintained during the planning and implementation to review progress and agree the next steps.
Informing participants and clinicians of the trial changes
Everyone affected by the trial changes was provided with information by the research team. A written information sheet was provided. Participants who were ‘active’ in the trial (i.e. recently referred or in the treatment phase) were provided with additional information and support regarding their ongoing care. A separate, simplified version of the information sheet was available for participants in the follow-up period.
Clinical teams and referring clinicians were informed of the trial changes. They received the same information as participants, and were offered additional guidance on ‘frequently asked questions’ and sources of additional support and advice, should they be required to respond to queries or concerns from participants.
A letter was sent to the responsible clinician within mental health services and the GPs for all participants involved in the trial to inform them of the changes to the trial.
Participants and clinical teams were informed of the changes to the study as a matter of urgency. It was considered essential that all those affected by the changes were informed by the research team first, before this information was in the public domain.
The provision of information to people affected by the trial changes was completed on a staged basis so that the processes used and information provided could be informed by early feedback from participants and clinical teams. However, the process of informing people affected by the trial changes was time limited and all those affected by the changes were notified within a reasonable time scale. Whenever possible, this was within 4 weeks.
Updated consent
No further trial-specific procedures (e.g. continuation of the IPDE, sessions with a PEPS therapist or follow-up assessments) were completed with participants until they had been informed of the trial changes. Participants in the trial were asked to sign an updated consent form to confirm receipt of the new information and indicate their ongoing agreement to participate in the trial. Verbal consent was accepted for continuation of follow-up assessments completed by telephone, although written consent forms were also requested.
Issues relating to participants
When the decision to stop recruitment and delivery of the trial therapy was made, 306 participants had been randomised. The number of participants at each stage of the trial is shown in Table 2.
| Stage of the trial | Number of participants |
|---|---|
| Randomised to PEPS therapy and active in the treatment phase of the trial | 19 |
| Randomised, active in follow-up | 210 |
| Randomised, completed trial follow-up | 54 |
| Randomised, withdrawn before completion | 23 |
| Active in screening phase | 38 |
| Referred but not yet started study-specific procedures | 42 |
Randomised to psychoeducation with problem-solving therapy and active in the treatment phase of the trial
Participants who had been randomised to receive PEPS therapy and were currently in the treatment phase of the trial were informed of the trial changes as soon as practically possible, and arrangements were made to ensure that the delivery of PEPS therapy was stopped in a clinically appropriate way. The TSC had confirmed that their recommendation to stop trial treatment did not advocate an abrupt stopping. Delivery of trial treatment in accordance with the protocol should cease; however, there was scope to meet with participants to explain the situation and to involve them in decisions about a structured and clinically appropriate end to trial treatment. What was clinically appropriate depended on the stage of PEPS therapy and the needs of individual participants. This was a clinical judgement made by the site principal investigator in consultation with clinicians involved in the trial, the participant’s usual clinical team and the participant, as appropriate. To underpin what was clinically appropriate we referred to national guidelines on treatment of PD. There are two National Institute for Health and Care Excellence (NICE) guidelines on the treatment and management of PD: one for borderline PD69 and one for antisocial PD. 70 Both emphasise the importance of endings and transitions. As a result, the study team identified a need to liaise with clinical teams to identify what support they could offer and how the investigators and trial therapists could assist.
Randomised, active in follow-up
Participants currently in follow-up were informed of the trial changes at their next scheduled follow-up appointment if this was due within 4 weeks. Participants were informed by letter before their next scheduled appointment, if this was necessary, to prevent an undue delay. Participants informed of the trial changes by letter were provided with contact details and invited to contact the study team should they have any queries or wish to discuss this further.
Active in screening phase
Participants who had not yet been randomised but were active in the screening phase of the trial were also informed promptly. Participants in the middle of the IPDE were given the choice to stop or continue this. Participants who completed the IPDE were offered feedback on the results of the IPDE, in accordance with the existing procedure for doing this at the end of the trial (this is distinct from the provision of psychoeducation).
Referred but not yet started study-specific procedures
There was a number of people who had not completed any study-specific procedures at the time of stopping, although some had given consent to join the trial. The research team contacted these people to thank them for their interest in the study and to inform them that recruitment to the trial had stopped.
Randomised, completed trial follow-up or withdrawn before completion
In addition, participants who were in follow-up and those who had completed their involvement with the trial (completed final follow-up or withdrew before completion) were informed and supported, as appropriate. Arrangements were made to contact participants who had completed their involvement in the trial and those who withdrew before completion. These participants were informed by letter and provided with a written information sheet.
The following people were not directly informed of the trial changes by the research team:
-
people who were referred to the PEPS trial, but were excluded before consent or declined to participate
-
participants who consented and withdrew before randomisation
-
participants who consented, but were not eligible for randomisation following screening.
Additional support for study staff
Study-wide guidance documents were prepared and maintained to ensure that everybody involved in informing participants and clinical teams, and responding to queries, including those responsible for staffing the trial helpline, had access to relevant written information to support them in this role.
Initially, local sites made arrangements for a limited local helpline to address queries and concerns from participants and clinical teams promptly. Arrangements for a trial-wide helpline were made, in the event that demand should exceed local capacity.
A press release was prepared and agreed by all parties so that a clear statement could be made in the event that the media picked up on the trial stoppage.
The national helpline and press statement were not used.
Documentation of the process
The process of informing participants and of stopping the delivery of PEPS therapy was clearly documented. Feedback given by participants was recorded to inform the ongoing process and ensure that lessons could be learned.
In addition to recording what information was provided and when, sites were also asked to document any questions, comments or concerns raised by participants and clinical teams, and to provide feedback to the trial co-ordinating centre on the issues raised.
Participants’ views on trial stoppage
Feedback was reviewed to identify learning points and enable reflection on the processes used in the trial and during implementation of the trial changes. 71 Specific feedback was received and documented from 37 participants from all trial stages. A number of recurring views were identified. Among the most frequently reported were that an initial increase in distress was expected when engaging in psychological therapies; stopping PEPS therapy led to a concern over the lack of alternative treatment options; that there were alternative interpretations of the finding that more adverse events were recorded in the PEPS group; and that therapy delivered within the trial could have been improved (e.g. needed to be longer). There were no complaints about how the trial changes were implemented.
Urgent safety measure
A substantial amendment (substantial amendment 08 dated 26 November 2012) sought retrospective approval from the Research Ethics Committee for implementation of the trial changes as an urgent safety measure. The procedure for informing participants and clinical teams, and supporting documents were submitted for ethical review and approval in parallel to implementation. Approval was received in December 2012.
Chapter 4 Results
Recruitment
Study recruitment commenced in August 2010. Between August 2010 and November 2012, 739 people were referred to the trial for assessment of eligibility. Of these, 444 people were initially assessed as eligible and provided consent to join the trial. The reasons for non-participation were not willing to provide consent (n = 113), not eligible (n = 49), unable to contact following referral to the trial (n = 106) and not recruited because of the early stoppage (n = 27). The reasons participants were deemed not eligible for enrolment are given in Table 3.
| Not eligible for enrolment in the trial | Total number of participants |
|---|---|
| Responsible clinician did not consider person had PD | 7 |
| Primary diagnosis of psychosis | 3 |
| Insufficient spoken English | 1 |
| Insufficient literacy, comprehension or attention | 3 |
| Receiving other psychological treatment for PD | 22 |
| Other | 13 |
| Total not eligible for enrolment in the trial | 49 |
One hundred and thirty-eight people provided consent to join the trial but withdrew or were excluded before randomisation. The reasons for withdrawal/exclusion before randomisation are presented in Table 4.
| Withdrawal/exclusion before randomisation | Total number of participants |
|---|---|
| PD diagnosis not confirmed by IPDE | 32 |
| Insufficient literacy identified at screening | 1 |
| Participant withdrew consent | 34 |
| Unable to contact | 23 |
| Early stopping of recruitment | 48 |
| Total withdrawn/excluded before randomisation | 138 |
Of the participants who consented but were not randomised, 67% were female and the average age was 38 years.
Participant flow
The CONSORT diagram in Figure 2 summarises the assessments completed at each time point.
FIGURE 2.
The CONSORT diagram. a, Not done is a combination of those participants who did respond to contact and those who chose not to attend the follow-up for that particular follow-up; b, two of the deaths occurred before and two after the trial was stopped.
Follow-up at 72 weeks after randomisation was completed for 62% and 73% in the usual treatment and PEPS arms, respectively. Follow-up rates for each time point by centre were as follows:
-
first follow-up: central and north-west London 75%, South Wales 89%, North-East England 80%
-
second follow-up: central and north-west London 61%, South Wales 73%, North-East England 74%
-
final follow-up: central and north-west London 66%, South Wales 76%, North-East England 62%.
The median (first quartile, third quartile) time between consent and randomisation was 6.1 (2.3, 11.9) weeks, and was 2.0 (1.0, 3.9) weeks between randomisation and first therapy session (PEPS only).
The mean times between randomisation and the first follow-up was 8.0 (SD 5.1) weeks, second follow-up was 28.8 (SD 6.9) weeks and final follow-up was 80.3 (SD 10.1) weeks.
Baseline characteristics of randomised participants
Table 5 summarises the randomised groups at baseline. Of the 306 participants, 230 (75%) were women and the mean age was 38.2 years (SD 10.9 years). The only variable with a notable imbalance between the arms at baseline was type of PD, with a greater proportion in the PEPS arm defined as complex (60% compared with 49% in the usual-treatment arm).
| Variable | Treatment arm | |
|---|---|---|
| Usual treatment (n = 152) | PEPS (n = 154) | |
| Age at randomisation (years), mean (SD) | 37.8 (11.0) | 38.6 (10.9) |
| Sex, n (%) | ||
| Female | 115 (76) | 115 (75) |
| Male | 37 (24) | 39 (25) |
| Age left full-time education (years), mean (SD) | 16.9 (3.3) | 17.2 (3.7) |
| Highest educational attainment, n (%) | ||
| None | 29 (19) | 24 (16) |
| GCSE | 16 (10) | 22 (14) |
| A-level | 45 (30) | 35 (23) |
| Vocational | 10 (7) | 10 (7) |
| Degree | 32 (21) | 36 (23) |
| Other | 20 (13) | 25 (16) |
| Missing | 0 | 2 (1) |
| Ethnicity, n (%) | ||
| White | 127 (83) | 129 (84) |
| Mixed | 9 (6) | 6 (4) |
| Black Caribbean | 6 (4) | 5 (3) |
| Black African | 0 | 2 (1) |
| Black other | 2 (1) | 0 |
| Asian Indian | 1 (2) | 0 |
| Asian other | 1 (1) | 0 |
| Other | 6 (4) | 12 (8) |
| Socioeconomic status, n (%) | ||
| Never worked and long-term unemployed | 96 (63) | 105 (68) |
| Routine and manual occupations | 28 (18) | 20 (13) |
| Intermediate occupations | 13 (9) | 9 (6) |
| Managerial and professional occupations | 15 (10) | 20 (13) |
| IPDE type (definitive), n (%) | ||
| Paranoid | 16 (11) | 13 (8) |
| Schizoid | 1 (1) | 4 (3) |
| Schizotypal | 0 | 0 |
| Antisocial | 31 (20) | 23 (15) |
| Borderline | 90 (59) | 93 (60) |
| Histrionic | 6 (4) | 2 (1) |
| Narcissistic | 3 (2) | 1 (1) |
| Avoidant | 56 (37) | 57 (37) |
| Dependent | 7 (5) | 4 (3) |
| Obsessive–compulsive | 20 (13) | 14 (9) |
| PD not otherwise specified | 10 (7) | 14 (9) |
| PD,a n (%) | ||
| Simple PD | 77 (51) | 61 (40) |
| Complex PD | 75 (49) | 93 (60) |
Table 6 shows baseline demographic and outcome variables further categorised by follow-up at 72 weeks and trial arm. Participants who completed the trial were slightly older, more likely to be female and more likely to have never worked or be long-term unemployed. There was no strong suggestion that different types of participants were followed up in the two trial arms, although white participants were over-represented among completers in the usual-treatment arm, while those with PD classified as simple were over-represented among completers in the PEPS arm.
| Variable | Non-completer | Completera | ||
|---|---|---|---|---|
| Usual treatment (n = 58) | PEPS (n = 41) | Usual treatment (n = 94) | PEPS (n = 113) | |
| Age at randomisation (years), mean (SD) | 36.2 (10.9) | 37.0 (11.3) | 38.9 (11.0) | 39.1 (10.7) |
| Sex, n (%) | ||||
| Female | 38 (65) | 27 (66) | 77 (82) | 88 (78) |
| Male | 20 (35) | 14 (34) | 17 (18) | 25 (22) |
| Age left full time education (years), mean (SD) | 17.1 (3.8) | 16.4 (2.8) | 16.8 (2.9) | 17.5 (3.9) |
| Highest educational attainment, n (%) | ||||
| None | 11 (19) | 6 (15) | 18 (19) | 18 (16) |
| GCSE | 7 (12) | 4 (10) | 9 (10) | 18 (16) |
| A-level | 18 (31) | 12 (29) | 27 (29) | 23 (20) |
| Vocational | 5 (9) | 2 (5) | 5 (5) | 8 (7) |
| Degree | 12 (20) | 8 (19) | 20 (21) | 28 (25) |
| Other | 5 (9) | 7 (17) | 15 (16) | 18 (16) |
| Ethnicity, n (%) | ||||
| White | 44 (76) | 34 (83) | 83 (88) | 95 (84) |
| Mixed | 6 (11) | 4 (10) | 3 (3) | 2 (2) |
| Black Caribbean | 4 (7) | 1 (2) | 2 (2) | 4 (3) |
| Black African | 0 | 0 | 0 | 2 (2) |
| Black other | 2 (3) | 0 | 0 | 0 |
| Asian Indian | 0 | 0 | 1 (1) | 0 |
| Asian other | 0 | 0 | 1 (1) | 0 |
| Other | 2 (3) | 2 (5) | 4 (4) | 10 (9) |
| Socioeconomic status, n (%) | ||||
| Never worked and long-term unemployed | 32 (55) | 27 (66) | 64 (68) | 78 (69) |
| Routine and manual occupations | 13 (23) | 4 (10) | 15 (16) | 16 (14) |
| Intermediate occupations | 7 (12) | 4 (10) | 6 (6) | 5 (4) |
| Managerial and professional occupations | 6 (10) | 6 (14) | 9 (10) | 14 (13) |
| PD, n (%) | ||||
| Simple PD | 28 (48) | 13 (32) | 49 (52) | 48 (42) |
| Complex PD | 30 (52) | 28 (68) | 45 (48) | 65 (58) |
| Baseline SFQ score | ||||
| Mean (SD) | 14.9 (4.9) | 14.7 (4.1) | 14.0 (4.2) | 15.1 (4.0) |
| Median (first quartile, third quartile) | 15 (12, 19) | 14 (11, 18) | 14 (11, 17) | 15 (13, 18) |
| Baseline HADS score | ||||
| Mean (SD) | 27.1 (8.1) | 28.6 (5.7) | 27.7 (7.3) | 27.0 (7.7) |
| Median (first quartile, third quartile) | 28.5 (21, 33) | 28 (24, 33) | 28 (23, 34) | 28 (23, 32) |
| Baseline EQ-5D health status score | ||||
| Mean (SD) | 43.2 (23.9) | 44.3 (20.4) | 40.4 (23.5) | 44.6 (23.0) |
| Median (first quartile, third quartile) | 45 (25, 60) | 45 (30, 59) | 39.5 (20, 60) | 40 (30, 60) |
| Baseline SPSI-R score | ||||
| Mean (SD) | 6.8 (3.1) | 6.2 (3.1) | 6.9 (3.4) | 6.5 (3.0) |
| Median (first quartile, third quartile) | 6.2 (4.6, 8.6) | 6 (4.2, 8.2) | 7.2 (4.3, 9.1) | 6.6 (4.4, 8.6) |
| Baseline three-problem average score | ||||
| Mean (SD) | 8.7 (1.1) | 9.0 (1.0) | 8.7 (1.1) | 8.6 (1.1) |
| Median (first quartile, third quartile) | 9 (8, 9.7) | 9 (8.7, 10) | 9 (8, 9.7) | 8.7 (8, 9.3) |
Duration of follow-up
Follow-up was greater in the PEPS arm at every time point (see Figure 2). Overall, 68% of participants were followed up at the 72-week primary end point. In addition to a greater proportion of completers, duration of engagement with the trial among non-completers was greater in the PEPS arm (Table 7). This resulted in a total of 178 and 203 person-years of follow-up in the usual-treatment and PEPS arms, respectively.
| Time active in PEPS trial | Treatment arm | |||
|---|---|---|---|---|
| Usual treatment | PEPS | |||
| Non-completer (n = 58) | Completer (n = 94) | Non-completer (n = 41) | Completer (n = 113) | |
| Mean number of weeks (SD) | 30.0 (25.7) | 80.2 (9.8) | 36.7 (23.4) | 80.3 (10.4) |
Uptake of psychoeducation with problem-solving therapy
Table 8 summarises how much PEPS intervention was offered to, and attended by, trial participants allocated to receive it. Median attendance at the individual psychoeducation sessions was approximately 90%, dropping to around 50% for the problem-solving therapy group sessions. Seventy (45%) participants completed the PEPS intervention as per protocol. The size of PEPS problem-solving therapy groups formed at randomisation and followed up at 72 weeks is reported in Table 9, along with estimated intracluster correlation coefficients for primary and secondary outcomes.
| Session types | Number offered | Number attended | Proportion offered/attended |
|---|---|---|---|
| Psychoeducation sessions | |||
| Mean (SD) | 3.4 (0.7) | 3.0 (1.0) | 0.9 (0.3) |
| Median (first quartile, third quartile) | 3 (3, 4) | 3 (3, 4) | 1 (0.8, 1) |
| Problem-solving group sessions | |||
| Mean (SD) | 11.8 (0.8) | 5.2 (4.0) | 0.5 (0.3) |
| Median (first quartile, third quartile) | 12 (12, 12) | 6 (2, 9) | 0.6 (0.3, 0.8) |
| Number of participants offered problem-solving sessions, n (%) | 133 (100) | ||
| Non-attenders, n (%) | 20 (15) | ||
| Attended 1–5 sessions, n (%) | 43 (32) | ||
| High attenders (minimum six sessions), n (%) | 70 (53) | ||
| Individual support sessions | |||
| Mean (SD) | 4.2 (1.9) | 2.3 (1.8) | 0.5 (0.4) |
| Median (first quartile, third quartile) | 5 (3, 6) | 2 (1, 4) | 0.5 (0.2, 0.8) |
| Outcome at 72 weeks’ follow-up | Estimated intracluster correlation coefficient (95% CI) |
|---|---|
| SFQ | 0.07 (0 to 0.24) |
| HADS | 0.11 (0 to 0.29) |
| SPSI-R | 0.01 (0 to 0.17) |
| Three main problems | 0.01 (0 to 0.15) |
| Size of cluster in problem-solving group | |
| As formed on randomisation | |
| Number of groups | 18 |
| Median (25th quartile, 75th quartile) | 6.5 (6, 8) |
| Minimum, maximum | 5, 12 |
| Followed up at 72 weeks | |
| Number of groups | 18 |
| Median (25th quartile, 75th quartile) | 5 (5, 6) |
| Minimum, maximum | 1, 10 |
Primary outcome
Primary analysis
Based on observed data, the mean SFQ score at 24 weeks was 13.5 points (SD 4.5 points), and for usual-care and the PEPS arm was 13.8 points (SD 4.5 points); the corresponding mean scores at 72 weeks were 13.9 points (SD 5.3 points) and 13.5 points (SD 4.8 points), respectively. However, the primary analysis was conducted based on multiple imputation of missing data.
There was no evidence of any clinically important difference between the arms based on the primary analysis (Table 10). Across the whole sample, the mean difference in SFQ score between baseline and the 72-week follow-up was –0.94 points (95% CI –1.54 to –0.32 points).
| SFQ score | Treatment arm | Adjusteda difference in meansb | 95% CI | p-value | |
|---|---|---|---|---|---|
| Usual treatment (n = 152) | PEPS (n = 154) | ||||
| Baseline, meanc (SE) | 14.3 (0.4) | 15.0 (0.6) | – | – | – |
| 24 weeks, mean (SE) | 13.9 (0.4) | 13.7 (0.4) | – | – | – |
| 72 weeks, mean (SE) | 13.8 (0.5) | 13.5 (0.4) | –0.73 | –1.83 to 0.38 | 0.19 |
Sensitivity analyses
There was no strong evidence of a between-group difference in the primary outcome from any of the sensitivity analyses (Table 11). As expected, the estimated treatment effect of PEPS among compliers was greater than that from the primary analysis, as were analyses based only on participants with observed data at 72 weeks. The lower confidence limits for these analyses exceed the prespecified minimum clinically important difference in SFQ score of two points. Although the primary analysis excludes any effect, a clinically important benefit of treatment cannot, therefore, be completely ruled out, particularly among participants who undertake all of the psychoeducation treatment phase and at least six group problem-solving sessions.
| Analysis type | Adjusted difference in means | 95% CI | p-value |
|---|---|---|---|
| (1) Primary analysis (MI) | –0.73 | –1.83 to 0.38 | 0.19 |
| (2) As (1) with adjustment for baseline PD type | –0.72 | –1.78 to 0.34 | 0.18 |
| (3) As (1), CACE estimate | –1.60 | –3.96 to 0.76 | 0.18 |
| (4) Observed outcome data | –1.07 | –2.24 to 0.09 | 0.07 |
| (5) As (4), CACE estimate | –2.15 | –4.39 to 0.09 | 0.06 |
Blinding
Unblinding of the researcher was reported for 48 (23%) participants who provided SFQ scores at the 72-week follow-up (Table 12). Mean scores were similar among participants in both arms whether or not unblinding was reported. This is unsurprising given that the SFQ is self-completed by participants.
| SFQ at 72 weeks by unblinding of researcher at final follow-up | Treatment arm | |
|---|---|---|
| Usual treatment (n = 94) | PEPS (n = 113) | |
| Unblinded | ||
| Mean (SD) | 13.9 (4.2) | 14.2 (4.4) |
| n | 12 | 36 |
| Blinded | ||
| Mean (SD) | 13.9 (5.4) | 13.1 (4.9) |
| n | 82 | 77 |
Subgroup analyses
There was no evidence that the effect of PEPS compared with usual treatment differed according to study centre, category of PD at baseline or borderline PD diagnosis at baseline (Table 13).
| SFQ score | Treatment arm | Interaction coefficienta | 95% CI | p-value | |
|---|---|---|---|---|---|
| Usual treatment | PEPS | ||||
| 72 weeks by site | |||||
| Central and north-west London, meanb (SE) | 13.2 (0.8) | 13.0 (0.7) | |||
| South Wales, mean (SE) | 13.4 (0.9) | 13.7 (0.6) | –0.37 | –3.19 to 2.45 | 0.30 |
| North-East England, mean (SE) | 14.8 (0.7) | 13.9 (0.7) | –1.34 | –3.82 to 1.14 | |
| 72 weeks by PD category | |||||
| Simple, mean (SE) | 12.5 (0.6) | 13.1 (0.7) | 0.03 | –0.63 to 0.69 | 0.93 |
| Complex, mean (SE) | 15.1 (0.7) | 13.8 (0.5) | |||
| 72 weeks by borderline diagnosis at baseline | |||||
| Not borderline, mean (SE) | 14.6 (0.5) | 13.1 (0.5) | – | – | – |
| Borderline, mean (SE) | 12.5 (0.8) | 14.1 (0.7) | –0.98 | –3.24 to 1.27 | 0.39 |
Secondary outcomes
Hospital Anxiety and Depression Scale
Table 14 summarises the HADS total score at all outcome assessment visits, and the estimated between-group difference at 72 weeks. There was no evidence of any difference between PEPS and usual treatment, although the 95% CIs were wide and includes a potentially clinically important effect (i.e. 1.5 points) in either direction. 73
| HADS score | Treatment arm | Adjusteda difference in means | 95% CI | p-value | |
|---|---|---|---|---|---|
| Usual treatment | PEPS | ||||
| Baseline, mean (SD) | 27.5 (7.5) | 27.4 (7.2) | – | – | – |
| 5 weeks, mean (SD) | 25.4 (9.8) | 24.5 (7.9) | – | – | – |
| 24 weeks, mean (SD) | 25.1 (8.6) | 23.7 (8.7) | – | – | – |
| 72 weeks, mean (SD) | 22.3 (11.8) | 21.0 (10.9) | –1.2 | –4.2 to 1.8 | 0.44 |
Three main problems
Table 15 summarises the three main problems mean scores at each time point and the estimated between-group difference at 72 weeks’ follow-up. Table 16 shows reported change at each follow-up time compared with previous assessment, and ordinal logistic regression of rated change at 72 weeks. Mean scores were reduced in both arms over time and overall, 108 (55%) participants reported some improvement at 72 weeks but there was no evidence of any difference between PEPS and usual treatment in either measure.
| Three main problems score | Treatment arm | Adjusteda difference in means | 95% CI | p-value | |
|---|---|---|---|---|---|
| Usual treatment | PEPS | ||||
| Baseline, mean (SD) | 8.7 (1.1) | 8.7 (1.1) | – | – | – |
| 5 weeks, mean (SD) | 7.7 (2.3) | 7.8 (1.7) | – | – | – |
| 24 weeks, mean (SD) | 7.5 (2.1) | 7.1 (2.1) | – | – | – |
| 72 weeks, mean (SD) | 6.6 (2.8) | 6.3 (2.8) | –0.3 | –1.0 to 0.5 | 0.48 |
| Three main problems score | Treatment arm | Adjusteda odds ratio | 95% CI | p-value | |
|---|---|---|---|---|---|
| Usual treatment | PEPS | ||||
| 5 weeks, n (%) | N = 110 | N = 136 | |||
| Very much improved | 3 (3) | 1 (1) | |||
| Much improved | 11 (10) | 22 (16) | |||
| Minimal improvement | 21 (19) | 59( 43) | |||
| No change | 29 (26) | 35 (26) | |||
| Minimally worse | 16 (14) | 4 (3) | |||
| Much worse | 14 (13) | 2 (1) | |||
| Very much worse | 6 (5) | 3 (2) | |||
| 24 weeks, n (%) | N = 93 | N = 116 | |||
| Very much improved | 5 (5) | 9 (8) | |||
| Much improved | 10 (11) | 26 (22) | |||
| Minimal improvement | 26 (28) | 38 (33) | |||
| No change | 24 (26) | 23 (20) | |||
| Minimally worse | 8 (9) | 7 (6) | |||
| Much worse | 12 (13) | 6 (5) | |||
| Very much worse | 6 (6) | 3 (2) | |||
| 72 weeks, n (%) | N = 90 | N = 108 | |||
| Very much improved | 8 (9) | 13 (12) | |||
| Much improved | 17 (19) | 27 (25) | |||
| Minimal improvement | 19 (21) | 24 (22) | |||
| No change | 19 (21) | 16 (15) | 0.65 | 0.38 to 1.12 | 0.12 |
| Minimally worse | 3 (3) | 3 (3) | |||
| Much worse | 10 (11) | 11 (10) | |||
| Very much worse | 7 (8) | 4 (4) | |||
Global Assessment of Functioning
Global Assessment of Functioning was included as a means of collecting data on the referring clinicians’ rating of change during the trial. However, during the course of the trial it became apparent that GAF completion rates were poor. The reasons for this were explored with site staff and identified as:
-
participants being discharged from secondary care services and therefore there not being a suitable clinician involved in their care to provide this rating
-
participants in services may not have a regular care team/care co-ordinator
-
even for participants for whom a care co-ordinator/regular clinical team is available, GAF scores are difficult to obtain. One reason for the low returns is resource issues within mental health services generally (e.g. budget cuts and job losses), which make any non-essential activities difficult to complete. Another may be lack of familiarity with the rating tool.
Continued efforts were made to obtain a rating from the original referrer or other member of the participant’s clinical team. The TMG agreed in advance that a minimum of 80% completion rate at baseline and 50% completion at follow-up would be required to enable a valid analysis of GAF data to be made. These rates were not achieved.
Mechanism of change
Working Alliance Inventory – Short Revised
The WAI examines the development of treatment alliance and was used to assess the effectiveness of the psychoeducation component in developing treatment alliance. The WAI is intended to be completed by clients and therapists. In the trial, the therapist form was completed by the PEPS therapist for participants allocated to PEPS therapy and a member of the participants’ usual-care team for participants receiving treatment as usual only. Participants were asked to complete the WAI client form in relation to the working alliance with the relevant clinician. However, for the same practical reasons as listed above for the GAF, completion rates were poor and did not reach the prespecified level required for a valid analysis.
Social Problem-Solving Inventory – Revised: short version
Table 17 summarises the SPSI-R total score at all outcome assessment visits and the estimated between-group difference in total score based on a repeated measures analysis that includes data from both 24 and 72 weeks’ follow-up. There was no evidence of any difference between PEPS and usual treatment. Analyses for each of the five SPSI-R subscales follow the same approach and found no evidence of any treatment effect (data not shown).
| SPSI-R score | Treatment arm | Repeated measures | |||
|---|---|---|---|---|---|
| Usual treatment | PEPS | Adjusteda difference in means | 95% CI | p-value | |
| Baseline, mean (SD) | 6.9 (3.3) | 6.4 (3.0) | – | – | – |
| 24 weeks, mean (SD) | 8.0 (3.6) | 7.7 (3.5) | – | – | – |
| 72 weeks, mean (SD) | 8.2 (4.0) | 8.0 (4.0) | 0.14 | –0.53 to 0.82 | 0.68 |
Service use
Table 18 summarises participants’ use of health services during the trial. Data were available for 140 (91%) and 126 (83%) participants in the PEPS and usual-treatment arms, respectively. There were 2041 contacts with health services from 135 individuals in the PEPS arm, compared with 2064 from 116 individuals allocated to usual treatment. When converted to annual rates, this equated to a mean of approximately 10 contacts per person per year. There was no evidence of any differences between groups in total health-service use, use of emergency or crisis services, or whether classified as scheduled or unscheduled.
| Service use | Treatment arm | Adjusteda IRR | 95% CI | p-value | |
|---|---|---|---|---|---|
| Usual treatment (N = 126) | PEPS (N = 140) | ||||
| Overall service use, n contacts (n individuals) | |||||
| Scheduled | 921 (87) | 970 (106) | |||
| Unscheduled | 604 (80) | 547 (94) | |||
| Unclassified | 539 (93) | 524 (118) | |||
| Total | 2064 (116) | 2041 (135) | |||
| GP contacts, n contacts (n individuals) | 1649 (80) | 1521 (98) | |||
| Emergency department, n contacts (n individuals) | 151 (47) | 142 (55) | |||
| Hospital admissions, n contacts (n individuals) | 48 (26) | 83 (34) | |||
| Hospital outpatient appointment, n contacts (n individuals) | 8 (5) | 19 (11) | |||
| Mental health service, n contacts (n individuals) | |||||
| Crisis resolution service | 30 (20) | 55 (28) | |||
| A&E liaison service | 6 (6) | 12 (11) | |||
| Other mental health service | 172 (106) | 209 (120) | |||
| Overall service use for emergency department, hospital admission, crisis resolution and A&E liaison service, n contacts (n individuals) | |||||
| Scheduled | 34 (27) | 49 (29) | |||
| Unscheduled | 195 (54) | 230 (65) | |||
| Unclassified | 6 (4) | 13 (10) | |||
| Total | 235 (85) | 292 (104) | |||
| Overall service use, n contacts (n individuals) | |||||
| 1–9 | 52 (45) | 51 (38) | |||
| 10–19 | 32 (27) | 50 (37) | |||
| 20–29 | 10 (9) | 20 (15) | |||
| 30–39 | 9 (8) | 7 (5) | |||
| > 40 | 13 (11) | 7 (5) | |||
| Overall service use (annual rate) during 72 weeks after randomisation, mean (SD) | 10.6 (11.0) | 9.9 (14.3) | 0.97 | 0.77 to 1.22 | 0.81 |
| Overall service use for emergency department, hospital admission, crisis resolution and A&E liaison service (annual rate), mean (SD) | 2.5 (3.1) | 2.4 (3.2) | 0.97 | 0.66 to 1.41 | 0.87 |
| Scheduled service use (annual rate), mean (SD) | 6.5 (7.4) | 6.0 (5.5) | 0.91 | 0.69 to 1.21 | 0.54 |
| Unscheduled service use (annual rate), mean (SD) | 4.4 (6.2) | 3.7 (4.3) | 0.87 | 0.60 to 1.26 | 0.47 |
When unclassified health service contacts were reassigned as either scheduled or unscheduled, there was a higher proportion of scheduled contacts in the PEPS arm than in the usual treatment arm (Table 19). However, 95% CIs were wide and included the null, indicating no strong evidence that participants in the two groups differed in scheduled use of health services.
| Assignment of service use type | Treatment arm | Adjusteda risk ratio | 95% CI | p-value | |
|---|---|---|---|---|---|
| Usual treatment | PEPS | ||||
| Overall service use if unclassified assigned as scheduled, n health service contacts (%) [n individuals] | |||||
| Scheduled | 1460 (71) [117] | 1494 (73) [134] | |||
| Unscheduled | 604 (29) [80] | 547 (27) [94] | 0.99 | 0.93 to 1.06 | 0.77 |
| Overall service use if unclassified assigned as unscheduled, n health service contacts (%) [n individuals] | |||||
| Scheduled | 921 (45) [87] | 970 (47) [106] | |||
| Unscheduled | 1143 (55) [108] | 1071 (53) [125] | 1.05 | 0.92 to 1.19 | 0.46 |
Adverse events
Twenty-eight individuals reported a total of 35 adverse events following referral to the PEPS trial and before randomisation. Of these, 14 participants were allocated to each arm, with 14 adverse events occurring in participants allocated to PEPS and 21 in participants allocated to usual treatment. Following randomisation, a total of 76 adverse events were reported by 39 individuals allocated to usual treatment and 117 events were reported by 60 individuals allocated to PEPS (Table 20).
| Adverse event report categories | Treatment arm | |
|---|---|---|
| Usual treatment (N = 152) | PEPS (N = 154) | |
| Participants with any adverse event, n (%) | 39 (26) | 60 (39) |
| All adverse event reports,a n events (n individuals) | ||
| Hospitalisation | 62 (33) | 100 (51) |
| Deathb | 0 | 4 (4) |
| Other | 14 (12) | 13 (12) |
| Total | 76 (39) | 117 (60) |
| Participants with multiple hospitalisations, n (%) | ||
| 1 | 19 (58) | 29 (56) |
| 2 | 6 (18) | 13 (26) |
| 3 | 3 (9) | 3 (6) |
| 4 | 4 (12) | 2 (4) |
| 5 | 0 | 1 (2) |
| 6 | 1 (3) | 0 |
| 7 | 0 | 2 (4) |
| 9 | 0 | 1 (2) |
| Mental health-related adverse event reports, n events (n individuals) | ||
| Planned/respite hospital admission | 2 (2) | 1 (1) |
| Self-harm including drug or alcohol overdose | 27 (16) | 42 (24) |
| Suicide or attempted suicide | 4 (3) | 7 (7) |
| Deterioration in mental health | 3 (2) | 9 (7) |
| Suicidal ideation | 6 (6) | 8 (6) |
| Other | 3 (3) | 4 (4) |
| Total | 45 (26) | 71 (37) |
| Participants with multiple self-harm reports, n (%) | ||
| 1 | 11 (69) | 16 (67) |
| 2 | 2 (12) | 3 (12) |
| 3 | 1 (6) | 1 (4) |
| 4 | 1 (6) | 3 (12) |
| 5 | 1 (6) | 1 (4) |
Four deaths were reported during the study: two were due to natural causes and two were suicides. All four participants were allocated to the PEPS arm. The nature and circumstances of these deaths did not seem related to the therapy.
During the trial, one serious adverse event was reported that had a suspected relationship to the trial and was not listed in the protocol as an expected occurrence. This was a non-fatal overdose resulting in an overnight hospital admission, which was precipitated by a number of stressors including anxiety about group work in relation to forthcoming PEPS therapy. This event was reported to the Research Ethics Committee, DMEC and TSC. A briefing was issued to all site staff reminding them of the importance of identifying anxiety in relation to group work and supporting participants with this, as appropriate.
Table 20 provides further details of hospitalisations, self-harm, and other mental health-related adverse events.
Table 21 shows the results of between-group analyses of reported adverse events. There were approximately 13% more participants in the PEPS arm who reported at least one adverse event, equivalent to an approximate 50% relative increase. However, when the difference in total follow-up between the arms is accounted for, this figure of 50% was attenuated to approximately 30%, with 95% CIs that span the null. Results of the analysis of total number of adverse events were similar, with an approximately 20% greater reported event rate in the PEPS arm, with a 95% CI that includes the null.
| Adverse events | RD, % (95% CI) | RR (95% CI) | IRR (95% CI) | |||
|---|---|---|---|---|---|---|
| Crude | Adjusteda | Crude | Adjusteda | Crude | Adjusteda | |
| Based on number of individuals (usual treatment, n = 39; PEPS, n = 60) | 13.3 (2.9 to 23.7) | 13.0 (2.9 to 23.1) | 1.52 (1.09 to 2.12) | 1.52 (1.09 to 2.13) | 1.35 (0.89 to 2.07) | 1.37 (0.91 to 2.05) |
| Based on number of adverse events (usual treatment, n = 76; PEPS, n = 117) | – | – | – | – | 1.35 (1.00 to 1.83) | 1.24 (0.93 to 1.64) |
Chapter 5 Health economics analysis
Objectives
Any new intervention requires the use of scarce health-care resources and so conducting economic evaluations to assess cost-effectiveness is crucial. Economic evaluations of interventions for PD are few in number, although cost-effectiveness analyses of specific therapies, psychotherapy more generally and whole programmes have been conducted. 74–76 The objectives of the economic evaluation in this study were:
-
to measure and describe the 6-month health-care and societal costs (with a detailed breakdown of care, benefits, lost employment, etc.) of participants in the PEPS trial at randomisation, and their association with clinical and demographic characteristics
-
to measure, describe and compare patterns of service utilisation by participants at second follow-up (post treatment, 24–26 weeks), third follow-up (week 40) and fourth follow-up (week 72) across the two groups (PEPS and usual treatment)
-
to compare care costs (health care and societal) at second, third and fourth follow-up
-
to assess the cumulative cost–utility of PEPs compared with usual treatment at second, third and fourth follow-up.
Methods
Analyses were detailed in an economics analysis plan, which was finalised prior to completion of data collection and database lock.
Use of health services, employment effects and welfare benefits
Comprehensive data were collected on all health, social care and other relevant services used by individual study members using a tailored version of the CSRI. 60 The CSRI was used at baseline and at second, third and fourth follow-ups, with information provided via participant self-report. The baseline interview covered resource use for the previous 6 months, while the subsequent interviews covered resource use since the last interview. The CSRI included the following broad categories of information:
-
contacts with health- and social-care professionals
-
use of day care services
-
use of inpatient care
-
contacts with criminal justice/legal services
-
medication
-
education and time absent from studies
-
employment and time off work
-
welfare benefits.
Cost calculations
The costs of each resource item were calculated using the best available unit cost estimates (see Appendix 12) and reported in 2012/13 prices. The cost of the PEPS intervention was estimated using information on the core resources required to deliver the individual and group sessions, and estimating specific costs for those inputs (see Appendix 13). Costs were calculated using data on the number of therapy sessions received by each participant. Lost employment costs for those in employment will be calculated by combining time off work with daily earnings. The value of benefits received was calculated but these are transfer payments and so were not included with the service or lost employment costs.
Quality-adjusted life-years
The EQ-5D was used to measure health-related quality of life. This consists of five domains (mobility, self-care, usual activities, pain/discomfort and anxiety/depression). Each of these received a score of 1, 2 or 3 corresponding to ‘no problems’, ‘moderate problems’ and ‘major problems’. Utility scores were attached to each health state based on these scores (a table of utility values has been produced by the Centre for Health Economics, University of York77). These utility scores were used to generate QALY gains over the follow-up period using area-under-the-curve methods and assuming a linear change between any two adjacent time points.
Analyses
Cost data are frequently skewed and this can lead to incorrect p-values and CIs when making comparisons between groups. To address this, we used a bootstrapped regression model, which makes no assumptions about the data distribution. Baseline data were analysed using a regression model to identify variables significantly associated with cost. If PEPS resulted in higher costs and better outcomes, then incremental cost-effectiveness ratios were to be computed. Uncertainty around these was explored using cost-effectiveness planes. To construct these, four regression models were run using 10,000 bootstrapped resamples. The models used service costs and QALYs at second and final follow-up as the dependent variables. The independent variables were the group identifier and the baseline measure of cost or utility score. The 10,000 coefficients for the group identifier variable are 10,000 estimates of the cost/outcome differences and these were plotted against each other. The planes show the probability of the PEPS group having (i) lower costs and better outcomes, (ii) higher costs and better outcomes, (iii) lower costs and worse outcomes, and (iv) higher costs and worse outcomes than supportive care alone. In addition, cost-effectiveness acceptability curves were generated using the net benefit approach in order to determine the likelihood that the intervention was the most cost-effective option. There is a theoretical, but unknown, value (represented by the term λ) that society would place on a 1-unit QALY gain as measured by the EQ-5D. Net benefit can be defined as:
where NB is net benefit, E is effectiveness (i.e. QALY gains compared with baseline) and SC is service costs. For example, if for a particular patient the QALY gained during the follow-up period is 0.05 and if their service cost is £400, then we can calculate their net benefit if we know λ. If λ = £0 then the net benefit is –£400, whereas if λ = £10,000 then the net benefit is £100. Net benefits for all patients were estimated by assuming different values for λ ranging between £0 and £50,000 in £5000 increments. A regression model was then used to determine the mean difference in net benefit between usual-treatment and PEPS groups for every value of λ. For each model, 10,000 regression coefficients for the group identifier variable were generated using bootstrapping and the proportion of these that were greater than zero indicated the probability that PEPS was cost-effective (i.e. it resulted in a mean incremental net benefit greater than zero). These probabilities were subsequently used to generate the cost-effectiveness acceptability curves.
Results
Service-use data are summarised in Table 22. In the 6 months before baseline assessment, over 90% of patients had GP contacts and these occurred approximately every month. Over half also had contacts with psychiatrists and around one-quarter of patients in both groups had inpatient admissions. Most other services were used by relatively few patients. Overall, there were few noticeable differences between the two groups at baseline, although the group randomised to PEPS did have more contacts with home treatment/crisis team members.
| Service | Time point | |||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 6 months to baseline | Baseline to second follow-up | Second to third follow-up | Third to final follow-up | |||||||||||||
| Usual treatment | PEPS | Usual treatment | PEPS | Usual treatment | PEPS | Usual treatment | PEPS | |||||||||
| n (%) | Mean (SD) contactsa | n (%) | Mean (SD) contactsa | n (%) | Mean (SD) contactsa | n (%) | Mean (SD) contactsa | n (%) | Mean (SD) contactsa | n (%) | Mean (SD) contactsa | n (%) | Mean (SD) contactsa | n (%) | Mean (SD) contactsa | |
| GP | 111 (94) | 7.6 (10.4) | 122 (90) | 6.4 (5.5) | 85 (72) | 5.9 (5.3) | 102 (76) | 6.2 (7.7) | 86 (73) | 5.3 (5.8) | 80 (59) | 5.6 (6.2) | 80 (68) | 7.7 (8.1) | 91 (67) | 8.1 (11.6) |
| Psychiatrist | 79 (67) | 3.1 (5.5) | 100 (74) | 2.7 (3.4) | 53 (45) | 3.1 (2.8) | 55 (41) | 2.4 (1.9) | 49 (42) | 2.4 (2.4) | 48 (36) | 2.3 (1.5) | 49 (42) | 4.2 (5.1) | 62 (46) | 2.7 (3.1) |
| Other doctor | 29 (25) | 3.7 (5.1) | 35 (26) | 3.3 (4.4) | 25 (21) | 3.6 (8.1) | 33 (24) | 3.3 (3.7) | 16 (14) | 2.1 (1.8) | 18 (13) | 4.1 (4.8) | 19 (16) | 3.6 (2.8) | 27 (20) | 4.7 (10.4) |
| Psychologist | 22 (19) | 4.0 (5.5) | 22 (16) | 5.6 (5.6) | 15 (13) | 8.1 (8.4) | 15 (11) | 5.3 (5.8) | 14 (12) | 8.9 (7.3) | 8 (6) | 8.1 (6.0) | 14 (12) | 10.4 (11.0) | 16 (12) | 7.7 (9.1) |
| Drug and alcohol advisor | 12 (10) | 5.9 (6.3) | 17 (13) | 9.4 (9.8) | 4 (3) | 6.3 (4.6) | 11 (8) | 20.7 (33.4) | 7 (6) | 19.9 (19.5) | 12 (9) | 13.3 (27.9) | 6 (5) | 16.8 (12.4) | 5 (4) | 6.6 (6.6) |
| Other counsellor/therapist | 21 (18) | 10.1 (8.5) | 27 (20) | 10.0 (9.9) | 13 (11) | 9.5 (10.2) | 19 (14) | 15.3 (26.7) | 14 (12) | 8.2 (7.0) | 17 (13) | 9.5 (8.5) | 10 (8) | 19.6 (17.1) | 22 (16) | 14.4 (14.2) |
| Home treatment/crisis team member | 34 (29) | 7.7 (9.5) | 56 (41) | 5.3 (6.7) | 26 (22) | 6.3 (10.5) | 32 (24) | 7.1 (10.4) | 17 (14) | 8.0 (19.9) | 17 (13) | 6.7 (7.2) | 18 (15) | 5.7 (5.1) | 23 (17) | 5.3 (4.9) |
| Assertive outreach team member | 2 (2) | 25.5 (2.1) | 1 (1) | 8.0 (–) | 1 (1) | 10.0 (–) | – | – | 1 (1) | 10.0 (–) | 1 (1) | 26.0 (–) | – | – | 1 (1) | 3.0 (–) |
| Early intervention team member | 3 (3) | 7.7 (3.8) | 2 (1) | 1.0 (0.0) | – | – | – | – | 2 (2) | 7.0 (4.2) | – | – | – | – | – | – |
| Social worker | 37 (31) | 6.4 (8.1) | 41 (30) | 8.2 (9.8) | 20 (17) | 14.3 (18.1) | 33 (24) | 4.9 (4.8) | 21 (18) | 6.8 (5.8) | 26 (19) | 6.9 (7.6) | 19 (16) | 6.7 (5.9) | 22 (16) | 9.0 (8.7) |
| Mental health nurse | 43 (36) | 9.8 (8.2) | 53 (39) | 10.8 (11.7) | 32 (27) | 9.9 (8.1) | 38 (28) | 9.8 (9.3) | 27 (23) | 10.4 (12.1) | 29 (21) | 5.4 (4.4) | 28 (24) | 9.3 (7.3) | 30 (22) | 14.6 (19.7) |
| Occupational therapist | 20 (17) | 7.3 (9.5) | 14 (10) | 6.1 (6.8) | 9 (8) | 10.2 (20.7) | 9 (7) | 5.3 (7.1) | 4 (3) | 2.8 (1.3) | 7 (5) | 2.7 (1.3) | 4 (3) | 11.0 (16.0) | 5 (4) | 3.0 (1.2) |
| Accident and emergency service | 46 (39) | 5.0 (17.5) | 56 (41) | 1.9 (1.4) | 27 (23) | 2.2 (2.6) | 34 (25) | 2.8 (5.9) | 22 (19) | 2.0 (1.1) | 29 (21) | 1.8 (1.8) | 33 (28) | 2.3 (1.8) | 28 (21) | 2.6 (3.9) |
| Drug/alcohol service | 2 (2) | 11.0 (12.7) | 7 (5) | 25.9 (43.0) | – | – | 4 (3) | 15.5 (18.1) | 2 (2) | 42.0 (25.5) | 4 (3) | 9.5 (4.9) | 1 (1) | 60.0 (–) | 2 (1) | 4.0 (2.8) |
| Community mental health centre | 9 (8) | 13.4 (11.3) | 9 (7) | 23.7 (46.8) | 4 (3) | 15.5 (13.1) | 6 (4) | 7.0 (7.5) | 3 (3) | 26.3 (21.5) | 3 (2) | 6.0 (2.0) | 6 (5) | 23.7 (20.9) | 2 (1) | 13.5 (17.7) |
| Day care centre/day hospital | 5 (4) | 17.2 (19.2) | 6 (4) | 18.0 (28.3) | 9 (8) | 21.4 (26.9) | 5 (4) | 10.6 (10.7) | 7 (6) | 16.0 (16.4) | 4 (3) | 46.0 (51.2) | 6 (5) | 24.3 (38.1) | 4 (3) | 27.5 (31.6) |
| Drop-in centre | 9 (8) | 31.7 (44.2) | 17 (13) | 26.7 (38.2) | 10 (8) | 15.5 (19.1) | 18 (13) | 10.9 (20.9) | 8 (7) | 27.1 (25.0) | 10 (7) | 15.6 (16.8) | 7 (6) | 33.3 (26.2) | 7 (5) | 26.6 (35.7) |
| Self-help/support group | 12 (10) | 21.0 (36.0) | 15 (11) | 18.1 (35.9) | 7 (6) | 9.1 (9.9) | 10 (7) | 29.0 (46.0) | 7 (6) | 18.7 (13.7) | 7 (5) | 12.4 (11.3) | 9 (8) | 7.7 (8.5) | 11 (8) | 9.1 (8.6) |
| Class/group at a leisure centre | 8 (7) | 24.5 (17.7) | 9 (7) | 13.3 (13.4) | 8 (7) | 26.9 (32.6) | 6 (4) | 17.2 (22.7) | 6 (5) | 15.2 (12.7) | 5 (4) | 9.8 (8.0) | 3 (3) | 7.7 (4.0) | 5 (4) | 5.4 (2.3) |
| Adult education class | 10 (8) | 8.5 (7.4) | 20 (15) | 19.0 (24.0) | 7 (6) | 13.4 (6.8) | 9 (7) | 22.4 (39.9) | 10 (8) | 15.6 (15.8) | 11 (8) | 18.1 (34.6) | 10 (8) | 11.4 (10.7) | 6 (4) | 20.5 (16.9) |
| Other day care activity provided by team | 10 (8) | 14.8 (13.3) | 10 (7) | 13.1 (14.8) | 3 (3) | 25.0 (2.6) | 4 (3) | 19.0 (11.9) | 7 (6) | 17.0 (15.8) | 5 (4) | 17.2 (11.3) | 3 (3) | 11.3 (12.9) | 2 (1) | 30.0 (5.7) |
| Inpatient admissionb | 31 (26) | 1.5 (0.9) | 35 (26) | 1.5 (0.9) | 17 (14) | 1.4 (0.6) | 25 (19) | 1.4 (0.9) | 13 (11) | 1.6 (1.0) | 15 (11) | 1.3 (0.6) | 14 (12) | 1.5 (0.9) | 20 (15) | 1.5 (0.9) |
During the period between baseline assessment and second follow-up, the vast majority of patients again had contact with GPs. In both groups, there was a reduction in contact with GPs, psychiatrists, home treatment/crisis team members, social workers and mental health nurses. Fewer were admitted to hospital. The only key difference between groups was in the use of a social worker, with 24% of patients in the PEPS group having contact, compared with 17% of patients in the usual-treatment group. However, the amount of contact with social workers was much higher in the usual-treatment group than the PEPS group, indicating a greater intensity of use.
Between the second and third follow-ups, there was a continued reduction in contact with home treatment/crisis team members and mental health nurses in both groups. Contact with GPs, psychiatrists and social workers was reduced in the PEPS group, but this trend was not observed in the usual-treatment group. Between the third and final follow-ups, there was a slight increase in contact with GPs, psychiatrists and home treatment/crisis team members in the PEPS group, which also saw a small increase in inpatient care. No noticeable changes in service use were observed in the usual-treatment group.
Mean service costs are presented in Table 23. At baseline, mean costs were highest for inpatient care, followed by contact with psychiatrists, GPs and social workers. Inpatient and GP costs were higher for the usual-treatment group, and the psychiatrist and social worker costs were higher for the PEPS group. However, the variation around the means was substantial, as indicated by the SDs. Total health-care costs were significantly higher for women than for men by an average of £2044 and were higher by an average of £5431 if the patient had a non-specified PD.
| Service | Time point | |||||||
|---|---|---|---|---|---|---|---|---|
| 6 months to baseline | Baseline to second follow-up | Second to third follow-up | Third to final follow-up | |||||
| Usual treatment | PEPS | Usual treatment | PEPS | Usual treatment | PEPS | Usual treatment | PEPS | |
| GP | 465 (1489) | 291 (380) | 267 (599) | 238 (430) | 198 (383) | 138 (231) | 289 (616) | 233 (477) |
| Psychiatrist | 472 (627) | 595 (729) | 396 (684) | 298 (482) | 359 (696) | 274 (481) | 479 (980) | 434 (895) |
| Other doctor | 76 (247) | 86 (239) | 98 (390) | 83 (205) | 24 (76) | 93 (397) | 78 (324) | 92 (341) |
| Psychologist | 89 (326) | 117 (406) | 135 (528) | 75 (333) | 133 (463) | 64 (316) | 143 (553) | 106 (492) |
| Drug and alcohol advisor | 32 (142) | 64 (249) | 11 (73) | 91 (581) | 64 (349) | 64 (478) | 46 (244) | 13 (92) |
| Other counsellor/therapist | 107 (335) | 140 (445) | 62 (267) | 146 (725) | 61 (235) | 79 (309) | 103 (446) | 150 (510) |
| Home treatment/crisis team member | 52 (167) | 52 (145) | 39 (260) | 44 (197) | 17 (100) | 17 (69) | 22 (78) | 20 (74) |
| Assertive outreach team member | 26 (203) | 1 (17) | 4 (45) | 0 (0) | 4 (45) | 14 (165) | 0 (0) | 1 (6) |
| Early intervention team member | 4 (28) | < 1 (3) | 0 (0) | 0 (0) | 3 (26) | 0 (0) | 0 (0) | 0 (0) |
| Social worker | 374 (1106) | 455 (1296) | 658 (2956) | 187 (520) | 253 (859) | 266 (879) | 234 (1160) | 284 (1189) |
| Mental health nurse | 212 (436) | 210 (424) | 172 (448) | 153 (413) | 130 (386) | 63 (179) | 139 (351) | 168 (529) |
| Occupational therapist | 49 (205) | 38 (239) | 33 (265) | 9 (42) | 3 (17) | 4 (18) | 28 (282) | 3 (14) |
| Accident and emergency service | 243 (1392) | 96 (159) | 63 (190) | 89 (396) | 47 (114) | 48 (137) | 81 (176) | 67 (257) |
| Drug/alcohol service | < 1 (4) | 3 (22) | 0 (0) | 1 (7) | 3 (27) | 5 (49) | 1 (10) | 1 (10) |
| Community mental health centre | 4 (27) | 22 (236) | 2 (11) | 14 (148) | 9 (85) | < 1 (1) | 41 (319) | 1 (6) |
| Day care centre/day hospital | 2 (10) | 6 (37) | 7 (38) | 3 (28) | 4 (21) | 5 (40) | 7 (45) | 3 (22) |
| Drop-in centre | 10 (64) | 28 (237) | 5 (26) | 7 (34) | 6 (28) | 3 (19) | 50 (504) | 4 (34) |
| Self-help/support group | 6 (39) | 59 (387) | 1 (7) | 4 (24) | 37 (378) | 1 (6) | 5 (32) | 4 (20) |
| Class/group at a leisure centre | 3 (18) | 8 (70) | 2 (12) | 2 (12) | 24 (252) | 5 (50) | 4 (37) | 4 (39) |
| Adult education class | 2 (7) | 31 (251) | 3 (16) | 4 (23) | 13 (100) | 18 (121) | 3 (20) | 5 (29) |
| Other day care activity provided by team | 18 (159) | 10 (77) | 1 (9) | 2 (18) | 2 (14) | 1 (6) | 3 (25) | 1 (7) |
| Inpatient admission | 1930 (4662) | 1600 (4815) | 1430 (5835) | 847 (3239) | 552 (2262) | 413 (2099) | 982 (4455) | 1314 (8139) |
Between baseline and the second follow-up, mean costs were again highest for the use of inpatient care, contacts with psychiatrists, GPs and social workers. For all of these four services, costs were higher in the usual-treatment group than in the PEPS group; the cost difference was especially high for inpatient care (usual treatment, £1430; PEPS, £847) and contacts with social workers (usual treatment, £658; PEPS, £187). Between the second and third follow-ups, the costs of inpatient care, psychiatrists and GPs were still higher for the usual-treatment group. Costs of social worker contacts were similar between the groups. Finally, between the third and final follow-ups, costs of psychiatrists and GPs were higher for the usual-treatment group, while the costs of inpatient care and social workers were higher for the PEPS group.
The total cost of all health-care services at baseline was £265 higher in the usual-treatment group than in the PEPS group (Table 24). This cost difference widened at the second and third follow-ups in favour of PEPS; however, the 95% CI shows the difference to be non-significant. By the final follow-up, the mean costs were £1174 higher for the usual treatment group than for the PEPS group, after adjusting for differences in baseline costs. However, the SDs were large and this difference was also non-significant.
| Health-care service use period | Treatment arm | Difference adjusted for baseline (95% CI) | |
|---|---|---|---|
| Usual treatment | PEPS | ||
| 6 months to baseline | 4178 (5780) | 3913 (5834) | – |
| Baseline to second follow-up | 3391 (7255) | 2296 (4081) | –1051 (–2470 to 367) |
| Baseline to third follow-up | 5334 (8434) | 3870 (5692) | –1391 (–3112 to 331) |
| Baseline to fourth follow-up | 8072 (10,598) | 6777 (10,550) | –1174 (–3720 to 1371) |
When the cost of the PEPS intervention was considered, the total costs for the PEPS group increased to £7727 per patient for the entire follow-up period, which is still lower than the total cost of the usual-treatment group (£8072 per patient). After adjustment for baseline, the cost for the PEPS group was £182 less than for the usual-treatment group.
At the 72-week follow-up, the SFQ score (the primary clinical outcome) is slightly in favour of the PEPS group, although the difference is not statistically significant (p = 0.19). Therefore, in a technical sense, PEPS was seen to be dominant as it resulted in lower service costs and an improved SFQ score. However, this does not take uncertainty around these estimates into account.
The utility values derived from the EQ-5D are shown in Table 25. At baseline assessment and at the second follow-up, the mean utility scores were slightly higher in the usual-treatment group. At the final follow-up, the mean utility score was slightly higher in the PEPS group. However, none of these differences was statistically significant.
| Assessment time point | Treatment arm, mean utility score (SD) | p-value | |
|---|---|---|---|
| Usual treatment | PEPS | ||
| At baseline | 0.3647 (0.3664) | 0.3422 (0.3505) | 0.6194 |
| At second follow-up | 0.4050 (0.3529) | 0.3731 (0.3709) | 0.4868 |
| At final follow-up | 0.3825 (0.3718) | 0.3895 (0.3668) | 0.8802 |
Quality-adjusted life-years generated using the area under the curve method are shown in Table 26. The mean QALY gain between baseline and second follow-up was 0.2161 for the usual-treatment group and 0.2108 for the PEPS group. However, the difference adjusting for baseline utility score was 0.0557 in favour of the PEPS group. By the final follow-up, the QALY gain in the usual-treatment group was 0.5706 and 0.5616 in the PEPS group. The adjusted difference was 0.0148 in favour of PEPS.
| Assessment time point | Treatment arm, mean QALY (SD) | Difference adjusted for baseline utility (95% CI) | |
|---|---|---|---|
| Usual treatment | PEPS | ||
| Baseline and second follow-up | 0.2161 (0.1961) | 0.2108 (0.2158) | 0.0057 (–0.0215 to 0.0328) |
| Baseline and final follow-up | 0.5706 (0.4624) | 0.5616 (0.5142) | 0.0148 (–0.0622 to 0.0919) |
In a technical sense, PEPS was seen to be dominant as it resulted in lower total costs and a greater QALY gain, after adjusting for baseline. However, this does not take uncertainty around these estimates into account.
The cost-effectiveness plane showing cost (including service costs for both groups and intervention cost for PEPS group only) and QALY differences at second follow-up is shown in Figure 3. At the second follow-up, 62.4% of the resamples showed lower costs and a greater QALY gain for the PEPS group while 30.6% of resamples showed lower costs and a lower QALY gain. Only 4.8% of the resamples showed higher costs and more QALYs for PEPS, and 2.2% showed higher costs and a lower QALY gain. The cost-effectiveness plane for the QALY gain and cost differences by final follow-up is presented in Figure 4. The most likely scenario with 36.8% of resamples was that PEPS resulted in lower costs and greater QALYs. Higher costs and more QALYs were revealed in 28.1% of resamples. Lower costs and fewer QALYs were shown by 17.9% of resamples and higher costs and fewer QALYs by 17.2%. Clearly, the level of uncertainty is substantial over the entire follow-up.
FIGURE 3.
Cost-effectiveness plane of cost and QALY differences at the second follow-up.
FIGURE 4.
Cost-effectiveness plane of cost and QALY differences at the final follow-up.
The cost-effectiveness acceptability curve for the whole follow-up period is shown in Figure 5. Even if society was not willing to attach any monetary value to QALY gains there would remain a likelihood of 58% that PEPS is the most cost-effective option. As a unit improvement is valued at higher levels, this likelihood increases. Although we do not know the true societal value that should be placed on a QALY gain, NICE uses a threshold in the region of £20,000–30,000. Between these two levels there is a 63.6–64.2% likelihood that PEPS is the more cost-effective option.
FIGURE 5.
Cost-effectiveness acceptability curves showing the probability that PEPS is most cost-effective option.
The above analyses focus on health-care costs. However, it is likely that people with PD will incur higher cost of criminal justice service as a result of their condition. Two types of criminal justice service were considered in this analysis: police contact and nights detained in a police cell. The use and costs of both services are shown in Tables 27 and 28. Costs of police contact were similar in the two groups. Although relatively few patients spent nights detained in a police cell, the cost was high. It can be seen that the cost was greatest for the PEPS group before baseline assessment (£211 per patient), but this cost was reduced during the follow-up period.
| Service | Time point | |||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 6 months to baseline | Baseline to second follow-up | Second to third follow-up | Third to final follow-up | |||||||||||||
| Usual treatment | PEPS | Usual treatment | PEPS | Usual treatment | PEPS | Usual treatment | PEPS | |||||||||
| n (%) | Mean (SD) contactsa | n (%) | Mean (SD) contactsa | n (%) | Mean (SD) contactsa | n (%) | Mean (SD) contactsa | n (%) | Mean (SD) contactsa | n (%) | Mean (SD) contactsa | n (%) | Mean (SD) contactsa | n (%) | Mean (SD) contactsa | |
| Police contact | 31 (26) | 1.8 (1.2) | 45 (33) | 3.1 (5.2) | 19 (16) | 1.9 (1.8) | 23 (17) | 2.9 (6.1) | 13 (11) | 3.4 (5.6) | 18 (13) | 2.6 (3.2) | 16 (14) | 4.0 (4.8) | 22 (16) | 2.9 (3.7) |
| Nights detained in a police cell | 4 (3) | 1.8 (1.5) | 10 (7) | 6.9 (18.7) | 1 (1) | 1.0 (–) | 2 (1) | 1.0 (0.0) | 1 (1) | 6.0 (–) | 2 (1) | 4.0 (4.2) | – | – | 2 (1) | 5.0 (5.7) |
| Service | Time point | |||||||
|---|---|---|---|---|---|---|---|---|
| 6 months to baseline | Baseline to second follow-up | Second to third follow-up | Third to final follow-up | |||||
| Usual treatment | PEPS | Usual treatment | PEPS | Usual treatment | PEPS | Usual treatment | PEPS | |
| Police contact | 16 (33) | 34 (110) | 10 (34) | 17 (90) | 12 (69) | 12 (48) | 18 (73) | 16 (60) |
| Nights detained in a police cell | 25 (165) | 211 (2137) | 4 (38) | 6 (50) | 21 (229) | 25 (252) | – | 31 (322) |
Information on the number of days lost from work as a result of health problems (absenteeism) and the cost impact of absence from work is given in Tables 29 and 30. The data relate only to those in employment, the assumption being that, given the level of unemployment in the economy, long-term absence from work does not result in lost output because others are employed instead. Lost work time was experienced by a small number of patients, but the costs were substantial. The lost employment costs were also similar between groups at baseline. These costs then fell for the PEPS group, but remained stable for the usual-treatment group.
| Service | Time point | |||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 6 months to baseline | Baseline to second follow-up | Second to third follow-up | Third to final follow-up | |||||||||||||
| Usual treatment | PEPS | Usual treatment | PEPS | Usual treatment | PEPS | Usual treatment | PEPS | |||||||||
| n (%) | Mean (SD) lost days | n (%) | Mean (SD) lost days | n (%) | Mean (SD) lost days | n (%) | Mean (SD) lost days | n (%) | Mean (SD) lost days | n (%) | Mean (SD) lost days | n (%) | Mean (SD) lost days | n (%) | Mean (SD) lost days | |
| Lost work | 19 (16) | 39.2 (58.4) | 19 (14) | 44.2 (56.0) | 16 (14) | 14.4 (23.7) | 13 (10) | 40.6 (71.0) | 12 (10) | 25.6 (41.4) | 9 (7) | 5.7 (4.9) | 12 (10) | 22.3 (28.4) | 10 (7) | 10.1 (7.4) |
| Cost | Time point | |||||||
|---|---|---|---|---|---|---|---|---|
| 6 months to baseline | Baseline to second follow-up | Second to third follow-up | Third to final follow-up | |||||
| Usual treatment | PEPS | Usual treatment | PEPS | Usual treatment | PEPS | Usual treatment | PEPS | |
| Production lossa | 664 (2848) | 654 (2698) | 205 (1032) | 411 (2567) | 274 (1566) | 40 (196) | 238 (1158) | 79 (344) |
Conclusion
The analyses presented in this chapter have focused on costs and cost-effectiveness. Over the follow-up period, costs were slightly lower in the PEPS group. There was substantial uncertainty in this result, as shown by the wide CIs. Utility scores derived from the EQ-5D were very low (average scores were typically around 0.4). The analyses demonstrated that QALY gains were very similar in the two groups, but with a very slight advantage in favour of the PEPS intervention after controlling for baseline differences. Technically, the PEPS intervention was ‘dominant’ in that it resulted in lower average costs and greater QALY gains. However, the uncertainty around both estimates means that, at a NICE threshold of £20,000–30,000, there is a 64% likelihood that the intervention is the more cost-effective option. Given all the information available, PEPS is somewhat more cost-effective but much caution is required in interpreting this result. We have not considered the logistics of implementation and that may outweigh the benefits from the intervention.
Chapter 6 Qualitative analysis
Objectives
The purpose of this analysis was to search for themes across the PEPS qualitative data set, which consisted of reports compiled by researchers of participant responses to semistructured interviews, conducted according to interview schedules developed for the study (see Appendices 14–17). These data were gathered at:
-
first follow-up (after psychoeducation), on the effects of psychoeducation (n = 87)
-
second follow-up (after problem-solving therapy), on the effects of problem-solving therapy (n = 61)
-
final follow-up (at 72 weeks), on experiences of usual treatment (n = 41) or PEPS therapy (n = 47).
The data were analysed using a thematic analysis. 78 Thematic analysis was chosen because of its ability to richly describe a large number of data by drawing out key points and presenting them in an easily accessible format,78 and it has been used to investigate engagement issues in people with PD. 79
Theoretical position
A pragmatic, flexible, inductive, thematic analysis was undertaken to provide largely face-value feedback on participant experience of the interventions, as described above. All semistructured interviews (n = 236) were included in this analysis. As the primary aim of the thematic analysis was to meet the objectives of the PEPS study, the focus was to provide a rich and nuanced reflection on participants’ experiences of usual treatment and PEPS as well as to explore, in as much detail as the data permitted, their perceived benefits or bad effects of the treatments under investigation.
Method
The interview transcripts were read by two researchers (ML and JD), who familiarised themselves with the entire data set being analysed. ‘Best’ answers to the research questions were captured in the form of selected excerpts or quotes as an initial step in the analysis. Through an iterative process, these quotes were organised into recurring patterns or themes. Finally, the themes identified separately were merged or differentiated and checked to ensure that there was good evidence for them.
In doing this, the following ‘lenses’ influenced the selection process: the notion of ‘prevalence’ (how frequently the idea occurred across the data set was an organising principle, although with the caveat that more instances does not necessarily mean that they are more important); and a notion of ‘keyness’ (the extent to which a quote or excerpt captured something important to the research question).
Themes
Feedback after psychoeducation
Psychoeducation followed the use of the IPDE to ensure that the individual was eligible for the study, and consisted of up to four sessions of an individual collaborative dialogue designed to build rapport with participants, inform them about their PD diagnosis and discuss its effects on interpersonal relationships and social functioning. The analysis found five themes pertaining to the individual psychoeducation sessions, which are summarised in Box 1 and then elaborated.
-
Collaborative one-to-one process.
-
Provided improved knowledge of traits and understanding of individual relevance and application.
-
Emotional impact and willingness to tolerate this and keep with the process.
-
Positive orientation: optimism, trust and motivation.
-
Barriers to progress: doubts and uncertainties.
Theme 1: a collaborative one-to-one process
In general, participants felt that psychoeducation had been a professional, constructive, well-structured discussion about their diagnosis and its implications for their social functioning and problem-solving.
I’ve found them useful to help me identify some of my patterns, and also enjoyable. Put things into context.
01/098
The [psychoeducation] sessions were simple to understand, not overelaborate – using simple down to earth speak. Everyone would benefit from learning.
03/080
Although I have been in therapy a long time and inpatient in various institutions and also seen many psychiatrists over the last 16 years this is the first time I’ve talked to anyone about my diagnosis. In therapy we talked about me and occasionally make reference to it. With psychiatrists it is always about symptoms. This has been reassuring to properly talk to someone about BPD [borderline PD] – a new context. Psychoeducation is definitely the main factor, without that phase the group sessions would not have worked. People would not have returned to the group sessions without the initial one to one.
01/099
Being supportive and somebody really listening to your feelings helps me feel secure and worthy as a person, that someone takes the time to listen.
02/009
I have become more curious and intrigued about my diagnosis. Finding out more about me. I can recognise and rationalise others’ behaviour and personality, so feel more able to understand them. I’m trying to apply this to me where I can.
03/024
Good to talk about things and have things explained.
01/148
I’ve got hope that I can deal with some of the things we have discovered in the sessions.
02/014
. . . contact with someone who was interested and understood the problems [was helpful]. I learned about PD generally and about the relation to my own problems and how they affect my relationship with others and my reactions to problems.
03/27
We learnt that I’ve got a PD, which has just been categorised as depression – hard to put into words. Found it helpful to breakdown what my problems are.
02/099
Theme 2: increased or new knowledge of the core concept of ‘trait’ (versus symptom), and a greater understanding of its relevance and application to the individual themselves
Participants had engaged in a meaningful process of self-understanding and were given a specific conceptual framework that made sense to them. This brought relief and a sense that they could apply this knowledge to their recovery:
Learned how my past has affected my personality.
03/01
I don’t feel as isolated; I’m not the only one receiving a PD diagnosis.
03/60
Learnt about my own traits and issues and how it effects me personally.
03/041
[IPDE] was very thought-provoking . . . as I answered some of the questions, I re-evaluated the scenario, situation.
03/04
Maybe I had in my mind I had a million problems and I’ve learnt it’s all caused by the need to have things perfect and get everything done. The trait is causing me many problems, I’m being irrational. It seems more manageable looking at it here, in this way.
02/046
Eye opener, empowering. I learned about my own traits. I am beginning to change . . . I learned that I am overly dependent.
03/62
I’ve got hope I can deal with some of the things discovered in the [IPDE] sessions.
02/14
. . . pin-pointing traits was helpful.
01/098
It was handy to be informed about other personality disorders.
01/099
What my traits are, the healthy ones and the not so healthy ones.
02/062
It’s not a specific thing with your personality, it’s different things.
02/108
A lot about myself, established ‘triggers’ for my behaviours. Learned that I have the skills to help myself.
03/04
Everyone has a personality and that disorder happens when a person experiences, perceives, copes or manages with issues around their feelings and emotions.
03/110
More about personality disorder. Interesting to have my personality disorder traits reflected back to me. Reassuring conversation relating to borderline personality disorder.
01/099
The fact that there are so many different types of personality disorder. I was able to put another perspective on my problems. I always thought it was coming from my depression. I’d never thought about it in relation to personality before.
01/120
Theme 3: emotional impact and willingness to keep with the process
Participants described the impact of facing and coming to terms with a diagnosis of PD. They described their own emotional responses to it and their concerns about how others might perceive and treat them.
Things had happened in our lives wherein we were let down by the system – and [but] the onus was put back onto ourselves for this . . . being described as having a personality disorder. We had failed in the first instance.
01/133
Triggered some historical memories about [my] father. I have not ‘switched off’ like I normally do – I have felt more focused.
03/82
Session 2 [psychoeducation] made me tearful. It felt like having the truth slapped in my face. Was overwhelming seeing all the boxes I had actually ticked. My mum has been a great support whilst in this study. I have got to spend more time with her, which brought us closer and she understands me better.
01/38
. . . I’ve also found it very hard and difficult, especially after sessions dealing with my emotions.
02/014
It’s been emotionally draining, but not negative. Feeling that I am not alone has been the biggest thing. Spending time with the research team and trusting the person doing the psychoeducation.
01/42
It was emotionally painful at times because it brought up some issues from the past that were painful – but I was able to deal with them.
03/027
Starting to admit things . . . helping me confront stuff. Helped me pinpoint issues. I have said I didn’t want to come but that’s only confrontation and I know it’s going to be okay.
01/47
After [psychoeducation] sessions you can feel quite vulnerable and confused about what you’ve talked about.
02/014
You are talking about things that are painful, but its part and parcel of working through an issue. No pain, no gain.
02/009
Has felt intense and hard. Has helped me to see things differently.
03/029
Daunting at first. I’ve found it difficult to say about all the bad bits . . .
02/029
Anxious, frightened initially about the questions. Still anxious but not as frightened.
03/093
When the truth hit me it made me feel emotional – I find emotions hard to deal with.
03/062
Overwhelming . . .
03/11
Theme 4: positive orientation – optimism, trust and motivation
Overall, the detailed IPDE assessment and psychoeducation sessions helped to prepare people and orient them for the next step. Many speakers articulated a positive motivation to build on their skills:
That there is hope, it’s not mysterious. What personality disorder is and the common aspects that affect lives. That I can do something about it.
03/123
When you speak out loud about things it makes it real and it makes you look at it, made me want to change it and make it better. I can make better choices.
02/098
Started to think about different ways of dealing with problems.
03/01
Learned that I have the skills to help myself.
03/04
Helped me really think about what things are a problem for me. Useful having that fed back to me. Some questions make me think of things I haven’t thought of before. How my natural personality is holding me back. Become more comfortable that I am not the only one with these problems. [Improved motivation by] showing me what I can get out of the sessions.
01/51
. . . things aren’t so bad as I thought they were.
02/056
Take a step back and look at what I thought were major problems, aren’t at all. Feel confident about sharing that [in problem-solving groups to follow].
01/46
It’s made me less apprehensive about going to somewhere new and talking to new people. It’s good I’ve met you [therapist] before the groups start or you’re meeting new people in the group and new therapists at the same time, it can be daunting.
02/026
I feel more motivated to join the group process.
03/39
Looking forward to problem-solving.
03/01
Theme 5: barriers to progress – doubts and uncertainties
Some participants found the information difficult to assimilate. The psychoeducation had not met their expectations. These misgivings or mismatches in their sense of what constituted meaningful help were barriers to progress.
I don’t know how it [psychoeducation] is helping me, I don’t. It released tension. I put it in my mind in the beginning that I didn’t fancy it.
02/015
I’ve not really understood them. I thought it was counselling, but it’s not really . . .
01/094
I’ve been unsure of the structure of the individual sessions, not sure what to expect . . .
01/124
. . . I thought that the personality statements were badly worded – slightly too open to interpretations – a bit vague.
01/099
Not really [benefited], just good to get started with some therapy.
01/102
I didn’t learn anything.
01/102
. . . I am a little confused about whether I learnt anything extra to what I already knew.
03/078
Can’t quite remember.
01/117
I felt bad at the time. Didn’t always agree with what was being said. Hard sometimes to recall the information.
03/03
Can’t remember most things.
03/37
Going through everything and seeing it in black and white, had feeling of being weak.
02/099
They weren’t long enough. There are just too many things that stress me to discuss in the sessions – need to be longer.
01/150
Learning more about myself and feeling worried.
02/108
It is not known if participants felt these impressions could have been raised and addressed during the course of the individual psychoeducation sessions. These follow-up data suggest that they were not resolved. This early feedback of potential difficulties could have been invaluable in engaging successfully with participants who were having difficulties and helping to address them.
Feedback after problem-solving therapy
Problem-solving therapy was a 12-session group intervention designed to teach people strategies for solving interpersonal problems. In addition, individual support sessions were also made available to participants. Six themes were found in this analysis, which are summarised in Box 2 and elaborated on below.
-
Positive experience.
-
Limited engagement.
-
Learning outcomes.
-
Vulnerabilities triggered – feeling exposed and unsatisfied.
-
Peer support seen as a positive resource.
-
Needed more sessions to master new skills.
Theme 1: positive experience
Many participants found problem-solving therapy to be a structured, positive, relevant intervention for them. They found that it was non-threatening and that by applying it they could take effective action successfully. Several responses evidence the fact that many people had grasped the method and felt encouraged about applying it.
It was a good course overall.
02/055
Just that I really enjoyed it. It helped me a lot, but you only get out what you put into it. You need to remember to use the skills.
02/008
Not what I expected it to be. First group I’ve stayed ‘switched on’ for. I learned how to stand back and think about something rather than just react.
03/82
I found it very beneficial to use those sessions [problem-solving]. The way it was presented I’ll give it 9/10.
02/023
. . . I think its been the most helpful thing I’ve had since been involved in mental health services. Pitched well as it doesn’t assume too much about the person beforehand.
02/026
What you’re doing is brilliant. I think the course should be longer, like a rolling course or open door. Once a week having an open door for people with problems to talk.
02/065
I think the group should be available more widely.
01/117
I really miss the place and coming here.
01/150
Managed to attend groups consistently despite feeling stressed.
03/003
I like the fact that it wasn’t about me, the fact anything can be broken down into smaller pieces using the steps.
01/098
Valuable experience, I learnt to mediate my responses, to learn to step back and just listen, not comment or criticise.
01/099
It’s reinforced that by going through a process again and again it helps and you can get better at it.
01/110
I’ve learnt that groups aren’t as scary as they seem. And the fact that so many options come up and it is helpful.
02/040
That I can control my impulsivity, that the glass is not always half empty, that I can be my own master, I just need to work on my skills and techniques.
02/008
[I] feel more confident in solving problems and feel capable of managing difficulties in a structured way.
03/039
[I learnt] to solve problems in a different way. Something more structured – easy to follow. I recognise the build up of a problem and can ID [identify] how I’m feeling then stop and work things out.
03/013
If you can stick at it long enough it gets better. Realise how it works and that everyone is there for similar reasons and not feeling like an individual in a group.
01/104
My general opinion is it’s something you’ve got to want to participate in. No use sitting there and having no input. Good to have other’s input and I’ve gained more confidence.
01/033
It was OK – didn’t think I’d like it at first but it was OK. Good idea and good to know there are things that can help.
03/020
Had impact on me as it made me use my mind more – thinking of options.
03/029
Absolutely brilliant. Although difficult to participate in some of them, I felt the actual problem-solving process was very, very useful.
02/023
It stopped me overdosing, it made me stop and think about things.
02/027
[The process was] helpful, I liked the fact that the group wasn’t big and the way we dealt with one thing at a time.
02/029
I think it brings a lot of mentally ill people together and gives them a bond in a group. It’s a good way to come together to bring experiences together.
02/052
It’s a lot different than I thought; I was expecting it to more emotional but it’s been relaxed and informal and I’ve got on with people better than I thought.
02/064
This was an encouraging and remarkable endorsement by a clinical population that is frequently challenging to engage. There is also implicit confirmation that the PEPS therapists were providing a valuable treatment that was not already routinely available. It looked as if for some participants it contained a collaborative approach to defining personal problems and setting priorities for intervention that had been identified as lacking in treatment plans, based purely on diagnostic assessments by clinicians. 80
Theme 2: limited engagement
Another robust finding of this analysis was the theme of limited engagement in problem-solving therapy, which seemed to include timing of the problem-solving therapy, resistance to the group format, sustaining the commitment required to complete the course, sometimes speakers felt that participants could not ‘relate’ to their problem or vice versa, and a deterioration in mental state.
I would have got more out of it at a different time. I wasn’t 100% there as I had so much going on in my head. If I had the chance I would like to try it again when all other things aren’t going on.
02/073
I am lucky that it wasn’t a group really, mostly just me who attended so . . .
01/84
I have learned a lot about myself, but due to physical health problems I’ve not got as much as I’d like to due to not being able to attend group often. Helpful in future to keep the six steps in mind. Can use it better when not it crisis – have been using problem-solving in recent separation of marriage.
03/01
Groups weren’t for me and I feel I’m missing out ‘cos I still need help. Didn’t like talking about other people – they could listen to me and think I’m a bad character.
02/015
All female group can go off on a tangent. Shame that it wasn’t better attended – lost people over course of the group.
01/57
May not have needed to be 12 weeks as same thing each week and group became smaller with time . . . Useful when people related to your problem as a group. Sometimes if people did not relate to my problem or made certain comments, it didn’t help, made me feel alone. But not always.
01/49
Maybe just that I found I couldn’t work with it over the last couple of weeks as I was in a really ‘bad’ place and couldn’t focus on it. I just couldn’t engage in it when I was feeling at my worst.
03/13
It was getting to me because I wanted to go, but found it difficult to be around the group and I felt bad then that I wasn’t attending.
02/068
I’m not sure I liked the group. I didn’t like that individuals were under the ‘spot light’ to bring their problems.
01/86
I could have gained more out of it if it hadn’t been at the wrong time and the problems with transport didn’t help. Things that were said and done in the group [had a bad effect]. I also felt inferior in the group because they sometimes used big words.
02/073
I did enjoy coming to the group, but my depression came in the last few weeks and that’s why I didn’t make it to sessions – wasn’t myself.
02/053
I didn’t give it a chance.
02/015
I didn’t like the fact that each week was about one person’s problem. I wanted more opportunity to talk about my problems. I did relate to some of the other’s problems, but not to all.
01/38
I found it hard going that there were so many sessions. I think a break in the middle of 2 weeks would have been good.
02/014
It seems likely that an obvious implication of this theme of insufficient engagement with a core component of treatment was a lack of mastery of the skills required for independent and effective application of problem-solving therapy to complex real-life problems.
Theme 3: learning outcomes
There was a strongly supported theme that problem-solving therapy did result in learning outcomes that participants could articulate for themselves as evidenced in the following quotations:
I’m a lot more calmer, I don’t lash out as much now. I learnt to talk about things rather then bottling things up then falling apart. I’ve now learnt I can talk things through, but I don’t always talk them through with the right people.
01/94
I feel as though I got a new set of tools which I can use on a daily basis – I know I am not a freak!
02/830
Strategies for my goals and ambitions. Learning alternatives to self-harm. Being reminded of importance of planning sleep and meals. Suggestions to keep diaries was useful. Also indirectly learned from others what they had learned. Printouts helpful so you could remember things. Learned about other resources – Crisis Line.
01/124
I’ve got in touch with Gofal [a Welsh mental heath and well-being charity] and other agencies to help me get back into work and have support in place.
02/023
The way we deconstructed the problem in stages was good, it took the excess away – by breaking it down and intervening gently it made the problem seem more manageable. By looking at things objectively it took away the emotional charge/shit feelings/scariness of the problem. I really benefited from learning this tool, I try to implement it as much as I can.
01/78
Even though it has been so hard, by hanging on through that day and letting it run its course, you can turn negative thoughts into positive thoughts. After 54 years I am finally starting to have positive thoughts – even if it’s just one to get me through the day. Nobody has given us the tools before. I have hope in my heart.
02/52
Really good philosophy and tools. It can be difficult and can fall by the wayside – need self-discipline. I learned about myself and how others perceive things – I was able to work out things logically – learnt a structure [for] soul searching. I realise I am a bit of an attention/reassurance seeker. I recognise I can manipulate to get my own way and how this can affect others. I can use Stop & Think more positively. The more I use Stop & Think, the easier I’ll find it. I want to be able to incorporate it into everyday life.
03/36
. . . a fusion of individual CBT [cognitive–behavioural therapy] in a group setting, but it wouldn’t have worked without the supplementary sessions . . . the group alone does not hold it. By listening to other group member’s problems and how they react to them . . . that was a learning curve for me to see myself in others. Hard journey to start a path of acceptance and self-compassion. I learned to mediate my responses, to learn to step back and just listen . . . usually my impulses are very quick and I’m out there without thinking . . .
01/99
. . . made me use my mind more – thinking of options. Group was caring and understanding. Good using six steps – put a different light on it. Made me try different things rather than just giving up. Understood more about self and built my confidence to ask questions and challenge others/offer others support. I want to use it more than I do, but that’s up to me to practise it so it feels easier to engage automatically.
03/29
These quotes provide convincing indications that adult learners had been engaged actively in meaningful learning with unequivocal signs of skills being brought within the participants’ reach. Traits associated with self-direction were being nurtured, including being methodical and disciplined, logical and analytical, collaborative and interdependent, open, as well as increasingly motivated and confident, as is consistent with the work of Candy. 81
Theme 4: vulnerabilities triggered – feeling exposed and unsatisfied
This theme related to having difficult feelings and negative social interactions, making unfavourable comparisons with others and being unable to reconcile the dissonance between what they expected and what problem-solving therapy actually involved. There were some indications of the socioeconomic or cultural differences in which their own lifestyle seemed far removed from the problems being addressed. For a few, it was too contrived, formal and academic.
After [problem-solving therapy] group taking feelings home with you with no one to share them with. Listening to other people’s problems, felt overwhelmed. Good to feel not alone, but sometimes it felt too much. Walked away with a lot spinning around in your head. Wanting to forget a lot of it. I felt I was the one with the problem (like when the psychiatrist wrote an inaccurate report). Nowhere else to go. Still lacked confidence. Some people dropped off. Wondering what happened to them. Concerned they were alright.
01/133
I didn’t like it at all. I’m a private person and didn’t want my dirty laundry to be aired. It was not for me. It blew my mind and I couldn’t face it. It benefited me in opening my eyes and knowing it’s not for me.
02/15
Sometimes I didn’t think any one listened to me and I would get upset . . .
01/148
Sessions would confuse my head coming from my lifestyle. My lifestyle is quite rough. I also struggle with being a bit quiet. And sometimes people having a laugh and I felt I wasn’t part of it. I needed to contribute, express more [but could not].
01/150
Groups shouldn’t be run over the Christmas months, should start in the new year as Christmas is distressing for people so they might/do miss lots of groups over the weeks due to being unwell. Should have been in a nicer environment, the room was too small. The research is a bit formal like it was a test or too academic.
01/104
The intensity of the group [had bad effects]. I started drinking a lot. It consumed my thoughts. I think if I hadn’t had other therapeutic [one-to-one] support during this time I would have made suicide attempts.
01/099
Made me regress, [I] know you (therapist) were correct, but I couldn’t change. Confirmed how I felt already. I felt depressed.
01/121
Some emotional factors, practical approach and the steps sometimes means you don’t have the facility to talk to a lot about why problems have come about. Would be best to have the groups alongside a more emotional focused group.
01/133
. . . as soon as the groups finished things went downhill again.
03/072
They were good but I don’t think it was good how it was set up. It was too planned with someone bringing a problem the following week, situation of ‘it’s not my turn’ to talk about my problem. So you are thinking of your own problem when someone is talking about theirs. The steps were too long. Sometimes you didn’t know your problem or what you wanted so you’d spend a long time on that bit and it dragged it out too much and I lost concentration . . .
(01/094)
Theme 5: peer support seen as a positive resource
The content of this theme related to overcoming negative stereotypes, vicarious learning from peers, being less judgemental, and overcoming social isolation.
Realised that others who have this diagnosis and that people with BPD [borderline PD] are not monsters. It was helpful to meet other people with BPD who are employed . . . I had a lot of personality issues at work and it is hard to keep a job. It was useful to meet other people who have personality disorders and can keep a job.
01/104
I’ve enjoyed it. Meeting people who are the same as me.
02/027
I feel I got a lot emotionally out of the group. I have been using my skills and I don’t feel angry if it doesn’t work, I just try something else. I am now part of a support group made up of ‘stop and think’ group members.
03/082
Huge meeting others with personality disorder, having others care, listen, take notice. Learnt to be less judgemental of others.
03/011
I found it reassuring that I wasn’t alone in the way I dealt with things. Others have problems too and being able to share. Got good helpful ideas off others.
03/013
Met a member of the group which developed into a friendship. Good to know I’m not on my own, others could identify with things I was saying, vice versa.
03/020
Sharing of info with other group members. Feel less isolated, ability to look at things differently breaking the cycle.
03/023
Sharing experiences with others and getting their support. Would feel more confident in engaging in other group work.
03/076
I made friends and learnt how to listen and negotiate with other group members.
03/083
I liked everyone in the group, I enjoyed it sociably, I liked the people very much.
01/098
Being in a group with people helped me be more sociable in my personal life.
01/100
Rewarding that when I put in options some people thought they were really good options. And some people came back to the group and said it worked, rewarding to be listened to.
01/133
It was nice being with people with different experiences – the social aspects. I feel more comfortable making small talk, having done it in a safe environment. Other people come up with things that you don’t think of.
02/026
Listening to others stories I can relate to, can put a context into it without needing to talk about it. Having another view point of comparison, having someone say ‘yes I can relate to this’.
02/064
The main thing I’ve learnt is that other people are dealing with similar problems and using a technique that works.
02/065
[I learnt] that there are options to problems. I was expecting mad people in the group but actually they were all quite normal which made me feel less threatened in a group setting.
01/104
Group was understanding, good using the six steps.
03/029
Theme 6: needed more sessions to master new skills
Many participants felt that more opportunities to master social problem-solving skills were needed. They wanted to be able to repeat the course and were worried that, after the course was over, there would be nothing to replace it.
What you’re doing is brilliant – I think the course should be longer, like a rolling course of open door. Once a week, having an open door for people with problems to talk.
02/65
More sessions as just warming up.
03/37
If I had the chance, I’d like to try it again when . . . other things aren’t going on.
02/73
Group could be longer. Just got used to people and was only half-way there when it stopped.
03/038
I hope we can have somewhere to go like this along the way in the future.
02/52
Found support sessions 1 : 1 valuable.
03/01
My only worry is that when this is all gone, I’m left with nothing again and that scares me.
02/32
Themes related to psychoeducation with problem-solving therapy based on feedback at 72-week follow-up
The following six themes were found in the interview data on final reflections by participants on PEPS therapy. They are summarised in Box 3 and then examples of quotes follow.
-
PEPS is a valid treatment.
-
Both psychoeducation and problem-solving therapy are necessary parts of it.
-
Peer support and not being the only one with difficult problems was reassuring.
-
Barriers to PEPS working.
-
Consistency of staff and process important for security and sustained participation.
-
Lack of ‘dose’ – needed more sessions and supervised practice.
Theme 1: psychoeducation with problem-solving is a valid treatment
Participants found PEPS to be credible, to mobilise their own resources, to provide positive, collaborative experiences with staff and peers, and to provide hope of adopting a self-directed approach to improving their lives.
Really good, well structured. Different depending on who got [participates] in it. Very useful for me. Skills based versus analysis, analytic introspective based. I think this is better for people with a personality disorder. Skills based model really useful. Differing and varying opinions useful because people had similar disorders, you could compare issues. Strategies for my goals and ambitions. Learning alternatives to self-harm. Being reminded of importance of planning sleep and meals. Suggestions to keep diaries was useful. Also indirectly learned from others what they had learned. Printouts helpful so you could remember things. Learned about other resources – Crisis Line.
01/124
I think everyone who was offered the chance to go to PEPS therapy doesn’t know how lucky they are. To go there and know what’s wrong with you and understand why you’re different and that you’ve got to work towards improving yourself. People don’t understand mental health, if you keep it in your head what you’ve learnt here – remember the people you met here and one part of the program will keep you alive.
02/052
[PEPS was] very good, it explained a lot to me.
02/058
Living again, not just existing, PEPS saved my life!
03/004
I hope it gets rolled out at the end of the study.
03/023
A lot easier to be in and keep employment when one has tools to manage . . . Also [applies to] everyday life. I’ve had long history of self-harming. Now when I get these urges, I dismiss them. Before, I felt I had to cut myself. Now I can step back, it’s worked for me. When born, not given a handbook. By listening to others [in PEPS] who have same problem and trying what’s worked for them, it’s helpful. Amazing what a little care and compassion can do. To hear another person say, ‘I can understand how you feel’ means a lot . . . Helped me be more open with my wife. I feel confident knowing she cares about me. If I have a [bad] day, I can tell her. Before would have bottled that up and would have felt like cutting myself. I used to wear a mask, now more open and honest. Know people care about me. No longer embarrassed about who or what I am.
01/127
I did have a benefit. I was thinking in a different way, stopped putting myself down as much and though more positively.
01/084
Felt like I could help other people who were going through things that I’d been through. It’s changed my whole attitude. If I think about all of my overdosing, it just seems like selfishness and not being prepared to deal with what I get myself into.
02/027
More confident dealing with things. Deal with things a lot better now than previously; used to have lots of meltdowns. Confidence to deal with things better and be around strangers.
02/029
Having ill people together, we came up with better answers – I think we need each other. Still need each other, need courses like PEPS, can mend each other, laugh and feel more positive. I think I deal with things much better since the course.
02/052
No longer making the same mistakes again and again, now coming up new counselling/solutions to [my] problems
02/056
Helped with social, recently discovered I have Asperger’s.
02/064
They [therapists] were lovely, really patient and understanding. Doctors normally make me feel crap and like they are looking down their nose at me. It was really important that PEPS people didn’t make me feel like that.
02/029
It’s a way of life now. Thank you for changing my life. It’s been an amazing experience and I’m very grateful for everyone who played a part in helping me get better.
02/056
Although I have given Q7 [rate the usefulness of PEPS] as a ‘1’, it is because I’ve let it slip. I do think it is really useful and works. I just need to find a way of reminding myself.
01/117
Learnt how to talk to people comfortably and overcome paranoia of people judging you.
03/001
These responses demonstrate that participants experienced PEPS therapy as combining interesting knowledge of their difficulties with practical coping skills and positive attitudes towards their future recovery.
Theme 2: both psychoeducation and problem-solving were necessary parts of psychoeducation with problem-solving
This theme demonstrated that many participants had grasped the rationale for PEPS and valued both component parts of it.
I think they work together, they complement each other – you need both. Whole structure very helpful.
01/098
Liked the individual ones – felt I could open up better one to one. Group sessions were fine, liked listening to other problems. I’ve never been good in large groups, was nice that it was a small group. Both parts useful in different ways, useful to have both, worked well together.
02/009
Both worked well together both equally useful.
02/020
Both useful and worked well together.
03/004
The two come hand in hand. If I had to chose, when everyone works in the group together and sharing our experiences was the best part.
02/052
Went hand in hand, couldn’t have one without the other.
02/056
Worked well together. The more educated you are the more able you are to deal with your problems.
02/062
Both worked well. One to one gave me confidence to go into the groups, confidence got me ready for the group.
03/001
Worked well together. One to one brought me outside myself and helped me understand myself which helped me face going to the groups.
03/013
I like being one-to-one, more direct and just about me; but then in the group It helps you to see other people’s views and hear other people who are worse off. [Both] useful for different things, worked well side-by-side. Also nice to have opportunity for individual support sessions.
02/004
Preferred the group because you realised you’re not the only one that behaves like that. Help each other to deal with things and individual session were alright.
02/005
Found psych-ed great, therapist was really understanding and non-judgemental, prefer one to one. Not a lot of people in the group was good. More attention to each person and good if nervous around big groups.
02/006
I think it should incorporate the two. I liked that we met the therapist and had a chance to discuss problems to work on. First day with the group is really hard, much easier when you at least know the therapist.
02/008
Theme 3: peer support and not being only one with personality disorder or difficult problems to solve was reassuring
[Benefited from] the support factor, wish they were still ongoing.
02/078
I met a couple of other girls that I’m still in touch with and call them when I’m having a bad day.
01/042
Had someone to discuss things and plan things with, it was nice to have support.
01/050
Being around others that knew where I was coming from and didn’t judge me. Really appreciated it.
03/013
Good to know other people’s views on problems, see things from different angles. Good way to start discussions and share experiences and let off steam.
02/006
That I’m not alone was such a huge relief. Even when you’ve got the same problem as someone, it’s not exactly the same but you understand each other better.
02/008
Theme 4: barriers
Participants highlighted various reasons why they gave up on PEPS. Group factors were a major reason, the size, composition and ‘feel’ of groups led many to reject this format. Some people realised that their personal mental health vulnerabilities were triggered by PEPS. Some participants were not yet in a reflective space, they felt caught in a spotlight and drawing attention to any perceived shortcomings was too much for them to cope with.
Not saying it didn’t work, but drop-outs in the group were unsettling. Changes in the group affected me; maybe a bigger group could have helped with this. More men as largely women group.
03/125
Didn’t attend problem-solving groups, couldn’t be in a room with strangers.
02/058
Only went to half of one group session – didn’t feel comfortable.
02/015
Only attend the individual session, found those helpful. Couldn’t attend group session due to other commitments.
01/150
Sometimes it can be awkward, everyone has different problems. Individual more helpful; didn’t find groups helpful.
01/063
Didn’t like the groups – would have been better if had more time to work with the therapist.
01/086
Might be a good idea to have a service user involved in explaining personality disorder – only they understand what it feels like. Would like refresher sessions of problem-solving and maybe include new information on what helps.
02/008
I think my weak point is that I let it slip. I don’t keep things up I need things put on a plate for me or I don’t do it.
01/084
Sometimes main message was lost. Noticed I was distracted by those who were less confident reading and writing. Wondered if it put people off, unsure if others got as much out of it.
03/004
I do think I’ve benefited but I’ve been really ill lately and I have a bad memory.
02/014
I struggle with paranoia and found that I couldn’t understand how others were finding the sessions useful. I judged myself and them but I have learnt that I shouldn’t judge them as it is acceptable that they could find it useful even if I didn’t.
01/099
Group sessions didn’t feel comfortable enough and being pushed too far into the deep end, made me want to give up.
02/039
When you are at your lowest sometimes wish you hadn’t gone because you know there isn’t a booklet to help you.
02/052
It drew attention to my bad attributes which meant I would leave group sessions feeling low and negative about myself. Also found that I would take on other people’s negative thoughts and start thinking those about myself.
01/002
Everything all together was overwhelming and I was unwell for a period afterwards. The premise of group was to bring a problem you wanted to work on. That exposure mixed with the immediate finality when the sessions ended was difficult.
03/123
Only went to one group session as didn’t get on with some of the participants.
03/118
It’s a good idea but not the group meetings; I prefer one to one. I don’t like talking about my problems in front of other people especially if I don’t know them. They might take me the wrong way. I’ve had bad experiences with this in the past.
02/015
One to one were very good; didn’t like the group session everyone knowing my business.
02/039
Personally it didn’t work, especially the group therapy. Went because the GP told me to but they were a waste of time.
01/063
Wasn’t very impressed with it, it didn’t seem to benefit me. I didn’t like the way the groups were. I didn’t like the way it used different people to talk each week – ‘spotlighted you’ I didn’t feel comfortable opening up to strangers.
01/086
Theme 5: consistency of staff and process important for security and sustained participation
Psychoeducation with problem-solving staff were valued for being sensitive and skilled. The format seemed acceptable to many participants:
Thought the therapists were really good, the way they explained things – not judgemental.
02/005
Routine helped me, consistent time and place.
02/008
Staff helpful and supportive, everyone got the chance to give input.
02/008
Therapists were very nice and down to earth.
02/009
From a workforce perspective, the fact that many people with PDs engaged with the staff and the process suggests that treatment specifically designed for people with PD need not be extremely intensive and costly.
Theme 6: lack of ‘dose’
A dominant theme in the feedback received on PEPS therapy was that people had not had sufficient exposure to the method and supervised practice in using it until it became a part of their own skill set that they could apply independently.
I think others found it difficult being open and dealing with their problems in front of others. I found it useful but ideally it might be useful to have a short refresher course a few months later.
02/014
Needed longer than 3 months in the group, worked well together.
01/084
Once sessions were over, I was like a zombie. I couldn’t do anything for a few days.
03/82
Not long enough – only just got into it and then it’s finished.
03/024
Looking for a short-term intervention to a long-term issue. Could have been better to hold the 12 sessions over 12 months, 12 weekly sessions were too much to process.
03/123
Better to spread therapy over 12 months, time frame was too short, very intense and full on. If over a longer period may be able to take things in better as remembering was difficult.
03/125
Didn’t last long enough. Waited a long time hear if in the group, then given only 2 weeks’ notice. Little follow-up.
03/036
Not enough time to go through everyone in the group and follow-up every problem. Problems never felt finished – so not solved.
03/024
It could have been longer, could have used the support for a bit longer.
02/009
Fabulous staff, could have lasted longer.
02/066
Pity it’s still not going on.
02/078
Groups felt like going back to school. [The process was] not long enough, needed more weeks.
03/024
After a while I slipped back into my old ways, would be good if it could keep going for longer maybe once a month.
02/006
I believe the spike in adverse events was due to the model not sustaining the support. It gave people help and then left them.
01/099
Ended to soon, only got to work on one problem. Would have been good to have chance to say more.
01/100
Content was good but not enough time to do everything.
03/123
12 weeks is not enough, should have been extended for as long as the person needs.
01/063
These comments suggest that this intervention was offered as a separate, stand-alone treatment. This is not ideal. Participants missed having maintenance sessions to consolidate and reinforce their acquisition of new skills. There seems to have been no carry over to ongoing mental health services participants were receiving. A probable implication is that as soon as the intervention was over, there was risk of it being eroded because it was not practised and improved. It was expecting a great deal of individuals with severe mental health problems to organise themselves to use a complex method unaided and unsupervised after the problem-solving course had been delivered. These data suggest that some participants floundered when the research intervention was suddenly stopped.
Themes related to usual treatment based on feedback at 72-week follow-up
Treatment as usual was provided by participants’ usual-care teams in accordance with normal practice. What constituted treatment as usual in the trial was variable. A summary of services mentioned by respondents in their feedback is given in Table 31. This analysis found two themes pertaining to treatments and support received as ‘usual treatment’:
-
helpful aspects of usual treatment
-
unhelpful aspects of usual treatment.
| Service | Qualifiers |
|---|---|
| Outpatient psychiatry | Regular follow-ups over several years |
| Inpatient admissions | As required |
| Art therapy | Weekly |
| Psychology | As agreed |
| Psychotherapy | As agreed |
| Keyworker from CMHT or voluntary agency | Regularly (fortnightly or monthly) as planned or when requested |
| Drug and alcohol team | Not specified |
| Counselling | As agreed |
| Planned meetings with GP | Regularly arranged |
| Community psychiatric nurse | Usually fortnightly or as agreed |
| Social worker | In role of keyworker usually fortnightly or as agreed |
| Day centre | Several days a week over the duration of several years |
| Crisis or emergency services | As required |
| Occupational therapy | As agreed |
| Advice on benefits | As required |
| Mentalisation groups | Weekly for several months |
| Mindfulness course | As per referral |
| Mental health advocacy | As required |
Theme 1: helpful usual treatment
Participants highlighted the usefulness of the practical help and support, being listened to and believed, and a sense that staff were caring and competent.
It’s been helpful, they gave me advice on staying positive and keeping my mind occupied.
01/34
Psychiatrist and social worker. Treatment from psychiatrist was excellent; [I] thought [the] social worker was very bad.
02/03
Floating support worker has been a huge help. Advocate has been hugely helpful in appointments [with] CPN [community psychiatric nurse] and psychiatrist – felt more believed and listened to.
03/07
Felt I could talk to my counsellor and they understood.
01/44
GP doesn’t listen or states the obvious. Mental health team have been great. CPN has brought forward appointments with the psychiatrist – monitoring me closely. Health-care assistant tries to motivate me to do things. They try to help even when I don’t think they can – don’t feel brushed off.
02/07
GP, psychiatrist and care co-ordinator – everyone was very kind, everyone was doing their best. I felt like I was being looked after.
03/10
I don’t know [what benefits I have had from treatment as usual] maybe a bit of community support has kept me out of hospital.
01/82
GP and CPN have been brilliant.
03/25
Yes, the psychologist I had was very good and understanding. She taught me different techniques and helped me to understand how my mind works.
01/87
I guess so – I’ve been quite stable. Art therapy helped me to understand my emotions and control them better.
01/93
Medication and CPN for past 18 months. CPN doesn’t always respond quick enough. Finally got meds [medications] balance right. CPN helps with problem-solving.
03/34
Mental health team have been really good. I only wish what is on offer now with psychology and psycho-education was available 20 years ago as I might have been a lot better.
01/111
There was no mention of specific PD services or care pathways. People valued knowing who was responsible for their care and how to contact them as a minimum.
Theme 2: unhelpful aspects of usual treatment
Individuals whose personal histories include being marred by close relationships picked up on staff attitudes and willingness to take action to help them. Some participants were affected by cut-backs. It was concerning to hear that some people felt that unless they deteriorated or experienced crises they were not taken seriously.
Waiting lists for treatment – art therapy are really bad. Also took a long time to see a psychiatrist and the mental health team. Quite scary to be kept waiting, to be left alone in those situations.
01/140
It’s all been very confusing. I was receiving support albeit intermittent[ly], but I moved house about a year ago and now my local GP will not refer me to services as he wants proof that I have previously been given a diagnosis of borderline personality disorder.
01/115
Only given six sessions with my mental health team. Not enough. Actually it’s worse at the end. Just about the time you feel you have developed rapport, the sessions end. Now I don’t even want to start. I don’t see use in having to keep re-explaining myself, to start building rapport with someone who will stop seeing me in a few sessions.
01/131
Don’t seem that bothered, seems as if they don’t have enough time – suggest things and then it takes ages for anything to happen.
02/11
Medication and seeing a professional every 4 months. Had a crisis – told there was no-one available to talk to [me]. Requested a copy of the notes the lead professional had made. They did not corroborate her experience of meetings. Felt ‘talked down to’ confronted the professional, put in a complaint.
03/14
Nothing, no treatment received. Medication – came off it in March this year [2012].
01/28
Some counselling through GP, but was stopped as funding was pulled. [GP/CMHT] really bad, not sympathetic, not aware of other services. Not at all bothered – didn’t see [my] desperation. Told [my] only hope was the crisis team.
02/25
Don’t feel I’ve had much treatment. Sessions with psychiatry. See GP regularly. Psychiatrist hasn’t given me suggestions or further support. Feel passed back [and] forth between GP and psychiatrist. Therapist at GP and sessions at ‘Workshop and Co’ all of which stopped, so I feel abandoned.
01/01
Psychologist then discharged and had no one. [It] took my daughter being placed on child protection list to have help.
02/30
Nothing since CPN at start of PEPS. Would have liked PEPS. Questionnaires made her think more – raised awareness [was] helpful.
03/33
Nothing – psychiatrist every 3 months waiting for psychologist. Bad experience of crisis team – [I felt] intimidated [they] didn’t understand. Fear of A&E [accident and emergency] – too many germs. Daughter is my carer. Self-management – felt categorised.
02/33
Unsatisfactory at all stages – both GP and CMHT. Difficulty being referred by GP basically told to make my own arrangements. CMHT not at all concerned. Just wanted numbers going through their doors. No longer have hope to speak to my GP. I just can’t depend on anyone at the moment. Before I had the feeling in the back of my head I could speak to my GP. Now I don’t have that feeling anymore.
01/108
I didn’t receive any treatment for my mental health.
01/40
Struggling for a long time. No service for what she needs help with. Refused psychotherapy as [she was] deemed ‘not safe’. But not safe without psychotherapy either. Struggling with change of CPN. No progress. Feels that what she says is acknowledged, but no support or way to deal with it after that [is offered].
03/107
Difficult to get care other than when I’m in crisis . . .
01/104
None – only see GP for alcohol related problems.
03/128
Discussion
The aim of this analysis was to provide information on participants’ experiences of usual treatment and PEPS therapy, with their perceived benefits and limitations. The majority of codes related to feedback on treatment aspects participants valued, as well as on obstacles they encountered. For many participants, having a thorough assessment of their personality traits and receiving detailed feedback on the meaning of the results made them feel valued and helped them to understand the challenges they faced if they were to overcome problems and recover valued roles. Furthermore, they grasped the rationale of the treatment to follow and saw the psychoeducation as pretherapy preparation, which was generally helpful in orienting them to the intervention to follow. Good staff performance emerged with a difficult-to-treat group feeling that they were listened to, cared for, supported and treated reliably and skilfully. The structure, training and supervision adopted by PEPS provide useful feedback to services wishing to develop specific care pathways for people with a PD.
Overall, this analysis provided strong indications that PEPS provided self-knowledge in a way that empowered participants to become agents of change. PEPS had an impact on their attitudes, which changed from frequently being disaffected to being engaged, hopeful and motivated to put effort into acquiring skills.
Several well-supported codes in this analysis related to challenges that needed to have been resolved in order for learning to progress from positive interest to successful acquisition of complex social and cognitive skills. Early signs were present that some people were being lost at the beginning of the process when they did not grasp the notion of how their own clusters of habits interfered with their social functioning and problem-solving. Unfortunately, subsequent learning could not proceed until individuals grasped the rationale. Given their personal experiences of being marred by their relationships, the transition from individual to group format was too difficult for many in this clinical population. Instead of labelling the problem-solving a group, it might have been less threatening to have presented it as a safe, non-invasive course. The analysis harvested important feedback on perceptions influencing motivation to engage or reject involvement in a group intervention including size, composition (including sex), efficiency in completing problem-solving cycles, group norms (arriving late, rejecting some contributions) and cohesion. PEPS did provide positive, even corrective, group experiences where groups were smaller, provided valuable suggestions for problem-solving options and challenged negative stereotypes about people meeting criteria for a PD. Unfortunately many people floundered at this stage and did not engage in group-based problem-solving. Follow-up, providing ongoing feedback on attempts to apply the skills and opportunities to repeat the course, emerged as being essential for the approach to become embedded in services and in people’s lives.
Closer links to educational research and knowledge about adult learning might have informed more accurate perceptions of prior learning needed for this quite sophisticated intervention to have met its outcome standards for proven efficacy. Clearly coping with high anxiety in social situations was relevant to many participants, but was not the focus of PEPS therapy.
The limitations of this analysis include the second-hand accounts of participants’ accounts captured on semistructured interview sheets. Some questions were repetitive and did not draw out meaningful elaborations or invite participants to provide more individualised responses. Future studies might benefit from using focus groups. These could use interview data to differentiate those people who were enthusiastic about the skills-based approach that they had been offered from those whose experience of PEPS had led them to reject this treatment approach. Focus group data could help us to understand what factors differentiated the groups and what might be done to increase access to clearly structured, evidence-based interventions.
Chapter 7 Discussion
Summary of findings
Clinical effectiveness
The findings of this trial showed that PEPS therapy plus usual treatment was no more effective than usual treatment alone on either the primary outcome or a range of secondary outcome measures. Specifically, no superior effect of PEPS therapy plus usual treatment over usual treatment alone at the 72-week follow-up on the primary outcome of social functioning, measured by the SFQ (adjusted difference in means –0.73 points, 95% CI –1.83 to 0.38 points; p = 0.19). The change score on the SFQ required to indicate the minimum degree of clinical significance was a reduction of 2 points. The CI of the difference in means crossed zero and did not include –2; thus, the null finding is unequivocal. Scores on the SFQ reduced in both arms of the trial from baseline to the 72-week follow-up, indicating some overall positive change. This may be attributable to being in a trial, bearing in mind that participants in the usual-treatment arm underwent a number of assessments that are not typical of usual treatment, although this observation can also be explained by regression to the mean. While the CI for the whole sample did not include zero, nor did it include –2 points (95% CI –1.54 to –0.32 points), hence any effect did not reach the requirement for minimum clinical significance.
At the 72-week follow-up, no treatment effects were evident for any of the secondary outcomes, namely scheduled and unscheduled health service use collected from GP records, mood measured by the HADS, or on the client’s assessment of change on the three problems they considered most important. The process of change, which was to improve social problem-solving skills as assessed by the SPSI-R, showed no group differences at the post-treatment point of measurement. There was consistency across all measures; there were no significant differences between the PEPS group and the usual-treatment group on any measure. Furthermore, there were no obvious effects at different points in follow-up.
The baseline characteristics of the two groups were similar, apart from a slightly higher proportion of participants with complex PD in the PEPS arm. Subgroup analysis of the SFQ at the 72-week follow-up showed no difference in outcome for complex versus simple PD. There was no difference for people with borderline PD.
About half of the people in the PEPS arm did not receive the full dose of treatment. Analysis of the primary outcome by compliance suggested a better outcome for those who received an adequate amount of treatment.
Cost-effectiveness
Psychoeducation with problem-solving shows slightly lower costs and higher QALYs, but differences are not large or significant and CIs are wide. There is a 64% likelihood that the intervention is cost-effective at the threshold used by NICE.
Adverse events
The stoppage of this trial because of concerns about safety is, to our knowledge, unique in trials of complex psychosocial interventions. 82 The DMEC, after studying unblinded data, alerted the TSC to the higher rate of adverse events in the PEPS arm of the trial. The TSC advised that recruitment to the trial be stopped, that treatment should discontinue and that all participants should be alerted to this safety signal. The TSC also firmly advised that follow-up should continue.
It quickly became apparent that stopping a trial of a complex psychosocial intervention because of reports of adverse events was both unique and potentially harmful to the participants, and would require additional, specific procedures to be implemented. We worked closely with the NCTU, in consultation with the TSC and site coinvestigators, to develop procedures for safe stoppage. These included advising the sponsor, advising clinical teams, writing information sheets for participants, devising clinically safe stopping procedures, contacting all participants and conferring with the research ethics committee. 83
By the end of the trial, both the number of adverse events, including serious adverse events, and the number of people reporting them were greater in the PEPS arm. Four deaths were recorded in the PEPS arm: two of these were from natural causes and two were suicides. The nature and circumstances of these deaths did not seem related to the therapy. The number of hospitalisations and the number of people hospitalised were greater in the PEPS arm. The majority of this difference is accounted for by incidents of self-harm and drug and alcohol overdose. Although there was a tendency for PEPS participants to report more adverse events, even after allowing for greater follow-up in the PEPS arm, the results are inconclusive (adjusted incidence rate ratio 1.24, 95% CI 0.93 to 1.64). It should also be acknowledged that the assumption underlying this analysis, that the occurrence of adverse events follows a Poisson distribution (that is, equally likely to occur at any time following randomisation), may not hold true.
Participant views
Interpretation of the findings is assisted by what the participants told us about their experiences of PEPS and usual treatment. Participants valued the one-to-one psychoeducation sessions, which built rapport and increased knowledge and understanding of PD, as intended. The problem-solving approach was also seen as helpful. There is, therefore, the important suggestion that service users’ perceptions of treatment being helpful may not translate into clinically meaningful effects for this group of people.
There were mixed feelings about group work: some felt supported by working with peers, whereas others felt the group setting to be challenging. There was a consistent view that the treatment was too short and the lack of follow-up support was considered a serious shortcoming. Opinions about usual treatment were variable: some individuals were positive about treatment received, while others felt dismissed and neglected. Worryingly, there was a view that the way to access services was to evidence a decline or bring about a crisis (e.g. harm oneself).
Strengths and limitations
Study design
This was a pragmatic trial with broad inclusion criteria (i.e. any PD). As such, it presents results that are generalisable to regular clinical practice in secondary care mental health services.
Treatment fidelity
Treatment fidelity is the extent to which an intervention is delivered as specified in the protocol. Without a high degree of fidelity, there is no way to determine whether unsuccessful outcomes reflect a failure of the model or failure to implement the model as intended. 84 A number of strategies were in place to maximise treatment fidelity. The PEPS intervention was specified in treatment manuals, containing information about the theory underpinning the treatment, the content of sessions, and the duration and frequency of sessions. Therapists were qualified mental health nurses or psychology graduates with clinical experience. Therapists were centrally trained by experienced clinicians and regular supervision was provided. Audiotapes of treatment delivery were scrutinised by the trainers to ensure that each therapist was adhering to the treatment specification and individuals were rated on competence checklists. None of the therapists failed to meet the competence criteria on any of the measures.
Measures
Validated outcome and process measures were used in this trial. With regard to secondary outcomes of service usage, the original idea was that service use might not decline as a result of treatment but that better use of the services offered might result. That is, treated individuals might attend scheduled appointments and thereby avoid crises that required emergency, unscheduled appointments. In the event, it was not easy to ascertain which appointments were scheduled and which were not, apart from the most obvious ones such as emergency department visits.
Sample size
The power calculation for this trial was based on a pilot study conducted with a comparable sample. As stated in the trial protocol,85 to detect a mean difference in SFQ score of 2 points (SD 4.53 points) at the 72-week follow-up with a two-tailed significance level of 1% and power of 80% with equal allocation to two arms, the requirement was 120 patients in each arm of the trial. Allowing for 30% drop-out, the aim was to randomise 340 participants in total (i.e. 170 participants per arm). The study fell marginally short of its target of 70% follow-up, with an overall follow-up at the 72-week end point being 68%. There was a difference in the follow-up rate between the usual-treatment arm (62% follow-up) and the PEPS treatment arm (73% follow-up), with the target being met in the latter. The numbers, however, were somewhat short of the desired 120 participants per arm at 94 in the usual-treatment arm and 113 in the PEPS arm. This was partly because the trial was stopped before 340 people could be randomised, with only 306 randomised before recruitment was halted. Furthermore, although this was one of the largest economic evaluations conducted in PD, the sample size may have been too limited to pick up important effects on the use and cost of specific services. Nonetheless, this trial has demonstrated that it is possible to recruit a large number of difficult-to-engage, community-dwelling individuals into a clinical trial and to retain a large proportion of recruits to an end point 72 weeks after randomisation. Further, the achieved sample size, although smaller than the target, still yielded sufficient precision to exclude a clinically meaningful effect.
Bias
Care was taken to minimise bias through independent randomisation, ensuring as far as possible that researchers collecting follow-up data were blinded to allocation, and conducting data entry and cleaning while blind to intervention status. However, researchers were unblinded in some cases, with more cases in the PEPS arm than in the usual-treatment arm (32% vs. 13%). Nonetheless, in the PEPS arm, the mean 72-week follow-up score on the primary outcome (the SFQ) was higher (i.e. in the undesired direction) for the unblinded cases, indicating no bias in favour of the PEPS group.
On average, non-completers in the PEPS arm remained in the trial for longer than non-completers in the usual-treatment arm. This, combined with the higher overall completion rates in the PEPS arm, contributes to more observed person-years of follow-up in the PEPS arm (203 person-years) than with usual treatment (178 person-years).
Bias in adverse event recording
Adverse event recording may have been subject to bias. Recording of adverse events during the trial depended on self-disclosure or a report by a member of the clinical team. The difference in reports of adverse events between arms could be explained in ways other than PEPS causing harm directly. These include differential reporting, for a number of reasons. Those in the PEPS arm were more in contact with therapists and so had more opportunity to report adverse events, and may also have felt more able to admit adverse events candidly. Similarly, there may be a greater degree of identification by staff of problems experienced by participants in the PEPS arm. In addition, clinicians may have been more likely to advise the research therapists about adverse events for those in PEPS therapy, whereas they managed the usual-treatment participants themselves, and so this may appear as more adverse events in the PEPS arm during the treatment phase. Alternatively, PEPS may encourage help-seeking, increasing the likelihood of hospitalisation, in which case adverse events would not be literally ‘adverse’. More systematic and objective recording of adverse events would have been desirable. Advance consent had not been obtained to access Health and Social Care Information Centre records, and gaining consent to do so from participants in the follow-up period proved difficult because of limited contact and availability.
Uptake of the intervention
Of those randomised to the PEPS arm, 4% (n = 6) received no intervention at all, 51% (n = 78) received a partial intervention (i.e. ≤ 5 group sessions) and 45% (n = 70) received the intervention as per protocol (i.e. ≥ six group sessions). In the pilot study, of the 87 people allocated to PEPS, 13% (n = 11) never attended, 1% (n = 1) attended psychoeducation only, 21% (n = 18) attended psychoeducation and fewer than five group sessions, and 66% (n = 57) attended psychoeducation and five or more group sessions. So, in the definitive study, fewer people received no treatment but also fewer received an adequate quantity of treatment. The planned 12-session group intervention actually lasted, on average, six sessions. That is, most participants did not receive the full treatment. CACE analysis attempts to estimate the effect of the intervention among those in the PEPS arm who adhered compared with those in the usual-care arm who would have adhered had they been allocated to receive it. As expected, when the proportion adherent is around 50%, the CACE estimates are approximately twice that of analyses that take no account of amount of intervention received. However the 95% CIs are wide and none of the sensitivity analyses lend strong support for different conclusions from the primary analysis.
Of the psychoeducation sessions offered (on average 3.4), 90% were taken up. Clearly, individual psychoeducation is far better attended than subsequent group sessions and the overall group non-completion rate is high. This difference may be explained in a number of ways. Participants’ enthusiasm for treatment may be stronger at the start of treatment and wane with time, they may prefer individual sessions or the content of psychoeducation may be more relevant to their problems.
To our knowledge, there is no review of treatment uptake and completion specifically in RCTs of psychological interventions. However, reviews of treatment completion of psychological therapies for PD using evaluations of any design indicate non-completion rates as low as zero and as high as 80%, with median or mean non-completion rates of between 29% and 35%. 33,86,87 Although the PEPS therapy non-completion rates appear higher than average, studies of non-completion use many different definitions of what constitutes non-completion of therapy and so comparisons may be misleading.
Personality disorder types
The treatment was offered to people with any type of PD. This was a pragmatic approach in that services often cannot offer separate treatments for different disorders and so treatments are offered only for the most common disorders (typically borderline PD) and people with the less common PDs may not be offered treatment at all. However, it may be that people with certain types of disorder have done better than others. Although the trial was not powered to conduct subgroup analyses, examination of those with borderline PD, which was the most prevalent disorder at baseline, showed no difference on the primary outcome at follow-up.
Non-standard usual treatment
The content of usual treatment was not standardised in this trial. Imposing standard treatment on the large number of services contributing to this trial was not considered a feasible option. Practice varied widely across services, and usual treatment was not a clearly prescribed procedure. Some participants commented that usual treatment could be well planned, holistic and reliably delivered, but in other cases it was unfocused, dismissive and unreliable. This issue (i.e. highly variable usual treatment) affected both those in the PEPS plus usual-treatment arm and those in usual treatment alone. The variability of usual treatment and the unclear treatment pathways for people with PD may have, in part, accounted for the loss to follow-up.
Observations about participants
Overall, participants in this trial were heavy users of health services, costing approximately £8000 per annum (based on the baseline data). This is comparable to the estimated treatment costs for schizophrenia but substantially more than the costs for bipolar disorder. 88 This is reflected in low quality of life, with QALYs of around 0.57 over the entire follow-up (i.e. out of possible 1.5 QALYs). The prevalence of PD in mental health services in England is at least 40%. 89 Coid et al. 1 have estimated a population prevalence of 4.4% in those aged 16–74 years and this would imply total health costs of around £16B per year.
Interpretation
We found no evidence to support the use of PEPS therapy alongside standard care within the UK NHS for improving social functioning of adults with PD living in the community. This differs from the pilot study, in which a significant improvement was found in the primary outcome of social functioning, measured, as in this trial, by the SFQ at the end of treatment, which was a mean of 24 weeks post baseline (range 21–28 weeks). 43 The between-group difference on the SFQ at the end point was –1.06 points (95% CI –1.99 to –0.18 points). In the pilot study, there were also significant improvements on the SPSI-R, although no significant improvements in service use.
A number of explanations can be put forward for the lack of significant effects observed in this full-scale trial compared with the pilot study. The full trial was superior in its design and methods: it was multisite, there was a larger number of participants (providing greater precision of estimated between-group differences) the follow-up period was considerably longer and the methods of imputing missing data were more sophisticated. Hence, greater confidence can be placed in these results. Evidence from meta-analyses shows that demonstration projects have larger effects than programmes delivered more widely, and high methodological rigour is related to smaller outcomes. 90 Pilot projects may produce larger effects because they are more manageable in size and they are innovative, hence they may benefit from closer supervision, higher programme fidelity and a greater motivational drive from the closely involved chief investigator.
Evidence for the effectiveness of psychological treatments for PD is marred by methodological weakness. Systematic review authors have concluded that many of the trials of PD treatments are of poor quality, have small sample sizes and use a wide range of non-comparable outcomes with different degrees of clinical importance (e.g. self-reported changes in personality beliefs vs. days in hospital). 11,91,92 This variation makes it difficult to draw firm conclusions about any one treatment, and there is a place for specifying core outcome measures of agreed importance to all. 93
Most of the treatments for PD that are evidenced as effective are of relatively long duration and are delivered by highly trained personnel, and hence are costly to deliver and are consequently of restricted availability. The question of whether or not brief interventions work for PD has been investigated recently,94 with insufficient available evidence to provide an answer. The null findings of the PEPS trial suggest that the intervention may have been too brief to have had an effect, at least as a stand-alone treatment divorced from good clinical care. This point of view was articulated by participants in the PEPS trial.
Although social problem-solving was addressed as a common need among people with PD, this may not be the case. Alternatively, this particular aspect of treatment may not have been a priority for all participants. Hence, brief ‘one-size-fits-all’ treatments appear not to be the way forward, but rather clinicians should base treatment plans on a thorough case formulation. Assessment and case formulation would also reveal who might benefit from group sessions and who might be averse to working in groups. Information from participants tells us that some participants enjoyed the support gained from other group members, whereas others found it difficult to discuss their own problems or to listen to others discussing their problems. This is reflected in the differential attendance rates for individual and group components. However, many interventions for people with PD are group based and it may be that there is a need to offer some pretherapy training in group-work skills so that people find it easier to engage. 95
Crudely set in the company of similar trials reporting the same outcome, PEPS is the largest and longest trial and, to our knowledge, most methodologically rigorous. A synthesis of these trials is presented in Figure 6. Although the synthesis of all trials does still suggest a modest effect, despite the PEPS data, the overall result is dominated by small bias-prone studies. In Figure 7 the studies are sorted by time, with a real trend across time showing that the more recent the study the less the effect of the intervention. This may be because of improvements in study design and conduct over the past 15 years.
FIGURE 6.
Synthesis of studies using SFQ as an outcome. a, PEPS 2016 refers to this report. df, degrees of freedom; IV, inverse variance.
FIGURE 7.
Changes over time in studies using SFQ as an outcome. a, PEPS 2016 refers to this report. df, degrees of freedom; IV, inverse variance.
Recently evidence has been accruing from trials that structured clinical management achieves equally good outcomes as specific treatments for PD. 98–101 Commenting upon this, Bateman102 speculated that in the past general psychiatric treatment, or treatment as usual, may actually have inadvertently been harmful and specialist treatments may have shown benefit primarily by minimising harm, possibly through preventing unco-ordinated care. Service users and providers have also highlighted the importance of co-ordinating and integrating psychological treatments within teams that can provide additional support for patients at times of crisis. 30 In the PEPS trial, treatment was delivered as a stand-alone therapy rather than being integrated into a co-ordinated package of care.
In the PEPS trial, although there was a higher number of adverse events reported in the treatment arm, it is also the case that the number of reported events was high in the usual-care arm: 76 adverse events reported by 39 people (26%) over a period of 72 weeks. One explanation for a higher number of reported events in the treatment arm of the PEPS trial is that the treatment stopped without any structured follow-up, thus leaving participants unsupported after a period of treatment. To have one’s support withdrawn may well be more damaging than to have had little or no support in the first place. As reported by participants, some individuals may be driven to dramatic and harmful gestures to access services.
Chapter 8 Conclusions
Implications for health care
The conclusion here is that PEPS therapy should not be promoted for people in secondary mental health services who are diagnosed with PD, at least not in the absence of a structured, comprehensive clinical care package. Harm is most likely to be caused by leaving people unsupported after the conclusion of brief interventions rather than by PEPS itself. However, any use of brief problem-solving interventions in practice should be conducted only with rigorous collection of data on adverse effects, in the context of the need for better awareness and measurement of adverse events in psychotherapy practice as a whole.
It seems reasonable to say that no specific treatment should be delivered to this group of individuals in the absence of good clinical care. This is an important and topical message when the Improving Access to Psychological Therapies initiative is being extended to people with ‘severe mental illness’, which includes PD. 103 Structured clinical management of people with PDs is an important aspect of treatment, it is not an alternative to specific treatments but rather the bedrock on which these treatments may be offered.
Some participants expressed reservations about group sessions, both in terms of the content being challenging and also concerning anxieties about being with other people. Others found group work to be an informative process and the support of others was deemed helpful. A general message is a greater investment in preparing people for group treatment may be worthwhile to prevent treatment non-completion. 104 The higher costs of group versus individual sessions may make individual preparation sessions cost-effective (£476 vs. £119, respectively).
Psychoeducation with problem-solving was also reported to have equipped a workforce of mental health professionals to offer consistently skilled responses to people presenting with PD. Although this intervention did not achieve the standard for significance, a positive working alliance and a willingness among many people with PD to master well-defined skills suggests that a promising start was made to developing general staff competencies and engaging people with PD in services.
Implications for researchers
Defining and reporting adverse effects of psychological therapies
The issue of defining and recording adverse events in trials of complex psychosocial interventions has been highlighted as of crucial importance. Consideration of potential harms has not been given adequate attention in clinical trials of psychosocial interventions with people with PDs or participants with other types of problems. 82,105,106 Specification of the nature of possible harms and identification of the type of person who may be susceptible to harm is important information to collect to establish risk–benefit balances that ought to be available to clinical services to inform the choice of treatments provided and to patients to help them make decisions about consenting to psychosocial treatment. Linden107 has offered definitions of a range of unwanted events and adverse treatment reactions and has constructed a checklist for recording the existence, severity and treatment-relatedness of these events that may be of value for systematic observation of harms in clinical trials. Jonsson et al. 105 offer suggestions for collecting and reporting adverse effects in RCTs, including providing a plan for how harms will be detected within the protocol for ethical review, using structured assessment methods such as checklists or rating scales, active and regular monitoring, and reporting the absence of harms as well as the occurrence of harms. At the consent stage it is worth asking participants for permission to access official records that may assist in the identification of adverse events, including centrally held NHS data via the Health and Social Care Information Centre.
Processes for stopping trials of psychological therapies
Consideration should be given in advance to procedures for stopping trials of psychosocial interventions, should this be necessary. The difficult and traumatic halting of PEPS was made more challenging by the need for additional, very specific procedures to be implemented. It was important that this was undertaken both swiftly and safely, and the procedure we followed has already been presented. 83 We hope to fully publish this experience to guide funders, sponsors, participants, trial units and triallists.
Recommendations for research
Methodological research
Measuring adverse events in psychological therapies
There is clearly a place for systematic research into the definition and prevalence of adverse effects of psychological therapies. We know of some ongoing work in Sheffield [www.shef.ac.uk/scharr/sections/hsr/mh/mhresearch/adeptproject (accessed 25 April 2016)] and await results of this important initiative.
Procedures for stopping trials of psychosocial interventions
We have discovered that there is not clarity around this issue. We suggest that Delphi work could inform a working party to draw on collective experience and consideration of specific examples in order to inform policy and the design and conduct of trials.
Core outcomes
There is a need to define and gain consensus for a core outcome data set for trials such as PEPS. We think a Core Outcome Measures in Effectiveness Trials initiative (COMET; www.comet-initiative.org), specific to psychological interventions for people with borderline personality difficulties is indicated, or this area of evaluative research will continue to be dogged by use of different measures, used with no agreed consistency.
Service delivery analysis
Pathways of care for people with personality disorder
Service research into defining and implementing pathways of care for people with PD is urgently needed. Given emerging evidence that good clinical care is as effective as ‘active’ treatments,95,98–100 more should be done to understand the barriers to, and facilitators of, the implementation of good care. Both qualitative and quantitative investigation is indicated.
Economics
Economic evaluations of interventions for people with personality disorder
Evidence from this study shows that people with PD are heavy users of mental health services and have poor quality of life in comparison with people with other disorders. Overall, participants in this trial had annual costs of around £8000 (based on the baseline data) and QALYs were around one-third of those of people with full health. Evaluations of treatments for people with PD should continue to address both costs and quality of life.
Evaluations
James Lind initiative
There are several ongoing James Lind initiatives in mental health care. It would seem that there could be a priority-setting exercise conducted for questions of relevance to the care of people with borderline-type problems.
Disaggregation of psychoeducation and problem-solving therapy
Although PEPS therapy as a whole was not effective, there were differences in uptake of the individual psychoeducation sessions and the group problem-solving sessions. Individual psychoeducation sessions were well attended, and this component of the intervention may be worth evaluating separately. However, care should be taken to embed this in good clinical care.
Active comparator
As good-quality clinical care has established effectiveness, future trials ought to be a comparison between good-quality routine clinical care versus good-quality clinical care plus the experimental intervention. This would answer the question as to what additional value was conferred by the intervention over and above that from good-quality routine clinical care. This type of design has not been employed, to date, in PD treatment trials. This would require specification and monitoring of good-quality clinical care. In addition, the impact of continuation of care after the end of the experimental treatment could be tested.
Adverse events
We plan to investigate adverse events further by accessing centrally held NHS data on deaths and hospitalisations for all PEPS trial participants. Because we do not have consent from all participants to do this, accessing this information without participant consent requires approval from the Health Research Authority Confidentiality Advisory Group. In light of the trial stoppage for safety reasons, we have a clear justification for requesting approval.
Other interventional research
This is a grossly under-researched area. There is a place for a broad overview of all evaluative studies in this area, to produce a map of where randomised research exists and where it does not. The number and power of studies in this area would then become obvious. In this way, guidance on what systematic reviews are possible can be created and, where relevant to NHS practice, those interventions in everyday practice that have not been fairly tested in trials could then be randomised within the context of routine care.
Final statement
This rigorous trial has established that PEPS therapy is not an effective treatment for improving social functioning in community adults with PD. This is despite PEPS being based on solid theoretical and empirical foundations,43,56 on therapy components with demonstrated effectiveness in dealing with a range of psychological problems31,36,37 and which has been deemed helpful by those who have received it. 49 This is not the first psychosocial intervention that has appeared promising – theoretically, empirically and by favourable participant judgement – but has proved in rigorous trials to be ineffective or even harmful. 108,109 Information from this study indicates that people with PD have poor quality of life and are heavy users of health and social services. It is important to continue to seek effective management and treatment for this group of troubled individuals. Good-quality research is the only way to provide evidence of what is genuinely effective, or ineffective, and is the best way to serve patients, their family and friends, and our wider society. Such research is expensive, but the costs of not conducting such research may well be greater.
Acknowledgements
We thank all those who took part in the trial and clinical staff at the participating sites for their support.
The PEPS Trial Collaborative Group acknowledges the support of the National Institute for Health Research, through the Mental Health and Primary Care Research Networks and the National Institute for Social Care and Health Research in Wales.
Contributions of authors
Mary McMurran (Professor of Personality Disorder Research) was the chief investigator and coauthored the final report.
Mike J Crawford (Professor in Mental Health Research, site principal investigator) contributed to the development of the grant application and trial protocol, contributed to the conduct of the study and contributed important intellectual content to the report.
Joe Reilly (Professor of Mental Health, site principal investigator) contributed to the development of the grant application and trial protocol, contributed to the conduct of the study and contributed important intellectual content to the report.
Juan Delport (Head of Psychological Services, site principal investigator) contributed to the conduct of the study, jointly conducted the qualitative analysis and prepared the qualitative results for publication, and contributed important intellectual content to the report.
Paul McCrone (Professor of Health Economics) wrote the health economics analysis plan, oversaw the health economics analysis and prepared the results for publication.
Diane Whitham (Research Manager, Clinical Trials Unit) contributed to the development of the grant application and trial protocol, contributed to the conduct of the study and contributed important intellectual content to the report.
Wei Tan (Medical Statistician, Clinical Trials Unit) analysed the clinical effectiveness data and contributed to preparing the results for publication.
Conor Duggan (Emeritus Professor of Forensic Psychiatry) contributed to the conception, design and conduct of the study, and contributed important intellectual content to the report.
Alan A Montgomery (Professor of Medical Statistics and Clinical Trials) oversaw the clinical effectiveness analysis and contributed to preparing the results for publication.
Hywel C Williams (Former Director of Nottingham Clinical Trials Unit) contributed to the development of the grant application and trial protocol, helped with the conduct of the study and contributed important intellectual content to the report.
Clive E Adams (Professor and Co-ordinating Editor, Cochrane Schizophrenia Group) contributed to the conception, design and conduct of the study, and contributed important intellectual content to the report.
Huajie Jin (Senior Health Economist) analysed the health economics data and contributed to preparing the results for publication.
Matthew Lewis (Research Assistant, Psychology) jointly conducted the qualitative analysis and preparation of the qualitative results for publication.
Florence Day (Trial Manager, Clinical Trials Unit) contributed to the development of the trial protocol, managed the conduct of the study and coauthored the final report.
The Psychoeducation with Problem-solving Trial Collaborative Group
Central Co-ordinating team
Mary McMurran (Chief Investigator), Florence Day (Trial Manager), Diane Whitham (Clinical Trials Unit Research Manager) and Keith Whitaker (Information Technology Programmer).
Statistical support
Paul Silcocks (until June 2010), Veronica Moroz (until December 2010), Samir Mehta (from February 2011 to December 2012), Justin Fenty (from April 2011 to September 2012), Wei Tan (from November 2012) and Alan Montgomery (from April 2013).
Trial Management Group
Mary McMurran, Florence Day, Diane Whitham, Paul Silcocks (until June 2010), Veronica Moroz (until December 2010), Samir Mehta (from February 2011 to December 2012), Wei Tan (from November 2012), Mike Crawford, Joe Reilly, Juan Delport, Clive Adams, Hywel Williams and Keith Archer (from January 2012).
Trial Steering Committee
Kate Davidson (Chairperson), Mike Clarke, John Norrie, Rachel Nickeas (until October 2011), Clare McLean (from March 2012 to October 2013) and Naomi Salisbury (from October 2013).
Data Monitoring and Ethics Committee
Glenys Parry (Chairperson), Gillian Haddock (Interim Chairperson February 2012 and from November 2012), Sheila Bird and Naomi Salisbury.
Clinical supervision
Steve Coupe.
Sites (number of participants randomised)
Central and north-west London (115)
Mike Crawford (Principal Investigator), Lavanya Thana (Research Assistant), Emma Naylor (Research Therapist until March 2011), Charlotte Cox (Research Therapist from April 2011 to September 2011) and Rosie Evans (Research Therapist from October 2011).
Louise Felix provided local clinical supervision for the PEPS therapists.
The research was supported by the following people: Madeline Dean, Dina Gazizova, Sarah Johnson, Antoinette McNulty, Lesley O’Connell, Lorraine O’Connell, Rosemary Stevenson, Raphael Underwood, Sharmini Rajenthran and Tendai Rushwaya.
South Wales (88)
Juan Delport (Principal Investigator), Katie Wood (Research Assistant until July 2013), Matt Lewis (Research Assistant from August 2013) and Serenna Jenkins (Research Therapist).
The research was supported by the following people: Tracey Brain, Delyth Braim, Sarah Hunt, Cat Johnston, Sandra Jumbe, Tracey Roberts and Daniel Webb.
North-East England (103)
Joe Reilly (Principal Investigator), Jemma Hill (Research Assistant until September 2011), Helen Beckwith (Research Assistant from January 2012 to August 2013), Claire Farrow (Research Assistant from August 2013) and Lorraine Fry (Research Therapist).
Stewart Mitchell provided local clinical supervision for the PEPS therapists.
The research was supported by the following people: Karen Bibbings, Philippa Bolton, Emily Clare, Gillian Cunningham, Jacqueline Harvey, Lesley Haley, Val Heard, Anthony Jones, Deborah Kemp, Saffra Knox, Sophie Lake, Cameron Martin, Paul Moran, Kerry Stott, Sophie Sultana and Corinne Vaughan.
Publications
McMurran M, Crawford MJ, Reilly JG, McCrone P, Moran P, Williams H, et al. Psycho-education with problem solving (PEPS) therapy for adults with personality disorder: a pragmatic multi-site community-based randomised controlled trial. Trials 2011;12:198.
McMurran M, Delport J, Jenkins S, Wall M, Day F. Recruitment to personality disorder treatment trials. Ment Health Rev J 2012;17:119–27.
Day F, McMurran M, Duley L. The process of stopping recruitment and trial treatment in a trial of psychological therapy for people with personality disorder following a safety alert. Trials 2013;14(Suppl. 1):13.
Day F, McMurran M, Duley L. Things often get worse before they get better: participants’ responses to the cessation of recruitment and trial treatment in a trial of a psychological therapy for people with personality disorder following a safety alert. (A missed opportunity for collaboration?). Trials 2013;14(Suppl. 1):94.
McMurran M, Wood K. A treatment goal checklist for people with personality disorder. Pers Ment Health 2013;7:298–306.
Data sharing statement
Data may be obtained from the NCTU according to their policy for sharing individual participant data from clinical trials.
Disclaimers
This report presents independent research funded by the National Institute for Health Research (NIHR). The views and opinions expressed by authors in this publication are those of the authors and do not necessarily reflect those of the NHS, the NIHR, NETSCC, the HTA programme or the Department of Health. If there are verbatim quotations included in this publication the views and opinions expressed by the interviewees are those of the interviewees and do not necessarily reflect those of the authors, those of the NHS, the NIHR, NETSCC, the HTA programme or the Department of Health.
References
- Coid J, Yang M, Tyrer P, Roberts A, Ullrich S. Prevalence and correlates of personality disorder in Great Britain. Br J Psychiatry 2006;188:423-31. http://dx.doi.org/10.1192/bjp.188.5.423.
- Rendu A, Moran P, Patel A, Knapp M, Mann A. Economic impact of personality disorders in UK primary care attenders. Br J Psychiatry 2002;181:62-6. http://dx.doi.org/10.1192/bjp.181.1.62.
- Department of Health . NHS Reference Costs 2006 07 2008. http://webarchive.nationalarchives.gov.uk/20130107105354/http://www.dh.gov.uk/en/Publicationsandstatistics/Publications/PublicationsPolicyAndGuidance/DH_082571 (accessed 25 April 2016).
- Fok ML, Hayes RD, Chang CK, Stewart R, Callard FJ, Moran P. Life expectancy at birth and all-cause mortality among people with personality disorder. J Psychosom Res 2012;73:104-7. http://dx.doi.org/10.1016/j.jpsychores.2012.05.001.
- Noren K, Lindgren A, Haellstom T, Thormaehlen B, Vinnars B, Wennberg P, et al. Psychological distress and functional impairment in patients with personality disorders. Nord J Personality 2007;61:260-70. http://dx.doi.org/10.1080/08039480701414973.
- Powers AD, Oltmanns TF. Personality disorders and physical health: a longitudinal examination of physical functioning, healthcare utilization, and health-related behaviors in middle-aged adults. J Pers Disord 2012;26:524-38. http://dx.doi.org/10.1521/pedi.2012.26.4.524.
- South SC, Turkheimer E, Oltmanns TF. Personality disorder symptoms and marital functioning. J Consult Clin Psychol 2008;76:769-80. http://dx.doi.org/10.1037/a0013346.
- Johnson JG, Cohen P, Smailes E, Kasen S, Oldham JM, Skodol AE, et al. Adolescent personality disorders associated with violence and criminal behavior during adolesence and early adulthood. Am J Psychiatry 2000;157:1406-12. http://dx.doi.org/10.1176/appi.ajp.157.9.1406.
- Soeteman DI, Verheul R, Busschbach JJ. The burden of disease in personality disorders: diagnosis-specific quality of life. J Pers Disord 2008;22:259-68. http://dx.doi.org/10.1521/pedi.2008.22.3.259.
- Binks C, Fenton M, McCarthy L, Lee T, Adams C, Duggan C. Psychological therapies for people with borderline personality disorder. Cochrane Database Syst Rev 2006;1. http://dx.doi.org/10.1002/14651858.cd005652.
- Duggan C, Huband N, Smailagic N, Ferriter M, Adams C. The use of psychological treatments for people with personality disorder: a systematic review of randomised controlled trials. Personal Ment Health 2007;1:95-125. http://dx.doi.org/10.1002/pmh.22.
- Kliem S, Kröger C, Kosfelder J. Dialectical behavior therapy for borderline personality disorder: a meta-analysis using mixed-effects modeling. J Consult Clin Psychol 2010;78:936-51. http://dx.doi.org/10.1037/a0021015.
- McMurran M, Crawford MJ, Nezu AM, Nezu CM. Handbook of Cognitive and Behavioural Therapies. New York, NY: Oxford University Press; 2016.
- Chambless DL, Hollon SD. Defining empirically supported therapies. J Consult Clin Psychol 1998;66:7-18. http://dx.doi.org/10.1037/0022-006X.66.1.7.
- Nur U, Tyrer P, Merson S, Johnson T. Relationship between clinical symptoms, personality disturbance, and social function: a statistical enquiry. Irish J Psychol Med 2004;21:19-22. http://dx.doi.org/10.1017/S0790966700008090.
- Seivewright H, Tyrer P, Johnson T. Persistent social dysfunction in anxious and depressed patients with personality disorder. Acta Psychiatr Scand 2004;109:104-9. http://dx.doi.org/10.1046/j.1600-0447.2003.00241.x.
- Skodol AE, Pagano ME, Bender DS, Shea MT, Gunderson JG, Yen S, et al. Stability of functional impairment in patients with schizotypal, borderline, avoidant, or obsessive-compulsive personality disorder over two years. Psychol Med 2005;35:443-51. http://dx.doi.org/10.1017/S003329170400354X.
- Blum N, Pfohl B, St John D, Monahan P, Black DW. STEPPS: a cognitive–behavioural systems-based group treatment for outpatients with borderline personality disorder: a preliminary report. Compr Psychiatry 2002;43:301-30. http://dx.doi.org/10.1053/comp.2002.33497.
- Blum N, St John D, Pfohl B, Stuart S, McCormick B, Allen J, et al. Systems Training for Emotional Predictability and Problem Solving (STEPPS) for outpatients with borderline personality disorder: a randomized controlled trial and 1-year follow-up. Am J Psychiatry 2008;165:468-78. http://dx.doi.org/10.1176/appi.ajp.2007.07071079.
- Bos EH, van Wel EB, Appelo MT, Verbraak MJ. A randomized controlled trial of a Dutch version of systems training for emotional predictability and problem solving for borderline personality disorder. J Nerv Ment Dis 2010;198:299-304. http://dx.doi.org/10.1097/NMD.0b013e3181d619cf.
- McMurran M, Nezu A, Nezu C. Problem solving therapy for people with personality disorders: an overview. Ment Health Rev J 2008;13:35-9. http://dx.doi.org/10.1108/13619322200800012.
- McMurran M, Nezu C, Nezu A, Morgan C, Bhugra D. Principles of Social Psychiatry. 2nd edn. Chichester: Wiley-Blackwell; 2009.
- D’Zurilla T, Nezu AM. Problem-Solving Therapy: A Positive Approach to Clinical Intervention. New York, NY: Springer Publishing Co.; 2007.
- Herrick SM, Elliott TR. Social problem-solving abilities and personality disorder characteristics among dual-diagnosed persons in substance abuse treatment. J Clinical Psychol 2001;57:75-92. http://dx.doi.org/10.1002/1097-4679(200101)57:1%3C75::AID-JCLP9%3E3.0.CO;2-6.
- Keltikangas-Järvinen LP, Pakaslahti L. Development of social problem solving strategies and changes in aggressive behavior: a 7-year follow-up from childhood to late adolescence. Aggressive Behav 1999;25:262-79. http://dx.doi.org/10.1002/(SICI)1098-2337(1999)25:4%3C269::AID-AB3%3E3.0.CO;2-K.
- Matthys W, Cuperus JM, Van Engeland H. Deficient social problem-solving in boys with ODD/CD, with ADHD, and with both disorders. J Am Acad Child Adolesc Psychiatry 1999;38:311-21. http://dx.doi.org/10.1097/00004583-199903000-00019.
- Nezu A, Wilkins V, Nezu C, Chang EC, D’Zurilla TJ, Sanna LJ. Social Problem Solving: Theory, Research, and Training. Washington, DC: American Psychological Association; 2004.
- D’Zurilla TJ, Nezu AM, Maydeu Olivares A. Social Problem-Solving Inventory – Revised (SPSI-R): Technical Manual. North Tonawanda, NY: Multi-Health Systems, Inc; 2002.
- Dixon-Gordon KL, Chapman AL, Lovasz N, Walters K. Too upset to think: the interplay of borderline personality features, negative emotions, and social problem solving in the laboratory. Personal Disord 2011;2:243-60. http://dx.doi.org/10.1037/a0021799.
- Crawford MJ, Price K, Rutter D, Moran P, Tyrer P, Bateman A, et al. Dedicated community-based services for adults with personality disorder: Delphi study. Br J Psychiatry 2008;193:342-3. http://dx.doi.org/10.1192/bjp.bp.107.043042.
- Malouff JM, Thorsteinsson EB, Schutte NS. The efficacy of problem solving therapy in reducing mental and physical health problems: a meta-analysis. Clin Psychol Rev 2007;27:46-57. http://dx.doi.org/10.1016/j.cpr.2005.12.005.
- McMurran M, Huband N, Duggan C. A comparison of treatment completers and non-completers of an in-patient treatment programme for male personality-disordered offenders. Psychol Psychother 2008;81:193-8. http://dx.doi.org/10.1348/147608308X288762.
- McMurran M, Huband N, Overton E. Non-completion of personality disorder treatments: a systematic review of correlates, consequences, and interventions. Clin Psychol Rev 2010;30:277-87. http://dx.doi.org/10.1016/j.cpr.2009.12.002.
- Banerjee P, Duggan C, Huband N, Watson N. Brief psychoeducation for people with personality disorder: a pilot study. Psychol Psychother 2006;79:385-94. http://dx.doi.org/10.1348/147608305X57987.
- Zanarini MC, Frankenburg FR. A preliminary, randomized trial of psychoeducation for women with borderline personality disorder. J Pers Disord 2008;22:284-90. http://dx.doi.org/10.1521/pedi.2008.22.3.284.
- Bell AC, D’Zurilla TJ. Problem-solving therapy for depression: a meta-analysis. Clin Psychol Rev 2009;29:348-53. http://dx.doi.org/10.1016/j.cpr.2009.02.003.
- Cuijpers P, van Straten A, Warmerdam L. Problem solving therapies for depression: a meta-analysis. Eur Psychiatry 2007;22:9-15. http://dx.doi.org/10.1016/j.eurpsy.2006.11.001.
- Hayward J, McMurran M, Sellen J. Social problem solving in vulnerable adult prisoners: profile and intervention. J Forens Psychiatry Psychol 2008;19:243-8. http://dx.doi.org/10.1080/14789940701752193.
- McMurran M, Blair M, Egan V. An investigation of the correlations between aggressiveness, impulsiveness, social problem-solving, and alcohol use. Aggressive Behav 2002;28:439-45. http://dx.doi.org/10.1002/ab.80017.
- McMurran M, Egan V, Duggan C. ‘Stop & Think!’ Social problem-solving therapy with personality disordered offenders. Crim Behav Ment Heal 2001;11:273-85. http://dx.doi.org/10.1002/cbm.401.
- Long CG, Fulton B, Dolley O, Hollin CR. Social problem-solving interventions in medium secure settings for women. Med Sci Law 2011;51:215-19. http://dx.doi.org/10.1258/msl.2011.010105.
- Stewart CD, Quinn A, Plever S, Emmerson B. Comparing cognitive behavior therapy, problem solving therapy, and treatment as usual in a high risk population. Suicide Life Threat Behav 2009;39:538-47. http://dx.doi.org/10.1521/suli.2009.39.5.538.
- Huband N, McMurran M, Evans C, Duggan C. Social problem-solving plus psychoeducation for adults with personality disorder: pragmatic randomised controlled trial. Br J Psychiatry 2007;190:307-13. http://dx.doi.org/10.1192/bjp.bp.106.023341.
- McMurran M, McMurran M, Howard RC. Personality, Personality Disorder and Violence. Chichester: Wiley-Blackwell; 2009.
- Tyrer P, Nur U, Crawford M, Karlsen S, McLean C, Rao B, et al. The Social Functioning Questionnaire: a rapid and robust measure of perceived functioning. Int J Soc Psychiatry 2005;51:265-75. http://dx.doi.org/10.1177/0020764005057391.
- McMurran M, Huband N, Duggan C. The role of social problem solving in improving social functioning in therapy for adults with personality disorder. Personal Ment Health 2008;2:1-6. http://dx.doi.org/10.1002/pmh.19.
- Crawford MJ. Can deficits in social problem-solving in people with personality disorder be reversed?. Br J Psychiatry 2007;190:283-4. http://dx.doi.org/10.1192/bjp.bp.106.031179.
- Paris J. Problem-solving therapy improves social functioning in people with personality disorder. Evid Based Ment Health 2007;10. http://dx.doi.org/10.1136/ebmh.10.4.121.
- McMurran M, Wilmington R. A Delphi survey of the views of adult male patients with personality disorders on psychoeducation and social problem-solving therapy. Crim Behav Ment Health 2007;17:293-9. http://dx.doi.org/10.1002/cbm.663.
- Personality Disorder: No Longer a Diagnosis of Exclusion. Policy Implementation Guidance for the Development of Services for People with Personality Disorder. London: National Institute for Mental Health in England; 2003.
- The Personality Disorder Capabilities Framework. Leeds: National Institute for Mental Health in England; 2003.
- Loranger AW. International Personality Disorder Examination (IPDE): DSM-IV and ICD-10 Interviews. Odessa, FL: Psychological Assessment Resources, Inc.; 1999.
- Diagnostic and Statistical Manual of Mental Disorders. Arlington, VA: American Psychiatric Association; 2000.
- The ICD-10 Classification of Mental and Behavioural Disorders. Geneva: World Health Organization; 1992.
- Fast Track 20 Questions. Leicester: National Institute of Adult Continuing Education; 2003.
- D’Zurilla TJ, Nezu AM. Problem-Solving Therapy: A Positive Approach to Clinical Intervention. New York, NY: Springer Publishing Co.; 1999.
- D’Zurilla TJ, Nezu AM. Problem-Solving Therapy: A Social Competence Approach to Clinical Intervention. New York, NY: Springer Publishing Co.; 1999.
- Zigmond AS, Snaith RP. The Hospital Anxiety and Depression Scale. Acta Psychiatr Scand 1983;67:361-70. http://dx.doi.org/10.1111/j.1600-0447.1983.tb09716.x.
- Horvath AO, Greenburg LS. Development and validaton of the Working Alliance Inventory. J Couns Psychol 1989;36:223-33. http://dx.doi.org/10.1037/0022-0167.36.2.223.
- Beecham J, Knapp M, Thornicroft G. Measuring Mental Health Needs. London: Gaskell; 2001.
- EuroQol Group . EuroQol: a new facility for the measurement of health-related quality of life. Health Policy 1990;16:199-208. http://dx.doi.org/10.1016/0168-8510(90)90421-9.
- Schulz KF, Altman DG, Moher D. CONSORT Group . CONSORT 2010 statement: updated guidelines for reporting parallel group randomized trials. Ann Int Med 2010;152:726-32. http://dx.doi.org/10.7326/0003-4819-152-11-201006010-00232.
- Zwarenstein M, Treweek S, Gagnier JJ, Altman DG, Tunis S, Haynes B, et al. Improving the reporting of pragmatic trials: an extension of the CONSORT statement. BMJ 2008;337. http://dx.doi.org/10.1136/bmj.a2390.
- Boutron I, Moher D, Altman DG, Schulz K, Ravaud P. CONSORT group . Methods and processes of the CONSORT group: example of an extension for trials assessing nonpharmacologic treatments. Ann Int Med 2008;148:W60-6. http://dx.doi.org/10.7326/0003-4819-148-4-200802190-00008-w1.
- Hopewell S, Clarke M, Moher D, Wager E, Middleton P, Altman DG, et al. CONSORT for reporting randomized controlled trials in journal and conference abstracts: explanation and elaboration. PLOS Med 2008;5. http://dx.doi.org/10.1371/journal.pmed.0050020.
- Dunn G, Maracy M, Tomenson B. Estimating treatment effects from randomized clinical trials with noncompliance and loss to follow-up: the role of instrumental variable methods. Stat Methods Med Res 2005;14:369-95. http://dx.doi.org/10.1191/0962280205sm403oa.
- Seivewright H, Green J, Salkovskis P, Barrett B, Nur U, Tyrer P. Cognitive-behavioural therapy for health anxiety in a genitourinary medicine clinic: randomised controlled trial. Br J Psychiatry 2008;193:332-7. http://dx.doi.org/10.1192/bjp.bp.108.052936.
- Research Governance Framework for Health and Social Care. London: Department of Health; 2005.
- Borderline Personality Disorder: Recognition and Management. London: NICE; 2009.
- Antisocial Personality Disorder: Treatment, Management and Prevention. London: NICE; 2009.
- Day F, McMurran M, Duley L. ‘Things often get worse before they get better’: participants’ responses to the cessation of recruitment and trial treatment in a trial of a psychological therapy for people with personality disorder following a safety alert. (A missed opportunity for collaboration?). Trials 2013;14. http://dx.doi.org/10.1186/1745-6215-14-S1-P94.
- Tyrer P, Johnson T. Establishing the severity of personality disorder. Am J Psychiatry 1996;153:1593-7. http://dx.doi.org/10.1176/ajp.153.12.1593.
- Puhan MA, Frey M, Büchi S, Schünemann HJ. The minimal important difference of the Hospital Anxiety and Depression Scale in patients with chronic obstructive pulmonary disease. Health Qual Life Outcomes 2008;6. http://dx.doi.org/10.1186/1477-7525-6-46.
- Priebe S, Bhatti N, Barnicot K, Bremner S, Gaglia A, Katsakou C, et al. Effectiveness and cost-effectiveness of dialectical behaviour therapy for self-harming patients with personality disorder: a pragmatic randomised controlled trial. Psychother Psychosom 2012;81:356-65. http://dx.doi.org/10.1159/000338897.
- Kvarstein EH, Arnevik E, Halsteinli V, Rø FG, Karterud S, Wilberg T. Health service costs and clinical gains of psychotherapy for personality disorders: a randomized controlled trial of day-hospital-based step-down treatment versus outpatient treatment at a specialist practice. BMC Psychiatry 2013;13. http://dx.doi.org/10.1186/1471-244X-13-315.
- Barrett B, Byford S. Costs and outcomes of an intervention programme for offenders with personality disorders. Br J Psychiatry 2012;200:336-41. http://dx.doi.org/10.1192/bjp.bp.109.068643.
- Dolan P, Gudex C, Kind P, Williams A. A Social Tariff for EuroQol: Results from a UK General Population Survey. York: Centre for Health Economics, York Health Economics Consortium, The University of York, NHS Centre for Reviews and Dissemination; 1995.
- Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol 2006;3:77-101. http://dx.doi.org/10.1191/1478088706qp063oa.
- Clarke M, Fardouly P, McMurran M. A survey of how clinicians in forensic personality disorder services engage their service users in treatment. J Forens Psychiatry Psychol 2013;24:772-87. http://dx.doi.org/10.1080/14789949.2013.862292.
- Huband N, Evans C, Duggan C, Khan O. Personality disorder traits and self-reported target problems in a treatment-seeking sample. Clin Psychol Psychother 2014;21:132-9. http://dx.doi.org/10.1002/cpp.1825.
- Candy PC. Self-direction for Lifelong Learning: A Comprehensive Guide to Theory and Practice. San Francisco, CA: Jossey-Bass; 1991.
- Duggan C, Parry G, McMurran M, Davidson K, Dennis J. The recording of adverse events from psychological treatments in clinical trials: evidence from a review of NIHR-funded trials. Trials 2014;15. http://dx.doi.org/10.1186/1745-6215-15-335.
- Day F, McMurran M, Duley L. The process of stopping recruitment and trial treatment in a trial of psychological therapy for people with personality disorder following a safety alert. Trials 2013;14. http://dx.doi.org/10.1186/1745-6215-14-S1-P13.
- Mowbray CT, Holter MC, Teague GB, Bybee D. Fidelity criteria: development, measurement and validation. Am J Eval 2003;24:15-340. http://dx.doi.org/10.1177/109821400302400303.
- McMurran M, Crawford MJ, Reilly JG, McCrone P, Moran P, Williams HC, et al. Psycho-education with problem solving (PEPS) therapy for adults with personality disorder: a pragmatic multi-site community-based randomised controlled trial. Trials 2011;12. http://dx.doi.org/10.1186/1745-6215-12-198.
- Barnicot K, Katsakou C, Marougka S, Priebe S. Treatment completion in psychotherapy for borderline personality disorder: a systematic review and meta-analysis. Acta Psychiatr Scand 2011;123:327-38. http://dx.doi.org/10.1111/j.1600-0447.2010.01652.x.
- Swift JK, Greenberg RP. Premature discontinuation in adult psychotherapy: a meta-analysis. J Consult Clin Psychol 2012;80:547-59. http://dx.doi.org/10.1037/a0028226.
- McCrone P, Dhanasiri S, Patel A, Knapp M, Lawton-Smith S. Paying the Price. The Cost of Mental Health Care in England to 2026. London: King’s Fund; 2008.
- Beckwith H, Moran PF, Reilly J. Personality disorder prevalence in psychiatric outpatients: a systematic literature review. Personal Ment Health 2014;8:91-101. http://dx.doi.org/10.1002/pmh.1252.
- Koehler JA, Lösel FA, Akoensi TD, Humphreys DK. A systematic review and meta-analysis on the effects of young offender treatment programs in Europe. J Exp Criminol 2013;9:19-43. http://dx.doi.org/10.1007/s11292-012-9159-7.
- Gibbon S, Duggan C, Stoffers J, Huband N, Völlm BA, Ferriter M, et al. Psychological interventions for antisocial personality disorder. Cochrane Database Syst Rev 2010;6. http://dx.doi.org/10.1002/14651858.cd007668.pub2.
- Stoffers JM, Völlm BA, Rücker G, Timmer A, Huband N, Lieb K. Psychological therapies for people with borderline personality disorder. Cochrane Database Syst Rev 2012;8. http://dx.doi.org/10.1002/14651858.cd005652.pub2.
- Gargon E, Williamson PR, Altman DG, Blazeby JM, Clarke M. The COMET Initiative database: progress and activities from 2011 to 2013. Trials 2014;15. http://dx.doi.org/10.1186/1745-6215-15-279.
- Omar H, Tejerina-Arreal M, Crawford MJ. Are recommendations for psychological treatment of borderline personality disorder in current UK guidelines justified? Systematic review and subgroup analysis. Personal Ment Health 2014;8:228-37. http://dx.doi.org/10.1002/pmh.1264.
- Ogrodniczuk JS, Joyce AS, Piper WE. Strategies for reducing patient-initiated premature termination of psychotherapy. Harv Rev Psychiatry 2005;13:57-70. http://dx.doi.org/10.1080/10673220590956429.
- Evans K, Tyrer P, Catalan J, Schmidt U, Davidson K, Dent J, et al. Manual-Assisted Cognitive–Behaviour Therapy (MACT): a randomized controlled trial of a brief intervention with bibliotherapy in the treatment of recurrent deliberate self-harm. Psychol Med 1999;29:19-25. http://dx.doi.org/10.1017/S003329179800765X.
- Davidson K, Norrie J, Tyrer P, Gumley A, Tata P, Murray H, et al. The effectiveness of cognitive behavior therapy for borderline personality disorder: results from the BOSCOT trial. J Person Disord 2006;20:450-65. http://dx.doi.org/10.1521/pedi.2006.20.5.450.
- Chanen AM, Jackson HJ, McCutcheon LK, Jovev M, Dudgeon P, Yuen HP, et al. Early intervention for adolescents with borderline personality disorder using cognitive analytic therapy: randomised controlled trial. Br J Psychiatry 2008;193:477-84. http://dx.doi.org/10.1192/bjp.bp.107.048934.
- Bateman A, Fonagy P. Randomized controlled trial of outpatient mentalization-based treatment versus structured clinical management for borderline personality disorder. Am J Psychiatry 2009;166:1355-64. http://dx.doi.org/10.1176/appi.ajp.2009.09040539.
- McMain SF, Links PS, Gnam WH, Guimond T, Cardish RJ, Korman L, et al. A randomized trial of dialectical behavior therapy versus general psychiatric management for borderline personality disorder. Am J Psychiatry 2009;166:1365-74. http://dx.doi.org/10.1176/appi.ajp.2009.09010039.
- McMain SF, Guimond T, Streiner DL, Cardish RJ, Links PS. Dialectical behaviour therapy compared with general psychiatric management for borderline personality disorder: clinical outcomes and functioning over a 2-year follow-up. Am J Psychiatry 2012;169:650-61. http://dx.doi.org/10.1176/appi.ajp.2012.11091416.
- Bateman AW. Treating borderline personality disorder in clinical practice. Am J Psychiatry 2012;169:560-3. http://dx.doi.org/10.1176/appi.ajp.2012.12030341.
- Talking Therapies: A Four Year Plan of Action. London: Her Majesty’s Stationery Office; 2011.
- McMurran M. Readiness to engage in treatments for personality disorder. Int J Forensic Ment Health 2012;11:289-98. http://dx.doi.org/10.1080/14999013.2012.746754.
- Jonsson U, Alaie I, Parling T, Arnberg FK. Reporting of harms in randomized controlled trials of psychological interventions for mental and behavioral disorders: a review of current practice. Contemp Clin Trials 2014;38:1-8. http://dx.doi.org/10.1016/j.cct.2014.02.005.
- Vaughan B, Goldstein MH, Alikakos M, Cohen LJ, Serby MJ. Frequency of reporting of adverse events in randomized controlled trials of psychotherapy vs. psychopharmacotherapy. Compr Psychiatry 2014;55:849-55. http://dx.doi.org/10.1016/j.comppsych.2014.01.001.
- Linden M. How to define, find and classify side effects in psychotherapy: from unwanted events to adverse treatment reactions. Clin Psychol Psychother 2013;20:286-96. http://dx.doi.org/10.1002/cpp.1765.
- Lilienfeld SO. Psychological treatments that cause harm. Perspect Psychol Sci 2007;2:53-70. http://dx.doi.org/10.1111/j.1745-6916.2007.00029.x.
- Rice ME, Harris GT, Harrison K, Rainey B. Handbook of Legal and Ethical Aspects of Sex Offender Treatment and Management. Chichester: Wiley-Blackwell; 2013.
- Curtis L. Unit Costs of Health and Social Care 2013. Canterbury: Personal Social Services Research Unit; 2013.
- Reference Costs 2011. London: Department of Health; 2012.
- 2009/2010 Fees and Charges Handbook. Kempston: Bedfordshire Police; 2009.
- Heslin M, Callaghan L, Barrett B, Lea S, Eick S, Morgan J, et al. Costs of the police service and mental health care pathways experienced by individuals with enduring mental health needs [published online ahead of print 17 March 2016]. Br J Psychiatry 2016. http://bjp.rcpsych.org/content/early/2016/03/10/bjp.bp.114.159129.long (accessed 26 April 2016).
- Office for National Statistics . Annual Survey of Hours and Earnings 2012 2013. www.ons.gov.uk/ons/dcp171778_335027.pdf (accessed 1 October 2014).
Appendix 1 Psychoeducation facilitators’ manual
Reproduced with permission from Banerjee P, D’Silva K, Huband N, Duggan C. Psychoeducation for People with Personality Disorder. Unpublished manual. Nottingham, UK: Nottinghamshire Healthcare NHS Trust; 2009.
Appendix 2 Psychoeducation participant booklet
Reproduced with permission from Banerjee P, D’Silva K, Huband N, Duggan C. Psychoeducation for People with Personality Disorder. Unpublished manual. Nottingham, UK: Nottinghamshire Healthcare NHS Trust; 2009.
Psychoeducation participant booklet (PDF download)
© Personality Disorder Service Nottinghamshire Healthcare NHS Trust
Appendix 3 Problem-solving therapy manual
Appendix 4 Competence rating scale: International Personality Disorder Examination
Appendix 5 Competence rating scale: psychoeducation
Appendix 6 Competence rating scale: problem-solving therapy
Appendix 7 Psychoeducation therapist fidelity rating
Appendix 8 Record check data collection forms
Appendix 9 Unblinding script
Final version 2.0: 3 December 2010
At the start of the study, the following information should be given:
The study involves looking at two different treatments for personality disorder. To find out which treatment is best we are asking you to complete some questionnaires and interviews with me. Both you and your mental health team will know what treatment you are receiving, but to make sure the results are fair, it is important that I don’t know which treatment group you are in until you have finished taking part in the research. I will remind you not to tell me this information each time I contact you about the research.
Each time the research assistant assesses the participant, a reminder about the need for blinding should be issued.
There is a particular risk of unblinding at the administration of the CSRI at the second follow-up. A specific instruction is recommended at this point:
I am going to ask you about the services you have received in the past 6 months. Remember, it is important that I don’t know which treatment group you were in. So, you should not tell me whether or not you received PEPS sessions (i.e. psychoeducation and Stop & Think sessions). Just don’t mention these and tell me only about your use of other services.
Appendix 10 Summary of trial amendments
| Stage of the trial | Amendment reference | Protocol version and date | Summary of changes |
|---|---|---|---|
| Amendments implemented before recruitment started | Substantial amendment 1 (15 June 2010) | Version 2.0, dated 15 June 2010 | The protocol was amended to include revised eligibility criteria, clarification of arrangements for the trial DMEC, a change to the follow-up schedule and addition of the service use record check as a secondary outcome measure |
| Amendments implemented during the recruitment phase | Substantial amendment 2 (17 August 2010) | Version 3.0, dated 17 August 2010 | The planned requirement for a standardised form of treatment as usual was removed from the protocol |
| Substantial amendment 3 (4 October 2010) | Version 4.0, dated 4 October 2010 | The protocol was amended to include additional competence checks for the IPDE and delivery of psychoeducation, and to clarify the publication policy. In addition, the amendment included a procedure for providing feedback of the participants’ IPDE results at final follow-up and to allow provision of a summary of the trial results after the final report | |
| Substantial amendment 4 (1 April 2011) | Version 5.0, dated 1 April 2011 | In response to advice from the TSC and DMEC the definition of recordable adverse events was expanded and simplified to ensure that all potentially relevant adverse events were recorded. The reporting procedure was also clarified | |
| Substantial amendment 5 (17 August 2011) | N/A | The decision was taken to allow problem-solving therapy to be offered in mixed-sex groups. The participant information sheet was updated to remove the reference to single-sex groups | |
| Substantial amendment 6 (02 February 2012) | N/A | The interview schedule at final follow-up for participants in the treatment as usual arm was updated to remove references to standardised treatment as usual as per the changes implemented in Substantial amendment 2 | |
| Substantial amendment 7 (5 July 2012) | N/A | Following the decision to provide IPDE feedback to participants who requested this, at the end of their involvement in the trial the study team developed a procedure and supporting documents to ensure that this feedback was provided sensitively and within an appropriate context to avoid causing distress or worry to participants. The procedure, letters and report template for the provision of IPDE results were submitted for review | |
| Substantial amendment 8 (26 November 2012) | N/A | Retrospective approval was sought from the Research Ethics Committee for implementation of the trial changes as an urgent safety measure, following the decision to stop recruitment and delivery of the trial intervention | |
| Amendments implemented during the follow-up phase | Substantial amendment 9 (3 April 2013) | Version 6.0, dated 3 April 2013 | This amendment included proposals for a number of measures to improve follow-up completion, including the introduction of payments to participants, the facility to post the SFQ to participants for completion and submission of a participant newsletter |
Appendix 11 Schedule of Data Monitoring and Ethics Committee and Trial Steering Committee meetings
| DMEC meetings | TSC meetings |
|---|---|
| 2010 | |
| 19 October 2010 | 17 March 2010 |
| 2011 | |
| 15 February 2011 | 23 March 2011 |
| 22 June 2011 | |
| 16 September 2011 | |
| 2012 | |
| 28 February 2012 | 13 January 2012 |
| 15 October 2012 | 16 March 2012 |
| 15 August 2012 | |
| 1 November 2012 | |
| Plus subsequent joint TSC/TMG meetings regarding early stopping | |
| 2013 | |
| 9 January 2013 | |
| 20 March 2013 | |
| 6 December 2013 | |
Appendix 12 Unit costs used in economic evaluation
| Service | Unit cost | Source |
|---|---|---|
| GP | £4 per minute of patient contact | Curtis (2013)110 |
| Psychiatrist | £362 per face-to-face contact | |
| Other doctor | £135 per 30 minutes of patient contact | |
| Psychologist | £134 per hour of client contact | |
| Drug and alcohol advisor | £54 per clinic consultation | |
| Other counsellor/therapist | £63 per hour of client contact | |
| Home treatment/crisis team member | £37 per hour per team member | |
| Assertive outreach team member | £49 per hour of patient contact | |
| Early intervention team member | £37 per hour | |
| Social worker | £226 per hour of face-to-face contact | |
| Mental health nurse | £74 per hour of face-to-face contact | |
| Occupational therapist | £44 per hour | |
| Accident and emergency service | £125 per attendance | |
| Drug/alcohol service | £38 per user session | |
| Community mental health centre | £38 per user session | |
| Day care centre/day hospital | £38 per user session | |
| Drop-in centre | £38 per user session | |
| Self-help/support group | £38 per user session | |
| Class/group at a leisure centre | £38 per user session | |
| Adult education class | £38 per user session | |
| Other day care activity provided by team | £38 per user session | |
| Inpatient admission | £345 per day (psychiatric); £577 per day (other) | Department of Health (2012)111 |
| Contact with police | £1 per minute | Bedfordshire Police Authority (2009)112 |
| Night detained in a police cell | £414 per night | Heslin et al. (2016)113 |
| National average wage | £26,500 per year | Office for National Statistics (2013)114 |
Appendix 13 Intervention costs
Individual psychoeducation sessions
Delivered by a band 6 mental health nurse, psychology graduate or other mental health professional.
1-hour session.
Unit cost = £119 per session.
Problem-solving group session
Delivered by two band 6 mental health nurses, psychology graduates or other mental health professionals.
2-hour sessions.
Average of 7.5 participants.
Unit cost = £63 per session.
Individual support as part of group therapy
Delivered by a band 6 mental health nurse, psychology graduate or other mental health professional.
1-hour session.
Unit cost = £119 per session.
Appendix 14 Qualitative interview schedule: post intervention – psychoeducation
Appendix 15 Qualitative interview schedule: post intervention – problem-solving therapy
Appendix 16 Qualitative interview schedule: 72-week follow-up – psychoeducation with problem-solving therapy
Appendix 17 Qualitative interview schedule: 72-week follow-up – usual treatment
List of abbreviations
- CACE
- complier average causal effect
- CI
- confidence interval
- CMHT
- community mental health team
- CONSORT
- Consolidated Standards of Reporting Trials
- CPN
- community psychiatric nurse
- CSRI
- Client Service Receipt Inventory
- DMEC
- Data Monitoring and Ethics Committee
- DSM
- Diagnostic and Statistical Manual of Mental Disorders
- EQ-5D
- European Quality of Life-5 Dimensions
- GAF
- Global Assessment of Functioning
- GP
- general practitioner
- HADS
- Hospital Anxiety and Depression Scale
- IPDE
- International Personality Disorder Examination
- NCTU
- Nottingham Clinical Trials Unit
- NICE
- National Institute for Health and Care Excellence
- PD
- personality disorder
- PEPS
- psychoeducation with problem-solving
- QALY
- quality-adjusted life-year
- RCT
- randomised controlled trial
- SD
- standard deviation
- SFQ
- Social Functioning Questionnaire
- SPSI-R
- Social Problem-Solving Inventory – Revised: Short Version
- STEPPS
- Systems Training for Emotional Predictability and Problem Solving
- TMG
- Trial Management Group
- TSC
- Trial Steering Committee
- WAI-SR
- Working Alliance Inventory – Short Revised