Evaluating mental health decision units in acute care pathways (DECISION): a quasi-experimental, qualitative and health economic evaluation

Article history

The research reported in this issue of the journal was funded by the HSDR programme or one of its preceding programmes as award number 17/49/70. The contractual start date was in March 2019. The final report began editorial review in November 2021 and was accepted for publication in September 2022. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The HSDR editors and production house have tried to ensure the accuracy of the authors’ report and would like to thank the reviewers for their constructive comments on the final report document. However, they do not accept liability for damages or losses arising from material published in this report.

Copyright statement

Copyright © 2023 Gillard et al. This work was produced by Gillard et al. under the terms of a commissioning contract issued by the Secretary of State for Health and Social Care. This is an Open Access publication distributed under the terms of the Creative Commons Attribution CC BY 4.0 licence, which permits unrestricted use, distribution, reproduction and adaptation in any medium and for any purpose provided that it is properly attributed. See: https://creativecommons.org/licenses/by/4.0/. For attribution the title, original author(s), the publication source – NIHR Journals Library, and the DOI of the publication must be cited.

2023 Gillard et al.

Mental health crisis care

People living with challenges to their mental health can experience an acute mental health crisis while living in the community. This can include intense feelings of depression, anxiety or paranoia, including feeling suicidal, and can result in the individual and/or their family and loved ones feeling that they are unable to cope without urgent, professional support. In England, people who are already using secondary mental health services in the NHS might be able to contact their care coordinator, an out-of-hours service or mental health crisis line, and be referred directly to crisis care services. For example, they might receive regular visits from a crisis resolution and home treatment (CRHT) team for a short period until the immediate crisis has passed or be admitted to psychiatric inpatient care for a period of assessment and/or treatment. People who are not already in contact with mental health services, or who are unable to wait to be referred to specialist care, will often present at hospital emergency departments (ED). People who self-harm or attempt suicide as a result of their distress can self-present at, or be transported to, an ED by emergency services. In ED, people will first be triaged by a member of the ED team and if a mental health need is identified – and once any immediate medical need (e.g. resulting from self-injury) is resolved – be seen by a member of the liaison psychiatry, assessed, and follow-up care arranged (including, if deemed necessary, a psychiatric inpatient admission).

Mental health crisis care, as described above, is under intense pressure in the UK1 and in equivalent systems internationally. 2–6 Visits to EDs for mental health issues are increasing while the number of available psychiatric inpatient beds is decreasing, resulting in challenges to the ED system7 and lengthy waits in ED for people in mental health crisis. 8 In England, the ED system has been described as near breaking point. 9 Approximately two in three of all people with multiple attendances at ED have been in contact with specialist mental health services or have had a previous acute psychiatric admission,10 with frequent attenders at greater risk of psychiatric inpatient admissions. 11 People presenting with a mental health issue are over six times more likely than people presenting with a physical concern to wait more than 4 hours at the ED12 (in England breaches of a 4-hour wait are a key performance indicator for EDs). Mental disorders are estimated to account for around 5% of ED attendances in the UK and almost 30% of acute inpatient bed occupancy and acute readmissions. 13 Poor experiences and low levels of satisfaction with mental health care all point to the ED as not being a far from ideal environment for support and treatment for mental health crisis. 12,14

Psychiatric inpatient stays can be costly,15 in some cases detrimental to mental health,16 disproportionately harmful to people from some minority ethnic groups,17 and reportedly unnecessary for as many as 17% of referred individuals. 18 Admissions following an acute crisis can be brief (often < 5 days), yet the effectiveness, cost-effectiveness and outcomes of short stays on psychiatric wards followed by early discharge is unclear. 19 Bed occupancy in inpatient psychiatric facilities is well above recommended levels with 91% of wards operating above the recommended occupancy rate. 13

The emergence of liaison psychiatry services has enabled mental health NHS trusts to provide responsive mental health assessment, advice and onward referral within emergency care settings, but wide variations in service provision20 remain and there are ongoing challenges to sustainability. 21 The introduction of CRHT teams22,23 and triage wards24 has offered little benefit in reducing contact with acute services, inpatient admissions or costs across the wider inpatient system compared with standard models of care, with ongoing staff concerns over the accuracy of triage decisions for mental health presentations in ED. 25,26 To address these growing challenges, policy in England has called for the development and evaluation of new, more effective models of crisis care as collaborations between health, mental health, social care, third sector and emergency service providers locally. 27,28 Alongside new street triage services,29 crisis houses30 and crisis cafes,31 some delivered in the third sector,32 psychiatric decision units (PDUs) have emerged as just one of a number of responses.

The psychiatric decision unit

Psychiatric decision units – also known in England as mental health decision units, crisis assessment units or assessment suites, among similar terms – have been set up in response to policy and the need to manage demand, to reduce unhelpful admissions to acute inpatient care, especially avoidable short admissions and expensive out-of-area or private admissions, and reduce mental health presentations at ED. International counterparts to the PDU fulfil a similar function,33 can be known as a psychiatric emergency service (PES), crisis stabilisation units or behavioural assessment units, and have become increasingly critical to the delivery of mental health crisis care. 34–36 This is particularly true for the USA where a third of hospitals are estimated to provide these emergency units,37 but they are also present in, for example, France,38 Singapore39 and Australia. 40

There is no single service specification for PDUs in England but rather a shared set of characteristics. PDU are short-stay facilities, based either on psychiatric or general hospital sites, for people in acute mental health crisis, offering time-limited care (typically up to 24–72 hours) including overnight stay. Units target people who experience repeat, often extended stays in ED, frequent use of other services such as the police and ambulance services, and have complex and frequent crisis-related needs but who might not benefit from a psychiatric inpatient stay. The focus of the units is on providing a comprehensive assessment in a calm, safe environment, offering therapeutic input as appropriate, and onward signposting and referral to a range of community-based care, both within and outside the NHS, hopefully breaking the cycle of repeat ED presentation and/or unhelpful acute admission. PDUs are distinguishable from triage or assessment wards; short-stay wards which typically accept all patients likely to require assessment or treatment in an inpatient setting. 41 Additionally, PDUs typically only accept informal patients (i.e. people not admitted assessment or treatment sections of the Mental Health Act), whereas triage wards will admit people formally under the Mental Health Act. Furthermore, as admission to PDU is not a formal inpatient admission, PDU staff are not required to complete inpatient treatment plans or the clustering tool for admission as they would on a ward, freeing up more time for individual face to face contact. Staff-to-patient ratio – at around 1 : 2 – is also typically higher than in an inpatient ward (typically around 1 : 4). 42 PDUs are often nurse led, supported by healthcare assistants (HCAs), with consulting input from psychiatry and other mental health professionals. PDUs can be co-located and share staffing with a Section 136 Place of Safety. Overnight accommodation is single sex, sometimes with flexible partitioning to enable the unit to respond to different numbers of male and female visitors, generally comprising reclining seating rather than beds. Units tend to be small with a capacity of around six to eight.

Efforts have been made to establish PDUs as an integrated part of the crisis care pathway. 43 PDUs accept referrals directly from liaison psychiatry teams based in ED, other mental health crisis services (e.g. CRHT and street triage teams), and sometimes community mental health services, primary care or third sector services – but do not generally accept self-referrals – with formal gatekeeping procedures in place. An example of the location of a PDU in the crisis care pathway in one of our sites can be seen in Figure 1, with the PDU playing a gatekeeping (assessment and referral) role for people identified by crisis services as having a higher level of need than might be met by intensive home treatment yet not in need of a Mental Health Act assessment, and from where a decision can be made about either subsequent inpatient admission or a return home with an appropriate support plan.

FIGURE 1.

Crisis care pathway map.

Aims and objectives

Although formal evaluations of recently developed PDUs in the USA and Australia have suggested that PDU-type units might reduce length of stay (LOS) in EDs and inpatient psychiatric admissions among people experiencing mental health crisis,37,40 evidence regarding the characteristics and effectiveness of PDUs in England is restricted to informal local evaluations. 44,45 While these reports suggest PDUs have potential to reduce demand on ED and psychiatric admissions, key data have not been reported (e.g. LOS in the ED) and study designs have not adequately accounted for variables that might confound comparison. In addition, PDUs in England have been developed organically, in response to policy and the pressure on services rather than the evidence base. There is a need for a clear description of the PDU model, including identification of key variables in unit configuration and function, and an understanding of how PDUs fit alongside other services in the crisis care pathway. It is possible PDUs introduce further fragmentation to the system, and, if not effective, may waste critical resources. As such, a formal evaluation of these services is urgently required to describe the model of care and generate much-needed knowledge about impacts, quality, and cost benefits.

The aim of the study is to ascertain the structure and activities of operational PDUs in England and to provide an evidence base for their effectiveness, costs and benefits, and optimal configuration. The study aim is addressed through the following research questions:

1. What is the range of hospital-based, short stay interventions internationally designed to reduce admissions to acute psychiatric inpatient care and what is their effectiveness?

2. What is the scope and prevalence of PDUs nationally and how are they configured?

3. How has the introduction of PDUs impacted on psychiatric inpatient admissions and ED psychiatric episodes/breaches?

4. What are the care pathways before and following an admission to the PDU?

5. What is the impact of the introduction of PDUs on inequalities of access to acute mental health services?

6. How do service users experience PDUs, as well as crisis care pathways before and after admission to PDU?

7. How are decisions made about referral and admission to PDU, and assessment and onward signposting and referral?

8. How do the economic costs and impacts of PDUs compare with areas without PDUs?

9. How do the costs for individual service users post PDU implementation compare with their costs prior to the introduction of PDUs to crisis care pathways?

A mixed-methods approach

This is a mixed-methods study in six work packages (WP): WP1 – systematic review and service mapping; WP2 – interrupted time series (ITS) analysis; WP3 – synthetic control study; WP4 – cohort study; WP5 – qualitative interview study; WP6 – health economic analysis. A mixed-methods approach is taken to address the range of research objectives identified in Chapter 1, incorporating a number of different types and sources of data necessary to answer a broad set of research questions. 46 The underlining framework is a multilevel organisational research approach proposing that findings at an individual level cannot be assumed to apply at a higher (e.g. population) level or vice versa, because the ‘nested complexity of organisational life’ impacts on the phenomena we are trying to understand or measure. 47 Drawing on Goffman’s multilevel frame analysis,36 it is necessary to ‘frame’ our enquiry at macro, meso and micro levels ‘to understand the pace, direction and impact of organisational innovation and change’37 as well as the interconnection between levels. This involves specifying, at each level, the construct we wish to test, how we will measure that construct, what our sample or data source will be, and what analytical approaches we will use. Best available data are used from a range of sources at each level to produce utilisable knowledge,35 informing the further development and implementation of PDUs nationally. We conceptualise our levels of enquiry as:

• Macro – national: how do policy, clinical guidance and other trends at a national level impact on the effectiveness and cost benefits of PDUs?

• Meso – organisational: how does the configuration of crisis care pathways (including the provision of other crisis care services) and the structure of PDUs at site level impact on the effectiveness and cost benefits of PDUs?

• Micro – individual: how do individual service user experiences of crisis care (including the PDU) and individual clinical staff decision-making processes along the pathway impact on the effectiveness and cost benefits of PDUs?

The specific way in which we frame research questions at each level and identify data sources and research methods for each of the six WPs is detailed in Table 1. Note that some questions (3 and 9) are broken down further with subquestions (a and b) and that on question, 3a is addressed in three different WPs (2–4), each using a different data set and, in some cases, covering different time periods. The challenges of this for data synthesis are addressed in Chapter 9. A diagram indicating how WPs are sequenced is given in Figure 2.

FIGURE 2.

Sequencing of work programmes.

Patient and public involvement in the project

A co-production approach to research has underpinned this project. The team leading the research has a strong track record in methodological development, support and evaluation of coproduction throughout a research project. In previous National Institute for Health and Care Research (NIHR)-funded research we developed a set of characteristics of co-produced research:

1. High-value research decision-making roles distributed across the research team (including team members with lived experience of using mental health services and/or mental distress, as well as clinical and university researchers).

2. Different interpretations of data within the research team owned and understood in terms of who offered the interpretation.

3. Consideration given to whether all members of the team are involved in the production of knowledge and the impact of this on the knowledge produced.

4. Methodological flexibility in the research process (where the scientific conventions of how research is usually might constrain the input of team members with lived experience).

5. Rigorous and critical reflection on how the research was done and the impact of this on findings.

6. Research outputs that report critically on how knowledge was produced. 48

Lived experience in the research team in this project comprised:

• co-investigator (KT): a qualitative researcher with many years’ experience of working from a lived experience perspective in mental health research

• three researchers explicitly employed to work from a lived-experience perspective (KA, LG, JL)

• Peer Expertise in Education and Research (PEER) group (a lived-experience research reference group at the lead site)

• Lived Experience Advisory Panel (LEAP): a group of eight people recruited nationally with lived experience of mental health distress, crisis and attending a PDU and or experience as an informal carer of someone with those experiences

• lived-experience representation on the project steering group.

We used lived experience in developing the project as follows:

• Two workshops with the PEER group discussing the study as a whole and the importance of employing a non-randomised study design in the context of crisis care, the ethics of use of routinely collected patient data in the cohort study, and identifying specific questions to be explored in qualitative interviews.

• Co-investigator KT contributed to developing the proposal from a lived-experience perspective, playing a key role in developing the co-produced approach to interpreting qualitative data sets,48 extended here to our data synthesis approach.

Using lived-experience during the research:

• A third workshop was held with the PEER group to support the application for NHS ethical approval for the study, including development of participant information sheets and informed consent procedures.

• The two researchers working from lived experience focused on development and delivery of WP5 data collection, analysis and write-up, as well as shaping and carrying out the WP1 mapping exercise and systematic review.

• The LEAP was facilitated by the two researchers working from lived experience with co-investigator KT’s support; the group met eight times during the study. The LEAP provided input into conduct of the study as it progressed, with input into material and application for ethical approval and the development of qualitative interview schedules.

• The LEAP also played an active role in the analysis of qualitative research data with members involved in the preliminary coding of interview data and in interpretive workshops.

• A second interpretive workshop involving the LEAP was held to ensure that lived experience informed the synthesis process bringing together data from all WPs to develop our final report.

• A final session of the LEAP was held to plan and design an applied output for the study (not reported here) aimed at helping people in crisis better negotiate the crisis care pathway.

• The lived-experience researchers on the team provided training to LEAP members as necessary, including around data analysis processes.

• Co-investigator KT oversaw patient and public involvement on the project and provided support to the lived experience researchers on the team, including facilitating a regular lived experience reflective space.

• There was lived experience on the study’s independent steering committee, brought by two committee members with who had made use of mental health crisis care, one of whom had also worked on a PDU in a support worker capacity.

We reflect on the impacts of patient and public involvement and our approach to coproduction in the study in the section on the Impact of patient and public involvement on the research.

Setting

With the exception of WP1, the study took place in four sites; sites were MHTs that had an operational PDU and the EDs at NHS hospitals in the same locality as the MHT that referred to the PDU. Key characteristics of study sites, including configuration of the PDU and other crisis care services available locally, are given in Table 2. Any changes in PDU or other crisis care provision during the time frame of the research, including those resulting from the COVID-19 pandemic, are indicated in parentheses.

Service mapping

Systematic review

Interrupted times series

Synthetic control study

Cohort study

Qualitative interview study

This WP combined a cross-sectional qualitative interview study with PDU staff and crisis care pathway staff referring to PDUs, with a longitudinal interview study with first-time visitors (service user) of PDUs.

Health economic analysis

The potential improved outcomes for mental health service users that may be associated with the creation of PDUs as an alternative option to traditional care pathways also have direct implications for resource use and costs. This is both for service users referred to PDUs, as well as broader impacts on local health economies, if there are spill-over benefits associated with implementation. Some previous analyses have indeed suggested that use of PDUs will lead to a reduction in both the number and length of inpatient admissions, as well as reduced attendance at ED, and therefore lead to a reduction in resource use and costs to healthcare systems. 78–81

The overall objective of the economic analysis was therefore to bring together findings from across WPs to identify potential impacts on the local health economy in each study area following the introduction of PDUs. The analysis draws on both the results of the ITS reported in Chapter 4 and the synthetic control study in Chapter 5 for site-specific decision model parameters on changes in area-level acute hospital ED attendances as well as area-level psychiatric hospital admissions (both informal and informal) at a (clinical) population-level following the introduction of PDUs.

As noted in the section on service mapping (see Chapter 3), the ITS provides data on service use over a 4-year period (spanning 2 years prior to and 2 years post PDU implementation) in MHTs and acute hospital EDs in three of the four sites. The synthetic control study (see the section on the systematic review) sought to match and compare mental health admissions and LOS, as well as ED attendances for the mental health and acute trusts in our four study areas with trusts with similar characteristics in areas of England where PDUs have not been implemented.

In addition to ED attendance and psychiatric inpatient admissions, contact with PDUs might be expected to have an impact on the use of community mental health services. Longitudinal cohort data have been used to explore these impacts. The section on the ITS sets out methods used for the cohort analysis undertaken and Chapter 6 describes how, in each of the four study areas, data have been collected on patterns of service contact and utilisation in the 9 months before and after an initial visit to a PDU. We have then estimated changes in resource use and costs to the NHS in the 9 months following a PDU visit for each of the four sites. To inform scenario modelling, regression analyses have been used to identify potential explanatory factors for differences in levels of cost. Selected service user subgroup analyses for costs also reflect different sociodemographic characteristics or factors associated with the use of mental health services.

The primary function of mental services is to improve health outcomes and not just impact on resource use and costs. Although we did not intend to directly look at changes in clinical outcomes, such as levels of mental distress, it has been possible to collect some self-report data on quality-of-life scores, using the EuroQOL EQ-5D-5L (EuroQol-5 Dimensions, five-level version) quality-of-life instrument for 148 participants referred to PDUs in two of the study areas, Lincoln and Sheffield. These data have been used to inform scenario modelling to help indicate the potential scope for improving quality of life of PDU service users. This can be useful, given that quality of life is the primary outcome measure used in economic analysis to inform decision-making in England, where judgements are typically concerned with assessing whether the additional costs incurred by a service are justified by quality-of-life gains.

Using these different sources of data chapter describes economic impact and the return on investment (RoI), from an NHS perspective, in each of the different study site areas. This compares the costs associated with PDU implementation with any subsequent cost offsets as well as additional costs incurred. Our service mapping review (see Chapter 3), where the resources required to implement PDUs in the different areas have been collected,82 has been used to estimate the costs of providing PDU services in each site; this review can also be used to explore the budgetary cost of expanding provision in England. In addition, we also look at impacts an individual service user level, scenarios that reflect the experiences of service users in qualitative analysis (see Chapter 7) have also been used to describe some potential individual journeys along service use pathways.

For all analyses costs are reported in 2019–20 prices, with costs for acute hospital-based contacts and specialists CMHTs taken from national reference cost. 83 Costs for psychiatric inpatient stays, as well as hourly costs for some additional community and hospital-based staff costs, are taken from the annual Unit Costs of Health and Social Care. 84 These staff costs estimate hourly costs using mean full-time equivalent basic salary for Agenda for Change (AfC) bands 4–9 of the April 2019 to March 2020 NHS staff earnings estimates. These estimates include salary overheads and other oncosts, including training, office, travel/transport, general supplies and utilities such as water, gas and electricity, as well as a share of capital overheads. The costs associated with PDUs have been estimated by applying these national salary costs to information provided by participating site area trust on the configuration of their PDU services, as well as previous publications that have estimated the costs of PDU (or similar) service provision. Discounting is not applied in this analysis, as only costs for up to 12 months are included.

Data synthesis

Synthesis of data from across WPs and across sites was conducted to critically appraise findings from the separate WPs, to provide insight into optimal configuration of PDUs in relation to the wider crisis care pathway and to inform potential future upscale and roll out of PDUs nationally. Data synthesis adopted a critical interpretive synthesis approach, as has been widely applied to the synthesis of quantitative and qualitative evidence in systematic reviews85 and the development of evidence-based practice. 86,87 In this approach, ‘constructs’ are derived from the various analyses (i.e. from descriptive analysis or hypothesis testing of quantitative data and thematic analysis of qualitative data) and mapped to an integrative grid that explores how those analyses interface. This enables the development of ‘synthesising arguments’ (analytical narrative) that offer explanatory insight into findings and inform applied learning from the research. An interpretive workshop involving members of the research team and LEAP was held to ensure that service user views and experiences informed this process alongside clinical and academic perspectives.

Following the workshop, a provisional set of constructs was specified, derived from preliminary findings from WP2–5. These constructs were further refined through discussion in the investigator team as we completed our analyses. The final data synthesis is presented in Chapter 9 and, together with our health economic analysis, provides the basis of our implications for policy and practice.

Changes from the proposal

Service mapping

A copy of the survey questionnaire has been published elsewhere. 82 This is an open access article distributed under the Creative Commons Attribution licence 4.0 (CCBY), which permits unrestricted use, distribution and reproduction in any medium, provided that the original work is properly cited. See https://creativecommons.org/licenses/by/4.0. The text and Tables 5 and 6 below include minor additions and formatting changes to the original. Survey responses were obtained from 50 of 53 NHS trusts with a relevant remit (94% response rate). The survey was completed by FOI officers, acute care pathway leads, service directors and lead nurses.

PDUs were present in a relatively small number of trusts, six (12% of trusts), with a further two planned but yet to open. 82 The locations of the trusts that had a PDU were Sheffield, Lincolnshire, Birmingham, Coventry and Warwickshire, South West London and Sussex. Of the PDUs planned, one would be in Nottinghamshire and one serving Rotherham, Doncaster and South Humber. Four decommissioned PDUs were identified – one in Leeds and three in Lancashire. Of the decommissioned PDUs, the unit in Leeds had operated with ward status and so staff had been unable to refer patients onward for inpatient care. This meant a protracted LOS for some in what was designed to be a short-stay unit with communal sleeping areas. Of the Lancashire units, one received an unfavourable quality report due to lengthy patient stays and dissatisfaction with unit layout and sleeping arrangements. 89 The Lancashire trust repurposed units as crisis assessment spaces without overnight stays.

All six PDUs were located on the mental health NHS trust site (see Table 5), although one of those was in a shared site by the MHT and the acute hospital. In five of six sites, the PDU was co-located with the trust’s place of safety (Section 136 facility). The majority of units were designed to reduce pressure on EDs, and half were designed to reduce inpatient admissions. Two PDUs had addition aims to reduce out-of-area placements and improve the patient experience. All PDUs facilitated overnight stays, with partitioned areas for sleeping in recliners rather than beds, and had a capacity of four to eight service users. All units only accepted voluntary patients. The majority of PDUs aimed to deliver both assessment and therapeutic input (four of six). All PDUs accepted referrals from liaison psychiatry, with the majority accepting referrals from CRHT teams and street triage (an outreach service run by the police and mental health services). However, substantial heterogeneity of pathways was also identified: referrals from third or voluntary sector services, police, general practitioner, approved mental health professional or self-referral when included in crisis care plan, were each only available at one trust. 82

PDUs had a high staff-to-patient ratio. In the day, the mean staff-to-patient ratio for nurses and HCAs combined was 1 : 2.1 [standard deviation (SD = 1.2)], rising at night to a mean of 1 : 2.3 (SD = 1.2). Staffing includes some allocated staff time from psychiatry (see Table 6). Although all units have a high staff-to-patient ratio, a sizeable difference was observed; units ranged from 1 : 1 staffing to 1 : 4 staffing. 83

Survey findings indicate that trusts with a PDU were approximately twice as likely than trusts without a PDU to have several crisis services, including crisis houses, crisis cafes or crisis drop-in services and acute day units (see Table 7). About half of trusts have hospital-based assessment services without overnight stays, and this is the same whether a trust has a PDU or not. The percentage of trusts with short-stay assessment wards was similar across trusts which have and do not have a PDU. 82

Systematic review

Twelve studies were included in the review. Figure 3 depicts the flow of information through the review. Study characteristics and further information regarding units evaluated in included studies can be seen in Table 8 and in Appendix 1, Table 33.

FIGURE 3.

Selection of studies for inclusion in systematic review.

Quality rating of included studies

Quality rating of each outcome of interest, for each study, are given for randomised (see Figure 4) and non-randomised studies (see Figure 5).

FIGURE 4.

Risk of bias ratings for outcomes from randomised studies.

FIGURE 5.

Risk of bias ratings for outcomes from non-randomised studies. Sources: Parwani et al.,37 Braitberg et al.,40 Tretheway et al.,45 Stamy et al.,79 Browne et al.,90 Gillig et al.,91 Kealy-Bateman et al.,92 Lester et al.,93 Mok and Walker,94 Schneider and Ross95 and Spooren et al. 96

We extracted 41 outcomes from 11 non-randomised studies, and four outcomes from a single RCT. 97 First, looking at the non-randomised studies, as indicated in Figure 5, we assessed there to be moderate risk of bias in a majority of outcomes considered in the review (27/41), from across most included studies (7/12). The most prevalent source of bias was potential bias in selection of reported result (in the absence of published protocols for most studies it was not clear that the full range of outcomes assessed in studies had been analysed and reported). Three outcomes from two studies91,95 were assessed as of serious risk of bias due to the risk of bias from confounding as the comparison groups were considered to be defined in a way which suggests that they are too dissimilar. Three further outcomes were at critical risk of bias because of additional biases identified or the bias due to confounding being rated as critical. Seven outcomes from one study79 were assessed as of low risk of bias. In the single RCT,97 two outcomes were assessed as of some concerns because of potential bias arising from the randomisation process, and two further outcomes of high concern because of, in addition, bias due to missing outcome data.

Synthesis of outcomes

Total emergency department length of stay

All four studies assessing total LOS in ED reported a significant reduction in this outcome. 37,40,80,94 Lester et al. 93 reported a mean decrease from 14.48 to 11.11 hours (p < 0.001), employing mixed model analysis using log-transformed ED lengths of stay. Parwani et al. 37 reported a reduction in median ED LOS from 155 minutes [interquartile range (IQR) 19–346] to 35 minutes (IQR 9–209) with autoregressive integrated moving-average modelling indicating this difference to be highly significant (p < 0.0001). Braitberg et al. 40 reported a reduction in median LOS from 328 minutes (IQR 227–534) in the pre-PDU period to 180 minutes (IQR 101–237; p < 0.001) in the post-PDU period, and Stamy et al. 79 a reduction from 351 minutes (IQR 204–631) to 334 minutes (IQR 212–517); a mean difference of 114 minutes (95% CI 87 to 143).

Where possible, data for ED lengths of stay were combined meta-analytically using mean difference random-effects models, with a pooled estimate for a reduction in total ED LOS of 164.24 minutes (95% CI −261.24 to −67.23 minutes; p < 0.001). The data from two studies could not be combined meta-analytically as one study did not report a measure of variance93 and another reported only reported medians. 37 An I² of 98% indicated high heterogeneity (see Figure 6). A GRADE starting rating of ‘low certainty, confidence or quality’ for outcomes of meta-analyses of non-randomised studies was upgraded to ‘moderate certainty’ due to the ROBINS-I ratings, indicating that authors believe that the true effect is probably close to the estimated effect.

FIGURE 6.

Forest plot of total length of ED stay in minutes. Sources: Braitberg et al. 40 and Stamy et al. 79

Emergency department wait time

One study40 reported a reduction in wait to be seen by a clinician in ED from a median of 68 minutes (IQR 24–130) in the control group to 40 minutes (IQR 17–86) in the experimental group (p < 0.001) and a reduction in median wait time for a mental health review from 139 minutes (IQR 57–262) to 117 minutes (IQR 49–224; p = 0.001). In another study,79 psychiatric boarding – time waiting in ED for a bed or transfer – decreased from 212 minutes (SD 119–536) to 152 minutes (SD 86–307) (mean difference 189 minutes; 95% CI 50 to 228). A reduction in long waits in ED was reported by one study,90 down from 12 patients per month who waited in ED for at least 24 hours in the pre-PDU opening period to only six 24-hour waits in the entire post-PDU period.

Emergency department environment: use of security services and restraint

Two studies40,90 assessed changes in adverse violent or threatening events that require a formal response from a security team (code grey events) and restrictive interventions from the pre- to post-PDU periods, although neither study was of high quality. Braitberg et al. 40 reported a reduction in events from 538 to 349 (p = 0.003) and a reduction in number of patients involved from 370 to 259 (p = 0.159). Browne et al. 90 reported a reduction in events in the ED but this was accompanied by an increase in events in the linked unit. Braitberg et al. 40 reported a reduction in number of patients experiencing any restrictive intervention from 338 patients (12.7%) to 255 patients (10.7%; p = 0.02), reduction in number of physical restraints from 339 (11.3%) to 224 (9.4%; p = 0.04), reduction in mechanical restraint from 275 (9.0%) to 156 (6.6%; p < 0.001) and reduction in use of therapeutic sedation from 250 (8.2%) to 156 (6.6%; p < 0.001). Browne et al. 90 reported a reduction in the total number of patients restrained from the pre- to post-PDU period, although size of cohort, total hours of restraint and average hours of restraint per patient were unclear.

Other emergency department outcomes

A single study79 reported no significant difference in the number of patients leaving ED without being seen between pre- and post-PDU opening periods, and similarly no difference in the combined number of patients leaving against medical advice. One study reported a reduction in numbers of patients brought to ED by street triage teams from the pre- to post-PDU opening periods but did not test this for significance. 45

Psychiatric inpatient admissions

A number of studies reported the impact of the PDU on inpatient psychiatric admissions. Parwani et al. 37 reported a significant reduction in the proportion of ED patients who experienced a psychiatric admission after the unit opened, from 42% to 25% (p < 0.001). Stamy et al. 79 reported that 301 (8.3%) patients presenting to ED were admitted to inpatient psychiatry in the ‘post’ period compared to 509 (15.3%) patients in the ‘pre’ period, representing a difference of −7.0 (95% CI −8.5 to −5.5). Lester et al. 93 reported that admissions to wards from ED reduced from 47.9% of presenting psychiatric patients to 38.0%, that discharge from ED rates were reduced by 39.1% to 28.2%, and that the total number of admissions to ward and transfers reduced from 58.8% to 50.9%. Gillig et al. 91 reported that 35% (156/435) of the intervention group were hospitalised from either ED or the PDU compared with 52% (181/348) of the control group, but these outcomes were assessed as being at serious risk of bias due to differences between the sites and populations they served. Tretheway et al. 45 reported that the number of patients admitted to a psychiatric ward by the ED-based liaison psychiatry team reduced from the pre- to post-PDU opening period but did not test for significance.

The data for psychiatric admissions from four studies where data were sufficiently well reported were combined meta-analytically (see Figure 7). As Gillig et al. 91 was assessed to be at critical risk of bias, a sensitivity analysis was conducted excluding the study. The combined OR is 0.55 (95% CI 0.43 to 0.68), with an I² value of 91%; data from 19,634 patients). The result was effectively unchanged in the sensitivity analysis (combined OR is 0.55; 95% CI 0.42 to 0.73, with I² of 94%). The GRADE rating for outcomes for meta-analyses of non-randomised studies of ‘low certainty, confidence or quality’ was upgraded to ‘moderate certainty’ due to the ROBINS-I ratings, indicating that we believe that the true effect is probably close to the estimated effect.

FIGURE 7.

Forest plot of inpatient psychiatric admissions. Sources: Parwani et al.,37 Stamy et al.,79 Gillig et al. 91 and Lester et al. 93

Psychiatric ward occupancy rates

Only one study94 reported data for occupancy rates for the ‘regular-stay unit’ as 94%, 98%, 99% and 95% in the pre-PDU period compared with 89%, 91%, 96% and 85% post PDU. The results are difficult to interpret as ‘regular-stay unit’ is not clearly defined, within month variance is not reported and no statistical analysis was conducted.

Length of psychiatric admission

One study93 reported a reduction in total time in hospital following ED presentation from a mean of 100.89 hours (median 46.15) hours to 91.00 hours (median 31.35), a significant reduction (p = 0.03) using log-transformed data in a mixed model. Van der Sande et al. 97 reported no significant difference in total number of inpatient days when time spent on the experimental unit was included; 33 days (SD 73.5) in the active group compared with 37 days (SD 83.0) in the control group. However, when inpatient days were compared excluding time spent on the experimental unit the difference was significant (z = −5.51, p < 0.001).

Hospitalisation in the follow-up period

Lester et al. 93 reported similar rates of hospitalisation across the groups following discharge from the unit (6.9% vs. 6.7%). Van der Sande et al. 97 reported a reduction in the number of patients who had a psychiatric inpatient admission during a 1-year follow-up in the PDU group (24%) compared with 38% in the control group but did not test for significance. Schneider and Ross95 reported a 30-day readmission rate for participants who stayed in the intervention unit similar to other patient samples, but this was deemed to be at serious risk of bias as the comparison groups were not clearly defined.

Psychometric outcomes

No psychometric outcomes (symptom scales or patient-reported outcomes) showed significant difference between the groups. Spooren et al. 96 (a study at moderate risk of bias) found no significant difference in scores on the General Health Questionnaire 2898 between groups at 1-month follow-up (t = –0.37, p = 0.715) or in the proportions of patients in each group who reported improvement in each group (t = 0.42, p = 0.677). Van der Sande et al. 97 reported no significant in any of the general symptom index [F(8,112) < 1, p = 0.72], hopelessness scale [F(1,110) = 2.14, p = 0.15] or Symptom Checklist-90; [F(8,110) = 1.03, p = 0.42].

Suicidality

Only one study, which described an experimental unit designed for those who had attempted suicide, reported data about changes to suicidality. 97 The study found that there was no significant difference in the number of suicide attempts per patient in the follow-up period (Z = 0.49, p = 0.62) and no difference in the probability of repeat suicide attempts in the follow-up period (hazard ratio of repetition for patients in the experimental group compared with the care as usual group was 1.24 (95% CI 0.68 to 2.27). For patients identified as at high risk of a repeat suicide attempt (a score of at least four on the Buglass and Horton scale),99 there was a non-significant difference in repeat attempts between experimental and control groups (log rank 2.69; p = 0.10). However, when gender was controlled for, males in the experimental group were found to be significantly more likely to repeat the suicide attempt (log rank 4.28; p = 0.04). Significantly more patients in the experimental group received outpatient care (including care specifically connected to the PDU) in the first year of follow-up (χ² = 37.42; df = 1, p < 0.001).

Deaths

Stamy et al. 79 reported a decrease in deaths – in the ‘pre’ sample of 23,336 individuals there was one fatality and in the ‘post’ sample (23,231 patients) there were no fatalities – but a power calculation was not reported and the result is unlikely to have sufficient power for conclusions to be drawn.

Health economic outcomes

Browne et al. 90 reported an annual reduction in the cost of one-to-one nursing in ED for those presenting with psychiatric problems of A$129,000, congruent with a reduction in time spent in ED and a decrease in the number of hours of one-to-one nursing care in the first 3 months after the unit fully opened compared with the same period in the previous year. However, no denominator or significance test was reported for this analysis. In the United States, Stamy et al. 79 reported additional revenue for the ED as a result of the experimental unit opening of US$404,954 in the 6 months and $861,065 annually.

Psychiatric decision units: structural characteristics and patterns of activity

Activity patterns and attendee characteristics from the first 2 years of PDU operation in each site are shown in Table 9. The number of PDU attendances was highest in BSMHFT, which has the largest capacity (eight) and several referral routes. Attendances were much less frequent in SHSCFT PDU, although the unit was closed for a 10-week period during 2020. Units were attended by a small majority of female service users, except for BSMHFT, where there was an even split between men and women. The distributions of attendees’ ages were comparable across sites, reflecting a tendency for about three-quarters of attendees to be aged 25–64 years. LOS on PDUs was longest for SWLSTG and shortest for LPFT. Precise LOS was not available for BSMHFT, although 43.4% and 39.4% of attendees, respectively, were discharged on the same day as admission or the next day, suggesting a relatively short LOS for most attendees. There were marked differences in the proportion of attendees subsequently admitted to psychiatric inpatient wards, ranging from just under one-third in LPFT to less than one in 10 in SHSCFT.

There were notable differences in the trajectories of attendance frequencies over the initial 24 months in each site (see Appendix 4, Figure 16). The number of attendances grew over the first 3 months post PDU implementation in BSMHFT, LPFT and SWLSTG sites but was steady thereafter for LPFT (70–80/month) and SWLSTG (50–60/month); in contrast, attendance levels in BSMHFT increased markedly from months 11–14 and afterwards the PDU maintained over 100 attendances/month. The frequency of attendances in SHSCFT was low initially, steadily climbing to 40/month by month 8 and largely maintaining that level thereafter aside from the closure period in March to June 2020.

Mental health trust outcomes

The catchment populations, annual rates and user characteristics of adult acute inpatient admissions and PLEs for each participating MHT, stratified by pre and post PDU implementation periods with pairwise comparisons, are described fully in Appendix 4, Table 34. Reliable data on inpatient admission and psychiatric liaison activity could not be sourced from SHSCFT and consequently they are not provided here nor included in subsequent ITS analyses.

In general, inpatient admission and PLE activity levels were considerably greater in BSMHFT (1772/year, 7446/year, respectively) and SWLSTG (1578/year, 4195/year) than LPFT (630/year, 1719/year). The frequency of inpatient admissions was comparable between pre and post PDU periods for BSMHFT and SWLSTG but higher in the latter period for LPFT. Across all trusts, the number of PLEs were significantly higher in the post PDU period. Individuals with previous (in last 24 months) inpatient admissions and PLE activity, respectively, made up a notable proportion of admissions (36.8–44.3%) and PLE (33.8–48.1%) in the study period, with a significant increase from pre to post PDU periods for the latter. There was a high proportion of women admitted to inpatient wards in SWLSTG (almost half) relative to BSFMHT and LPFT (40–45%). The (broad) distributions of age were more comparable, with approximately four of five admissions of service users aged 25–64 years. Overall, informal admissions made up about half of the admissions in BSMHFT and SWLSTG and over 60% in LPFT, although all trusts observed a significant post PDU decrease in the proportion of admissions that were informal. Length of inpatient stay was highest in BSMHFT where only 1 in 10 service users admitted stayed 5 days or less and lowest in SWLSTG where 1 in 5 had short stays. In the post-PDU period, length of inpatient stay increased in SWLSTG and LPFT as indicated by significant changes in median days and/or decreased frequency of 0- to 5-day stays, but there was a small but reliable decrease in LOS in BSMHFT. Daily bed occupancy significantly increased in the post-PDU period for all trusts where data were available. Out-of-area admissions, available for SWLSTG only, showed a marked decrease (from 75.5/ to 47.0/year) following PDU implementation.

Interrupted times series outcomes: psychiatric inpatient admissions and psychiatric liaison episodes

Weekly aggregated data concerning the frequency of informal acute adult inpatient admissions in pre and post PDU periods in participating MHTs are presented graphically in Figures 8–10. Corresponding parameter estimates from ITS analysis and estimates for secondary outcome measures are shown in Tables 10–12 (see Appendix 4, Figures 17–36 for accompanying graphical depictions).

FIGURE 8.

Impact of PDU implementation on informal inpatient admissions in BSMHFT. Note: The black vertical line represents implementation of the PDU.

TABLE 10 - Changes in level and trend of (weekly) informal acute adult inpatient admission and liaison psychiatry activity post PDU implementation in BSMHFT

p < 0.05.

p < 0.01.

p < 0.001.

Note

Bold indicates significant changes.

	Initial trend (pre PDU)	Step change (post PDU)	Trend change (post PDU)	Post PDU trend
	B (95% CI) weekly change	B (95% CI) step change	B (95% CI) weekly change	B (95% CI) weekly change
Results using Poisson GLM
Informal inpatient admissions	−0.001 (−0.003 to 0.001)	−0.176 (−0.334 to −0.018)a	0.003 (0.001 to 0.006) a	0.003 (0.001 to 0.005) c
	−0.07%	−16.16%	0.33%	0.30%
Informal inpatient admission with previous admission (< 24 months)	−0.003 (−0.005 to 0.0001)	−0.052 (−0.286 to 0.182)	0.005 (0.002 to 0.009) b	0.003 (0.0003 to 0.006) a
	−0.26%	−5.06%	0.53%	0.29%
All inpatient admissions	−0.0001 (−0.001 to 0.001)	−0.047 (−0.150 to 0.056)	0.001 (−0.001 to −0.002)	0.001 (−0.0004 to 0.002)
	−0.01%	−4.56%	0.07%	0.09%
Results using GLM with identity link
Formal inpatient admission (%)	0.039 (−0.030 to 0.108)	6.308 (0.813 to 11.803) a	−0.156 (−0.246 to −0.067) c	−0.112 (−0.174 to −0.050) c
	0.04% points	6.31% points	−0.16% points	−0.11% points
Length of inpatient stay
Five days or less inpatient admission (%)	0.003 (−0.030 to 0.036)	−0.924 (−3.729 to 1.881)	0.020 (−0.028 to 0.069)	0.028 (−0.012 to 0.069)
	0.003% points	−0.92% points	0.02% points	0.03% points
Log mean days	0.002 (−0.0003 to 0.004)	−0.170 (−0.369 to 0.029)	−0.001 (−0.004 to 0.002)	0.001 (−0.002 to 0.003)
	0.21% points	−15.63% points	−0.11% points	0.07% points
Daily bed occupancy (mean)	0.147 (0.073 to 0.221) c	−1.248 (−4.049 to 1.554)	−0.171 (−0.252 to −0.090)c	−0.027 (−0.056 to 0.002)
	0.15 beds	−1.25 beds	−0.17 beds	−0.03 beds
Results using Poisson GLM
PLEs	0.002 (0.002 to 0.003) c	−0.108 (−0.159 to −0.056) c	−0.0004 (−0.001 to 0.0003)	0.002 (0.001 to 0.003) c
	0.24%	−10.22%	−0.05%	0.21%
With previous episode (< 24 months)	0.005 (0.004 to 0.006) c	−0.158 (−0.226 to −0.090) c	−0.003 (−0.004 to −0.002) c	0.002 (0.001 to 0.003) c
	0.48%	−14.58%	−0.28%	0.20%

Inpatient admission and psychiatric liaison episode activity in Birmingham and Solihull Mental Health NHS Foundation Trust

Data indicate a significant decrease in the number of informal admissions immediately following PDU implementation (by 16%) but an increase in informal admission rate in the post PDU period (of 0.33%/week; see Figure 8, Table 10). There was a smaller (non-significant) drop in informal admissions upon implementation of the PDU for those with a recent previous admission, but a similar, reliable post-PDU increase in admission rate. These findings were mirrored by a significant increase in the proportion of formal admissions immediately after PDU implementation (from 52% to 58%) but a steady decrease in the weekly proportion of formal admissions in the 2 years following PDU implementation, reflecting a highly significant change in rate from pre to post PDU.

While there was no effect (either short- or long-term) of the PDU on overall admission rates or on admitted service users’ length of inpatient stay, there was a significant difference in trend of daily bed occupancy between pre and post PDU periods, reflecting a shift from a pre-PDU increase of 0.15 beds/week to a more stable pattern of daily bed occupancy post PDU (decrease of 0.03 beds/week). PLE frequency in the pre-PDU period was significantly increasing, by 0.24%/week, but decreased immediately following PDU implementation (by 10%). There was no significant change in the underlying increasing weekly trend post PDU, however, with PLE frequency continuing to increase (by 0.21%/week) in the post-PDU period. However, there were highly significant short- and long-term decreases in weekly frequency of PLE considering episodes by individuals with a recent previous episode, albeit in the context of an increasing weekly pre-PDU trend in this group of service users.

Inpatient admission and psychiatric liaison episode activity in Lincolnshire Partnership NHS Foundation Trust

The weekly frequency of informal admissions decreased immediately following PDU implementation by more than 10%, although this failed to reach significance (see Figure 9, Table 11). Nevertheless, there was a significant (long-term) change in pre- to post-PDU trend; in the 2-year post-PDU period, the number of informal admissions decreased by almost half a per cent/week. A similar pattern emerged (at a monthly level) considering only those informal admissions by service users with a recent previous admission. Prior to PDU implementation, weekly admission (formal and informal) frequency was increasing by 0.29%/week. This pattern was reversed in the post-PDU period where admissions decreased significantly by 0.18%/week, representing a highly significant change pre- to post-PDU trend (of 0.45%/week). The proportion of admissions that were formal was significantly increasing each week before PDU implementation. There was a further (immediate) increase (from 30% to 40%) on PDU implementation, and the pre-PDU trend continued in the post-PDU period pattern at the same rate (increase of 0.14% points/week). The PDU had less impact on length of inpatient stay, although the proportion of short stays, which was increasing prior to PDU implementation, did stabilise in the post-PDU period. There were significant trends during pre- and post-PDU periods indicating increasing PLE frequency, although when considering only those episodes by individuals with a recent previous episode there was a significant pre- to post-PDU trend decrease in (monthly) episode frequency.

FIGURE 9.

Impact of PDU implementation on informal inpatient admission in LPFT. Note: the black vertical line represents implementation of the PDU.

TABLE 11 - Change in level and trend of (weekly) informal inpatient admissions and liaison psychiatry activity post PDU implementation in LPFT

p < 0.05.

p < 0.01.

p < 0.001.

Note

Bold indicates significant changes.

	Initial trend (pre PDU)		Step change (following PDU)	Trend change (following PDU)	Post PDU trend
Results using Poisson GLM	B (95% CI) weekly change	B (95% CI) step change		B (95% CI) weekly change	B (95% CI) weekly change
Informal inpatient admissions	0.001 (−0.001 to 003)	−0.113 (−0.313 to 0.086)		−0.004 (−0.008 to −0.001) b	−0.005 (−0.007 to −0.002) b
	0.09%	−10.72%		−0.45%	−0.45%
All inpatient admissions	0.003 (0.001 to 0.005) b	−0.031 (−0.180 to 0.119)		−0.004 (−0.007 to −0.002) c	−0.002 (−0.003 to −0.0001) a
	0.29%	−3.03%		−0.45%	−0.18%
	Monthly change	Step change		Monthly change	Monthly change
Informal inpatient admission with previous admission (< 24 months)	0.010 (−0.008 to 0.028)	0.036 (−0.264 to 0.335)		−0.030 (−0.053 to −0.007) a	−0.017 (−0.034 to −0.001) a
	1.00%	3.64%		−2.96%	−1.72%
Results using GLM with identity link	B (95% CI) weekly change	B (95% CI) step change		B (95% CI) weekly change	B (95% CI) weekly change
Formal inpatient admission (%)	0.143 (0.049 to 0.236) b	10.275 (2.425 to 18.126) a		−0.025 (−0.152 to 0.103)	0.139 (0.050 to 0.228) b
	0.14% points	10.28% points		−0.03% points	0.14% points
Length of inpatient stay:
≤ 5 days inpatient admission (%)	0.085 (0.016 to 0.154) a	−2.502 (−8.444 to 3.441)		−0.064 (−0.157 to 0.029)	0.012 (−0.056 to 0.080)
	0.09% points	−2.50% points		−0.06% points	0.01% points
Log mean days	−0.003 (−0.007 to 0.001)	0.083 (−0.243 to 0.409)		0.001 (−0.004 to 0.006)	−0.002 (−0.004 to −0.0001) a
	−0.27%	8.64%		0.12%	−0.21%
Daily bed occupancy (mean)	0.007 (−0.007 to 0.018)	0.938 (−0.252 to 2.13)		−0.014 (−0.031 to 0.004)	−0.009 (−0.023 to 0.005)
	0.01 beds	0.94 beds		−0.01 beds	−0.01 beds
Results using Poisson GLM	B (95% CI) weekly change	B (95% CI) step change		B (95% CI) weekly change	B (95% CI) weekly change
PLEs	0.002 (0.0003 to 0.003) a	0.005 (−0.104 to 0.114)		0.001 (−0.001 to 0.003)	0.004 (0.003 to 0.005) c
	0.18%	0.51%		0.13%	0.39%
	Monthly change	Step change		Monthly change	Monthly change
With previous episode (< 24 months)	0.039 (0.027 to 0.051) c	−0.002 (−0.187 to 0.183)		−0.028 (−0.043 to −0.014) c	0.009 (0.0003 to 0.018) a
	3.97%	−0.22%		−2.79%	0.93%

Inpatient admission and psychiatric liaison episode activity in South West London and St George’s Mental Health NHS Trust

Prior to PDU implementation, there was a significant trend indicating an increase (of 0.27%/week) in informal admission frequency. Informal admissions decreased by almost 20% immediately following implementation of the PDU, and there was a highly significant change in pre- to post-PDU (long-term) trend, reflecting a decrease of 0.5%/week (see Figure 10, Table 12). The effect of the PDU on informal admissions for service users with a recent previous admission was very similar, although the post-PDU trend also indicated a significant weekly decrease. A significant change of weekly trend from pre- to post-PDU periods was also observed when considering all admissions (formal and informal), suggesting that PDU implementation arrested the prior trend of increasing admission frequency. The opening of the PDU in SWLSTG had an immediate impact on the proportion of patients admitted formally (increase by almost 7% points, from approximately 43% to 50%) and there was a significant change (weekly increase) in the pre- to post-PDU trend. There was little indication of an effect of PDU implementation on length of inpatient stay or daily bed occupancy rates over the study period. In a similar manner, there was no significant impact of PDU implementation on PLE frequency, which was increasing over the pre-PDU period (both for all service user episodes and for those by service users with a recent episode).

FIGURE 10.

Impact of PDU implementation on informal inpatient admissions in SWLSTG. Note: the black vertical line represents implementation of the PDU.

TABLE 12 - Changes in level and trend of (weekly) informal inpatient admissions and liaison psychiatry activity post PDU implementation for SWLSTG

p < 0.05.

p < 0.01.

p < 0.001.

Note

Bold indicates significant changes.

	Initial trend (pre-PDU)	Step change (following PDU)	Trend change (following PDU)	Post PDU trend
	B (95% CI) Weekly change	B (95% CI) Step change	B (95% CI) Weekly change	B (95% CI) Weekly change
Results using Poisson GLM
Informal inpatient admissions	0.003 (0.001 to 0.004) b	−0.218 (−0.357 to −0.078) b	−0.005 (−0.007 to −0.003) c	−0.002 (−0.004 to 0.0001)
	0.27%	−19.56%	−0.49%	−0.19%
Informal inpatient admission with previous admission (< 24 months)	0.003 (0.001 to 0.005) a	−0.219 (−0.402 to −0.037) a	−0.005 (−0.008 to −0.002) c	−0.002 (−0.005 to −0.0001) a
	0.28%	−19.70%	−0.53%	−0.24%
All inpatient admissions	0.002 (0.001 to 0.003) c	−0.089 (−0.184 to 0.007)	−0.003 (−0.004 to −0.001) c	−0.001 (−0.002 to 0.0002)
	0.22%	−8.50%	−0.28%	−0.09%
Results using GLM with identity link
Formal inpatient admission (%)	−0.036 (−0.098 to 0.026)	6.85 (1.685 to 12.017) b	0.109 (0.021 to 0.197) a	0.068 (−0.005 to 0.141)
	−0.04% points	6.85% points	0.11% points	0.07% points
Length of inpatient stay:
≤ 5 days inpatient admission (%)	−0.044 (−0.093 to 0.005)	1.110 (−3.478 to 5.697)	−0.005 (−0.076 to 0.065)	−0.044 (−0.097 to 0.010)
	−0.04% points	1.11% points	−0.01% points	−0.04% points
Log mean days	−0.002 (−0.005 to 0.0003)	0.128 (−0.059 to 0.315)	0.002 (−0.001 to 0.005)	0.0003 (−0.002 to 0.002)
	−0.021%	13.66%	0.24%	0.03%
Daily bed occupancy (mean)	0.035 (0.001 to 0.068) a	−1.234 (−3.492 to 1.024)	−0.029 (−0.069 to 0.010)	0.014 (−0.009 to 0.037)
	0.04 beds	−1.23 beds	−0.03 beds	0.01 beds
Results using Poisson GLM
PLEs	0.002 (0.001 to 0.003) c	0.018 (−0.050 to 0.087)	−0.001 (0.002 to 0.0003)	0.001 (0.001 to 0.002) c
	0.23%	1.86%	−0.07%	0.14%
PLEs with previous episode (< 24 months)	0.003 (0.002 to 0.004) c	0.075 (−0.034 to 0.184)	−0.001 (−0.003 to 0.0003)	0.001 (−0.0001 to 0.003)
	0.30%	7.79%	−0.15%	0.13%

Acute trust outcomes

The catchment populations, annual rates and user characteristics of ED mental health attendances for each participating acute trust, stratified by pre- and post-PDU implementation periods with pairwise comparisons, are described fully in Appendix 4, Table 35. In general, attendances were more frequent in SWBHFT (2242/year) and STHFT (2200/year) than SGUHFT (1980/year) and ULHFT (1620/year). Annual rates were comparable between pre- and post-PDU periods in SWBHFT and in SGUHFT but higher in the latter period for LPFT and lower post PDU in STHFT. A small majority of attendances in SWBHFT (55.1%), SGUHFT (61.1%) and STHFT (62.9%) were by service users who had previously attended (for any reason) in the last 24 months; there was a marked decreased in STHFT from pre (67.0%) to post PDU (53.8%), however. The overall proportion was much lower in ULHFT (34.4%) most likely because these attendances considered only those individuals with previous attendances involving a mental health presentation.

The sex of attending service users was, overall, evenly split between women and men in STFHT, SGUHFT and ULHFT but women only made up 39.5% of the attendees in SWBHFT over the study period, although this significantly increased post-PDU implementation. Approximately two-thirds of service users were aged 25–64 years in STHFT and ULHFT. This compared with three-quarters of attendees in SGUHFT, although the proportion of younger service users (aged 18–24 years) significantly increased post PDU. In all trusts, more than half of ED mental health attendances were by individuals arriving by ambulance or police. This was particularly apparent at ULHFT (66.8%), although there was a significant reduction in ambulance/police arrivals post PDU. Rate of admissions to an acute trust ward from ED was also higher in ULHFT (19.6%) relative to other trusts (8.6–12.7%) where significant decreases in acute trust ward admissions in the period following PDU implementation were observed. Median LOS for ED mental health attendances varied between 3 and 4 hours, depending on the trust, with 4-hour breach rates ranging from just under one-quarter in SWBHFT (23.8%) to almost 40% (39.2%) in ULHFT. While length of ED stay significantly decreased from pre to post PDU in SWBHFT, STHFT and ULHFT, it increased in SGUHFT. Further, the proportion of 4-hour breaches significantly increased post PDU in SGUHFT (by 2.4% points) and ULHFT (by 5.0% points).

Interrupted times series outcomes: emergency department mental health attendances

Weekly aggregated data concerning the frequency of ED mental health attendances in pre- and post-PDU periods in participating MHTs are presented graphically in Figures 11–14. Corresponding parameter estimates from ITS analysis and estimates for secondary outcome measures are shown in Tables 13 and 14 (see Appendix 4, Figures 37–52 for accompanying graphical depictions).

FIGURE 11.

Impact of PDU implementation on weekly number of ED mental health attendances in SWBHFT. Note: the black vertical line represents implementation of the PDU.

Emergency department mental health attendances in Sandwell and West Birmingham Hospitals NHS Foundation Trust

The data indicate that prior to PDU implementation, there was a highly significant increase in ED mental health presentations in SWBHFT (by approximately 0.5%/week). There was a significant decrease in the number of ED mental health attendances immediately following PDU implementation (by around 20%) but there was little impact on long-term trend, and ED attendance frequency continued to rise in the post-PDU period (by 0.4%/week; see Figure 11, Table 13). The pattern was similar when considering only those attendances by service users with a (recent) previous ED attendance, although the initial period was characterised by a more marked weekly increase in frequency (by approximately 0.8%/week) and there was a significant change (decrease) in trend from pre to post PDU. There was little impact of PDU implementation on the proportion of ED mental health attendances via ambulance or police or rates of 4-hour breaches, although there was a significant drop (> 10%) in length of ED stay immediately following PDU implementation. Curiously, the proportion of ED mental health attendances discharged to an acute trust ward decreased significantly in the period before PDU implementation but increased by about 4% on opening of the PDU, although was still steadily declining in the post-PDU period.

Emergency department mental health attendances in United Lincolnshire Hospitals NHS Foundation Trust

While there was no evidence of an immediate effect of PDU implementation on ED mental health attendances in ULHFT, there was a significant increase (by 0.3%/week) in attendance frequency in the post-PDU period, reflecting a change in pre- to post-PDU trend (see Figure 12, Table 13). PDU implementation had no significant short- or long-term effects on the proportion of ED mental health attendances where individuals arrived by ambulance or police, which was significantly decreasing in the pre-PDU period. There was, however, a significant increase from pre to post-PDU in the proportion of attendances where service users were discharged from ED to an acute trust ward, reversing the decreasing pre-PDU trend. There was evidence for a small but reliable weekly increase in (mean) LOS in ED following PDU implementation but no effect on 4-hour breach rate.

FIGURE 12.

Impact of PDU implementation on weekly number of ED mental health attendances in ULHFT. Note: the black vertical line represents implementation of the PDU.

TABLE 13 - Changes in level and trend of weekly ED mental health attendances in SWBHFT and ULHFT ED post PDU implementation

p < 0.01.

p < 0.001.

p < 0.05.

Note

Bold indicates significant changes.

	Initial trend (pre-PDU)	Step change (following PDU)	Trend change (following PDU)	Post-PDU trend
	B (95% CI) weekly change	B (95% CI) step change	B (95% CI) weekly change	B (95% CI) weekly change
SWBHFT
Number of ED mental health attendances	0.005 (0.001 to 0.008) a	−0.229 (−0.352 to −0.105) b	−0.001 (−0.004 to 0.002)	0.004 (0.003 to 0.005) b
	0.47%	−20.43%	−0.06%	0.40%
Number of ED attendances for those with previous attendance (< 24 months)	0.008 (0.004 to 0.011) b	−0.268 (−0.406 to −0.129) b	−0.004 (−0.007 to −0.0004) c	0.004 (0.003 to 0.005) b
	0.79%	−23.48%	−0.37%	0.40%
Arrival by ambulance/police (%)	0.010 (−0.141 to 0.161)	−1.818 (−7.169 to 3.533)	−0.005 (−0.155 to 0.145)	0.004 (−0.047 to 0.054)
	0.01% points	−1.82% points	−0.01% points	0.004% points
Discharge to acute trust ward (%)	−0.128 (−0.225 to −0.031) a	4.021 (0.579 to 7.463) c	0.086 (−0.011 to 0.183)	−0.041 (−0.074 to −0.008) c
	−0.13% points	4.02% points	0.09% points	−0.04% points
Length of stay in ED
4-hour breach (%)	0.061 (−0.112 to 0.235)	−6.00 (−12.25 to 0.247)	−0.014 (−0.187 to 0.158)	0.045 (−0.003 to 0.094)
	0.06% points	−6.00% points	−0.01% points	0.05% points
Log mean minutes	0.0005 (−0.001 to 0.002)	−0.121 (−0.202 to −0.040) a	0.0001 (−0.002 to 0.002)	0.001 (−0.0003 to 0.001)
	0.05%	−11.43%	0.01%	0.06%
ULHFT
Number of ED mental health attendances	0.0001 (−0.001 to 0.001)	0.047 (−0.066 to 0.161)	0.003 (0.001 to 0.004) a	0.003 (0.001 to 0.004) b
	0.01%	4.84%	0.26%	0.27%
Number of ED attendances for those with previous attendance (< 24 months)	0.002 (−0.001 to 0.004)	0.095 (−0.084 to 0.274)	0.0005 (−0.003 to 0.004)	0.002 (0.0003 to 0.004) c
	0.18%	9.98%	0.05%	0.24%
Arrival by ambulance/police (%)	−0.074 (−0.132 to −0.016) c	1.702 (−3.693 to 7.097)	0.036 (−0.040 to 0.111)	−0.050 (−0.113 to 0.012)
	−0.07% points	1.70% points	0.03% points	−0.05% points
Discharge to acute trust ward (%)	−0.094 (−0.148 to −0.041) a	3.885 (−0.513 to 8.283)	0.121 (0.056 to 0.186) b	0.027 (−0.014 to 0.068)
	−0.09% points	3.89% points	0.12% points	0.03% points
Length of stay in ED
4-hour breach (%)	0.005 (−0.052 to 0.063)	1.38 (−3.87 to 6.63)	0.032 (−0.045 to 0.110)	0.032 (−0.031 to 0.096)
	0.01% points	1.38% points	0.03% points	0.03% points
Log mean minutes	−0.0001 (−0.001 to 0.001)	0.029 (−0.043 to 0.100)	0.001 (0.0002 to 0.002) c	0.001 (0.0001 to 0.002) c
	−0.01%	2.94%	0.12%	0.11%

Emergency department mental health attendances in Sheffield Teaching Hospital NHS Foundation Trust: interrupted time series outcomes

There was a pre-PDU trend indicating a significant decrease (by 0.16%/week) in ED mental health attendances. There was a non-significant decrease in attendance frequency immediately following PDU implementation but a significant increase in pre- to post-PDU trend over the longer term (a single year post PDU in the case of STHFT), although weekly attendances did not increase in the post-PDU period (see Figure 13, Table 14). These effects were in the same direction, although more marked, when considering only those attendances by individuals with a recent ED attendance. In the pre-PDU period, the proportions of mental health presentations in ED where individuals arrived by ambulance or police and/or were discharged to an acute trust ward were decreasing significantly. However, following PDU implementation, significant short- and long-term increases in the rate of arrivals by ambulance or police and a significant increase in pre to post trend for attendances with discharge to acute trust ward occurred. LOS in ED was also significantly decreasing prior to implementation of the PDU, both with respect to (mean) minutes in ED and proportion of 4-hour breaches, although there was no significant (short- or long-term) impact on this trend following PDU implementation.

FIGURE 13.

Impact of PDU implementation on weekly number of ED mental health attendances in STHFT. Note: the black vertical line represents implementation of the PDU.

Emergency department mental health attendances in St George’s University Hospital NHS Foundation Trust

There was little evidence for either an immediate or long-term effect of the introduction of the PDU on attendances in SGUHFT. There was a small but reliable decrease (of 0.06% points/week) in the proportion of attendances where individuals arrived via ambulance or police in the post-PDU period (see Figure 14, Table 14). The rate of ED attendances with discharge to an acute trust ward increased significantly following PDU implementation and there was a highly significant change in pre to post PDU trend, although the proportion remained in steady decline in the post-PDU period. Average LOS in SGUHFT ED increased in the post-PDU period (by approximately 0.2%/week), significantly reversing a pattern of decreasing LOS prior to PDU implementation, although there was little evidence for an impact of the PDU on 4-hour breach rate.

FIGURE 14.

Impact of PDU implementation on weekly number of ED mental health attendances in SGUHFT. Note: the black vertical line represents implementation of the PDU.

TABLE 14 - Changes in level and trend of weekly ED mental health attendances in STHFT and SGUHFT post-PDU implementation

p < 0.05.

p < 0.01.

p < 0.001.

Note

Bold indicates significant changes.

	Initial trend (pre-PDU)	Step change (following PDU)	Trend change (following PDU)	Post-PDU trend
	B (95% CI) weekly change	B (95% CI) step change	B (95% CI) weekly change	B (95% CI) weekly change
STHFT
Number of ED mental health attendances	−0.002 (−0.003 to −0.00001) a	−0.040 (−0.199 to 0.118)	0.005 (0.0001 to 0.009) a	−0.001 (−0.006 to 0.005)
	−0.16%	−3.96%	0.46%	−0.08%
Number of ED attendances for those with previous attendance (< 24 months)	−0.003 (−0.005 to −0.001) b	−0.085 (−0.288 to 0.117)	0.007 (0.0004 to 0.013) a	−0.003 (−0.009 to 0.004)
	−0.32%	−8.19%	0.68%	−0.25%
Arrival by ambulance/police (%)	−0.288 (−0.446 to −0.129) c	10.128 (3.279 to 16.976) b	0.226 (0.034 to 0.418) b	0.033 (−0.138 to 0.204)
	−0.29% points	10.13% points	0.23% points	0.03% points
Discharge to acute trust ward (%)	−0.144 (−0.182 to −0.106) c	−0.965 (−4.466 to 2.536)	0.206 (0.125 to 0.287) c	0.030 (−0.066 to 0.125)
	−0.14% points	−0.97% points	0.21% points	0.03% points
Length of stay in ED
4-hour breach (%)	−0.102 (−0.161 to −0.043)c	6.810 (−0.633 to 14.253)	−0.027 (−0.217 to 0.162)	−0.025 (−0.304 to 0.253)
	−0.10% points	6.81% points	−0.03% points	−0.03% points
Log mean minutes	−0.002 (−0.003 to −0.001) c	0.089 (−0.040 to 0.218)	−0.0004 (−0.004 to 0.003)	−0.001 (−0.006 to 0.004)
	−0.17%	9.31%	−0.04%	−0.10%
SGUHFT
Number of ED mental health attendances	−0.001 (−0.002 to 0.0005)	0.087 (−0.010 to 0.184)	−0.0001 (−0.002 to 0.001)	−0.001 (−0.002 to 0.0002)
	−0.0.06%	9.12%	−0.01%	−0.08%
Number of ED attendances for those with previous attendance (< 24 months)	−0.0003 (−0.002 to 0.001)	0.062 (−0.051 to 0.175)	−0.0004 (−0.002 to 0.001)	−0.001 (−0.002 to 0.001)
	−0.02%	6.43%	−0.04%	−0.09%
Arrival by ambulance/police (%)	0.014 (−0.042 to 0.070)	−3.215 (−8.000 to 1.570)	−0.067 (−0.140 to 0.006)	−0.059 (−0.116 to −0.003) a
	0.01% points	−3.22% points	−0.07% points	−0.06% points
Discharge to acute trust ward (%)	−0.172 (−0.208 to −0.137) c	5.374 (3.041 to 7.706) c	0.133 (0.093 to 0.173) c	−0.035 (−0.059 to −0.010) b
	−0.17% points	5.37% points	0.13% points	−0.04% points
Length of stay in ED
4-hour breach (%)	−0.035 (−0.090 to 0.020)	2.576 (−2.116 to 7.268)	0.075 (−0.003 to 0.153)	0.041 (−0.018 to 0.101)
	−0.04% points	2.58% points	0.08% points	0.04% points
Log mean minutes	−0.001 (−0.002 to −0.0002) a	0.006 (−0.066 to 0.078)	0.003 (0.001 to 0.004) c	0.002 (0.001 to 0.003) c
	−0.11%	0.59%	0.27%	0.17%

Pooled estimates of impact of psychiatric decision unit implementation

Table 15 shows the fixed and random effect pooled estimates for primary ITS outcomes concerning informal inpatient admission and ED mental health attendance activity. Significant heterogeneity in site outcomes were observed for all estimates except informal inpatient admission step change, which suggested a robust decrease (of 16.6%) in informal admission frequency following PDU implementation across participating sites. While fixed effects suggested that, overall, there was a significant decrease in informal admission frequency trend from pre to post PDU (by about 0.2%/week), reversing a steady weekly increase in trend prior to PDU implementation, after adjustment for random effects, the pre-to post-PDU change in weekly trend was not significant. The introduction of PDUs had no overall effect on level or trend in mental health presentations at ED. Although fixed-effects modelling suggested a highly significant weekly increase in ED attendances in the post-PDU period, this was not significant after adjustment for random effects.

Meta-analysis of secondary ITS outcomes indicated that (after adjustment for random effects where appropriate) PDU implementation had significant overall effects on formal inpatient admission percentage (step change increase by 7.3%), PLE frequency in individuals with recent episode activity (pre-to post-PDU trend decrease of 0.25%/week), proportion of ED mental health attendances with discharge to acute trust ward (step change increase of 3.2% points, pre-to post-PDU trend increase of 0.14% points/week) and length of mental health ED stay (pre-to post-PDU trend increase of 0.10%/week). Fixed- and random-effect pooled estimates for secondary ITS outcomes are detailed in Appendix 4, Tables 36–38.

Psychiatric decision unit and pathway reconfiguration

Key changes to the crisis care pathway (other than PDU introduction) within the duration of the study period, elicited through semistructured interviews with strategic managers and contact with trust staff in each site, are presented in (superimposed on) graphs showing ITS analyses of PDU implementation on primary outcomes in Appendix 4, Figures 53–56. Explanation of these changes to the crisis care pathway and a more detailed list including minor changes are provided for each site in Appendix 4, Tables 39–42.

Implementation of the PDU often occurred as part of or concurrently with reconfiguration or expansion of crisis care services at participating sites. The influence of each major service initiative (that occurred sufficiently distant in time from the study period start/end and from PDU implementation) on primary outcomes was (separately) examined in ITS analyses that included each in conjunction with PDU implementation (see Appendix 4, Tables 43–46 for parameter estimates and detailed commentary). These analyses tended to reaffirm the significant decrease in informal inpatient admissions following PDU implementation in BSMHFT, LPFT and SWLSTG. But they also suggested some earlier initiatives significantly affected short-term admission rates, such as the introduction of street triage services in BSMHFT (decrease of 20.5%) and the opening of a psychiatric intensive care unit in LPFT (increase of 34.7%). In addition, the introduction of some services post-PDU implementation were also potentially impactful, particularly in sites where significant long-term post-PDU decreases in admissions were observed (LPFT and SWLSTG). For example, there was an immediate decrease in informal observed admissions after the launch of a crisis vehicular response unit (15 months post PDU) in LPFT (by 40.1%) and long-term changes in trend (decreases) in weekly admission rates in SWLSTG following the introduction of crisis cafe (20 weeks post-PDU) and street triage services (30 weeks post-PDU).

The impact of additional crisis care pathway changes on ED mental health attendances was more mixed. A Secure Quality Involvement initiative specifically intended to reduce repeat ED mental health attendances in STHFT introduced at the start of the time series is likely relevant to the highly significant decreases in attendances observed prior to PDU implementation, although expansion of the liaison psychiatry service (step decrease of 26.3%) and closure of assertive outreach and intensive rehabilitation teams (increase of 1.0%/week) in this period also significantly influenced short- and long-term ED activity. Also, there were immediate decreases in ED mental health attendances in SGUHFT following introduction of a street triage service (by 16.7%/week) and subsequent development/expansion of the PDU service (by 26.6%/week). But changes in crisis care pathway appeared to have less impact in SWBHFT and ULHFT.

Data cleaning and selection of control trusts

After all exclusions 38 (3 treated and 35 controls) MHTs and 133 (6 treated and 127 controls) acute NHS trusts were included. The data set for the 37 MHTs extracted from HES-APC to proxy admissions to a mental health inpatient ward included 549,343 admissions for patients aged between 18 and 75 years from December 2012 to January 2021. The data set for the 133 acute trusts extracted from HES-ED data set to proxy mental health attendances at ED contained 1,775,237 attendances for patients aged between 18 and 75 years from December 2012 to January 2021. Patient-level data were aggregated to trust level to create 24-month data series pre and post implementation at each trust.

For each of the treated and control trusts, we plotted the primary outcomes during the 24 months before and 24 months after the opening of the PDU. SGUHFT was excluded due to low data counts, indicating that not all data were submitted to HES. Selected months for the study periods for the treated trusts also had to be excluded from the analysis due to data recording and quality issues. See Table 4 for updated pre and post intervention study periods for each of the treated studies. Three control MHTs and 12 control acute trusts that had clearly irregular patterns of the outcome as a result of reporting errors, or non-submission of data to HES, were also excluded. The 20 and 10 closest peers for each of the 5 remaining treated acute trusts and the 3 remaining treated MHTs, respectively, were identified from the remaining pool of 115 control acute trusts and 32 control MHTs as described in the methods (see Appendix 5, Table 47).

Characteristics of patients

Control trusts were carefully selected based on their similarity to the treated trusts across a range of variables. These variables, which include data on capacity, workforce availability, patient characteristics, management structures, regional characteristics (including index of multiple deprivation and rurality), target similarity across the trust and catchment population as a whole, rather than across mental health services and patients presenting with mental health issues. This process thus aims to ensure that treated and control trusts are similar in their potential capacity to deal with patients presenting with mental health issues, rather than the actual capacity, but should account for a wide variety of unobserved variables that impact on the outcomes studied here. We further control for the specific characteristics of patients at risk of the outcomes studied here in the synthetic control method.

To assess whether the controls trusts were similar to the treated trusts in terms of key patient characteristics of patients with a mental health admission or ED psychiatric attendance, we compared the average distribution of key patient characteristics across the treated trusts to the average distribution across the pool of 20 selected control trusts, and across the excluded control trusts, in the 24 months prior to the opening of the PDU (see Appendix 5, Tables 48 and 49).

Outcomes analysed

The GSC method was unable to provide adequate estimates for some of the outcomes in particular trusts due to a lack of fit in the pre-intervention period, or evidence that the result was obtained by extrapolation. These issues can arise either as a result of variability in the outcome in the treated trust that is not matched by any of the control trusts, often occurring because of data sparsity in the treated trust (e.g. KHFT had < 1000 mental health admissions in both the pre- and post-intervention periods), or evidence that the result was obtained by extrapolation.

The following outcomes related to admissions to a MHT inpatient ward failed:

• Mental health admissions and average LOS on a mental health inpatient adult ward: LPFT (lack of pre-intervention fit).

The following outcomes related to ED psychiatric attendance failed:

• ED psychiatric attendances: KHFT (lack of pre-intervention fit)

• percentage ED psychiatric attendances less than 4 hours: UHBFT (lack of pre-intervention fit)

• percentage ED psychiatric attendances less than 12 hours: all trusts except UHBFT and STHFT (lack of pre-intervention fit)

• percentage ED attendances admitted: KHFT and ULHFT (evidence of extrapolation).

Estimated trends in hospital use

Tables 16 and 17 summarise the estimated impact of the opening of the PDU on each of the outcomes during the study period after the opening of the PDU at each treated trust and in a meta-analysis at the MHTs and acute trusts, respectively. Hospital use can vary over time even without changes to the way that care is delivered and it would be misleading to attribute normal variation in hospital use to the opening of the PDU. To check this, we report the p-value, which is the probability that that an effect of at least the magnitude of that observed could have arisen by chance. If this probability is low (here we assume that a low p-value is one that is < 5%), the estimate is considered to represent a significant difference between the hospital use in the treated trust compared with the control trusts.

As an alternative approach to presenting these data, Appendix 5, Figures 57 and 58 show the trends in each of the outcomes for the treated trusts (red lines) and the counterfactual trend estimated by the GSC method (blue line) using a pool of 10 control trusts at the MHTs and 20 at the acute trusts, respectively. The blue line is our estimate of the trend in the treated trust if the PDU had not been implemented. Note that the two lines are similar, by design, in the period prior to the opening of the PDU (i.e. to the left of the first grey dashed line), as we aimed to find a counterfactual that tracked the outcome in the treated trust over this period. The difference between the two lines in the study period after the opening of the PDU (i.e. to the right of the grey dashed line) provides an estimate of the impact of the PDU on that outcome.

Sensitivity analysis

In sensitivity analyses – including just the nearest 10, rather than the nearest 20, control trusts in the control pool for the acute trust outcomes and vice versa for MHT outcomes – all significant results were replicated either as a significant result with a similar effect size, or with a similar effect size but not significant. As results are robust to choice of controls, we are reasonably confident that the results we are seeing are not just by chance alone.

Sample characteristics

Pre and post first psychiatric decision unit visit findings by site (primary cohort)

Pre-pandemic comparisons by site

As described in Chapter 2 for the cohort study, the same set of pre- and post-first PDU visit comparisons were made for our additional, pre-COVID cohort (see Appendix 6, Table 51), and then service use for the pre-COVID cohort, in both pre- and post-PDU visit period (see Appendix 6, Table 52), compared with our primary cohort (see Appendix 6, Table 53). These are presented below for all sites except SHSCFT, where the PDU had yet to be opened in the pre-COVID period.

Equalities impact assessment

There were a small number of differences in sociodemographics observed between samples in our primary cohort and a recent sample of all people using mental health services in each site (see Table 19). In all sites, the PDU sample was significantly younger that the population of people using each trust’s services as a whole. There were no other differences between groups in SWLSTG and LPFT. In BSMHFT, first-time visitors to the PDU were also less likely to be female (171/387; 44.19%) compared with the trust-wide population (37,669/69,513; 54.19%; z = 2.62, p < 0.01) and less likely to be White British (257; 66.41% vs. 41,019; 59.01%; z = 2.41, p = 0.02). In STHFT, the PDU sample was more likely to be black than the trust population (11/204, 5.39% vs. 828/31, 190 2.65%; z = 2.06, p = 0.04), although numbers were low in the PDU sample.

Characteristics of the sample

Service users (first-time visitors to PDU) and PDU and referral pathway staff were recruited at all four study sites. Recruitment took place between 4 November 2019 and 27 February 2020 at the SWLSTG, LPFT and SHSCFT sites – roughly coincident with the recruitment period for the WP4 cohort study – and between 17 May 2021 and 1 September 2021 at the BSMHFT site. Follow-up interviews of service users (at all sites except BSMHFT) were conducted between 8 and 10 months post discharge, with the final follow-up interview taking place on 12 November 2020.

Thematic analysis

The final coding framework, as applied to the whole qualitative data set, can be found in Appendix 7, Table 54. Thematic analysis of both service user and staff interviews was produced and is presented below, organised by themes and subthemes. For service user interviews, themes follow a roughly chronological journey from ‘arrival’ to ‘discharge, signposting and onward referral’. Themes for staff interviews begin by considering ‘clarity’ around different staff perspectives on the function of PDUs and who the units might work best for, the referral pathway to the PDU, and then the PDU environment and experience of working on the units.

In quotation of data from verbatim transcripts of interviews, single speech marks are used for reported speech or thought and ellipsis for any truncation of data. Where names of people, places or services are anonymised, general terms are given in square brackets. Square brackets are also used for other insertions for clarity. No other annotation is used. Participant identification numbers following quotes indicate site number (as above), role (S = service user; P = PDU staff, R=pathway staff) and identification number and, for service user interviews only, indication of whether the data were collected in the baseline or follow-up interview (B = baseline; F = follow-up). To note, most interviewees, where relevant, used the term accident & emergency (A&E) to refer to the ED. We have not amended this usage in the data.

Service user interviews

Staff interviews

Overall approach and aims

The objective of the health economic analysis is to bring together findings from earlier chapters to identify potential impacts on the local health economy in each study area following the introduction of PDUs. The analysis makes use of the findings of all earlier chapters in this report. Using these different types of information, the chapter describes the economic impact and the RoI, from an NHS perspective, in each of the different study site areas. This compares the costs associated with PDU implementation with subsequent cost offsets as well as additional costs incurred. In addition, we also look at some impacts at an individual service user level; individual service scenarios provide illustrative estimates of changes in quality of life that would be needed for PDUs to be considered cost-effective from an NHS perspective.

All costs are reported in 2019–20 prices, with costs for acute hospital-based contacts and specialists community mental health teams taken from national reference costs. 83 Costs for some psychiatric inpatient stays, as well as hourly costs for some additional community and hospital-based staff costs, are taken from the annual Unit Costs of Health and Social Care. 84 These staff costs, estimate hourly costs, using mean full-time equivalent basic salary for AfC bands 4–9 of the April 2019/March 2020 NHS staff earnings estimates and include salary overheads and other oncosts. Further information on unit costs used in this chapter and their sources are reported in Appendix 8, Table 55.

Estimating the costs of psychiatric decision units

Our service mapping review (see Chapter 3), where the resources required to implement PDUs in the different areas have been collected,82 has been used to estimate the costs of providing PDU services in each site. An online survey completed by senior staff within PDUs included basic features on the structure of each unit, capacity, average and maximum LOS. The survey also collected information on staffing levels for both the day and night shifts in each PDU. This included type and number of staff members as well as their NHS AfC pay bands. We used these data to estimate the annual costs of running each PDU service. We assumed that there were two shifts, each of 12 hours in length and that the service operated 365 days per year.

The cost per PDU visit varies considerably from £741 in BSMHFT to £4800 in SHSCFT, with cost per PDU service user also ranging between £996 and £7442 (see Table 21). This is partly due to the considerable variation in staff ratios across the four MHTs, as well as differences in the level of throughput. BSMHFT, which has very short average stays of 12 hours, has an average of 1253 PDU visits for 932 service users per annum, compared with 438 visits by 283 service users in STHFT.

Area-level economic impact

The ITS reported in Chapter 4 and the synthetic control study in Chapter 5 have examined the impact of PDUs on area-level acute hospital ED attendances, as well as area-level psychiatric hospital admissions (both informal and informal) at a (clinical) population-level. In this section, using the results of these analyses we estimate economic impacts from an NHS perspective that may be associated with the implementation of PDUs at an area level. Chapters 4 and 5 set out in detail the methods used and results of the two approaches. As noted in Chapter 4, the ITS provides data on service use over a 4-year period (spanning 2 years prior to and 2 years post PDU implementation) in MHTs and acute hospital EDs in three of the four sites. The synthetic control study (described in Chapter 5) sought to match and compare mental health admissions and LOS, as well as ED attendances, for the mental health and acute trusts in our four study areas with trusts with similar characteristics in areas of England where PDUs have not been implemented.

While there are some similarities in the conclusions reached from each method there are also differences in findings. In this section, we look at potential impacts of using both of these approaches independently.

Return on investment

An objective of this chapter was to draw on findings of earlier chapters to identify potential impacts on the local health economy in each study area following the introduction of PDUs. One element of this is a RoI, from an NHS perspective, in each of the different study site areas. Essentially this compares the costs associated with PDU implementation with observed changes in costs of health services.

Table 23 summarises the RoI for each of the four sites. Here we have assumed that all the impacts on costs from the ITS and synthetic cohort studies are independent and cumulative. Even with this assumption, the limited impacts of PDU implementation on resource use reported by the analyses in Chapters 4 and 5, means that the level of direct annual RoI for most areas is low, being just 2.41% of PDU annual costs in SWLSTG, 8.89% in LPFT and 9.05% of PDU annual costs in STHFT. The exception is BSMHFT, which because of the highly significant reduction in average length of inpatient stay of 15.5 days, has an annual return of 1868% compared with the costs of PDU investment. Our survey in Chapter 3 indicates that the four PDU services were operating between 63% and 83% of full capacity; if services were operating at full capacity and assuming a proportionate change in cost impacts would have a negligible impact on the RoI in three areas but would increase to a maximum of 2990% in BSMHFT (see Appendix 8, Table 56 for full results).

Our original intention was to use the results of the RoI analysis to look at the potential costs and benefits of scaling up access to services across England, but this is not appropriate given the uncertainty around the levels of return, which are low with the clear exception of BSMHFT. While the synthetic control method indicates that the difference in mental health inpatient LOS following the introduction of the PDU in BSMHFTis highly significant, this method does not allow us to isolate the effects of the PDU from other factors that could contribute to difference in LOS. The introduction of the PDU would not be expected to have such a major effort on LOS, given its focus on reducing the number of short (< 5 days) rather than long hospital stays; the relative contribution to average LOS of other innovative services introduced in the same time frame in BSMHFT, including the more intensive use of liaison psychiatry, using the rapid assessment, interface and discharge service needs to be explored. The limited change on resource use seen in both the ITS and synthetic cohort studies in the other areas should also not be interpreted as indicating that there is no economic case for investing in PDUs, but it does indicate that they may be unlikely to offset a substantial proportion of their cost to the NHS.

That said, there are additional resources saved on which we have not placed an economic value; these include any reductions in waiting time in ED. The synthetic cohort study in Chapter 5 pointed to significantly lower waiting times in the ED at KHFT in comparison with control sites. While we have valued contacts with EDs using tariffs that do not vary by time spent waiting, reducing waiting times potentially frees up time of ED staff to see and treat other patients. Furthermore, evidence from previous analysis suggests that individuals detained under section 136 of the Mental Health Act may wait in EDs on average up to 12 hours compared with a general population average of 2.5 hours, owing to the complexity of their needs. This, it has been suggested, is the equivalent of being able to treat 10 other patients in the same time period. 100 Our study also does not look at whether PDUs services have displaced existing local mental health services (e.g. have replaced triage wards or other community-based assessment services) or are instead an addition to existing services. There may also be impacts on the use of specialist CMHTs that we do not know about. The costs of providing PDUs will be lower and the RoI greater if they have substituted for some existing services.

It should also be stressed that the time period for this analysis is still relatively short; if individuals benefit from contact with PDUs then their long-term mental health service use costs may be considerably lower, but our analysis at most only covers a 2-year post-PDU implementation period. Untreated poor mental health can have consequences that last for many years or even decades, so a longer time frame may also be needed to fully capture these consequences. Our return on analysis has been conducted from an NHS perspective, but there may be other impacts on other agencies, such as the police and local authorities arising from any improvement in mental health outcomes for service users.

Critically, the most fundamental impacts of all are the short-, mid- and long-term outcomes on PDU service users. Longitudinal analyses are still needed at an individual service user level that compare the long-term costs and outcomes of PDU service users directly with those of individuals that have not used these services. If PDUs help improve levels of recovery from poor mental health as well as social inclusion, this will also have economic benefits for these individuals such as economic independence and wider society (higher levels of productivity from participation in education, paid employment, family caring and volunteering). These could not be included in our RoI analysis. PDUs will also have impacts on quality of life. Another fundamental question is whether contact with PDUs leads to incremental gains in quality of life compared with care as usual.

The remaining sections of this chapter look at some of these missing potential individual-level impacts of PDUs, by drawing on analysis of service use and pathways of care observed in cohort study in Chapter 6, as well as the qualitative analysis in Chapter 7. We also examine the quality of life of individuals attending PDUs and therefore there is scope for substantial improvements in quality of life following PDU contact.

Changes in cost of mental health services following psychiatric decision unit assessment

In addition to ED attendance and psychiatric inpatient admissions, contact with PDUs might be expected to have an impact on the use of community mental health services. Longitudinal cohort data collected as part of decision have been used to explore these impacts. Chapter 6 describes how in each of the four study areas data have been collected on patterns of service contact and utilisation in the 9 months prior to and following an initial visit to a PDU. We have then examined differences in resource use and costs to the NHS in the 9 months after a PDU visit for each of the four sites. Unit costs are drawn from NHS national reference costs and the Personal Social Services Research Unit’s Unit Costs of Health and Social Care used in estimating these costs. Specific sources are reported in Appendix 8, Table 55.

Table 24 shows summary results of pairwise comparisons on per-PDU and post-PDU costs, using 2000 bootstraps, bias corrected and accelerated to account for the non-normal distribution of costs (full results for each site are given in Appendix 8, Tables 57–60). Overall increased contacts with services and inpatient hospitalisations meant that post-PDU costs are significantly greater than pre-PDU costs in all four areas (p = 0.000). Mean total costs post PDU were £6945 greater in BSMHFT £8446 in LPFT, £5803 in SWLSTG and £7908 in SHSCFT. Similarly, the mean costs of inpatient stays were significantly greater in all four sites: £4696 greater in BSMHFT £5059 in LPFT, £3671 in SWLSTG and £5179 in SHSCFT. The cost of liaison psychiatry costs was significantly lower in the post-PDU in LPFT (p = 0.001) and SWLSTG (p = 0.000) although the absolute level of costs was low. Use of both CMHTs, as well as crisis/home treatment teams, were also significantly greater in all four sites. Potentially PDU visits may have led to different patterns of onward referral and signposting to community-based services.

Insights into the quality of life of psychiatric decision unit service users

The primary function of mental health services is to improve health outcomes and not just impact on resource use and costs. Although the study did not intend to directly look at changes in clinical outcomes, such as levels of mental distress, it has been possible in the study to collect some self-report data on quality-of-life scores, using the EuroQOL EQ-5D-5L quality-of-life instrument. 101 The instrument has five domains: mobility, self-care, ability to perform usual activities, pain and depression/anxiety levels, with a score of 1 indicating perfect quality life and a score of 0 being equivalent to death. The five response levels for each domain are: no problems, slight problems, moderate problems, severe problems and extreme problems. The instrument is widely used to generate estimates of quality adjusted life-years used to help assess the cost-effectiveness of health system interventions.

In our analysis quality-of-life data can inform scenario modelling to help indicate the potential plausibility for improving quality of life of PDU service users. This can be useful given that quality of life is the primary outcome measure used in economic analysis to inform decision-making in England, where judgements are typically concerned with assessing whether the additional costs incurred by a service are justified by the quality-of-life gains made.

PDU service users in the cohort study in two of the study areas, LPFT and STHFT were invited to complete the EQ-5D-5L at baseline and at 9-month follow-up. Individuals were able to post their completed instruments back to the PDU and these questionnaires could then be linked to individual service user identifiers. Complete responses were received from 148 participants, 107 from LPFT and 41 from STHFT.

Currently there is no accepted value set in England for the five-level version of EQ-5D, so in line with current recommendations from the National Institute for Health and Care Excellence,102 we have used the crosswalk mapping function developed to map five-level values to the existing English three-level value set103 to generate utility scores. Figure 15 shows the distribution of EQ-5D-5L scores at baseline. The average utility score of just 0.14 is extremely low, being for example, much lower than average values reported for young people in England who have self-harmed of 0.68. 104 41% of these scores were less than zero, indicating that these individuals had utility scores that could be viewed as worse than death; these very low values in a high proportion of individuals might be expected, given that many individuals attending PDUs may have been experiencing suicidal thoughts.

FIGURE 15.

Distribution of EQ-5D-5L scores in reporting participants at baseline in LPFT and SHSCFT.

While this sample is small, the data suggest that there is potential for substantial changes in utility values with appropriate treatment; if PDUs do lead to such improvements then the economic case for investment will be strengthened. Although only 28 follow-up responses at 9 months on utility scores were returned to the study team, average utility scores in these 28 individuals were much greater at 0.41, compared with 0.10 for the 28 at baseline. While too small to have clinical meaning, nonetheless pairwise analysis indicated that this 0.30 improvement in utility scores was significant (p = 0.002).

Individual service user scenarios

Decision has shown that PDUs support individuals with a wide range of differing personal and social circumstances, as well as differing mental health problems. Our analysis of service use in the 9 months after initial visit to the PDU for the longitudinal cohorts in all four sites indicates this is associated with significantly greater levels of both community and inpatient mental health service utilisation than received in the 9 months pre-PDU visit.

A key question is whether service pathways followed after PDU visits are more appropriate to service user needs and lead to better outcomes than would otherwise be the case. The time that PDU staff have to influence service pathways is very limited, usually no more than 48 hours per stay, at which point a decision must be made as to whether a service user is simply discharged home, referred or signposted to a local crisis team or other community mental health service, or transferred to an acute psychiatric ward.

Ideally, this question is best addressed through longitudinal analyses comparing the experiences of individual service users who come into contact with mental health services as a result of crises and are referred to PDUs, with those service users who do not have this option. While we do not have such data, we can illustratively look at the level of impact that a PDU would potentially have to have to be considered cost-effective. This can be done by creating simple scenarios looking at potential experiences and outcomes for hypothetical individual service users.

We can construct some scenarios drawing on information on the cost of PDU service provision, together with observed patterns of short-term service use drawn from the cohort analysis, as well as quality-of-life utility scores we have collated from 148 PDU service users. In addition, we also refer to qualitative research in Chapter 7 on whether PDUs are perceived as helpful in recovery.

We have illustrated the potential economic case for 16 hypothetical PDU service users, 4 in each of our PDU sites. Sex in the cohort analysis was associated with increased service use costs in the 9 months post PDU use, so our scenarios include two men and two women from each site. Our cohort analyses also indicate that costs are highly skewed, with a small number of individuals having substantial inpatient mental health stays in each area, while the majority of individuals only use community services post PDU visit. To reflect this cost variation simply, four of the scenarios look at median incremental costs post PDU visit compared with pre-PDU service use, as there was no use of inpatient services (‘low cost’), while the other four scenarios apply mean costs identified in each of the cohort sites which do account for inpatient costs (‘high cost’).

We do not know what impacts PDUs have on quality of life, but we can estimate the minimum level of quality adjusted life-year (QALY) gain that would be needed for the PDU intervention to have a cost per QALY below the accepted cost-effectiveness threshold of £30,000 for each of the 16 scenarios. Some service users in qualitative interviews also pointed to the role that PDUs can play in signposting towards support to help stay or return to employment, for instance signposting to recovery colleges and employment service-related information. One individual also expressed an interest in getting a job in future in a PDU. Equally, some service users did not have good experiences in post-PDU support to obtain employment and also mentioned the practical difficulties sometimes faced when wanting to return to an existing job, particularly after a long period of absences. The desire to maintain or return to employment was an issue raised in interviews so we also considered how varying the proportion of individuals who are in employment at 9 months post PDU might also impact on cost per QALY gained from a societal perspective.

As Table 30 indicates for the low-cost scenarios where service users do not spend time in inpatient care, then the QALY gains over 9 months required to produce a cost per QALY below £30,000, range from 0.06 for a woman or man in Birmingham to 0.19 for a woman in Sheffield. If on average over 9 months an additional 30 days were spent in work, valued conservatively at the minimum wage rate in 2020 for the over 25s, then the cost per QALY would fall further, in the case of Birmingham almost being cost neutral.

For the high-cost scenarios in Table 31 when service users do spend time in inpatient care, then the QALY gains required to produce a cost per QALY below £30,000 range from 0.25 for a man in SWL to 0.42 for a man in Sheffield. If on average individuals spend an extra 15 days in work over 9 months then the cost per QALY gained could range between £25,000 and £27,000.

These estimates are purely illustrative, but they do indicate, particularly for service users who do not need to make use of inpatient care, that only very modest improvement in quality of life might be required for the PDU intervention to be considered cost-effective. For high service uses the picture is more complex, with a much higher level of improvement in quality of life needed for the PDU to be considered cost-effective. Our earlier collection of data on quality of life, did however indicate that mean changes in quality-of-life utility values of 0.3 were seen in the very small number of individuals who provided these utility values at 9 months post PDU. These scenarios are also potentially conservative as they assume that there would be no increase in use of services for individuals who do not make use of PDUs; there will also be impacts on the use of acute hospital services that were not included in service use data in the cohort analyses. The scenarios only cover a 9-month period, if any quality of life and productivity benefits are sustained beyond this time period then the case for investment would be strengthened.

Data synthesis

We saw evidence of some effect of the introduction of PDUs on our primary outcomes of informal acute psychiatric admissions and mental health presentations at ED but, given the complexity of the study, have an opportunity to use the full range of data we collected to better understand those findings. Synthesis of data from across WPs, developed through our synthesis workshop and subsequent discussion in the research team (as described in Chapter 2) produced the following set of ‘synthesising arguments’ that are indicative of our overall findings from the research (the analyses that support each argument are presented in Table 32). Note that these propositions are data-driven and as such do not always correspond directly to our initial research questions:

1. PDUs reduce overall levels of informal acute psychiatric admissions, but there is no clear trend in informal admissions after units open.

2. First time PDU visitors (about 40% of whom are relatively new to mental health services) experience increases in admissions of all types and community mental health service use in the months following their first PDU visit as they are connected with care and support (some service users report being able to access the care they need for the first time after their first PDU visit).

3. Following PDUs opening there is an increase in formal admissions (but with no clear trend in formal admissions after units opening).

4. There is little overall change in inpatient psychiatric service use as a whole after PDUs open, with no clear evidence of an effect on numbers of overall admissions, no evidence of change in bed occupancy or proportion of 0- to 5-day admissions (there is some evidence of reduction in average LOS), suggesting that PDUs might have a general impact on the balance of admissions away from informal and towards formal admissions.

5. There is little evidence of additional benefits of PDUs for people at high risk of psychiatric admission (i.e. people with previous admissions/diagnoses of psychosis), but there is evidence to suggest that PDUs provide a safe, calming environment) for people with a high level of acute need who might be too at risk for immediate discharge from ED but who are unlikely to benefit from an admission (PDU population is most likely to comprise this group, including people with diagnoses of mood or personality disorders).

6. PDUs can provide improved quality of care for this group of people (e.g. people with complex emotional needs) who might benefit from a PDU where there is a longer LOS with substantial opportunity for in-depth therapeutic input (including psychosocial interventions and higher staff-to-patient ratio) and onward referral/signposting to a wide range of community support.

7. PDUs have no overall effect on level or trend in mental health presentations at ED, but do have the potential to reduce presentations at ED where they are specifically set up to do so (i.e. higher capacity, shorter maximum LOS).

8. PDUs have no overall effect on liaison psychiatry episodes in ED, mental health presentations of people at risk of attending (people with previous attendances) or on 4-hour breaches or average length of wait except, potentially, where the PDU is specifically set up to impact mental health presentations in ED.

9. Conversely, while larger units with a shorter maximum LOS might be more likely to reduce pressure on ED, PDU staff can feel pressurised to admit inappropriate referrals and discharge service users while still feeling suicidal; referral and signposting to appropriate aftercare might also suffer where LOS on the PDU is shorter (i.e. configuring PDUs in this way, primarily to reduce pressure on ED, is unlikely to improve quality of care).

10. The effectiveness of PDUs in impacting either informal psychiatric admissions or mental health presentations at ED is associated with the aims of units and the way in which units are configured; in practice priorities for PDUs reflect trends in outcomes prior to PDU opening and the existence of other initiatives to address either psychiatric admissions or mental health presentations at ED.

11. A lack of clarity of function, differences in expectation and poor communication between PDU team and referral pathway staff, and inappropriate referrals mitigate the potential effectiveness of units; this reflects systematic review of the literature on similar units internationally which indicates that units with a clear purpose and configuration to impact ED wait times are effective in doing so, and units that primarily aim and are configured to reduce psychiatric admissions are effective in doing so (service users’ experiences of the crisis care pathway and of PDUs negatively impacted by a lack of clarity in unit function).

12. The impact of PDUs on cost to trusts of psychiatric admissions and ED attendances are marginal at most sites, resulting in only small savings; potential savings result from shorter length of psychiatric inpatient stay, and some site level savings relate to ED presentations and arrivals by ambulance compared to control but these do not offset cost of running PDUs (except at the BSMHFT site, where a substantial reduction in psychiatric inpatient LOS was observed in one analysis).

13. PDUs with a short LOS and low staff to patient ratio have considerably lower cost per visit (and can impact mental health ED presentations) than PDUs with longer LOS and higher staff to patient ratio (which can impact informal psychiatric admission and improve quality of care for people in crisis).

14. Significant additional costs of both inpatient and community mental health services are incurred by first time visitors to PDU as they are connected to care post first visit.

15. PDUs generally do not offer a simple financial return on investment in the short term, but the potential for reducing longer term costs of care as people access more appropriate community care (and less emergency care) and experience improved quality of life as a result indicate that PDUs may be cost-effective in at least some scenarios.

16. Changes in wider configuration of the crisis care pathway (e.g. introduction of street triage services) can support and reinforce the effects of introducing PDUs on both psychiatric admissions and ED mental health presentations (this was especially the case at BSMHFT, where a substantial reduction in length of psychiatric inpatient stay might be attributable to the introduction of an enhanced psychiatric liaison service).

17. Impact of changes in crisis care pathway on mental health presentations at ED are less clear (introduction of street triage and expansion of PDU provision to include psychosocial intervention reduced ED presentations in SWLSTG).

18. No overall evidence that PDUs might reduce inpatient admissions for sociodemographic groups over-represented in acute mental health care (although in one site, BSMHFT, indicated that, potentially, younger, male, non-White British service users might benefit from the introduction of PDUs); noting the exploratory nature of the analyses, costs of post-PDU care were generally higher for women and older people (i.e. people who already have better access) although cost of care was higher for non-White British service users in BSMHFT (people with insecure housing appeared to access less care post-PDU visit).

19. While working on the PDU can be rewarding and supportive for staff, the high-pressure nature of the role is also demanding and might mitigate the extent to which PDUs can provide higher quality of care.

20. Sustained benefits of PDUs (including where PDU visits are experienced positively) are likely to be mitigated by the capacity of crisis and community mental health services as a whole; where there are not sufficient appropriate options to refer or signpost for people (within mental health services and the wider community sector) or where the experience of discharge is hurried, then the potential longer-term effects of PDUs on ED presentations and psychiatric admissions might be diluted.

21. The pandemic did not have a major impact on the overall level of service use for people post first visit to the PDU, although there were often fewer face to face and more remote community contacts (compared with a similar pre-pandemic cohort), some reduction in CRHT team contacts and fewer inpatient admissions in rural areas (offset by increase in remote contacts).

22. The pandemic increased pressure on PDUs, even where there was an increase in capacity in the unit or other elements of the crisis pathway (e.g. the introduction of psychiatric ED provision), exacerbating many of the issues impacting quality of care described above.

Results in context

It is helpful to consider the implications of our findings in terms of the multilevel approach to addressing organisational research questions we proposed in Chapter 1,47 discussing the wider relevance of our findings at micro (individual), meso (service and organisational) and macro (policy) levels.

Strengths and limitations

This was a quasi-experimental study, and as such individual workstreams are reporting using the Transparent Reporting of Evaluations with Nonrandomised Designs for non-randomised/quasi-experimental studies. 109 We identified a range of limitations in the research, within and across WPs. Individual WPs analysed data from different time periods and different samples, and so not all of the comparisons we make in our data synthesis are direct triangulation between data sets. For example, because PDUs opened on different dates, WPs 2 and 3 are not synchronous across sites (interrupted time series and synthetic control studies cover the same time periods within each site). The WP4 cohort study and WP5 interview study were synchronous across sites, but as a result did not take place at the same time as ITS and synthetic control studies. In addition, because of availability of data, while the ITS analysed ED data from SGUHFT in SWLSTG and from SWBHFT, the synthetic control study used ED data from KHFT and UHBFT, respectively.

In our ITS study, we spent considerable time working with business intelligence staff within trusts to collect relatively precise data on both psychiatric admissions and mental health presentations at ED. However, data based on diagnostic coding, such as ED mental health attendances, may be more vulnerable to error, and mental health coding has been shown to be problematic in data sets derived from routine hospital data. 110 In our synthetic control study, classification of ED data was determined by diagnostic coding or a ‘mental health reason for visit’ entry applied by staff in the ED. Admissions to mental health acute inpatient wards are also not explicitly coded as such in HES-APC and we proxied these admissions using the main specialty of the consultant or the treatment function of the episode (and might have included admissions to eating disorders or other specialist wards, or acute admissions to community-based crisis and home treatment teams, for example). Incidence of both our ED and MHT outcomes was substantially higher in the synthetic control study than the ITS as a result, potentially diluting or confounding effects and going some way to explaining differences in effects observed in the two studies.

However, within-site comparisons within each WP are all methodologically appropriate and clearly defined. While it is possible that effects observed in WPs 2 and 3 are at least in part explained by differences between comparators, conditions outside of trusts or by other changes in the delivery of care (indeed we explore this to some extent in secondary analyses in the ITS; see Chapter 4), creation of both the counterfactual in the synthetic control study and the extended time series in WP2 are methodologically robust and were risk-adjusted and included sensitivity analyses where possible.

There were some issues with data within WPs. In WP2, MHT data from Sheffield included a number of missing variables and there were missing data from large numbers of admissions during the times series period. As a result, psychiatric admission data from Sheffield were omitted from the interrupted times series analysis. In the WP3 synthetic control study, reporting requirements of MHT data to NHS Digital did not distinguish psychiatric admissions as formal or informal, capture liaison psychiatry episodes in ED or allow us to specifically identify outcomes for a population with previous admissions; hence we have a slightly smaller set of outcomes for the synthetic control study (compared with the ITS), without informal psychiatric admissions as a comparable primary outcome. Further, there were likely changes in mental health coding practice over the 4-year study period in participating sites, complicating interpretation of trends over time, and for a small number of variables in some trusts, data sets did not cover the entire 4-year period of interest; this decreased the power of corresponding WP2 ITS analyses and their impact in pooled analyses of outcomes.

The late addition of BSMSTG as a site to WP5 meant that we had no follow-up interview data for service users in BSMSTG and could not explore the post-discharge pathway there. We did collect sufficient baseline data from service users, as well as a range of PDU and pathway staff data, to sufficiently explore the model of care in BSMSTG. We were unsuccessful in recruiting a diverse service user sample to WP5; as a result, we did not systematically capture the experiences for black, Asian and minority ethnic visitors to the PDU and were unable to further elucidate findings from our equalities impact analysis in WP5. In WP6, because we relied on patient record data and did not directly recruit and follow-up a cohort of PDU visitors, we did not have sufficient structured quality-of-life data to formally assess cost benefit in terms of quality of life-years (although data we collected from WP5 participants did enable us to include quality of life in our modelling work). Our whole study design was predicated on a synthesis of routinely collected service use data and experiential insight from qualitative interview. This offers an extremely cost-efficient approach to research but as a result we missed out on participant reported outcomes (including quality of life) that are collected through (costly) one-to-one interview.

Nonetheless, our mixed-methods approach46 was a strength of the study as a whole. Given that an experimental study design was not practical, employing two robust quasi-experimental methods offered both within-site and externally controlled estimates of effect enhancing confidence in our findings. The critical synthesis method made systematic use of data from across WPs to generate and explore potential explanatory arguments that might usefully inform research, policy and practice, rather than prioritise and discount analyses solely on the basis of reliability. The inclusion of a cohort study enabled us to explore the impact of PDUs on service use at an individual level while qualitative research, with staff and service users, was indicative of the complex relationship between experience (quality) of care and clarity of purpose of PDUs, and the potential effectiveness of different configuration units. As a result, we were able to give carefully consider if and how different configurations of PDU, in the context of the crisis care pathway as a whole, impact outcomes in different ways, providing us with opportunity to draw nuanced conclusions on PDU configuration in relation to policy and efforts to manage demand on services across acute and mental health NHS trusts that would not have been possible without our mixed method and critical synthesis approach.

Impact of patient and public involvement on the research

There were a number of important impacts of patient and public involvement and of our coproduced approach to research on the study and findings:

• First, and importantly, in a study where most WPs were quantitative in nature, reducing research questions to measures of service activity, the qualitative WP – where the was most patient and public involvement input – was essential to the interpretation of data.

• Researchers working from lived experience on the team and through the LEAP ensured that the experiential perspective on PDUs remained front and foremost throughout the study, and particular as we began to analyse and make sense of all of our data.

• This insight, together with that of the PEER group who helped develop the study and the lived experience researchers on the team, ensured that our qualitative interview questions were very specifically focused on key aspects of the experience of mental health crisis and of staying on a PDU.

• The lived experience researchers, with input from the LEAP at our interpretive and synthesis workshops,48 also emphasised key aspects of the data that play a crucial role in shaping our data synthesis and overall conclusions. This work underscored:

  1. the high levels of suicidality among visitors to the units

  2. the importance of both unit environment and the depth of talking and listening required of unit staff to impact such levels of distress

  3. the crucial importance of having sufficient time on the unit to begin to address issues for people

  4. the sense that the impact of the unit was wholly mediated by the capacity of the crisis pathway to continue to support people post-discharge.

These understandings provided a lens through which to make better sense of the balance of impact, cost and benefit of introducing PDUs.

After discussion the lived experience team decided on overt disclosure of their lived experience to people they were interviewing in WP5, both in participant information sheets and in introduction to the qualitative interview. At the end of the interview two questions were asked about what it had been like talking with a researcher with lived experience and whether the interviewee thought this had made any difference. An in-depth analysis of these responses will be the subject of future publications, but a preliminary analysis suggests a marked impact in terms of ease of talking, ability to share difficult experiences, and feelings of being listened to, heard and believed from service user participants.

The co-investigator responsible for supporting the patient and public involvement in the project facilitated regular reflective sessions for the researchers with lived experience. This gave the team an opportunity to openly reflect and gain support in a safe environment about the experience of using their lived experience in the project and of the impacts it had had on them. This was particularly important as many of the interviews including accounts of distressing experiences and feelings of suicidality. It was also an opportunity to continuously reflect and monitor on how patient and public involvement was being used in the study and what was working well and what could be done differently.

Recommendations for future research

Recommendations for future research are both methodological and substantive in nature. While our study benefited from employing a number of different approaches to estimating the impact of introducing a new service model on mental health and ED service use, there is need for research to better understand how the way in which data are coded in national data sets impact the specificity and precision of estimates, especially with MHT data, and how to best interpret differences in estimates obtained from proxies for specific populations derived national data sets and directly from electronic patient record. This understanding would help in the efficient design of studies of service-level/pathway change in the future where a quasi-experimental approach is most appropriate.

In our study we relied on a combination of service use and qualitative, experiential data. We did not assess individual-level clinical or psychosocial outcomes, in part because recruiting a large cohort to do so adds considerable cost to the research and a clear rationale is needed for doing so. However, as a result we struggled to obtain sufficient quality-of-life data and potentially missed out on outcomes that would have shed more light on quality of care (which emerged as important in our qualitative analysis) and provide more conclusive evidence on the potential cost-effectiveness of PDUs. Future research might usefully build on our study to target a group or groups of people who might reasonably be expected to benefit from PDUs and evaluate the impact on a wider range of mental health and psychosocial outcomes, either using an experimental or quasi-experimental design. Alternatively, more attention might be paid to the use of routinely collected patient reported outcome and experiences measures, such as, for example, DIALOG+,111 to enrich our understanding of the impact of service-level change. Further research might also usefully refine, methodologically, a longitudinal approach to qualitative enquiry about experiences of care pathways.

While we were able, in our equalities impact work, to demonstrate some potential for PDUs to impact access to care for underrepresented groups, we did not manage to systematically integrate this work across workstream. Future research might develop, a priori, analysis plans that better explore associations between demographics and outcomes as we did in our cohort study and health economic analyses, while sampling for qualitative research needs a more proactive approach in order to obtain meaningfully diverse samples.

Our study revealed the challenges around evaluating the impact of the introduction of a new service model in the context of an evolving care pathway where other service-level initiatives are also likely bring about change. While we demonstrated some success both in isolating effects and identifying complementary effects, there is a need for a systems-level approach to investigating mental health crisis care building on similar studies elsewhere,50,112,113 including the role that the third and voluntary sector might best play in providing an effective network of crisis support. 32 As designed, our study also lacked an economic perspective on the commissioning and decommissioning of other crisis care services alongside our PDUs. Such an approach might usefully be applied to similar policy driven initiatives that seek to change the wider landscape of health and social care.