Patient involvement in improving the evidence base on mental health inpatient care: the PERCEIVE programme

Phase 1: mapping out the dimensions of inpatient care

In this initial phase, three stakeholder/reference groups were held to map out the dimensions of inpatient care as a topic guide. In developing VOICE, the stakeholders were service users with experience of research and inpatient care who were not currently using local services. National voluntary organisations were also represented. In developing VOTE, the stakeholders consisted of senior nursing staff. Each group built a topic guide that was flexible in order to encourage as creative a list as possible. Members of these two groups also then constituted the reference groups for the user-led and staff parts of the study.

Phase 2: measure generation

For both VOICE and VOTE, purposive sampling was used to reflect local inpatient demographics with regard to gender, age and ethnicity when recruiting participants who were either staff or service users of an inner London UK mental health trust.

Samples

Service user participants for the feasibility study were recruited from acute wards and psychiatric intensive care units (PICUs), and test–retest participants were engaged on acute wards. For the larger psychometric testing phase, participants were recruited from acute wards. Service users were eligible if they could provide informed consent and they had been present on the ward for at least 7 days during the 4-week data collection phase. All nursing staff from all grades (health-care assistants, entry-level qualified staff, clinical charge nurses and team leaders) were asked to participate in the psychometric testing of VOTE. All potential participants gave written informed consent following an explanation of the study.

Phase 3: feasibility and acceptability

The final measures were intended as self-report tools, so studies of feasibility and acceptability were conducted to evaluate the burden of administering and completing the measure. In the feasibility study, participants completed the measure including two additional questions assessing whether or not the measure was easy to complete and understand. Acceptability was assessed using two additional questions collected from participants. Two tests which indicate readability, the Flesch reading ease test (the recommended score is between 60% and 70%), were assessed.

Phase 4: psychometric assessment

Phase 1: mapping out the dimensions of inpatient care

Phase 2: measure generation

Psychometric assessment

Phase 4: psychometric assessment

Can VOICE and VOTE distinguish differences in views?

Detained participants were shown to hold more negative perceptions of inpatient services. This is similar to the satisfaction studies showing that service users who are admitted involuntarily are less satisfied with their care. 93 More recently, research has identified that lower levels of satisfaction are linked with the accumulation of coercive events and perceived coercion. 94,95 Our findings from the focus groups present a more complex picture and one that is worth further analysis.

We anticipated, but did not find, any differences on either VOICE or SSS-RES scores for ethnicity. Previous quantitative studies have shown differences for legal status but not ethnicity,96,97 whereas qualitative research has revealed that BME users hold relatively poor perceptions of acute care. 16,98 Our study was set in areas of London with high proportions of people from BME groups. 99,100 Staff demographics tended to mirror those of inpatients and it may be that services were better tailored towards BME groups. In addition, asking users for their perceptions while they were in hospital may have inhibited openness and honesty, particularly on sensitive issues.

In this study, staff had slightly more positive perceptions of job satisfaction than staff in other studies. 101–103 Staff with negative perceptions of the daily pressures of working on an acute ward and barriers to change also had poor job satisfaction.

In previous studies, work stressors have differed between groups including occupational status and gender. 43,48 Of all the demographic characteristics included in this study, only country of birth showed a significant effect on staff perceptions of the daily pressures of acute ward working, with staff born outside the UK showing more positive perceptions than those born within the UK. This may be a direct result of the inner London location of the trust under study, as a large percentage of the client group and the nursing workforce are not UK born.

However, the mean VOTE scores were most negative overall in the direct care staff groups (staff nurses/nursing assistants) compared with more positive scores in those who occupied more managerial roles (clinical charge nurses and team leaders). Although we did not address whether or not the stressors differed significantly between these groups, it is true that both nursing assistants and staff nurses spend the most time in direct client contact. Therefore, their more negative perceptions might be one of the inevitable aspects of delivering therapeutic acute care for patients with severe mental illness at their most distressed. Furthermore, staff in leadership roles are more likely to be involved in the planning stages of new changes and, therefore, have an increased sense of control and responsibility over them, while those in direct care roles are more likely to be involved in delivering changes. These important issues, revealed through stakeholder involvement, require further exploration to discover key drivers for these perceptions that might then be subjected to management and other interventions.

Are VOICE and VOTE different from other measures?

The benefits of the participatory model used to develop VOICE and VOTE are visible in both the breadth of the construct investigated and the rich content of the items. Although the VOICE total scores were correlated, there were also distinct differences in content between VOICE and the comparison satisfaction measure. We believe that this is because of the use of a participatory methodology. The factor analysis of VOICE discovered two main factors: security and care. In particular, the safety and security issues were given more weight in VOICE and items on diversity were included that did not appear in the conventionally generated measure. Although items regarding the physical environment and office procedures featured in the SSS-RES,78 these issues were not deemed as important by the service users in our study and, therefore, were not included in VOICE. However, other items referred to ‘care received’ overlapped with some items in SSS-RES.

It is often assumed that the only construct to measure is satisfaction with acute care. However, there are difficulties in encapsulating complex sets of beliefs, expectations and evaluations in satisfaction measures. Caution should be taken when making inferences from the results of such measures, as they may not accurately reflect the views of service users. 104 VOICE is unique in that it captures service users’ perceptions and we anticipate that this will depict the inpatient experience more accurately.

The factor analysis conducted during the development of VOTE revealed three novel factors. Factor 1, ‘workload intensity’, seems consistent with previous work identifying the effects of poor resources such as inadequate staffing and too few beds on occupational stress for nurses. 9 Factor 2, ‘teamwork’, is similar in content to the concept of ‘job satisfaction’ characterised by leadership, educational attainment, pay and stress. Factor 3 represents a new subscale that explains the lower correlations to the IWS and Maslach Burnout Inventory (MBI) subscales. It crystallises two ideas. First, it reflects some issues relating to burnout. 46 Second, it draws together some of the feelings of fear and uncertainty that are associated with working with distressed clients in an unstable environment. This complex issue was revealed through stakeholder involvement and adds a new and unique aspect to the current library of measures that capture staff perceptions of stressors in acute inpatient settings. This issue clearly requires further exploration in the field, as one of the fundamental aspects of delivering therapeutic acute care is in the ability to work with clients with severe mental illness at their most distressed.

Further developments

Three further measures grew out of this WP; please refer to Appendices 1–3.

Strengths and limitations

This study was conducted in London boroughs with high levels of deprivation, ethnic minority populations and psychiatric morbidity99,100 and, therefore, may not be directly generalisable to other settings. In addition, our focus groups and expert panels included a high proportion of participants from BME communities. Although this is a strength, it may be that different items would have been produced by other groups. Furthermore, as recruitment for VOTE was limited to staff from acute wards it is not clear whether or not VOTE would be suitable for other settings such as forensic or rehabilitation wards.

It is impossible to accurately assess inpatient care without involving the people directly affected by that service. Developing outcome measures that are valued by service users and staff is essential in evaluating and developing inpatient services. The main strength of this piece of research is that it fully exploits a participatory methodology: staff and service users were involved in a collaborative way throughout the whole research process. VOICE and VOTE are the only robust measures of acute inpatient services designed in such a way. This method has resulted in measures that encompass the issues that staff and service users consider most important and that are acceptable, easy to understand and complete by a diverse staff group and service users with a range of diagnoses and severity of illness.

Clinical implications

VOICE and VOTE are measures of the experience of living and working on acute wards. This is consistent with recent National Institute for Health and Care Excellence (NICE) guidelines 2014. 104,105 The factor structures demonstrated what was key in this experience for each group. Service users were concerned with the care provided and staff/patient interaction as well as security on the wards. Nurses were concerned with workload intensity, team dynamics and interaction anxiety. Although interaction anxiety could lead to an avoidance of service users and, therefore, compromised care, there was evidence from the post-intervention focus groups for WP3 that service users understood how pressured staff were and made allowances for it. These findings suggest that leaders in the NHS need to consider putting in place support structures for staff if care is to be optimised for service users. The 2013 Care Quality Commission (CQC) report106 paints a mostly bleak picture of inpatient care which corresponds to our findings and they call for similar measures. The RCN ran a consultation on control, restraint and rapid tranquilisation practices in 2013, which greatly exercised our focus group participants.

Research implications

It would be worth investigating more thoroughly the variation in VOICE and VOTE and what such variation might predict. Furthermore, a more detailed investigation could be undertaken for the WP4 data. We need to find out whether or not, within the triage system, the admission ward had different VOICE and VOTE configurations (total scores and individual factors) from the long-stay wards as well as looking at this over time. We also have test–retest data from a forensic ward and these could be further examined in terms of the VOICE and VOTE scores to investigate if there are differences between wards. Other ongoing research is examining whether or not VOICE scores are sensitive to a peer support intervention on acute wards.

Methods

Reprinted from International Journal of Nursing Studies, Vol. 47 Issue 7 Sharac J, McCrone P, Sabes-Figuera R, Csipke E, Wood A, Wykes T, Nurse and patient activities and interaction on psychiatric inpatients wards: a literature review, pp. 909–17 Copyright (2010), with permission from Elsevier. 15

We followed the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines by setting up a search strategy as well as independently checking papers for inclusion and searching reference lists. Initially, the electronic databases EMBASE (1980–2008), MEDLINE (1950–2008) and PsycINFO (1806–2008) were searched with the following keywords: ‘inpatient’, ‘mental’, ‘psychiatric’, ‘hospitaliz(s)ed’, hospitaliz(s)ation’, ‘time’ and ‘activity’. A final check of the literature was made on 30 November 2009. Included studies were limited to those published after 1970 that were written in English and reported activity on psychiatric inpatient wards. Abstracts were reviewed by JS and PM to assess eligibility for inclusion. The reference lists of included studies were checked by PM for further relevant literature. The criteria for inclusion were that (1) the study was based in an inpatient setting and (2) that it measured or observed and recorded time spent by patients and/or staff on different activities. Given that studies differed substantially in their methods and contexts, we did not attempt to combine the results in a formal way. In addition, given the study heterogeneity we did not have predetermined measures of quality but we do discuss study characteristics in the text.

Results

The search strategy identified 86 papers of potential relevance. After reviewing abstracts, two studies were initially included. A further 11 studies were included following examination of reference lists. Study characteristics are summarised in Table 8.

Four of the studies were performed in England, three in Australia, two in Scotland and one each in Finland, Canada, the USA and Northern Ireland. Before the findings from the studies are described, we will summarise the methods used and make some inference about the quality of the studies.

Studies used data collected through interviews, questionnaires, audits of attendance registers, and observations of staff and patients on wards at varying time intervals, usually recorded with an observational instrument and coded according to specific categories.

Discussion

This review identified studies that have examined how staff and patients on acute inpatient psychiatric wards spend their time and how much time is spent in staff–patient interaction. Studies published over a 35-year period from seven countries revealed quite consistent and clear findings: (1) on average, around 50% of staff time is spent in contact with patients, (2) the more senior the staff member, the smaller amount of time they spend with patients, (3) the amount of time spent delivering ‘therapy’ is probably in the region of 4–20% and (4) relatively little patient time is spent in contact with staff and much is spent in isolation. There is also evidence to suggest that staff time with patients is reducing over time13 and that increasing staff numbers may not result in more time spent with patients. 118 The finding that staff spend relatively little time in direct contact with patients and that patients spend little time with staff do not go hand-in-hand as this depends on the staff-to-patient ratio.

The review also examined the methods used in the studies, most of which observed staff and/or patients over a period of time. A number of these studies were relatively small, including only limited numbers of staff. However, others were more representative in that they covered activity across different wards and sometimes across different hospitals. The majority of studies observed activity for only a limited period of time (sometimes just 1 week) and, as such, may not reflect typical activity patterns, but this is less of a concern in the multiward/site studies. This was recognised as a problem and it is perhaps inevitable given the limited research time that they would have been faced with. Most studies also focused only on daytime shifts. A further recognised problem with the studies is that observers might influence the very activity that they are measuring.

Other methods to measure activity are self-report and collection of data from audits/administrative systems. Self-report by nurses may result in recall bias and an incentive to ‘inflate’ time spent in activities considered more ‘worthwhile’. Self-report and observational methods have been compared (on non-psychiatric wards) and differences found. 126 Nurses reported significantly more time spent on patient care (40% compared with 33% observed) and ward-related activities (7% compared with 3% observed). However, although significant, these differences are not particularly large. The authors also report that observational methods were more acceptable to staff than self-report, which perhaps reflects the busy workload on inpatient wards.

Methods

Initially, the electronic databases EMBASE (1947–2014 week 4), MEDLINE (1946–2014 week 4), and PsycINFO (1806–2014 week 3) were searched with combinations of ‘inpatient’ or ‘admission’ or ‘admitted’ or ‘hospital$’ in the title plus ‘healthcare util$’ or ‘health care util$’ or ‘healthcare cost$’ or ‘health care cost$’ or ‘cost$’ or ‘econ$’ or ‘financ$’ in the title plus ‘mental’ or ‘psychiatric’ in the title. Abstracts were reviewed by PM to assess eligibility for inclusion and checked by LK. The criteria for inclusion were that (1) the study was based in an inpatient setting and (2) that it reported an economic evaluation of a specific intervention rather than a model of care. As in the previous review, we did not attempt to combine the results in a formal way.

Results

The search strategy identified 236 papers. None of these met the search criteria.

Discussion

Inpatient care constitutes a substantial proportion of mental health-care expenditure and yet there appear to have been no economic evaluations of interventions that take place on inpatient wards. Although this is worrying, it was not an unexpected finding given that there is little clinical evidence regarding inpatient interventions and it is rare for economic findings to be produced in the absence of clinical results.

Methods

The target population of the decision model were inpatients with schizophrenia. SCT was compared with treatment as usual (TAU). The economic value of SCT was assessed from a health-care perspective and a time horizon of eight months was assumed so that no discounting of costs or benefits was required. There were only limited data on care pathways for this intervention and the outcomes experienced by patients. Therefore, a simple decision tree structure was chosen (Figure 1).

FIGURE 1.

Decision tree structure.

Evidence was collected on the risk of relapse for each trial arm over the time horizon of the model.

For the purposes of this exploratory model, the general psychopathology subscale of the Positive And Negative Symptom Scale (PANSS) was chosen as an outcome measure. PANSS is a commonly used, disorder-specific, non-preference-based instrument. It measures symptoms in patients with psychosis rather than quality of life. This means that cost-effectiveness estimates based on this measure are not easily comparable to alternative interventions in other disease areas. However, our model uses an equation estimated by Siani et al. ,130 which predicts EuroQol-5 Dimensions (EQ-5D) utility values based on PANSS scores. The EQ-5D utility values were then used to calculate quality-adjusted life-years (QALYs). QALYs combine the health-related quality-of-life values and the length of time that this quality is enjoyed into a single summary metric.

Relevant clinical trial evidence was identified based on two recent systematic reviews of the SCIT literature and studies citing these. 131,132 The evidence on the reduction in the PANSS subscale was synthesised using a fixed-effects model for continuous measures assuming that there was no correlation between baseline and follow-up values within each trial arm. The meta-analysis was conducted in Stata^® (version 11; StataCorp LP, College Station, TX, USA). 133 To date, there are only a few studies investigating the durability of treatment effects in SCT. 131 For this reason, the treatment effect was assumed to decrease in a linear fashion up to the end of the time horizon of the model.

We identified only one small observational study by Hodel et al. 134 (n = 19) investigating the effect of SCT on relapse rates. It suggested that this intervention may reduce rehospitalisation at 8-month follow-up. Conservatively, the reduction in relapse rate was assumed to lead only to cost saving but not improvements in quality of life. Based on results from elsewhere in the programme, the average length of admission was assumed to be approximately 45 days. To determine the cost of inpatient stay, data were derived from a nationally recognised source. 134 The length of SCT in the literature was variable, ranging from 6 to 24 weeks. 135 Based on the studies used to inform the model, treatment duration of 8 weeks with two 45-minute sessions per week led by a psychotherapist was assumed. The cost of a clinical psychologist was used as a proxy for the cost of a psychotherapist. 136

In addition to calculating the differences in costs and benefits of the intervention, the cost-effectiveness of the interventions was compared using a net monetary benefit approach. 137 If λ is defined as the value that the decision-makers assign to increasing QALYs by one unit and E and C are the incremental costs and differences between the two treatments, the net benefit of the intervention can be defined as NB = λE – C. Based on this framework, a probabilistic sensitivity analysis was conducted for a range of λ values from £0 to £40,000 per QALY (i.e. the probability that SCT was cost-effective was calculated based on random draws from the distribution of the input parameters). The results of this analysis were displayed on a cost-effectiveness plane as well as a cost-effectiveness acceptability curve. In addition, a threshold analysis was conducted by examining at which point the net benefit of SCT would be zero assuming that the reduction in relapse rates in the observation study was biased upwards.

Results

Three relevant studies were identified in the literature review. 138–140 The weighted mean reduction in PANSS score attributable to SCT compared with TAU was estimated to be –3.49 (95% CI –8.01 to 1.04) (Figure 2). The study by Hodel et al. 134 suggested that SCT reduced the risk of relapse by 26% (95% CI –70% to 17%) percentage points. The cost of the intervention was calculated to be £1632 and the savings for an averted hospitalisation were £15,615. In the base-case scenario, SCT was estimated to both increase QALYs by 0.009 (95% CI –0.007 to 0.018) and decrease the costs by £2532 (95% CI –£3282 to £8110); Figures 3 and 4 show the joint distribution of costs and benefits. In other words, the model indicated that it was the dominant treatment option. In addition, the probability that SCT was the most cost-effective was found to be high (> 75%) at all levels of willingness to pay for a QALY improvement. The threshold analysis suggested that in order to be at least as equally cost-effective as TAU, SCT needed to reduce the risk of hospitalisation, on average, by 8.5% to 10.5% depending on the willingness to pay for a QALY improvement.

FIGURE 2.

Forest plot of meta-analysis: reduction in general symptoms PANSS.

FIGURE 3.

Cost-effectiveness plane: mean costs and effects of social cognition relative to TAU with 95% confidence region.

FIGURE 4.

Cost-effectiveness acceptability curve.

Discussion

One should be cautious not to overstate the results of this exploratory model because some strong assumptions were used to be able to construct a coherent framework based on relatively sparse data. For example, the mapping or ‘cross-walk’ approach employed in this study has been described as a ‘second best’ solution and introduces additional uncertainty and errors into estimates141 Although we showed that the reduction in risk of hospitalisation of less than half of the base-case scenario may be sufficient for SCT to be cost-effective, it is highly uncertain to what extent the findings by Hodel et al. 134 are generalisable. As noted by Henderson,135 research on SCT ‘remains very much in the developmental stage’. Based on current evidence, there is little robust evidence regarding its effectiveness but the results of this model suggest that it might have the potential to be a cost-effective intervention. However, before making the transfer from a developmental stage to routine clinical practice, further knowledge regarding the effects of SCT on quality of life and in the long term is required.

The objective of this section has been to demonstrate the feasibility of using a modelling approach to assess the cost-effectiveness of inpatient-based therapeutic interventions. As will have been apparent, this approach is dependent on the quality of the data available. Although it can circumvent the need to collect primary data from patients, it still requires robust estimates to be made of model parameters and to explore any uncertainty around these estimates in an appropriate manner. Having said that, the modelling approach is generally feasible and, as long as it is conducted with transparency, it represents an accepted way of generating information on cost-effectiveness.

Methods

Reproduced from Sabes-Figuera R, McCrone P, Sharac J, Csipke E, Craig TK, Rose DS, et al. Developing a tool for collecting and costing activity data on psychiatric inpatient wards. Epidemiol Psychiatr Sci 2012;21:393–9. 142 © Cambridge University Press 2012.

In developing the tool we conducted interviews with inpatient staff to discuss its content and structure. The professionals interviewed included two ward managers, three OTs, one nurse and one therapy and benefits co-ordinator. All were working at psychiatric inpatient units in the South London and Maudsley NHS Foundation Trust. The interviews lasted for approximately 30 minutes and the main objective was to identify the group activities that take place on the wards for which professionals were most likely to have contacts with inpatients. Staff sometimes provided detailed schedules of patient activity on their wards. The first draft of the tool was then circulated for comments and suggestions to a group of experts including psychiatrists, psychologists and nurses.

Twenty-five service users from inpatient psychiatric wards were then interviewed using the questionnaire. The objective was to determine the acceptability of the tool, service users’ opinion about difficulties in answering the questions and to evaluate how practical it was to collect data in this way. Two focus groups were conducted in parallel with this feasibility component. The first included eight OTs, with seven nurses from an inpatient ward comprising the second. Participants were asked for their opinions about the tool and for any suggestions for improvements.

The process of consultation with staff resulted in the development of a tool that allows activities specific to an inpatient ward to be measured. The list of group activities was amended to include only the activities that had taken place in the previous week. Interviewers list the availability of activities (based on the ward calendar for the relevant week) and, therefore, each version of the tool is unique to that particular ward and time period. Initial questions on contacts with members of staff were shortened. The staff contacts section was also revised so that service users could indicate the name of professionals they have met even if they did not know or remember their job title. The tool is shown in Appendix 9.

This next study assessed the level of congruence between the information provided by service users using the final version of the tool and information that could be obtained from other sources. It took place in two stages. Initially, information on the activities attended by 41 service users over the preceding 7 days was obtained from the OT of each ward and the number of contacts with nursing staff members was collected from the patient’s records for the same period. This information was also collected from the same patients and period using the tool.

Information on the number and duration of staff contacts and activities attended by 22 additional service users was collected over a 14-hour direct observation period (08.00–22.00). An adapted 1-day version of the CITRINE questionnaire was used to collect the relevant information from the same service users over this period. (This stage was added later and so the patients are different from the 41 previously included.)

Contacts with staff members were combined with unit costs measured in UK pounds for the financial year 2007/8 from an established source. 109 In the first phase (n = 41) we assumed an average nurse contact of 15 minutes. Activity costs were calculated specifically for the study based on session duration, preparation required, staff involved and materials for each activity provided on each ward. These data were combined with unit costs of staff time109 and with information on the average number of service users attending each activity to obtain an estimated individual cost for each group activity. Activities were subsequently classified in categories according to their type and cost. (A list of all unit costs is available from the authors.) An average of these activity costs was used in the analyses presented here. The significance of differences in reported and observed/recorded mean resource use was assessed using a paired t-test and the agreement was further assessed using the concordance correlation coefficient.

Results

The CITRINE tool typically took between 5 and 10 minutes to complete and most of the service users who participated in the feasibility study found it easy or very easy to answer the questions. Table 9 reports 7-day data from the CITRINE and from information covering the same period from OTs and case notes for 41 service users. The mean age was 37 years and 19 of the participants were women. Most participants (68%) had a primary diagnostic of psychosis or bipolar disorder and the average length of stay at assessment was 43 days. However, there was great variation in this represented by a SD of 66 days.

Service users report attending more activities than is contained in case notes (see Table 9). Attaching an average unit cost of activities suggests a cost difference of £10 per person. Case notes also report more one-to-one nursing contacts, resulting in a cost difference of £4 per person. Although neither difference is large, the discrepancy in the number of activities reported is statistically significant.

The comparison of data obtained from the 1-day observational study and the 1-day version of the tool shows good congruence in terms of activities attended and psychiatrist contacts. However, the congruence in terms of contact with nurses and other staff is less good. The differences in the 1-day version of the tool were significant or of borderline significance. The concordance correlation coefficient for total costs from this part of the reliability study was 0.79. The cost implications for the difference in nursing contact are important given that the observational data suggest that this accounts for one-third of the total cost.

Discussion

The process of developing the CITRINE tool has involved input from a wide range of staff involved in providing care to those receiving psychiatric inpatient services (nursing staff, OTs, ward managers and psychologists). Furthermore, key input to the process has been provided by the recipients of this care. The tool collects data directly from service users. From a pragmatic perspective, this is appropriate as it is the most practical way of obtaining the breadth of activity data that we require for economic studies. In addition, it emphasises the service user’s perspective in reporting activity. Of course this could mean that activities that are entirely unmemorable will not be recalled or costed (e.g. conversations with nurses that could be very similar every day such as ‘you have a telephone call’, ‘do you want to go for a walk?’). However, this type of very brief contact that is not remembered by service users is not the focus of the tool.

The result is a tool that has the objective of collecting information on the therapeutically relevant activities and staff contacts that take place on psychiatric wards and at the same time is acceptable to service users and relies on them as the main source of the relevant information. There have been other attempts to describe interventions and services provided in inpatient mental health settings, such as the International Classification of Mental Health Care. 143 However, this instrument is a tool to classify services according to type of care, rather than one that provides information on the specific services that patients receive.

The main function of the tool will be its use in economic studies, combined with information on relevant unit cost of the services and staff time. Therefore, its level of accuracy in reflecting what is actually taking place within psychiatric wards is paramount. The reliability study carried out offers some reflections on this issue. In particular, the questionnaire (1) is a good source of information on the number of activities attended by service users and on contacts with psychiatrists, (2) may provide acceptable information on interactions between service users and non-nursing care staff (OTs, psychologists, social workers, etc.) and (3) may be more limited in recording time spent in service user–nursing staff interaction.

In relation to nursing staff contacts, there are some aspects that should be considered. First, this type of contact is common and, consequently, creates difficulties for accurate recall. Second, service users might report only contacts that they think are significant or meaningful for them. This classification is subjective and can result in some contacts being labelled as ‘non-contacts’ by service users. For example, a member of staff may have spent some time asking how a patient was and would regard this as a contact, but it may not have been recognised as such by a patient if it was very brief or unwanted. This may have been the key reason for the discrepancy between the observed nurse–patient contacts and the patient-reported contacts. The mental health status of some of the service users may affect their ability to provide accurate information, although data on this are lacking. Although time spent with nursing staff needs to be measured correctly, the difficulty seems to apply only to one-to-one contacts. Contacts as part of organised activities are more readily measured and, therefore, the disagreement over total cost is limited.

The alternative sources of information on inpatient psychiatric activities and staff contacts are not free of problems. Registers and electronic databases are designed to support clinical care and not to record activities within wards. Furthermore, there might be intra- and interward variability on the level of completion of these and on the accuracy of the information recorded. Observational data are an alternative but require too many resources to be a realistic when studies involve large numbers of individuals/wards with data collected at several time points.

There are limitations with this work. First, we did not conduct the full range of psychometric tests, although we did assess validity and reliability. It should be stressed that we do not regard this as a clinical instrument but rather as a simple recording schedule to ascertain more accurately the use of care on inpatient wards. We anticipate that the tool being adapted for different circumstances and extensive tests on one particular version would have limited usefulness. Second, and related, we did not assess inter-rater reliability. The content of the tool, though, is such that ambiguity of responses should be limited and, hence, inter-rater reliability is not as important as with a tool for which interviewer interpretation is required. Third, the activities relate to wards in one particular hospital. It may be that in another setting a more complex array of activities would present more of a challenge. However, the tool does require the listing of specific activities at the start and so this problem is reduced. Fourth, the relatively poor concordance on contacts with nurses is a cause for some concern. This has been discussed, but the wording of the question may not be clear. Greater definition of what constitutes a contact may be required. Fifth, service users may not always have been clear about the profession of the person they had contact with. However, we have dismissed this because no service users mistook a psychiatrist for nurse and this is likely to be the category of error that would most skew the costs data. Self-report data are associated with such problems. Sixth, to rely on self-report data might be a limitation of the tool although a number of studies have suggested that patient recall of service use is acceptable. 144–146 Asking for the duration of contacts may be difficult but these durations are meant to be approximations.

Methods

Results

Of those service users who fulfilled the inclusion criteria, 62% agreed to participate in the study. Service use data were available for 402 service users; however, some values for sociodemographic and clinical variables were missing for some participants or their length of stay was < 7 days. It was decided to include in the analysis only participants with complete data and the final sample size was 334. It was found that those with incomplete data had lower GAF functioning scores (35.6) than those included (39.8) (p < 0.01).

Characteristics of the sample are shown in two tables (see Tables 10 and 11). Two-thirds had a diagnosis of schizophrenia or bipolar disorder and a similar proportion had been admitted under the Mental Health Act. The average length of stay at the point of assessment was almost 7 weeks, but with a wide range. The GAF scores indicate a high level of disability. As stated above, the VOICE was designed so higher scores reflected lower quality of care. Therefore, those service users with a score of 38 or lower felt that the quality of care received was good. Alternatively, those with scores of 95 and higher, perceived that inpatient care was of low quality. In our sample, 21% had a VOICE score of 38 or lower and 5% had a score of 95 and higher. Therefore, most of the sample felt that the quality of care was neither good nor bad.

Table 12 summarises the data collected with the CITRINE instrument, providing information on the number of therapeutic activities attended and number and type of one-to-one care contacts reported by participants during the period of 7 days. Almost 10% of the sample stated that they had not had any one-to-one contact and more than one-fifth reported that they had not attended any activities. The average number of one-to-one care contacts for the whole sample was less than one per day.

More than 40% of participants reported no contact with a member of nursing staff in the past week and for those who did have contact, their frequency was slightly higher than one contact every 2 days (Table 13). Participants with no nursing contacts were not more likely to have contact with other staff members (and so this does not indicate a problem of distinguishing between care staff). The number of participants who reported contacts with the other types of care staff was low, except for psychiatrists, with whom almost three-quarters of the sample had a weekly average of 1.6 contacts.

The service use data were combined with the unit costs for 2008 to obtain the costs of therapeutic group activities and one-to-one care contacts during the previous week. There was significant variation within the different group activities in the cost per participant attendance. This variation is the result of three factors: first, the different qualifications and number of professionals involved; second, the duration of the activities and the time necessary for their preparation; and, finally, the number of participants who attended each activity. This number varied from 2 to 14.

Table 14 shows the average cost of group activities and one-to-one care contacts over the past week for the 334 participants. Costs were highest for time with a psychiatrist and accounted for more than half of the cost of care contacts. The cost of group activities represented a small proportion of the total cost, as did contacts with nursing staff. If the total figure, cost of group activities and one-to-one care contacts for the 1-week period is translated to a cost per day, the value is slightly more than £30 per day. There is substantial variation between individuals, as reflected by the SDs.

The regression analysis of the variation in costs of one-to-one contacts (Table 15) showed that older participants had lower costs. A higher score on the VOICE instrument was related to lower cost, indicating that participants with a worse perception of inpatient care also use fewer inpatient resources. For instance, a decrease in the VOICE score of 10 points would imply, assuming other variables do not change, an increase in costs of £11. There were significant cost differences between inpatient wards. Regarding the activities model, the VOICE score was, again, significantly associated with cost (see Table 15). Service users with a higher level of education had higher activity costs. (Subsequent analyses revealed this to be caused by more engagement with activities rather than use of higher-cost activities.) The results of the two models also show the positive relationship between the two types of cost.

Discussion

It was striking that 40% did not report contact with nursing staff. The results for therapeutic activities attended do not indicate that the low level of contacts with staff was compensated by more attendances. We have estimated that the cost of care contacts and therapeutic activities is around £30 per day, which is just slightly higher than 10% of the average cost per bed-day of £288 for 2008/9. 149 This figure of £288 includes the costs of other resources not considered in the present study such as cleaning, maintenance, food, laundry, drugs and other treatments, etc. Crucially, the figures here do not include staff time that is not spent in contact with patients. In relation specifically to nursing staff costs, Bowers and Flood150 analysed expenditure data (i.e. a ‘top-down’ approach including non-contact time) for 136 wards and they found a cost per bed-day equal to £90, while in the current study we found that the daily cost of nursing staff contacts was £4.20. The difference could be partially caused by an under-reporting by service users of any contacts and, specifically, nursing staff contacts. However, our review of studies that measured nursing and patient activity and interaction on psychiatric inpatient wards found proportions of staff time spent in direct contact with patients (24–48%) and/or providing specific therapeutic interventions (4–20%).

High scores on the VOICE questionnaire, representing low satisfaction, were related to low costs of staff contacts and activities; however, the direction of the association is not clear. It is possible that attending fewer activities and having fewer contacts with staff is a result of a worse perception of care received, or low levels of care received may lead to poor perceptions of care. In relation to sociodemographic characteristics of the patients, age was inversely related to staff contact costs, while service users with ≥ 12 years of education had activity costs 23% higher than those with fewer years of education. Nevertheless, it is interesting that McCrone et al. 151 found that older people and those with lower levels of education had higher costs of community mental care in a sample of individuals with severe mental illness. The age effect is the opposite of what was found here and may be related to other factors (e.g. physical health in the elderly and/or chronicity relating to poor function). In addition, it should be recognised that these costs may be influenced by the structure of services; staff are present on inpatient wards (i.e. they are available), whereas in the community it may be more complex to arrange a contact.

These findings need to be considered while taking account of the limitations of the study. First, a relevant proportion of inpatients, almost 40%, refused to participate in the study. It was not possible to collect information on demographic or clinical characteristics for this group of patients, therefore preventing a comparison with those who agreed to participate. However, subsequent data collection as part of WP4 indicates that those who consented were representative of people admitted to the wards. There are also some limitations of the CITRINE questionnaire which were discussed earlier.

The low costs found reflect low levels of reported interaction between care professionals and inpatients. This is concerning because it has been established that in inpatient care, outcomes are associated with the level of attention patients received from staff. 152 The relationship found in this study between costs and service users’ views of the quality of care, measured by the VOICE instrument, emphasises these previous findings.

Procedures

Ethics approval for the study was granted by Bexley and Greenwich Research Ethics Committee (reference: 07/H0809/49). Researchers approached all eligible patients following consultation with ward staff. Researchers gave each participant an approved information sheet explaining the study, answered any questions and gained signed informed consent. Participants were free to withdraw at any time. Handovers and ward meetings were used to publicise the study with the nursing staff. Recruitment lasted for a period of 4 weeks at each of the time points. Data were stored in a locked office and entered into the database, and every entry was double checked by a different researcher to ensure accuracy.

The setting

This study was carried out in two mental health trusts and covers several distinct geographic areas.

In the first trust, we studied wards serving three borough catchment areas. The first borough serves an inner city population that has a deprivation index of 29.73 (ranking 41 out of 326 English boroughs163). Five wards participated in this study: three men’s wards and two women’s wards. Each had 18 beds. The catchment area also provided an intensive care ward comprising14 beds but this was not included in the study.

The second borough was located in a more suburban area, with a deprivation index of 22.76 (ranking 107 out of 326 boroughs163). Three wards participated in our study: one men’s ward and two women’s wards. Two wards had 22 beds. One of the women’s wards was unique in that, although considered part of the acute service, it was a ‘ward in the community’. This ward had only eight beds and accepted service users only if they were stable and not on a section. The ward was a detached house with a large garden located in a suburban street and the ward ethos was to provide as many therapeutic activities as possible.

The third borough had an average index of deprivation score of 31.24 (ranking 29 out of 326 boroughs163). Four 18-bedded wards provide acute inpatient care. One ward provided beds for women and one ward for men. One of wards was a specialist service, an early intervention unit accepting service users of both genders with psychosis from across the entire trust.

The final four wards were in a separate trust. This area was also mixed socioeconomically. Two of the wards served an urban borough with a deprivation score of 31.94 (ranking 28 out of 326 boroughs163). Both wards were mixed gender and had 18 beds. They also provided a PICU (not included in the study). The final two wards served two boroughs, both of which were suburban. The first had a deprivation score of 16.71 (ranking 174 out of 326 boroughs163) while the second had a score of 14.95 (ranking 203 out of 326 boroughs163). These were also mixed gender and provided 18 beds.

Participants

Service users using inpatient services on the participating ward at the time of data collection were eligible to take part in the study if they met the following inclusion criteria: they had been on the ward for a minimum of 7 days, they were able to communicate in English and they were well enough to consent. The exclusion criterion was that the participant had previously participated in the trial. There were no diagnostic exclusion criteria.

Nurses of any grade were eligible to be included in the study. Temporary agency staff were required to have completed seven shifts in the previous month in order to be eligible so that we could be sure that they were sufficiently familiar with the ward environment to complete the measures.

Main outcomes

For all measures obtained by questionnaires it was a requirement that > 80% of the questions were completed. Responses with < 80% completion were counted as missing.

Secondary outcomes

Background information

Intervention

Following consultation of NICE guidelines, trust clinical leads, ward managers and nursing staff directly involved with each ward, eight interventions were chosen, based on evidence of feasibility and outcome in wards. Ward managers suggested that three of these interventions should be compulsory. These were SCIT,129 CBT-based communications training for nurses (cofacilitated by a service user educator) and computerised cognitive–remediation therapy (CRT) (to involve OTs). In addition, pharmacists were recruited to run a medication education group. The wards then had a choice to receive training in two of the following five interventions: Hearing Voices Group, Emotional Coping Skills Group, Problem Solving Skills, Relaxation/Sleep Hygiene and Coping with Stigma group. Each ward made their choices based on what activities were already occurring on the ward or the client mix. Please see www.perceive.iop.kcl.ac.uk/doorways.html168 for a summary of each. A modified SCIT intervention was carried out on forensic wards as a value added study (see Appendix 4).

Statistical methods

Baseline data

A total of 1108 service users took part in the study. This is 70% of the population of service users who were eligible to participate at the time of the assessments. A total of 560 staff members consented to take part and we had valid data from 539 (444 pre-training intervention and 280 post training). Some contributed more than one set of data.

Outcomes and estimation

Costs and cost-effectiveness

What is the effect of the intervention on the cost of perceived contacts?

We estimated that the total cost of the intervention across the 16 wards was approximately £156,000 which amounted to a cost of £10 per patient-week given an average number of 18 patients per ward and an average post-intervention follow-up of 55 weeks (Table 22). About 80% of this cost was attributable to the opportunity cost of nurses attending the training sessions. The average cost of perceived staff contacts over the course of a week was £259 before the intervention and £242 after the intervention. Our regression analysis suggested that the intervention increased perceived costs by £12 (95% CI –76 to 98; p-value = 0.774) but this difference was not statistically significant (Table 23).

TABLE 22 - Cost of the intervention (WP3)

Item	Number of units	(Average/total) number of participants	Number of sessions	Session duration (hours)	Cost per hour (£)	(Sub)total cost (£)
Opportunity costs of time spent training
Nurse	16	53	1	3.5	41	121,258
OT CRT training		19	7	1	34	4522
Clinical psychologist instruction costs
Training Sessions	16	1	4	3.5	60	13,440
Activity cofacilitation	16	1	17	0.5	60	8160
CRT		19	7	1	60	7980
Total						155,360

TABLE 23 - Results for cost of perceived service use (WP3)

Model	Coefficient	95% CI	SE	p-value
Average intervention effect	12	–76 to 98	44	0.77
Time effect (6 months): linear	14	–36 to 5	10	0.14
Intercept	293	238 to 360	32	< 0.001

Figure 6 shows the uncertainty surrounding the effects of the intervention on cost and VOICE estimates in combination. We estimated that the intervention most likely improved VOICE scores but increased costs of service contacts at the same time (i.e. 50% of our bootstrap replicates were in the north-west quadrant). It was unclear what the value of increasing the score by one unit was but, even at a willingness to pay of £100 per improvement in VOICE, the probability that the intervention was cost-effective was low (Figure 7). In the cost-effectiveness analysis using VOTE, results differed somewhat from the previously reported separate estimations because to combine these two end points, the analysis had to be undertaken at the unit level. In the cost-effectiveness acceptability plane using VOTE, the probability that the intervention led to deterioration of staff perceptions and reductions in costs was highest with 41% followed by the probability that the intervention would lead to deterioration in VOTE and an increase in cost with 29% (Figure 8). Regardless of the willingness to pay for improvements in VOTE, our analysis suggested that, with a probability of being cost-effective of 40%, it was unclear whether or not the intervention was cost-effective if staff perception was the only relevant outcome measure (Figure 9).

FIGURE 6.

Cost-effectiveness plane using VOICE with 95% confidence ellipse (WP3).

FIGURE 7.

Cost-effectiveness acceptability curve using VOICE (WP3).

FIGURE 8.

Cost-effectiveness plane using VOTE with 95% confidence ellipse (WP3).

FIGURE 9.

Cost-effectiveness acceptability curve using VOTE (WP3).

Adverse events

No adverse events were reported as a result of the intervention.

Summary

The DOORWAYS study sought to improve the perceptions of the ward environment among those living and working on acute inpatient mental health wards by training staff to provide activities and increase the number and quality of therapeutic activities taking place on these wards. For service users, the measured perceptions of the environment moved in the direction expected and, for those who were admitted compulsorily, this was significant. But nursing staff had more complex views on the inpatient environment. We also found that we were able to make an impact on the day-to-day life of the wards in a number of ways. However, there was only limited indication that the intervention was cost-effective at the service user level (depending on how much a unit improvement in the VOICE is valued) and even less likelihood that the intervention was cost-effective at the ward level with regard to the staff measures, especially our main outcome, VOTE.

Impact on wards

The associations between social interaction and taking part in therapeutic activities and their impact on service users’ improvement is not a new one. 11 Not only do activities break up the monotony of time spent on wards,10 they also provide a forum in which service users can interact with each other, which we know is important. 34 Therefore, we attempted to increase the activities on the wards and had moderate success in this. As the intervention consisted of training nurses to deliver activities, we had no control over the actual number of sessions that were actually run, apart from the ones the psychologist was involved in. In spite of this, we were successful in increasing the average number of activities on the 16 wards in this study. All of these wards already had a set weekly activity schedule so it is likely that some of the existing activities were dropped and replaced by the ones in this study. This may be one reason why our improvement of an average of two extra activities a week was not statistically significant. Sainsbury’s Centre for Mental Health16 specifically recommended that more evidence-based therapeutic activities are run on wards, while NICE158 recommends psychological talking therapies be offered to those with psychosis. Although very much valued,159 the majority of groups already being run on the ward were activities such as cookery or bingo, and so we believe that the introduction of evidence-based therapeutic activities was a positive one. The cost of the intervention was small; spending per patient did not increase and, in fact, may have decreased over time as budgets were cut. However, the number of contacts and activities did increase, which service users considered to be valuable. Furthermore, it is probably true, as others have found,40,41 that administrative duties and crisis management often means that therapeutic activities are relegated to the bottom of the ward’s to-do list. Service users in the focus groups recognised this:

Sometimes the nursing staff, they’re quite occupied with what they’re doing around the unit. They don’t really have much time, they very rarely have much time to themselves to tell you the truth, they probably have a break once in the shift.

Anonymous service user

It is perhaps not surprising that our study was not able to singlehandedly change this. In light of that, increasing the mean number of activities from six to eight can be considered a success.

One of the results of the increased pressure on inpatient wards is the tinderbox atmosphere that is associated with violence. 8,9 During the period involved in the study, we recorded a total of 67 incidents of violence. Forty of these were on non-intervention wards while 27 were on intervention wards. We estimated this to be a 14% reduction in the rate of violence on intervention wards. Although we cannot claim statistical significance for this result, changes in relatively rare events are difficult to find and the study was not powered for this secondary outcome. Viewed in this light, and knowing about the background service pressures, this was a positive outcome and is indicative of an improved atmosphere on the wards following our intervention.

Impact on service users

The primary aim of this study was to improve the experiences of service users, which has been highlighted in many recent reports as vital in evaluating the performance of a service (e.g. Francis18 and CQC156). Our study, which began in 2009, was ahead of this recent focus. Although both perceptions and satisfaction of those using services moved in the right direction, we did not reach the generally accepted threshold of statistical significance for all those on the ward and so we cannot rule out the role of chance in producing these differences. But we were able to show improvements for those who are admitted under a legal section. We chose to investigate longer-term effects as these seem to have more relevance for the NHS, whereas other studies of changes in inpatient care have tended to investigate shorter-term effects, such as a few weeks. As wards aim to discharge service users as soon as they are able to do so, service users do not typically spend 6 months on the ward and so it was not possible to use repeated measures to assess their views of how activities had improved the environment, but this would have increased the power to detect small but consistent effects. We also only analysed the intention-to-treat effects of providing an intervention (training staff) on the outcome (change in perceptions of the ward). This is shown as the link between the intervention and the effects in green in the model (Figure 10). However, the effects may not be direct and the model shows other potential linkages that we might investigate through our database in the future. For instance, the effects of the intervention might be mediated by the effect on the number of interventions that are provided. Moderators include the turbulence on wards, such as the changes we noticed in ward leadership.

FIGURE 10.

Model of potential relationships between intervention and outcome (WP3).

Furthermore, this study was set against a backdrop of budget cuts and service reorganisations, which clearly had an impact on the wards, producing variability and a secular time trend. Our study also did not have sufficient power as it was not possible for us to collect data from all wards at all time points because we extended the study. Running such a complex trial on a large number of wards is a significant logistical challenge and will always present serious practical and analytical difficulties to researchers.

Our baseline analyses169 found that both activity and one-to-one sessions were associated with service users’ perceptions of the quality of the care they were receiving. These findings demonstrate that even those on an acute ward can engage in and appreciate the activities provided and the meaningful interactions they can have with staff, which is reflected in their perceptions of the ward. An often repeated justification for not running activities in the first place is that people are too ill to take part. Our study proved that this is not the case and that symptom severity was not associated with taking part in fewer activities. 169 Furthermore, our post-intervention focus group supported this finding:

I found the recognition one with the faces where you start off with the blank face and the face changes quite interesting because it shows that you can’t judge someone by a first look especially in a place like this because there’s lots of emotions and feelings going on in the place so it’s actually to get to know the person behind the, what they, I found that quite interesting.

Anonymous service user

Nursing staff can also take heart in this finding that their efforts are recognised, appreciated and valued.

Feedback from our focus groups showed that many service users were not aware of the activities taking place. This could be for a number of reasons: a lack of clearly displayed ward timetables, no introduction to the ward on admission, no mention of it in the community meetings, and no one inviting them to take part. Furthermore, both service users and staff reported that activities were sometimes cancelled at the last minute, but our data still suggests, on average, an increase in activities over time despite these cancellations. There was also some feedback that the activities, or the way in which they were run, could have been modified. Both groups reported that the activities became boring over time with repetition:

Yeah they repeat the same things again. That’s the only thing that I find difficult. They say ‘oh DOORWAYS meeting’ and I think oh what’s the point in going to a DOORWAYS meeting when it’s the same thing as last week. So there needs to be a bit more of a change going on.

Anonymous service user

An alternating schedule of activities planned over several weeks might have overcome this problem. Service users also reported that some activities were childish or like being in school again, so perhaps the complexity/level of the activities needs to be raised.

Impact on nursing staff

The results from the nursing staff were more complex. In spite of DOORWAYS requiring nurses to engage and interact with service users more frequently, there were no effects on interaction anxiety, as measured by the VOTE subscale. Nurses on the wards in the post-intervention focus groups reported that their confidence was growing following training. In the post-DOORWAYS focus group, a nurse described how they felt after our project finished:

Because we were a bit unsure of ourselves . . . But I think once we got into it we didn’t have a problem at all, we got more and more confident and it’s not a problem anymore to, and I think it’s helped us to run other groups because we’re more confident in that so now we can run other groups as well without any problem.

Anonymous nurse

We did find that workload intensity, job satisfaction and overall VOTE scores worsened slightly but not significantly and therefore opposite to our hypothesis of an improvement. Much of this effect we consider to be a reflection of the changes in the local NHS services during the study. Three out of the four catchment areas in our study reduced the number of wards and budgets were also reduced. In one case, staff had to reapply for their own jobs and some wards did not have ward managers but were run by junior staff or overseen on a part-time basis only. In light of this turbulent background, it is not surprising that being asked to deliver more activities had an effect on their view of the ward. This consequence was also reflected in an increase in the MBI emotional exhaustion scale. However, it is interesting that there was no effect on the personal accomplishment and depersonalisation scales of the MBI, or on the interaction anxiety measure of VOTE. Therefore, the effects, although important, are not solely explained by the increases in activities. Pragmatic health services research needs to consider such issues in terms of interpreting significant (or non-significant) effects. A similar issue is reported in Chapter 5 when we discuss that even in the absence of any intervention, the view of the services deteriorated over time. However, these staff findings may also be a feature of a clustered trial; with relatively few numbers of clusters and a large variability in the situation of each cluster it is always possible for results to be shifted by extreme conditions on a small number of clusters. In other words, a few events or circumstances negative enough on two wards would be enough to push the results into significance. One of our future aims is to interrogate the data to see if there was a consistent perception on all wards.

Costs and cost-effectiveness

The intervention required investment in training and, when apportioned over the wards and service users, this amounted to £10 per patient-week. The cost of meaningful service contacts and activities increased (non-significantly) by £12 per patient-week following the intervention (including the intervention cost). This does not reflect an actual increase in expenditure but rather a relative increase in resources from a fixed amount. For example, if the actual cost per patient-week of an inpatient stay is £2100 (7 days at £300 per day), then £12 per week extra represents a shift of 0.5% towards per direct patient care. The cost-effectiveness acceptability curve shows that an improvement in the VOICE would have to be valued very highly in order for the intervention to be cost-effective. It was unclear whether or not the intervention could be regarded as cost-effective using VOTE.

Strengths and limitations of this study

A major strength of this study was the use of the stakeholder perspective in the main outcomes from the service users and staff members. No study to date, to our knowledge, has had such a focus. Although entirely well intentioned and very much welcomed, many of the strategies and programmes aimed at improving inpatient services are developed by senior management or bodies such as the Royal College of Psychiatrists. 161 We involved the service users and nurses using the wards on a day-to-day basis and ensured this programme delivered what mattered to them.

There were a number of reasons behind our decision to use a stepped-wedge design for this investigation. First, as the intervention was at the ward level, a cluster-based design was clearly required in this context. Second, a stepped-wedge design allows nursing staff on all wards to receive the training. This is in distinction to a regular cluster randomised trial in which only half of the clusters would be allocated to the intervention arm. Third, it was not possible for us to train all wards at the same time owing to limits in the number of researchers and psychologists employed. By staggering the timing of the intervention, we could reach more wards with the same number of researchers.

The economic evaluation was conducted from the perspective of the service user with their perception of meaningful contacts/activities being the driver of costs. Service user data relied on self-report and this may have some limitations in terms of recall accuracy. However, we have assumed that if the contact was meaningful then it would be recalled.

Research implications

Future research needs to use the tools of implementation science to uncover the factors that might predict successful implementation. Although we were able to show that activities had increased and we have anecdotal evidence of where it proved to be relatively successful, we also know that there was much variance unaccounted for. We will be investigating some of these exploratory hypotheses on our current data set to best suggest the remit for future studies. A better understanding is needed of the strategies for knowledge integration, especially in high throughput inpatient settings. We also plan to explore differences between wards with special reference to their culture and those differences produced by boroughs and trusts. We will also be carrying out further analyses investigating the impact of the intervention on clinical indicators such as PANSS scores, NOSIE and GAF. The results from the economic evaluation indicate the shift that is likely towards more direct care provision and this approach should be developed in further studies.

Clinical implications

One of the most common complaints about inpatient services is the extreme boredom and lack of therapeutic activities occurring on the wards (e.g. Walsh and Boyle34 and CQC170). We successfully increased the number of activities and received positive feedback from those participating in the study, demonstrating that, with effort and dedication, it is possible to increase the number of activities. This was carried out with little increase in the costs and showed a transfer of costs to more patient valued care. Although there was variation in terms of implementation success, it should be possible to adopt our design to increase the skill and confidence of nursing staff, which has often been found to be perceived as poor. 29 This alone would be a positive outcome, not well captured in our current study but revealed in our qualitative analysis from the focus groups. Our baseline analyses also showed that symptom severity did not have an impact on the uptake of activities and participation in activities was related to more positive perceptions of the wards, demonstrating that they are a much valued component of inpatient services regardless of illness severity. This belies the belief that the acutely ill cannot take part in meaningful activities and also supports the view that more therapeutic activities could be of value and are appreciated.

Conclusions

This programme of research had some positive and some surprising results. Service users’ perceptions of their care were moving in the right direction and, for those who were admitted compulsorily, there was a significant effect. Nurses appear to be working in such a pressured system that their perceptions were mixed in spite of the positive views of our project discussed in the focus group. The number of activities taking place and the number of violent incidents also moved in the right direction. A much larger study of this kind is needed in order to detect statistically significant effects, as power was low even after we increased the number of wards.

The setting

The study was carried out in one large mental health organisation responsible for all mental health services. The defined geography covered by this health-care trust comprises four catchment areas, each of which has its own inpatient hospital provision prioritised for catchment area residents but service users from any area, including other areas in the UK, may be admitted. This WP focused on two of these areas, one of which operated a triage service.

Study 1: is a triage system more efficient than traditional care?179

Study 2: service user and staff perceptions of care

These data were collected four times at 6-month intervals over an 18-month period (largely coterminous with the comprehensive admission census described above).

This series of cross-sectional samples and their timing was deliberately chosen because of concern that a single cross sectional assessment would be open to the effects of seasonal changes and rare events, such as unexpected organisational changes that could not be controlled for in the analysis.

We carried out two sets of comparisons. In the first we compared staff and service user perceptions in the triage system (triage ward plus remaining wards) with those in routine care. In the second set of comparisons, we investigated specific differences between types of ward.

Study 1: is the triage system more efficient than traditional care?

There were a total of 935 admissions to the triage system and 899 to routine care over our census year beginning on 1 January 2009. There were few differences and none of the differences between the characteristics of people admitted in the two systems reached statistical significance (Table 30). Psychosis accounted for the largest diagnostic group in both systems and around 40% were involuntary admissions. For a fair comparison of the two systems, all admissions were analysed regardless of whether or not service users were resident in the catchment area. Admissions from individuals recorded to be homeless were assumed to be inside the catchment area. In both areas, a similar proportion of all admissions were for ‘out of area’ residents (routine care 21% and triage 24%).

Are there differences in length of stay between the two systems?

Although more admissions to the triage system had rapid discharge, both systems have service users with lengthy admissions. A univariable analysis showed that although the length of inpatient stay was, on average, lower in the triage system, the difference was not statistically significant (mean difference –3.87 days, 95% CI –12.20 to 4.47 days; p = 0.364). A multivariable analysis (Table 31) taking account of service user characteristics also failed to provide evidence of a difference between the systems (mean difference 0.53 days, 95% CI –9.77 to 7.21 days).

TABLE 31 - Length of inpatient stay (WP4)

Multivariable linear regression model including a random intercept to account for clustering effects by service user.

Bootstrapped SEs and percentile-based CIs were estimated using 1000 repetitions.

Number of admissions, n = 1722; number of individuals, n = 1367; average number of admissions per person, n = 1.3.

ICC = 0.49.

	Days	Bootstrap SE	p-value	95% CI (percentile based)
Service system
Routine (in catchment)	Reference
Triage (in catchment)	–0.53	4.37	0.90	–9.77 to 7.21
Interaction between system and catchment area
Admission from outside catchment area (for routine care, reference)	–13.39	5.86	0.02	–24.20 to 1.25
Service system by catchment area (interaction, additional effect of triage)	10.02	7.83	0.20	–5.98 to 23.93
Diagnosis
Schizophrenia	Reference
Other psychotic disorders	–12.32	5.53	0.03	–24.18 to 2.50
Bipolar and related disorders	–4.76	8.96	0.60	–20.78 to 14.74
Depressive disorders	–9.34	5.82	0.11	–21.04 to 2.19
Neurotic and anxiety disorders	–19.58	10.61	0.07	–36.37 to 3.68
Personality disorders, etc.	–22.28	8.63	0.01	–39.54 to 6.14
Substance misuse	–27.02	4.9	< 0.001	–37.74 to 17.84
Other/unspecified	–2.17	9.54	0.82	–17.64 to 18.56
Demographic and clinical factors
Age	0.06	0.16	0.70	–0.25 to 0.38
Previous number of admissions (log-scale)	9.36	3.33	0.005	3.94 to 16.59
Admission status
Voluntary	Reference
Compulsory assessment	1.67	2.66	0.53	–3.60 to 7.31
Involuntary treatment section	69.83	5.65	< 0.001	57.92 to 79.55
Fixed residence
Yes	Reference
No	69.54	9.65	< 0.001	51.35 to 89.76

The use of HTT also differed between systems with a greater proportion of service users from triage followed up by the HTT (32%) at discharge than from the routine care system (26%, OR 1.50, 95% CI 1.12 to 2.01; p = 0.006). Repeating the multivariable analysis of length of stay but including the additional days spent with HTT showed that the mean length of acute care is actually somewhat longer in the triage system (mean difference 7.83 days, 95% CI –1.62 to 15.63 days; p = 0.076).

Figure 11 shows probability density distributions of length of inpatient stay with and without involvement of the HTT in both systems. It is apparent that more admissions to the Triage system resulted in a rapid discharge but both systems had a long ‘tail’ of admissions that were in excess of 60 days. The use of HTT differed. In the routine care system, HTT duration was largely focused around one to two weeks following discharge from hospital whereas in the Triage system service users generally had longer contact with the HTT. When the two components were considered together, as a period of acute care, there were no differences between the systems. Looking in detail, we noticed that for the acute care episodes of short duration, both systems used a similar percentage of days on home treatment to days in inpatient care. However, for the longer care episodes we noticed that the Triage system discharges individuals to HTT care for longer.

FIGURE 11.

Length of care components (WP4). (a) Inpatient stay; (b) home treatment; (c) acute care; and (d) inpatient care/acute care. a, Probability density functions estimated using a kernel-smoothing function calculated on full sample, distribution censored at 100 days; and b, locally weighted scatterplot smoothing curves, censored at 100 days. Acute care = length of stay including any use of Home Treatment. Acute care admissions lasting ≤ 100 days accounted for 85% of the full sample. Reproduced from Efficacy of a triage system to reduce length of hospital stay. Williams P, Csipke E, Rose D, Koeser L, McCrone P, Tulloch AD, et al. Brit J Psychiat 2014;204:480–5. 179 Reproduced with permission of Royal College of Psychiatrists via PLSClear.

Reproduced from Efficacy of a triage system to reduce length of hospital stay. Williams P, Csipke E, Rose D, Koeser L, McCrone P, Tulloch AD, et al. Brit J Psychiat 2014;204:480–5. 179 Reproduced with permission of Royal College of Psychiatrists via PLSClear.

A sensitivity analysis that excludes the use of leave periods (where a service user is allowed home temporarily and his/her bed re-used) from the measure of length of stay, shows the average number of days spent on the ward was lower in the Triage system (mean difference: –8.36 days, 95% CI –16.54 to –1.51; p = 0.030).

About one in three of the admissions in each system resulted in a re-admission within 1 year of discharge. A multivariable model showed no significant difference in the probability of re-admission between the two systems (odds of re-admission: triage vs. routine = 1.05, 95% CI 0.77 to 1.44; p = 0.758) (Table 32). Results from the univariable analysis were (odds of re-admission: triage vs. routine = 0.90, 95% CI 0.80 to 1.22; p = 0.925).

TABLE 32 - Re-admission to inpatient care (WP4)

Multivariable logistic regression model shown was constructed using a stepwise variable selection procedure retaining all variables that had an association with re-admission (p < 0.2). Candidate variables that were tested and removed (p > 0.2) were residency outside catchment area, use of home treatment at the end of index admission, gender and age.

n = 781 individuals.

	Odds ratio	SE	p-value	95% CI
Service system
Routine care system	Reference
Triage system	1.05	0.17	0.758	0.77 to 1.44
Ethnicity
White	Reference
Black	1.2	0.21	0.291	0.85 to 1.70
Other	0.58	0.17	0.071	0.32 to 1.05
Employment
Unemployed/retired/education	Reference
Employed	0.72	0.18	0.187	0.43 to 1.18
Marital status
Single	Reference
Divorced/separated/widowed	0.51	0.12	0.005	0.32 to 0.82
Married/cohabiting	0.39	0.13	0.004	0.21 to 0.75
Diagnosis
Schizophrenia	Reference
Other psychotic disorders	1.3	0.29	0.238	0.84 to 2.03
Bipolar and related disorders	1.42	0.37	0.181	0.85 to 2.37
Depressive disorders	0.6	0.19	0.1	0.32 to 1.10
Neurotic and anxiety disorders	0.64	0.24	0.23	0.30 to 1.33
Personality disorders, etc.	2.34	1.03	0.054	0.99 to 5.55
Substance misuse	1.34	0.37	0.29	0.78 to 2.31
Other/unspecified	0.38	0.17	0.028	0.16 to 0.90
Any admissions prior to the first admission in 2009
No	Reference
Yes	2.7	0.43	< 0.001	1.97 to 3.69
Residential mobility
No	Reference
Yes	0.49	0.13	0.006	0.29 to 0.81

Study 2: the experience of service users and staff

Four hundred and fifty-four service users were recruited, which is 61% of all those eligible to take part. They were mostly single men of non-white ethnicity, aged, on average, 40 years and who were unemployed. Most had experienced a previous admission with a median of 4 years since their first admission, and the most common diagnosis was psychosis (56%). Mean NOSIE and GAF scores were similar between systems. The mean NOSIE score of the sample was 16.0 and the mean GAF (level and symptom scores) were 43.3 and 42.2, respectively (Table 33).

Do experiences differ between the triage and routine care systems?

Service users

The detailed results for these analyses are shown in Table 34. No significant difference was found in perceptions of the ward between the two systems (VOICE scores adjusted mean difference –0.77, 95% CI –4.44 to 2.90; p = 0.68). However, this analysis demonstrated that perceptions of wards deteriorated over time. For every increase in phase (6-month interval), the mean VOICE score for the whole sample increased (coefficient 2.72, 95% CI 1.00 to 4.43; p < 0.005). This result remained significant after adjusting the model for clustering. The results remained stable in the two sensitivity analyses adjusting for the number of days spent on the ward until interview. The exploratory model that separated the triage ward from the remaining triage system wards also found no evidence of differences between the three types of ward.

TABLE 34 - Staff demographic characteristics by care system (WP4)

For staff with repeated measures, the mean of the time points responded was taken.

	Routine care system (N = 150)	Triage system (N = 134)
Age (years),a mean (SD)	36.09 (10.22)	35.41 (9.14)
Gender: female, n (%)	85 (57)	87 (65)
Ethnicity: non-white, n (%)	85 (58)	89 (67)
Education group (highest achieved)
Vocational	53 (40)	50 (42)
University degree (undergraduate)	60 (46)	59 (49)
University degree (postgraduate)	18 (14)	11 (9)
Length of employment on ward (months)	20.5 (1–300)	18 (1–207)
Band (highest achieved)
2	10 (8)	6 (6)
3	36 (27)	22 (21)
4	2 (2)	0 (0)
5	53 (40)	51 (48)
6	24 (18)	23 (22)
7	8 (6)	5 (5)

No significant differences or associations were found in satisfaction between triage and routine care or between routine care and the remaining wards in the triage system (Figure 12). However, there was a trend towards lower satisfaction in the triage ward compared with the remaining locality wards (mean difference –6.91, 95% CI –14.66 to 0.84; p = 0.08). This remained stable in the two sensitivity analyses adjusting for the number of days spent on the ward until interview. A trend of deteriorating satisfaction with phase of the study was also observed (p = 0.07).

FIGURE 12.

Service user VOICE for each system by phase (WP4) by (a) system; and (b) system and ward type.

One potential explanation for the deteriorating service user perceptions is that we also found a trend of decreasing length of stay with phase. As the phase increased by one unit, the average total length of stay for participants decreased by 16.16 days (coefficient –16.16, 95% CI –26.84 to –5.49; p = 003). This finding remained in the sensitivity analysis which included clustering by ward (Figure 13 and Table 35).

FIGURE 13.

Service user SSS-RES for each system by phase (WP4) by (a) system; and (b) system and ward type.

TABLE 35 - Results for primary service user outcomes (WP4)

Linear regression covarying for time assuming a linear relationship.

p < 0.05 after performing a sensitivity analysis of the same model with robust SEs adjusted for clustering at ward level. ICC coefficient (ward) were 0.06, 0.07 and 0.06, respectively. Significance of all other results did not change.

Linear regression covarying for phase assuming a linear relationship. SEs were adjusted using bootstrapping with 10,000 replications. Five different seeds were used and all estimates were consistent to 1 decimal place. Seven participants with < 7 days from admission until interview were removed.

Reproduced from Csipke et al. 184 © The Royal College of Psychiatrists 2016. This is an open access article distributed under the terms of the Creative Commons Attribution (CC BY) licence.

	Coefficient	SE	p-value	95% CI
VOICE modela
Triage system	–0.77	1.87	0.681	–4.44 to 2.90
Phase	2.72	0.87	0.002b	1.00 to 4.43
Sensitivity analysesa
Triage system (excluding seven participants with number of days until interview of < 7 days)	–0.54	1.88	0.774	–4.23 to 3.16
Triage system (adjusted for days until interview)	–0.78	1.87	0.678	–4.45 to 2.89
SSS-RES modela
Triage system	–1.77	2.7	0.512	–7.07 to 3.53
Phase	2.29	1.24	0.066	–0.15 to 4.74
Sensitivity analysesa
Triage system (excluding seven participants with number of days until interview of < 7 days)	–1.53	2.71	0.572	–6.85 to 3.79
Triage system (adjusted for days until interview)	–1.79	2.7	0.508	–7.09 to 3.51
Total length of stayc
Triage system	–12.35	13.72	0.368	–39.25 to 14.54
Phase	–16.16	5.45	0.003b	–26.84 to 5.49

Table 36 provides results of a number of exploratory analyses of the potential predictors presumed to be associated with service user experience (VOICE) from prior research. Worse service user experience (i.e. higher VOICE scores) was associated with higher skilled worker groups, increasing severity of symptoms and those who were admitted for compulsory treatment. A strong association with the number of contacts service users make with staff and better service user experience is associated with more staff contacts. The trend with phase and with severity of symptoms remained significant in this adjusted model.

TABLE 36 - VOICE exploratory model (WP4)

p < 0.05 after performing a sensitivity analysis of the same model with robust SEs adjusted for clustering at ward level. ICC coefficient (ward) = 0.017. Significance of all other results did not change.

Linear regression of VOTE on sociodemographic and clinical variables selected using a stepwise selection procedure. Candidate variables that were tested in the model but removed owing to low ability to predict outcome were as follows: ethnicity, age, diagnosis, functioning (GAF level), number of previous admissions, number of days on ward, and number of recorded activities (in part 7 from CITRINE).

	Coefficient	SE	p-value	95% CI
System (triage vs. routine)	–1.59	1.8	0.379	–5.13 to 1.96
Phase	2.27	0.84	0.007a	0.63 to 3.92
Gender (female vs. male)	–3.79	1.8	0.036a	–7.33 to 0.25
Socioeconomic status
Lower professionals vs. higher professionals	–1.52	2.37	0.523	–6.18 to 3.15
Semi- and non-skilled workers vs. higher professionals	–6.76	2.93	0.021a	–12.52 to 1.01
Never worked and long-term unemployed vs. higher professionals	–6.08	3.15	0.054	–12.28 to 0.12
Compulsory treatment orders
1 month vs. informal	2.84	2.62	0.28	–2.32 to 7.99
≥ 6 months vs. informal	7.63	2.04	< 0.001a	3.62 to 11.64
Severity of symptoms (NOSIE)	0.28	0.12	0.021a	0.04 to 0.52
Number of contacts with service staff (in last 7 days from CITRINE)
Simple effect	–2.35	0.45	< 0.001a	–3.23 to 1.47
Quadratic effect	0.07	0.02	0.001a	0.03 to 0.11

Nursing staff

The same models were examined for staff but in these models we used time as a categorical variable. Interactions between system and phase were significant for VOTE [χ²(3) = 11.03; p = 0.01] and for MBI [χ²(3) = 8.89; p = 0.03] but not for the IWS model [χ²(3) = 5.80; p = 0.12].

There was no significant difference in staff experience at baseline between the two systems (adjusted mean difference: 1.68, 95% CI –2.05 to 5.41; p = 0.38) but staff experience in both systems changed over the period of the study and these changes differed between systems. In the routine care system, there was a deterioration in the VOTE measure at 6 months (in phase 1) (mean difference: 6.58, 95% CI 3.83 to 9.33; p < 0.001), but this was not seen in the triage system (mean difference 0.94; p = 0.503) and the difference between the two systems was significant (mean difference –5.64, 95% CI –9.53 to –1.75; p = 0.005). By the end of the study (phase 3), there was evidence of deterioration compared with baseline in both the routine care system (mean difference 4.80, 95% CI 1.30 to 8.30; p = 0.007) and the triage system (mean difference 4.78, 95% CI 1.93 to 7.63; p = 0.001) and no evidence that this was a differential change (p = 0.994). These results remained in a sensitivity analysis adjusting for clustering by ward (Figure 14 and Table 37).

FIGURE 14.

Staff outcomes for each system by phase (WP4). (a) VOTE; (b) MBI; and (c) IWS. a, Phases in which difference between systems was statistically significant.

TABLE 37 - Full results for the regression models of the staff primary outcomes (WP4)

p < 0.05 after performing a sensitivity analysis of the same model with an additional level of clustering at the ward level. Significance of all other results did not change.

Mixed-model linear regression including a random effect for individuals and covarying for phase (categorical, coefficients use the baseline as a comparison). A system by phase interaction was tested in each model and results are presented where this interaction was shown to be significant from a likelihood ratio test.

ICC coefficients for the models were 0.71, 0.64 and 0.72, respectively.

Reproduced from Csipke et al. 184 © The Royal College of Psychiatrists 2016. This is an open access article distributed under the terms of the Creative Commons Attribution (CC BY) licence.

	Coefficient	SE	p-value	95% CI
VOTE modelb
System
Routine care system	Reference
Triage system	1.68	1.90	0.38	–2.05 to 5.41
Interaction: system (routine) X phase
Baseline (routine)	Reference
Phase 1	6.58	1.40	< 0.001a	3.83 to 9.33
Phase 2	0.25	1.60	0.99	–3.11 to 3.16
Phase 3	4.80	1.79	0.007a	1.30 to 8.30
Interaction: system (triage) X phase
Baseline (triage)	Reference
Phase 1	0.94	1.41	0.50	–1.81 to 3.70
Phase 2	–0.13	1.40	0.92	–2.90 to 2.63
Phase 3	4.78	1.46	0.001	1.93 to 7.63
MBI modelb
System
Routine care system	Reference
Triage system	0.86	2.35	0.72	–3.75 to 5.47
Interaction: system (routine) X phase
Baseline (routine)	Reference
Phase 1	5.02	1.82	0.006a	1.44 to 8.60
Phase 2	2.47	2.01	0.22	–1.47 to 6.40
Phase 3	8.61	2.24	< 0.001a	4.21 to 13.01
Interaction: system (triage) X phase
Baseline (triage)
Phase 1	–0.87	1.90	0.65	–4.60 to 2.86
Phase 2	1.06	1.87	0.57	–2.60 to 4.72
Phase 3	1.76	1.89	0.35	–1.95 to 5.46
IWS modelb
System
Routine care system	Reference
Triage system	8.06	3.63	0.03a	0.95 to 15.16
Phase
Baseline	Reference
Month 6	8.13	2.50	0.001a	3.23 to 13.05
Month 12	5.06	2.66	0.06	–0.16 to 10.27
Month 18	11.90	2.82	< 0.001a	6.36 to 17.43

Similar results were seen for the other staff measures. Staff ‘burnout’ as measured by the MBI in the triage system appeared to be relatively stable throughout the study timeframe but an exploratory analysis separating the triage ward from the locality wards in that system showed that burnout was highest on the locality wards, even when compared with staff in the routine care system (mean difference 3.76, 95% CI –7.28 to –0.25; p = 0.04). Staff were also more dissatisfied in the triage system than in the routine care system (IWS mean difference 8.06, 95% CI 0.95 to 15.16; p < 0.05) and this increased over the study for both systems, with the most dissatisfaction in the final phase compared with baseline (mean difference 11.90, 95% CI 6.36 to 17.43; p < 0.001). The largest difference appeared to be between the routine care system and the triage locality wards (mean difference 9.03, 95% CI 1.19 to 16.87; p = 0.03) that had the lowest level of satisfaction. No evidence was found for a difference between the routine care system and the triage ward itself (mean difference 5.77, 95% CI –4.82 to 16.37; p = 0.29).

What services do people receive in the two systems, what are their costs and what is the cost-effectiveness?

Despite similar levels of nurse staffing, service users in the triage systems reported about half the number of contacts with nurses than those in the routine care system (Table 38). The cost of meaningful contacts with OT and activities was also found to be statistically significantly lower in the triage system. The total costs of staff contacts were estimated to be £41 lower in the triage system than the routine care system in the unadjusted analysis (95% CI 7 to 75), but this estimate was highly unstable over time and between wards. Figure 15 shows the development of the costs of perceived service use across the study. There was a marked reduction in staff contacts in the routine care system; therefore, after adjusting for potential confounders, the triage system was estimated to have lower costs in the first two phases of the study only. On the other hand, at phase 2, costs were found to be statistically significantly higher in the triage system. Two units (an early intervention ward and one of the acute wards) in the routine care system had notably higher estimated mean costs than the other wards (£294 and £305 vs. £159). Therefore, the use of cluster robust SEs increased the width of CIs considerably, such that differences were no longer statistically significant in the unadjusted comparison. In the adjusted comparison the costs of service use in the triage system remained statistically significantly lower at phase 0 and higher at phase 2 than the routine care system.

TABLE 38 - Service use and costs (WP4)

Among users of the service.

Statistically significantly higher cost in routine care system than triage system at the 5% level.

Statistically significantly higher cost in non-triage wards than triage wards in triage system at the 5% level.

Type of contact	Routine care system (N = 223)			Triage system (N = 226)
				Non-triage wards (n = 147)			Triage ward (n = 79)
	% accessing service	Mean (SD) number of contactsa	Mean (SD) costs	% accessing service	Mean (SD) number of contactsa	Mean (SD) costs	% accessing service	Mean (SD) number of contactsa	Mean (SD) costs
Nurse	64	6 (16)	31 (66)b	60	3 (3)	16 (27)	65	3 (2)	18 (23)
Any doctor	78	2 (1)	123 (135)	86	2 (1)	112 (101)	84	3 (2)	140 (157)
Care co-ordinator	19	2 (2)	9 (29)	34	1 (1)	15 (29)	24	1 (1)	12 (32)
OT	26	2 (2)	7 (22)b	2	1 (1)	1 (5)	1	1 (n/a)	0 (0)
Other care professional	25	2 (2)	17 (45)	39	2 (1)	22 (54)c	9	1 (0)	4 (19)
Activities	71	4 (4)	25 (33)b	71	2 (2)	13 (21)c	37	2 (1)	3 (8)
Total			212 (198)			179 (148)			160 (171)

FIGURE 15.

Cost of perceived service use (WP4). a, Phases in which difference between systems is statistically significant after adjusting for confounders.

Within the triage system, service users on the triage ward were significantly less likely to take part in activities and have contact with other care professionals. The overall cost of perceived service use was marginally lower in the triage ward than other wards in the triage system (mean difference –17, 95% CI –62 to 27), but this difference was not statistically significant and equally unstable across the different phases of the study as in the comparison of the two systems. The cost of meaningful contacts was statistically lower in the triage system at phase 3, but the mean estimate was not consistently lower across the study. In summary, although there was some indication that service use patterns in the two systems might be different, the results of the service use analysis were not robust and these differences may be attributable to other factors. Similarly, our data suggested that there were no substantial differences between the overall cost of perceived care received in the triage ward and long-stay wards in the triage system.

The cost-effectiveness analysis also did not provide clear support for either of the two systems. Using VOICE, there was a high probability that the triage system was more cost-effective (Figure 16) at any willingness to pay and that it would lead to both a decrease in costs and improvement in service user satisfaction (Figure 17). However, the aforementioned lack of robustness in mean cost estimates over the time horizon of the study should be kept in mind. In addition, in the case of VOTE, the analysis suggested that the triage system would probably lead to a deterioration in staff perception (67%) (Figure 18). Therefore, only if little value was placed on this outcome measure and it was the only outcome measure would the triage system be cost-effective (Figure 19).

FIGURE 16.

Cost-effectiveness plane using VOICE with 95% confidence ellipse (WP4).

FIGURE 17.

Cost-effectiveness acceptability curve using VOICE (WP4).

FIGURE 18.

Cost-effectiveness acceptability curve using VOTE (WP4).

FIGURE 19.

Cost-effectiveness plane using VOTE with 95% confidence ellipse (WP4).

Strengths and limitations

Triage systems are one attempt to improve the efficiency and efficacy of acute inpatient psychiatric services and the current study is, to our knowledge, the first attempt at a rigorous evaluation of the approach. Our data included a comprehensive account of all admissions and discharges across two similar geographical catchment areas and linked this with a contemporaneous assessment of service user and staff perceptions of the quality of inpatient care in both systems. A limitation of our study is that it describes just one triage system in one inner city location provided by one NHS trust at one period in history. With these limitations in mind, the results of our investigation suggest that although a triage ward does achieve a rapid turnover of acute cases, this has little effect on the overall length of stay across the system and, in our data at least, a triage system cannot be said to provide a clear advantage over traditional care. There are three possible explanations for our observations. First, and most plausible, is the continuing pressure to improve efficiency in routine care that has made traditional catchment area services more like ‘triage’ than they might have been had we conducted the study 10 years ago when the triage system was first promoted. Second, it may be that the major contribution to length of stay has less to do with the vigour or success of crisis management and more to do with delayed discharge at the ‘other end’ of the system, with substantial numbers of service users remaining in hospital because of failures in the delivery of community and social care including access to adequately supported accommodation. Finally, there is the possibility that triage wards encourage admissions of some service users who would not be hospitalised in a routine care service and that our service user data were insufficiently sensitive to pick this up.

Although it could be expected that service users might express a preference for one system over the other for a variety of reasons, we did not find this to be the case. Even when accounting for other variables (e.g. socioeconomic status) that are associated with differences in perceptions, there is no significant difference in perceptions of care by service users being treated in the two systems. Even when looking at the triage ward on its own, service users did not view it differently from the other wards. The only change in service users’ perceptions and satisfaction was a deterioration in these measures over time that occurred in both systems and coincided with a move to shorter lengths of stay and a considerable reduction in the budgets of both systems, including a ward closure in the routine care system.

The perceptions and burnout of nursing staff did not differ between the two systems, although both got worse over time, which mirrored service user perceptions. Although we suggested that working on a high-pressure triage ward would increase burnout and dissatisfaction, we did not support this hypothesis. In fact, when looking at the wards themselves, a difference did emerge. Burnout scores were highest on the triage system’s locality wards compared with the triage ward itself and the routine care wards, and the triage system had less satisfied nurses than routine care. When teased apart into types of ward, nurses on the locality wards in the triage system had the lowest satisfaction. This finding can be explained as the consequence of the triage ward system, as the service users with the more persistent and challenging behaviours tend to concentrate over time on the locality wards, particularly if community residential alternatives are slow coming forward or not available. As noted in our introduction, staff in acute wards often report that as many as one-third of the service users have delayed discharges because of failures in ‘social’ care, and the protracted stay contributes to both staff and service user dissatisfaction. This may be compounded in a triage system as the triage ward is perceived not just as the innovation attracting interest and attention but also as a setting in which there is a steady turnover of service users with none of the problems of long-stay service users.

In terms of the costs, the analyses suggest that the cost of meaningful staff contacts might be lower in the triage system, however, this was not significant. Similarly, there was no clear evidence for differences between the triage ward and long-stay wards in the triage system in terms of activity costs.

A striking and unexpected finding in the study was that both service users and staff reported a worsening in satisfaction with inpatient services across the period of our research regardless of whether or not they were managed in triage or routine care. Sadly, these data chime with other reports from elsewhere in the NHS that suggest a more pervasive influence on staff morale and patient experience working across the wider organisation in recent years. 18,170 They are also surely not the only data to demonstrate this decline in qualitative experience coinciding with a period of budget tightening and system pressure to increase efficiency that was reflected in our data by a universal reduction in length of hospitalisation and continuing very high levels of occupancy on the wards.

This issue was in the news as the then Deputy Prime Minister, Nick Clegg, made a speech on 20 January 2014,185 alongside a publication by the Department of Health,3 promising action to direct attention at improving the quality of care in our hospitals that should certainly improve the service user and carer experience. At the same time, the Guardian4 also ran a feature describing the harmful effects of bed closures on inpatient services from the point of view of a former psychiatric nurse revisiting his previous place of employment and finding it much deteriorated from a decade ago.

Clinical implications

Although the results of the study do not suggest any benefit of triage system over routine care, it would be wrong to conclude that the approach does not have some merit or may not be worthy of further study. Our data show that, when considered in isolation, the triage ward is able to discharge a greater proportion of hospitalised service users in the 7- to 10-day period than is achieved by standard care and there is no apparent increase in re-admission of these patients as was thought likely. Furthermore, contrary to expectations, staff in the triage ward were no more likely to report burnout than staff in routine care. What is clear is that there may be consequences for staff morale and satisfaction on the remaining wards in such a care pathway and this may require special attention.

Implications for future research

As noted in the section Strengths and limitations, this is a study of one triage system in one area of the inner city in London. This was the only service available at the time. Since then, triage systems have been adopted in several areas and are also contained in our own data in WP3. Future research would benefit from the inclusion of more of these new sites and perhaps a more detailed appraisal of the clinical characteristics of service users as there are anecdotal concerns that triage might encourage hospitalisation of service users who would otherwise be managed as effectively by crisis HTTs.

In conclusion, our study suggests that introducing triage wards is not enough to address the wider challenge of shortening the length of hospitalisation, as this is also determined by the availability of social care in the community, including access to appropriate housing and community support, much of the provision of which lies outside the control of the NHS.

Conclusion

The study demonstrated that, for an inner city acute service, triage wards confer no specific benefit to length of stay and they are not preferred by either staff or service users. Contrary to our hypothesis, triage ward staff did not exhibit more detrimental effects than staff on other wards. There is also some limited evidence that nurses on wards that are part of the triage pathway experience lower morale and less satisfaction with their role. 184

Service user involvement in the governance structure

Service users were members of the Steering Committee and the Data Monitoring and Ethics Committee. However, their attendance was poor and although each meeting did have service user involvement, this could have been improved. We will, in the future, encourage attendance through a more proactive approach and organise meetings at times more suitable to service users (e.g. evenings or weekends).

Design of the PERCEIVE programme

The programme had a coapplicant who is a service user researcher (DR) and has experience of psychiatric inpatient care including of detention and compulsory treatment. However, it should not be thought that what is being brought to the table is some idiosyncratic ‘personal’ experience. Having used inpatient services for four decades and also having a research career, DR has both conceptualised her own experience in terms of the research literature and conceptualised the research literature in terms of her experience and, crucially, others who share it. Her understanding is scientifically and socially shaped.

DR led WP1 (LIAISE), which consisted of developing outcome measures from the perspectives of service users and nurses. To do this, LIAISE utilised a model which DR has used in developing patient-generated PROMs since 200717 and she also contributed to the design of the other three WPs in terms of the research priorities of service users and the kinds of interventions that they would find acceptable and helpful. She helped in the advancement of the conceptualisation of DOORWAYS (WP3) as not just an increase in activity and psychological therapy but a whole-ward enhancing intervention that might be expected to lead to a better patient experience. She also supplied a dose of common sense in terms of the difficulties that could be expected and resistance to organisational change in the NHS, particularly in mental health services.

Service user involvement in measure development

Work package 1 developed a measure from the service user perspective. It was a user-led piece of work and the measure is called VOICE. It also set out to develop a measure from the nurse perspective (VOTE). These measures were not measures of ‘satisfaction’ as they are fraught with difficulties,104 but aimed to tap directly into experiences and perceptions of acute care. For the service user measure, we employed a service user researcher. The first phase followed a participatory approach in that both participants and researchers had experience of service use. It began with four focus groups made up of people who had been inpatients in the local trust in the previous 2 years, with one group made up exclusively of people who had been detained. However, in the event, all groups included people who had been detained. The four focus groups each met twice and only one person across all four groups did not return for the second meeting. This indicates that they were very engaged with the process and it might be argued that this was because it was user led. The details of the rest of the measure generation can be found in Chapter 2. One further point worth noting is that the measure was subject to full psychometric testing and was found to be robust. This is interesting as the population on whom it was tested was an inpatient population and standard textbooks suggest that the ‘subjects’ for psychometric testing should be ‘cognitively intact’. Many of our participants had diagnoses of psychosis which might be thought to compromise them in this task and so, again, it may be the fact of being user generated that makes the measure amenable even to people who have poor mental health because it is about things that really matter to them at that moment.

In order to further assess the impact of service user involvement on research, our programme was included as part of a study carried out by the NIHR Mental Health Research Network. The service user researcher from the LIAISE WP was interviewed by an independent evaluator for studies adopted by the Mental Health Research Network (MHRN). She had this to say:

There’s a huge amount of benefit to employing service user researchers. It’s very valuable to be able to say to service users this is who I am. When you’re running a group or an interview, it breaks down those traditional barriers. Imagine if you had a psychiatrist running a focus group asking people about issues on the ward – how open are they going to be about that? There is something about a shared experience that makes people feel more comfortable and more open and honest in what they say. I remember when I was talking to one participant she said ‘I feel like I can tell you everything, you understand me and I can really talk to you’. So I think it makes a massive difference.

Once VOICE was complete, it was used in WP3 (DOORWAYS), the trial of activities and psychological interventions, and WP4 (BETTER PATHWAYS), the triage ward study. The service user researcher collected data and there were other service users collecting data as well. The recruitment rates were no different between service user researchers and the rest of the recruitment team – all were uniformly high.

How does VOICE perform?

In the battery of measures for service users to complete, there was also a conventionally derived satisfaction one. In WP4 (BETTER PATHWAYS), VOICE was sensitive to differences in activities that participants were engaged in but the conventionally derived one was not. In WP3 (DOORWAYS), VOICE did not find that perceptions improved overall post interventions, which was to be expected as the number of activities taking place did not increase to a significant level. However, it was able to detect differences in those who were compulsorily detained.

Extending service user involvement to other stakeholders

INVOLVE, the programme set up by NIHR to promote patient and public involvement in research, states that mental health professionals do not count as the ‘public’ for involvement purposes. However, we did extend our model of measure development to nursing staff. We proceeded in exactly the same way as for VOICE to produce a measure named VOTE. In this case, we recruited nurse researchers to work with focus groups and expert panels of nursing staff and then to collect data on the wards. The participants in the groups and panels were current staff. We also employed a nurse researcher as part of the team of researchers collecting data for WPs 3 and 4.

CITRINE measure development

For the development of the measure of patient-level costs of inpatient care, CITRINE, we also included both service users and nurses. Service user researchers provided advice on the whole project and also collected data for reliability and validation exercises. In the measure development, staff members were interviewed about a variety of issues related to activities of nursing staff in relation to inpatients. Focus groups in the form of ward-based community meetings were held with service users in the initial measure development and to discuss the presentation of the self-report measure. Following advice from service users, the final measure was adapted in presentation to ensure that it would be suitable for as many people as possible by providing some support for completion (see Chapter 3 for details).

Service user involvement in data collection

In addition to the close involvement of stakeholders in all measure developments, service user researchers were also part of the research teams responsible for data collection in all other areas of the programme grant. These staff members were recruited on the same basis as all members of the team (for their professionalism and expertise as researchers) but also required the added expertise of experience as a service user.

It is difficult to assess their impact on the team, but a number of issues did arise for those service users who had received inpatient care. In particular, some of the trauma associated with their own admission reasserted itself for them. This was dealt with through close supervision that also included a senior service user researcher who also had experience of admission as well as a senior clinical psychologist.

Although service user involvement was seen by the research team as a benefit, there was resistance by some ward staff who were concerned that service users were able to access case notes (with participant consent). However, the professionalism of our staff and close supervision of the team was able to reduce the distress to our staff and overcome the ward staff resistance.

Service user impact on analyses

Service users were involved in the development of all the analysis plans and were especially involved in the analyses in WP1, in which they specifically discussed with statisticians the best approach to the factor analyses and the assessment of the measure psychometrics. Service users were particularly important in helping to understand the statistical output and often were able to fall back on the qualitative data we had collected as well as their own experience to ensure that the team understood the meaning of tables and also to lead the secondary or exploratory analyses by providing potential hypotheses to test. A final set of focus groups were carried out, facilitated by a service user researcher, with the aim of understanding if the inputs to WP3 had been helpful and if there were any other issues that needed consideration in the analysis or follow-up of the programme grant.

Service user impact on dissemination

In keeping with our ethos, all papers had at least one service user author. Service user researchers also presented data from the programme at local, national and international research meetings. During the programme, we kept stakeholders involved in our programme by updating them on the progress through newsletters for which service users contributed and acted as editors. A further round of dissemination is planned when the current programme is completed.

Interim feedback has also been delivered and is planned to be delivered to service managers in order to ensure that the programme provides useful local data for service planning. The first of these meetings was with the Medical Director, Clinical Service Directors and Clinician Lead for the inpatient services to feedback the data from WP4. This feedback also allowed these stakeholders to suggest new analyses for future data exploration which may be of use locally. We have also planned similar exercises with service managers and ward managers as well as local service users.

Service user impact on protocol developments (add-on studies)

During the process of carrying out the programme grant protocol and as we analysed its results, other issues emerged. These became the focus of add-on studies.

Clinical and organisational implications for mental health inpatient units

This programme demonstrates the valuable contribution that stakeholder research makes. Our consistently high recruitment rates demonstrate that people are eager to help create evidence for good practice. In light of the Francis report,188 trusts need to increasingly focus on stakeholder experience as a measure of service delivery.

Our study found that the higher the numbers of meaningful contacts that service users had with nurses, the better were their perceptions of services. So, despite the negative media presentation of services, including inpatient mental health services, those receiving such services appreciate quality communication with nurses. We also discovered that regardless of how acutely ill a person may be they are still able and, in fact desire, to take part in therapeutic activities. This is in opposition to the general concerns that staff had about whether or not service users in such acute stages of mental ill health were able to engage in therapeutic activities. These findings suggest that NHS leaders and service directors need to consider support structures for staff to create an ethos on wards where therapeutic activities and interactions are of equal importance to the completion of paperwork and the management of crises.

Increasing activities did not lead to a worsening of staff perceptions of services. This is clearly important as the extra burdens may have affected the quality of interactions with inpatients. This was also a time during which services across both trusts were being cut and our results are perhaps not surprising. As noted above, this finding demonstrates the importance of service director and manager support to ensure that wards are staffed, organised and run adequately not only for the benefit of the services users but also for the nurses themselves. This could also contribute to greater staff retention, which is an issue across the NHS. Nurses often fed back to our team that they would like to be able to run more activities and engage more with service users as those are the things that motivate one to choose a career in psychiatric nursing. The economic analyses revealed that there was a modest cost increase following the DOORWAYS intervention. This is not an expenditure increase but rather a redistribution of already allocated resources towards more direct care. There was limited suggestion of cost-effectiveness of DOORWAYs, but only if a high value is placed on a unit improvement in the VOICE. In terms of staff perceptions, the intervention did not appear cost-effective.

Our study did not find any clear advantage of a triage system over that of routine care. We found that the introduction of a triage system did not result in a reduction in the length of stay, nor were perceptions of the wards very different between the systems. However, it did not lead to an increase in re-admissions, as feared, and costs were not significantly different from non-triage services. On its own, the triage ward functions as it should (discharging people rapidly) and, as expected, the nurses did not report more burnout. What was unexpected was that staff on the locality wards are finding it very challenging. In the locality wards, nurses reported the most burnout and this needs further exploration. The triage system does not work in isolation and the effects on locality wards especially on what can improve staff moral would be a possible next step in its evaluation.

Setting

The study took place in an inner city NHS mental health trust. Four 18-bed adult acute wards participated. Data collection took place in April–May 2012 (measure development), June–August 2012 and July–August 2013 (pilot study). Inclusion criteria for service user participants were that they had been present on the ward for 3 days prior to data collection and were able to provide informed consent. Inclusion criteria for staff were that they had been working on the ward for at least 4 weeks. Demographic data were collected by self-report from both groups, while clinical data from the service users were collected from NHS records.

Measure generation

Questionnaires were produced through a participatory methodology. 17 Two service user researchers reviewed the literature and produced topic guides which formed the basis of interviews with staff and service users on the participating wards. The interviews were audio-taped, transcribed and thematically analysed by the service user researchers using NVivo 9 (QSR International, Warrington, UK). Key themes identified were then used to develop questionnaire items.

Analysis

The internal reliability of the measure was assessed using Cronbach’s alpha83 with data from the questionnaire phase of the study. Test–retest reliability was carried out with participants who completed the measures on two occasions. Test–retest reliability was estimated for each item using weighted Kappa (quadratic weighting to emphasise the importance of minor differences between time points) and alpha statistics. Based on previous research, we expected that service users who were involuntarily admitted and those from BME groups would show more negative perceptions of the ward environment.

We assessed any differences with one-way ANOVA on views on the ward design.

Analyses were carried out using Stata 12.1 and Statistical Product and Service Solutions (SPSS) version 20 (IBM Corporation, Armonk, NY, USA).

Results

Measure generation phase

Inclusion criteria for our measure generation sample were that participants had been admitted to an acute ward, a mother and baby unit or were deemed to be in an acute episode by their community treatment team and were being treated at home. Furthermore, they were mothers with a child aged < 1 year, or pregnant at the time. Finally, they were able to provide informed consent.

VOICE itself was used as a basis for the topic guide for the semistructured interview, along with input from the women’s mental health team, which also included service user researchers and advisors. Interviews were carried out because focus groups were not feasible. The material from the interviews was audio-taped and subject to thematic analysis.

Quality assurance phase

Inclusion criteria for the feedback group were that they had used perinatal mental health services in the previous 5 years and they were willing to provide informed consent.

The draft of the questionnaire was taken to the feedback group and it was also presented to the women’s mental health team and feedback was sought.

Measure generation phase

Sixteen women agreed to be interviewed about their perceptions of their care and needs. Their average age was 34 years (SD 3.6 years) and they were evenly split between having a white or BME background. Two of the women (12%) were pregnant at the time of the interview. Four women were recruited from a standard acute inpatient ward (25%) and the remaining individuals were on mother and baby units. Half of participants had in the past experienced other types of care. No one who was being treated in the community was recruited.

Seven participants were diagnosed with depression (44%). Three individuals (19%) had multiple diagnoses, two (12%) had a form of psychosis, one (6%) had a diagnosis of bipolar disorder and the remaining three had no diagnoses.

Measure review and improvement phase

Six participants agreed to feed back on the draft questions. The mean age was 37 years (SD 9.4 years) and all were white. Three reported they had had depression, two reported bipolar and the remaining one did not wish to disclose their diagnosis. All participants had multiple types of care and half of them all three types.

One of the key things raised was the lack of questions regarding medication side effects. It was suggested we add a question despite it not coming up in the interviews and the women’s health team agreed with this. They also helped whittle down those questions which were too similar to others and chose the best one. Furthermore, they helped with the phrasing of certain questions.

We also discussed the draft measure with the women’s health research team as it was designed to be eventually used by them. Originally conceptualised as consisting of three separate measures for each type of service (mother and baby unit, Acute, Community) they decided that we should try to phrase it in a way that could apply to all settings. After much deliberation, we made only one question optional (which was related to breastfeeding, which not all women may be doing). We currently have a measure of 27 items.

Study design

This study was designed to develop and test this self report measure. The measure was generated through a process of stakeholder (staff) involvement. 19–21 This approach directly involved nursing staff in order to ensure measures that captured an accurate picture of an acute care ward. The process of measure development used for VOCALISE (see Appendix 10 for full measure) was an adaption of that used for VOTE.

Mapping out the dimensions of inpatient care

In this initial phase, stakeholder groups were held to map out the dimensions of inpatient care as a topic guide. In developing VOCALISE, an interview schedule was refined from the literature and the focus group data as part of a consultation exercise between two nurses, one who was a senior nurse from the trust under study, an expert in qualitative methods/service user researcher and a clinical psychologist.

Measure generation

Psychometric analysis

The final measure was intended as a self-report tool so studies of feasibility and acceptability were conducted to evaluate the burden of administering and completing the measure. In the feasibility study, 40 participants completed the measure including two additional questions assessing whether the measure was easy to complete and understand. Acceptability was assessed using two additional items collected from 115 participants. Test–retest reliability was assessed with participants who completed the measures on two occasions with an interval of 6–10 days. Internal consistency was assessed using Cronbach’s alpha.

Whether or not the new measures truly reflected the experiences of those providing care in the acute inpatient services (face validity) and covered the full spectrum of views (content validity) were explored as a result of the participatory methodology during the instrument development phase. We hypothesised that staff with positive perceptions of barriers to change would also have high levels of job satisfaction. 199 This was tested using two groups: staff with positive perceptions of job satisfaction and staff with negative perceptions of job satisfaction.

Exploratory analyses

Mapping out the dimensions of inpatient care

As the concept of ‘perceptions of barriers to change’ is abstract, the reference group recommended that the interviewee be asked to consider a scenario in practice in which a significant change to clinical practice had occurred. A topic guide for interviews was created.

Measure generation

Psychometric assessment

Design and procedure

This study is a cluster randomised single-blind controlled trial with crossover design. Four forensic inpatient wards took part with two wards randomised to receive SCIT and two wards randomised to TAU. Following the intervention phase, those randomised to TAU were offered SCIT. All participants were assessed twice: at intake and after the intervention.

Participants

Participants were recruited from adult male forensic psychiatric wards in collaboration with a single inner city London NHS trust. Inclusion criteria were: aged 18–65 years with sufficient English to consent to research, complete assessments as well as the intervention.

Intervention

Therapy consisted of a 16-session group facilitated twice a week with a 45-minute session. Emotion perception was the focus of the first six sessions and the following seven sessions were assigned to ToM, attributional style and underlying processes such as jumping to conclusions and tolerating ambiguity. Integration, which focuses on generalising skills to participants’ real-life social situations, was covered in the final two sessions. All sessions end with the setting of homework that is reviewed at the beginning of the following session.

Measures

Analysis

Results

Discussion

The modified SCIT intervention seems feasible with 16 out of 21 randomised to SCIT completing at least 12 out of the 16 sessions. Among completers, there was an attendance rate of 90%, which was similar to a previous study implementing SCIT within an inpatient forensic setting. 201 The intervention drop-out rate of 24% is similar to previous community-based SCIT groups204 but is high in comparison with previous inpatient SCIT groups. 201 The difference may be because of the differing recruitment processes (i.e. Combs et al. 201 recruited from a pool of inpatients that had already shown an interest in attending whereas, in the current study, participants were recruited from a pool of eligible inpatients with no prior indication of interest).

We found an improvement in one measure – emotion recognition in the SCIT group compared with TAU, which replicates previous findings. 201,204 Perceiving and understanding emotions is the primary focus of the first six sessions of SCIT and this skill is revisited throughout the programme, which might have produced the positive effect in this domain. A change in ToM or attribution style may require longer therapy and more exposure to social situations. In previous research, there have been mixed findings for ToM, with some SCIT studies showing improvements204 and others showing none. 211 The non-significant finding in attribution bias might have been influenced by the already low AIHQ ratings at baseline which may be a consequence of the therapeutic focus of these forensic wards.

The therapists identified some challenges with running the groups in this setting, mainly regarding practicalities, staffing arrangements and participant engagement. However, ward staff were enthusiastic about SCIT and after discussing issues were motivated to further support the intervention by being flexible with their activity timetable, prioritising the group over other ward tasks and in encouraging participants to engage and attend. A recent independent study evaluating the feasibility of a 20 session SCIT group within a community mental health setting experienced similar challenges in implementation. 203

The positive views of service users and the subjective feedback showed that participants not only found SCIT helpful in reaching these personal goals, but also that it was generally enjoyable and beneficial, and the sort of group they would like to be offered again.

In conclusion, this study has replicated previous research in a new setting with an adapted programme that may be more suitable to an inpatient environment. It has demonstrated that SCIT is a feasible and acceptable group-based intervention for forensic inpatients. There are promising indications for the effectiveness of SCIT in improving emotion perception in this setting, which is an important domain of social cognition210 (see Table 42).

Work package 3 DOORWAYS: service user analysis

Work package 3 DOORWAYS: staff analysis

Work Package 4: BETTER PATHWAYS Service User analysis

Work Package 4: BETTER PATHWAYS staff analysis