Notes
Article history
The research reported in this issue of the journal was funded by PGfAR as project number RP-PG-0707-10093. The contractual start date was in January 2009. The final report began editorial review in July 2015 and was accepted for publication in July 2016. As the funder, the PGfAR programme agreed the research questions and study designs in advance with the investigators. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The PGfAR editors and production house have tried to ensure the accuracy of the authors’ report and would like to thank the reviewers for their constructive comments on the final report document. However, they do not accept liability for damages or losses arising from material published in this report.
Declared competing interests of authors
none
Permissions
Copyright statement
© Queen’s Printer and Controller of HMSO 2017. This work was produced by Killaspy et al. under the terms of a commissioning contract issued by the Secretary of State for Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science Park, Southampton SO16 7NS, UK.
Chapter 1 Background to the REAL research programme
This programme of research focuses on one of the most socially excluded groups in society: people with longer-term mental health problems whose needs are such that they require inpatient rehabilitation. The majority of this group have a diagnosis of schizophrenia. 1 The National Service Framework for Mental Health2 implemented specialist community mental health teams in England (assertive outreach, early intervention and crisis resolution services) that have reduced reliance on inpatient services. 3 However, a proportion of users of these and other mental health services have such complex problems that they continue to require lengthy hospital admission. 4,5 It has been estimated that, at any time, up to 10% of people with schizophrenia are in receipt of inpatient rehabilitation with the aim of recovering adequate social function to live outside hospital. 6
Although there is good evidence for specific interventions [such as medications, cognitive–behavioural therapy (CBT) and family psychoeducation] that can improve outcomes for people with a diagnosis of schizophrenia,7 these can only be delivered in well-resourced services and to individuals able to engage with them. Most people are referred for inpatient rehabilitation after the National Institute for Health and Care Excellence (NICE) guideline algorithm for the treatment of schizophrenia has been exhausted. 7 Many people with severe and enduring mental health problems, such as schizophrenia, experience ongoing active symptoms of illness and impairments in cognition and conation, social stigma and the secondary handicaps consequent on the illness. By definition, those who are referred for rehabilitation are those whose problems are of such complexity or severity that they have not been able to be discharged home following an acute admission. These problems include treatment resistance (non-response to first-line medications), which occurs in up to 30% of people affected;8 cognitive impairment (most commonly affecting executive function and verbal memory) and pervasive negative symptoms such as apathy, amotivation and blunted affect;9–11 and coexisting problems such as substance misuse, premorbid learning disability and developmental disorders, such as those on the autism spectrum. 1 These kinds of complex problems contribute to major impairments in social and everyday functioning and challenging behaviours that impede recovery and increase the risk of adverse outcomes. 6
Although the National Service Framework for Mental Health brought major investment in specialist community services across England, there appeared to be a simultaneous disinvestment in rehabilitation services. 12 At the same time, there was a rapid growth in independent sector provision for people with complex mental health problems who required longer-term care in hospital, nursing and residential care home settings. This phenomenon has also been noted elsewhere in Europe13 and has been critically referred to as a ‘virtual asylum’. 14 This rise in so-called ‘out-of-area treatments’ (OATs) has been of great concern, as these facilities are often geographically displaced from the service users’ area of origin, leading to social dislocation, and criticisms of the quality of care in some have been made with regard to the lack of implementation of statutory Care Programme Approach processes and institutional cultures that lack a rehabilitative ethos. 15 In addition, the inadequacy of systems for monitoring the quality of OATs and whether or not individuals have an ongoing need for the level of support they provide has been highlighted. 16 In addition to these concerns, OATs are around 65% more expensive than locally provided NHS rehabilitation services. 17
As well as the significant clinical challenges this service user group poses for professionals, the care of this group constitutes a major resource pressure for the NHS and social services. Depending on what is included in the estimate, the costs amount to 25–50% of the total mental health budget. 18 The identification of approaches and interventions that can reduce the need for inpatient care, even by a small reduction in length of stay, will have a very large impact on the mental health budget. Therefore, understanding which approaches are best able to promote service users’ progress towards greater independence and successful community discharge is highly clinically relevant and potentially cost-effective for the NHS. Despite the high levels of need of rehabilitation service users and the high costs of care for this complex group, there is currently very little evidence for effective interventions available to guide mental health rehabilitation practitioners.
A common focus in rehabiliation services is occupational therapy; a previous national telephone survey of rehabiliation services in England found that almost all had at least one full-time occupational therapist (OT). 19 It has long been known that facilitating service users’ activity reduces negative symptoms,20,21 and there is some evidence that this may also lead to improvements in social function through the promotion of motivation and daytime structure. 22–24 Nevertheless, there is evidence that the level of activity of users of acute inpatient services is alarmingly low: in one survey in a London trust, service users spent < 17 minutes per day in an activity other than sleeping, eating or watching television. 25 There are, however, very limited published data on the number and types of activities undertaken in inpatient rehabilitation services. Shimitras et al. 26 found that, although users of these services spent more time sleeping than did community rehabiliation service users, they also spent more time engaged in active leisure activities. Other community samples have also found that people with schizophrenia spend a large amount of time engaged in passive activities such as sleeping and watching television. 27,28 This suggests that although inpatient rehabilitation may improve activity levels, these gains are not sustained following discharge. Inpatient rehabilitation services, therefore, need to work with community services to enable service users to extend and maintain their range of community activities. The government’s Social Exclusion Unit report of 200429 highlighted the role of education, training, volunteering, arts, leisure and sports in promoting community participation for mental health service users. To date, interventions that aim to achieve this have not been evaluated. Although the importance of staff facilitation of service user activities has been highlighted,30 prior to the Rehabilitation Effectiveness for Activities for Life (REAL) programme there had been no randomised controlled trials to test the efficacy of interventions to train and engage staff in promoting service user activities in rehabilitation units.
Our programme of research aimed to increase the evidence base in mental health rehabilitation to inform how best to focus resources on this especially complex patient group, and identify improvements in care that could potentially reduce the length of inpatient rehabilitation admission required and the need for expensive longer-term OAT placements.
Aims and objectives of the REAL research programme
The area of mental health rehabilitation has previously been referred to as ‘an evidence-free zone’. 19 Little is known about the characteristics of users of these services, the components of care delivered, the costs of these services and which approaches are effective. This research programme aimed to address this paucity of evidence through four main objectives:
-
to provide a detailed understanding of the scope of current NHS mental health rehabilitation service provision in England, including the characteristics of their service users and the content and costs of care delivered
-
to develop a staff training intervention to facilitate service users’ activities and improve their social functioning
-
to test the clinical effectiveness and cost-effectiveness of the staff training intervention through a randomised controlled trial
-
to carry out a longitudinal study to identify the components of care associated with better service user outcomes.
The research questions we specified in our original proposal were:
-
What is the current provision of NHS mental health rehabilitation services in England and does it reflect relative levels of socioeconomic deprivation and variation in clinical need?
-
What is the range of quality of these rehabilitation services in England?
-
What are the characteristics of the users of these services and do they vary between services?
-
Do areas with poorer-quality NHS rehabilitation services have higher proportions of service users placed out of area?
-
Is training front-line staff to promote service users’ activities for community living a cost-effective intervention to improve poorer-quality services?
-
Is service user activity associated with better clinical and social outcomes?
-
Is greater quality of the rehabilitation service associated with better outcomes for service users?
Objective 1 and research questions 1–4 and 7 were addressed by phase 1 of our research programme, the national scoping exercise of rehabilitation services (project months 1–24).
Objective 2 was addressed by phase 2 of the programme, the development of a staff training intervention to enhance service users’ activities: the ‘GetREAL’ intervention (project months 18–24).
Objective 3 and research questions 5 and 6 were addressed in phase 3 of the programme, a cluster randomised controlled trial to investigate the clinical effectiveness and cost effectiveness of the ‘GetREAL’ intervention (project months 25–60).
Objective 4 and research question 7 were addressed in phase 4 of the programme, the cohort study (project months 25–60).
Summary of progress and outputs
Please note that all project months refer to a start date of 1 April 2009.
Phase 1 was completed on time (project month 24), despite recruitment exceeding the original estimate of 90 units (133 units were recruited). The analysis of the phase 1 quantitative data is complete, and a paper presenting the findings has been published in the British Journal of Psychiatry. 31 The main analysis of qualitative data is complete, and the findings were fed into phase 2 (the development of the staff training intervention to facilitate service users’ activities). Further analysis is ongoing as part of a doctor of philosophy (PhD) project being carried out by one of the REAL programme researchers (Nicholas Green). The main findings are being prepared for publication.
Phase 2 was completed on time (project month 24). The protoype GetREAL staff training intervention manual was drafted and refined through consultations with representative groups of mental health OTs, service users and staff of mental health rehabilitation units. The GetREAL teams were recruited and started in post in February 2011. The intervention was piloted in two units and further refined in response to piloting. A paper describing the development of the GetREAL intervention has been published in the British Journal of Occupational Therapy. 32
Phase 3 started in April 2011 (project month 25) and was also completed on time. The main results, including the health economic analysis, have been published in Lancet Psychiatry. 33 A total of 40 units participated (randomly selected from the sampling pool of 64 eligible units identified in phase 1: those scoring below the median on our quality assessment tool and that had at least eight beds). Randomisation was carried out by an organisation independent of the research team, and units that agreed to participate were randomised in batches of 10 on an equal basis either to receive the ‘GetREAL’ intervention or to continue to deliver usual care. Staff focus groups and individual qualitative interviews with service users at 10 intervention sites, which were purposively selected to represent a range of location and size, were carried out on average 6 months after the GetREAL team left. The results of the qualitative component have been published in BMC Psychiatry. 34 A further paper describing the role of the OTs in facilitating change in the intervention sites was published in the British Journal of Occupational Therapy. 35
Phase 4 commenced in June 2011 (project month 27). A 3-month delay to the start of phase 4 was agreed by the Programme Management Group (PMG) so that baseline data collection for the phase 3 units could be prioritised. This did not lead to any delay in the completion of phase 4. The main results from phase 4 have been published in BMC Psychiatry. 36
An additional component to the programme was also carried out, which was led by Dr Sarah Cook at Sheffield Hallam University and funded by an underspend in the programme budget that facilitated the agreement of a 12-month no-cost extension to complete this extra work. This component commenced in April 2014 and comprised a ‘realistic evaluation’ of the intervention developed in phase 2 and a cluster randomised controlled trial that assessed the intervention in phase 3. This component has been completed and the results have been published in the Journal of Advanced Nursing37 and BMC Psychiatry. 38
Programme preparation activities
The National Institute for Health Research (NIHR) project contract officially started on 1 January 2009. The recruitment of our research team was then undertaken; the team comprised two full-time researchers (Nicholas Green and Isobel Harrison) and a part-time project manager (Melanie Lean). Ms Lean commenced in post in March 2009, Mr Green in April 2009 and Ms Harrison in May 2009. The NIHR kindly agreed a 3-month no-cost extension to our contract to acknowledge the time-lag between the official contract start date and the date on which the research team came into post.
Application for ethics approval for the programme was made in April 2009 and full approval for the whole programme was received on 9 June 2009 from the South East Essex Research Ethics Committee (reference 09/H1102/45). The programme was adopted by the Mental Health Research Network on 29 September 2008. Clinical studies officers from the Mental Health Research Network gathered information about the rehabilitation services in areas that they covered, as well as relevant contact details for these services. These details were added to those from our previous telephone survey of mental health rehabilitation services in England, carried out in 2004, that identified 90 short-term (i.e. length of stay of up to 12 months) inpatient mental health rehabilitation units. 19 Once our research team was in post, it contacted all NHS mental health trusts in England to confirm whether they had an inpatient or a community mental health rehabilitation unit that accepted patients referred from acute admission wards. The research was conducted in keeping with usual research governance guidance, and local approvals were gained at each site that had eligible mental health rehabiliation unit(s). Along with the chief investigator, members of the research team were involved in preparing the final version of the questionnaires for each phase of the research programme, in assisting with the application for ethical approval and with submissions for local research and development approvals at each site. The researchers were trained by HK in the use of the all study materials and piloted these prior to use.
The ‘realistic evaluation’ was an additional component to the programme that was undertaken to increase our understanding of the resuts of phase 3 of the programme, the cluster randomised controlled trial.
There were three aims:
-
to investigate factors associated with variation in units sustaining the skills and changes in practice during the reinforcing stage of the GetREAL intervention
-
to investigate whether or not uptake of the intervention, particularly during the reinforcing stage, was associated with outcome
-
to recommend modifications to the GetREAL intervention for testing in a future trial.
Application for ethics approval for the realistic evaluation was made in March 2014 and appproval was received on 15 July 2014 from the Faculty Research Ethics Committee at Sheffield Hallam University (reference 2013–4/HWB/HSC/STAFF/19/SHUREC1). Three part-time researchers were employed to carry out this component: Sadiq Bhanbhro (0.8 full-time equivalent), a senior researcher, and Melanie Gee (0.4 full-time equivalent), an information scientist, were both employed from 2 June 2014 to 31 March 2015, and Helen Brian (0.3 full-time equivalent), a research assistant, was employed from 14 July 2014 to 31 March 2015.
Chapter 2 Phase 1: national scoping exercise of rehabilitation services
Method
Phase 1 commenced in July 2009. The researchers contacted each service identified during the preparatory phase to gain consent from the service manager for the unit’s potential participation in phase 1. Services that agreed were then visited by one of the two researchers for up to 1 week in order to gain informed consent from the unit manager, from whom detailed descriptive data about the unit were gathered, and an assessment of its quality was made using the Quality Indicator for Rehabilitative Care (QuIRC).
The QuIRC was developed through an international study conducted between 2007 and 2010, led by HK and funded by the European Commission: the Development of a European Measure of Best Practice in Institutional Care (the DEMoBinc project). 39 It is a standardised, reliable and comprehensive toolkit for the assessment of the quality of inpatient and community-based units for people with longer-term mental health problems. During the project, a systematic review of the international literature and care standards pertaining to longer-term mental health units was carried out,40 along with a three-stage international Delphi exercise involving service users, carers, mental health professionals and advocates in 10 European countries41 to ascertain the ‘critical ingredients’ for recovery for this service user group. The QuIRC was designed to assess these aspects of care. It is completed by the unit manager through a face-to-face interview and takes 45–60 minutes to complete. It comprises 145 questions that are collated to give percentage ratings of seven domains of care (living environment, therapeutic environment, treatments and interventions, self management and autonomy, social inclusion, human rights and recovery-based practice). These ratings include data on the setting (hospital or community) and size (number of beds) of the unit; the average length of stay; the proportion of male and female service users; the proportion of service users detained under the Mental Health Act;42 the degree of disability of service users; the staffing of the unit (staff numbers of different disciplines); staff training in rehabilitative skills (e.g. CBT, family interventions, recovery-based practice and motivational interviewing); the provision of staff supervision; staff turnover, vacancies and disciplinaries; the unit’s provision of evidence-based pharmacological and psychosocial interventions, occupational therapy and the facilitation of community activities (education, employment and leisure); interventions for physical health care and promotion (such as smoking cessation programmes, dietary advice, and access to and support for exercise); the therapeutic milieu; collaborative and individualised care planning; service users’ involvement in their own care and the running of the unit; the protection of service users’ human rights, including their privacy and dignity, access to advocacy; the response to challenging behaviours including the use of de-escalation, control and restraint, and seclusion; and the quality of the built environment. The QuIRC has excellent inter-rater reliability,43 and the ratings gained from the unit manager have been shown to reflect service users’ experiences of care and the degree to which the unit promotes their autonomy. 44
Each service received a detailed report on the quality of its service. These reports were generated by the web-based QuIRC application (www.quirc.eu): data were entered by the project manager and a 12-page PDF report was produced for each unit, which first described the unit and then reported its performance on the seven domains of care as percentages, displayed on a ‘spider web’-style diagram. The mean percentage scores are shown for the unit alongside those of all similar (hospital or community based) mental health rehabilitation units in England. Further details were then given on the aspects of care assessed in each domain so that the staff could reflect on how they could potentially improve their unit’s rating.
Additional data on the diagnostic and risk profile of service users were gathered from unit managers in an anonymised, aggregated form. The managers also gave details of the unit’s annual budget; the unit’s assessment process for new referrals; the number of serious incidents in the preceding 12 months; the provision of local community rehabilitation services; the provision of local supported accommodation; the number of service users discharged to different types of unsupported/supported accommodation in the last 12 months locally and elsewhere; and the involvement of the consultant rehabilitation psychiatrist in funding/placement panels for service users requiring supported accommodation and in reviews of service users placed outside the local area.
A rating of the psychiatric morbidity of the area served by the unit was also made using the Mental Illness Needs Index (MINI)45 relevant to the postcode for each unit. This index provides a rating of psychiatric morbidity based on socioeconomic demographic data from the 2001 census, with a standardised mean of 1.0. A MINI rating of 1.2 represents an area with mental health needs 20% higher than the national average.
Service users who gave informed consent participated in a research interview that took approximately 30 minutes. After collecting sociodemographic data, confirming diagnosis and length of history from the case notes, the researchers interviewed service users to rate their autonomy, quality of life and experiences of care and the therapeutic milieu of the unit. Autonomy was assessed using the Resident Choice Scale (RCS);46 the service user rates the degree to which they have choice over 22 aspects of daily activities and the running of the unit on a four-point scale (‘I have no choice at all about this’, ‘I have very little choice about this’, ‘I can express a choice about this but I do not have the final say’ and ‘I have complete choice about this’). The RCS has a maximum possible score of 88. Quality of life was assessed using the Manchester Short Assessment of Quality of Life (MANSA);47 the service user rates 12 aspects of their life on a scale from 1 (couldn’t be worse) to 7 (couldn’t be better), and a total mean score of between 1 and 7 is generated. Their experiences of care were assessed using the Your Treatment and Care (YTC)48 questionnaire, which has been used in the UK in service user-led assessments of mental health services. The service user rates 25 items related to their care (e.g. ‘I know who my doctor is’) as ‘yes, ‘no’ or ‘don’t know’. The total number of ‘yes’ answers is summed, giving a maximum possible score of 25. Service users’ views on the unit’s therapeutic milieu were assessed using the General Milieu Index (GMI);49 the service user rates their general satisfaction with the unit, with staff and with other residents, and the degree to which they feel the unit facilitates their confidence and abilities, on a scale of 1 to 5 (from ‘not at all’ to ‘very much’), and a total mean score between 5 and 25 is generated. An assessment of service user function was also made by the researcher using the Global Assessment of Functioning (GAF) scale50 to take this into account as a potential mediator between service quality and clinical outcomes. All service user interviews were completed within 1 month of the unit manager interview.
Data analysis
The specific research questions we detailed in our original proposal that would be addressed in phase 1 were:
-
What is the current provision of rehabilitation services in England?
-
Does it reflect relative levels of socioeconomic deprivation and variation in clinical need?
-
What is the range of quality of rehabilitation services in England?
-
What are the characteristics of the users of these services, and do they vary between services?
-
Do areas with poorer-quality rehabilitation services have higher proportions of service users placed out of area?
To address these questions, a subgroup of the PMG (the chief investigator, the study statisticians and Professor King) agreed on the data analysis plan for phase 1 (see Appendix 1) with the following four objectives:
-
to determine the current provision and quality of inpatient mental health rehabiliation units in England
-
to describe the characteristics of users of these services
-
to investigate whether or not unit quality is related to service user characteristics and the psychiatric morbidity of the local area
-
to investigate whether or not service user outcomes are related to the quality of the unit.
The study statisticians, RO and LM, carried out the data analyses using Stata version 11 (StataCorp LP, College Station, TX, USA). Descriptive data reporting on our first two objectives are presented as frequencies and percentages or means and standard deviations (SDs), as appropriate.
With regard to our third objective, multiple linear regression was used to investigate which covariates were associated with unit quality (QuIRC domain scores). Our sample size (133 units) allowed us to estimate up to eight coefficients in each model, using the rule of 15 observations per coefficient estimated to achieve adequate precision. The covariates selected a priori were location of unit (hospital or community; units within hospital grounds were recategorised as community, as previously they were found to be more similar in profile to community-based units than to hospital wards);43 psychiatric morbidity of the area local to the unit; percentage of male service users; mean age of service users; service users’ mean GAF score; and percentage of service users detained involuntarily.
Our fourth objective was to investigate whether or not unit quality (QuIRC domain scores) was associated with service user outcomes, namely autonomy (RCS), quality of life (MANSA), experiences of care (YTC) and therapeutic milieu (GMI). Our sample included 739 service users, with 616 service users having complete data on all variables. Using an intracluster correlation of 0.04, an average cluster size of 14 and the rule of 15, this allowed us to estimate up to 27 coeffcients with adequate precision in each model. As YTC data were left skewed, they were categorised into tertiles and analysed using the proportional odds regression with robust standard errors to account for clustering within unit. The assumption of proportional odds required by the model [that the odds ratio (OR) comparing one level of a particular covariate with another level was constant across all categories of the YTC] was satisfied by the data. Other outcomes were analysed using linear marginal models based on generalised estimating equations. 51 All variables (covariates and outcomes) apart from the MANSA had only a small proportion of missing values.
As part of the secondary analyses, we also used multiple imputation based on chained equations52 to estimate the missing MANSA values. To determine which variables were included in the imputation model, all unit and service user variables were first tested univariably using logistic regression models with robust standard errors, allowing for clustering within centres, to examine whether or not they were significantly associated with missing MANSA data (a binary variable was created by allocating 1 to any service user with a missing MANSA value and 0 otherwise). Three unit-level and two service user variables associated with missing MANSA data were identified. The unit-level variables were all from the QuIRC: access to a psychiatrist, other interventions provided and percentage of patients detained involuntarily. The service user variables were ‘would like more leisure time’ and tertiles of the experiences of care measure (YTC). A number of additional variables were included in the model for consistency, as they had been agreed as variables of interest in the models for the other outcomes and for the analysis of MANSA without imputation. For service user level these were age, sex and GAF score, and for unit level these were MINI score for the area local to the unit, all QuIRC domain scores, a dichotomous variable to indicate whether or not the unit had ≥ 4 domains scoring less than the national median, and location of the unit (hospital or community). To control for clustering of the data within centres in the imputation model, an estimate of the random effects (obtained by fitting a multilevel model) was also added. As with the complete case analysis, the age, sex, GAF score and location of the unit and each QuIRC domain separately (seven models) were included in the marginal model based on generalised estimating equations after multiple imputation.
The results of these analyses are expressed per 10 percentage point change in QuIRC domain score.
Results
Response
All of the 60 NHS mental health trusts in operation in England in 2009 had at least one inpatient (or community-based equivalent) mental health rehabilitation unit. Three trusts declined to participate in the study, two failed to complete local research governance approval within the phase 1 study period and, therefore, could not be included, and three trusts closed their rehabiliation units prior to participation. The response rate was, therefore, 87% (52/60). A total of 133 units were identified, with a median of two units per trust [interquartile range (IQR) 1–2]. These units had a total of 1809 beds, of which 1641 (91%) were occupied. A total of 442 service users occupied beds that were designated respite or continuing care. Of the remaining 1199 service users, 129 (11%) lacked the capacity to be able to give informed consent to participate in a research interview, 20 (2%) lacked adequate English for participation, 108 (9%) were unavailable as they were on planned leave from the unit, 12 (1%) were unavailable as they were absent from the unit without leave, 191 (16%) declined to participate and 739 (62%) were interviewed. The response rates of service users across units ranged from 40% to 100% but were not found to have any association with unit quality (QuIRC domain scores).
Unit characteristics
The unit characteristics are shown in Table 1. The majority of units were based in suburban areas in the community, and they had a mean of 14 beds and a mean of 16 admissions in the previous year. Overall, 124 (93%) units provided single bedrooms only and 85% provided separate women-only and mixed-sex communal areas. Most service users were admitted from acute admission wards or directly from the community, with a small number coming from secure settings. Overall, 42% of units used standardised measures in their assessment process and 89% used them routinely after admission. All units had input from a psychiatrist and all units were staffed by nurses and support workers, but 21 (17%) had no access to a clinical psychologist and 13 (10%) had no access to an OT. The total mean staff per unit was 21 (SD 6), and a mean of 20% (SD 15%) had left in the previous 2 years. The mean staff-to-service user ratio was 1.58 (SD 0.47).
Characteristic (N = 133) | n (%) | ||
---|---|---|---|
Unit location | |||
Inner city | 26 (20) | ||
Suburbs | 96 (72) | ||
Rural area | 11 (8) | ||
Unit type | |||
Hospital ward | 15 (11) | ||
Community based | 79 (59) | ||
Within hospital grounds | 39 (29) | ||
Unit has max. length of stay | 27 (20) | ||
Max. length of stay (years), mean (SD) | 1.7 (0.5) | ||
Staffing (N = 127) | No access | Access outside unit | Works in unit |
Psychiatrist | 0 | 38 (30) | 89 (70) |
Clinical psychologist | 21 (17) | 65 (51) | 41 (32) |
OT | 13 (10) | 21 (17) | 93 (73) |
Nurse | 0 | 0 | 127 (100) |
Support worker | 0 | 0 | 127 (100) |
Social worker | 27 (21) | 93 (73) | 7 (6) |
Volunteer | 67 (53) | 41 (32) | 19 (15) |
Arts therapist | 66 (52) | 53 (42) | 8 (6) |
Ex-service user(s) work in unit | 40 (31) | ||
Ex-service user(s) on payroll (n = 37) | 24 (65) | ||
Years unit open, mean (SD) | 10 (6) | ||
Beds | Mean (SD) | ||
Beds available in the unit | 14 (5) | ||
Beds occupied | 13 (5) | ||
Percentage of beds occupied | 91 (12) | ||
Mean (SD) | Median (IQR) | ||
New admissions last 12 months | 16 (17) | 10 (6–19) | |
From acute wards | 9 (8) | 6 (3–11) | |
From community | 5 (13) | 1 (0–3) | |
From low secure units | 1 (2) | 1 (0–1) | |
From another rehabilitation unit | 1 (2) | 0 (0–2) | |
Unit manager views on service user functioning (N = 132) | Mean (SD) | Median (IQR) | |
% service users able to do most things without assistance | 48 (31) | 41 (25–74) | |
% service users able to do some things without assistance | 38 (29) | 33 (19–50) | |
% service users able to do very little without assistance | 14 (17) | 9 (0–25) | |
% service users who are difficult to engage with | 34 (23) | 29 (19–47) | |
Unit manager estimates of service user interventions | Mean (SD) | Median (IQR) | |
% families receiving psychoeducation (N = 128) | 10 (21) | 0 (0–12) | |
% service users receiving CBT (N = 124) | 14 (27) | 0 (0–18) | |
Hours per day service users spend doing planned activity | 4 (1.6) | 4 (3–5) | |
% service user who regularly participate in activities on unit | 76 (24) | 80 (63–100) | |
% service users who regularly participate in activities in community | 70 (31) | 75 (47–100) | |
Local supported accommodation beds | Mean (SD) | Median (IQR) | |
Nursing homes | 18 (19) | 14 (2–30) | |
Residential care homes | 44 (58) | 22 (1–75) | |
24-hour supported tenancies | 30 (77) | 13 (1–30) | |
< 24-hour supported tenancies | 38 (30) | 40 (10–66) | |
Floating outreach | 37 (83) | 20 (2–36) | |
Discharges in last 12 months | Mean (SD) | Median (IQR) | |
Nursing homes | < 1 (1) | 0 (0–1) | |
Residential care homes | 1 (1) | 1 (0–2) | |
24-hour supported tenancies | 2 (3) | 1 (0–3) | |
< 24-hour supported tenancies | 1 (3) | 0 (0–2) | |
Floating outreach | 1 (2) | 0 (0–1) | |
Other | 4 (8) | 1 (0–3) | |
Out-of-area placement | 1 (1) | 0 (0–1) | |
Service users ready for discharge, awaiting suitable accommodation | 2 (2) | 2 (1–3) | |
QuIRC domain scores (possible range 1–100%; N = 133) | Mean (SD) | ||
Living environment | 73 (10) | ||
Therapeutic environment | 68 (6) | ||
Treatments and interventions | 62 (6) | ||
Self-management and autonomy | 73 (9) | ||
Recovery-based practice | 71 (9) | ||
Social inclusion | 63 (12) | ||
Human rights | 75 (8) |
A mean of 1 (SD 2) member of staff was trained in family psychoeducation and a mean of 1 (SD 2) service user was receiving this intervention in each unit. A mean of 1 (SD 2) member of staff was trained in CBT and 2 (SD 3) service users were receiving it per unit. Most (85%) unit managers reported that their service users usually received < 10 CBT sessions. All units used individualised care plans and 126 (95%) provided individualised programmes of activities. Most (95%) unit managers reported that their service had links with local community sports facilities. A wide range of other community links was also reported: 74% of units had links with local churches or other religious organisations, 61% had links with local entertainment venues such as cinemas, 53% had links with local cafés and 20% had links with other community organisations. In 18% of units, at least one service user was reported to be attending a mainstream employment scheme and a mean of 1 (SD 1) service user was attending a local college. Overall, a mean of 70% (SD 22%) of service users in each unit were prescribed atypical antipsychotic medication. Very few were prescribed more than two antipsychotic medications (25 service users across all units) and a mean of 33% (SD 20%) were prescribed clozapine.
There were few serious incidents (n = 35) or staff disciplinaries (n = 21) in the preceding 12 months across all units. Of those unit managers who answered the question, 30 out of 96 (31%) reported that the consultant psychiatrist was involved in agreeing funding of out-of-area placements, and 58 out of 98 (59%) reported that they were involved in reviewing people placed outside the local area. Half of the unit managers who answered the question reported that the service had a local community rehabilitation team (53/104, 51%).
Service user characteristics
The majority of service users were white men, with a mean age of 40 years, a median 13-year history of contact with mental health services, and four previous admissions. The majority had a diagnosis of schizophrenia or schizoaffective disorder (81%). The median length of the current admission was 18 months, and one-third of service users were currently detained involuntarily. Almost half had a previous history of self-neglect or self-harm, and over half had a history of assault on others. There were very high levels of satisfaction with care (YTC) and the average GAF score suggested moderate levels of symptoms and impairment of social and occupational functioning (Table 2).
Characteristic (N = 739) | n (%) |
---|---|
Age (years), mean (SD) | 40 (13) |
Male | 475 (64) |
White | 595 (81) |
Age at leaving full-time education (years), mean (SD) | 17 (2) |
Employment (N = 739) | |
Employed (paid or voluntary) | 8 (1) |
Unemployed | 679 (92) |
Retired | 32 (4) |
Other | 20 (3) |
Living situation before admission (N = 735) | |
Alone | 134 (18) |
With partner | 22 (3) |
With parents | 79 (11) |
With children aged ≤ 18 years | 5 (1) |
With children aged > 18 years | 9 (1) |
Other | 486 (66)a |
Housing before admission (N = 738) | |
Owner–occupier | 86 (12) |
Rented flat/house | 126 (17) |
Hostel | 33 (4) |
Sheltered | 12 (2) |
Residential home | 14 (2) |
Hospital ward | 459 (62) |
No fixed abode | 8 (1) |
Diagnosis (N = 702) | |
Schizophrenia | 511 (73) |
Schizoaffective disorder | 57 (8) |
Bipolar disorder | 59 (8) |
Other | 75 (11) |
Psychiatric history, median (IQR) | |
Years of contact with mental health services (n = 594) | 13 (6–22) |
Previous admissions (n = 522) | 4 (2–9) |
Length of current admission (months) (n = 586) | 18 (9–46) |
Time in rehab unit (months) (n = 572) | 8 (4–19) |
Mental Health Act status (N = 630) | |
Detained during this admission | 427 (68) |
Currently detained | 203 (32) |
Previous admission to secure unit | |
High secure unit (n = 599) | 40 (7) |
Medium secure unit (n = 600) | 84 (14) |
Low secure unit (n = 601) | 117 (19) |
Prison (n = 600) | 110 (18) |
Risk | |
Assault on others ever (n = 599) | 349 (58) |
Assault in last 2 years (n = 599) | 127 (21) |
Serious assault last 2 years (n = 568) | 43 (8) |
Self-harm ever (n = 604) | 271 (45) |
Self-neglect (n = 604) | 295 (49) |
Social inclusion markers | |
Voted in last general election (n = 736) | 144 (20) |
Has a bank/PO account (n = 735) | 623 (85) |
Has charge of own finances (n = 736) | 534 (73) |
Standardised outcome measures, mean (SD) | |
Autonomy | |
RCS (possible range 22–88) (n = 672) | 61 (7) |
Quality of life | |
MANSA (possible range 1–7) (n = 616) | 4.6 (0.8) |
Experiences of care | |
YTC (possible range 1–25) (n = 711) | 24 (4) |
Therapeutic milieu | |
GMI (possible range 5–25) (n = 720) | 18 (4) |
Social functioning | |
GAF score (possible range 1–100) (n = 739) | 54 (9) |
Factors associated with unit quality
Table 3 shows that the mean age of service users in a unit was associated with scores in five of the seven QuIRC domains, increasing age being associated with decreasing scores. The largest reduction in QuIRC domain score was in the social inclusion domain, where it decreased by 0.37 percentage points for each year of mean age [95% confidence interval (CI) –0.64 to –0.10 percentage points]. The percentage of service users detained involuntarily was also associated with a decrease in four QuIRC domain scores. The largest reduction was in the self-management and autonomy domain, which reduced by 0.12 percentage points for each percentage point increase in those detained (95% CI –0.17 to –0.06 percentage points). The percentage of men per unit was negatively associated with the social inclusion and therapeutic environment domains, having a greater influence on the former, where for each percentage point increase in male service users the social inclusion domain score reduced by 0.11 percentage points (95% CI –0.20 to –0.03 percentage points). For each point increase in MINI, the living environment domain score decreased by 6.78 percentage points (95% CI –11.09 to –2.47 percentage points). Units located in the community had a therapeutic environment score 3.58 percentage points higher (95% CI 0.11 to 7.05 percentage points) than hospital-based units after adjusting for other variables in the model.
Variable | Coefficient (95% CI) |
---|---|
Living environment | |
Unit based in community (vs. hospital) | 4.47 (–1.08 to 10.03) |
Male service users (%) | –0.02 (–0.09 to 0.05) |
Service users’ mean age (years) | –0.12 (–0.33 to 0.09) |
Service users’ mean GAF score (range 1–100) | 0.01 (–0.26 to 0.28) |
Service users detained involuntarily (%) | –0.08 (–0.14 to –0.01) |
Unit location MINI score | –6.78 (–11.09 to –2.47) |
Therapeutic environment | |
Unit in community | 3.58 (0.11 to 7.05) |
Male service users (%) | –0.04 (–0.09 to 0.00) |
Service users’ mean age (years) | –0.27 (–0.40 to –0.14) |
Service users’ mean GAF score | 0.10 (–0.07 to 0.27) |
Service users detained involuntarily (%) | 0.00 (–0.04 to 0.04) |
Unit location MINI score | –0.07 (–2.77 to 2.62) |
Treatments and interventions | |
Unit in community | 2.97 (–1.13 to 7.07) |
Male service users (%) | –0.03 (–0.08 to 0.02) |
Service users’ mean age (years) | –0.19 (–0.34 to –0.03) |
Service users’ mean GAF score | 0.04 (–0.16 to 0.24) |
Service users detained involuntarily (%) | 0.02 (–0.03 to 0.07) |
Unit location MINI score | –0.27 (–3.45 to 2.91) |
Self-management and autonomy | |
Unit in community | 2.15 (–2.45 to 6.75) |
Male service users (%) | –0.02 (–0.08 to 0.04) |
Service users’ mean age (years) | –0.18 (–0.35 to –0.01) |
Service users’ mean GAF score | 0.08 (–0.14 to 0.30) |
Service users detained involuntarily (%) | –0.12 (–0.17 to –0.06) |
Unit location MINI score | –2.82 (–6.38 to 0.75) |
Human rights | |
Unit in community | –0.75 (–5.44 to 3.94) |
Male service users (%) | –0.02 (–0.08 to 0.03) |
Service users’ mean age (years) | –0.08 (–0.26 to 0.10) |
Service users’ mean GAF score | 0.12 (–0.11 to 0.35) |
Service users detained involuntarily (%) | –0.08 (–0.13 to –0.02) |
Unit location MINI score | –1.50 (–5.14 to 2.13) |
Recovery-based practice | |
Unit in community | 3.84 (–1.04 to 8.72) |
Male service users (%) | –0.05 (–0.12 to 0.01) |
Service users’ mean age (years) | –0.25 (–0.44 to –0.07) |
Service users’ mean GAF score | 0.13 (–0.11 to 0.36) |
Service users detained involuntarily (%) | –0.07 (–0.13 to –0.01) |
Unit location MINI score | –0.28 (–4.06 to 3.50) |
Social inclusion | |
Unit in community | 3.09 (–4.01 to 10.19) |
Male service users (%) | –0.11 (–0.20 to –0.03) |
Service users’ mean age (years) | –0.37 (–0.64 to –0.10) |
Service users’ mean GAF score | 0.09 (–0.25 to 0.44) |
Service users detained involuntarily (%) | 0.01 (–0.07 to 0.09) |
Unit location MINI score | –4.21 (–9.72 to 1.29) |
Unit quality and clinical outcomes
Very few service users were discharged from the rehabilitation service to an out-of-area placement (see Table 1), but unit managers were unable to provide data on the number of service users placed out of area by other services in their trust. Data were, therefore, not available for the investigation of the association between unit quality and the use of out-of-area placements. With regard to other service user outcomes, clear associations were found. Most QuIRC domains were positively associated with experiences of care (YTC) (Table 4). For example, a 10 percentage point increase in the treatments and interventions domain score resulted in an odds of 1.56 (95% CI 1.17 to 2.08) for scoring in the highest tertile on the YTC compared with the lower two tertiles. All QuIRC domains were positively associated with autonomy (RCS). The largest of these associations was for the therapeutic environment domain, for which a 10 percentage point increase was associated with an increase in the RCS of 3.43 (95% CI 2.04 to 4.81) points. All QuIRC domains were significantly associated with service users’ ratings of the units’ therapeutic milieu (GMI). Here the QuIRC domain with the strongest influence was therapeutic environment, where a 10 percentage point increase was associated with an increase in GMI of 1.18 (95% CI 0.61 to 1.75 points). The quality-of-life scores (MANSA) appeared to be associated with living environment and self-management and autonomy, but here a 10 percentage point increase was associated with very small increases in MANSA scores. Repeating this analysis using imputed MANSA data gave similar results.
Variable | ORa or coefficientb (95% CI) |
---|---|
Experiences of care (YTC)a | |
Living environment | 1.27 (1.05 to 1.54) |
Therapeutic environment | 1.50 (1.09 to 2.08) |
Treatments and interventions | 1.59 (1.19 to 2.13) |
Self-management and autonomy | 1.43 (1.15 to 1.77) |
Human rights | 1.34 (1.06 to 1.69) |
Recovery-based practice | 1.20 (0.98 to 1.48) |
Social inclusion | 1.34 (1.15 to 1.56) |
Autonomy (RCS)b | |
Living environment | 1.75 (0.83 to 2.67) |
Therapeutic environment | 3.43 (2.04 to 4.82) |
Treatments and interventions | 3.19 (1.98 to 4.41) |
Self-management and autonomy | 2.37 (1.42 to 3.33) |
Human rights | 2.24 (1.16 to 3.32) |
Recovery-based practice | 2.38 (1.50 to 3.25) |
Social inclusion | 2.06 (1.44 to 2.68) |
Therapeutic milieu (GMI)b | |
Living environment | 0.73 (0.33 to 1.12) |
Therapeutic environment | 1.20 (0.62 to 1.77) |
Treatments and interventions | 0.68 (0.10 to 1.26) |
Self-management and autonomy | 1.10 (0.72 to 1.47) |
Human rights | 0.92 (0.45 to 1.38) |
Recovery-based practice | 0.85 (0.47 to 1.22) |
Social inclusion | 0.44 (0.17 to 0.71) |
Quality of life (MANSA)b (complete case) | |
Living environment | 0.09 (0.02 to 0.15) |
Therapeutic environment | 0.02 (–0.13 to 0.17) |
Treatments and interventions | 0.00 (–0.14 to 0.16) |
Self-management and autonomy | 0.09 (0.01 to 0.17) |
Human rights | 0.04 (–0.06 to 0.15) |
Recovery-based practice | 0.03 (–0.05 to 0.12) |
Social inclusion | 0.00 (–0.06 to 0.06) |
Quality of life (MANSA)b (imputed) | |
Living environment | 0.09 (0.01 to 0.17) |
Therapeutic environment | 0.03 (–0.12 to 0.18) |
Treatments and interventions | 0.01 (–0.14 to 0.16) |
Self-management and autonomy | 0.10 (0.02 to 0.18) |
Human rights | 0.06 (–0.05 to 0.17) |
Recovery-based practice | 0.05 (–0.04 to 0.13) |
Social inclusion | 0.00 (–0.07 to 0.06) |
Discussion
The national survey component of phase 1 represents the first in-depth study of NHS mental health rehabilitation units in England. The high participation rate strengthens the generalisability of our findings. However, we acknowledge that there are many rehabilitation units in the independent sector that were not included in this study (owing to the limitations of our resources), and we were also unable to include NHS units that were designated as ‘forensic’, ‘secure’ or ‘locked’ rehabilitation. Our initial estimate of the number of NHS inpatient mental health rehabilitation units likely to be in operation was based on our previous telephone survey of practitioners in this field,19 which suggested that there would be around 90 units. However, during the scoping phase of our national survey, we identified 150 NHS mental health rehabilitation units. We decided to aim to include them all and therefore made the decision that we could not expand our study to include secure or independent sector facilities. We were also aware that the independent sector may be a less stable setting for the longer-term aspects of our programme, being perhaps subject more to ‘market forces’ than the NHS.
We found that all NHS trusts had at least one rehabilitation unit catering for a group with particularly complex needs; the relatively high levels of clozapine prescription and extended length of admission corroborate previous descriptions of the treatment-resistant nature of illness in this service user group. 6 Around half required assistance with some or most activities, and one-third was difficult to engage in activities.
Although the recommended range of supported accommodation needed to help individuals move on to more independent living53 was usually provided, ‘delayed discharges’ affected about 14% of service users, suggesting inadequate provision of local supported accommodation and ‘silting’ of the care pathway. Service users were discharged only rarely from a rehabilitation unit to an out-of-area placement, but the local psychiatrist with responsibility for the rehabilitation service was commonly not represented on the local placement panel. Current guidance emphasises the importance of this involvement in ensuring the appropriate placement of service users in facilities that are tailored to their needs, that opportunities for local treatment and support have been fully explored prior to a placement being made out of area, and that there is ongoing review of an individual’s suitability for local repatriation at the earliest opportunity. 54 Our results suggest that this guidance is inadequately followed.
Units with a higher proportion of older patients, male patients and patients detained involuntarily were of poorer quality, although the degree of association between these service user characteristics and service quality was small. However, the psychiatric morbidity of the local area had a greater impact on service quality than the nature of the users placed there, although it influenced only one quality domain, namely the living environment. As areas with greater deprivation tend to have higher levels of psychiatric morbidity,45 there may be a greater pressure on resources in these areas, with less available for investment in the built environment. Community-based units appeared to fare better than inpatient units on only one domain of quality (therapeutic environment), corroborating previous reports of the less ‘institutional’ culture of non-hospital settings. 40 However, our findings suggest that, in all other respects, the rehabilitation delivered in hospital-based units was of a similar quality to that in community-based units.
Most unit managers reported high levels of participation in activities on and off the unit, despite around one-third of service users being difficult to engage. The high level of individualised activity programmes may have facilitated this. However, very few service users were receiving psychological interventions as recommended by NICE. 7 This is not surprising, as few staff were adequately trained to deliver these. In the current economic context, further investment in clinical psychologists seems unlikely, but a greater focus on training and supervising nurses and other staff to deliver psychological interventions has been shown to be possible in some psychosis services,55 although problems with sustainability have also been identified. 56,57 This might include so-called ‘low-intensity’ psychological interventions that, although lacking a strong evidence base, may be easier for service users with complex mental health problems to engage with and assist with addressing some of the comorbidities and other issues that impede progress towards successful community discharge (e.g. anxiety management, relapse prevention, motivational interviewing).
There were strong indications that the quality of care provided by units was associated with service users’ autonomy, experiences of care and perception of their therapeutic environment, although it was not associated with quality of life. The cross-sectional nature of our data means that we cannot be sure of the direction of these associations, but they are encouraging. Although it may be difficult to prevent the destructive impact of chronic psychotic illness on quality of life, it seems that rehabilitation services are providing a positive experience of care that facilitates individuals’ autonomy. In addition, the quality-of-life measure we used considers social and community aspects of life that may simply be outside the current experiences of this service user group, who, because of the severity of their symptoms and impairments, have been admitted to an inpatient rehabilitation unit. Our cross-sectional data could not assess the longitudinal associations between the quality of rehabilitation services and service users’ quality of life, but these were explored in later phases of the REAL research programme.
Our findings represent the first comprehensive description of NHS mental health rehabilitation services in England and provide national ‘quality benchmarking’ data for these services. We found a positive association between quality of care and clinical outcomes in these services, suggesting that interventions that improve the quality of care provided are likely to promote service users’ autonomy and experiences of care. As the promotion of autonomy is the main goal of rehabilitation,19,53 ongoing investment in these services is needed to continue to deliver high-quality care that promotes service users’ recovery and abilities so that they can leave hospital and live as independently as possible in the community. Additional investment in the local supported accommodation pathway is, therefore, also needed to ensure that service users have an appropriate place to move on to when they are ready to leave the rehabilitation unit.
Finally, this phase of the REAL study has also shown that collection of comprehensive service quality assessment data on a national scale is possible when a tailored, reliable and well-validated tool is available. The QuIRC has been incorporated into the Royal College of Psychiatrists’ Accreditation for Inpatient Mental Health Services, which ensures that the ongoing standardised assessment of the quality of these services can continue.
Phase 1 health economic component
Phase 1 also included an investigation of the costs of care for inpatient mental health rehabilitation service users.
Staffing levels and staff contacts
Data on staffing levels in rehabilitation units were obtained from a senior staff member by the researchers during phase 1 data collection, as described previously. Information on whether or not service users had access to staff of different disciplines, as well as on the number of full-time equivalent staff members of each discipline employed in the unit, was also collected. Most commonly, these were psychiatrists, nurses, OTs and support workers. Therefore, the health economic component of phase 1 focused on these four ‘core’ disciplines. The costs were calculated based on data on recent salary and on cost estimates for health and social care services in England. 58 As part of the research interviews carried out with service users in phase 1, information on health and social care service use over the previous 4 weeks was collected using an adapted version of the Client Service Receipt Inventory (CSRI). 59 The data gathered were corroborated through review of the case notes (when informed consent for this was provided). Based on estimates of average group sizes in the study, the unit costs for individual sessions were estimated by dividing the number of group sessions by 3.8 and added to the number of one-to-one contacts to compute the overall number of contacts for the respective professionals.
Statistical analysis
To assess the strength of the association between the quality of care as assessed by the QuIRC43 and staffing levels, product-moment correlation coefficients were calculated. Moreover, we explored the association between service user characteristics and the number of self-reported contacts with the four aforementioned ‘core’ professional disciplines in a complete case analysis. To avoid confounding between individual and unit effects, we used the percentage difference of the staff contacts relative to the unit mean as the dependent variable and the demeaned predictors as independent variables in our regression model. Owing to the highly skewed nature of the service use data, we computed bias-corrected and accelerated bootstrap standard errors with 1000 replications. The choice of predictors of service use cost was informed by a review of related studies. 60
Results
As reported earlier, with a response rate of 87%, 52 NHS mental health trusts across England participated in the study. Service use information was collected on a total of 131 units and 739 service users, amounting to 59% of those able to provide consent. The mean (SD) age of service users was 40 years (13 years), and the majority were male (64%) and white European (81%), with 35% currently being detained involuntarily under the Mental Health Act. 42 Schizophrenia was the most common diagnosis (71%), and the mean (SD) GAF score was 54 (9). The units were predominantly community based (59%) and the mean (SD) sample size per unit was 5.6 (2.2).
A mean of 1.57 (SD 0.47) members of staff per bed were employed in the rehabilitation units (Table 5). As expected, most of these were either nurses or support workers; however, there was a relatively large variation in staffing levels between units. Among the ‘core’ staff these two disciplines also accounted for the largest fraction of staffing costs (56% and 27%, respectively). The quality of inpatient care was statistically significantly associated with nurse and overall staffing level, albeit with a relatively low coefficient in absolute terms. With the exception of OT contacts, there was no significant correlation between staffing levels and the mean number of self-report contacts.
Staff type | Number of full-time equivalent staff per bed | Mean (SE) cost per bed-day | Correlation with QuIRC (p-value) | Correlation with number of mean self-report contacts with the professional (p-value) | n | ||
---|---|---|---|---|---|---|---|
Mean (SD) | Min. | Max. | |||||
Psychiatrist | 0.04 (0.05) | 0.01 | 0.36 | 28 (3) | 0.08 (0.42) | 0.04 (0.66) | 108 |
OT | 0.08 (0.05) | 0.01 | 0.36 | 10 (1) | 0.02 (0.82) | 0.29 (< 0.01) | 93 |
Nurse | 0.68 (0.2) | 0.25 | 1.14 | 122 (3) | 0.24 (0.01) | 0.04 (0.69) | 122 |
Support worker | 0.63 (0.24) | 0.13 | 1.41 | 58 (2) | 0.08 (0.39) | –0.1 (0.27) | 122 |
Total number of unit and visiting staff | 1.57 (0.47) | 0.69 | 2.8 | n/a | 0.27 (< 0.01) | n/a | 119 |
The regression results at the service user level, as shown in Table 6, suggest that female service users had 17% (95% CI 2% to 35%) fewer contacts with psychiatrists than did male service users. There was a statistically significant association between nurse contacts and service users’ GAF scores, with every unit increase relative to the unit average associated with a 1% increase in GAF score (95% CI 0.2% to 2.2%). In the regression analysis on OT contacts, if the service user was currently involuntarily detained in the rehabilitation unit they were predicted to have reported 32% fewer contacts with an OT (95% CI 9% to 57%). Finally, for each year of age, service users had 0.9% (95% CI 0.2% to 1.8%) fewer contacts with support workers than the unit average.
Predictor/statistic | Staff type | |||||||
---|---|---|---|---|---|---|---|---|
Psychiatrist contacts (n = 628) | Nurse contacts (n = 630) | OT (n = 548) | Support worker (n = 608) | |||||
Coefficient (SE) | 95% CI | Coefficient (SE) | 95% CI | Coefficient (SE) | 95% CI | Coefficient (SE) | 95% CI | |
Female | –17.1 (8.5) | –34.9 to –1.7 | 10.6 (9.7) | –8.2 to 31.3 | 17.7 (13.4) | –7.6 to 45.3 | 23.7 (13.1) | –1.9 to 50.6 |
Age | –0.5 (0.3) | –1 to 0.2 | 0.1 (0.3) | –0.5 to 0.8 | –1.2 (0.6) | –2.2 to –0.1 | –0.9 (0.4) | –1.8 to –0.2 |
GAF score | 0.3 (0.6) | –0.7 to 1.5 | 1.2 (0.5) | 0.2 to 2.3 | 0.9 (0.8) | –0.7 to 2.5 | 0.5 (0.8) | –1 to 2 |
White European | 18 (13.8) | –9.8 to 43.5 | –1 (13.2) | –26.4 to 25.4 | 24 (18.8) | –14 to 57.1 | 2.5 (18.7) | –39.1 to 36.8 |
Involuntary admission | 13.5 (9.8) | –4 to 34 | –5.5 (6.2) | –16.4 to 8.1 | –32.2 (12.2) | –56.2 to –8.4 | 7.5 (12) | –13.5 to 31.8 |
Discussion
The health economic component of phase 1 of the REAL study provides an account of staffing levels and service use in a representative sample of NHS mental health rehabilitation units in England. Given the lack of published evidence in this area, this may aid decision-makers in planning processes. In comparison with acute psychiatric care settings in the UK, the number of nurses per bed appeared to be lower in rehabilitation units and the variation of nurses per bed between units was higher. 61 Owing to the cross-sectional nature of the sample, it should be noted that the associations we found are not necessarily causal. We found that nurse staffing levels were positively associated with quality of care. Therefore, it may be of value to consider more carefully the reasons for and consequences of staffing variations in the context of budget allocations. In this study no information on staff seniority was collected and only ‘core’ staff members were considered, which limits the accuracy and comprehensiveness of the reported costings. However, the findings suggest that the share of nurse costs was lower than in those in the only other study on the costs of rehabilitation care that the authors are aware of, which was carried out in Scotland in the 1980s,62 suggesting reduced investment in these services over the last 30 years.
The reasons for the low levels of correlation between staffing levels and mean number of contacts in this study were unclear. Non-response bias, lack of information on contact duration, variations in the number of contacts over time or misclassification of staff types may be part of the explanation. However, it may also suggest that, in a setting in which a certain number of health-care staff is designated to care for service users for a certain proportion of their time, to estimate average resource it may be more effective to measure staffing levels rather than rely on self-report data. This would also suggest that it is of value to go beyond staff ratios or direct measurement of time staff spent with service users, as in previous studies, and consider the perception of the amount of care received from the service user’s perspective. 63
Phase 1 qualitative component
Background
In phase 1 we aimed to examine the current provision of NHS mental health rehabilitation services in England using a cross-sectional, quantitative method. The data gathered would allow an identification of the characteristics of service users and services and an exploration of factors related to service quality. However, such services are inherently complex and influenced by a range of factors, including contextual and cultural factors, that may not be easily explained by quantitative methods alone. The NIHR reviewers of our proposal suggested that we add a qualitative component to section 1 of the research programme to address these issues and that we include a social scientist in our research team. In response to the reviewers’ suggestions, we invited Professor Gerard Leavey to join the team as a coinvestigator. He has extensive expertise in qualitative health services research and has previously collaborated with other members of the research team. One of the researchers, NG, registered for a PhD with the Division of Psychiatry at University College London (supervisors MK and HK) in 2010 and has led the data analysis and write-up of the phase 1 qualitative component as part of his PhD project.
The principal aim of the qualitative component in phase 1 was to identify aspects of policy and practice in mental health rehabilitation that could potentially be included in the GetREAL intervention developed in phase 2. However, we were also interested in the more detailed elements of service provision, structural and cultural, that facilitate excellence in rehabilitation services or, conversely, act as barriers. The objectives of this component were, therefore, to explore staff and service users’ views of the purpose of rehabilitation, their experiences of these services and the facilitators of and barriers to successful rehabilitation.
Method
Recruitment and data collection
We aimed to recruit two or three ‘front-line’ staff and two or three service users from 10–15 units for participation. Ideally, we would have selected units purposively to represent a range of sizes, locations (hospital or community based), geographic spread and quality representative of all units included in phase 1. Similarly, we would, ideally, have selected service users and staff on the basis of characteristics that were representative of our national survey data. However, as the qualitative component was carried out concurrently with the collection of the section 1 national survey data, these characteristics were not available. Nevertheless, we sought to include units of different sizes, locations and geographic locations. Service users were approached for participation with the aim of ensuring maximum variation, with a range of sexes, ages and lengths of stay, and, similarly, staff were approached for recruitment to try to ensure a range of disciplines, seniority, sex and experience. Informed consent was gained from all participants before they were interviewed.
Separate topic guides for staff and service users were developed in consultation with members of the PMG and the North London Service User Research Forum (SURF). These incorporated the ethos of the unit, aspirations and expectations of rehabilitation, the activities available on the unit, links with the community, service users’ access to community resources and activities, and the facilitators of and barriers to successful rehabilitation (see Appendices 2 and 3).
The interview was piloted in two units. A total of four staff and four service users were interviewed, and the data were transcribed. These recordings and transcriptions were reviewed by GL and HK to ensure that the interviews were being carried out consistently and consideration was given to possible amendments to the topic guides. In fact, no amendments were felt to be needed, as, although the interviews were structured to accommodate the core thematic interests set out in the topic guide, they were intended to be flexible, allowing interviewees to introduce new issues for exploration.
Each staff interview lasted around 30 minutes. Service user interviews tended to be shorter. The interviews were recorded, and the project manager and another administrator subsequently transcribed them. In addition, following each visit, the researchers recorded their observations about each unit, taking detailed field notes on the environmental context of the services such as the unit’s location, decor and furnishings, and on staff interaction with patients, the range and type of activities provided and the general ambience of the unit.
Data analysis
The data analaysis was carried out by NG under the supervision of GL and HK. The transcripts were imported into specialist software (Atlas.ti 6; Atlas.ti Scientific Software Development GmbH, Berlin, Germany) for analysis. As we were not pursuing any particular hypothesis or seeking to develop theory in this component but, rather, to add rich contextual information about the activities, functioning and quality of rehabilitation units, we adopted a straightforward approach to the analysis, using a standard coding and thematic procedure. However, although this assumes a more ‘realist’ approach to the phenomena in question, an interpretive engagement with the data is still possible. A grounded theory approach was not possible because the data for the qualitative component could be collected from units only during visits carried out for section 1 quantitative data collection. As such, interviews were usually undertaken with consideration of the time and availability of staff and service users. In addition, the data had to be collected within the time frame available for section 1 of the REAL study that did not allow time for concurrent qualitative data analysis. It was also recognised that the interviews required a relatively tight structure, constrained by the need to provide information for the development of the intervention in section 2 of the REAL study. Additionally, this type of interview schedule was appropriate, given the staff time limitations and service users’ difficulty with verbal performance due to the negative symptoms of psychosis. For these reasons, a thematic approach was deemed both pragmatic and useful.
Nicholas Green read and coded the transcripts. We began by using the topic guide as the initial coding frame covering the main ares of interest and then added codes and other key themes that were detected in the texts. We then developed an analytical framework to organise the data into themes. Members of the team (NG, HK, GL, IH and ML) discussed the coding and the emergent themes in depth during regular seminars, which also incorporated the exploration of relationships between thematic areas. Thus, we used conceptual maps to explore relationships and connections in the data, and also identified a series of detailed themes that were employed for further analysis.
The themes used to further consider the data were:
-
What ‘helps and hinders’ service users of mental health rehabilitation services in their recovery?
-
What is the nature and quality of the relationships between staff and service users?
-
What are the roles and responsibilities of front-line staff working within mental health rehabilitation units?
-
What are the potential benefits of multidisciplinary working?
-
What are the barriers within and between disciplines with regard to the effective delivery of rehabilitation services?
Results
Characteristics of units, staff and service users interviewed
A total of 26 service user interviews and 22 staff interviews were carried out by the two researchers (NG and IH) between September 2009 and July 2010. Of the service users, 24 of the 26 who completed qualitative interviews had also participated in the quantitative research interviews, the results of which are reported in Chapter 1. The participants were recruited from 12 units across eight trusts. The unit characteristics are shown in Table 7.
Characteristic | Phase 1 qualitative study units (N = 12), n (%) | All phase 1 units (N = 133), n (%) |
---|---|---|
Unit location | ||
Inner city | 3 (25) | 26 (20) |
Suburbs | 7 (58) | 96 (72) |
Rural area | 2 (17) | 11 (8) |
Unit type | ||
Hospital ward | 2 (17) | 15 (11) |
Community based | 9 (75) | 79 (59) |
Within hospital grounds | 1 (8) | 39 (29) |
The units ranged in size from 8 to 20 beds, with a mean of 15 beds. Nine units provided facilities for both men and women, and three were male only. Bed occupancy was 92%. All units were staffed by nurses and support workers, and all had a consultant psychiatrist. All but one had at least one OT working on the unit part-time (ranging from 0.2 full-time equivalent to 0.3 full-time equivalent). Only three units had a clinical psychologist working in the unit, but eight of the remaining nine had access to psychology from elsewhere in the trust. Staffing was therefore similar to that in all inpatient rehabilitation units across England.
Of the 22 staff interviewed, 15 were nurses (three of these were unit managers), three were OTs, three were support workers and one was an activity worker. Twelve of the staff interviewed were female and all but one were white European. Their mean age was 40 years (range 22–56 years).
The service user participant characteristics are shown in Table 8. Two-thirds of the 26 service users interviewed were male, with a mean age of 35 years. The majority had a diagnosis of schizophrenia. The mean length of stay on the unit was 15 months (range 4 months to 4 years). Overall, those who participated in qualitative interviews were generally similar to service users of all inpatient rehabilitation units across England, except that fewer were currently detained under the Mental Health Act42 (4% vs. 32%).
Characteristic | Phase 1 qualitative study participants (N = 26), n (%) | All phase 1 service users (N = 739), n (%) |
---|---|---|
Age (years), mean (SD) | 35 (11) | 40 (13) |
Male | 19 (64) | 475 (64) |
White | 22 (85) | 595 (81) |
Age at leaving full-time education (years), mean (SD) | 16 (2) | 17 (2) |
Diagnosis | N = 23 | N = 702 |
Schizophrenia | 19 (73) | 511 (73) |
Schizoaffective disorder | 1 (4) | 57 (8) |
Bipolar disorder | 1 (4) | 59 (8) |
Other | 2 (8) | 75 (11) |
Psychiatric history, mean (SD) | N = 23 | N = 594 |
Years’ contact with mental health services | 12 (11) (n = 22) | 16 (12) (n = 522) |
Previous admissions | 4 (4) (n = 20) | 6 (6) (n = 586) |
Length of current admission (months) | 33 (46) (n = 19) | 19 (59) (n = 572) |
Length of current admission to rehabilitation unit (months) | 15 (32) | 19 (33) |
Mental Health Act status | N = 24 | N = 630 |
Detained during this admission | 11 (42) | 427 (68) |
Currently detained | 1 (4) | 203 (32) |
Social functioning | N = 24 | N = 739 |
GAF score (range 1–100), mean (SD) | 57 (5) | 54 (9) |
Main themes
The results of the five main themes are described sequentially. Within each of the main themes, a number of subthemes emerged, and these are also reported.
Theme 1: what ‘helps and hinders’ service users of mental health rehabilitation services in their recovery?
With regard to this theme, four subthemes were identified: (1) the suitability of the service user for rehabilitation and the importance of the pre-admission assessment; (2) the unit ethos and approach provided; (3) the quality and suitability of the built environment; and (4) the facilitation of activities.
Generally, staff felt that, for them to be able to assist people in their recovery, it was essential that the service user was prepared or ‘suitable’ for rehabilitation.
One of the things that was established before I started, was that was quite a clear referral pathway into us, so people have to meet criteria to be deemed suitable for the placement.
Unit 0503, female, unit manager, 49
We like to make sure that people are in a position to benefit from the service, if people aren’t ready we will happily wait and reassess them at a later date when they might be in a slightly better position to engage with the services that we can provide.
Unit 1803, female, qualified nurse, 26
People have got to feel that coming here is going to be of benefit to them and that they are going to get something out of it.
Unit 1802, female, OT, 36
Readiness or ‘suitability’ for rehabilitation was predicated on the provision of multidisciplinary pre-admission assessments, as highlighted by the following staff comment:
We have a referral form that we send out that we feel covers most of the areas that we are interested in and that gives us a taster of the person. We then go and see the person, read through their notes, meet with the staff that they work with, look at any assessment work that’s been done, meet the person, have a chat with them, find out what they want, what their expectations are, what, what they want from us, we will tell them about us, what we can do for them, what we can offer them realistically and we go from there.
Unit 1703, female, qualified nurse, 43
In some cases the preadmission assessment would be further discussed within the multidisciplinary team (MDT) to ensure the appropriateness of the placement.
That information then comes back to the ward, where we have a MDT referrals meeting, where we will discuss that person in depth and then that may lead to a further assessment.
Unit 1703, female, qualified nurse, 43
Staff also indicated that it was important to allow time for referred service users to have pre-admission visits or periods of trial leave to the unit so that their suitability could be further assessed.
A visit helps to kind of ease anxieties, from both sides [staff and service user] and get an idea partly of what we’ve been referred is true and for them [service user] to get an idea of what we’re like.
Unit 1803, female, qualified nurse, 26
However, some staff reported that assessment processes were, on occasion, compromised because of pressure on acute admission beds. Thus, some service users were transferred to the rehabilitation unit prematurely and/or inappropriately.
We have a small number of clients, that really, ideally, are not ready for rehab[ilitation] and maybe need an alternative placement, but unfortunately there was a period when we were having to fill the beds.
Unit 0901, female, manager, 40
[T]he only reservations that people [staff] have are of whether people coming here are suitable for rehabilitation. And that’s a concern, ‘cause it’s always a concern that people are being pushed out of acute units.
Unit 1501, male, qualified nurse, 46
I think it’s become more acute as there’s less acute beds, there’s more pressure to kind of get people out quicker, so some people who are a little more acutely ill when they first come here.
Unit 0701, male, qualified nurse, 42
. . . sometimes we do favours for the acute services and take people who are perhaps are still a little bit too acutely ill for us . . .
Unit 2503, female, OT, 37
The inappropriateness of these placements, and the resulting problems, quickly become obvious. One of the biggest issues for staff was a lack of motivation among service users.
If clients don’t feel that there is going to be any benefit in coming here, and they can’t see that there is anything that we can provide that’s going to help them in any way, then they probably don’t really have that motivation to work with us.
Unit 1802, female, OT, 36
Despite these difficulties, staff still seemed to be willing to work with all of those who were admitted to their service, even if the specific aims of their treatment were not clear:
I think there is always something that they can benefit from, something that we can offer them. Whether it’s the full rehab[ilitation] package or um, or just something little that they can then, but even just a bit of education on their medication or you know, or social skills or something like that.
Unit 1802, female, OT, 36
I think some patients will get more out of it than others, but that greatly depends on their motivation and their wanting to change.
Unit 1801, male, qualified nurse, 32
However, although staff emphasised the importance of a thorough initial assessment, service users were often vague when questioned about the referral and admission process. In an apparent contradiction of staff’s assertions of service user preparation, service users generally appeared passive and rather uninvolved in the assessment and decision-making process related to their admission into rehabilitation. Often, their only source of information was fellow service users who were currently in, or had previously used, the rehabilitation unit.
I didn’t know anything. I was told just like a basic house where you have rooms in the house with other patients.
Unit 1803, male, service user, 42
I thought it would be more or less the same sort of set up, as [name of previous unit] and, erm, and really I didn’t have any idea really apart from, erm, it was, erm, it wasn’t a secure unit.
Unit 1802, male, service user, 45
All I knew was from people that had been in here in this house [rehabilitation unit].
Unit 1803, male, service user, 42
Although service users were generally positive about the idea of rehabilitation, most had little understanding of its specific purpose:
I suppose . . . somewhere to fix my problems . . .
Unit 1401, male, service user, 23
Only one service user seemed to have a better understanding, describing it as somewhere:
. . . to help me get back into the community and settle and start doing normal things, like start getting a job and my own place and stuff like that.
Unit 1502, female, service user, 26
When asked about the ethos and approach of the service, many staff spoke about the need for the team to be motivated to work with service users with longer-term needs. Many felt that both staff and service users should be clear about the aims of treatment. Several staff emphasised the importance of the service providing a flexible and individually tailored approach that was sensitive to the changing needs of service users.
I think you need a really good level of enthusiasm in the staff.
Unit 0901, female, support worker, 22
I think the approach should be one that is definitely based on evidence-based interventions um and that there is a clear pathway so that when people come they know what’s expected of them and that there will be a beginning, middle and an end.
Unit 0503, female, manager, 49
I think ideally a rehab[ilitation] unit, I mean it’s, it would evolve with the person.
Unit 0701, male, qualified nurse, 42
It would be an environment that was supportive, um, proactive, I would say a nurturing environment that was sort of going to help – help and support the individual.
Unit 1501, male, qualified nurse, 27
I suppose it’s patient-led so that patients are actually telling me, oh I’d like to do this or I’d like to try this, and that we can be responsive to what they’d like us to do.
Unit 0503, female, manager, 49
Service users reported that the rehabilitation units provided more individual freedom and autonomy than did acute admission wards.
. . . well you get more freedom, and you’re not with people who have just become ill . . .
Unit 1703, female, service user, 55
They don’t push you, they don’t push you do this or push you to do that, they just leave you to your own devices, let you do your own thing, sort of thing.
Unit 1703, male, service user, 51
However, service users’ perspectives on structure and autonomy tended to be based on personal tastes. Thus, other service users felt that the ‘regime’ in the rehabilitation unit was too inflexible, particularly in relation to getting up in the morning. Other complaints were made regarding the inflexibility of medication times and meal choices (although these were no different from the institutional routine of any hospital unit).
There are set times for medication, so, well, patients are aware of the times of medication, and therefore it’s just sort of routine you know?
Unit 1802, male, service user, 45
You have to decide what you want your meal to be in 2 days in advance and I guess that’s not really what you have to do in the real world.
Unit 0701, male, service user, 21
A number of service users spoke positively about the opportunity that they had in the rehabilitation unit to try things out, and valued the staff’s support and willingness to help them engage in activities.
I have come from being, wrapped in cotton wool, right to, you know, this life centre, of go and enjoy it, do what you want you want to do, and the staff will say, they will support you.
Unit 1802, male, service user, 45
The 12 units visited varied greatly in terms of their built environment. Only one had been purpose built to provide rehabilitative care. Of the remaining 11, two were hospital wards, two were in buildings that had been designed and previously used for another purpose and the remaining seven had been converted from mainstream community-based residences. Changing the use of the building from its original purpose had led to some compromises with regard to what the service was able to provide, but the buildings were generally in a good state of repair, both structurally and decoratively. However, some were more ‘homely’ in style than others.
Some staff identified and described how the building had a positive effect on the rehabilitative process. However, there were a number of concerns voiced by staff about inadequacies related to the built environment that detracted from service users’ rehabilitation, including overcrowding, shared bathrooms, a lack of male- or female-only spaces and a lack of dedicated space for occupational therapy activities.
I think it is a fairly modern, well-equipped, comfortable environment.
Unit 1401, male, manager, 39
Here we have a series of practice flats and I think they’re quite good and they offer an opportunity to self-cater and um, also so that they can socialise they are equipped with lounge, a bedroom, a kitchen, washroom, bath and they are separate from the unit but they’re not, they have, they can come to us at any time.
Unit 1501, male, qualified nurse, 46
. . . it would be nice for people to have en-suite facilities . . .
Unit 1802, female, OT, 36
. . . all of the stuff, I have to do like that is in the communal kitchen areas, which is quite tricky, well it can be sometimes, quite tricky, about managing that, because if you’re teaching people skills or getting them to relearn skills that they haven’t used for a long time, particularly people that might have anxieties, or very low confidence . . .
Unit 1802, female, OT, 36
The staff working in units based in hospitals questioned the suitability of rehabilitation being provided in these settings. More directly, staff were concerned that the concept of rehabilitation was contradicted and undermined by situating this work in an institutional environment.
I personally think that the hospital setting isn’t necessarily ideal for the client group.
Unit 1703, female, qualified nurse, 43
. . . and just the setting we’re in, the buildings really, it’s a really old hospital and it looks like a really old hospital, here which doesn’t foster feelings of you know, people want to look after themselves, it fosters kind of institutional/institutionalisation.
Unit 0701, male, qualified nurse, 42
. . . because we are in a hospital, we have to fit into the hospital situation, in terms of appointments, and pharmacy and people coming and going and that does obviously put limitations on things [sic: the ‘things’ referred to here are activities].
Unit 1703, female, nurse, 43
Service users were generally positive about the quality of their accommodation. When these facilities were available, having access to a private bathroom and kitchen facilities was regarded as particularly important.
You have your own room, your shower is in your own room, and you can make coffee and that when you want it.
Unit 0901, male, service user, 29
You have your own room with a cooker in and microwave and things in [studio flats for pre-discharge use].
Unit 1803, male, service user, 21
However, regardless of the environment and the geographic location (suburbs, inner city or a rural area), emphasis was placed on the importance of being close to community facilities and/or having good transport links to facilitate service users’ access to the community. Indeed, it was felt that proximity to local services helped to attenuate the sense of institutionalisation that some units created.
It’s really, really important, that were not so much in the middle of nowhere, so you can use trains, buses, and things like that, so that’s pretty useful for an ideal setting. I don’t think a rehab[ilitation] unit is going to be very effective in the middle of nowhere really because, we are supposed to be encouraging people to, access the community, and planning for when they go home, or go elsewhere, that they can support themselves.
Unit 2503, female, activity worker, 33
A lot of it is about, being able to access community services and start doing the things that people are going to be doing when they leave here and get them in to routines around that, and I think I have to say we are very lucky, where we are based, we are so near the town, and so near all the services.
Unit 1802, female, OT, 36
In 11 of the 12 units, the researchers noted that activities appeared to be taking place both on and off the unit. Activity programmes were often clearly displayed on noticeboards, and individualised programmes were filed in service users’ case notes. A variety of activity equipment was also observed in the units and there were often photographs on display of outings and activities that service users had participated in. However, in one unit, although an activity programme was on display, staff reported that the activities listed were not actually being delivered.
Many staff highlighted the need for rehabilitation services to provide service users with opportunities to develop or relearn skills, not just through providing activities on the unit but also through facilitating and supporting them to access community activities. This bridging function of their work is illustrated in the following comments:
To me this [rehabilitation] is all about building, you know, daily living skills, instilling confidence in people that perhaps has been shattered or perhaps they never had it at all.
Unit 1803, male, support worker, 48
It’s about helping people being able to function when they leave here, so whether it’s about learning cooking skills, so they can do that when they leave here, or whether it’s about accessing community services, so that when they leave here they are, going out and having that social contact, they are going to access things of interest, whether it’s vocational stuff, like setting up with work, you know, developing peoples routines, erm, it should all relate to, how that’s going to benefit people to move forward, to become more independent.
Unit 1802, female, OT, 36
I think, part of, one of the main parts of our, our role is to re-engage people with the community. You know and the only way you’re going to do that is to get them out of this unit to experience that.
Unit 1401, male, unit manager, 39
Definitely, that’s really important, really important, say, you know, if people want to go back to education, you would do that, to facilitate it . . . if they want voluntary work, or paid work.
Unit 2503, female, activity worker, 33
We do have links with the local college and we have several people that do courses at one of the local centres . . . they do basic literacy, community skills and creative writing.
Unit 1703, female, qualified nurse, 43
I meet up with a community walking group, and the reason I do that is because I couldn’t run a walking group myself here, but the reason I take people there is because that’s again something that people can follow through in the community when they have left.
Unit 1802, female, OT, 36
. . . well yes, I mean we go, there are local college do offer a number of courses for example we had a chap who’s been doing art, we have a chap who is doing horticultural college . . .
Unit 1501, male, qualified nurse, 46
Staff in some units had found ways to maximise service users’ access to activities by holding the activities in the evenings when service users were more likely to be up and about and when the units were less busy. Staff also noted that offering a variety of activities was important for engaging different service users. However, the purpose of these activities was not always that clear.
We do evening activities on a regular basis for the patients, but they’re quite social and diversionally orientated.
Unit 0503, female, manager, 49
On Thursday we have an activity which we arrange for all service users. Now that tends to be very socially oriented, so that would either be a place of interest . . . a sporting/leisure activity . . . or we’ll go out to, somewhere for an extended period, so we’ll you know, either go, maybe to do a walk for example in the lakes.
Unit 1401, male, unit manager, 39
I think it’s just a case of organising something and then seeing if you rope in as many patients as you can, going out, whether that be a pool group, or going out for a meal, and stuff like that, and . . . sometimes you can persuade the patients, talk them round into going out.
Unit 1801, male, qualified nurse, 32
However, simply motivating services users to leave the unit was a considerable challenge.
It’s all about trying to get them off the unit in the day time, get them out, that’s where we tend to have problems, ‘cause they don’t want to go out. So it’s trying to get them off, and get them motivated.
Unit 1401, female, qualified nurse, 56
Staff also recognised and reported working with service users to provide an individualised programme of activities: a plan that was guided by the service user’s interests and at a pace they could manage.
You know rather than us saying you know this is what you need to do, it’s very much more now working with the residents here and seeing what they want to do and working at a pace where they feel comfortable with.
Unit 1502, male, qualified nurse, 42
We do change things and personalise things, for individual people, you know, what their interests are, what their aims are, what their goals are.
Unit 1802, female, OT, 36
Although the staff’s interviews suggested that providing a variety of activities was a major focus of their work, this was contradicted by a commonly held perception of service users that the units were dull and devoid of stimulation, that they spent much of their time watching television or smoking.
I don’t really do anything. I just go out to the shop and come back. Some days you’ve got some OT but it’s not every day. I mean you don’t have a lot to do.
Unit 0901, male, service user, 28
It’s so easy to be lazy, you know, just get, get you med[ication]s, and sit around smoking, drinking tea and coffee.
Unit 1802, male, service user, 45
For other service users, the experience was more positive, and some particularly valued having the autonomy to choose what they wanted to do.
. . . and here, they don’t push you, they don’t push you do this or push you to do that, they just leave you to your own devices, let you do your own thing, sort of thing.
Unit 1703, male, service user, 51
You have got a bit of freedom, you know, you can go out and go for a walk or something like that.
Unit 1803, male, service user, 21
The majority of service users reported that they attended to basic activities of daily living to different degrees and with different levels of support.
Everyone has to tidy their rooms once a week and um everyone has to do their laundry.
Unit 1803, male, service user, 21
Budgeting, cooking, washing your clothes, certain chores . . .
Unit 1501, male, service user, 51
Well basically, I done the shopping, I do cooking and stuff like that so it gives me something to do.
Unit 1803, male, service user, 42
Self-catering which is cooking, and buying some food from the shop.
Unit 0701, male, service user, 26
In addition, many service users also reported having taken part in a wide range of other activities:
. . . like quad biking, walking ponies, horse riding, segways, martial arts in the forest, loads of things.
Unit 0701, male, service user, 22
Playing the computer, we got guitars and things, we got a drum set, a set a keyboard, a guitar room and all that here.
Unit 0901, male, service user, 29
However, some service users felt that they were made to participate in activities despite not finding them appropriate or likely to be beneficial.
I didn’t feel like I needed, needed that group at all, I mean I went along to one, and I mean, I couldn’t get anything out of it.
Unit 1803, male, service user, 21
I just don’t like it [activity groups] – I haven’t really got a choice you know.
Unit 0701, male, service user, 26
Theme 2: what is the nature and quality of the relationships between staff and service users?
Staff recognised the importance of developing good working relationships with service users during their inpatient admission. This was seen as essential in assisting service users in their recovery, and staff seemed to gain job satisfaction from this aspect of their work. Two subthemes related to this emerged: (1) the benefits of a good working relationship and (2) the time needed to build the relationship. When asked which aspect of the service they were most proud of, one member of staff reported:
I suppose the relationship with the clients. I think we’ve all got a pretty good relationship.
Unit 0901, female, support worker, 22
Staff recognised that working in mental health rehabilitation services over a relatively long period of time gave them an opportunity to develop therapeutic relationships with service users. In contrast to working on acute units, staff were able to see individuals progress, which contributed to their job satisfaction.
I think the thing that I like the most is that you can actually . . . forge a really meaningful therapeutic relationship with someone because they’re here for a long time and get to know somebody really well and see the journey if you like, because on acute it’s very sort of intense isn’t it?
Unit 0503, female, manager, 49
You form good relationships with people over time that sort of carry on even when people have been discharged. People come back for things like the lunch group here and meet up again and that’s good yeah.
Unit 1502, male, qualified nurse, 42
I enjoy having the time to develop working relationships with the people I’m supporting, I enjoy supporting them and seeing them do well, and hopefully leave here, and not come back.
Unit 0701, female, qualified nurse, 45
I think it’s really warm and friendly here, and I think a lot of people have come back to say, what a friendly place it is, and we have also had service users come back, to pop in for a cup of tea, and, that’s been lovely, and seeing that they are getting on really, really well with their lives.
Unit 2503, female, activity worker, 33
Staff felt that forming a good relationship with service users over time was a very important part of the work that they were doing, and that it reduced the barriers between them and helped the service user feel more able to discuss specific issues.
. . . you could build a really good therapeutic relationship with the person and essentially you get far more information, because they’re far more relaxed.
Unit 0901, female, manager, 40
. . . people aren’t going to engage unless they want to and it’s about kind of I guess finding something they enjoy and developing rapport with that individual um so much as you know what they enjoy you might not know how to get them involved in that unless you have a good relationship with them and you know other things about them and they trust you to help them get involved in things.
Unit 1803, female, qualified nurse, 26
I enjoy the fact that there’s no quick fixes, that people are complex and this, you can’t say there’s this thing wrong with them – do this and it will all get better – you have to really understand somebody and really get to know them.
Unit 2503, female, OT, 37
Staff often reflected that it took a considerable amount of time to build up therapeutic rapport with service users, and they seemed to particularly value the fact that working in the rehabilitation service, where service users tended to stay for longer periods of time, gave them this opportunity, whereas other mental health services did not.
I really enjoy it, yeah I really enjoy it, I find I get far more interaction with service users than I have done in my short nursing practice.
Unit 1501, male, qualified nurse, 27
I find it much more satisfying than in acute services where it’s a very quick turnover, it’s, it’s down to medication, it’s very quickly out of the door and you have a sad feeling that people will be back because you have only really scratched the surface, where here you can get a much more in-depth knowledge of people.
Unit 2504, female, qualified nurse, 49
However, service users reported more mixed experiences of their relationships with staff. Although the majority reported that they found staff to be supportive, some had negative views.
Actually the majority of nurses I think are great and I really like them and yeah I do get on well with most of the nurses here, yeah.
1803, male, service user, 21
When I first came here I didn’t feel like new I just fitted in like straight away, and the staff were really helpful.
Unit 1502, female, service user, 26
The staff are very good here, they are very supportive.
Unit 1801, female, service user, 55
I get on with them alright but they can be a bit of a pain, to be honest.
Unit 1401, male, service user, 32
I would say that the staff can be very over bossy and unkind.
Unit 1501, male, service user, 32
Theme 3: what are the roles and responsibilities of front-line staff working in mental health rehabilitation units?
Five subthemes were identified relating to the overarching theme of roles and responsibilities of mental health rehabilitation service staff: (1) pathways into working in mental health rehabilitation, (2) staffing levels and skill mix, (3) staff training, (4) staff roles and (5) staff satisfaction.
Staff in rehabilitation services fell into two main categories with regard to how they came to be working in rehabilitation services: those who had chosen to work in rehabilitation and those who had been redeployed there, seeing it as an easier option than other services.
I’ve chosen to work primarily in rehabilitation for most of my professional life.
Unit 1501, male, qualified nurse, 46
I come here, because I’d been on acute previously to that, so I’d been on acute for 10 years and it was getting too much, it was getting too stressful, and it was making me ill to be quite honest, so I had to move really, so I’ve been here since last May.
Unit 1401, female, qualified nurse, 56
Staff frequently reported that the unit did not have enough staff to work with service users one to one in the way they would like to. Many units had only two staff per shift available outside core office hours.
I think we could do with more numbers [staff] to be honest, we could do more one to one, in fact we could do a lot more one-to-one work.
Unit 0503, male, qualified nurse, 52
I think it’s adequately staffed . . . we meet our, our minimums, our basics. But a lot more staff means that we would be able to do a lot more things. More rehab[ilitation]-focused work, so you could actually allocate someone to, you know, on a daily basis to cook for someone, to shop with someone. Sometimes you just can’t do that, you know, with your resources.
Unit 1401, male, unit manager, 39
Up until 2 weeks ago we got two OTs on the ward, myself and another occupational therapist, it meant we could concentrate both on the in-ward activities and the psychoeducational groups and diversional activities, the on-ward services and also that we can focus on one-to-one on getting back out to the locality. I am now having to cover two people’s jobs.
Unit 1703, male, OT, 42
Despite this, most staff felt that their team had an adequate range of skills; however, this seemed to be interpreted with regard to the interests of staff members rather than the multidisciplinarity of the team and specific skills they had been trained in:
. . . a mixture of old and young and a mixture of kind of skills and experiences across the board from people who have got excellent cooking skills, to others who have far more in the way of kind of life experiences, which has got its benefits.
Unit 1803, female, qualified nurse, 26
I think it’s a very good mix actually. Um, if you look at the staff nurses, you’ve got myself, another staff nurse, who qualified the same time, so we’re newly qualified, you know just over a year. Then you got a couple of other staff nurses that have been qualified, well one for about 6 years, and then another like 20 years and then you’ve got the Sister, 20 odd years. So it’s a good mix.
Unit 1501, male, qualified nurse, 27
Staff recognised the benefits of attending training for their continuing professional development. However, staff often reported that resources were not always available to fund their place on courses, that courses were often full and that providing adequate cover to release staff for training was problematic.
I think training is hugely important for everybody.
Unit 1703, female, qualified nurse, 43
I think over time even little bits of training can make a significant difference in, not just in how you do your work, but in kind of how you feel about your work and every year you go to the same manual handling course, it’s a bit dull really, it’s not very mind expanding.
Unit 1501, male, qualified nurse, 46
I’m trying to get on the mentor’s course, it’s run through the university, I did get a place, but I had to kind of wait a year and it then happened that I couldn’t attend, but there’s no kind of professional, forward-looking development really.
Unit 1502, male, qualified nurse, 42
That [access to training] is getting more and more limited I would say, I mean, I just had a support worker knocked back for the OT training because there was only two places funded this year. I think there’s always more you can offer and I think um, I’d like to see more people accessing the psychosocial interventions degree that there is.
Unit 2503, female, OT, 37
[T]o be honest, one of the bigger problems is actually the backfill to people, is if you send someone off on a course, if there is no funding to cover them when they’re absent, that’s probably a bigger problem than the cost of a course.
Unit 2503, female, OT, 37
Owing to the recognised difficulties in accessing training, senior members of the unit staff (managers/psychologists/senior OTs) sometimes provided in-house training.
It tends to be in-house stuff that we do ourselves, within the current professionals either the [unit psychologist], or other psychologists would run.
Unit 0701, male, qualified nurse, 42
The previous consultant did training in rehab[ilitation] and around the recovery model, and a psychologist would also bring things to the table like CBT, um maybe incorporate, you know, interview techniques, and stuff like that, so if what we can get from each other.
Unit 0901, female, manager, 40
We’ve got an OT that works here, and does onsite training as well, you know, about how to engage with people.
Unit 1401, male, manager, 39
Support workers were particularly aware of their lack of training:
. . . we weren’t really given any help in what to do, sort of just go talk to people. I suppose it’s the best way to get experience is just to go out and do it, but yeah didn’t have any training.
Unit 0901, female, support worker, 22
This was also noted by some service users.
If you’ve got a problem they can’t help you, you know, they’re support staff, they don’t get no training.
Unit 1703, male, service user, 53
People are not trained, you know, the care workers and support staff, I mean.
Unit 1801, female, service user, 34
Nursing staff seemed less able than other members of the MDT to define their role. Generally, they described their role as one of co-ordinating other staff and functions within the unit rather than delivering these themselves.
I’m just, I’m an overseer really. I just kind of make sure things are getting done. You know so it would be nice to be involved more in there.
Unit 1501, male, qualified nurse, 46
The role of unqualified staff and support workers seemed much clearer both to the support workers themselves and to other staff. Most described it as providing ‘hands-on’ and practical support to service users, working in collaboration with other staff under the supervision of the qualified nurses.
It’s usually the support workers that have to go out for appointments, hospital appointments and stuff, they’re also the ones that are expected to do most of the, kind of, tasks on the unit, like lunch and activities and stuff.
Unit 0901, female, support worker, 22
They’ve come very much as health support workers, but they support people in their rehabilitation. They have got really good skills, I mean some of them have never worked in mental health or even the health service before, they have actually come with a real range of life skills and abilities.
Unit 2503, female, OT, 37
I believe that on the one hand that the backbone of the NHS, be it general or psychiatric, are support workers, who are often not necessarily qualified but people of a certain age who are prepared to just get in and muck in and do stuff.
Unit 1703, male, OT, 42
The OTs were also clear about their role and remit within the service.
We are trying to help someone achieve the maximum amount of independence and autonomy whilst you know getting whatever support they might require.
Unit 1703, male, OT, 42
It’s about supporting people to become as independent as they can, supporting people, to be able to go and do the things that they, partly need to do but want to do, and that’s all about fulfilling dreams and hopes, and is very goal orientated, strength based, that’s what OT is.
Unit 1802, female, OT, 36
We can concentrate both on the in-ward activities, and the psycho-educational groups and diversional activities, kind of getting people up and doing arts and crafts and computers, to kind of involve all the patients in the in-ward services and also that we can focus on one to one on getting back out to the locality where they came from and want to go back to.
Unit 1703, male, OT, 42
However, although most OTs had a good understanding of their role, they often felt that other staff did not fully understand it, seeing OTs as people who could run groups and organise outings.
. . . the staff have seen it [occupational therapy services] as – oh the OT’s here, they are going to do a cinema trip, they are going to do, a this trip, a that trip, and I don’t do that here . . .
Unit 1802, female, OT, 36
One member of the nursing team, a unit manager, echoed this view:
It’s not just for say example, OT doing just shopping stuff um, maybe they need to be involved in other things because they have to have time for assessments, one to one with clients and stuff, and they’re fantastic when it comes to running groups and they facilitate groups, they also you know, implement new groups as well.
Unit 0901, female, unit manager, 40
A number of service users were able to identify the specific tasks and responsibilities of staff from different disciplines who had assisted them in their rehabilitation.
The nurses help you with things, erm, do care plans and things like that to help you improve, and improve on my mental health as well.
Unit 1502, female, service user, 26
They [the nurses] make you better with tablets.
Unit 0503, female, service user, 25
I use a technique, one technique is to keep a diary, the occupational therapist has, erm, been showing me how to use a diary, how to make entries, to check it every day, and you know it’s important.
Unit 1802, male, service user, 45
However, commonly, service users tend to describe the role of the staff group collectively rather than the respective roles of individual disciplines:
The staff, they help you keep positive, they tell you don’t think about it, try to think positive.
Unit 1801, female, service user, 34
The staff are very friendly and they get you motivated, they’re on your case . . .
Unit 1803, female, service user, 26
Overall, staff appeared satisfied with their work and appeared to gain great satisfaction from seeing people progress in their recovery.
I enjoy rehab[ilitation] very much, I’ve worked in rehab for a while and worked in rehab in other places, and um, it is, it’s kind of where I find my skills fit.
Unit 2503, female, OT, 37
I enjoy the interaction with the residents, um, I enjoy the different characters, I enjoy seeing the people make progress.
Unit 1803, male, support worker, 48
I enjoy having the time to develop um working relationship with the people I’m supporting, I enjoy supporting them and seeing them do well.
Unit 0701, female, qualified nurse, 45
. . . that’s one of the reasons that I enjoy rehab[ilitation] so much, you can see them go through their journey, and then they leave, and you then you think ‘oh’, but you know they have to move on and you think, well it’s a positive step because now they’ve moved on it means they’re doing really well.
Unit 0901, female, unit manager, 40
Theme 4: what are the potential benefits of multidisciplinary working?
As mentioned previously, all units were staffed by nurses and support workers and all had a consultant psychiatrist. The vast majority had at least one OT, and most also had a psychologist working in the team part-time or had access to one. Staff reported that having a MDT was important, but the specific benefit of this was not always clear.
I think it would be good, it’s good to have a true multidisciplinary team I don’t think it definitely needs to be an OT but it’s good to have as many different disciplines.
Unit 2504, female, qualified nurse, 49
I think there needs to be multidisciplinary approach within the team also. So you need, you know we’re lucky enough to have an OT, so you need an OT, I think alongside that OT, you, there needs to be a technical instructor, that’s focused on activities.
Unit 1401, male, unit manager, 39
It’s good working with the nursing team and with um medical, and with the psychology, so we seem to interlink a lot, and I think that’s fantastic.
Unit 0901, female, unit manager, 40
On some units, staff reported, in positive terms, an overlap in the roles of different disciplines, and a number of staff suggested that there could be benefits to more generic working and less rigid role demarcation between the different disciplines.
We have a very good team, very tight knit team and work very well together, and we do cross over into each other’s areas, and I think because of all the work we do we are quite confident in doing that.
Unit 1703, male, OT, 42
I think a blurring between some of the disciplines would be nice as well, for, for nurses and support workers to help them undertake some of the assessments that the OTs use, and er, also for nurses to work with, when we do people work with the groups, and help with group work with the psychologist.
Unit 1703, female, qualified nurse, 43
However, occasionally, staff reported a lack of mutual support between disciplines.
I don’t think they [OTs] get enough support here from the nursing staff.
Unit 0901, female, support worker, 22
Theme 5: what are the barriers that different disciplines encounter with regard to the fulfilment of their roles?
A recurrent problem reported by staff of all disciplines was that the unit was understaffed and they did not have enough time to spend with service users as a result of competing priorities.
I think that’s a big thing there, be patient, trying to do different things, but most of all having the time to spend with them, that is the very important thing because we don’t, as the qualified nurses, we don’t.
Unit 1401, female, qualified nurse, 56
Sometimes when you’re busy doing bits, you don’t really get the sense that you are getting anything done, and sometimes you want to sort out things with your client but you’re not able to do that because you’re too busy dashing around doing a little bit of this and a bit of that, and managing small bits to keep the unit running.
Unit 1801, male, qualified nurse, 32
I got quite a few staff off on long-term sick at the moment, when we’ve got our full establishment operating we’re able to do a lot more social inclusion work.
Unit 0503, female, unit manager, 49
I personally could see work for another qualified OT. I don’t enjoy, is the amount of work that there is and there just being me, so it goes back to the staffing thing. It can be frustrating at times.
Unit 1802, female, OT, 36
I am now having to cover two people’s jobs and it means that stuff is falling by.
Unit 1703, male, OT, 42
It can be understaffed a lot though, um, when I first got here, in the first month, it probably was understaffed most shifts.
Unit 0901, female, support worker, 22
Many nursing staff noted that the amount of administrative work had increased in recent years and that this meant they did not have adequate time to work directly with service users, which was a source of dissatisfaction.
The trouble is being kind of one of the senior staff nurses here, is that you often get stuck in the office and you have to do all of the paperwork and you don’t . . . it’s the support workers who get out and about and take them out to different places and ahh, they probably see more of the benefits of the work than I do.
Unit 1501, male, qualified nurse, 46
It’s the admin[istration] and the paperwork side of things that since we’ve become a foundation trust we just seem to have a lot of audits. We audit the audits!
Unit 1502, male, qualified nurse, 42
Some service users were also acutely aware of how much time nurses spent in the office:
I’m a patient, remember writing is finished, who comes first the writing or the patient. You know I’m ill, I’m saying I’m ill and they have to go and write it down you know.
Unit 1703, male, service user, 53
Unit managers often reported that they were under pressure from their seniors to discharge people owing to a lack of beds in the system, but there was often not enough suitably supported accommodation for service users to move on to.
We’ve often got people here who don’t really need to be here but they’re waiting for 24-hour supported accommodation.
Unit 0503, female, unit manager, 49
Some nurses reported that they found some of their colleagues’ practices to be rather rigidly outdated and that these colleagues lacked the motivation to update their practice through training opportunities.
There are nursing staff here that have been here for a very, very long time, and they don’t like change, and it causes a lot of – it causes a lot of friction among staff.
Unit 1401, female, qualified nurse, 56
However, yes there is always going to be some people that will get set in one way of doing things, right and that may affect the way they work with people.
Unit 0701, female, qualified nurse, 45
I think if, this service, does stay open, and we get [. . .], the newer type client group that we have started to get, there may be some members of staff, that may need a little bit more sort of training, because we are used to caring for people, who . . . who have needed long-term care.
Unit 1801, female, qualified nurse, 40
It’s the old school I think that drag their heels a bit regarding taking up of additional training.
Unit 0503, female, unit manager, 49
As well noting the inadequacies of suitable facilities within the unit (such as appropriately equipped kitchens for assessing and working with service users, as noted previously), OTs also reported that they found it difficult to maintain up-to-date information about community resources that service users could engage with.
It can be quite tricky, knowing about what’s out there, what’s available, it’s very time-consuming, it is hard work, because a lot of these things do change every year, so trying to keep up with it, [laugh] it does take up quite a lot of your time.
Unit 1802, female, OT, 36
Summary of the main themes identified in phase 1
The qualitative component of phase 1 of the REAL study aimed to investigate the facilitators of and barriers to successful mental health rehabilitation from staff and service user perspectives.
A number of related themes and subthemes were identified. These can be broadly organised into the ethos of the unit, the resources available and the demands on the service.
Ethos
There was a clear emphasis on providing a recovery-based practice that encompassed collaborative working between staff and service users as well as therapeutic optimism. Staff highlighted the importance of assessing service users for their suitability for rehabilitation and reported how this process was undermined when the pressure on acute beds forced them to accept individuals who were not yet stable enough to engage with rehabilitation.
There was also clear recognition of the benefits of forming a good therapeutic alliance between staff and service users and the time required to achieve this. Staff and service users acknowledged the importance of activities as an important part of the rehabilitation process.
Resources
Staff were more aware than service users of the negative impact of a poor built environment on people’s rehabilitation. However, more obvious were the problems of inadequate staffing that were noted by both staff and service users. Staff were aware of the difficulties in delivering good rehabilitation when there were inadequacies in staff numbers, skill mix and attitude. The role of staff training, support and supervision were also acknowledged.
Demands
As well as inadequate resources, other barriers to rehabilitation included the competing priorities on staff time (such as administrative duties), a lack of supported accommodation for people to move on to from hospital and poor awareness or provision of community-based resources for staff to link service users with as part of their rehabilitation.
Synthesis and discussion
The qualitative component of phase 1 of the REAL study was designed to help us better understand the nature of working in mental health rehabilitation services and the challenges faced by staff and service users. In undertaking this strand we aimed to identify areas that are required to improve the delivery of services and that could be incorporated into the design of the GetREAL intervention in phase 2 of the study. In a thematic analysis of the interview data we have attempted to make explicit some of the ‘real-world’ realities and dynamics of service provision in NHS mental health rehabilitation services, contrasting the aspirations of staff with the often harsher realities experienced by service users.
In illustrating these sometimes disparate views, we take the position that the ability of services to provide optimum rehabilitative activities is a systemic issue in which positive outcomes (service user motivation and skills development for independent living) are predicated on multiple and interdependent components that are situated, additionally, within the demands, functions and processes of mental health services overall. Thus, failure or blockage in one component tends to have negative consequences throughout the system.
We will briefly demonstrate some of these consequences with regard to different types of systemic challenge. Some barriers to effective care may be external in origin and resource driven (financial) and, therefore, less amenable to internal staff intervention (e.g. when acute-bed shortage prompts a premature transfer to rehabilitation). As we noted previously, staff tend to underline the need for good service user preparation and assessment prior to transfer; staff need to understand the purpose and expectations of the rehabilitation unit. However, service users were often unaware of this process and much less able to describe the specific aims of rehabilitation. Staff noted that building positive therapeutic relationships with service users was an important component of successful rehabilitation, and that this took time. This was valued by both staff and service users. However, this measured and careful approach to progression in rehabilitation was sometimes difficult to achieve, and some service users appeared disengaged and resentful of the expectations that the service had of them. The presence of such service users is likely to lead to deterioration in collective motivation and has the potential to impact negatively on the morale of staff and other service users who are more able to engage with the available support. Similarly, a key element of staff satisfaction is the relatively longer and deeper engagement with service users in rehabilitation than with those on the acute wards. The ability to support service users and see them through to successful discharge provides a visible product of achievement, which is undermined by pressures to accommodate inappropriate referrals. As we have described elsewhere in the report, a sense of service users being ‘stuck’ transfers and permeates among staff and is palpable within the entire environment.
Other barriers may be more appropriately viewed as discrete, internal problems of the service, but they may nevertheless have their origins in the wider organisational culture. The issue of how MDTs function within rehabilitation services is vital. It is an issue that impacts on staff and performance, through leadership, goal-setting and task co-ordination, communication, skills and knowledge transfer, and morale. Various studies in the NHS and in other settings have highlighted the failure of health-care teams to set aside time for regular meetings to define objectives, clarify roles, apportion tasks, encourage collaboration and manage change. Poor communication also arises from differences in status, power, educational background, assertiveness of members of the team and assumptions about the source of leadership. 64 These problems also emerge from the staff interviews in our study. Although many highlighted the importance of multidisciplinary working, reinforced throughout the interviews with nursing staff, we noted that nursing staff were less able than staff of other disciplines to clarify their role and remit, and there was some evidence that they did not fully understand the OT’s role. This is a clear indication that staff may lack team cohesion and direction. Thus, long-established hierarchical relationships within rehabilitation services and the ‘boundary maintenance’ attached to different roles appear to undermine collective goal-setting and collaborative problem-solving. Ideally, all rehabilitation staff should have a composite picture of a client’s needs and all should be active in designing and assisting with activities, guided by professional expertise (psychiatry, psychology and occupational therapy). However, participants reported a tendency towards individualist contribution rather than collaborative working. In terms of staff resources and patient outcomes this is, at best, inefficient and, at worst, corrosive.
Again, this problem is compounded by the demands of competing priorities (particularly administrative duties) that detract some nursing staff, who have limited time, from one-to-one work with service users. Although nursing staff regarded this as a source of frustration, support workers were available and tended to do the most ‘hands-on’ work with service users. Support workers are enthusiastic and keen to be involved in rehabilitation but their potential appears to be somewhat disregarded within service teams. This is disappointing, given that their lack of training was acknowledged as problematic by both staff and service users. The failure to intentionally incorporate support workers in the design, planning and undertaking of rehabilitation activities seems a lost opportunity, particularly so when training is limited. Training tends to be provided ‘in house’ by other members of the team and represents an opportunity for the MDT to team build, resolve conflicts and set goals.
Despite the barriers staff cited to performing their role, most reported a high level of job satisfaction. This finding has been corroborated by a national study of staff morale in mental health services that found the highest level of job satisfaction and lowest level of burnout was among staff of rehabilitation services. 65 Encouragingly, there was a strong sense of purpose and therapeutic optimism, and staff really seemed to enjoy their work. This positive attitude has been acknowledged as a particularly important aspect of recovery-orientated services. 66 Given that the client group that rehabilitation services work with is particularly complex and, by definition, difficult to treat, this positive therapeutic culture is not only an impressive achievement, but also recommended in the guidance on commissioning mental health rehabilitation services. 67 Nevertheless, the barriers to staff being able to perform their duties adequately should also be acknowledged. Staff shortages, difficulties in accessing appropriate training and a lack of availability of appropriately supported accommodation for service users to move on to all impede the delivery of an effective and efficient service. Rehabilitation services have historically been subject to disinvestment, with negative consequences for people with severe and complex mental health problems. 17 Our results suggest that there are ongoing issues of inadequate resourcing of rehabilitation and supported accommodation services.
Some limitations to the qualitative component of phase 1 need to be acknowledged. The findings presented here should not be regarded as a generalised view of all rehabilitation units in the NHS in England; rather, our analysis of a range of units reveals some of the distinctive challenges to providing high-quality care for service users within such environments. Although ideally units, staff and service users would have been purposively sampled, this was not possible owing to the logistic and time constraints of data collection in phase 1. However, the characteristics of those interviewed were similar to those of the whole phase 1 sample. Nevertheless, there may have been a sampling bias, as those with specific issues they wished to discuss may have been more likely to agree to participate.
Qualitative interviews could not be carried out with service users who were too unwell to give informed consent to participate and therefore those interviewed were likely to be further forward in their recovery than all users of rehabilitation services. As with all qualitative research, the researcher analysing the data subjectively influenced the themes and subthemes that emerged. We minimised this source of potential bias by using a topic guide to steer interviews, with the main topics as the starting point for the thematic analysis. Nevertheless, we are satisfied that we have managed to accumulate a wide range of perspectives and experiences that accurately reflect the challenges of service users and staff within rehabilitative settings.
Summary of main findings from phase 1
-
Our findings from phase 1 of the REAL study represent the first comprehensive description of mental health rehabilitation services in England. Our survey provided national ‘quality benchmarking’ data for these services. We found a positive association between quality of care and clinical outcomes, suggesting that interventions that improve the quality of care provided are likely to promote service users’ autonomy and experiences of care.
-
Our health economic component found a positive association between staffing, particularly the provision of nursing staff, and quality of care. However, the staffing of rehabilitation units is lower than others in mental health inpatient services, and appears to have decreased in the last 30 years.
-
Our qualitative component highlighted the reliance on untrained support workers to carry out much of the ‘hands-on’ work with service users in these settings. Staff commonly cited increasing administrative duties and understaffing as reasons that they could not work directly with service users as often as they wanted to. There was a strong sense that the increased pressure on beds in the inpatient system was leading to units having to accept service users considered ‘unsuitable’ or not yet able to engage productively in rehabilitation. Nevertheless, staff appeared to hold authentic therapeutic optimism and to derive great satisfaction from building therapeutic relationships with service users over the longer term and supporting them to achieve successful discharge.
As the promotion of autonomy is the main goal of rehabilitation, ongoing investment in these services is needed to continue to deliver high-quality care that promotes recovery. Investment in the local supported accommodation pathway is also needed to ensure that service users have an appropriate place to move on to when they are ready to leave the rehabilitation unit.
Use of phase 1 results for phase 2
The findings from phase 1, particularly the qualitative component, were used to identify a number of potential barriers to providing activities that we incorporated into the development of the GetREAL staff training intervention in phase 2.
These included:
-
maximising staff availability to support service users’ engagement in activities
-
identifying alternative spaces for activities when there are inadequacies in the built environment (e.g. if there is no activity room/communal kitchen)
-
challenging staff assumptions about whose responsibility it is to provide and facilitate activities
-
clarifying local community resources for activities
-
facilitating access to community activities as well as activities on the unit.
Chapter 3 Phase 2: development of a staff training intervention to enhance service users’ activity in inpatient mental health rehabilitation units
Aim
The aim of phase 2 was to develop a staff training intervention to engage the users of inpatient mental health rehabilitation units in activities.
Phase 2 was led by Dr Sarah Cook and Cathy Hill of Sheffield Hallam University.
Developing and refining the intervention
The original framework
Our original proposal (unpublished) described the theoretical basis and structure of staff training intervention in some detail, as follows:
The training model will be based on a three-stage model of change68 which distinguishes between Predisposing, Enabling and Reinforcing processes. The Predisposing stage aims to facilitate a focus on the need for change and gain local service ‘sign up’. 69 The Enabling Stage involves identifying and removing barriers to change, team-level action planning and the development of new necessary skills. 70 Finally the Reinforcing Stage involves maintaining changes once they are in place, identifying and implementing team changes and monitoring approaches in order to reinforce sustainable change. 71
Predisposing Stage: aimed at highlighting the drawbacks of current practice combined with awareness of the importance of occupation/activity as a solution. A consultation meeting/seminar with senior service managers and senior clinicians to explain the programme, gain support and identify local champions will be held, facilitated by Helen Killaspy, Frank Holloway and Tom Craig. The consultation report written after Phase 1 will be used as a basis for discussion about the individual service’s strengths and areas for improvement. Successful approaches and examples of good practice identified from across the country in Phase 1 will be shared and ideas of how the GetREAL programme can be tailored to meet the needs of the local rehabilitation service discussed. The programme will build on local expertise and explore how links with community resources can be increased. Areas of the programme that may strengthen the service’s statutory quality assessments by the Healthcare Commission will be highlighted.
Enabling Stage: a brief (one day) training course for nurses and unqualified staff (health care assistants, activity co-ordinators, Support, Time and Recovery workers) of each inpatient rehabilitation service. These training courses will be delivered by the GetREAL team members. The training course will be tailored to resources in each service and demonstrate motivational techniques72 and simple interventions to encourage service users’ activities. Staff will be taught to tailor their approach to the service user using the trans-theoretical model of change: pre-contemplation; contemplation; preparation; action; maintenance; and termination. 73 The delivery style of the course will model these theoretical principles and motivational methods. The course will be repeated in the first one to two weeks of the programme to maximise attendance by all relevant staff. A further training day a month later will boost staff motivation and skills and use group problem solving to overcome barriers encountered. Examples of situations where implementation has been problematic will be explored together using solution focused approaches and suggestions agreed on how to tackle these. Where systemic barriers are identified, these will be discussed by the GetREAL team with service managers and, where required Helen Killaspy, Frank Holloway and Tom Craig will discuss the issues with the relevant senior managers in order to try to resolve them. The GetREAL team OT and activity worker will work with staff in the inpatient rehabilitation services daily for four weeks after the training course to give intensive, hands on support for staff to gain confidence in the implementation of the techniques and interventions learned during the training course. The GetREAL workers will tailor their approach according to the local service setting but will aim to work alongside staff on a day to day basis, using real life examples of service users’ difficulties with motivation and engagement in activities to model and explore with the ward team possible approaches that may be effective.
Reinforcing Stage: during the final week of the intervention, the GetREAL team members will meet with the service manager to consider how best the skills acquired can be incorporated into the usual structures and processes of the service e.g. through regular review of the type and amount of each service user’s activities at Care Programme Approach meetings and through staff reflection on strategies to facilitate activities at care planning meetings and in staff supervision sessions. Ongoing support to staff and trouble-shooting will be provided by the GetREAL teams by email contact. The GetREAL programme will be piloted in one service and modified in response to feedback. SC and CH will lead the development of the training course and will provide ongoing mentoring and supervisory support for the GetREAL teams. We will seek advice from the College of Occupational Therapists, the Royal College of Nursing and the Royal College of Psychiatrists as to whether the training course can be developed with their formal approval and/or in line with the award of a recognised qualification in mental health rehabilitation. Recruitment of GetREAL team members will take place during the latter half of Phase 2 to allow a period of training ready for Phase 3.
The intervention was further developed through an iterative process of consultation with various experts in mental health occupational therapy and mental health rehabilitation and piloting. Table 9 lists these activities, the individuals involved and the associated time frames.
Process or event | People/venue | Date |
---|---|---|
Consultation workshop at COT conference | Brighton (Sarah Cook and Marieke Wrigley) | 24 June 2010 |
Generation of themes from the workshop | Sarah Cook | 27 June 2010 |
First draft of the manual | Sarah Cook and Cathy Hill | July–September 2011 |
Development of the manual at 1-day workshop with PMG members | Gemma Dorer, Marieke Wrigley, Helen Killaspy, Tom Craig, Frank Holloway, Sarah Cook and Cathy Hill | 22 September 2010 |
Consultation with SURF | Helen Killaspy | 24 November 2010 |
Detailing the support and supervision structures for the GetREAL teams | Sarah Cook, Helen Killaspy and Gemma Dorer | 18 October 2010 |
Development of the fidelity criteria and checklist | Sarah Cook, Helen Killaspy, Frank Holloway, Tom Craig and Michael King | 3 December 2010 |
Consultation with five rehabilitation units in the NHS | Barefoot Lodge – Oxleas NHS Foundation Trust (Helen Killaspy) | 14 January 2011 |
Ruby Lodge – Doncaster (Helen Killaspy by telephone) | 11 February 2011 | |
Macmillan Close – Mapperley, Nottingham (Helen Killaspy and Sarah Cook) | 25 January 2011 | |
Swallownest Court, Rotherham (Sarah Cook) | 24 January 2011 | |
Broomhill House, Gedling Nottingham (Sarah Cook by telephone) | 9 February 2011 | |
Induction of the intervention teams | Camden and Islington NHS Foundation Trust, Gemma Dorer, Helen Killaspy and Melanie Lean | 7–11 February 2011 |
Training the intervention teams and refining the GetREAL manual and training materials | GetREAL teams including the service user experts, trained by Sarah Cook and Tim Mundy in Sheffield | 14–18 February 2011 |
Piloting the GetREAL intervention for 5 weeks in two NHS units, including the predisposing meetings (by Tom Craig and Helen Killaspy to Barefoot Lodge and by Frank Holloway and Helen Killaspy to Ruby Lodge) | GetREAL teams Barefoot Lodge – Oxleas NHS Foundation Trust; Ruby Lodge – Doncaster |
28 February–1 April 2011 |
Manual and fidelity criteria agreed by steering group | Steering group | 18 March 2011 |
Post piloting refinements day workshop | GetREAL team, Helen Killaspy, Sarah Cook, Melanie Lean and Gemma Dorer | 5 April 2011 |
Final version of the GetREAL manual produced with training materials | Lara Freeman and Deborah Taylor | 8 April 2011 |
Application for course endorsement from the COT for unit staff attending the GetREAL training | Helen Killaspy and Sarah Cook | 12 April 2011 |
Consultation workshop at College of Occupational Therapists conference
Our abstract (see Appendix 4) to facilitate a consultation workshop at the 2010 annual conference of the College of Occupational Therapists (COT) was accepted and published in the conference programme and book of abstracts. Sarah Cook (REAL coapplicant) and Marieke Wrigley (REAL project collaborator) facilitated this workshop, which was attended by 18 OTs who practise in mental health units focused on recovery and rehabilitation. We particularly wanted to find out what practising therapists found useful in their day-to-day settings in engaging service users in activities so that we could identify specific occupational therapy practices in the GetREAL intervention.
Using the results from phase 1, a clinical scenario was devised and presented to workshop attendees; this described a mental health rehabilitation unit that presented a number of specific challenges in terms of delivering activities for service users (Box 1). The attendees were asked to work in small groups to consider how they might address these challenges, and they wrote their ideas on flip charts. The ideas were discussed in the larger group, and consensus was reached on the most useful ideas by asking attendees to place ‘voting stickers’, with or without additional comments, on the generated ideas. After the workshop the flipcharts and voting stickers were typed out and collated. These data were analysed by SC into major themes and actions that were then considered for potential inclusion in the enabling stage of the GetREAL intervention. Two major themes emerged: ‘approaches’ (service user involvement, valuing and supporting staff, recovery model) and ‘actions’ (involve patients, support and train staff, audit skills and interests of patients and staff, expand interests of patients and staff, engage patients in activities, link with community resources, promote sustained change).
-
You have been approached to do a piece of consultancy work with a group of staff and patients on an inpatient rehabilitation ward.
-
The ward is located in an inner-city area of South London and is a 15-single-bedded, mixed-sex ward. The average length of stay is 18–24 months and the aim is to move patients on to more independent living.
-
The staffing is two qualified nurses and three support workers per shift and sessional input of 2 hours per week from a consultant psychiatrist.
-
Currently there is no OT input, but there is a plan to have 0.5 WTE band 6 OTs starting in about 1 month.
-
Staff group is seen as lacking in motivation and resistant to new ideas and ways of working.
-
Patients are seen as lacking in motivation and observed as ‘sitting around doing nothing’.
-
Poor care planning.
-
Management is viewed as being task driven and outcome obsessed.
-
Staff view is that there is a lack of opportunities for training and development.
-
In addition, lack of money to support activities.
WTE, whole-time equivalent.
Sarah Cook and Cathy Hill then drafted the GetREAL intervention manual, which was based on the framework in the original proposal and expanded to include the relevant suggestions from the consultation workshop. The manual was underpinned by relevant theory from the disciplines of occupational therapy and organisational psychology (particularly change management). Approaches relevant to the common values of both disciplines were included and organised into a cohesive intervention. Some examples of the materials are shown in Figure 1 and Box 2.
Use one or more of these tools to help people think about what occupations and activities interest and motivates them. It is important for staff to think about how they themselves spend their time and what interests them, and the importance of engaging in meaningful and rewarding activities for their self-esteem, physical health, mental stimulation and general well-being.
The Activity Card Sort is a comprehensive instrument for assessing participation in occupational performance of instrumental, social-cultural and leisure activities.
The UK Modified Interest Checklist gathers information on a client’s strength of interest and engagement in 74 activities in the past, present and future. Interests are listed in nine categories that focus on different types of activity choices.
The Time Wheel is a way of representing the amount of time we spend on different types of activities, such as leisure, work, self-care and sleep. Or it can use occupational science concepts of ‘doing’, ‘being’, ‘becoming’ and ‘belonging’. These would need explaining and discussing. Coloured card circles can be placed on top of each other and cut. These are then turned to represent the amount of time spent.
Development of the manual with the REAL Programme Management Group
The intervention manual was further developed in a 1-day workshop with the members of the REAL PMG who work clinically in rehabilitation units. In particular, we focused on the content of the two training days that take place at the beginning and the end of the 5-week enabling stage. We also discussed the scoping and engaging of key people in NHS management of each unit in the predisposing stage and the scoping of local resources within and outside the units. We also discussed the support and supervision of the GetREAL teams and how to provide this, and the importance of ensuring ‘model fidelity’ through encouraging the OTs to keep a daily reflective diary and to reflect on progress at key points in the 5-week enabling stage.
Detailing the support and supervision structures for the GetREAL teams
Adequate support and supervision was considered a very important part of the GetREAL intervention. We expected the two GetREAL teams to meet many challenges as they travelled round the country from unit to unit. We did not want them to divert from the manualised intervention in their efforts to solve local problems, nor did we want them to feel isolated and unsupported. Three types of supervision were agreed (line management, clinical supervision and adherence to the GetREAL intervention). The focus, frequency and the responsible supervisor for each is shown in Table 10.
Who | What | When | How |
---|---|---|---|
Activity workers receive line management and clinical supervision from the OTs | Line management Contract Leave: sick leave, annual leave, maternity leave Training, continuing professional development Annual appraisal Performance issues Personal issues |
As needed; informal or formal appraisal at start and 6-monthly | Face to face |
Clinical supervision Facilitation of reflection and creative problem solving Monitoring performance |
Each week | Face to face | |
OTs Line management from Gemma Dorer (who subsequently left and was succeeded by Louise Reynolds) – line manager in Camden and Islington NHS Foundation Trust. Louise will carry out annual appraisals, with feedback from Sarah and Tim and others as appropriate Clinical supervision and supervision for adherence to the intervention by Sarah Cook (OT) and Tim Mundy (Organisational Change consultant) Louise is also happy to discuss clinical matters in her supervision sessions where appropriate |
Line management as above | At least 3 monthly, and more often as needed | Negotiated – face to face or telephone/SkypeTM (Microsoft Corporation, Redmond, WA, USA) |
Clinical supervision Negotiated between OTs, Sarah and Gemma/Louise Facilitation of reflection and creative problem-solving Adherence to intervention manual Sarah and Tim – refer to fidelity criteria checklist |
At least twice per intervention cycle | Email, telephone or Skype with Sarah Face to face approximately 6-weekly with Gemma/Louise |
|
Service user consultants receive support from the OTs and guidance from Helen Killaspy | Via their collaboration agreements for their role as service user experts | As required but at least once per 5-week cycle | Face to face |
Everyone Tim facilitates |
Peer group reflective supervision | Once per intervention cycle | E-mail, telephone, meetings (suggest 6 weekly) |
Consultation with service users
A summary of the GetREAL intervention was presented by the chief investigator, Helen Killaspy, to the North London SURF. This group comprises 12 members with severe and enduring mental health problems, some of whom have been patients in rehabilitation units. The group members were asked for their views on the structure and content of the proposed intervention, and they gave important feedback on the importance of goal-setting as a facilitator to achieving planned activities. This was subsequently incorporated into the manual and fed back to the GetREAL teams.
Development of the GetREAL fidelity checklist
Sarah Cook developed a GetREAL intervention fidelity checklist, with input from the PMG. This checklist was piloted at the pilot intervention sites and modified to the version included in Appendix 5. At the end of each enabling stage on each intervention unit, the GetREAL OT and SC completed the fidelity checklist, which was then sent to the chief investigator (HK) for completion of the items about the predisposing stage. The final scores of the reinforcing stage were added at the end of the 12-month intervention period by the project manager (ML), who kept a log of contacts between the GetREAL teams and the intervention sites.
Consultation with NHS mental health rehabilitation units
The study statistician (LM) selected six units randomly from our sample pool of eligible units identified in phase 1 (i.e. those that scored below the median on the QuIRC and had at least eight beds) for piloting/consultation. Five responded, of which two (noted below in bold) were purposively selected for piloting (one near London and one in the north of England):
-
Barefoot Lodge, Oxleas NHS Foundation Trust
-
Ruby Lodge, Doncaster
-
Macmillan Close, Mapperley, Nottingham
-
Swallownest Court, Rotherham
-
Broomhill House, Gedling, Nottingham.
We sent each unit manager the manual and then asked for their feedback approximately 2 weeks later. Discussion and feedback took place in face-to-face meetings with the unit managers and other members of their team for three units and by telephone discussion with the unit manager for the other two. The feedback received was very positive and, in general, validated the content and approach of the intervention. One important issue that was raised through this consultation process was that the OTs in post in the intervention units, especially the senior OT in the intervention team, might feel threatened by the input of the GetREAL intervention teams. Having taken this on board, we discussed with the GetREAL teams ways in which, early on in their time on the unit, they could seek out OT staff, present the intervention as helping them in their work role and negotiate how to work productively together. This aspect was also emphasised at the predisposing meeting.
Induction of the GetREAL teams
An induction programme was organised for the GetREAL teams in the first week of their posts (Table 11). As they were employed by Camden and Islington NHS Foundation Trust, they were obliged to attend mandatory training for the first 2 days. Other induction activities were more specific to the GetREAL intervention.
Monday 7 February | Tuesday 8 February | Wednesday 9 February | Thursday 10 February | Friday 11 February |
---|---|---|---|---|
St Pancras Hospital, London | St Pancras Hospital, London | Highgate Mental Health Centre, London, and Camden and Islington Rehabilitation services | GetREAL team base at Highgate Mental Health Centre | Department of Mental Health Sciences, Royal Free Hospital, London |
9 a.m.–5 p.m. | 9:30 a.m.–1 p.m. | 9 a.m.–3 p.m. | 9 a.m.–5 p.m. | 9 a.m.–1 p.m. |
Local employing trust induction (Camden and Islington NHS Foundation Trust) | Local employing trust induction | 9–10 a.m.: meet with Helen Killaspy | Day to read the GetREAL manual | Helen Killaspy – overview of the whole project (feedback about how much detail to include when delivering similar overview in each unit) |
Orientation to Camden and Islington NHS Foundation Trust and rehabilitation services with Gemma Dorer (tour of rehabilitation services, meeting with relevant people, etc.) | Begin to go through manual in outline form | |||
Early finish – travel home for those living further away |
Training the intervention teams, refining the manual and training materials
A full week of training for the two GetREAL teams was organised at Sheffield Hallam University. It was attended by the GetREAL OTs, one of the activity workers (due to recruitment problems) and the two service user experts. The training was delivered by SC and Tim Mundy (who succeeded Cathy Hill as the organisational change consultant). Having the service users with us was very beneficial. It helped to maintain the service user focus and to build the teams’ cohesiveness. The topics covered are shown in Figure 2.
The programme of training is shown in Table 12. The training methods mirrored those used in the intervention: acknowledging and building on people’s strengths, modelling behaviours and skills, practice and problem-solving.
Monday 14 February | Tuesday 15 February | Wednesday 16 February | Thursday 17 February | Friday 18 February |
---|---|---|---|---|
1 p.m.: styles of learning within the GetREAL interventions – starting with you (practical exercise) | Manual and detail of intervention: OT approaches and training days Fidelity to the GetREAL intervention Amendments to manual |
Practice with each other, make amendments and any agree any further materials Prepare materials for the pilots starting 21 February |
Manual and the organisational development cycle: reviewing the GetREAL teams’ work; sustainability and the tipping point Final amendments to manual |
Final queries Practice online supervision on Skype Verify practical details of pilot sites, dates of travel, etc. |
During the training, minor amendments and refinements were made to the intervention manual as we rehearsed elements of the intervention.
The GetREAL staff did very useful work in developing the training materials and resource pack to be used in the initial training sessions with unit staff. These tools were particularly to help unit staff communicate with their service users about activities and interests that each individual had or would like to take up. The tools included:
-
time line – ‘a day in the life of me’
-
occupational balance – colour wheel
-
set of laminated activity cards (photographs)
-
the interest checklist.
This week was crucial to promoting the GetREAL teams’ ownership of the intervention, as well as training them in techniques and processes. The trainers delivered a shortened version of the training to the second activity worker, who started later.
Piloting the GetREAL intervention
The intervention was piloted in two units: Barefoot Lodge (Oxleas NHS Foundation Trust) and Ruby Lodge (Rotherham Doncaster and South Humber NHS Foundation Trust). The piloting included the predisposing visits by Tom Craig and Helen Killaspy to Barefoot Lodge, and by Frank Holloway and Helen Killaspy to Ruby Lodge. This reinforced the importance of this component of the intervention, which aims to gain sign-up for the intervention from senior managers and clinicians, and to ensure that the GetREAL teams are welcomed and that all necessary practical arrangements are in place. These arrangements include booking staff cover so that all unit staff can attend the training sessions during the enabling stage, and gaining access to computers and electronic patient records. The pilot cycles enabled the two new intervention teams to try out the intervention, and to identify and address obstacles.
Post-piloting refinements from day workshop
The piloting of the GetREAL intervention did not highlight any specific problems with the content of the intervention, but it provided useful learning for the GetREAL teams. In particular, it highlighted the need to identify staff willing to try out the techniques for encouraging service user activities immediately after the training day in the first week. Although staff tended to give very positive feedback about the training day, they then tended to be caught up in their usual day-to-day tasks and unavailable to work with the GetREAL teams. The GetREAL OTs thus adopted a more assertive approach to ensure take-up of opportunities for modelling the techniques by identifying individual staff at the end of the training day who would commit to working with them in subsequent days. This approach was successfully implemented at intervention units. The GetREAL teams, HK, CH and SC met to agree minor refinements to the intervention manual after the piloting. These were mainly minor clarifications of processes. The schedules and materials for the staff training sessions delivered during the enabling stage were also completed and minor amendments to the fidelity checklist were agreed.
Final version of the GetREAL manual produced with training materials
The process of consultation and piloting resulted in the GetREAL intervention manual undergoing eight iterations. The structure, values and core actions remained the same throughout, which corroborated the original approach. The final version was theoretically coherent, allied to practice and practically feasible. The final version is available in Appendix 6.
Application for endorsement from the College of Occupational Therapists
Enquiries to both the Royal College of Nursing and the COT were made to find out whether or not these organisations might endorse the staff training element of the GetREAL intervention (the enabling stage). We thought that the unit staff who attended the initial and final training sessions would appreciate a certificate of attendance endorsed by a national body. The Royal College of Nursing’s position on their accreditation was that it was more suitable for longer courses, and so an application with the relevant justifications and evidence was prepared for the COT, and this was agreed. The COT logo was added to the attendance certificates received by staff who participated at the intervention units (see Appendix 7).
Chapter 4 Phase 3: cluster randomised controlled trial to investigate the clinical effectiveness and cost-effectiveness of the ‘GetREAL’ staff training intervention
Aim
To evaluate the clinical effectiveness and cost-effectiveness of the GetREAL staff training programme to improve service user engagement in activities in mental health rehabilitation services scoring in the lower range of quality of care in England.
Hypothesis
Patient activity in rehabilitation services that receive the GetREAL staff training intervention will increase to a significantly greater degree than patient activity in services that continue with usual rehabilitative care.
Design and setting
A single-blind, cluster randomised controlled trial in which inpatient mental health rehabilitation services across England were the unit of randomisation. We chose a cluster design to prevent the effects of the training influencing untrained staff were we to randomise staff rather than services. A cluster design reduces this potential ‘contamination’ of the intervention between trial arms.
Trial registration
The trial is registered as Current Controlled Trials ISRCTN25898179 (www.controlled-trials.com/ISRCTN25898179).
Trial Steering Group
The trial was overseen by a Trial Steering Group chaired by Professor Philippa Garety of King’s College London. The other members were Professor Diana Rose, Professor of Service User Research at King’s College London; Professor Morven Leese, Professor of Statistics at King’s College London; Ms Genevieve Smyth, Mental Health Lead at COT; the chief investigator (HK); and the study statistician (LM). The Trial Steering Group met for the first time on 10 August 2011 and on four further occasions during the trial.
Participants
All inpatient mental health rehabilitation services scoring below the median on the QuIRC rating of service quality during phase 1 of the REAL programme were eligible for inclusion. We added one further exclusion criterion subsequent to our original proposal: units with ≤ 8 beds were excluded. This was for two reasons: (1) because the GetREAL intervention might be somewhat overwhelming for small staff teams in small units; and (2) to ensure an adequate number of service users on whom outcome data would be available, as sufficient numbers needed to be recruited for the GetREAL intervention to be evaluated within the phase 3 time frame. This additional criterion led to the excluson of three units across England.
Practical arrangements for allocating services to trial arms (randomisation)
From our previous national survey of rehabiliation services,19 we estimated that around 45 services would be eligible for phase 3. Allowing for a 20% non-participation rate, we estimated that around 35 services would agree to participate. In fact, 133 services participated in phase 1, of which 64 were eligible for phase 3. An initial 35 units were randomly selected from the pool of eligible phase 1 units by the study statistician (LM). Four units declined to participate; in three cases this was because of planned reconfiguration to their services. As a result, we increased our sampling pool to randomly select and recruit 40 units so that we could secure an adequate sample size on which to test our primary hypothesis.
These 40 units were randomly allocated on an equal basis to receive the GetREAL intervention or to continue providing usual care. Randomisation was staggered, in batches of 10, to allow sufficient time for the researchers to gather baseline data and for the GetREAL teams to deliver the intervention at each site. Randomisation was carried out independently of the research team by the Aberdeen Randomisation Service. As there was a relatively narrow spread of QuIRC ratings across the 133 units in phase 1, we decided that our original proposed stratified randomisation approach (using the lower five strata of QuIRC ratings) was not necessary to match services for quality ratings in each randomisation arm.
Interventions
Intervention arm
The units allocated to this arm received the GetREAL staff training intervention as developed in phase 2 and described in detail in Chapter 3 and in Appendix 6. Each of our two GetREAL teams delivered the intervention in 10 inpatient mental health rehabiliation units between April 2011 and August 2012.
Usual service arm
The units allocated to this arm continued to deliver their usual service and were able to use any resources at their disposal to provide maximum care for service users. There were no restrictions on the work of these teams.
Treatment fidelity
At the end of each unit’s intervention period, the supervising OT (SC) completed a pro forma, together with the GetREAL team’s OT and a senior member of the research team who had attended the predisposing meetings (HK). This recorded the delivery of 24 specific aspects of the GetREAL intervention, with each completed item achieving a score of 1 (see Appendix 5).
Informed consent and masking of researchers
The researchers collected baseline data in four units (two allocated to receive the GetREAL intervention and two comparison units) within the 4-week period prior to the GetREAL teams starting their intervention (i.e. from March 2011 to July 2012). All service users in each unit were eligible for participation in the study, and the researchers approached them to explain the purpose and process of the study. The service users were given a participant information sheet and had an opportunity to ask questions about the study. Those who were judged as having capacity to give informed consent and those who declined to participate were not interviewed. This process was repeated for baseline and follow-up data collection. In keeping with the approval granted by the South East Essex Research Ethics Committee for the whole REAL study, we were able to gather data from staff and case notes for those participants who lacked capacity to give informed consent. We made a concerted effort to minimise the chances that our researchers would be unmasked by stressing to unit staff, throughout the time we were in contact with them, that they should not reveal to the researchers whether or not they had been randomly assigned to receive the GetREAL training intervention. Any unmasking of researchers was reported to the PMG. We assessed the influence of unmasking by asking the researchers to record their views about which units received the intervention and which were comparison sites after they had collected the 12-month follow-up data. When unmasking occurred at baseline data gathering, the second researcher gathered the 12-month follow-up data to minimise bias. All 12-month follow-up data were collected between March 2012 and July 2013.
Primary outcome
The primary outcome was the degree to which service users were engaged in activity over the previous week, assessed using the time-use diary. 74 This measure rates patients’ activities during four periods each day: morning, lunchtime, afternoon and evening. The degree of engagement in activity as well as the complexity of the activity is rated on a scale of 0–4 for each time period, giving a maximum possible score of 112. The diary is completed retrospectively during a structured interview with the participant. The scale has good inter-rater reliability. 74 If a patient lacked capacity to give informed consent for participation in a face-to-face interview, the information about his or her activities in the preceding week was gathered from the case records and discussions with his or her primary nurse. Outcomes were assessed 12 months after baseline data collection to investigate whether or not the GetREAL staff training intervention was associated with sustained change in the unit’s practices.
Secondary outcomes
-
Service users’ social functioning as rated by a key staff member using the Life Skills Profile (LSP). 75 This measure comprises 39 staff-rated items each rated on a four-point Likert scale, with the most positive response scoring 4 and the least positive response scoring 1, giving an overall score ranging from 39 to 156.
-
Length of admission (days).
-
Percentage of service users discharged/ready for discharge in the last 12 months.
-
Percentage of service users discharged to an ‘out-of-area placement’ in the last 12 months (this identifies problems with the availability of suitable local supported accommodation to which patients can be discharged).
-
Staff attitudes towards service users’ progress were assessed using the question ‘I expect this person to be able to move on to a more independent setting within the next 12 months’. This assessed therapeutic optimism, which has been cited as a key factor in service users’ recovery. 66 A key staff member, such as the primary nurse, rated their view on a five-point Likert scale. This was converted into a binary score for analysis (very likely/likely vs. neither likely nor unlikely/unlikely/very unlikely).
-
Service quality was assessed using the QuIRC,43,44 which was completed by the unit manager. This tool comprises 145 questions on service quality and provision. It reports percentage ratings on seven domains of care: living environment, therapeutic environment, treatments and interventions, self-management and autonomy, social interface, human rights and recovery-based practice.
Data collection
Descriptive data on all patients were collected from staff and case notes as follows: demographics (age, sex, ethnic group), diagnosis, length of history and length of current admission. The primary and secondary outcome measures were completed as described above. Potential mediators of outcomes were also assessed, including the staffing of the unit (collected from the unit manager), patients’ overall functioning (assessed using the GAF scale50), substance use (assessed using the staff-rated Clinician Alcohol and Drug Use Scales)76 and challenging behaviours that may make community placement difficult [assessed using the staff-rated Special Problems Rating Scale (SPRS)]. 77
Data management
Data were entered into the study’s Microsoft Access® (Microsoft Corporation, Redmond, WA, USA) database by the researchers. Range and logic checks were built in to assist with data cleaning. Ten per cent of data were double entered to check for data entry errors, with an error rate set at 5%, above which all data would be double entered. The error rate was < 5% and, thus, no double data entry was required. Data were checked and cleaned by the statistician before analysis. 78
Sample size
Based on a two-sample t-test for the time-use diary score at the 12-month follow-up, it was estimated that 129 service users would be required in each arm for the detection of an effect size of 0.35 SD between the intervention and comparison groups, with 80% power and 5% significance level. This was inflated to 186 service users in each arm from a minimum of 31 clusters (inpatient rehabilitation units) to allow for the clustering of service users within units, assuming an intracluster correlation coefficient of 0.04 and an average cluster size of 12. The sample size calculation was performed using Stata version 11.
Data analysis
We followed Consolidated Standards of Reporting Trials (CONSORT) guidelines79,80 for the analysis of cluster randomised trials and for the presentation of our results. We published a study protocol including details of our approach to the data analysis prior to carrying this out. 78 Our data analysis plan is included in Appendix 8.
Service users’ descriptive characteristics were summarised using mean (SD), median (IQR) or proportions, as appropriate, and by trial arm. Our intervention was at the unit (staff team) level, and the primary outcome was assessed at the service user level. Our main inference was at the service user level, as the aim of the intervention was to improve service user engagement in activities. The unit-level outcomes were of secondary interest. Random-effects linear regression was used for the primary outcome, adjusted for the baseline value of the time-use diary score, to evaluate the effect of the intervention. However, some service users assessed at follow-up were different from those assessed at baseline, as some had been discharged and new service users had been admitted to the unit. Therefore, the mean baseline score for each unit (based on the service users present in the unit at the baseline data collection point) was used in the model, rather than the scores for the individual service users at baseline. Bias attributable to missing data and predictors of missingness was investigated. The analysis was then adjusted for predictors of missingness associated with the outcome to preserve the missing at random mechanism. Assumptions of normality of the residuals were investigated using normal plots. The agreement between staff and service user time-use diary scores was examined by plotting the two scores against each other. As there was a moderate degree of departure from the straight line of agreement, the primary analysis consisted of service user ratings only. In a sensitivity analysis, the time-use diary score for service users who lacked capacity to participate in the assessment was imputed using the staff-rated scores and other potential predictors in the imputation model. 81 Imputation was undertaken using the method of chained equations and clustering by centre was adjusted for by including the estimates of the random effects as a predictor in the imputation model. A supportive analysis was also carried out at the unit level by comparing the mean time-use diary scores across the trial arms using a linear regression weighted by cluster size and adjusting for baseline mean scores. A sensitivity analysis was also carried out adjusting for the length of admission in the unit (at the 12-month follow-up point) and the GetREAL intervention fidelity score. A further sensitivity analysis was carried using the staff-rated outcome for all service users.
For the secondary outcomes, appropriate statistical models allowing for clustering were used for outcomes measured at the service user level, and linear regression based on the cluster summary measures was used for outcomes measured at the unit level. All analyses were carried out on an intention-to-treat basis using Stata version 11.
Results
Response
Figures 3 and 4 show the recruitment of units and service users at baseline and the 12-month follow-up. Forty units were recruited and 20 were randomly allocated to receive the GetREAL intervention. Over the course of the trial one intervention unit closed (suburban, community based). At baseline, there were 260 potential participants in the intervention units and 265 in the comparison units. A total of 79% were recruited in each arm (206 and 211, respectively). Of these, 53 (20%) intervention unit participants and 51 (19%) comparison unit participants lacked capacity to complete research interviews, and so for these participants only staff-rated time-use diary scores were gathered.
At the 12-month follow-up, there were 259 potential participants in the 19 intervention units, of whom 174 (67%) were recruited, and 246 in the 20 comparison units, of whom 170 (69%) were recruited. Of these, 18 (11%) intervention unit participants and 11 (6%) comparison unit participants lacked capacity to complete the research interviews.
Unmasking
There were six episodes of unmasking (15% of units), all of which occurred in intervention units during baseline data collection. In these units, the second researcher carried out the follow-up data collection. At the end of the trial the researchers correctly identified 29 out of 39 (74%) units as either intervention or comparison.
Unit characteristics
Table 13 shows the main unit characteristics at baseline and at the 12-month follow-up. Most were based in suburban areas in the community. Both the intervention and the comparison units had been in operation for a median of at least 10 years, and both had a median of 15 beds with almost full occupancy. Fewer intervention units than comparison units had access to a clinical psychologist (69% vs. 90%), although around half of the units in both groups had a clinical psychologist actually working in the unit. There were no other noteworthy differences in staffing, and staff turnover in the last 12 months was similar. The intervention units had a higher level of service user turnover in the previous 12 months at baseline (69% vs. 44%). Very few units reported discharging patients to out-of-area placements in the previous 12 months at baseline or at follow-up, and this secondary outcome was, therefore, not investigated further. There were no differences between intervention and comparison units in QuIRC domain scores. No further differences, other than those noted at baseline, were found at 12 months.
Characteristic | Baseline | 12-month follow-up | ||
---|---|---|---|---|
Comparison units (N = 20), n (%) | GetREAL units (N = 19), n (%) | Comparison units (N = 20), n (%) | GetREAL units (N = 19), n (%) | |
Unit location | ||||
Inner city | 7 (35) | 9 (47) | 7 (35) | 9 (47) |
Suburbs | 13 (65) | 10 (53) | 13 (65) | 10 (53) |
Unit type | ||||
Hospital ward | 1 (5) | 5 (26) | 1 (5) | 5 (26) |
Community based | 19 (95) | 14 (74) | 19 (95) | 14 (74) |
Beds, median (IQR) | ||||
Beds on the unit | 15 (12–18) | 15 (10–20) | 15 (12–18) | 15 (11–18) |
Percentage of beds occupied | 95 (92–100) | 93 (82–100) | 96 (86–100) | 90 (78–100) |
Staffing | ||||
Psychiatrist works in the unit | 18 (90) | 17 (89) | 20 (100) | 18 (95) |
Access to psychiatrist | 2 (10) | 2 (11) | 0 (0) | 1 (5) |
Clinical psychologist works in the unit | 10 (50) | 10 (53) | 13 (65) | 12 (63) |
Access to clinical psychologist | 8 (40) | 3 (16) | 7 (35) | 4 (21) |
OT works in the unit | 18 (90) | 17 (89) | 16 (80) | 17 (89) |
Access to OT | 0 (0) | 2 (11) | 4 (20) | 1 (5) |
Nurse works in the unit | 20 (100) | 19 (100) | 20 (100) | 19 (100) |
Support worker works on the unit | 20 (100) | 19 (100) | 20 (100) | 19 (100) |
Access to social worker | 17 (85) | 15 (79) | 19 (95) | 15 (79) |
Turnover | ||||
Percentage of staff turnover, median (IQR) | 15 (10–20) | 11 (6–16) | 12 (7–22) | 10 (7–13) |
Percentage of service user turnover, median (IQR) | 44 (23–80) | 69 (25–120) | 56 (31–89) | 70 (24–90) |
Percentage of service users discharged in last 12 months/ready for discharge but no suitable placement, median (IQR) | 65 (35–90) | 80 (39–145) | 77 (49–103) | 85 (41–113) |
Units who discharged to an out-of-area placement | 4 (20) | 2 (11) | 4 (20) | 3 (16) |
QuIRC domain scores (%), mean (SD) | ||||
Living environment | 74 (9) | 72 (9) | 73 (9) | 73 (9) |
Therapeutic environment | 62 (6) | 65 (5) | 66 (6) | 67 (5) |
Treatments and interventions | 58 (7) | 61 (7) | 62 (7) | 65 (6) |
Self-management and autonomy | 69 (6) | 68 (8) | 70 (7) | 71 (6) |
Human rights | 72 (9) | 72 (7) | 71 (7) | 75 (6) |
Recovery-based practice | 65 (7) | 66 (9) | 68 (6) | 69 (7) |
Social inclusion | 55 (10) | 57 (9) | 62 (13) | 65 (10) |
Service user characteristics
Table 14 shows service users’ characteristics at baseline and at the 12-month follow-up. The majority were male, with a mean age of 43 years, a diagnosis of schizophrenia, schizoaffective disorder or bipolar affective disorder and a median history of contact with mental health services of 15 years. A minority in both groups were rated as having problematic use of alcohol or illicit substances. More of those in the intervention units had previously been treated in a secure unit and more were currently detained on a forensic section, although this was not reflected in differences in the history of risk to others between the two groups. The most prevalent risk was self-neglect, which was reported for around two-thirds of patients.
Characteristic | Baseline | 12-month follow-up | ||
---|---|---|---|---|
Comparison units (N = 211), n (%) | GetREAL units (N = 206), n (%) | Comparison units (N = 174), n (%) | GetREAL units (N = 170), n (%) | |
On unit at baseline | – | – | 73 (43) | 55 (32) |
Sociodemographics | ||||
Male | 134 (65) | 127 (66) | 113 (66) | 121 (70) |
Age (years), mean (SD) | 43 (12) | 43 (14) | 44 (12) | 44 (13) |
White | 157 (75) | 141 (73) | 130 (76) | 129 (74) |
Diagnosis | ||||
Schizophrenia | 137 (68) | 141 (76) | 118 (72) | 126 (76) |
Bipolar affective disorder | 15 (7) | 9 (5) | 14 (8) | 11 (7) |
Schizoaffective disorder | 15 (7) | 14 (8) | 10 (6) | 13 (8) |
Psychiatric history | ||||
Years of contact with mental health services, median (IQR) | 15 (9–25) | 15 (8–26) | 16 (8–27) | 14 (8–24) |
Previous admissions, median (IQR) | 4 (2–7) | 5 (2–10) | 4 (2–8) | 4 (2–8) |
Previous involuntary admissions, median (IQR) | 2 (1–4) | 3 (1–5) | 2 (0–4) | 2 (1–5) |
Length of current admission (months), median (IQR) | 24 (11–48) | 29 (13–86) | 27 (13–59) | 26 (11–80) |
Length of current admission in rehabilitation unit (months), median (IQR) | 9 (4–23) | 14 (5–28) | 13 (5–25) | 10 (5–29) |
Current admission involuntary | 142 (73) | 144 (80) | 116 (76) | 126 (80) |
Currently detained involuntarily | 92 (46) | 117 (65) | 74 (48) | 108 (68) |
Risk history | ||||
Previous high secure admission | 9 (5) | 8 (5) | 4 (3) | 8 (5) |
Previous medium secure admission | 23 (12) | 26 (15) | 13 (9) | 22 (14) |
Previous low secure admission | 41 (21) | 47 (27) | 26 (18) | 55 (35) |
Ever detained on forensic section | 26 (13) | 33 (19) | 20 (14) | 30 (19) |
Currently detained on forensic section | 18 (9) | 28 (16) | 16 (11) | 27 (17) |
Ever been in prison | 39 (21) | 37 (21) | 122 (83) | 109 (70) |
Prison last 2 years | 3 (2) | 1 (1) | 0 (0) | 0 (0) |
No history of violence | 84 (44) | 65 (37) | 66 (45) | 59 (38) |
Violence > 2 years ago | 74 (39) | 78 (44) | 53 (36) | 59 (38) |
Violence < 2 years ago | 32 (17) | 34 (19) | 29 (20) | 36 (23) |
Worst act of violence within last 2 years | ||||
Threatened someone with weapon | 10 (5) | 11 (6) | 14 (9) | 10 (6) |
Assault but victim did not require hospital inpatient treatment | 27 (14) | 22 (12) | 54 (36) | 64 (41) |
Assault and victim required hospital inpatient treatment | 2 (1) | 5 (3) | 4 (1) | 4 (3) |
Homicide | 0 (0) | 0 (0) | 1 (1) | 2 (1) |
Sexual offence ever | 19 (10) | 17 (9) | 13 (9) | 20 (13) |
Fire-setting ever | 27 (14) | 27 (15) | 17 (12) | 26 (17) |
Self-harm ever | 63 (32) | 76 (43) | 59 (40) | 63 (41) |
Recurrent self-harm last 2 years | 6 (3) | 7 (4) | 8 (5) | 3 (2) |
Self-neglect last 2 years | 129 (67) | 107 (61) | 92 (63) | 94 (61) |
Problematic alcohol use | 19 (9) | 12 (6) | 13 (7) | 16 (10) |
Problematic illicit substance use | 11 (6) | 16 (8) | 13 (7) | 7 (4) |
Functioning, activity, challenging behaviours | ||||
Staff consider unlikely to be ready for discharge in next 12 months | 88 (42) | 78 (41) | 70 (41) | 77 (45) |
GAF score, mean (SD) | 52 (7) | 51 (7) | 52 (8) | 51 (9) |
LSP score, mean (SD) | 125 (17) | 123 (20) | 127 (18) | 127 (18) |
Time-use diary score, mean (SD) | 46 (11) | 45 (11) | 47 (12) | 48 (11) |
SPRS score, SD (range) | 1 (0–3) | 1 (0–3) | 1 (0–3) | 1 (0–2) |
Fidelity to the GetREAL intervention
Items from the fidelity assessment for which all units achieved 100% were removed for the purposes of the analysis (Table 15). The mean score for the remaining items was 13 (SD 2). The units in the comparison group were scored 0 on the fidelity measure for statistical analysis.
Criterion | Percentage of units in which criterion was achieved |
---|---|
Pre-disposing stage | |
Pre-disposing meeting held with the unit’s senior team members attended by at least one of the REAL research steering group’s senior psychiatrists (HK, FH, TC) to explain the purpose of the GetREAL intervention and gain senior staff ‘sign-up’ | 80 |
Dates for the first GetREAL training day(s) for unit staff, and release of staff to attend, are agreed with the unit manager before the GetREAL team arrive | 85 |
Unit manager agrees to provide unit keys and, when possible, IT access/e-mail accounts for the GetREAL team OT and activity worker | 90 |
Enabling stage | |
At least two members of the GetREAL team deliver the initial staff training | 100 |
At least 50% of the unit staff attend | 95 |
Initial evaluation forms are completed by all staff attending | 90 |
Action plans agreed for the next 4 weeks | 95 |
GetREAL team work alongside unit staff for at least 5 weeks including the training days | 100 |
At least one structural change/enhancement agreed to facilitate service user activities | 100 |
Any other structural/process changes made secondary to GetREAL team’s suggestions | 90 |
Individual goal-setting (regarding activities) carried out and recorded in care plans for at least 50% of service users on the unit | 50 |
At least two members of the GetREAL team deliver the final staff training | 100 |
At least 50% of the unit staff attend | 55 |
Certificate of attendance (at both training sessions) awarded to at least 50% of unit staff | 70 |
Reinforcing stage | |
At the end of the enabling stage, action plan for the unit to continue the GetREAL work for the next 12 months is agreed | 100 |
Action plan circulated to all unit staff by the GetREAL team | 100 |
At the end of the enabling stage, activity is included in at least 50% of service users’ individual care plans | 80 |
Link person identified to keep e-mail contact with the GetREAL team for up to 12 months | 90 |
GetREAL team members continue offering e-mail contact for 12 months | 100 |
Link person makes contact with GetREAL team at least once during the 12-month period | 45 |
Supervision and support of the GetREAL team | |
GetREAL service user consultants supported by the OTs through face-to-face/e-mail/telephone discussion as required | 100 |
GetREAL activity workers supervised weekly by the OTs during each intervention period | 100 |
GetREAL OTs supervised at least three times per intervention period by SC/TM | 75 |
GetREAL OTs have line management meeting with the employing trust’s (Camden and Islington NHS Foundation Trust) senior OT (GD/LR) once per cycle | 100 |
Primary outcome
A total of 342 participants had a completed staff proxy time-use diary score at follow-up, and 308 had a self-reported score. Only one participant had only a self-reported score, and 35 participants had only a staff proxy score. The agreement between the time-use diary ratings completed by service users and staff was not considered adequate for staff scores to be substituted for scores of service users who lacked capacity (in 23% of the cases the service user rating was at least 5 points higher than the staff rating and in 20% of cases the staff rating was at least 5 points higher than the service user rating). However, as the percentage of missing service user-rated time-use diaries was low at the 12-month follow-up, it was decided (1) not to substitute data from staff but to carry out analyses using the service user and staff ratings separately; and (2) to undertake multiple imputation to impute service user-rated data from staff-rated data and other predictors in a secondary analysis.
Multivariable analysis identified three predictors of missing data that were also associated with the primary outcome: white ethnic group (OR 0.13, 95% CI 0.03 to 0.56), years since first contact with mental health services (OR 1.07, 95% CI 1.01 to 1.13) and GAF score (OR 0.81, 95% CI 0.72 to 0.90). The results of the primary outcome analyses are shown in Table 16 (the comparison unit scores are the reference category). There were no statistically significant differences in primary outcome between groups, even when adjusting for predictors of missing data, length of admission, staff turnover and GetREAL fidelity score. No significant difference was found when only staff-rated time-use diary scores were used or when the service user time-use diary scores were analysed at the unit level rather than as individual scores.
Primary outcome | Coefficient (difference in mean) (95% CI) |
---|---|
Service user time-use diary (complete case) score adjusting for mean baseline unit time-use diary score only | 1.444 (–1.351 to 4.238) |
Service user time-use diary score after multiple imputation adjusting for mean baseline unit time-use diary score only | 1.430 (–1.372 to 4.232) |
Service user time-use diary score, unit level, adjusted for baseline unit mean score and weighted by cluster (unit) size | 1.227 (–1.748 to 4.202) |
Service user time-use diary (complete case) score adjusting for mean baseline time-use diary score and predictors of missingness (GAF score, ethnicity, length of contact with services) | 0.680 (–2.405 to 3.764) |
Staff rated time-use diary score adjusting for mean staff-rated baseline time-use diary score and predictors of missingness | 0.548 (–2.393 to 3.490) |
Service user time-use diary (complete case) score adjusting for mean baseline time-use diary score, predictors of missingness and staff turnover in last 12 months | 0.914 (–2.640 to 4.468) |
Service user time-use diary (complete case) score adjusting for mean baseline time-use diary score, predictors of missingness, length of admission and GetREAL treatment intervention fidelity | –1.378 (–15.090 to 13.154) |
Secondary outcomes
The results of our analyses of secondary outcomes are shown in Table 17 (the comparison unit scores are the reference category). There were no differences between intervention and comparison units in any secondary outcomes.
Secondary outcome | Coefficient (95% CI) |
---|---|
LSP score | –0.905 (–6.533 to 4.723) |
Length of current admission in rehabilitation unit | –7.268 (–18.226 to 3.689) |
Length of current admission | –12.533 (–44.804 to 19.739) |
Staff expectations of patients moving on in next 12 months (OR) | 1.049 (0.473 to 2.326) |
Percentage of patient turnover per unit | 1.064 (–30.457 to 32.585) |
Percentage of staff turnover per unit | –4.390 (–14.863 to 6.084) |
Percentage of discharged/ready to be discharged per unit | 0.107 (–1.746 to 1.961) |
Unit QuIRC domain scores (%) | |
Living environment | –0.103 (–5.521 to 5.314) |
Therapeutic environment | 0.874 (–2.580 to 4.328) |
Treatments and interventions | 2.571 (–1.679 to 6.822) |
Self-management and autonomy | 1.311 (–2.896 to 5.517) |
Human rights | 3.109 (–1.293 to 7.511) |
Social inclusion | 0.591 (–3.609 to 4.790) |
Recovery-based practice | 2.853 (–4.707 to 10.412) |
Discussion
This is the first large-scale randomised controlled trial to investigate the effectiveness of a specific training intervention for mental health rehabilitation unit staff aimed at improving service users’ engagement in activities. Our results showed that the intervention had no clinical advantage over usual care.
We minimised the number of missing data by collecting our primary outcome through face-to-face interviews with patients and by using ratings collected from staff and case notes for those who lacked capacity to participate. Our secondary outcomes were gathered from case notes and staff. Only patients who had capacity and declined consent, and those on leave from the unit and therefore unavailable for interview, could not be included. We adjusted our analyses for predictors of missingness that were associated with the primary outcome and used an intention-to-treat approach. Our researchers were unmasked in 15% of units but, when this occurred, the second researcher was able to collect the 12-month follow-up data to minimise observer bias. Nevertheless, the researchers were able to identify three-quarters of units correctly in terms of whether or not they had received the GetREAL intervention.
As we were assessing all service users present in each unit at 12 months after randomisation, loss to follow-up of individual service users assessed at baseline was not an issue. However, given the potential for changes (and closures) to services, especially at a time of economic downturn in the NHS, we increased the number of units recruited from 31 to 40 to ensure that we had enough units in the study at the 12-month follow-up. This was appropriate, as one intervention unit closed down during the course of the study. The median size of the units (15 beds) was slightly larger than we had expected based on our previous national survey (14 beds) but, despite this, we were unable to gather the 12-month follow-up data on our target 372 patients (186 per trial arm). This appears to have been due to fewer service users being assessed as lacking capacity (such that data could be gathered from staff) at follow-up, and to more service users who had capacity refusing consent. Although the study was somewhat underpowered owing to the higher observed intracluster correlation coefficient and the slight under-recruitment of service users into the study, the loss of power was compensated for by the smaller actual cluster size (8.8 rather than the 12 we included in our sample size estimate) and through the use of a more efficient analysis at the service user level that adjusted for baseline scores. The supporting unit-level analyses concurred with the results of the primary service user-level analysis. Our results, therefore, appear robust.
There are a number of possible explanations for the lack of effectiveness of the intervention. First, although the fidelity scores for the units that received the training were relatively high, very few units made spontaneous contact with the GetREAL teams once the teams had left the unit. This may suggest that the unit staff did not continue to use the techniques they had practised while the GetREAL teams were with them. It is possible that we would have found a difference in service user activity if we had assessed this at the end of the enabling stage but, as the aim of the intervention was to upskill staff and embed changes in practice into the unit, we chose to assess outcomes 12 months after the GetREAL teams had left. It is also possible that the training benefited some service users immediately and that those who became more active were discharged and replaced by less active, more impaired people by the time of the 12-month follow-up.
Underlying cultures in health-care settings are difficult to change, and reverting to previous practice may, therefore, explain why our intervention failed to show effectiveness at 12 months. A longer period may be required to improve organisational structures, train unregistered staff and change ‘a culture of long-term custodial care to one of active treatment and rehabilitation’. 82 In addition, a more intensive reinforcement process may have sustained the intervention (e.g. through regular supervision with the link person identified to oversee the action plan in each unit). 57
The study was carried out during a time of turbulence in the NHS. The economic downturn meant that many units faced pressure to increase productivity and reduce costs. Unit managers and staff may, therefore, have found it impossible to continue to implement the GetREAL practices in the context of competing priorities.
Another possible explanation for our results is that the service users were too severely impaired in functioning to benefit from the intervention. Their mean time-use diary scores were lower than those of other patients with a shorter duration of psychosis83 and the severity of our study participants’ problems is also reflected in their lengthy contact with mental health services and low mean GAF scores. The study may therefore have been too ambitious in expecting a specific intervention to be able to improve engagement in activity. A more complex intervention that incorporates a number of evidence-based interventions for the treatment of psychosis in addition to the GetREAL intervention might, therefore, be more effective. However, the units in this study generally had the appropriate range of staff to deliver the evidence-based interventions recommended for people with psychosis,7 and units scored relatively highly on the treatment and interventions domain of the QuIRC that incorporates these routine interventions. Nevertheless, our results from phase 1 of the REAL study highlighted the low level of evidence-based psychological interventions being used.
A further possibility is that because the intervention units had a higher service user turnover than the comparison units, staff repeatedly had to accommodate a higher level of morbidity among new admissions and were unable to adjust their approach to engaging their service users in activities, thus preventing the intervention from having an impact. In addition, our primary outcome was based on the assumption that increased service user activity is beneficial, whereas it may be that those with severe symptoms found it overstimulating and avoided the activities staff were trying to engage them in.
Finally, we know that the quality of care in English mental health rehabilitation units is higher than that in other countries84 and, although we included units in the trial that scored below the median on quality across England, the lack of effectiveness may represent a ceiling effect.
Conclusion
This rigorously designed, cluster randomised controlled trial failed to detect evidence of effectiveness of a specific staff training intervention aimed at increasing service user engagement in activities in inpatient mental health rehabilitation units. Further trials of interventions that take account of these results are needed for the treatment of people with severe and complex psychosis.
Phase 3: health economic component
Phase 3 also included a health economic component to investigate the cost-effectiveness of the GetREAL intervenion.
Cost-effectiveness of the GetREAL intervention
The number of service use contacts was assessed using an adapted version of the CSRI. 59 This recorded specific care inputs [type of care, professional providing it, number and duration of contacts received by each service user over the previous 4 weeks (or since admission to the unit if < 4 weeks previously)]. In addition, the CSRI recorded details of services used outside the rehabilitation unit during the same period. Unit cost data from Curtis58 were used to obtain an estimate of the perceived cost of these contacts. When possible, information from a study on inpatient care in the UK was used to estimate the mean duration of service use. 85 Otherwise, it was assumed that contacts lasted for 30 minutes on average. Average group size was also estimated from this same study, which investigated service use in inpatient psychiatric wards,85 and unit costs for individual attendees at group sessions were divided by five.
In addition, the cost of the intervention was estimated by combining the cost of the time spent by the GetREAL team on delivering the intervention and the opportunity cost of the staff attending training sessions. The cost-effectiveness of the GetREAL intervention was determined using an incremental cost-effectiveness ratio. Specifically, the cost of achieving an extra 1% of time spent in activities was calculated. To account for both clustering of the data and correlation between costs and outcomes, a bivariate multilevel model was estimated. The uncertainty around this estimate was assessed using a cost-effectiveness plane and a cost-effectiveness acceptability curve.
Results
As reported above, there were no statistically significant differences between intervention and comparison units in primary or secondary outcomes. The difference in cost of perceived service use in the intervention units and comparison units was also not statistically significant (coefficient £195, 95% CI –£49 to £440). The average cost of the intervention was estimated to be approximately £102 per month per patient (Table 18).
Travel and accommodation cost | Days spent at each unit | Reimbursement per day (£) | Product (£) |
---|---|---|---|
OT | 25 | 100 | 2500 |
Activity worker | 25 | 100 | 2500 |
Patient expert | 3 | 100 | 300 |
Salary cost | Days spent at each unit | Cost per working day (£)a | Product |
OT | 25 | 188 | 4689 |
Activity worker | 25 | 111 | 2766 |
Patient expert | 3 | 111 | 332 |
Opportunity cost of staff attending training sessions | Number of staff-hours trained | (Weighted) cost per hour (£) | Product |
Nurses | 48 | 39 | 1871 |
Support workers | 57 | 21 | 1189 |
OT | 8 | 36 | 278 |
Other | 9 | 85 | 875 |
Total (£) | 17,300 | ||
Average number of occupied beds | 14.2 | ||
Assumed longevity of treatment in months | 12 | ||
Average cost per patient per month (£) | 102 |
An overview of the differences in the number of contacts with specific staff is shown in Table 19. With the exception of the cost of contacts with nurses, the costs between the two groups were comparable. The incremental cost of an extra 1% increase in activity levels was £101.
Staff type | Comparison units | GetREAL units | ||||
---|---|---|---|---|---|---|
Percentage using service | Mean (SE) contactsa | Mean (SE) costs in £b | Percentage using service | Mean (SE) contactsa | Mean (SE) costs in £b | |
Psychiatrist | 48 | 1.9 (0.1) | 107 (9) | 47 | 2 (0.1) | 105 (9) |
Other medical specialist | 21 | 2 (0.2) | 44 (6) | 25 | 2 (0.1) | 50 (6) |
Clinical psychologist | 12 | 2.9 (0.3) | 22 (4) | 12 | 3 (0.2) | 18 (3) |
OT | 25 | 6.9 (0.6) | 9 (1) | 32 | 8 (0.6) | 14 (2) |
Social worker | 8 | 1.7 (0.2) | 16 (4) | 11 | 2 (0.2) | 21 (4) |
Counsellor/psychotherapist | 1 | 2.7 (0.7) | 1 (1) | 1 | 3 (0.4) | 2 (1) |
Volunteer | 4 | 2.7 (0.6) | 1 (0) | 4 | 4 (0.4) | 1 (0) |
Arts therapist | 1 | 3.3 (0.8) | 1 (0) | 2 | 4 (1.1) | 1 (0) |
Care co-ordinator | 11 | 1.9 (0.2) | 6 (1) | 14 | 2 (0.2) | 8 (1) |
Other | 4 | 2.6 (0.5) | 6 (2) | 5 | 2 (0.4) | 5 (2) |
Nursec | 99 | 8.3 (0.3) | 614 (24) | 100 | 9 (0.4) | 690 (30) |
Support workerc | 81 | 7 (0.8) | 147 (16) | 85 | 7 (0.7) | 145 (14) |
Total | 973 (35) | 1059 (39) |
The cost-effectiveness plane suggested that most replications were situated in the north-east or the north-west quadrant (Figure 5). It was unclear what value decision-makers would place on increasing the level of activity in rehabilitation wards, but the cost-effectiveness analysis suggested that a willingness to pay of > £100 per percentage point increase in time spent on activity would be required for the intervention to be more likely to be cost-effective than not (Figure 6).
Discussion and conclusion
The economic analyses suggested that the GetREAL intervention was more likely to increase costs, but the difference in costs between intervention and comparison units was not statistically significant. In addition, it should be noted that an increase in the number of nurse contacts need not necessarily mean an increase in service costs as implied by our calculation. It may be that, as a result of the intervention, nurses spent a greater proportion of their time interacting with service users but this would not actually be associated with extra service costs in a real-life situation. Our cost-effectiveness analysis suggested that a willingness to spend > £100 for each percentage point increase in time service users spent engaged in activity would be required for the intervention to be more likely to be cost-effective than not. In other words, it is equivocal whether or not the intervention can be considered cost-effective. However, bearing in mind the possible explanations for the lack of effectiveness of the intervention detailed in the previous section, if the intervention were to be refined such that it enabled a small increase in service user activity, it would be more cost-effective.
Phase 3: qualitative component
The qualitative component of phase 3 was led by Professor Gerard Leavey. The aim was to investigate staff and service user experiences of the GetREAL staff training intervention. There were two main objectives:
-
to provide contextual information that could inform our interpretation of the quantitative results of the cluster randomised controlled trial that assessed the efficacy of the GetREAL intervention
-
to identify the barriers to and facilitators of successful implementation of the GetREAL intervention.
Methods
Recruitment and data collection
We purposively sampled 10 units that had received the GetREAL intervention on the basis of four characteristics: type (hospital or community based), location (urban, suburban, rural), size and which of the two GetREAL teams had worked with them. All front-line staff of any discipline who worked on these units were eligible to participate. We also carried out one-to-one qualitative interviews with one or two service users from each of these units who had been inpatients at the time the GetREAL intervention was delivered. We recruited service users to reflect a range of sexes, ages and lengths of stay on the unit. Written informed consent was gained from all participants prior to interview.
Separate topic guides for staff focus groups and service user interviews were developed in consultation with members of the PMG and SURF. The staff focus group topic guides included specific questions relating to the three stages of the GetREAL intervention (predisposing, enabling and reinforcing), such as the process by which staff were informed of the fact that they were going to receive the training, their views on the content and usefulness of the training day in the first week of the enabling stage, the specific interventions used to engage service users in activities, and whether or not these continued to be used after the GetREAL teams left at the end of the enabling stage. We sought to clarify staff views on the particular strengths and limitations of the intervention, its delivery, implementation and sustainability (see Appendix 9). With regard to service user interviews, the topic guide focused on their recall of the GetREAL team’s work in the unit, whether or not they were able to give examples of specific changes to the activities they were offered during and since that time, and their general views on the impact of the intervention (see Appendix 10).
The focus groups and service user interviews in the first two units were facilitated by GL and HK, with ML and a medical student assistant, Henrietta Gordon, in attendance. Subsequent focus groups were facilitated by ML and cofacilitated by Henrietta Gordon. The focus groups were conducted in a quiet and confidential space within the unit, and we endeavoured to ensure that there were minimal interruptions. The groups were led by the facilitator, while the cofacilitator made observational notes. The service user interviews were conducted by ML only. The focus groups and service user interviews were digitally recorded and transcribed by ML. Transcriptions from the first four units were reviewed by GL to ensure that the interviews were being carried out consistently. Although the topic guides set out a clear structure for the interviews, ML was trained so she could allow flexibility in allowing interviewees to introduce unanticipated issues for exploration. Each staff focus group lasted approximately 60 minutes and each service user interview lasted approximately 30 minutes.
Data analysis
Data analysis was carried out by ML under the supervision of GL and HK. The transcripts were imported to specialist software (Atlas.ti 7) for analysis.
We adopted a similar approach to data analysis as in phase 1, using a standard coding and thematic procedure. The topic guide was used as the initial coding frame covering the main areas of interest. Further codes and themes that were detected in the transcriptions were added. An analytical framework was developed to organise the data into themes. The coding and the emergent themes were discussed in depth by ML, GL and HK at regular meetings held throughout phase 3. These included consideration of the inter-relationships between themes.
Participants
Focus groups were conducted with staff of 10 units between 2 and 9 months (mean 6 months) after the GetREAL team had completed the enabling stage of the intervention and left the unit. These units had a mean 19 beds; half of the units were community based and half were hospital based. Four were located in the inner city, three were in suburban areas and three were in rural areas. Five units had received the intervention from one GetREAL team and five had received it from the other GetREAL team. The focus groups had an average attendance of six staff members, with a total of 58 staff participating (three unit managers, 25 nurses, 19 support workers, seven OTs, one clinical psychologist, one food technician, one activity worker and one student nurse).
A total of 12 service users were recruited for interview from seven of the 10 units (four community and three hospital units). Seven were inpatients on units that received the staff training intervention from one GetREAL team and the other five were on units that had received the GetREAL intervention from the other team. Descriptive characteristics were available for only nine (as three declined the researcher access to their case notes). Six had a diagnosis of schizophrenia, one had a diagnosis of schizoaffective disorder, one had a diagnosis of personality disorder and one was categorised as ‘other’ diagnosis. Six were detained under the Mental Health Act (two on Section 37/41, three on Section 3 and one on Section 37). 42 Three were voluntary patients.
Findings from the staff focus groups
From the staff focus groups, five main themes are presented:
-
the functioning of the unit prior to the intervention; difficulties in engaging service users in activities
-
staff preparation for, and expectations of, the GetREAL intervention
-
the experience of the GetREAL intervention
-
maintaining changes in practice in the reinforcing stage
-
barriers to sustaining changes to practice.
Main themes
The functioning of the unit prior to the intervention; difficulties in engaging service users in activities
Staff commonly reported that they found it difficult to engage service users in activities. On occasion, this difficulty to engage was directly related to service users’ fluctuating mental health, particularly as a consequence of negative symptoms. As we shall elaborate in later sections, this had a demoralising impact on staff motivation:
People’s mental states on an inpatient unit are very up and down, so you could have a breakthrough one week, and then the following week the same service user is back to square one again, so it’s hard. And I suppose you, after time, you do start feeling demotivated yourself – you feed off one another.
OT, unit 0902
Well half the problem is overcoming the initial resistance in doing anything – with negative symptoms especially.
Unit manager, unit 3704
We noted a view among nursing staff that this client group was difficult to motivate. The following dialogue between staff in one unit (4204) illustrates this:
We’ve got the things there, the groups there, but a lot of them [service users] don’t want to engage.
. . . and you can’t force them either, you know, it’s their choice isn’t it? As much as it’s a rehabilitation unit and we would like to, we can only encourage them can’t we?
Sometimes we’ll have a film afternoon you know and it just ends up the staff, nobody else [laughs].
Staff preparation for, and expectations of, the GetREAL intervention
Some staff had not received much information about the GetREAL intervention prior to the team’s arrival on the unit, and therefore had few expectations of what the intervention would involve. This may have been because unit managers had been told to give minimal information to staff in order to prevent the researchers being unmasked as to the unit’s allocation as intervention or comparison (the researchers collected baseline data at each unit involved in phase 3 within the 4 weeks prior to the GetREAL team’s arrival).
From a personal point of view I didn’t have an idea about what the intervention would be, so it’s difficult to comment.
Staff nurse, unit 0804
I think there was a bit of confusion, because of the randomised controlled trial and us being told not to say things about who’s having an intervention, that was a bit confusing when one of the researchers were here and I knew, I did know at that point and I was like ‘oh no I mustn’t mention this’ [group laughs] and he was like ‘don’t mention that’, and I was thinking, ‘what shouldn’t I mention? Anything or . . .’ but obviously they’ll know because they’ll come and look at the board and that’s the GetREAL intervention, and you know so I think that could have been more clearly written down – when you see so and so, don’t mention anything about . . . you know as explicitly as that, ‘cause if you don’t know what you’re not meant to say, you could say the wrong thing.
Staff nurse, unit 2902
Overall, staff seemed positive about the arrival of the GetREAL teams:
I expected a lot because there was nothing happening on this ward.
Staff nurse, unit 0804
. . . a sense of optimism for unit . . .
Unit manager, unit 3704
A different approach to thinking, to working and to patients.
Activity worker, unit 0502
Nevertheless, we noted a degree of ambivalence among staff about the GetREAL team’s intentions. A few staff seemed to have felt threatened by the team’s arrival:
I think I felt they were just going to come in and sort of um tell us what we weren’t doing and what we should be doing.
OT, unit 4203
You know when you have a stranger coming in, you feel a bit threatened, you know, and you feel like you are being monitored but you soon got used to that yeah because they were very friendly, very approachable.
Staff nurse, unit 0804
The experience of the GetREAL intervention
To varying degrees, most staff felt that the GetREAL teams easily assimilated into their units, and reported positive experiences of working with the team.
They’ve come in, and just took on board how we work and slotted in.
Unit manager, unit 0502
They were just very supportive, they were always there to help us if there were anything that we were interested in doing.
Health-care assistant, unit 2902
They were really hands-on and they, I think they did fit in really well, very quickly. But they also . . . knew where they work . . . so like part of the team but separate, and I think that was really very professional.
Clinical psychologist, unit 0102
However, this positive regard was not universal; staff on one unit did not view the GetREAL team as part of their team during their time on the unit:
I felt they were quite external . . . the nursing staff on here do 12-hour shifts, so for someone to pop in for a couple of hours, you never felt as though they were part of the nursing team ‘cause we’re here for 12.
Charge nurse, unit 4203
Many staff suggested that the GetREAL teams’ work was associated with an increase in service user engagement with activities. On one unit, the degree to which service users had started to engage in community activities was striking:
Sometimes there’s nobody in the building to do the groups with . . . they’re all out doing stuff.
Staff nurse, unit 2902
Seven subthemes emerged from the staff perspective that appeared to underpin the processes by which the GetREAL intervention facilitated these changes. These appeared to interact in a dynamic way rather than representing a linear process. These were (1) the opportunity for reflective practice as a team; (2) questioning existing structures and processes; (3) providing a more person-centred service; (4) giving permission to change working practice; (5) building staff confidence; (6) broadening staff responsibilities to include service user activities; and (7) motivating staff and service users.
A sense of defeatism or of being ‘stuck’ in the rehabilitation units permeated the focus group discussions, whereby staff conveyed the perception that service users were often unable to progress and they, the staff, felt trapped and ineffective. The work of the GetREAL teams appeared to lead to staff members reflecting on the way they provided, and engaged service users in, activities. Staff also valued the opportunity that the intervention provided them for thinking together as a team about their approach.
It had an enormous benefit in that it were joint working and collaborative working and that it was going to bring everyone together, as one team, working in one direction.
Occupational therapy instructor, unit 2902
We all met as a team then and we discussed a plan as to how we’re going to achieve that in the short term and long term . . . to discuss the strategy, how we are going to get there, how we are going to achieve it, so I think that was a good opportunity for all of us to exchange ideas and think how can we improve the service. I think that was good . . . to think about it and reflect what we’ve done at work and what we could have improved on or do better.
Staff nurse, unit 0102
. . . it’s probably the first time that we’ve been able to get together totally as a team since, well after 2 years, so it was really helpful that that was included as part of the study.
Clinical psychologist, unit 0102
The chance to plan and reflect on areas of need also provided the opportunity for staff to put structures in place that could facilitate meeting these needs. For instance:
. . . after that [the intervention], we came up with a kind of protocol for therapeutic outings, so that it was clear, how much money could be afforded for therapeutic outings and in fact, we incorporated the option to have a therapeutic outing that was completely of a client’s own choice, it didn’t have to fall into a certain sort of category of you know and they can say well I would like to do this and we had an allowance that the client could claim to do an activity. And that still stands really, and that was a new, that came out of REAL. And obviously we had to limit it to a couple of times a week maximum. But most clients haven’t take much advantage of it, but some have, you know, have used it to very great effect to do their own thing.
Unit manager, unit 3704
As well as promoting the importance of reviewing structures and processes that could enable service user activities, the GetREAL intervention also seemed to allow staff to become more flexible in their approach. For instance, staff reported that they no longer necessarily provided groups or activities at certain times, but were better able to respond to service users’ needs and wishes for specific activities.
We are more flexible now, whereas before we were kind of more rigid you know so rather than thinking oh no we have to wait until this time and that day to do [activities]. So we use our time a lot better now.
Staff nurse, unit 0102
Some aspects of working arrangements were not open to renegotiation with staff. For instance, a ward manager described staff members’ reluctance to work flexible shift times that would allow more meaningful activities for service users, such as going to the cinema:
They get into the groove and they kind of build their life around the shift system. They don’t want to do a 9 to 5 or a 12 to 8 so they could go to the pictures or something.
Unit manager, unit 3704
The GetREAL teams seemed to bring an objective, outside perspective that helped staff to see the potential to deliver their service differently. Staff cited specific examples of how practices such as cooking and laundry had been modernised as a direct result of the intervention:
The morning shift had to get everybody’s laundry done in that shift . . . sort of at our convenience. Why does it have to be like that? Why can’t we be a bit more flexible? And it was just having some fresh eyes coming in and seeing.
OT, unit 2902
This external perspective provided by the GetREAL teams, and the process of reviewing practice, allowed staff to reflect on whether or not they were really meeting service users’ individual needs.
[The unit] was bad at looking at people as individuals. It was like well this is how we’ve done it and we’ve done this for years and yeah we ain’t gonna change.
OT, unit 2902
Some patients are harder to engage, but . . . now I just try harder to find something that they like, that will kind of motivate them, personally, like more individually rather than as a whole, trying the same thing for everybody, but now I try to really pick something that will get them like ah, OK!
Support worker, unit 3704
They [the GetREAL team] also showed us ways of talking one on one to see what they actually like. It’s alright us putting groups on, but if they don’t like them . . .
Nursing assistant, unit 4204
The GetREAL team appeared to play an important role in providing permission for staff, particularly support staff, to move beyond the perceived confines of their role.
I’m coming as a band 2, band 3 – it felt like we weren’t allowed to do anything . . . ‘cause it was their job role to do all that, but whereas now it feels we’re able to do it.
Health-care assistant, unit 2902
It’s given the nursing assistants sort of carte blanche to do more and to get involved more to make a difference.
Staff nurse, unit 2902
Staff also referred to the hierarchy within the service that acted as an important constraint on what they felt permitted to do.
For a lot of health-care assistants, there’s issues of power and who actually gives permission . . . and I think, depending on how the structure works, the hierarchy works, it perhaps is hard for some people to feel like actually I think it would be really good to do swimming, to do whatever. But am I? Is that alright on this shift?
Staff nurse, unit 2902
Activity workers particularly benefited from the developing role and greater sense of confidence and I think with that has also come much greater initiative in terms of setting up certain activities and ways of working with our patients.
Consultant psychiatrist, unit 0804
It’s given us more confidence to know we can take them out, where personally I didn’t feel that I could beforehand.
Health-care assistant, unit 2902
One senior member of staff noted a sense of increased confidence and agency among the team in relation to activities.
I think they did help with structuring and how to go about it, but I think with some people they gave them the confidence and the encouragement as well to go ahead with it rather than, just an idea, they kind of helped them put it into action. Staff feel, not that they couldn’t before, but I think there’s a renewed confidence in saying – I’d like to do this kind of group, how do I go about it? – and then before you know it, within 3 or 4 weeks, the posters are up, the group’s sort of set up and away it goes.
Staff nurse, unit 3106
It’s been a process that has enabled a number of key people within the team to be confident in terms of developing an innovative, professional repertoire of behaviours and activities within the unit actually and for the team to feel comfortable with that. It’s a big difference.
Consultant psychiatrist, unit 0804
A number of staff reported that the intervention reminded them that the promotion of service user activities was an important task for the whole team:
Yeah it just reminds everybody that it’s an MDT responsibility, not just the OT’s.
Charge nurse, unit 4203
Prior to the GetREAL team coming, the occupational therapy staff tended to run those groups and I think there was perhaps a view from the nursing staff that if they were doing a group it was almost seen as an add-on extra. And I think what the GetREAL team was trying to say was ‘no, it’s not an add-on extra, this is part of everybody’s role’.
Occupational therapy instructor, unit 2902
Staff also spoke of the benefit of having an ‘identified lead for an activity’, which represented an important change in accountability for the provision of activities:
I think now that we have also identified who is leading on a particular activity so that they will be answerable if it takes place or not. Rather than before we used to just diarise ‘today is bingo’ but nobody is really taking [it].
Staff nurse, unit, 0902
Staff reported an increased sense of motivation among the team during the period in which the GetREAL teams were working with them:
There was a renewed enthusiasm . . . it did make people think I could do something and I could start it and did.
Staff nurse, unit, 3106
Staff felt that the GetREAL intervention empowered and motivated not only them, but also their service users. This, in turn, had a positive effect on staff morale:
We see them happy as well, makes us happy if we see them engaged, yes it gives us satisfaction.
Health-care assistant, unit 0902
Maintaining changes in practice during the reinforcement stage
The data from the focus groups suggested that, once the teams had left, it was difficult to sustain the positive impact of the GetREAL teams on practice regarding service user activities.
[The GetREAL team] was like a brief period of an injection of, it was quite intense for a short period of time, but, the outcomes in that immediate time were that people were quite enthused and I think, there’s quite a few other cases of spun off with their own ideas and coming up with groups and being enthused to carry it on, and putting it on the board and using their own interests, like a dancing group. That was encouraged so, for that period of time . . . and you know we are to continue it, but some of these groups are still continuing with the white board.
Staff nurse, unit 2902
I think the energy that the intervention brought to the team lasted a few months.
Unit manager, unit 3704
If they [the GetREAL team] stayed longer and all of these people [ward staff] stayed out of the office, it’s been fantastic, but these people now have all gone back to their own little positions, and us little people are still left on the floor trying to muddle around to get it right and it is hard for us.
Nursing assistant, unit 3301
As part of the enabling stage, the GetREAL team trained and supported ward staff in the use of tools specifically designed to help encourage communication and discussion with service users about meaningful activity. It appears, however, that staff did not continue to use these tools as part of routine practice after the GetREAL team left:
Yeah, I haven’t seen that [the activity wheel tool] used in a long time. I think at first, but I don’t think it’s, in my experience I haven’t maintained it.
Staff nurse, unit 3106
I think some people have [used the activity tools]. If I’m honest, I haven’t.
Staff nurse, unit 4204
They left some [activity tools] with us yeah. I think they’re in the activity cupboard.
Unit manager, unit 3704
. . . the flashcards, that was very good and positive . . . we have used in most [service users]. I don’t know whether there’s a bit of a slow in terms of going back what we were doing in the initial stages, because it may be some kind of . . . bit of fade somewhere a little bit, because they would have gone through a bit of um, a very busy period, let’s put it that way [laughs] . . .
Staff nurse, unit 2902
An important component of the reinforcing stage of the intervention was the unit’s implementation of the action plan they had drawn up and agreed with the GetREAL team. However, the focus groups indicated that staff rarely referred to their plan:
We did talk about having an activity, not necessarily supervision, but discussion on a sort of fairly regular, every 2 months or whatever basis, to look at how we were doing on the action plan. So that was one way of trying to follow it, like you say and make sure everyone has got it. That hasn’t happened, but you know . . . I think annual leave . . .
OT, unit 2902
I meant to print it out, because I hadn’t gone back to look at that for a long time, and after the training, we did go back to it a couple of times, and tweaked it, but we haven’t done in the last 4 months, 5 months at all. But again, it just slipped off the agenda. That’s what happens.
Unit manager, unit 3704
They left us a box of tools didn’t they and a nice file with all the activities planned and where we could go and where we could find them and how to, you know do them sort of thing. There is a file for that in the office. Is it still in the office?
Nursing assistant, unit 4204
Only one unit out of those that took part in the focus groups reported that they were continuing to use their action plan and referred to it in the focus group interview:
. . . so really what we’ve done is kept updating the ‘to-date’. We’ve not changed any of the agreed aims, agreed actions or by whom; it’s just this column [points to it] that’s updated. And we spoke about it this morning just very briefly and there are individual things that are ongoing, but are individual to the patients, which we didn’t feel we should put in there because it was say, individual, not everybody’s doing it, but it does show that we’ve achieved over and above . . .
Charge nurse, unit 4203
There also appeared to be some resentment about how the link person for the reinforcing stage of the intervention was chosen:
Well we got volunteered didn’t we, I mean, the OT and I ended up leading it and we were saying – oh when did we volunteer for that then? – we didn’t know that we were leading it did we until this came [action plan] and it was like, by who? And it was oh, oh it’s us?
Charge nurse, unit 4203
When it came to right who wants to lead these things, there was a bit of silence from those people. And I found that interesting, but it did make people think and make people take initiative partly didn’t it.
OT, unit 4203
Furthermore, in a number of units staff were not aware of who their designated link person was, despite noting that the implementation of the action plan would have benefited from such a lead person, which suggested that they had not read the action plan:
I don’t know, maybe if somebody was responsible for reviewing it, but that would still be, like if you had a lead person, but then they’d have to have an interest . . .
Staff nurse, unit 0102
The follow-up e-mail contact offered as part of the reinforcing stage of the intervention did not appear to be used or considered useful by staff.
The main contact is like e-mail only and it’s something that you’d need someone in person to kind of facilitate a group with you and just give you a different idea of how, you know what if you try going about it this way.
Staff nurse, unit 0102
If we’d have says you know, would they have come and got us more members of staff or come and done more work with, what would they have done, to be honest?
Nursing assistant, unit 3301
Barriers to sustaining changes in practice
Four subthemes emerged that appeared to be important in understanding why the staff found it difficult to sustain in practice those changes that they had appeared to embrace enthusiastically during the enabling stage, when the GetREAL teams had worked with them. These subthemes were competing priorities, roles and responsibilities, service user factors and misunderstanding the aims of the intervention.
Many staff reported that they had to spend a lot of their time completing paperwork and other tasks, and did not have sufficient time to engage in face-to-face activities with service users.
Well I don’t particularly enjoy it, I’d sooner be with patients than sat filling forms in myself, but it just justifies a moment in time, with all the cutbacks in all this country, you’ve got to justify what you’re doing, and it just makes you feel better that it’s another job you’re doing.
Staff nurse, unit 0502
We can only afford something like 10 minutes per shift, because there are other tasks to do, so . . . we have limited time in order to give them that quality time . . . they can approach a staff anytime and say sit down while you have a chat, and that’s fine but then you have to go back and do other things.
Staff nurse, unit 0902
Many staff saw paperwork as a priority because they knew that their performance would be questioned if it was not completed.
You wouldn’t get pulled up on what activities you’re not doing, you get pulled up on your paperwork that’s not filled in, or you haven’t ticked these boxes.
Staff nurse, unit 3106
Activities were often viewed as less important than other tasks (such as medication administration) and were likely to be abandoned when the unit was short of staff.
The needs of the ward, with staffing we’ve got, comes before activities.
Unit manager, unit 3301
I think we still lack a good strong skill base for that very, very basic motivating work with clients . . . there are staff on the team that are very, very, very good at that and others who just don’t seem to . . . and unfortunately I think it’s a slightly attitudinal thing. That it’s not the most important part of their job really.
Unit manager, unit 3704
Despite the enthusiasm for the changes in practice while the GetREAL team were working in the unit, some nursing staff appeared to revert to a previous position about their remit once the team left, whereby they did not consider engaging service users in activities to be their responsibility.
We’ve got to do our own job and then try to fit other people’s jobs in.
Staff nurse, unit 3301
If we nurses do all this, what would be the role of the OT . . . If my role just covered activities like an OT, then that would be easy.
Charge nurse, unit 4203
Even one unit manager felt that there was no capacity in the team to focus on service user activities:
Nobody’s actually took over where they [the GetREAL team] left off. Because I mean, each and every one of us, have got enough on our job role as it is.
Unit manager, unit 3301
Similarly, some of the previous defeatism centring on the severity of service users’ symptoms and the difficulty in engaging them in activities seemed to have re-emerged once the GetREAL teams left.
I think a lot of it depended on their mental status, the clients you’re working with. You could work with those who are interested to do that particular activity, you know you could find two or three people interested, but I think mental state . . . is probably the reason why some of the activities failed in the first place.
Staff nurse, unit 0902
You need that consistency with the clients wanting to engage in it really. And getting something out of it . . .
Staff nurse, unit 3106
Staff also felt that their relationships with their service users were impeded by the power imbalance inherent in inpatient mental health settings:
As the primary nurse or care co-ordinator of your clients, they mostly sometimes try to choose what they say to you or how they engage with you because they think this might affect them . . . I used to have a more close relationship when I was like a support worker with clients, because they think of, ‘I can say anything to him, it can’t affect me’, but as a qualified nurse now they’re a bit distanced, they can’t say anything to me, he’s going to document that, it’s going to affect me and he’s going to say that in ward round.
Staff nurse, unit 0902
It was also suggested that the service users who had been present on the units while the GetREAL teams had worked there were unrepresentative of the unit’s usual patients, who were not as easy to engage:
When they [the GetREAL team] came, they were dropped into a golden opportunity when we did have a fair few clients that you could motivate.
Unit manager, unit 0502
We noted that the aims of the enabling stage of the intervention may not have been fully communicated to or understood by all staff. Some staff appeared not to have understood that it aimed to increase their confidence and skills in engaging service users in activities and to embed these new skills into their future routine practice.
So probably since they last came here, we’ve had eight new people in, who have not had that involvement from the GetREAL service and started off on that kind of train of thought really.
Staff nurse, unit 3106
Some staff felt that the enabling stage was too short to have been able to change practice, and did not appear to realise that they needed to continue to invest in the process to ensure that change was sustained:
There was the stumbling block of getting ideas achieved in 5 weeks.
Unit manager, unit 0102
There was also some evidence of misunderstanding of one of the key concepts of the intervention, that of using a personalised approach to engaging service users in activities that are meaningful to them, rather than a ‘one size fits all’ approach.
Loads of people say, I’m not doing jigsaws, I’m too old to be doing jigsaws, I’m a man . . . I don’t want to draw. Some clientele love to do activities, and love to participate, whereas you’ve got a patient like [name removed] who’s my patient, lord knows I’ve tried with him, and he thinks he’s not ill. I’m not doing jigsaws! I’m not painting! So I think you know, no matter how much you try sometimes, it’s very, very . . .
Staff nurse, unit 0502
Summary of main findings of phase 3 staff focus groups
The focus groups with staff of units that had received the GetREAL intervention provided useful contextual information to assist our understanding of the quantitative findings of the cluster randomised controlled trial that assessed the intervention’s efficacy.
The following themes emerged from the data:
-
The functioning of the unit prior to the intervention and difficulties in engaging service users in activities influenced the degree to which the service was receptive to the aims of the intervention.
-
Staff expectations of the GetREAL intervention were not always clear at the time the intervention teams arrived, leading to some difficulties in engaging the team quickly.
-
Staff experience of the GetREAL intervention was generally very positive and a number of benefits were noted:
-
the opportunity for reflective practice as a team
-
questioning existing structures and processes
-
providing a more person-centred service
-
giving permission to change working practice and step outside usual roles, building staff confidence
-
broadening staff responsibilities to include service user activities
-
motivating staff and service users with regard to activities.
-
However, there were considerable barriers to sustaining the changes to practice, including:
-
competing priorities and demands on staff time
-
resistance to changing usual roles to promote responsibility for activities among the whole team
-
reports that some service users were too unwell to engage
-
a fundamental misunderstanding by some staff about the fact that the intervention aimed to provide staff with the skills and confidence to engage service users in activities after the intervention team left.
Findings from the service user interviews
From the interviews with service users, four themes emerged: experiences of the GetREAL teams; impact of the GetREAL teams on unit activities; service users’ interest in activities; and insights into difficulties facing staff in facilitating activities after the GetREAL team left.
Service users’ experiences of the GetREAL teams
Most service users’ recollection of the GetREAL teams was poor:
To be honest I can’t really remember them.
Service user 1, unit 0902
I don’t remember them at all.
Service user 1, unit 0804
However, some recalled the specific activities that they did with the GetREAL teams:
I remember doing some photography.
Service user 1, unit 2902
They tried to organise activities, um get us involved in the running of the unit, um, just basically that really, getting us involved in different activities.
Service user 2, unit 3106
This service user then went on to describe the benefit that the GetREAL team provided to their unit:
[Having the intervention team on the ward was] a good thing because it got people motivated to do different things, instead of just sitting in front of the telly [laughs].
Service user 2, unit 3106
. . . yeah, you see it was good, because they weren’t from here, and we helped them to get information. They’d say to us, where’s the library, and if we didn’t tell them, one of us would go with them. And it were like gaining um, a friendship, do you know what I mean?
Service user 1, unit 3106
Another service user spoke of how the GetREAL team worked to increase service user-led activities on the unit:
. . . she’s um, doing nails and massage. She’s been trained to do it, but she’s a patient. I think that’s what them two ladies were trying to get us more to do when they came actually. Getting more things, um, us patients can do together. Um, I’m sure they did, I’m sure . . . yeah, if one of you’s got, like that [patient]’s got, she’s got the ability to do nails and massage, then she can share it around five other women, you know . . . that’s like being self-sufficient.
Service user 1, unit 3106
Impact of the GetREAL teams on unit activities
A number of service users reported that the GetREAL team had led to an increase of activities available in the unit:
Have you noticed a change since they came to the unit?
Um, there’s more activities involved, um they do a rambling club, and a lunch group and things like that.
Oh OK, and what do you do for lunch group?
Um a different person cooks a meal and then all the others come and . . .
Come and eat?
Yeah [laughs].
Lovely and do you participate in that?
Yeah yeah.
Service user 2, unit 3106
OK, um, so when you first moved here, what was the average week like?
I can’t really remember. There wasn’t many activities going on, no, they seem to have improved those over the past couple of months really.
Oh really? OK.
Yeah, I thought it was the ladies coming, that helped you know set that up.
Service user 2, unit 3106
A newsboard. We’ve got one in every bungalow, and that was something else they did. They um, set up the boards um, we’d already got them, but they did it, making it look brighter. Yeah I can remember that. And like making posters, telling – bright colours and that – telling you what’s happening and when it’s happening and whose doing it and everything. Um and you can like look back at them before the actual, before the actual thing is taking place, and you can think, hmm, I think I’ll look forward to that.
Service user 1, unit 3106
However, others reported that although there had been more activities available while the GetREAL teams were working on the unit, these were no longer being facilitated by unit staff.
I remember doing some photography.
You did some photography. And was that something that you just started doing when they came here?
That was just something that I just started doing when they came here yeah.
OK so you didn’t do it beforehand?
I didn’t do it beforehand, but I had been interested before in photography.
Oh yeah, lovely. And so what kind of photos do you take here now? Is it when you head out on your activities or just around the unit here?
I haven’t been doing it since [GetREAL team] were here.
So you’ve stopped since they left or . . .
I stopped since they left. I haven’t got a camera of me own and haven’t been given, haven’t been given a camera.
Oh OK, but you’d like to, take some.
I’d like to take some yeah, of other people here.
Service user 1, unit 2902
Yes there was activities . . . but when they was here, it was better.
Service user 2, unit 0804
. . . no it’s the same . . . we need a few more activities, or different activities to get people interested in what, in what they want.
Service user 1, unit 3301
Service users’ interest in activities
Most service users interviewed seemed easily able to identify activities they were interested in.
What would you like to do here on the ward . . . is there anything that you like to do that would be good?
Social events and that.
Social events OK, any particular kind of social events, like a party or a disco or what kind of things?
I’m talking social events, like social events outside.
You want some social events outside?
Yeah I want some social events outside not inside, not parties, not discos.
So you want to be able to interact.
I want to eat supper [?] with other people outside and, have personal conversations and study . . .
Service user 1, unit 3301
So you’re quite interested in a wide range of things?
Yes I am, I’m interested in drawing and going to art group every Thursday.
Service user 1, unit 2902
Some service users recognised that their interest in certain activities had changed over the course of their illness:
You’ve got some musical instruments and things here as well.
Yeah.
Do you play them often?
No.
Why don’t you play them?
I used to play guitar but I went off, through my, because I didn’t feel well for so many years.
Service user 1, unit 3301
This service user described that although when first admitted he participated in more group activities, as his mental health had improved he preferred to do more independent activities:
OK. So would you say, kind of over the past 2 years or so that you’ve lived here, has there been any change in the amount of activities or things that are going on?
Well when I first came here, I only used to go to a few groups, the men’s group or something like that, and that was twice a week, and then during the week, we’d sometimes go to a garden centre now and again, you know, and then we used to go to pictures and then bowling, and that was within the month or sometimes these were within the week, you know, but as I’ve gradually come along, over the past two and a half years, I’ve more or less been changed to do my own thing, and I don’t go to men’s group anymore, but I do sometimes go out with them when they go into the garden centre, and we do go to a snooker club once a week.
Service user 1, unit 4204
This example also illustrates the importance of offering an individualised approach to activities, as service users’ interests may change:
And painting, yep, and you enjoy doing that? Have you been doing that the whole time you’ve been on the unit?
Um, um, I think that was quite a while ago I didn’t attend it, when I first came.
Was it running then or was it just something that you didn’t think you were interested in at the time?
Um, yeah I wasn’t very interested.
Yeah, what changed your mind about . . .
Well I know, I just, I think, I think, no, I just decided to go for it.
Service user 1, unit 0804
Others chose not to engage simply because they were not interested in what was on offer:
[OT] mentioned to me that you have an allotment that people go to.
I don’t use it.
You don’t go to that. You don’t fancy gardening . . .
I don’t mind gardening, but I don’t really fancy . . . I help with the gardening at home, yeah.
Service user 1, unit 4203
Service users’ insights into difficulties facing staff in facilitating activities after the GetREAL team left
Service users described a wide variety of groups and activities available to them on the units, including a DVD group, a rambling group, gardening, pampering and beauty groups, knitting, embroidery and other crafts, and various cookery groups. Group outings were also often mentioned (e.g. to the seaside or a local art gallery). Some service users also mentioned that they were attending classes at a local college. However, although there appeared to be a number of opportunities for activities, several service users reported that staff were too busy to talk to them or to assist them with specific things they wanted to do that were not part of the group programme.
And do you find that the staff maybe talk to you more about what kind of things you’re interested in doing . . . or?
Ah no not really, because they’re that busy, yeah they don’t get a chance to be like on a one to one. But they might talk to you in a group or something like that.
Is that something you would like if they . . .
If they could talk one to one? Yeah.
Service user 2, unit 3106
One service user described how he enjoyed building model kits, but spoke of difficulty in getting staff to help him with his project, and said that he needed to ask multiple times:
I bought a little matchstick here, ehm, you build like a mechanical thing.
Is that like Meccano or something?
It’s made out of wooden.
Ahh.
I bought one in the past, but I haven’t started it yet, but through the week I asked staff if they would help me but . . .
But they haven’t had time to help you or?
Well I haven’t brought it to them, to ask the, the staff.
But you might ask them to help you?
I did ask, but I haven’t asked them again.
Service user 2, unit 3301
Service users were aware of the pressure on staff due to high workloads, lack of resources and staff shortages, and understood that these impacted on the provision of activities. For example, this service user speaks about their unit’s walking group:
It should happen every week, but the lady that helps us with it, she’s not always here, so you don’t get . . . nobody else seems to take it on, whether they’re too busy or not I don’t know.
And you said that there’s a rambling club.
A walking group.
A walking group is it, excellent. Do you take part in that?
I do yeah, and they go to different places . . . we’ve been all over really . . .
And do you enjoy going on those walks and getting out?
Yeah I do! I missed this week because there’s not, there’s staff absences – off sick, so there’s not been enough staff to run that. So I missed it this week.
Service user 1, unit 3106
This service user described how bingo was postponed because there were not enough funds to buy prizes:
I play bingo on the weekends or something like, but we haven’t had that this week or last week.
OK, do you know why it hasn’t been on this week or last week?
We haven’t had the money.
Haven’t had the money oh OK. For prizes?
Yeah.
Service user 1, unit 3301
Although the GetREAL teams had aimed to engage all staff in the task of facilitating activities, some service users reported that most activities were still being facilitated by OTs and activity workers:
And so what happens in recovery group. What do you do in that group?
Ehm, just talk about stuff and anxiety and anger.
About anger?
Yep, and social problems with the family.
Problems with your family, so it’s like a therapy type group is it?
Yeah.
And who runs that?
One of the OTs.
Service user 1, unit 3301
However, others reported that other staff were proactively encouraging them to engage in activities:
Um, I think I’ve only really been doing the [pampering] groups over the last year. I don’t . . . I think I always just try to get out of it, but then I’ve just met [a ward staff member] who works here and she’s kind of turned me around and convinced me that it’s a really good idea to get yourself involved in things, so I started it recently, within the last year.
Aha, and how do you feel now that you that you are involved in those. Do you agree with [the staff member]?
I do agree yeah. It’s made my life much better.
Service user 1, unit 0804
Discussion
Main findings
The qualitative component of phase 3 provided helpful insights into possible reasons for the lack of efficacy of the GetREAL intervention found in the cluster randomised controlled trial. Staff often reported that they had experienced difficulties in engaging service users in activities prior to the GetREAL team’s arrival, and appeared positive and enthusiastic about the impact of the techniques and changes in practice that were made during the enabling stage. However, these benefits were not sustained during the reinforcing stage because the unit staff reverted to their previous practices once the intervention teams left; there appeared to be minimal implementation of the action plans that the GetREAL teams had drawn up and agreed with the units. This appeared to be due to four main reasons: competing priorities (particularly ‘paperwork’); a reversion to previous rigidity around role boundaries that abdicated responsibility for engaging service users in activities, rather than seeing this as an important task for the whole team; the placement of the onus for engagement in activities on service users, and the citing of the severity of symptoms as an insurmountable barrier to such engagement; and a failure to understand that the GetREAL intervention aimed to facilitate lasting change in the unit’s practices.
Some of the service users who were interviewed had no recollection of the GetREAL teams, but those who did often reported that the team’s presence had led to a greater range of activities being on offer in the unit. Although there appeared to be a broad range of group-based activities available across units, there was little sense of a personalised approach from staff that tailored an individual’s activities to their interests and recovery. Many service user groups remained the responsibility of occupational therapy staff, and service users demonstrated insight into the competing demands on nursing and support staff time that impeded staff’s availability for activities.
Strengths and limitations
The purposive sampling employed in the qualitative component ensured the participation of staff of different levels of seniority, from different disciplines and unit types (hospital/community based) and based in different settings (urban/suburban/rural). There was also a good range in the time since the staff of the unit had participated in the enabling stage of the intervention, when the GetREAL teams had worked alongside them. In addition, the facilitation of the focus groups by non-researchers (the programme manager and a medical student) may have helped to increase participants’ willingness to speak openly about their experiences of the GetREAL intervention. However, logistical constraints meant that the participants were usually the staff who happened to be present on the day that the focus interviewers attended. Although the unit managers often attempted to agree a date and time when the maximum number of staff who had experienced the enabling stage of the intervention were likely to be available (such as during the usual team business meeting), inevitably some of those who wished to participate may not have been available. Thus, some important insights may have been missed. A similar sampling bias applies to the service user participants; those with specific issues that they wished to discuss may have been more likely to agree to participate. In addition, interviews could not be carried out with service users who were too unwell to give informed consent to participate and, therefore, those service users interviewed were likely to be further forward in their recovery than others on the unit.
Interpretation
The strengths of the GetREAL intervention, as described by the staff focus groups, are considerable. We have outlined a process in which the GetREAL intervention tackled many of the challenges faced by rehabilitation units, such as poor team-working, low staff morale and an absence of motivational techniques. However, it seems clear from the focus group findings that the third stage of the intervention, the reinforcing stage, may have been inadequate, and this provides some explanation as to why the intervention was found to lack effectiveness when the outcomes were assessed at the 12-month follow-up. This requires further consideration.
The intervention was specifically designed to help unit staff gain the confidence and skills to engage service users in activities, and to enable changes in the structures and processes of the unit to sustain these once the GetREAL teams left at the end of the enabling stage. Our results suggest that the enabling stage was successful but that, in most units, the gains in staff confidence and enthusiasm dissipated soon after the GetREAL teams left. The lack of implementation of the action plan may have been the result of a lack of ‘sign-up’ (this was evident in units where staff did not appear to have read the plan or to know who their link person was) or a lack of senior support for continuing to invest in it (although this remains speculative as it was not a clear theme in the focus groups). During the 12 months after the enabling stage, the NHS underwent one of its most turbulent periods as a result of the economic recession. One of the intervention units closed down and many others were subject to uncertainty over their future and to talk of reconfiguration. Some staff were reinterviewed for their own jobs and downbanded as part of the response to funding cuts. It is not difficult to imagine how such major changes and threats to job security could have led to the GetREAL action plan paling into insignificance. One unit manager reported that they had ‘taken their eye off the ball’ in relation to the action plan in the context of these major pressures.
Staff often reported that competing priorities made it difficult for them to make time for face-to-face contact with service users and to facilitate activities. Increasingly, mental health staff are expected to record data to provide evidence of specific targets and outputs that are mandated by senior management. A large amount of staff’s time is spent recording their own activities and ‘form-filling’, and electronic patient record systems can be frustratingly cumbersome and poorly supported by inadequate IT systems, making this process very slow and time-consuming. Staff time for face-to-face interaction with service users is reduced in order to meet these demands. 86 Alongside this, mental health units are notoriously difficult places to work; engaging with the emotional and physical disturbance of mental disorder on a day-to-day basis requires resilience and emotional intelligence from staff as well as support from senior members. Withdrawal to the office to avoid the ‘front line’ has been previously noted,86 but an increasing shortage of staffing will compound the pressure on those staff who are present. Our results showed that such resource issues impacted negatively on staff’s ability to prioritise service user activities.
There was also evidence that some nursing staff were inflexible in terms of the responsibilities associated with specific roles. Resistance and resentment were expressed in relation to the promotion of the idea that service user activity is everybody’s business. This rigid, boundaried demarcation of responsibilities is clearly unhelpful in the context of facilitating service user activities. Although OTs do have professional expertise in the area of activities and meaningful occupation, there is usually only one such staff member per unit. Our results from phase 4 (see Chapter 5) corroborate the previous studies cited in the background to this final report, that have shown a higher degree of service user activity to be associated with better clinical outcomes. Michael87 states that ‘in rehabilitation, nurses work alongside service users to provide an enabling role in everyday activities’. Contemporary definitions of mental health rehabilitation also emphasise this aspect. 19 Mental health rehabilitation staff therefore need to have, and to use, the appropriate skills for engaging service users in activities. A single member of staff cannot attend to this adequately for all service users in a unit. This may be one reason that, in the units studied, many activities continued to be delivered as group programmes rather than as a more personalised approach tailored to an individual’s interests and recovery.
Engaging service users in activities is not an easy task. People with complex mental health problems in inpatient rehabilitation units often have severe negative symptoms and lack motivation. However, our phase 1 results found that service user characteristics associated with chronicity and severity of illness (age, sex, being involuntarily detained and GAF score) were not associated to a clinically significant degree with the quality of care in inpatient mental health rehabilitation units. In other words, although this service user group presents specific challenges in terms of engagement, it is possible to deliver high-quality care in these settings (including the facilitation of activities). However, some staff appeared to place the onus for responsibility of engagement on the service user rather than on staff; some staff reported that their unit’s service users were too unwell to engage, or that the GetREAL teams had been successful in this because they had happened to be on the unit at a time when the service users were somehow easier to engage. Although it may be true that the profile of service users has changed to some degree over recent decades, with the development of an increasingly community-based mental health system (such that those patients in hospital are at the most severe end of the spectrum),88 these are, nevertheless, the patients that contemporary mental health rehabilitation services work with. Furthermore, the positive impact on staff and service users reported during the enabling stage suggests that many service users could be engaged successfully in activities.
The barriers to implementation of the action plan, discussed by all staff focus groups, emphasise both resource and service user factors. Changing these is not within the control of front-line staff. The Job Demand-Control-Support model may be relevant here,89,90 whereby staff burnout and job satisfaction are mediated by the demands of the role (workload, role conflict), the support from peers and seniors, and the degree of control the worker has over the demands of the job. Van der Doef and Maes91 argue that interactions with service users constitute a particular additional demand on health-care professionals. These factors may have resulted in a loss of team strategy and cohesion, and a defensive retreat to inflexible role boundaries that led to staff abdicating their responsibility for service user activities. Instead of the newly found collaborative enthusiasm for team-working on agreed objectives, the issue of service user activities appears to have reverted to being the sole responsibility of OTs. The contrast between this and the positive experience that support staff reported of being ‘given permission’ to step outside their usual role during the enabling stage is also noteworthy. As support staff are supervised by nursing staff, it appears that the benefit this experience brought was lost once the enablers of this change, the GetREAL teams, left. The concept of permission also relates to the low degree of decision-making autonomy many staff felt they had in inpatient settings. 88 This, in turn, presumably reinforced inflexibility in role boundaries and remit. It is also likely to have impacted on staff morale and ward atmosphere.
Conclusion
The results of the qualitative component of phase 3 of the REAL study helped in our understanding of the results of the cluster randomised controlled trial. The intervention appeared to be well received, and provided an opportunity for staff to step outside their usual roles and review their approach to engaging with service users. It appeared to lead to some specific changes in the structures and processes within the unit, and to positive attitudes and behaviours of staff at all levels. These factors appear to have encouraged service users to engage in activities. However, these benefits were not sustained once the GetREAL teams left. Staff seemed to lack the motivation, authority and support to continue to enact their new skills, and quickly reverted to previous practices.
Difficulties in implementing interventions in mental health services into routine practice are well recognised. Tansella and Thornicroft92 have described the barriers to implementation as occurring at the national, local and individual level. Although secular changes do not usually impact on RCTs, as participants in both arms are subject to them, in this study the severity of this economic recession certainly had an impact on staff morale, and the resultant turbulence in the system may well have destabilised any impetus for change that the enabling stage had tried to establish. This would have reduced differences in effectiveness between units in the two trial arms. It is understandable that staff would be hesitant to invest their energy in new tasks that might improve their service when it could be closed at short notice. 93 Local factors included the degree to which senior staff supported implementation of the changes in practice described in each unit’s action plan and gave permission for unit managers and other staff to continue to focus on these alongside other tasks. Redfern et al. 93 suggest that development of a culture that is receptive to changes in practice requires strong leadership and actively engaged ‘agents of change’. Although the GetREAL teams appeared to act in this role during the enabling stage, the role did not appear to pass successfully to the link person during the reinforcing stage. At the individual level, our results suggest that rigid role definition, inadequate resources and competing priorities among frontline staff were the main barriers to sustaining the intervention. These findings are clearly useful in considering how the intervention could be adapted to improve its effectiveness.
Summary of main findings from phase 3
-
In phase 3 of the REAL study we carried out a rigorous cluster randomised controlled trial to investigate the efficacy of a staff training intervention that aimed to facilitate engagement in activities for users of inpatient mental health rehabilitation units.
-
Our results showed no difference in clinical effectiveness or cost-effectiveness between units that received the intervention and comparison units that continued to provide usual care 12 months after recruitment into the study.
-
Data from our measure of unit fidelity to the intervention, and from focus groups with staff of the units that received the intervention, suggest that although the predisposing and enabling stages of the intervention were well received, there was a reversion to previous practice during the reinforcing stage, after the GetREAL teams left.
-
Rigid role definition, inadequate resources and competing priorities among frontline staff were the main barriers to sustaining the intervention. Strengthening this part of the intervenion through ongoing supervision of staff teams, and stronger ‘buy-in’ of senior management to support staff in continuing to focus on service user activities alongside other tasks, may provide for a more effective intervention.
Chapter 5 Phase 4: naturalistic cohort study
Background
In phase 1 of the REAL study, we surveyed the nature and quality of mental health inpatient rehabilitation services across England. In phase 3, we evaluated the staff training intervention developed in phase 2 that was implemented in less well-performing services. Phase 4 complemented these other phases of the programme, comprising a naturalistic cohort study carried out in better-performing services that aimed to investigate the characteristics of services and service users associated with desirable clinical outcomes.
Design and setting
A prospective cohort study involving service users of units that scored above average for quality on the QuIRC in phase 1. Our original protocol aimed to recruit an average of 10 services users from each of 35 inpatient mental health rehabilitation units (i.e. 350 service users) and to describe their characteristics at recruitment and 12 months later. This phase ran concurrently with phase 3.
Eligibility criteria
All 67 services that scored above the median total QuIRC score in phase 1 were eligible for inclusion in phase 4. There were no exclusion criteria.
Participant recruitment
All service users of each eligible unit were eligible to participate in the study and the researchers approached them to explain its purpose and process. A participant information sheet was provided and the service users were given the opportunity to ask questions about the study. Those who were assessed as having capacity to give informed consent but who declined to participate were not recruited. The approval gained from the South East Essex Research Ethics Committee (reference 09/H1102/45) for the REAL programme allowed for the inclusion of participants who lacked capacity to give informed consent to participate.
Measurement of outcomes
Our primary outcomes assessed at the 12-month follow-up were:
-
social function as measured by the LSP75
-
successful community discharge, that is, without readmission or placement breakdown (service users considered ‘ready for discharge’ who were awaiting a vacancy in suitable accommodation were also included in our analyses)
-
length of admission in the rehabiliation unit.
Phase 4 did not involve research interviews with service user participants. All data were gathered from case notes and staff-rated measures. This approach was pragmatic given the fact that the two researchers were concurrently recruiting and interviewing service users in phase 3, and it minimised the burden on service users of research interviews. Data were gathered on service user participants at recruitment from a review of case notes and staff-rated measures, as follows: demographics (age, sex, ethnic group); diagnosis; length of history; length of current admission; previous admissions; previous involuntary admissions; risk history; social function, assessed using the LSP;75 engagement in activities, assessed using the staff-rated version of the time-use diary;74 and function, assessed using the GAF scale. 50 Potential mediators of outcomes were also assessed through staff ratings: quality of care in the unit was assessed using the QuIRC43,44 completed by the unit manager; staffing of the unit and the availability of specific interventions were included in the QuIRC; service users’ substance use was assessed using the Clinician Alcohol and Drug Use Scales;76 and challenging behaviours that may make community placement difficult were assessed using the SPRS. 77
At the 12-month follow-up, the researchers contacted a key informant to clarify whether or not the service user had been successfully discharged, and the details of community placements were recorded. When service users had been discharged from the unit, the key informant was either the service user’s community care co-ordinator or their keyworker at their supported accommodation. For service users who remained in the inpatient setting at follow-up, the key informant was their primary nurse. These key informants were asked to complete the LSP to assess service users’ social functioning. The data on length of admission were available from the unit manager and case notes, and were therefore available for all service users assessed at recruitment.
Data management
Data were entered into the study’s Access database by the researchers. Range and logic checks were built in to assist with data cleaning. Ten per cent of data were double entered to check for data entry errors with an error rate set at 5%, above which all data would be double entered. The error rate was < 5% and, thus, no double data entry was required. Data were further checked and cleaned by the statistician before analysis.
Data analysis
A subgroup of the PMG (the chief investigator, the study statisticians and MK) agreed on the data analysis plan for phase 4 (see Appendix 11). Descriptive characteristics of the cohort (demographics, and clinical and service characteristics) were summarised using mean (SD), median (IQR) or proportions, as appropriate, at baseline and 12-month follow-up. Summary statistics were calculated for all standardised assessments at baseline and follow-up, providing percentages or mean (SD), or median (IQR), as appropriate.
Random-effects regression models were used to investigate the relationship between service user characteristics, service factors (QuIRC domain scores) and outcomes, accounting for clustering of service users within units using random-effects models. As the analyses were exploratory, no sample size calculation was performed. However, when fitting the models, we followed the rule of 10 events per variable94 to ensure that the coefficients were estimated with adequate precision. A complete-case analysis was performed. As the analyses are considered to be exploratory, the estimates of association are presented with their 95% CIs.
An intuitive analysis was carried out to identify the service and service user factors associated with successful discharge and successful discharge plus readiness for discharge. Categorical or binary explanatory variables with < 5% prevalence were omitted to reduce chances of model instability. A univariable analysis of pre-specified factors considered to be potentially associated with the outcome of interest was first carried out. Variables that had a p-value of ≤ 0.15 were considered in the mutivariable model. Backwards elimination was then carried out, with a significance level of 5%.
Results
A total of 50 of the 67 units that scored above the median total QuIRC in phase 1 agreed to participate in phase 4. Within these units there were 540 potentially eligible service users, of whom 346 (64%) gave informed consent to participate, and 16 (3%) who lacked capacity to give consent were also recruited. At the 12-month follow-up, seven participants had died, four had emigrated and two had withdrawn consent. Of the remaining 349 participants, the researchers gathered 12-month outcome data on 339 (97%). Figure 7 shows the participant flows in phase 4.
The participant characteristics were very similar to those of users of all rehabilitation services across England, as reported in phase 1. These characteristics did not vary between recruitment and follow-up, and are reported at recruitment only in Table 20. Two-thirds (65%) were male, the vast majority (90%) were white and most had a diagnosis of schizophrenia or schizoaffective disorder (78%). Participants’ median length of contact with mental health services was 12 years and they had a median of four previous admissions, two of which had been involuntary. Their current admission was 18 months long at the time of recruitment, with 7 of those months spent in the rehabilitation unit. Over two-thirds (71%) had been detained involuntarily during the current admission and around half were still detained. One-fifth (20%) had previously been an inpatient in a secure unit and 9% were currently detained on a ‘forensic’ section of the Mental Health Act. 42 The majority (81%) had never been in prison, but four (1%) had been imprisoned within the last 2 years. Just under half had physically assaulted other(s) at some time; this had occurred within the last 2 years for 20%. Serious assaults (resulting in the victim requiring hospital treatment, or resulting in homicide) were uncommon (5%), but four participants had killed somebody, all four incidents having occurred > 2 years previously. More prevalent was the risk of self-neglect and self-harm (65% and 17%, respectively, within the last 2 years). Mean staff ratings of service users’ functioning and activity were low, reflecting the severity of illness in this group. Problematic substance use was relatively uncommon (alcohol 7%, illicit substances 4%).
Characteristic | n (%) |
---|---|
Sociodemographics and diagnosis | |
Male | 235 (65) |
Age (years), mean (SD) | 39 (13) |
White | 324 (90) |
Schizophrenia | 238 (66) |
Schizoaffective disorder | 36 (10) |
Bipolar disorder | 26 (7) |
Other | 49 (14) |
Contact with mental health services | |
Time since first contact (years), median (IQR) | 12 (6–20) |
Previous admissions, median (IQR) | 4 (2–7) |
Previous involuntary admissions, median (IQR) | 2 (0–4) |
Length of current admission (months), median (IQR) | 18 (9–38) |
Length of current admission in rehabilitation unit (months), median (IQR) | 7 (3–15) |
Current admission involuntary | 250 (69) |
Currently detained involuntarily | 174 (48) |
Previous admission to special hospital (ever) | 9 (3) |
Previous admission medium secure unit (ever) | 30 (9) |
Previous admission low secure unit (ever) | 70 (19) |
Currently detained on forensic section | 32 (9) |
Of whom: Section 37 | 18 (56) |
Of whom: Section 37/41 | 14 (44) |
Risk history | |
Assault on others > 2 years ago | 96 (27) |
Assault on others in the last 2 years | 68 (19) |
Serious assault > 2 years ago | 20 (four homicides) (5) |
Serious assault in last 2 years | 4 (1) |
Sexual offence > 2 years ago | 17 (5) |
Sexual offence in the last 2 years | 3 (1) |
History of fire-setting > 2 years ago | 15 (4) |
History of fire-setting in the last 2 years | 10 (3) |
Overdose or self-harm > 2 years ago | 83 (23) |
Overdose or self-harm in the last 2 years | 58 (16) |
Recurrent self-harm in the last 2 years | 21 (6) |
Self-neglect in the last 2 years | 220 (61) |
Function, mean (SD) | |
Mean GAF rating | 53 (8) |
Mean LSP rating | 128 (15) |
Mean time-use diary (activity) rating | 49 (11) |
Challenging behaviours | |
SPRS, median (IQR) | 0 (0–2) |
Problematic alcohol use | 24 (7) |
Problematic substance misuse | 15 (4) |
Table 21 shows the characteristics of the 50 units that participated in phase 4. Most were community based (88%) and located in suburban areas (86%). These units were slightly smaller than phase 3 units (mean 13 beds) but staffing was similar; all units were staffed by nurses, support workers and psychiatrists, and most had an OT and a psychologist on the team or had access to these disciplines. The proportion of units with a psychologist increased over the course of the study. Over one-third of units employed ex-service users on the staff team. With regard to the treatments and interventions offered, over one-third of service users were prescribed clozapine and none was prescribed more than two antipsychotics. Most (75%) had an informal carer (a family member or friend) involved in their care, but family interventions had been carried out with only 7% of service users in the previous 12 months. Over 10% of service users had received CBT in the 12 months before recruitment (this increased to 13% during the course of the study). Over half of the unit managers rated the degree to which they felt comfortable with a close friend or relative of theirs receiving treatment in the unit as ‘very happy’. The quality of the units, as assessed by the QuIRC, remained higher than that of the units that participated in phase 3 (as expected).
Characteristic | Baseline | Follow-up | ||
---|---|---|---|---|
n or (median) | % or (IQR) | n or (median) | % or (IQR) | |
Unit location | ||||
Inner city | 5 | 10 | ||
Suburbs | 43 | 86 | ||
Rural area | 2 | 4 | ||
Unit type | ||||
Hospital ward | 6 | 12 | ||
Community based | 44 | 88 | ||
MINI score for local area of unit, mean (SD) | 1.09 | 0.363 | ||
Staffing | ||||
Psychiatrist works in the unit | 47 | 94 | 50 | 100 |
Access to a psychiatrist | 3 | 6 | 0 | 0 |
Clinical psychologist works on the unit | 26 | 52 | 36 | 72 |
Access to a clinical psychologist | 20 | 40 | 11 | 22 |
OT works on the unit | 39 | 78 | 41 | 82 |
Access to an OT | 7 | 14 | 6 | 12 |
Nurse works on the unit | 50 | 100 | 50 | 100 |
Support worker works on the unit | 50 | 100 | 50 | 100 |
Social worker works on the unit | 1 | 2 | 2 | 4 |
Access to a social worker | 39 | 78 | 45 | 90 |
Ex-service user(s) work in the unit | 19 | 38 | 20 | 40 |
Ex-service user(s) on the payroll | 15 | 79 | 17 | 85 |
Staff turnover | (14) | (8–21) | (9) | (4–17) |
Staff-to-service user ratio | (1.8) | (1.6–2.2) | (1.9) | (1.8–2.3) |
Beds | ||||
Beds on the unit | (13) | (10–16) | (13) | (10–16) |
% beds occupied | (95) | (89–100) | (94) | (86–100) |
Turnover | ||||
% staff turnover in last 12 months | (14) | (8–21) | (9) | (4–17) |
% service user turnover in last 12 months | (82) | (55–113) | (92) | (56–120) |
Interventions | ||||
Percentage taking clozapine | (37) | (25–45) | (39) | (25–45) |
Percentage taking multiple antipsychotics | (0) | (0–0) | (0) | (0–0) |
Percentage of service users with carer involvement | (76) | (64–90) | (75) | (56–87) |
Percentage of service users who had family intervention in the last year | (7) | (0–28) | (7) | (0–25) |
Percentage who received CBT in the last 12 months | (13) | (0–33) | (9) | (0–38) |
Unit manager ‘very happy’ for their friend/relative to receive care on unit | 28 | 56 | 27 | 54 |
QuIRC domain scores (%), mean (SD) | ||||
Living environment | 79 | (7) | 79 | (7) |
Therapeutic environment | 68 | (6) | 71 | (5) |
Treatments and interventions | 65 | (7) | 65 | (8) |
Self-management and autonomy | 75 | (6) | 77 | (7) |
Human rights | 77 | (8) | 80 | (6) |
Recovery-based practice | 72 | (7) | 74 | (7) |
Social inclusion | 65 | (12) | 65 | (11) |
Table 22 shows participant outcomes at the 12-month follow-up. Over half (56%) were successfully discharged, and a further 14% were considered ready for discharge but no suitable vacancy in supported accommodation had been identified for them to move to. There was a small improvement in mean staff ratings of service users’ social functioning (LSP score) from 128 to 132. Staff ratings of service users’ functioning (GAF score) and activity (time-use diary score) also improved slightly over the 12 months.
Twelve-month outcome | Baseline, mean (SD) | 12-month follow-up, n (%) |
---|---|---|
Discharged from the rehabilitation unit | 219 (66) | |
Ready to be discharged from the rehabilitation unit | 48 (14) | |
Discharged or ready to be discharged from the rehabilitation unit | 267 (80) | |
Successfully discharged from the rehabilitation unit | 187 (56) | |
Successfully discharged or ready to be discharged from the rehabilitation unit | 235 (71) | |
Length of admission (months), median (IQR) | 18 (9–38) | 24 (15–48) |
Length of admission in the rehabilitation unit (months), median (IQR) | 7 (3–15) | 16 (10–23) |
LSP rating, mean (SD) | 128 (15) | 132 (15) |
GAF score, mean (SD) | 53 (8) | 56 (9) |
Time-use diary (activity) rating, mean (SD) | 49 (11) | 51 (15) |
Table 23 shows the results of the regression analysis investigating the association between the quality of the unit, as assessed using the QuIRC, and service users’ social function (LSP score) at the 12-month follow-up. None of the QuIRC domain scores appeared to be associated with service users’ social function.
QuIRC domain | Adjusteda coefficient (95% CI) | Unadjusted coefficient (95% CI) |
---|---|---|
Living environment | –0.02 (–0.31 to 0.27) | 0.11 (–0.16 to 0.38) |
Therapeutic environment | –0.06 (–0.41 to 0.28) | –0.11 (–0.43 to 0.21) |
Treatments and interventions | –0.18 (–0.45 to 0.08) | –0.20 (–0.45 to 0.04) |
Self-management and autonomy | –0.03 (–0.34 to 0.27) | 0.10 (–0.19 to 0.38) |
Human rights | –0.05 (–0.30 to 0.21) | 0.03 (–0.21 to 0.27) |
Recovery-based practice | –0.09 (–0.38 to 0.20) | –0.04 (–0.32 to 0.24) |
Social inclusion | –0.06 (–0.23 to 0.11) | –0.06 (–0.22 to 0.09) |
Table 24 shows the results of the regression analysis investigating the association between unit quality and length of admission in the rehabilitation unit. Given the small size of the coefficients and the confidence limits, it seems that none of the QuIRC domains was associated with length of admission in the rehabilitation unit.
QuIRC domain | Coefficient (95% CI) |
---|---|
Loge length of admission | |
Living environment | 0.001 (–0.006 to 0.009) |
Therapeutic environment | 0.004 (–0.004 to 0.013) |
Treatments and interventions | 0.001 (–0.006 to 0.008) |
Self-management and autonomy | 0.003 (–0.004 to 0.010) |
Human rights | 0.005 (–0.002 to 0.011) |
Recovery-based practice | 0.000 (–0.007 to 0.007) |
Social inclusion | 0.001 (–0.003 to 0.005) |
The results of the further multivariable exploratory analyses are shown in Tables 25 and 26. Table 25 shows the variables associated with successful discharge at the 12-month follow-up. The communication subscale of the LSP (which assessed service users’ social skills), the time-use diary score (which assessed service users’ level of activity) and the recovery-based practice domain of the QuIRC (which assessed the unit’s performance on this aspect of care) were found to be positively associated with successful discharge. The length of service users’ current admission and the percentage of service users in the unit who had received CBT in the 12 months prior to recruitment were associated with a reduced chance of successful discharge.
Variable | OR (95% CI) |
---|---|
Length of current admission (months) | 0.99 (0.99 to 1.00) |
LSP communication subscale score | 1.13 (1.04 to 1.24) |
Time-use diary (activity) score | 1.03 (1.01 to 1.05) |
QuIRC recovery-based practice domain score (%) | 1.04 (1.00 to 1.08) |
% service users in the unit who received CBT in the year before recruitment | 0.99 (0.98 to 1.00) |
Variable | OR (95% CI) |
---|---|
Any history of fire-setting | 0.35 (0.13 to 0.92) |
Any self-harm | 2.02 (1.16 to 3.51) |
Length of current admission | 0.99 (0.99 to 1.00) |
Time-use diary (activity) score | 1.05 (1.02 to 1.07) |
When participants who were ready for discharge and awaiting a suitable placement in the community were included in the analysis, along with those who had achieved a successful discharge at the 12-month follow-up, length of current admission was, again, found to be associated with a reduced chance of successful discharge/readiness for discharge, as was any history of fire-setting. Service users’ level of activity (time-use diary score) was positively associated with successful discharge/readiness for discharge, as was a history of self-harm (see Table 26).
Discussion
Phase 4 of the REAL study comprised a naturalistic cohort study investigating outcomes for users of inpatient mental health rehabilitation units that were assessed as being of above-average quality during phase 1. By including better performing units, we aimed to identify the aspects of good-quality care that were most likely to be associated with supporting service users in improving their social function and achieving successful community discharge.
Main findings
The majority of service users achieved a successful community discharge within the 12-month follow-up period, and there were small improvements in the ratings of social function. The multivariable regression models did not appear to identify any association between the seven QuIRC domains assessing the quality of care provided in the units and better clinical outcomes for service users (social function or length of admission in the rehabiliation unit). However, a number of factors were identified that were associated with successful discharge. These were repeated including service users who were considered by staff to be ready for discharge but who were awaiting a suitable community placement, in order to avoid bias due to lack of availability of appropriate local supported accommodation. Both analyses found that the degree to which service users were engaged in activities at baseline was positively associated with successful discharge/readiness for discharge, but service users who had been in hospital longer were less likely to achieve this positive outcome. In the first model, service users’ social skills at baseline (the communication subscale of the LSP) were also found to be positively associated with successful discharge, as was the degree to which the rehabiliation unit operated with a ‘recovery’ orientation.
Although the negative association between fire-setting and discharge or readiness for discharge is not surprising, less easy to explain is the negative association between receiving CBT and successful discharge.
Implications of the findings
These results are important in helping to inform the practice and interventions that are most likely to help service users with complex needs progress in their rehabilitation. The finding that service users’ activity at baseline was associated with successful discharge/readiness for discharge at 12 months provides corroboration of the appropriateness of our focus on service user activities during phases 2 and 3 of the REAL study. The association with social skills is also of interest. The evidence for the effectiveness of social skills training for people with schizophrenia has not been considered adequate for NICE7 to recommend routinely offering it. However, a meta-analysis of 22 trials of social skills training95 found it to be associated with improvements in psychosocial functioning and negative symptoms, although problems with heterogeneity of methods and result reporting limited the robustness of the findings. 96 Nevertheless, the Scottish Intercollegiate Guidelines Network guidance on the management of schizophrenia97 states that ‘social skills training may be considered for individuals diagnosed with schizophrenia who have persisting problems related to social skills’. It may be that further studies are required to investigate the potential benefit of specific social skills training for people with complex mental health problems who are referred to rehabilitation services.
Our finding of a positive association between successful discharge and recovery-orientated practice is of interest. Policy-makers strongly encourage recovery-orientated practice in mental health services. 98 It incorporates a focus on therapeutic optimism and collaborative working with service users to agree together the goals of treatment and support, rather than the more traditional approach of a professional-led treatment plan with the service user as a passive recipient. 99 Mental health rehabiliation services were early adopters of the recovery approach,66 and current commissioning guidance describes them as operating with this style and these values. 67 One specific aspect of recovery-orientated practice, namely the employment of ex-service users as members of the team, occurred in 38% of units (and in one-third of units across England in phase 1). The recovery-based practice domain of the QuIRC also includes many other aspects of care, including an assessment of the degree to which collaborative care planning practices are employed and the therapeutic optimisim of the staff. We believe that our results provide the first empirical evidence of the possible benefits of recovery-orientated practice.
We also identified factors associated with less chance of successful discharge/readiness for discharge. The greater the percentage of service users per unit who had received CBT in the year before recruitment into the study, the less likely successful discharge was. Although this could be interpreted as suggestive of a negative effect of CBT, there is generally good evidence of CBT’s effectiveness in people with psychosis and NICE7 recommends it for treatment in this group (although a recent meta-analysis has suggested that the effect size is smaller than previously thought). 100 A more likely explanation is that service users with the most complex needs are more likely to receive CBT as part of the range of interventions aimed at improving symptoms and functioning. In other words, the association with CBT may be simply related to the severity of their symptoms. This explanation concurs with the finding that service users who had been in hospital longer were less likely to achieve successful discharge/readiness for discharge. In other words, those with the most complex and treatment-resistant symptoms tend to remain in hospital longer and are, perhaps, more likely to be offered more interventions over time.
A history of fire-setting was also associated with less chance of successful discharge/readiness for discharge, although only 7% of the cohort had such a history. Challenging and dangerous behaviours have previously been noted to make individuals difficult to discharge from hospital. 6,77 Arson is an especially challenging behaviour, and many supported accommodation providers are, understandably, reluctant to offer placements to people with this kind of serious risk history. Conversely, we found that a history of self-harm (which had occurred for 41% of the cohort) was associated with a greater chance of successful discharge/readiness for discharge. This seems a rather paradoxical finding. Perhaps those who self-harm have less severe negative symptoms and are more motivated to act (albeit in a detrimental manner) than those with more severe negative symptoms whose level of function is so poor that it impedes community discharge. Self-harm may also indicate the presence of mood symptoms that are generally associated with a better prognosis than negative symptoms alone. It should also be borne in mind that this factor included self-harm at any point in the person’s history, and such acts may have occurred many years earlier.
Strengths and limitations
Although our analyses in phase 4 were exploratory, we can have some confidence in our findings. We recruited a large sample from across most of the better-performing inpatient mental health rehabilitation units in England. Our follow-up rate was excellent, with primary outcome data on successful discharge collected on 97% of our cohort. This is also a ‘hard’ dichotomous outcome that does not rely on subjective opinion. Our decision to use staff-rated outcomes also minimised the number of missing data on service users’ social functioning at the 12-month follow-up. Nevertheless, service user-rated outcomes, such as quality of life and satisfaction with treatment and support, would have allowed us to report on a more comprehensive range of perspectives on the concept of ‘meaningful’ clinical outcome.
Conclusion
Our results support the therapeutic optimism that is enouraged in mental health rehabilitation services. We found that the majority of service users in our cohort study were successfully discharged to the community within 12 months (without relapse, readmission or community placement breakdown), despite the severity and complexity of their mental health problems that led to their referral to these specialist services. Rehabilitation services are therefore succeeding with an especially complex group. We found that successful discharge was associated with units operating an approach that incorporated recovery-based practice. Higher levels of service user activity and social skills were also associated with a greater chance of successful discharge. Service users with more complex needs and challenging behaviours (specifically, fire-setting) were less likely to achieve successful discharge. This suggests that further research is needed to identify effective interventions that enhance recovery-based practice, service user activities and social skills.
Although our findings were derived from better-performing services in England, the results are of obvious relevance for improving the quality and effectiveness of services for people with longer-term and complex mental health problems in lower-quality units, not just in England, but in many European countries and developing nations, where the quality of care may be much worse.
It is concerning that 14% (1 in 7) service users whom staff considered ready for discharge could not leave the unit because no suitable community accommodation was available. This represents an inefficient use of resources and needs to be addressed urgently to ensure that service users are supported in the least restrictive environment appropriate to their needs. More investment in community-based supported accommodation is, therefore, required. This should include specialist accommodation for the small percentage of service users whose challenging behaviours, such as a history of fire-setting, impede their moving on.
Phase 4: health economic component
Methods
As with the analyses conducted in phase 3, we used an adapted version of the CSRI59 to collect information on the number of contacts with health-care professionals in the unit and in the community over the last month at both time points (baseline and 12-month follow-up). We used data compiled by Curtis58 to cost these service contacts, assuming that each had an average duration of 30 minutes unless information from a comparable study was available to suggest otherwise. 85
Service use and costs were compared between baseline and follow-up. We compared baseline and follow-up cost of contacts and estimated the statistical significance of the change in the total over time using a non-parametric bootstrap approach with appropriate correction for clustered data. 101 In addition, we estimated the association between baseline service user characteristics and baseline and follow-up cost of service contacts. For this purpose we used a random-effects regression model with robust standard errors. We pre-specified the list of potential predictors based on our knowledge of the literature. It included service users’ sex, age, GAF50 score, admission status (involuntary or not), subscale D of the SPRS77 assessing challenging behaviours (i.e. incontinence, risk of suicide, self-harm) and, where applicable, baseline costs. We then estimated the univariate association between QuIRC43,44 domains and costs at follow-up, as well as the interaction between service quality and service user characteristics. We also estimated the mean change in social function as assessed by the LSP75 rating and costs for the entire sample and subgroups, as well as subgroups defined by sex, length of illness and social function at baseline. We combined these estimates to compute a ‘cost–outcome’ ratio (i.e. the average change in costs divided by the average change in LSP75 score).
Results
Table 27 shows the service use (percentage of service users having contact with different staff and mean contacts) at baseline and 12-month follow-up. Service use generally declined over the 12-month follow-up period, with the exception of the percentage of service users having contacts with support workers, which stayed around the same, and the percentage of service users having contact with care co-ordinators, which underwent a large increase. Of those having contact with nurses, the number of hours decreased substantially, whereas for support workers the opposite was evident.
Service | Baseline | Follow-up | ||
---|---|---|---|---|
% using service | Mean (SE) contacts | % using service | Mean (SE) contactsa | |
Psychiatrist | 97 | 2.1 (0.1) | 66 | 1.7 (0.1) |
Other medical specialist | 44 | 2.9 (0.1) | 25 | 2.3 (0.2) |
Clinical psychologist | 19 | 3.2 (0.2) | 11 | 2.4 (0.3) |
OT | 57 | 7.9 (0.4) | 23 | 6.2 (0.7) |
Social worker | 7 | 1.8 (0.3) | 10 | 1.8 (0.2) |
Counsellor/psychotherapist | 0 | 4 (0.0) | 1 | 7 (1) |
Volunteer | 8 | 4.1 (0.6) | 2 | 3.3 (0.4) |
Art/music/dance therapist | 2 | 3.8 (0.7) | 2 | 3.5 (0.4) |
Care co-ordinator | 45 | 2.3 (0.1) | 62 | 2.8 (0.2) |
Advocate | 2 | 2.1 (0.5) | 1 | 1.2 (0.4) |
Probation officer | 1 | 2.5 (1.5) | 0 | 4 (0.0) |
Other | 12 | 3.5 (0.5) | 6 | 4.7 (1.2) |
Nurseb | 99 | 16.4 (0.6) | 90 | 9.2 (0.6) |
Support workerb | 64 | 11.8 (0.9) | 63 | 22.7 (2.7) |
As shown in Table 28, about 55–60% of the perceived costs of contact were accounted for by nurses at both time points, followed by psychiatrist and support worker costs. There was a statistically significant reduction in the cost of service use over the time horizon of phase 4, with a bootstrap estimate of –£710 (95% CI –£888 to –£514). This decrease was largely due to a reduction in nurse costs.
Service | Mean baseline cost (SE) | Mean follow-up cost (SE) | Mean cost change (SE) |
---|---|---|---|
Psychiatrist | 245 (9) | 134 (9) | –112 (12) |
Other medical specialist | 135 (10) | 61 (8) | –76 (12) |
Clinical psychologist | 36 (5) | 15 (3) | –23 (5) |
OT | 31 (2) | 11 (1) | –21 (3) |
Social worker | 15 (4) | 21 (4) | 5 (6) |
Counsellor/psychotherapist | 1 (1) | 2 (2) | 2 (2) |
Volunteer | 2 (0) | 0 (0) | –1 (0) |
Art/music/dance therapist | 1 (0) | 1 (1) | 1 (1) |
Care co-ordinator | 27 (2) | 44 (4) | 18 (4) |
Art/music/dance therapist | 1 (0) | 4 (1) | 3 (1) |
Care co-ordinator | 2 (1) | 1 (1) | 0 (2) |
Other | 24 (7) | 14 (6) | –11 (6) |
Nurse | 1216 (46) | 678 (47) | –529 (64) |
Support worker | 204 (13) | 241 (16) | 35 (19) |
Total | 1938 (56) | 1229 (66) | –710 (82) |
The univariate analysis results (Table 29) suggested that the percentage scores on the ‘living environment’ and the ‘human rights’ domains of the QuIRC were positively associated with cost of service contacts at follow-up (one percentage point increase in the ‘living environment’ score was associated with an increase in costs of £18; one percentage point increase in the ‘human rights’ score was associated with an increase in costs of £30).
QuIRC domain | Coefficient (95% CI) |
---|---|
Living environment | 18.0 (4 to 32) |
Therapeutic environment | 11.8 (–5.7 to 29.4) |
Treatments and interventions | –6.6 (–25.6 to 12.4) |
Self-management and autonomy | –4.2 (–22.1 to 13.7) |
Human rights | 29.6 (7.7 to 51.5) |
Recovery-based practice | 19.8 (–1.3 to 40.9) |
Social inclusion | 5.9 (–14.8 to 26.5) |
There was no evidence that service quality modified the relationship between cost of service use at follow-up and service user characteristics, except for the interaction between the ‘self-management and autonomy’ domain score and service users’ LSP score at baseline (Table 30).
QuIRC domain | Coefficient (95% CI) with interaction between QuIRC and sex | Coefficient (95% CI) with interaction between QuIRC and length of illness | Coefficient (95% CI) with interaction between QuIRC and LSP |
---|---|---|---|
Living environment | 7 (–14.6 to 28.6) | –0.1 (–1.4 to 1.2) | 0.4 (–0.3 to 1) |
Therapeutic environment | –31.9 (–63.2 to –0.5) | 1 (–1 to 3) | 0.5 (–0.5 to 1.6) |
Treatments and interventions | –3.2 (–35.6 to 29.2) | 0.1 (–1.7 to 1.9) | 0.9 (–0.1 to 1.9) |
Self-management and autonomy | –16.2 (–56.8 to 24.4) | 0 (–1.6 to 1.6) | 1.2 (0.4 to 2.1) |
Human rights | 16.2 (–14 to 46.4) | –0.3 (–2.1 to 1.5) | 0.7 (–0.5 to 1.9) |
Recovery-based practice | 1 (–40.1 to 42.1) | 0.3 (–1.5 to 2.1) | 1.1 (–0.1 to 2.4) |
Social inclusion | 9.5 (–23.7 to 42.8) | –0.6 (–1.9 to 0.7) | 0.6 (–0.4 to 1.6) |
The regression results suggested that, other things equal, male service users and white service users had a lower amount of service use at baseline (Table 31). Better functioning, as assessed by GAF score at baseline, was negatively associated with both baseline and follow-up costs. Service users with more challenging behaviours on the type D subscale of the SPRS (which includes self-harm) reported a higher number of costly contacts at baseline, but the effect on follow-up costs was not statistically significant. However, there was a trend towards higher follow-up costs for those service users with more challenging behaviours on the type A SPRS subscale (which includes fire-setting).
Dependent variable, predictors | Baseline cost (n = 351) | Follow-up costs (n = 326) | ||||
---|---|---|---|---|---|---|
Coefficient | SE | 95% CI | Coefficient | SE | 95% CI | |
Male | –280.3 | 126.7 | –528.6 to –32 | –206.8 | 166.4 | –532.9 to 119.3 |
White | –423.4 | 209.7 | –834.3 to –12.4 | 312.9 | 276.2 | –228.4 to 854.3 |
GAF score | –15.7 | 8 | –31.4 to –0.1 | –31.1 | 13.5 | –57.6 to –4.6 |
Age | 2.4 | 4.8 | –7.1 to 11.9 | 2.6 | 7.4 | –11.9 to 17.1 |
Type D behaviours (SPRS) | 188.4 | 91.7 | 8.7 to 368.2 | 133.7 | 122.7 | –106.8 to 374.3 |
Type A behaviours (SPRS) | 154.9 | 122.5 | –85.2 to 395.1 | 329.9 | 168.6 | –0.6 to 660.4 |
Involuntarily admitted | 38.5 | 134.8 | –225.7 to 302.6 | 40.5 | 165.1 | –283.1 to 364.1 |
Baseline cost | 0 | 0.1 | –0.1 to 0.2 | |||
Constant | 3156.6 | 608.8 | 1963.3 to 4349.9 | 2607.5 | 914.1 | 815.8 to 4399.2 |
The mean change in costs and benefits in the different subgroups that we investigated were comparable (Table 32); therefore, the ‘cost–outcome ratios’ were similar. One exception was the analysis of service users with high and low LSP scores at baseline, which had a decrease and larger increase in LSP scores, respectively; however, this may be due to regression to the mean.
Subgroup | Change in LSP (SE) | Change in cost (SE) | Cost–outcome ratio | n |
---|---|---|---|---|
Overall | 3.7 (0.9) | –709.9 (82.2) | –191.9 | 334 |
Men | 3.2 (1) | –680.3 (101.3) | –212.6 | 213 |
Women | 4.4 (1.8) | –762.3 (140.7) | –173.2 | 121 |
Length of illness > median | 3.8 (1.3) | –583.6 (125.3) | –153.6 | 174 |
Length of illness ≤ median | 3.5 (1.2) | –846.4 (103.8) | –241.8 | 160 |
LSP score > median | –3.9 (1) | –738.8 (97.7) | 189.4 | 161 |
LSP score ≤ median | 10.7 (1.3) | –682.8 (130.4) | –63.8 | 173 |
Discussion
The health economic component of phase 4 showed that there was a decrease in the costs of care over the 12 months of the cohort study, although the costs of contacts with support workers remained more or less stable and the costs of contacts with care co-ordinators increased. This concurs with the findings from the main results of phase 4, as the majority of service users were successfully discharged to the community and hence had fewer contacts with the inpatient MDT and more contacts with their community care co-ordinator. The majority of staff in supported accommodation that most service users moved on to are support workers, explaining the stability of costs associated with this staff group. Quality of care was not associated with costs of care when adjusted for service user age, sex and social functioning.
Less severe symptoms and higher functioning (higher scores on the GAF scale) were associated with lower costs of care, presumably because service users who were less unwell were more able and therefore had less need of staff support. This has been demonstrated previously. 102 Service users with a history of fire-setting were less likely to be discharged in our sample. Therefore, there was a trend towards higher costs at follow-up among those with problems on the SPRS subscale that included this behaviour.
In general, few variables were predictive of cost. This implies that (1) key factors may not have been included in the models, (2) supply-side features have greater influence on resource use or (3) service use exhibits substantial random fluctuation.
Our cost–outcome ratio analysis showed that the cost of every point increase in our measure of social function (LSP) was around £200. The mean score increased by 4 points over the 12-month cohort study, an improvement that would therefore cost around £800 per service user to achieve (£67 per month). This seems a relatively small investment to make to improve social functioning and achieve the high rate of successful discharge reported earlier.
Summary of main findings from phase 4
-
We carried out a naturalistic cohort study, following 349 service users of 50 inpatient mental health rehabilitation units for 12 months.
-
We achieved a very high follow-up rate (97%).
-
We found that over half (56%) of service users were successfully discharged to the community.
-
The more active service users had been at the start of the 12 months, and the better their relationships with others (social skills), the more likely they were to be discharged successfully.
-
The degree to which the unit adopted a ‘recovery orientation’ (i.e. involved service users in planning their own care and maintained hope for their move on from the unit) was also associated with successful discharge.
-
Service users with longer admissions and those with a history of fire setting were less likely to achieve successful discharge.
-
The mean score on our measure of service users’ social function increased by 4 points over the 12 months. The service costs associated with each increased point was around £200, so an improvement of 4 points therefore cost around £800 per service user to achieve (£67 per month). This seems a relatively small investment to make to achieve the high rate of successful discharge we found.
Chapter 6 Realist evaluation of the GetREAL intervention
Introduction
In phase 3 of the REAL study, we carried out a cluster randomised controlled trial to investigate the efficacy of a staff training intervention (the ‘GetREAL’ intervention) that aimed to increase service user engagement with activities in inpatient mental health rehabilitation units. Following the pre-disposing stage, in which senior psychiatrists visited each host organisation to gain their support, the enabling stage of the intervention was delivered by two ‘GetREAL’ teams who worked with staff in the units for a period of 5 weeks. During this time, the teams trained the staff in skills to facilitate service user engagement in activities and identified appropriate changes to the unit’s structures and processes that could support this. They agreed an action plan with each unit’s staff team with the aim of the team continuing to implement the new skills and processes after the GetREAL team left (the reinforcing stage of the intervention). The GetREAL teams continued to provide support through e-mail contact, as requested, over the following 12 months.
Our cluster randomised controlled trial programme found no statistically significant differences in service user activities at the 12-month follow-up between intervention and comparison units. 33 The qualitative component of phase 3 of the REAL study included focus groups with staff at the intervention units. These revealed that the increased staff skills and changes in practice that were facilitated in units by the GetREAL teams during the enabling stage of the intervention were not sustained during the reinforcing stage (once the GetREAL teams had left the units). 34 This may well explain the lack of effectiveness of the intervention. The results from our cohort study (phase 4 of the REAL study) identified that service users with higher levels of activity at recruitment had better clinical outcomes 12 months later. 36 This suggests that the aim of the GetREAL intervention (to increase service user activity) was appropriate.
Rationale and design
Together, the results of the phase 3 trial, its qualitative component and the phase 4 cohort study justified further work to try to understand, in more depth, which aspects of the GetREAL intervention may have been weaker and may have potential for strengthening in future studies. Asking the question ‘did an intervention work or not’ is not that meaningful for a complex intervention, as it cannot clarify the specific components of the intervention that are effective and whether or not the setting in which it is implemented has an impact on efficacy. Therefore, we applied a realist methodology to investigate the GetREAL intervention further. Realist methods pose the questions, ‘how or why does an intervention work, for whom does it work, and in what circumstances does it work?’. (Realist methodology is described more fully below in relation to the methodology for the rapid realist review.) This enquiry contributed to the process evaluation that is recommended to accompany trails of complex interventions, especially when the intervention is found to be ineffective overall or in certain contexts. 103
The four main aims were:
-
to review the factors that impact on sustainability of staff training interventions in mental health rehabilitation units
-
to further our understanding of the factors associated with variation in units sustaining the skills and changes in practice during the reinforcing stage of the GetREAL intervention
-
to investigate whether or not uptake of the GetREAL intervention, particularly during the enabling and reinforcing stages, was associated with outcome
-
to recommend modifications to the GetREAL intervention for testing in a future trial.
We addressed the aims using two complementary approaches: a rapid realist review of the literature to identify theories on what sustains long-term change in practice, and a qualitative case study analysis to identify the factors associated with better uptake of the GetREAL intervention using existing data collected during the cluster trial in phase 3. Finally, we carried out a quantitative data analysis (random-effects modelling) to explore whether or not ratings of uptake and receptiveness of the GetREAL intervention in all the units that had received the intervention during the cluster randomised controlled trial were positively associated with 12-month outcome ratings of service user activity and social function.
The rapid realist review informed the focus of the realistic evaluation by producing theories about long-term change in practice, and these theories were then tested using qualitative data from three purposively selected case study units that had received the GetREAL intervention. The exploratory analysis using random-effects modelling allowed us to triangulate these quantitative results with the findings from the case studies.
This chapter comprises three parts. In the first part, we describe the rapid realist review. In the second part, we present findings from the qualitative case study analysis. In the third part, we present the results from our exploratory quantitative data analysis (random-effects modelling). We then summarise the key findings emerging from the integration of the three parts of the realistic evaluation, and provide recommendations for modification of the GetREAL intervention based on these.
Rapid realist review
This section of the chapter describes the process and results of the rapid realist review.
Rationale for the review
The purpose of this review was to inform a realistic evaluation of the cluster randomised controlled trial that assessed the efficacy of the GetREAL intervention. In this evaluation we were seeking to identify and understand the factors associated with better uptake of the GetREAL staff training/change intervention, and to investigate the association between uptake of the intervention and 12-month outcomes.
The final product of this review is the articulation of realist programme theories that provide plausible explanations [expressed as context–mechanism–outcome (CMO) configurations] as to the contextual circumstances within which a recovery-oriented staff training/change programme leads to lasting change, and the mechanisms that are operating to produce this desirable outcome. The realist programme theories identified in the rapid realist review provided a framework for analysis of the data collected during phase 3 of the REAL programme when the GetREAL intervention was assessed through the cluster randomised controlled trial (see Theory-led findings).
A secondary purpose of the review was to present a synthesis of the complexity of the staff, unit and organisational factors that influence the uptake of a staff training/change programme to increase recovery-based practice, something that had not been undertaken in this way before.
Objectives and focus of the review
The over-riding question for the review was ‘When multidisciplinary teams working in a mental health inpatient rehabilitation setting participate in a work-based training/change programme aimed at increasing their engagement with recovery-oriented practice, what factors (configurations of context and mechanism) enable, or inhibit, lasting change in practice?’. Although the GetREAL intervention was specifically focusing on increasing service users’ engagement in activities, preliminary scoping searches revealed that there was relatively little published literature relating specifically to staff training interventions for increased service user activities. In order, therefore, to be able to engage with a sizeable (but still relevant and useful) body of literature, the review team (MG, SB, HB and SC) agreed to expand the focus of the review from training to increase service user activities, to training to increase recovery-oriented practice more generally. In support of this decision was the fact that, in the cohort study carried out in phase 4 of the REAL research programme, the degree of recovery-oriented practice of the inpatient mental health rehabilitation units was found to be positively associated with successfully discharging service users to the community.
Definitions of recovery-oriented practice are numerous; for the purposes of this review, we drew from the Joint Commissioning Panel for Mental Health’s guidance on mental health rehabilitation services,67 which refers to valuing service users as partners in a collaborative relationship with staff to identify and work towards personalised goals, and encompassing the values of hope, agency, opportunity and inclusion. However, we did not make judgements on how recovery principles were defined or applied in the literature, but accepted the authors’ various approaches.
The concept of ‘lasting change in practice’ is also subjective in terms of how long it will last. In essence, we were interested in observable change – of staff attitudes, behaviours and/or working practices – that continued beyond the immediate aftermath of the training programme. Recognising that many training/change programmes are long term of themselves, for the purpose of the review (and for consistency with the GetREAL intervention) the working definition of long term was at least 6 months after the end of the programme if it was finite, or 12 months after its commencement if it was ongoing.
Methodology
Realist review methods were used. This is a theory-driven methodology that involves:
-
articulating (as ‘programme theories’ expressed as relationships between context, mechanism and outcome) key assumptions behind why interventions are believed to work
-
using existing research as case studies with which to test and refine those theories in order to produce plausible explanations as to ‘what works for whom in what circumstances and in what respects’ (p. 74). 103
Realist reviews aim to produce explanations that can be transferred across different contexts and populations; as such, they are wide-ranging in the evidence consulted, are time-consuming to conduct and can often suffer from ‘scope creep’. 104 The time frame to conduct this review was short, and, in order to be of practical use to the realist evaluation, the scope and output necessarily needed to be as tightly focused as possible on the specific context of the GetREAL intervention. Accordingly, it was appropriate to streamline the process and use a rapid realist review methodology, drawing from (but not replicating) the methodology described by Saul et al. ,105 which is ‘intended to incorporate the theory specification of a realist review and the boundary clarification aim of a scoping review’. 105 The main output of this review reported here was, therefore, the articulation of programme theories, to provide a framework for the analysis of existing data already collected for the phase 3 trial.
Stakeholder consultation
Key to the rapid realist review methodology, as advocated by Saul et al. ,105 is the involvement of a local reference group (LRG) and an expert panel (EP): the former, to ensure that the review is appropriately focused and that the findings have utility for the context of the GetREAL intervention and its evaluation; the latter, to ensure that the findings have wider validity. These groups were therefore created for our review. Our LRG consisted of members of the original GetREAL project team, engaged to ensure that the review would produce results that are directly relevant to the local context of the project. Our EP comprised researchers and practitioners who are actively engaged in conducting work in the area under review, engaged to ensure that the review was appropriately focused and the evidence was appropriately interpreted to produce results that have a wide validity. The terms of reference of the LRG and EP groups are provided in Appendix 13. Both groups were consulted to expedite the discovery of relevant documents for the generation of candidate programme theories; the LRG was consulted for validation and prioritisation of candidate programme theories. We met the LRG to report, and obtain feedback, on our draft findings from this review and the realistic evaluation; EP feedback on the review will be sought when further theory testing and refinement has taken place.
In addition to input from the LRG and EP, the REAL programme documents, including the GetREAL manual, were available to the review team as a fruitful and readily available source of potential programme theories, thus further facilitating and expediting the review process.
Overview of the methodology
Whereas a conventional systematic review takes place in a stepwise manner with one stage following another in a logical sequence, realist review methodology is more complex, with different processes taking place in parallel and informing one another. This creates challenges when it comes to reporting the methodology and the results, as the two are inextricably linked. To assist the reader, Figure 8 is a flow diagram showing, in boxes, the main steps to the review, presented for clarity as a logical progression but in practice being performed less sequentially. The column to the left of the figure indicates corresponding sections in this methodology part of this chapter where each box or group of boxes is described; the bracketed text to the right of the figure indicates corresponding sections in the following results part of this chapter. The chapter sections are also cross-referenced with the box step numbers for further clarity. Within this structure, as far as possible, the Realist And Meta-narrative Evidence Syntheses: Evolving Standards (RAMESES) publication standards for realist synthesis106 have been followed.
Preliminary scoping work
This corresponds to step 1 in Figure 8. Initially, it was necessary for the review team members (MG and SB) who were new to the occupational therapy/recovery field, to obtain rapidly an understanding of the inpatient mental health rehabilitation setting. To this end, three reviewers (MG, SB and HB) visited a local inpatient mental health rehabilitation unit (not involved in phase 3 of the REAL programme) that helped the team understand some of the constraints and opportunities for engagement with recovery-based practice. This, combined with preliminary readings of the GetREAL intervention documentation and literature provided by the LRG and EP, enabled the team to start to understand some of the contextual constraints/opportunities to staff engagement with recovery, and some potential areas of conflict between staff members in the MDT. Drawing from soft systems methodology (a systems approach to help stakeholders gain an understanding of a complex problem situation),107 a rich picture was created to illustrate the different staff disciplines, service users and other actors, and relationships between them, the local unit environment, wider organisational environment and broader economic/social environment in which the unit is situated. This rich picture served as a framework on which to hang exploratory discussions in the first few meetings of the review team; the rich picture was added to as new factors emerged, and was instrumental in the early stages of theory development.
Searching processes
This corresponds to step 2 in Figure 8. The aim of the searches was to find literature to use to develop theories about why and how recovery-oriented training programmes in the context of inpatient mental health rehabilitation are supposed to ‘work’, and who for, and theories about why they might not have worked.
Sources of these theories, and the order in which they were sought and engaged with, were:
-
project documentation and notes relating to the REAL project
-
sources identified from the LRG and EP members
-
strategy-led bibliographic database searches
-
grey literature searches
-
reference and citation searches of key papers identified above.
REAL project documentation
The starting point was a consideration of the GetREAL intervention itself: the review team had access to the GetREAL training manual, publication outputs and other outputs in progress, and notes from dissemination events and conferences presentations including points raised during discussions with audience members. These sources are listed in Appendix 14. All of these provided an insight into the ways in which the GetREAL research team expected this intervention to ‘work’ and some initial ideas on possible explanations as to why it worked less well than expected, including potential areas of tension between different staff disciplines.
Local reference group and expert panel consultation
The LRG and EP members were contacted by e-mail to request that they forward any published or unpublished materials that, in their opinion, would be of use in answering our research question for the rapid realist review. This yielded several relevant documents that were not discovered by other means; as discussed below (see Document selection: other searches), where documents from fields other than mental health inpatient rehabilitation were suggested by the LRG and EP members, they were considered carefully.
Strategy-led bibliographic database searches
The bibliographic database search strategy was developed by the lead reviewer (MG), who is an experienced information scientist. The strategy used search terms relating to the type of intervention of interest (workplace staff training/learning for recovery-based practice) and the setting (inpatient mental health rehabilitation units). There were no limitations in the strategy as to the type of paper, geographical setting or date, but results were limited to English language only. Search terms were developed through use of the citation pearl growing technique108 in conjunction with consultation with the review team.
The databases searched were Applied Social Sciences Index and Abstracts (via ProQuest, to 8 September 2014), Cumulative Index to Nursing and Allied Health Literature Complete (via EBSCOhost, to 5 September 2014), The Cochrane Library (including Cochrane Database of Systematic Reviews, Health Technology Assessment database, Cochrane Central Register of Controlled Trials) (via Wiley, to 8 September 2014), MEDLINE (via EBSCOhost, to 5 September 2014), PsycINFO (via ProQuest, to 5 September 2014), Scopus (via Elsevier, to 5 September 2014) and Web of Science (via Thomson Reuters, to 8 September 2014). An indicative search history is provided in Appendix 15.
It is acknowledged that the focus of these database searches was relatively narrow, whereas ideally in a realist review they are broader, drawing from literature in other fields where it could be reasonably inferred that the same mechanism(s) might be in operation. 106 However, as noted above, the time to undertake this review was limited (hence the need for a ‘rapid realist review’), and the specific objective of the review was to inform a realistic evaluation of a staff training programme set in this particular context; therefore, it was appropriate to keep the searches tightly focused for maximum relevance to the evaluation. 105 As discussed below (see Document selection: other searches), the net was cast more widely for the literature forwarded by the LRG and EP members, and for the grey literature and reference/citation searches.
Grey literature searches
For the grey literature searches, the following websites were browsed between 5 November 2014 and 10 November 2014, for relevant materials [where appropriate, following up ‘leads’ by further general internet (Google; Google Inc., Mountain View, CA, USA) searches]: Department of Health, www.dh.gov.uk; NIHR, www.nihr.ac.uk/; NICE, www.nice.org.uk/; Together for Mental Wellbeing, www.together-uk.org/; Schizophrenia Commission, www.schizophreniacommission.org.uk/; Mental Health Network of the NHS Confederation, www.nhsconfed.org/networks/mental-health-network; Centre for Mental Health, www.centreformentalhealth.org.uk/; Emergence, www.emergenceplus.org.uk/; Mental Health Foundation, www.mentalhealth.org.uk; Mind, www.mind.org.uk; Personality Disorder, www.personalitydisorder.org.uk; Rethink, www.rethink.org; Scottish Recovery Network, www.scottishrecovery.net/; and Royal College of Psychiatrists, www.rcpsych.ac.uk.
Reference and citation searches
Reference (backwards) and citation (forwards) searches were carried out in respect of the most relevant documents identified in the searches above.
Selection and appraisal of documents
This corresponds to step 3 of Figure 8. Different selection criteria were used for the strategy-led bibliographic database searches, and the other searches.
Document selection: strategy-led bibliographic database searches
A screening tool was developed, piloted and used to identify those documents from the strategy-led database searches that were relevant to theory generation, based on the inclusion/exclusion criteria in Box 3.
Inpatient rehabilitation unit as defined in page 12 of the Commissioners Guide67 (i.e. low secure; high dependency; community; complex care).
ExcludeAcute mental health units; day care units/day centres.
Training type IncludeDescribes a staff training/learning programme for existing staff.
ExcludeTraining is not for existing staff (e.g. induction for new staff would not be relevant), or no training, or training for service users not staff.
Aim of training IncludeIncreased engagement with recovery-based practice (i.e. service user engagement in activities, self-management, work, volunteering, autonomy/living skills and self-care, hope, arts therapies, healthy living – diet, smoking cessation, exercise).
Also include (following team discussion)Psychosocial rehabilitation and ‘behavioural therapy’ (which can include social skills training).
ExcludeTraining for purposes other than psychiatric rehabilitation (e.g. medication checking).
Target of training IncludeMore than one disciplinary staff group (e.g. OT and nurses; nurses and support workers).
ExcludeSingle staff group.
Consideration of lasting change IncludeAt least a consideration of the factors that might facilitate/inhibit lasting change in recovery-based staff practice, attitudes or behaviour (whether or not this is actually evaluated for the training programme described).
ExcludeNo consideration of lasting change.
Additionally, at the same time as screening for relevance for theory generation, documents were screened for potential use as case studies for further theory testing and refinement, which is ongoing work to be reported separately. For this purpose, two additional criteria were used: first, that the document needed to report primary research; and, second, that the document needed to report a follow-up and an evaluation of lasting change in recovery-based staff practice, attitudes or behaviour. Screening was performed by the whole review team (MG, SB, HB and SC), following initial piloting with a subset of 10 papers to ensure that the whole team had a shared understanding of the screening criteria, for consistency and inter-rater reliability.
Document selection: other searches
The lead reviewer (MG) selected relevant sources from the project documentation, LRG and EP, grey literature searches and reference and citation searches. For these searches, the inclusion/exclusion criteria were less rigid: essentially, anything that had something useful to say about the context of inpatient mental health rehabilitation, and/or staff training and lasting change, was included. By virtue of these papers being specifically suggested by the stakeholder groups themselves, coming via highly pertinent websites, or being cited by, or citing, highly relevant papers, it was appropriate to lessen the rigidity of the inclusion criteria.
Document appraisal
As is appropriate for realist reviews, studies/papers were not excluded on the basis of ‘quality’ checklists, as ‘studies that are technically deficient in some overall sense may, if inspected closely, still provide trustworthy nugget of information to contribute to the overall synthesis’. 109 Using an approach consistent with the RAMESES publication standards for realist reviews,106 in tandem with the data extraction for theory development and testing, there was an appraisal of the contribution of any section of the data within relevant documents, based on the criteria of relevance (to theory building/testing) and rigour (whether or not the methods used to generate that piece of data were credible and trustworthy).
Data extraction
This corresponds to step 5 in Figure 8. As is usual with realist reviews, for the purposes of theory identification and development, relevant sources of literature were read and ‘engaged with’, extracting theories (formal theories of change, or otherwise) on how a recovery-oriented training/change programme was supposed to work, was thought to work or was thought not to work, and/or relevant contextual information. (Typically, such information was located in the introductory and discussion sections of documents, as opposed to formal ‘findings’.) Thus, note-taking or jottings, cross-referenced to the originating source, took the place of formal data extraction initially. For those jottings that fed into those programme theories that we prioritised for further exploration, additional notes were made against the jotting to indicate whether or not the jotting came directly from a finding of the study, whether it was the author’s opinion or speculation as to what was going on, or whether or not it came from official guidance. This then assisted the synthesis stages described below. Additionally, for those primary research documents that ultimately fed into the prioritised candidate programme theories, a data extraction matrix was used to capture details about the setting, research aims and design, intervention and outcomes, and appraisal based on relevance and rigour (see Document appraisal). Data extraction was undertaken by the lead reviewer (MG).
Analysis and synthesis
Data analysis and synthesis was predominantly undertaken by MG, but the emerging programme theories were discussed with the whole review team, and SC contributed to simplification and consolidation processes.
Identification of candidate programme theories
As the main output from this review was programme theories expressed as CMO configurations, the review group also needed a shared understanding of what we meant by ‘context’, ‘mechanism’ and ‘outcome’ for this process to be meaningful. As Pawson110 states, ‘Programmes do not come in pre-ordained chunks called contexts, mechanisms, and outcomes. Rather, these terms take their meaning from their function in explanation and their role in testing those explanations’ (p. 26). We found it helpful to work ‘backwards’ from outcome (the most straightforward of the three), and our shared understanding was as shown in Box 4.
What ‘happens’: the intended or unintended consequences of what is going on. Outcomes can be proximal, intermediate or final.
MechanismThe generative force that leads to the outcomes. It can be thought of as the response, reasoning and reaction, and, ultimately, the behaviour of the subjects/participants, to the resources or capabilities offered by or embedded in a programme.
ContextSomething that can ‘trigger’ or modify, or even block, a mechanism. The context may be provided by the programme, or it might relate to a broader contextual ‘backdrop’ within which the programme operates.
The analysis and synthesis processes went hand in hand. There were several stages to the process of progressing from the initial, informal ‘jottings’ (384 of them) from the documentary sources, to the creation of candidate programme theories (CMO configurations). Throughout the process, a careful audit trail was maintained in order to be able to trace the final candidate programme theories (after various iterations) to their original sources.
In summary, the stages of analysis from jottings to programme theories comprised identification of high-level outcomes and thematic analysis under each outcome (step 5 in Figure 8); initial CMO configurations; rationalisation and consolidation of these initial configurations using graphical methods and analysis under mechanism (step 6 in Figure 8); new (fewer) CMO configurations; and theory prioritisation through identification of key outcome of interest (step 7 in Figure 8). Because findings (theories) were emerging as the analysis proceeded, the processes are described more explicitly, with reference to these emerging findings (see Interim findings: identification of candidate programme theories). The end result was 50 candidate programme theories (CMO configurations) relating to seven mechanisms and one outcome.
Theory prioritisation
This corresponds to step 8 in Figure 8. As advocated by Saul et al. ,105 the LRG was then consulted to check that our initial theorising had some validity, and to help with prioritising theories for testing and refinement. Ideally, we would have conducted a Delphi panel exercise to obtain consensus,111 but time did not allow for more than one round of consultation (a survey of expert opinion) in our study. Therefore, an online questionnaire was developed, which is provided in Appendix 16. In the questionnaire we asked, for each of the seven mechanisms, how important the LRG member felt this was, by rating it on a five-point Likert scale (extremely important, very important, somewhat important, of little importance, not important). The questionnaire defined ‘importance’ as ‘interest, value, and relevance’ for long-term change in recovery-based practice, for MDTs working in a mental health rehabilitative setting. Then, under each mechanism, we listed the relevant candidate programme theories and asked them to indicate, for each mechanism, which three theories they thought were the most important and worthy of further investigation.
Many of the candidate programme theories were quite lengthy and a little unwieldy at this stage, in an attempt to capture nuances of detail that came from the literature. However, in order to make the theories more accessible to the LRG, these theories were abridged for brevity and clarity.
For example, the full candidate programme theory RS2 (RS, theory 2) read:
When the staff have high levels of job satisfaction/low burnout, they are likely to be engaged and motivated by the change programme. Increased engagement/collaboration with service users during the training programme can increase job satisfaction and hence further motivation towards recovery-based practice. When the staff feel that they are working in a supportive organisation, with collaboration between different staff disciplines and where they can seek support from colleagues and supervisors, this will also mitigate stress/burnout and increase job satisfaction. Conversely, where there is a high prevalence of stress, low job satisfaction, and burnout amongst staff disciplines, those members of staff affected are unlikely to be engaged and motivated by a change programme (any change programme). When staff perceive service users as ‘difficult’ to engage, they are more likely to have low job satisfaction.
The ‘essence’ of this theory was that high levels of job satisfaction and low levels of burnout tend to lead to engaged and motivated staff. The theory tried to capture various contextual factors emerging from the literature that might affect job satisfaction and burnout. 112–114 These all related to supportive organisations/colleagues and collaboration (between staff, and with service users). Therefore, our simplified statement of the candidate programme theory RS2 read: ‘When the staff have high levels of job satisfaction and low burnout, they are likely to be engaged and motivated by the change programme, fostered by supportive organisations/colleagues and collaboration’. For each abridged candidate programme theory presented to the LRG, the key features (e.g. job satisfaction and low burnout) were printed in bold to help provide further focus to the reader.
As will be presented in the results section (see Theory prioritisation: consultation with the local reference group), the responses from the LRG enabled us to prioritise two mechanisms and, below those, seven theories (step 9 in Figure 8).
Results
Interim findings: identification of candidate programme theories
This section describes more fully the steps taken to progress from the initial extractions/jottings, to the programme theories (CMO configurations), and the interim findings that fed into those theories.
In constructing the CMO configurations, we found it helpful to work ‘backwards’ from outcome. Therefore, the first step was to identify the outcomes our jottings related to, and five high-level outcomes of interest/importance were identified (Box 5). Each jotting was therefore initially mapped to one or more relevant high-level outcomes (even though in some instances, the jotting itself seemed to relate to more of an intermediate outcome). This corresponds to step 5 in Figure 8.
O1: high level of attendance of training.
O2: engaged staff during training.
O3: management buy-in to training.
O4: long-term change in practice.
O5: little change in practice.
Jottings under each outcome were then thematically analysed and given approximate descriptive labels. For example, jotting 384 from Corrigan et al. ,115 reading ‘Strategies which help staff members to cope with emotional exhaustion and depersonalisation may lead to greater optimism about the development and implementation of behavioural strategies’, was given the label ‘burnout’ under outcomes O4 and O5.
Label by label within each high-level outcome, the jottings were read across in order to try to obtain a sense of what was going on. We then asked what ‘lay beneath’: what was it about the context that was triggering a particular mechanism in the individual (and what we thought that mechanism was), to result in the outcome? Answering this led us to, for each high-level outcome, a catalogue of corresponding context and mechanism pairs – and, in some cases, context, mechanism and intermediate outcome configurations. Some of the intermediate outcomes became, in turn, contexts feeding into later CMO configurations. As is typical with realist methodology, this was an iterative process: CMOs were being developed in parallel to screening, reading and annotating the documents found in the searches.
In total, 103 initial CMO configurations were generated (outcome O1, high level of attendance: 9; outcome O2, engaged staff during training: 24; outcome O3, management buy-in to training: 3; outcome O4, long-term change in practice: 36; outcome 5, little change in practice: 31). This process took place quite rapidly, and the initial wordings for CMO configurations were quite rough-and-ready. Moreover, some of the CMO configurations seemed similar, and some of them included multiple possible contexts or mechanisms in and/or configurations. Further work was therefore needed to further develop and consolidate the configurations that we had. This was facilitated by a graphical method to map the configurations and obtain a ‘whole picture’ perspective, and common contexts and mechanisms were found to apply to multiple configurations. From this ‘whole picture’ perspective, and discussion within the project team, a relatively small set of underlying mechanisms emerged in association with each of the outcomes: O1, high level of attendance; O2, engaged staff during training; and O4, long-term change in practice (and its reciprocal, O5, little change in practice). For the purposes of theory refinement, outcome O3, management buy-in, was treated as a context for the other outcomes. The theories were then re-examined and rationalised under the relevant mechanisms, resulting in 77 new CMO configurations.
Theory prioritisation: by outcome
As it would not be practical to test and refine all 77 configurations, there was a need for theory prioritisation. This was achieved in two steps: first, by focusing on those outcomes of most relevance to the review question (step 7 in Figure 8); and, second, by consulting the LRG to identify the most important mechanisms and the most important configurations (step 8 in Figure 8).
We decided to focus on outcomes O4 and O5 (long-term change in practice or little change in practice) for maximum relevance to the research question. (For the purposes of further analysis we decided to group these together and label the outcome as long-term change.) Although the other outcomes were undoubtedly important for any training programme, we already knew that a high level of attendance of training, and staff engagement during training, did not necessarily lead to lasting change in practice. This left 50 CMO configurations relating to seven underlying mechanisms, leading to long-term change. We gave the mechanisms shorthand labels as follows: recovery is important (RI), supported change (SC), recovery is realistic (RR), resourced for recovery (RFR), recovery is everyone’s responsibility (RER), reinforced direction (RD) and receptive staff (RS). These mechanisms are described in the findings section (see Contexts and mechanisms for lasting change: the 50 candidate programme theories).
Document flow diagram
Figure 9 shows a document flow diagram illustrating the search and screening processes, the number of papers found at each step and their relevance.
Document characteristics
In addition to the GetREAL intervention materials listed in Appendix 14, 50 documents were informative for the generation of the candidate programme theories for long-term change. Of these, 27 documents contributed to the seven priority candidate programme theories. These documents comprised 15 reports of primary studies,82,116–129 one draft manuscript (Mona Eklund, Department of Health Sciences, Lund University, Sweden, 29 September 2014 and 31 January 2015, personal communications), one Cochrane review,130 one thesis,131 two opinion pieces,132,133 five grey literature reports/guidance67,134–137 and two book chapters. 138,139 The 23 papers that fed into other (not prioritised) candidate programme theories for long-term change are listed in Appendix 17.
Table 33 summarises the key characteristics of the 15 documents reporting primary studies that fed into the priority theories.
Authors | Programme theories informed by study | Country | Design/summary | Findings/utility to review |
---|---|---|---|---|
Marlowe et al.116 | SC5, SC4, RI1 | USA | Comparative case report between two units with 100 and 38 staff members Staff residential training programme on a model psychosocial treatment unit, and follow-up academic consultation |
Poor fidelity to intervention, few empirical data, but case studies provide useful pen-portraits of the two units |
Birkmann et al.82 | SC5, SC2, RD1 | USA | Case report, c. 1000 direct-care staff Collaboration between academic department and 550-bed state psychiatric hospital, including formal undergraduate and in-service staff training in group facilitation skills/therapeutic communication skills, discharge planning and psychiatric rehabilitation practice; improving organisational processes, management processes and internal communications; fostering a recovery focus |
Good attendance of in-service training and improvement of organisational processes. Collaborative development of 3-year strategic plan. Few before-and-after comparison data, but paper describes achievements and acknowledges some barriers |
Le Boutillier et al.117 | RS3, SC4 | UK | Qualitative: 10 focus groups and 32 structured interviews Exploratory study of staff in five mental health service community care teams to investigate what staff say they do to support recovery and to identify perceived barriers/facilitators associated with providing recovery-oriented support |
Context is community care, not inpatient. Main theme: competing priorities. Subthemes: health process priorities where clinical systems dictate the direction of practice, business priorities, staff role perceptions. Also lack of shared understanding of what ‘recovery’ actually meant |
Way et al.118 | SC4, RI1 | USA | Programme evaluation through questionnaires, WAS and WES, 2 weeks before training and 12–15 weeks afterwards Mandatory 3-day training programme (NY Core Curriculum) including recovery module, for all staff of adult and forensic mental health facilities, having direct contact with service users. Intensive evaluation for 3 of 20 participating institutions, after 12–15 weeks |
Significant increase in staff perception that ‘what the recipients say makes a difference in their treatment’, that ‘staff spend time talking to and doing things with recipients’ and in staff’s ‘believing recipients would get out of the hospital and not come back’. Similar findings were found from service user questionnaires. Significant increases in WAS Support Scale for both staff and service users. In addition a significant increase in staff’s perception of autonomy |
Bartholomew and Kensler119 | SC4, RS4, SC3, SC2, RI1 | USA | Case report Implementing IMR first in transitional unit and then other complexes including acute and admissions, in a state psychiatric hospital. Four phases to programme include staff training, and ongoing supervision with consultants. IMR client- and clinician-rating scales were administered at baseline and every 6 months |
Moderate level of implementation fidelity, but clinical competence in IMR varied significantly across facilitators and areas of the hospital. IMR rating scales results from pilot and first year showed positive trends but data were incomplete and sample size too small for statistical significance. Institutional barriers to the programme are cited (e.g. resources), and need for inclusion of recovery goals in patient’s treatment plan. Authors mainly draw from anecdotal information |
Strating et al.120 | RD4, RS1, RER5 | Netherlands | Multiple case report Evaluation of four quality improvement collaboratives in long-term mental health care, including on focusing on recovery-oriented care |
Of marginal relevance: the jotting used in the theory generation came from the introductory section |
Burdett and Milne121 | SC4, RD1, RFR4, RI1 | UK | Qualitative exploratory: 11 staff interviews Identifying ‘settings events’ that influence staff’s use of behavioural therapy, after 3-year ‘innovation’ (training/programme) period |
Most important ‘settings event’: the course. Also important: nursing officer and peer support; psychologist support; rewards of working with patients. Barriers: lack of equipment and feedback, too many rules and regulations. Small sample size and no before-and-after comparison data |
Linhorst122 | SC4 | USA | Qualitative: staff focus groups of 21 executive, 23 middle management and 157 direct care staff of four long-term psychiatric facilities Exploratory study to identify key issues when creating and executing state-wide PSR services by learning from the development, implementation, and maintenance of Missouri’s existing inpatient psychiatric programmes |
Key issues identified: understanding (‘conceptualising’) PSR; operationalising client choice; doing PSR with forensic clients; intensive training; training community mental health agencies; staff roles and role expansion/flexibility; involvement in community activities; continued contact with clients in the community; evaluating both PSR process and outcome, and including client evaluation of services. Little supporting evidence is presented |
Donat et al.123 | SC4 | USA | Case report; 234 staff in psychiatric facility Evaluation of 2-day staff training workshop on behavioural methods. Comparison of knowledge of behavioural methods. Inventory performance of direct care staff pre and post training |
Knowledge levels increased most for nursing staff. No supporting data are presented |
Corrigan et al.124 | SC3 | USA | Staff survey of 47 nursing, professional and administrative staff of extended care units in state hospital Using a peer nomination strategy to identify a subgroup of ‘behavioural advocates’ (champions) from the ranks of line-level staff working in a state hospital. To investigate differences between advocates and non-advocates in terms of their perceptions of behavioural innovations |
Advocates, compared with non-advocates, reported significantly fewer barriers to implementing behavioural interventions, perceiving institutional constraints and philosophical opposition to be less an impediment than their peers. Advocates also had significantly greater knowledge of behavioural principles, but still only identified 45% of items correctly |
Nancarrow et al.125 | SC2, RER5 | UK | Mixed methods: systematic review and survey of 253 staff from 11 community rehabilitation and community care teams Merging review and survey staff perceptions data using qualitative content analysis to arrive at a framework that identifies characteristics and proposes competencies that support effective interdisciplinary team work |
Identified 10 competencies of an effective interdisciplinary team. Different context (community rehabilitation/intermediate care, not mental health and not training/change management) but still of utility to the review |
Pollard et al.126 | SC2 | Israel | Brief case report and randomised controlled trial In-service training programme for different staff disciplines in a 350-bed psychiatric facility with acute and chronic units. Practitioner’s Beliefs, Goals, and Practices in Psychiatric Rehabilitation Questionnaire was administered to control (waiting list) (n = 27) and training (n = 28) groups before and after training |
Significant improvements in programme groups observed on ‘staff directed paradigm’ and ‘evidence-based practices’ factors, and overall total score. Results do not distinguish between staff from acute/long-term units or between different staff disciplines. The case descriptive data are useful |
Narevic et al.127 | SC2 | USA | Case report Describes staff training, staff support groups and increased programmatic behaviour monitoring over study period in a 170-bed skilled nursing facility, and (routine) data on incidents of physical aggression towards peers and objects |
Descriptions of staff support groups might be generalised to other contexts |
Valinejad128 | SC2 | UK | Case report, and evaluation from 10 post-training questionnaires 10 × 2-hour weekly training sessions for multidisciplinary staff team to raise awareness in role of ‘psychological approaches’ in care of clients with long-term health needs. Post-training questionnaires assessed perceptions of usefulness and areas where further training is needed. On average, there were 12 attendees per week and nine people attended at least 60% of the programme |
Staff comments related to staff–patient interactions, more empathy and change in attitude. Further areas for training identified using clients as case examples, dealing with challenging behaviours and aggression, and cognitive approaches with mental illness. Identified need for MDT teaching sessions. The training intervention is also described in some detail |
Ahmed et al.129 | SC2, RD1, RFR4, RER5 | USA | Case report Describes organisational transformation towards recovery-based care in a psychiatric hospital, in association with academic partners. Includes compulsory, annual staff attendance of 3-hour recovery workshops, and unit problem identification and goal-setting |
No formal evaluation and little about impact of training: describes the organisational systemic changes, including creation of ‘recovery teams’ that include the patient. Acknowledges organisation and external constraints |
Despite a large volume of literature meeting the inclusion criteria, there were relatively few primary studies reporting the evaluation of long-term change in behaviour following a recovery-oriented staff training intervention in the context of mental health rehabilitation. As can be seen from Table 32, of the primary studies that fed into the priority theories, three studies that evaluated change following a training programme did so via a before and (immediately) after comparisons,123,126,128 and two performed comparisons over longer periods. 118,119 Most of the primary studies provided anecdotal, case-study type data from which contexts and mechanisms leading to long-term change could be inferred, but not demonstrated.
Main findings
Contexts and mechanisms for lasting change: the 50 candidate programme theories
As described above (see Identification of candidate programme theories), the steps of data analysis, starting from the jottings taken from the phase 1 search results, led initially to a very complex, ‘messy’ picture – as might be expected from a change intervention for a multidisciplinary staff group with complex relationships with service users, typically under pressure for time, resources and competing demands. However, several stages of theory consolidation and refinement (steps 5–7 in Figure 8) led us to postulate that there were essentially just seven mechanisms that may or may not be triggered within individual members of staff to lead to a long-term increase in recovery-based practice. The mechanisms were given the shorthand labels RI, SC, RR, RFR, RER, RD and RS. Box 6 provides a statement of each mechanism.
Each postulated mechanism starts with:
When staff groups of a mental health inpatient rehabilitation unit have taken part in a training programme aimed at increasing their engagement with recovery-based practice, they will make long term changes and increase their engagement with recovery-based practice if . . .
This is stated only once under the first postulated mechanism.
RI: when staff groups of a mental health inpatient rehabilitation unit have taken part in a training programme aimed at increasing their engagement with recovery-based practice, they will make long-term changes and increase their engagement with recovery-based practice if they feel that recovery is important to themselves individually and to the organisation.
SC: they feel encouraged/motivated/supported by management and colleagues to change.
RR: they, and the service users, feel that working collaboratively towards recovery is realistic.
RFR: they feel they have the resources to do so and/or barriers (individual, group or organisational) have been removed.
RER: they, and the service users, feel that recovery is everyone’s responsibility – all staff, all service users.
RD: they know exactly what is expected of them, and this clear direction is continually reinforced.
RS: they feel involved, valued, enthusiastic and engaged in the programme.
For these mechanisms, we had 50 candidate programme theories postulating in more detail how specific contextual factors might cause these mechanisms to fire. These contextual factors could be classed as factors relating to one of the following groups: training/change programme, resources, organisational structures and systems, culture and climate, and staff team. Figure 10 summarises the context and mechanism groups that were present in our 50 candidate programme theories relating to long-term change.
Table 34 shows which context groups trigger which mechanisms, and the specific candidate programme theories that articulate each context group/mechanism combination. Short summaries of the content of each theory, organised by mechanism, are shown in Box 7. A full statement of each of the candidate programme theories in the format in which they were presented to the LRG for validation and prioritisation is shown in Appendix 18.
Context groups | Mechanism | ||||||
---|---|---|---|---|---|---|---|
RI | SC | RR | RFR | RER | RD | RS | |
Training/change programme | RI2: performance linked to service user feedback RI3: training fits job description/continuing professional development requirements RI5: staff identify need for change RI6: training is repeated/refreshed RI8: action planning |
SC2: regular supervisions/collaborative meetings SC3: change agent/champion role SC6: involvement of relevant professional groups in programme development |
RR1: service user involvement in programme design/delivery RR10: quick wins demonstrate progress |
RFR7: positive, collaborative culture between staff and service users |
RER1: shared training across staff groups RER2: training ensures shared understanding of recovery RER5: action plan developed collaboratively |
RD2: training is practical and specific RD3: training is repeated/refreshed RD4: clearly articulated action plan RD5: regular supervisions |
RS1: action plan developed collaboratively RS5: all staff receive training RS6: programme is part of research project RS7: programme is tailored to staff group |
Resources | RFR1: strong community and family links RFR3: adequate shift handovers RFR4: adequate staffing capacity, time, space, resources |
RER4: administrative burdens, competing work priorities | |||||
Organisational structures and systems | RI1: management endorsement and prioritisation RI4: consistent with unit mission |
SC4: management support, supported role flexibility SC5: organisation structures, etc., modified to support change |
RFR6: change built into existing organisational structures, etc. | RD1: new staff activities reflected in organisational structures | RS3: incorporate recovery into existing change programme | ||
Culture and climate | RI7: recent major negative event | SC1: publicly recognise, reward, incentivise programme successes | RR5: service users and staff work together RR8: staff stress, burnout, job satisfaction RR9: encouraged autonomy, positive risk-taking supported |
RFR7: positive, collaborative culture between staff and service users | RS2: high job satisfaction, low burnout RS4: climate of job uncertainty, fear |
||
Staff team | RR2: peer support workers working with staff RR3: staff understand recovery is non-linear RR4: staff find service users with complex needs hard to engage with RR6: paradigm shift from custodial/protective model to recovery-based care RR7: appropriate medication regime |
RFR2: shift/working pattern flexibility RFR5: appropriate medication regime |
RER3: staff groups reflect together RER6: role flexibility, common understanding and support between staff groups |
When staff groups of a mental health inpatient rehabilitation unit have taken part in a training programme aimed at increasing their engagement with recovery-based practice, they will make long-term changes and increase their engagement with recovery-based practice if they feel encouraged/motivated/supported by management and colleagues to change.
Theory SC2: regular supervisions/collaborative meetingsRegular supervisions or collaborative meetings between staff groups and the training team, and/or staff members and a local change lead (‘champion’), when these take place within a supportive organisational culture, will help staff members feel supported by their peers and managers in the change programme.
Theory SC3: change agent/champion roleAppointing individuals from the staff team to ‘champion’ or act as a ‘change agent’ for the programme serves to persuade, encourage and empower (i.e. support) other staff members to change. To be effective, those individuals should have optimism, good interpersonal skills, management support and the respect of colleagues, and be influential. Ensuring that the ‘champion’ is associated with a role, rather than a single individual or individuals, ensures long-term continuity: dependence on key enthusiastic individuals runs the risk of staff members losing that support if those individuals leave or move to other departments.
Theory SC4: management support, supported role flexibilityIt is important that staff groups feel supported by their management to change. This support can be realised through active management endorsement and prioritisation of the programme, and encouraging change (e.g. getting involved themselves, endorsing an action plan for change, quantifying progress, incorporating external drivers to change). This support will help the staff feel encouraged to change even if increased engagement with recovery entails moving outside their traditional occupational role. Management support of positive risk-taking is also desirable.
Theory SC5: organisation structures, etc., modified to support changeIf the management team modify organisational structures, processes and systems (e.g. working practices, responsibilities, policies, documentation and performance reviews) to facilitate the move towards recovery-based practice, staff members will feel supported by management in changing their practices.
Receptive staffWhen staff groups of a mental health inpatient rehabilitation unit have taken part in a training programme aimed at increasing their engagement with recovery-based practice, they will make long-term changes and increase their engagement with recovery-based practice if they feel involved, valued, enthusiastic and engaged in the training/change programme.
Theory RS1: action plan developed collaborativelyStaff will feel engaged, valued and involved (and hence receptive to change) if an action plan for change has been developed collaboratively between different staff groups and service users. If the action plan utilises existing strengths and experiences within the team and service users, this will also contribute to the sense of feeling valued. Conversely, if an action plan is imposed on staff members, they will be unlikely to feel engaged, valued or involved.
Theory RS3: incorporate recovery into existing change programmeIn an organisation that has undergone much recent change, staff members may feel, at best, unenthusiastic about an additional training/change programme to increase recovery-based practice or, at worst, unenthusiastic, disillusioned or downright pessimistic. Incorporating recovery into an existing change programme may help with staff engagement and enthusiasm.
Theory RS4: climate of job uncertainty, fearIn an organisation that has a climate of uncertainty and fear (e.g. a context of economic cutbacks and job losses), it is likely that staff members are likely to feel disillusioned, and uninterested or pessimistic in/about a change programme. We propose that this negative organisational context therefore blocks the mechanism RS – staff will be unlikely to feel involved, engaged or valued.
Theory prioritisation: consultation with the local reference group
As described in the methods section (see Theory prioritisation), online questionnaires were sent to the LRG (step 8 in Figure 8). Responses were obtained from 6 out of 12 LRG members. The responses to the importance of the seven postulated mechanisms are summarised in Table 35. That none of the respondents classified any of the mechanisms as being of little importance or not important gave all mechanisms some validity.
Mechanism | Extremely important | Very important | Somewhat important | Of little importance | Not important |
---|---|---|---|---|---|
RI | 0 | 6 | 0 | 0 | 0 |
SC | 4 | 1 | 1 | 0 | 0 |
RR | 0 | 4 | 2 | 0 | 0 |
RFR | 1 | 4 | 1 | 0 | 0 |
RER | 3 | 0 | 3 | 0 | 0 |
RD | 2 | 3 | 1 | 0 | 0 |
RS | 3 | 3 | 0 | 0 | 0 |
The responses to the questions for which the LRG members were asked to indicate their ‘top three’ theories under each mechanism are provided in Appendix 19.
The review team met to discuss these responses and agreed, in the absence of any overwhelming indication from the LRG as to important (or unimportant) mechanisms and theories, that a pragmatic approach to prioritising the theories would be to identify the top two mechanisms by importance and then, under these top two mechanisms, to identify those theories with the most ‘top three’ votes. From Table 34 it can be seen that the top two mechanisms identified were SC and RS. From Appendix 19 it can be seen that for the mechanism SC, four theories (SC2, SC3, SC4 and SC5) can be considered as ‘top’, having four or five votes each. For the mechanism RS, three theories (RS1, RS3 and RS4) can be considered as ‘top’, having three or five votes each. To aid the reader, the top seven priority theories are given in Box 7. (Theory prioritisation is step 9 in Figure 8.)
As an end product, the rapid realist review provided a list of 50 candidate theories including seven prioritised theories. The seven priority theories were tested with the available evidence from the qualitative case study data. In addition, four other (non-priority) theories from the list, which were found to correspond with the qualitative case study data, were also tested with available evidence and are presented in the following section.
Qualitative case study analysis
This section of the chapter presents findings of the case study analysis that explored the factors associated with variation in units sustaining the skills and changes in practice that were imparted during the GetREAL intervention. The analysis was informed by the candidate programme theories identified in the rapid realist review that were ‘tested’ using data relating to three inpatient rehabilitation units that had received the intervention.
Methods
We used a case study design based on multiple sources of existing data collected during phase 3 of the REAL programme. These data included staff focus groups and service user participant interviews carried out during the qualitative component of phase 3 of the REAL study; the GetREAL team OTs’ daily reflective practice diaries; the unit staff evaluation forms; the GetREAL intervention fidelity assessments; and supervision notes compiled by the GetREAL team members and their supervisors at the end of the enabling stage of the intervention in each unit. The data sources are included in Box 8.
AP: action plan.
CN: charge nurse.
FS: fidelity sheet.
NA: nursing assistant.
OTi: occupational therapist instructor.
RD: reflective diary.
SFG: staff focus group.
SN: staff nurse.
SR: supervision record.
SUI: service user interview.
TEN: training evaluation notes.
Purposive sample
A purposive sample of three inpatient mental health rehabilitation units was drawn from the units that received the GetREAL intervention (and remained open until the end of the cluster randomised controlled trial). Rather than representation, the sampling aimed to achieve a range of unit characteristics: location (urban, suburban, rural), type (hospital or community based), whether or not staff had received any training in recovery-based practice, and change in the phase 3 trial primary outcome (scores on the time-use diary74) between baseline and 12-month follow-up. Thus, the following selection criteria were used:
-
The unit took part in phase 3 of the REAL programme and received the GetREAL staff training intervention.
-
The unit staff took part in a focus group and service users participated in a qualitative interview during the qualitative component of phase 3 of the REAL programme.
-
The unit had either a high, mid or low mean change score in service user activity (as assessed using the time-use diary74) during the cluster randomised controlled trial in phase 3 of the REAL programme.
-
Complete data were available for the unit (GetREAL action plan, GetREAL fidelity assessment, staff focus group transcript, at least one service user interview transcript, unit staff evaluation notes, GetREAL team OTs’ daily reflective diaries, GetREAL team’s supervision records).
Of the 20 units that received the GetREAL intervention, one closed during the course of the trial and was excluded from follow-up data collection. Of the remaining 19, 11 did not meet the criteria for the case analysis because they had not participated in the qualitative component of phase 3 of the REAL programme. Based on the mean change in unit time-use diary scores between baseline and 12-month follow-up (Table 36), three units with low, mid and high change scores were selected for the case studies.
Unit code | Difference in time-use diary scores (follow-up minus baseline) | Location | Type | Number of beds | Team (staff working on the unit at baseline) | Recovery-based practice formal training received | |||
---|---|---|---|---|---|---|---|---|---|
Psychiatrist | Psychologist | OT | Baseline | 12-month follow-up | |||||
0102 | 3 | Urban | Hospital | 14 | Yes | Yes | Yes | Yes | Yes |
0804 | 5 | Urban | Hospital | 26 | Yes | Yes | Yes | Yes | Yes |
2902 | –6 | Rural | Community | 31 | No | No | Yes | Yes | Yes |
3106 | 2 | Suburban | Community | 18 | Yes | Yes | Yes | Yes | a |
3301 | 4 | Urban | Community | 25 | Yes | Yes | Yes | Yes | Yes |
3704 | –2 | Urban | Community | 20 | Yes | Yes | Yes | a | a |
4203 | 7 | Suburban | Hospital | 15 | Yes | Yes | Yes | Yes | Yes |
4204 | 4 | Urban | Community | 15 | Yes | Yes | Yes | Yes | Yes |
Unit 4203 had the largest change and unit 2902 had the smallest change in time-use diary scores. The mean change score for the remaining five units was 4. Two units (units 3301 and 4204) had a mean change score of 4. Unit 3301 was selected because a complete data set (see above for the selection criteria) was available.
Accessing data
Written informed consent was gained from the OTs who had led the two GetREAL teams to access their reflective diaries. Data were collated for the three units selected for the case studies: unit action plans (n = 3), unit fidelity rating sheets (n = 3), staff focus group transcripts (n = 3), service user interview transcripts (n = 4), GetREAL team supervision records (n = 6), GetREAL team OTs’ reflective diary entries (n = 26), and unit staff GetREAL training evaluation notes (n = 9). We also collated unit characteristics that had been gathered from the unit manager using the QuIRC at baseline during phase 3 of the REAL programme (n = 3). A total of 56 data items, constituting 308 pages, were reviewed and analysed.
Data analysis
The qualitative analysis was carried out by SB and HB, under the supervision of SC. The qualitative data from the three selected units were stored and managed using NVivo software (version 10; QSR International, Warrington, UK). Both researchers read the transcripts to familiarise themselves and prepare for the analysis. As we were following the ‘realist’ approach to support or challenge the theories identified through rapid realist review of literature, we used a blend of ‘realist’ and traditional ‘framework’ analysis approaches. The initial analysis was carried out using the coding, indexing and charting techniques of the framework analysis approach. 140 This was followed by an iterative process of mapping evidence against theories identified from the literature to challenge or support them. The qualitative case study analysis was carried out using the following steps:
-
All textual data were entered into NVivo software and coded with an index of themes and subthemes.
-
Data for each theme were entered into a matrix to analyse themes across the data sources and cases.
-
The characteristics of selected units were constructed from the data.
-
The seven priority candidate programme theories identified in the rapid realist review were tested by collating supporting evidence for them from the case study data sources.
-
When data supported or challenged other theories identified in the rapid realist review, this was also noted.
-
The data were synthesised and a final interpretation of emerging patterns and explanations was made in relation to the candidate theories.
Findings
Unit profiles
This unit had the highest mean change in primary outcome (service user activity as assessed using the time-use diary74) between baseline and 12-month follow-up. This unit had opened within the past 5 years. It was based in a suburban hospital setting and had 15 beds. All of the beds were occupied at baseline and at 12-month follow-up in phase 3 of the REAL programme. The staff working on the unit included a psychiatrist, a clinical psychologist and an OT, as well as nursing and support worker staff (QuIRC). Those involved in the ‘sign-up’ meeting (the pre-disposing stage of the GetREAL intervention) were the unit manager, an activity worker and two staff nurses (fidelity sheet unit 1). The initial GetREAL staff training workshop was attended by 18 out of 24 (75%) staff, and the final workshop was attended by 9 out of 24 (36%) staff (fidelity sheet unit 1).
This unit had opened > 5 years ago. It was based in an urban area in the community and had 25 beds. All of the beds were occupied at baseline and at 12-month follow-up in phase 3 of the REAL programme. The staff working on the unit included a psychiatrist, a clinical psychologist and an OT, as well as nurses and support workers (QuIRC). Those involved in the ‘sign-up’ meeting (the pre-disposing stage of the GetREAL intervention) were the unit manager and clinical psychologist (fidelity sheet unit 2). The initial GetREAL staff training workshop was attended by 24 out of 36 (67%) staff, and the final workshop was attended by 12 out of 36 (33%) staff (fidelity sheet unit 2).
This unit was community based. It had 31 beds and was located in a rural area. At baseline, 81% of the beds were occupied and at the 12-month follow-up 94% of the beds were occupied. The unit staff included nurses, support workers and an OT. A psychiatrist and a clinical psychologist provided input to the unit but were not considered part of the unit staff numbers (QuIRC). Those involved in the ‘sign-up’ meeting (the pre-disposing stage of the GetREAL intervention) were the unit manager, an OT, an activity worker, a senior service manager, the psychiatrist and clinical psychologist (fidelity sheet unit 2). The initial GetREAL staff training workshop was attended by 28 out of 36 (78%) staff, and the final workshop was attended by 8 out of 36 (22%) staff (fidelity sheet unit 3).
Thematic findings
The index of initial themes and subthemes used to code the data is shown in Table 37. The iterative process of framework analysis generated the following four main themes that appeared to contribute to long-term change: the staff’s receptiveness to the GetREAL teams, the impact of the GetREAL teams, maintaining initial enthusiasm and the GetREAL legacy.
Themes | Subthemes |
---|---|
Predisposing | People involved from units in sign-up |
Receptiveness to GetREAL teams | Expectations |
Know-how prior to training | |
Positive views | |
General perception of staff about GetREAL | |
GetREAL training | GetREAL training workshop |
Attendance level | |
Staff views on training day | |
Staff views on training facilitators/educators | |
Fresh perspective | |
Staff engagement during training | |
Went well during training | |
Challenges/issues/gaps | |
Improvements for next time | |
Change in practice | Goal setting |
Planning activities | |
Progress in activities | |
Meaningful activity | |
Motivation for change | |
Types of activities | |
Links with community teams | |
Structural changes | Shift patterns |
Changes to structure | |
Service user engagement in activities | Dealing with challenging people |
Benefits | |
Dealing with hierarchy | Permission |
Barriers | |
Managing continuity | |
GetREAL legacy | Maintaining the legacy |
Post GetREAL contact | |
Success/knock-on effect | |
Sustainability | |
Action plan | |
Achieved by GetREAL team |
From the data we identified differences in the general receptiveness of staff to the GetREAL teams. Across the three units, some staff members reported that they had heard that the GetREAL team was coming and some reported that they had not. Some staff indicated that they were unsure about the purpose, whereas those who had attended the pre-disposing meeting had a clearer idea. Staff often reported that their unit manager had not given then much information about the intervention prior to the GetREAL team’s arrival. (Note: the research team asked the unit managers to avoid telling staff about the intervention until after the researchers had collected baseline data in phase 3 in order to keep the researchers masked as to whether or not the unit had been randomly selected to receive the intervention. As there was often very little time between baseline data collection and the GetREAL team arriving, it was not possible for unit managers to inform the whole staff team in advance.)
The findings from three staff focus groups (one in each unit) and the GetREAL team reflective diary notes did not provide a consistent picture of the information staff had about the GetREAL intervention before the team arrived on the units. The focus group participants were able to state quite precisely the level of information they had received before the start of the intervention. This varied between knowing that the team was coming and knowing what they would be doing. For instance:
The OT didn’t know we [GetREAL team] were coming until last Thursday but expects her peers to be interested and wants better briefing. Many of the staff didn’t know much about the REAL programme and phase 3.
RD, unit 1
We weren’t aware of what it [GetREAL] was about . . . it was do it with activities and their [patients’] mental health and stuff, but, actually sort of what people are going to, what the intervention was.
SFG, unit 2
We just had a few days’ notice that they [GetREAL team] were coming, but we didn’t actually know what they were about.
SFG, unit 3
In the staff focus groups, participants were able to share their concerns. For instance, a charge nurse (CN) in unit 1 was concerned about the presence of the GetREAL team on the ward:
I was worried it might be a bit unrealistic because the GetREAL team were extra people on the ward, so of course the engagement and the activities were going to be more.
SFG CN, unit 1
Similarly, OTs in unit 3 appeared confused about whether the GetREAL team would be working just with the nurses on the unit or with them (SFG OT, unit 3). However, this was quickly clarified by the team, who explained that they worked with all staff on the unit.
Unit staff reported both positive and critical views about the GetREAL teams. For example, the unit manager of unit 1 said:
It was a good thing to have intensive involvement for a few weeks from REAL and thought this was valuable and helpfully was keen to embed with staff members who are closely involved.
RD, unit 1
However, in the same unit the OT had different views:
I was very apprehensive about them [GetREAL team] coming and thinking, oh XXXX, someone is going to be coming watching over us, more watching over us rather than giving, ah working with us . . . ooh is she going to be watching me and judging what I’m doing?
SFG OT, unit 1
The OT in unit 2 was of the view that those in the GetREAL team could be useful allies and would assist with their work (RD and SFG, unit 2). However, the GetREAL OT had some concerns about this:
The unit OT seems to take on a lot – when activities do not happen because some staff members make excuses she took this on herself and this seemed unsustainable.
RD, unit 2
The unit 3 staff members were quite positive about the GetREAL intervention:
I was totally positive about the GetREAL because as an OT, I believe in occupation as being therapeutic.
SFG OT1, unit 3
Similarly, a staff nurse said:
I viewed it [GetREAL] positively and thought it was good idea because we’re here for the patients, for them to express themselves, to work out activities and to enthuse us. It brought some good perspectives.
SFG SN1, unit 3
However, another staff nurse in the same unit felt that there had been inadequate planning for the intervention:
It [GetREAL] just started, stopped and then no one mentioned anything until I saw it in the diary.
SFG SN2, unit 3
Furthermore, the OT in unit 3 felt that the GetREAL team focused on working with the nurses and suggested that they should have done more work with the therapists (SFG OT3, unit 3).
Staff and service users’ views varied as to the extent and type of impact the GetREAL teams had. From the data we identified many examples of how it had had a positive impact on the unit team, including improving the collaboration between unit staff, increasing their confidence in certain skills and motivating them. There was also recognition that it had led to an increase in the range of activities offered to service users and encouraged service users’ involvement in planning activities.
GetREAL has promoted staff and service users’ involvement in activities, which is valuable.
SFG CN, unit 1
After GetREAL the unit staff have better understanding of complexities of the unit.
SFG, unit 2
It [GetREAL] had an enormous benefit in that it was joint working and collaborative working and that it was going to bring everyone together, as one team, working in one direction, and offering the service users here a greater range of meaningful activities, not necessarily just groups but meaningful activity, in the widest sense.
SFG OTi, unit 3
In unit 1, although a great deal of work had already gone into promoting service users’ access to the community before the GetREAL team arrived, the service users were not involved in planning their days and nor were they regularly consulted about activities. The GetREAL team helped the unit’s OT to change the unit’s community meetings to a more participative style, and to plan activities on the unit with service users (RD and SFG, unit 1). This change was well received by service users, which pleased staff.
Across all three units, the service user experts who cofacilitated the staff training days in the first and last weeks of the GetREAL enabling stage seemed to inspire staff to give more thought to how they could develop their approach to service user involvement. The staff focus group participants felt that, during the enabling stage, the unit team were energised and motivated, more confident and more engaged in thinking about how best to support service users (SFG, units 1, 2 and 3). The GetREAL teams had also had a directly positive effect on service users, empowering them to suggest activities that they wanted to do (SFG, unit 3).
The data also suggest that the GetREAL teams gave the unit OTs support and helped them develop their skills in actively engaging other members of the MDT in planning and leading activities with service users. This was achieved through modelling collaborative action planning, sharing occupational therapy concepts and skills with all staff, and encouraging them to try out the activities suggested by service users. There seemed to be a general shift in attitude, whereby, at the start of the enabling stage, many unit staff had viewed activities as the job of the OT, but the stage it progressed more staff seemed to acknowledge the importance of activities for service users’ recovery, and to see this as something that the whole team should enable (RD and SFG, unit 1).
In unit 3, the OT was very enthusiastic about the GetREAL intervention, especially about its potential to improve multidisciplinary collaboration and disseminate the message that engaging service users in meaningful activity was a role for all staff on the unit. However, the therapist felt that the intervention was a little too focused on the nurses, and that more time should have been given to working alongside OTs. However, at the staff focus group 6 months later, some nursing staff complained that they felt they were expected to do the OT’s job as well as their own (RD and SFG, unit 3).
The ‘maintaining initial enthusiasm’ theme refers to the longer-term sustainability of the changes introduced during the enabling stage of the GetREAL intervention. The data from the three units revealed that once the enabling stage had started, staff members and service users were enthusiastic about the intervention. For example, in their focus group, the unit 3 staff reflected on how the team had motivated them to start different activities with service users (e.g. dancing or attending the local gym). They found the training interesting, and felt listened to and supported. Staff indicated that the changes they made during the intervention were successfully sustained and further developed over the following few months. However, we did not have specific data available to assess if this momentum was maintained over the longer term.
Staff across the three units seemed to appreciate outsiders (the GetREAL teams) stimulating them to look at things afresh. They also appeared to gain satisfaction from seeing service users responding positively to the changes that were facilitated by the GetREAL teams, and reported feeling more confident in engaging service users in activities (SFG, units 1, 2 and 3). However, the service user interviews revealed how some activities introduced during the enabling stage (photography, visiting the aquatic centre) did not continue owing to a lack of equipment (camera) or insufficient numbers of available staff to accompany them (SUI, unit 3).
In unit 1, some of the activities initiated in the enabling stage, involving all staff members, were sustained, such as working on a local allotment. This appeared to have been helped by positive feedback to staff from service users about how much they enjoyed it. In unit 1 there was also evidence that staff members, mainly nurses and the OT, were continuing to refer to the action plan and review the actions they had agreed to continue after the GetREAL team left (SFG unit 1). The staff focus group in unit 3 also suggested a lasting benefit from the enabling stage, in that service users were reported to be accessing the community more for activities, to the point where:
Sometimes there is nobody left in the building to take part in the activities in the unit.
SFG, unit 3
However, the unit 2 staff members were less positive, admitting:
Nobody actually took over where the GetREAL team left off because each and every one of us staff members has got enough on their job roles.
SFG, unit 2
Theory-led findings
The seven priority candidate theories identified in the rapid realist review were examined further to assess how far the published literature identified in the review and the data collated for the case studies were able to support or corroborate them. Three theories were categorised as belonging to the mechanism RS and four to the mechanism SC.
The literature suggests that staff groups will be receptive to making long-term changes and increasing their engagement with recovery-based practice if they feel involved, valued, enthusiastic and engaged in the training/change programme. Contextual factors that might promote, or inhibit, this sense of enthusiasm, engagement, involvement and feeling valued include characteristics of the training/change programme, organisational structures and systems, and the local culture and climate (see Table 34).
The case study data and evidence from the literature review for each of the three priority candidate theories within the RS mechanism are presented below. A summary of the main characteristics of the papers reporting on primary studies referred to below is provided in Table 33.
The literature suggested that staff will feel engaged, valued and involved (and hence receptive to change) if an action plan for change has been developed collaboratively between different staff groups and service users. If the action plan utilises the existing strengths and experiences within the team and service users, this will also contribute to the sense of feeling valued. Conversely, if an action plan is imposed on staff members, they will be unlikely to feel engaged, valued or involved.
Action plans for each unit were available in the data sets for all three case studies. An initial, working action plan was drafted by the GetREAL team OT after the initial scoping week of the enabling stage of the GetREAL intervention, and then in the final week a longer-term action plan was agreed with the staff team. The case study data show that staff and service users were engaged in developing and reviewing the action plans during the GetREAL intervention. The GetREAL team had individual and group meetings with staff members and service users in all units to gain their input and feedback on the action plans.
In team meetings staff were encouraged to think and give ideas how patients can be more involved in activities and those can be included in the action plan.
RD, 15 March 2012
The GetREAL team had a meeting with the Consultant Psychiatrist (CP) in which CP gave context and an overview of the current direction and drivers for the unit Action Plan particularly around collaborative goals and he was very pleased with the barriers that had been identified.
RD, 20 February 2012
The staff appreciated the Action Plan review and its focus on success of the team and individuals. They also particularly responded to the way it is communicated what had been done to everyone.
TEN, p. 1
The psychologist gave feedback on Action Plan. She felt the focus of the Action Plan was appropriate and could see where it fits in to the longer term vision for development of the team. She also gave useful information and added some ideas to consider to the plan.
RD, 23 January 2012
The Action Plan was discussed with unit staff members and they were positive about it. Most of staff had seen the Action Plan. Got buy-in from Manager and Deputy Manager followed by the sign-up of other staff members.
RD, 4 July 2011
The Action Plan was provided to staff as they come on shift and talk this through. For those who did not attend the training session, also explain the salient points and how to lead the content of the Action Plan.
RD, 8 July 2011
Service users (SUs) were responding positively to use and sharing information that is new to the team-have commented on the Action Plan and were seeing the value of different approaches in Action Plan. The GetREAL team had identified the key individuals to be involved in specific parts of the plan.
RD, 21 June 2011
It seems that priority theory RS1 is corroborated by the evidence, which clearly shows that action plans in the three case study units were developed collaboratively with staff and service users. They reported finding it useful and considered it helpful in providing direction for their practice (SFG, AP, FS units 1, 2 and 3). In terms of longer-term change in practice, the staff focus group participants of unit 1 confirmed that they were continuing to refer to their action plan and update it 6 months after the GetREAL team left (SFG, unit 1). This suggests that they had incorporated an ongoing, iterative process to ensure that the document remained ‘live’. However, in units 2 and 3, staff focus group participants reported that they were no longer using their action plans (SFG, units 2 and 3).
The Implementing Recovery through Organisational Change programme briefing paper136 describes a team recovery implementation plan (TRIP) instrument, which is a plan for teams to implement a recovery approach that includes coproduction and ongoing review. The TRIP has been used in a number of mental health organisations. 136 The authors stress the importance of coproduction: for staff, at all levels, and service users to be involved on an equal footing in discussions about the current situation and achieving consensus on ways forward. This requires staff members to be honest about real external or organisational constraints that they experience, something that the authors acknowledge may be difficult.
Action plans also feature in our priority candidate theories RD4 and RI8 (see Appendix 18 for a statement of these theories). In theory RD4, we propose that the development of a clearly articulated action plan, which is regularly referred to and updated over a long period of time, provides staff with a clear sense of what is expected of them: they have a sense of ‘reinforced direction’ – a separate mechanism, potentially triggered by the same contextual factors. In theory RI8, we propose that the absence of an action plan following up a training programme leads staff members to conclude that increasing their recovery-based practice is not a priority for them, thus blocking the mechanism RI and leading to little lasting change in practice.
In an organisation that has undergone much recent change, staff members may feel, at best, unenthusiastic about an additional training/change programme to increase recovery-based practice or, at worst, unenthusiastic, disillusioned or downright pessimistic. We propose that incorporating recovery into an existing change programme – ideally, one that is viewed positively by staff members – may help with staff engagement and enthusiasm.
All three case study units had been involved in at least one change programme before the GetREAL intervention. Units 1 and 3 had engaged with the Productive Ward initiative, and unit 2 had undergone a programme to enable the implementation of the recovery approach. In addition, all three units had programmes of staff training in recovery-based practice.
The Productive Ward programme was integrated into the GetREAL intervention in this unit. In their focus group staff reported that they had had some success in engaging service users in activities during the GetREAL programme because the organisation and culture of their unit had already been improved by the Productive Ward programme (SFG, unit 1). In particular, the Productive Ward programme had improved communication between staff, including communication about and planning of activities for individual patients. The focus group participants emphasised that the Productive Ward process prepared the unit for the GetREAL team and that both fitted together well. This suggests that it was possible to adapt the Productive Ward process to integrate the GetREAL intervention and that this helped to embed both sets of changes in the long term. Both programmes improved communication and freed up staff time for the supporting of service user activities (SFG, unit 1). Before the GetREAL intervention, 19% of the unit staff members had received formal recovery-based practice training. This had increased to 85% at the time of the 12-month follow-up data collection in the REAL phase 3 trial (QuIRC).
This unit had adopted a recovery approach before the GetREAL team arrived (SFG, unit 2). Details of exactly what this involved were not available from our data sources, but the GetREAL OT observed that the existing recovery approach appeared to have no specific impact on practice (RD, unit 2). Before the GetREAL team arrived, the formal recovery-based practice training had been completed by 27% of the unit staff, a figure that increased to 97% at the time of the 12-month follow-up (QuIRC).
The staff focus group participants reported that they had engaged in the Productive Ward initiative before the GetREAL team arrived, which had included a focus on improving the activities available on the unit. They felt that the GetREAL intervention built on this (SFG, unit 3). However, in this unit only 5% of staff members had received formal recovery-based practice training before the GetREAL team arrived, and this had increased only marginally, to 7% at the time of the 12-month follow-up (QuIRC).
Our data provide some support for this priority candidate theory. In unit 1 the GetREAL intervention appears to have been fairly easily integrated into the already-established Productive Ward programme. In particular, staff were able to identify that communication about activities had been addressed prior to the GetREAL team’s arrival (SFG, unit 1). This unit maintained their enthusiasm for both change programmes, and scored higher on the primary outcome assessed at the 12-month follow-up in the REAL phase 3 trial (service user activities) than the other two units. Unit 3 staff also believed that the GetREAL intervention was timely, as it built on the change they had begun through their Productive Ward programme (SFG, unit 3). There was no indication, however, that both change programmes were integrated, and unit 3 staff reported that they could not sustain enthusiasm for the GetREAL changes long term. This does, therefore, support the theory that a lack of integration with existing change programmes can limit enthusiasm for longer-term change.
Similarly, we did not have explicit support for this theory from the published literature on mental health rehabilitation services, although the difficulty of prioritising and operationalising recovery in addition to other, potentially competing, organisational change processes has been acknowledged. 117 ‘Change fatigue’ is also a recognised phenomenon in the NHS;141 in the discussion following the symposium held in 2014 that disseminated the results from the whole REAL programme to participating staff, one psychologist commented that there was ‘too much change, too many initiatives already going on’. Our results suggest that if change programmes are synergised and integrated then this can amplify benefits and minimise ‘change fatigue.’
In an organisation that has a climate of uncertainty and fear (e.g. in a context of economic cutbacks and job losses), it is likely that staff members will feel disillusioned and uninterested in, or pessimistic about, a change programme. We propose that this negative organisational context blocks the mechanism RS: staff will be unlikely to feel involved, engaged or valued.
No relevant evidence in unit 1 data.
Unit 2 was under threat of reconfiguration or closure during the GetREAL intervention. Commissioners were considering different options for providing more cost-effective services, either in the form of a smaller rehabilitation unit and a separate personality disorder unit, or through closing the unit and investing in a community rehabilitation team (RD, 1 February 2012). In addition to this uncertainty, two staff members were off on long-term sick leave and this had affected the overall ward environment and staff involvement in activities. For instance, ‘long-term sickness . . . made some staff feel that they have to burden themselves with organising activities on top of their regular duties’ (RD, unit 2). It was also noted in the GetREAL team OT’s reflective diary that ‘the long-term sick leave . . . had a negative impact on how well everything is organised and whether everything runs according to plan’ (RD, unit 2).
No relevant evidence in unit 3 data.
We did not find sufficient evidence from our case studies to support or challenge the theory. Similarly, little supporting evidence was found in the published literature. However, during phase 3 of the REAL programme, as stated earlier in this report, during the 12 months after the enabling stage of the GetREAL intervention, one of the units that had received the intervention closed down and others experienced a climate of uncertainty in a period of turbulence in the NHS due to the economic recession. It may be overly ambitious to try to engage members of staff with a training/change programme at all in such difficult circumstances. For this reason, in the USA, a pilot site that was ‘not in crisis’ was purposively chosen to test a complex intervention in mental health rehabilitation facilities, the Illness Management and Recovery programme. 119 In order to select such a site, it would be necessary to have the tools to first assess the situation in that site (i.e. the degree of uncertainty or threat to employment experienced by staff).
An uncertain or fearful cultural climate will also increase levels of staff stress and burnout. These factors feature in our proposed theories RS2 and RR8 (see Appendix 18 for a statement of these theories). In theory RS2, we propose that staff members with high levels of job satisfaction and low burnout are likely to feel engaged, motivated and ‘receptive’ to change. Conversely, increased levels of stress and burnout will make those staff members less likely to be engaged or motivated by any training/change programme. In theory RR8, we propose that staff members affected by stress and burnout will be more likely to perceive a threat more readily and/or make negative attributions towards service users – thus blocking the proposed mechanism to longer-term change, that ‘recovery is realistic’.
The literature suggests that staff groups will make long-term changes in practice if they feel supported, both by management and colleagues, to do so; this is one of our postulated mechanisms of long-term change. Contextual factors that might promote, or inhibit, this sense of support, encouragement and motivation include characteristics of the training/change programme, organisational structures and systems, and the local culture and climate (see Table 33).
The corroborative evidence that emerged from our literature review and case studies for each of the four priority candidate theories included under the SC mechanism is presented below. A summary of the main characteristics of the primary study papers referred to below is provided in Table 32.
The literature suggests that regular supervision or collaborative meetings between staff groups and the training team, and/or staff members and a local change lead (‘champion’), that take place within a supportive organisational culture, will help staff members feel supported by their peers and managers in the change programme.
Data show that all clinical staff members had a named supervisor. At the time of baseline data collection for phase 3 of the REAL programme, it was reported that staff met with their supervisor individually at least weekly (QuIRC). However, at the 12-month follow-up point, the frequency of supervision was reported as every 2–6 weeks, and this was being given as group rather than individual supervision (QuIRC).
All clinical staff had a named supervisor and they were reported to have one-to-one supervision meetings every 2–6 weeks at the time of baseline data collection. This frequency was maintained at the 12-month follow-up point. In addition, group supervision was started during the 12 months at a frequency of every 2–3 months (QuIRC).
All clinical staff had a named supervisor in this unit as well and they were reported as meeting individually every 2–6 weeks both at baseline and at the 12-month follow-up point. In this unit, group supervision was also used at least weekly at baseline and at 12-month follow-up (QuIRC).
The available evidence suggests that, in all three units, individual and group supervision was provided routinely. This suggests a supportive organisational culture. However, our available data sources did not include staff reports on how useful they found supervision or its content.
The importance of line management supervision is discussed in relation to theory SC4 (see Priority theory SC4: management support, supported role flexibility) and theory SC5 (see Priority theory SC5: organisation structures, etc., modified to support change). The current theory, SC2, relates to clinical supervision or meetings outside the remit of line management supervision; these meetings may or may not have involved the original GetREAL training team. Based on the presented information, it appears that only some aspects of the theory are supported.
Built in to the GetREAL intervention was provision for the nominated link person (in effect a unit ‘champion’) in the intervention units to make e-mail contact with the GetREAL teams in the 12-month period after the teams had left the unit, rather than receiving face-to-face supervision. However, those volunteering to be a champion were new the job, and they had no formal training or support for acting in this role. More regular and formal supervision from the research team (three times over a 14-month period) was built in to a programme in northern Sweden, aimed at enriching psychiatric day centres for attendees (including increasing their engagement with meaningful activities). Quantitative results of this randomised controlled trial142 found no difference between participants in the intervention and control groups regarding the value linked with the day centre activities, in the satisfaction they derived from everyday activities in general, or in their or health and well-being. However, the staff focus groups revealed high levels of satisfaction with the research team supervisions: ‘Staff’s reflections and feelings of guidance and confirmation were essential for the development and implementation of the enrichment intervention’ (Mona Eklund, Department of Health Sciences, Lund University, Sweden, 29 September 2014 and 31 January 2015, personal communication).
The desirability of multidisciplinary staff supervision to share concerns and problem-solve is recognised by the Joint Commissioning Panel for Mental Health guidance for commissioners of mental health rehabilitation services. 67 Notwithstanding the need for line managers to ‘buy in’ to recovery-based practice, providing supervision outside the normal line management structure may be beneficial if it encourages staff from different groups to feel that they can work and reflect together, in a supportive environment. Weekly supervisions with university consultants were built into the Illness Management and Recovery programme, mentioned earlier, that was implemented in the USA. 119 These supervisions supported ‘the development of trusting relationships and an environment in which participants freely engaged in role plays, clinical exercises, and videotaping of sessions’. Multidisciplinary staff groups that persist beyond the end of a training programme can help to create an ongoing shared vision126 and can be used as a mechanism for a continuous update on good practice and for synthesising approaches to mental health rehabilitation between different professional groups. 128 Timing such group meetings appropriately, and having open or rotating membership, can enable maximum participation from different staff groups working on different shifts; however, it may then take additional time to develop a true sense of group ‘cohesion’ and purpose. 127 Additionally, some staff members may feel uncomfortable sharing their feelings or concerns in a group context if they see work-related stress as normal, or if they see themselves as ‘carers’, not needing care themselves. 139
This theory refers explicitly to a supportive organisational culture. Self-evidently, supervisions and staff meetings will be most effective if the unit staff work well as an interdisciplinary team. Within the context of rehabilitation/community care, the 10 competencies of an effective interdisciplinary team proposed by Nancarrow et al. 125 include shared values, a culture of trust and consensus, intrateam communication and collaboration, and collaborative decision-making. These team competencies would seem to be equally applicable to, and desirable in, interdisciplinary teams working in mental health rehabilitation.
Providing opportunities for staff to reflect together also helps to foster a sense that recovery belongs to everyone: ‘recovery is everyone’s responsibility’. Therefore, this context also feeds into this mechanism, in our theory RER3 (see Appendix 18 for a statement of this theory).
The literature suggests that appointing individuals from within the staff team to ‘champion’ or act as a ‘change agent’ for the programme serves to persuade, encourage and empower (i.e. support) other staff members to change. To be effective, those individuals should have optimism, good interpersonal skills, management support and the respect of colleagues, and be influential. We also propose that ensuring that the ‘champion’ is associated with a role, rather than a single individual or individuals, ensures long-term continuity: dependence on key enthusiastic individuals runs the risk of staff members losing that support if those individuals leave or move to other departments.
The GetREAL intervention manual (see Appendix 6) states that ‘The GetREAL team will encourage posts to involve promoting and maintaining activity, rather than post-holders’. However, when GetREAL was implemented, individuals tended to act as champions, rather than there being a specific post, as evidenced below.
It was mentioned in the GetREAL team OT’s diary that a nursing assistant had been identified as a champion in this unit (RD, unit 1).
The GetREAL team OT’s diary reported that an OT and a psychologist were enthusiastic about the intervention, and hence both were considered ‘champions’ but not appointed formally (RD, unit 2).
Although no one was identified as a ‘champion’ in this unit, the concept was evidenced in the unit staff members’ discussions. For instance, the comment was made that the service user consultant was proactive and genuinely interested in users’ and staff engagement in activities, and was valued and considered a ‘champion’ by the unit staff (SFG, unit 3).
Overall, we did not find sufficient evidence to test the theory. However, based on available information we could say that the idea of ‘champion’ existed in all three units.
The role of change lead or ‘champion’ is discussed in relation to theory SC3, change agent/champion role (see Priority theory SC3: change agent/champion role). It has been suggested that the creation of champions is a possible strategy for lessening any sense of coercion experienced by staff who are expected to change their behaviour. 119 In an effort to operationalise the recovery model through implementing an Illness Management and Recovery programme in a unit of a psychiatric hospital in the USA,119 keen volunteer staff members acted as champions to lead on the programme design and delivery. The findings of a Cochrane review130 of randomised controlled trials that aimed to evaluate the effectiveness of local opinion leaders in improving the behaviour of health-care professionals and patient outcomes were inconclusive; opinion leaders appeared comparable with other mechanisms for enhancing engagement of health-care professionals with evidence-based practice (such as distributing educational materials, audit and feedback, educational outreach), the effectiveness of opinion leaders varying within and between studies. This finding is entirely consistent with what might be expected in a complex setting.
What qualities should local champions for change possess? Corrigan132 describes champions as ‘yeoman clinicians who exhibit sufficient excitement and knowledge to shepherd rehabilitation innovations through implementation and maintenance phases of program development’. They have ‘communication skills that help them express complicated ideas simply . . . [and] . . . good interpersonal skills that serve them well in building consensus among peers’. 132 A survey of 47 nursing, professional and administrative staff of extended care units in a hospital in Illinois, USA,124 found that peer-nominated champions possessed more programmatic optimism than their peers. These individuals reported significantly fewer barriers – specifically, institutional constraints and philosophical opposition – to implementing behavioural interventions than those who were not nominated.
Corrigan132 advocates psychologists as an appropriate staff discipline to take on the role of champion. An OT would be another possible candidate, having a professional vested interest in the programme. Programmatic optimism and communication skills are not enough, however; a champion is powerless if not supported by management, or if not well embedded in the MDT. As McCracken and Corrigan138 observe, ‘Institutional constraints may result in a catch-22 situation in which evidence-based practice therapists have little influence over the institutional constraints because these factors place them outside the power structure needed to influence the institutional factors’ (p. 236).
The literature suggests that it is important for staff groups to feel supported by their management to change. This support can be realised through active management endorsement and prioritisation of the programme, and encouraging change (e.g. management getting involved themselves, endorsing an action plan for change, quantifying progress, incorporating external drivers to change). This support will help the staff feel encouraged to change, even if increased engagement with recovery entails moving outside their traditional occupational role. To this we may add that management support of positive risk-taking is also desirable.
Data confirmed that a pre-disposing meeting (the GetREAL sign-up meeting) was held with each unit’s senior staff. At this meeting, dates of the staff training workshops and plans for release of staff (arranging ‘backfill’) to attend were agreed (FS, units 1, 2 and 3). Case study data revealed that in all three units the senior staff members were also involved in the development of the units’ action plans and that they endorsed them.
The senior management and nurse manager in units 1 and 2 were clearly supportive of the GetREAL intervention from the start.
The senior management were truly multidisciplinary team in their approaches and despite the pervasive medical model the medical staff members were very involved in GetREAL.
RD, unit 1
The charge nurse gave feedback and believed that it was really valuable that the GetREAL team promoted more involvement as she believed the ward doesn’t offer client centred care.
RD, unit 1
The leadership team have decided to include key actions from the REAL study in one inclusive document bringing together a range of strategic pieces of work.
RD, unit 1
Reception in the leadership meeting was positive and supportive. The acting manager showed her support for the team increasingly being involved in role sharing around activities.
RD, unit 2
Acting manager took a lot from the training both in terms of what we were discussing and from observing the responses provided by the team. Sometimes the acting manager was very proactive in engaging service users in activity planning discussions.
RD, unit 2
Most senior staff referred positively to the intervention and said it was useful to get everyone to think about activities or get some support regarding this.
RD, unit 2
In this unit, although the management team and OT attended the pre-disposing meeting, the nurses later said that they had not known what to expect and what was expected of them at the start of the intervention (SFG, unit 3). Further, unlike in units 1 and 2, the unit manager did not require that all staff members attend the training workshops but decided instead to see how many of them would ‘buy in’ to the intervention by inviting those interested to attend (RD, unit 3). This may have implied that the manager did not feel that the programme was important for all staff. A higher proportion of staff (three-quarters) attended the initial training session than in the other two units in our case studies, but only one-fifth attended the final training session (a considerably lower figure than in the other two units).
It appears that in units 1 and 2 the management team actively endorsed GetREAL as a change programme. However, in unit 3 there was less clear support for the programme from senior staff. Unit 3 had the smallest change in primary outcome scores (service user activities) at the 12-month follow-up, which suggests perhaps that senior managerial support is a key component for successful implementation of a programme that aims to change or improve practice. However, this is a tentative interpretation, as many inter-relating factors will have had an impact on outcomes.
In a small, older, exploratory study121 of the opinions of staff of inpatient rehabilitation wards, the factor that most influenced staff use of behaviour therapy was attendance at the training course (100%), followed by nursing officer and peer support (91%). Marlowe et al. ’s116 case report compared the experiences of two mental health inpatient units in separate hospitals in Florida, USA, that underwent a staff training programme to implement psychosocial rehabilitation. Although fidelity to the programme was poor and there was little empirical evidence to enable a formal evaluation, the unit with supportive management fared better. In both units, the paraprofessional staff were initially enthusiastic. In unit A, ‘the unit management staff met with them after their initial training and requested feedback. Suggestions were acted upon, such as the shortening of the residency training program’. In unit B, ‘the resistance of professional staff eventually led to resentment and discouragement since paraprofessionals could not perform any of the new skills they were learning without professional support’. 116 The same applied to higher management (hospital administrator) support. In both units the hospital administrators gave support to the programme verbally, but only in unit A was this followed through: ‘The hospital administrator for Unit A was accessible and highly involved in the entire process . . . by contrast, [academic partner] staff infrequently met with the hospital administrator of Unit B‘. 116 Drawing conclusions from largely anecdotal information, Bartholomew and Kensler119 stated, regarding an illness management and recovery programme developed and implemented in a psychiatric hospital in New Jersey, USA, that ‘Ultimately, it was the support of the Chief Executive Officer and ongoing discussions with the leaders who had concerns that allowed the project to eventually achieve broad support’.
It is desirable for facilitators of managerial support (support from local managers as well as executive management) to be built into any training/change programme. In the Core Curriculum training programme,118 hospital executive staff with the ability to implement hospital-wide changes attended the 3-day (mandatory) training programme along with all staff having direct contact with service users. The authors observed that ‘executive staff members were present at the ward training to stress the importance of the training and to resolve problems, and psychiatrists were in attendance and actively participated’. 118
Of course, management buy-in to a change programme, in word and in deed, cannot be guaranteed, even if specific provision is made for encouraging management support. A behaviour management in-service training programme for staff in an extended care unit in Lincoln, NB, USA,131 included additional modules for supervisory staff, which included the assignment of in vivo exercises for the supervisors to carry out. In practice, the take-up of these in vivo exercises was poor among nursing supervisors, compared with social work, occupational therapy and recreational therapy supervisors. The author observed that ‘The main message of the Supervisors’ Modules, that ongoing training and supervision of direct care staff is a necessary part of behaviour management, was apparently lost on many of the supervisors’. 131 In the same study, the registered nursing staff were found to view the ward in a more optimistic manner, as measured by the Ward Atmosphere Scale, than other professional staff and technician staff. Therefore, it was suggested that the supervisory nurses would have little motivation to change the social milieu if they already saw it in a positive light. 131 Perhaps, therefore, nursing supervisors have the most to learn from a recovery-oriented training programme? This was borne out by a case study presenting a (short-term) evaluation of a behavioural methods training programme for staff in a psychiatric facility in Virginia, USA: before-and-after data from the Knowledge of Behavioural Methods Inventory found that nursing staff benefited most from the training, compared with psychiatric aides and mental health workers. 133
Management endorsement and prioritisation also feature in our proposed theory RI1 (see Appendix 18 for a statement of this theory). In this theory, endorsement and prioritisation by management helps the staff feel that ‘recovery is important’ to the organisation: a separate mechanism, potentially triggered by the same contextual factors.
In addition to management support for role flexibility to increase engagement with recovery-based practice, staff members may need to feel supported by management to make positive risks. The report on rethinking risk,143 cited by Boardman and Roberts,144 states that ‘constructive and creative risk-taking is a vital part of a patient’s rehabilitation and risk averse practice is detrimental to this process’. A focus group and interview-based exploration of staff identified the barriers and facilitators associated with providing recovery-oriented support in five mental health trusts in England:117 ‘The relationship of recovery to the statutory clinical obligation of risk management was seen as a competing priority. Staff felt they would encourage recovery support through positive risk-taking if they were better supported by the organisation’. Positive risk-taking also features in our proposed theory RR9 (see Appendix 18 for a statement of this theory). In this theory we propose that if there is an organisational culture that encourages autonomy and supports positive risk-taking in the pursuit of recovery, staff will feel empowered to increase their engagement with recovery-based practice: they will feel that ‘recovery is realistic’ – again, a separate mechanism, potentially triggered by broadly the same contextual factors.
The literature suggests that if the management team modify organisational structures, processes and systems (e.g. working practices, responsibilities, policies, documentation and performance reviews) to facilitate the move towards recovery-based practice, staff members will feel supported by management in changing their practices.
Shepherd et al. 145 state that guidance for individual practitioners needs to be ‘integrated into day-to-day practice through the design of appropriate job roles and job descriptions and effective supervision’ (p. 9).
Our case studies provided some evidence that the structures, processes and systems in unit 1 fostered receptiveness to the GetREAL intervention.
The unit has a well-functioning multidisciplinary team that plans together at fortnightly CTMs.
RD, unit 1
The consultant explained that they wanted to merge with other initiatives.
RD, unit 1
The unit staff members have already begun making changes to some of their systems and show a willingness and motivation to make things work even better for them and the patients.
RD, unit 1
In unit 2, the GetREAL team’s OT felt that structural and management issues, such as inadequate staff supervision and line management, hindered good practice and performance management (RD, unit 2).
Unit 3 was the largest of the three units in our case studies, and the GetREAL OT noted that the structure was very hierarchical. For instance, it was reported that nursing assistants had to get permission from seniors for ‘everything’ and did not feel it was their job to facilitate activities (RD, unit 3).
An example of an organisational system that impacts on staff behaviour is the requirement to complete various records of activity for audit purposes. In unit 3, such a system was in operation and included staff routinely recording the number of hours of activity service users engaged in. However, data from the staff focus group suggest that staff did not see this system as meaningful. For example, the hours recorded did not include the time service users spent by themselves doing their chosen activities, only those where they engaged in an activity with staff individually or in a group. Activities that are meaningful to service users include those relating to daily living (self-care, cleaning, shopping, cooking), leisure activities and vocational activities such as study or work, all of which may take place on the unit or out in the community. As illustrated in the quotation below, when staff felt that there was little value in recording service users’ activities, this did not encourage change in practice.
I think there were some questions perhaps about what meaningful activity was for people, what counted and certainly looking at the outcome of the first report and matching that against national standards and seeing where we came there, I thought ‘oh that really doesn’t feel like what goes on at all’ and therefore if you’re going to put more in and measure it, are you actually going to get anything accurate. But that’s also part of the fact that the journey like you were saying, has been going on, particularly intensely in the last 2 years, in terms of the whole service review for here as an active sort of rehab[ilitation] recovery unit, and that has been about changing the culture and how we work really, and incorporating meaningful activity into everyday things. I thought well, there’s this snapshot, but its perhaps only getting a part of that, because we knew there was so much more work to do um, and so, in terms of sort of some people saying they only had a very little bit of, um, number of contact minutes or hours, as that came out in some of the aspects of that survey, really it depends what, the staff were used to discussing and counting as meaningful activity with those individuals.
SFG AN, unit 3
To conclude, we did not find sufficient data to robustly test this theory, although there was some evidence that unit 1 appeared to have more facilitative structures to support the intended changes. It is, of course, no trivial undertaking to modify organisational structures, processes and systems to facilitate long-term change towards increased recovery-based practice. Doing so takes time, and it might be necessary to ‘fix’ the organisation in order for the training/change programme to be effective. Birkmann et al. 82 describe a collaborative project between a university and a hospital in New Jersey, USA, aimed at transforming the hospital’s philosophy of care to a greater focus on recovery. In addition to staff training programmes, the project involved improving the structure and process of daily ‘life management’ or ‘community’ meetings between staff and service users on the ward. The need to improve these meetings was identified through an initial needs assessment performed by the academic faculty. Patient rounds, treatment team meetings and treatment plan documents were also targeted.
In order to be able to implement change (any change) in an organisation, there needs to be clarity about who has the authority to make/approve the changes. This was posed as an important contextual factor explaining the differences between units A and B in a case report116 comparing the experiences of two units from separate hospitals in Florida, USA, which underwent a staff training programme to implement psychosocial rehabilitation. For unit A, ‘there were clear lines of authority and decision making, which resulted, for example, in the expeditious implementation of the new agitation/seclusion policy’. In contrast, in unit B, ‘the consultants were never able to determine who had the authority to approve changes on the unit’. This led to some procedures being duplicated because ‘approval was never obtained to drop the duplicated aspects’. 116 Staff line management supervision (through appraisals, performance reviews, etc.) also needs to incorporate the reinforcement of desirable (recovery-focused) behaviour. Citing Fuoco and Christian,146 Vangen131 states that ‘maintenance and generalization of training effectiveness can be promoted by contingent feedback for staff performance, going from continuous to intermittent performance feedback, cuing desired performance, and providing ongoing supervision and training throughout an individual’s employment’. As is discussed, however, in relation to theory SC4 (management support, role flexibility), buy-in from supervisory staff, in particular nursing supervisors who are not well versed in recovery principles or do not perceive them as important, is not guaranteed (see Priority theory SC4). The importance of broader supervision and support is discussed in relation to theory SC2 above (see Priority theory SC2).
A key organisational policy that can/should be modified to incorporate recovery-based practice is the risk management policy. Boardman and Roberts144 propose a revised approach to risk assessment and management based on ‘person-centred safety planning’ and present an outline framework for operationalising this new approach. (Management support for positive risk-taking is discussed in relation to theory SC2; see Priority theory SC2.)
Building the desired change into (modified) organisational processes also features in our proposed theory RD1 (see Appendix 18 for a statement of this theory). In this theory we propose that reflecting new recovery-based activities that the staff are expected to do, in organisational structures, processes and systems, gives the staff a sense of ‘reinforced direction’ (i.e. they know what is expected of them). If there was conflict between the training/change programme goals and these organisational processes, the staff members would probably feel confused and revert to their usual practices. This is, therefore, a separate mechanism, but it is triggered by the same broad contextual factors.
Other (non-priority) theories with supporting evidence from the data
The LRG prioritised the previous theories during the rapid realist review. Some of the other theories were found to resonate with the case study data, and these are briefly reported below.
When staff groups of a mental health inpatient rehabilitation unit have taken part in a training programme aimed at increasing their engagement with recovery-based practice, they will make long-term changes if they, and the service users, feel that recovery is everyone’s responsibility – all staff, all service users.
Providing opportunities for different staff groups to reflect together, obtain feedback, monitor their progress and identify areas for further change helps staff to feel that recovery is a shared responsibility.
There was some support for this theory arising from the qualitative data from all three case study units. The more successful unit (unit 1) appeared to have a more open and supportive staff culture that facilitated the aims of the GetREAL intervention, to enhance shared responsibility for engaging service users in activity. For instance, the GetREAL OT observed that:
The Unit 1 multidisciplinary team is democratic by the consultative style and humility of the consultant and other professionals . . . but that the nursing staff may remain quite hierarchical and didactic in their planning of work.
RD, unit 1
The GetREAL intervention was seen to encourage team reflection and decision-making, as it was reported that:
[T]he turning point has been the team meeting, where the Action Plan was reviewed – this prompted staff to think about how the patients can be more involved in their treatment by planning their own week.
RD, unit 1
Furthermore, the GetREAL team observed that the unit 1 OT was skilful at encouraging nursing staff to support and share responsibility for activities.
The data from unit 2 suggested that at the start of the enabling stage there were barriers to staff taking on shared responsibility, but over time and through shared discussions the GetREAL team facilitated staff in improving their communication and collaboration:
The team planned away days for discussion and collaborative planning . . . In which the OT will input information about MOHOST [an occupational therapy assessment tool] to one of these away days and other topics are included: how do we talk to each other? How do we keep each other informed?
RD, unit 2
Furthermore, the GetREAL team’s OT noted in their reflective diary that the unit 2 staff members:
[D]iscussed communication ideas for activity planning and agreed to put activities in to the diary and add a column in the allocation sheet to support staffing and prioritising of activities. Also discussed visual prompt and manager has ordered an extra white board for the office following the discussion.
RD, unit 2
The data from unit 3’s staff focus group suggest that during their group supervision meetings the staff usually reflected on the work they had done and how to improve it. They mentioned that they found it useful to reflect on their work as a team. In addition, staff members found the noticeboard very useful as an indicator of how well they were doing in focusing on service user activities, and this also aided handover between shifts because:
[I]t was listed on the noticeboard what all service users are doing on daily basis.
SFG, unit 3
When staff of a mental health inpatient rehabilitation unit have taken part in a training programme aimed at increasing their engagement with recovery-based practice, they will make long-term changes and increase their engagement with recovery-based practice if they, and the service users, feel that working collaboratively with service users towards recovery is realistic.
Involvement of current or former service users in the design and/or delivery of the training programme will persuade staff that both recovery and collaboration is achievable and realistic for service users.
Service users were involved in both the design and delivery of the GetREAL intervention from the start of its development. Disappointingly, the GetREAL teams found that the case study units did not involve service users in planning and providing activities on their units to any great extent. The GetREAL teams attempted to rectify this by inviting staff to work with them to involve service users in making choices and run community meetings, thus demonstrating service users’ potential. This had some positive responses from staff, adding tentative support to this theory.
The staff nurse of unit 1 identified that:
The underlying issue is the lack of service user involvement in their own care let alone the running of the unit. It is very disempowering. Staff don’t even ensure that service users discuss and sign their care plans.
SFG SN, unit 1
Similarly, the GetREAL team’s OT noted that:
[T]he care planning does not involve the patients directly and when explored it is left to the nursing staff to discuss and seek collaboration.
RD, unit 1
Later, the GetREAL team involved service users in planning the week’s activities by sitting with them one to one, encouraging them to plan their own week and use the weekly timetable sheets (RD, unit 1).
The team identified that service users in unit 2 were underengaged. Although the team found this challenging to address, after organising a service user forum meeting they felt that service user involvement was progressing (RD, unit 2).
In unit 3, staff focus group participants felt that the GetREAL service user consultant had been an important contributor to the intervention. This encouraged the staff to put forward plans for more service user involvement in the unit and to invite in ex-service users (SFG, unit 3). The GetREAL team’s OT observed that the unit’s activity worker had started collecting ideas about what service users would like to do through running ‘taster sessions’, although the response was limited and the service users identified things that were already happening on the unit, rather than suggesting new activities (RD, unit 3).
If ‘quick wins’ in change towards increased recovery-based practice are identified, implemented and promoted, the staff groups will feel that they have made progress and that further change is achievable. Staff who were previously reluctant to engage may be newly motivated to engage with the programme.
There was support for ‘quick wins’ having a positive impact on staff changes. For instance, in unit 1:
[S]ome of the nursing staff mentioned that they had found the weekly timetable useful and seen the benefits of promoting service user involvement.
TN, unit 1
The unit 2 data showed that staff members gradually started to talk more about activities, introducing one-to-one activities with service users and responding to individual service users’ suggestions for specific activities (RD, unit 2). The GetREAL team’s OT observed that the staff members in unit 2 who had previously not been involved in the activities or shown some reluctance started putting themselves forward to cover the groups. For example:
[A] member of the team who previously spent the majority of his shift in the office was making a concerted effort to be out of the office with service users today.
RD, unit 2
Furthermore, it was reported by the unit 2 staff that:
[T]hey feel as a result of the GetREAL input staff are spending more one to one time with service users.
SFG, unit 2
Unit 3 staff focus group participants stated that ‘the GetREAL OT has set up a new white board with the programme of activities and groups for the week that was owned by the nurses and will be updated each Sunday’ (SFG, unit 3). This was done to encourage the staff members to look at the board and organise appointments around the activities, so that these were prioritised, and to manage shifts around activities (SFG, unit 3).
Staff will increase their engagement with recovery-oriented practice if they feel that they have the resources to do so and/or if barriers (individual, group or organisational) have been removed.
Sufficient flexibility in their shift/working pattern to enable participation in activities outside their ‘normal’ working day. A lack of flexibility impedes continuity of service user engagement/activities between one member of staff/one week and the next.
The qualitative data identified the importance of flexibility of shift and working patterns and problems when shift working is inflexible. For instance, the GetREAL team’s OT observed that many of the nursing staff members in unit 1 were supportive of service user activity but did not always know what was planned until the day they came into work. It became clear from the activity planning meeting that the unit’s OT did not plan the activities with the rest of the team (RD, unit 1).
They also noted in their reflective diary that ‘there are some people on the ward who say “that’s not a nursing job, I’m not doing it” but they have taken part in the GetREAL training workshops’ (RD, unit 1).
Data from unit 2 suggested that greater role flexibility led to positive outcomes. For example, nursing assistants became increasingly engaged in established activities and in suggesting new ones, and nurses became increasingly supportive of this by managing shifts to better accommodate these. It was noted in the GetREAL team’s reflective diary that the unit staff appeared keen to get involved with activities and, over time, this was true even of those staff members who had earlier stated that they felt activities to be the OT’s job (RD, unit 2).
The unit 3 staff were split into two teams and worked across two floors of the unit. This avoided the need for bank staff to cover staff absence, but it was observed by the GetREAL team’s OT that ‘Staff members go off for a week or two between shifts on both floors. This breaks continuity in getting activities up and running all following through what service users want’ (RD, unit 3).
Summary of findings from rapid realist review and case studies
The main aims of the realistic evaluation were (1) to identify and understand the factors associated with better uptake of the GetREAL staff training/change intervention and (2) to investigate the association between uptake of the intervention and the 12-month outcomes.
The rapid realist review identified seven priority theories of how change towards a more recovery-orientated approach might be facilitated in the context of the types of units that participated in the REAL study (‘context–mechanism–outcome’ theories).
The two that were rated as most important by the LRG were RS and SC. These were tested against case studies of three units that had participated in the cluster randomised controlled trial assessing the efficacy of the GetREAL intervention, using qualitative data collected through the staff focus groups carried out in phase 3 of the REAL study and other study materials. Evidence was found to support the RS priority theory (collaboration of staff and service users in developing action plans to support the change processes facilitated through the GetREAL intervention; synergising the GetREAL intervention with other programmes of change). There was also evidence for the importance of the ‘supporting change’ priority theory (regular supervision with a focus on the specific actions required in the GetREAL intervention; senior management support for the intervention including support for flexibility in multidisciplinary roles; sharing responsibility for the successful implementation of the intervention; and supporting changes to the structures and processes of the unit that could facilitate the intervention).
Quantitative analysis of the association between units’ uptake of the GetREAL intervention and service user outcomes
The aim of the quantitative analysis was to investigate the association between units’ uptake of the GetREAL intervention and service user outcomes at the 12-month follow-up. Data were included from all 19 units that received the GetREAL intervention and remained in the study such that 12-month outcome data were available. The quantitative results were then triangulated with the qualitative findings from the realistic evaluation case studies. 147
Methods
A quantitative rating instrument was developed to measure the uptake and receptiveness of each unit to the GetREAL intervention. A quantitative data analysis using random-effects modelling was then carried out using these ratings to investigate associations with service user outcome data collected at the 12-month follow-up in phase 3 of the REAL programme (service user activity as assessed by the time-use diary74 and social function as assessed by the LSP). 75
A categorical variable was developed from a combination of the GetREAL fidelity rating used in phase 3 of the REAL programme and the qualitative data used in the realistic evaluation, to rate each unit that received the GetREAL intervention dichotomously as ‘good’ or ‘poor’ in terms of their uptake and receptiveness of the intervention. Uptake was defined as the acceptance or adoption of a new product or idea, or the number of people wanting to do something, such as use a service or study a particular subject. Receptiveness was defined as having the quality of receiving, taking in or admitting, or being ready and willing to accept something such as a new idea. These definitions were operationalised and incorporated into a draft instrument by SC. ‘Uptake’ was rated using the following items from the GetREAL fidelity rating sheets: attendance of staff at the two training events, and whether or not the units had spontaneously corresponded with the intervention teams during the 12-month reinforcing stage. The ‘receptiveness’ of each unit was rated using items identified from the realist review of context and mechanisms associated with sustainable change in recovery-based practice within rehabilitation mental health units: ‘receptive staff’, ‘recovery is realistic’ and ‘resourced for recovery’. The draft instrument, consisting of 26 items, was piloted by SC with the data from the three case study units used in the realistic evaluation. The researchers carrying out the case study analyses (SB and HB) and the reviewer of the literature (MG) were consulted on the validity of the ratings and the items in the ratings instrument. The scores reflected both the qualitative and quantitative data for these three units (higher for unit 1, lower for unit 3). Amendments were made to the instrument, which included deleting some items considered of less direct relevance to the study aims.
The final rating instrument comprised 23 items, as shown in Table 38. This was used to rate all 19 units. Data were extracted from the GetREAL intervention fidelity rating sheets, the daily reflective practice diaries completed by the OT of each GetREAL intervention team during the first week of the enabling stage, the staff initial training day evaluations, supervision notes and initial action plans. For each unit, each item was given a score of 1 (good) or 0 (poor) when supporting data existed for that unit, and marked as ‘not known’ if there were no supporting data. A rating for each unit was calculated as the mean of all of the scored items.
Uptake data from fidelity sheets | Score | Evidence |
---|---|---|
Unit codes | ||
2. Dates for the first GetREAL training day(s) and release of staff agreed before the GetREAL team arrive | ||
5. Attendance at initial training day for at least 50% of staff | ||
11. Individual goal-setting (regarding activities) is carried out and recorded in care plans for at least 50% of service users | ||
13. Attendance at final training day by at least 50% of staff | ||
17. At the end of the 5 weeks, activity is included in at least 50% of service user individual care plans | ||
18. A link person is identified to keep e-mail contact with the GetREAL team | ||
Data from Microsoft Excel® (Microsoft Corporation, Redmond, WA, USA) file; criteria from original fidelity criteria At the end of 5 weeks visit by the GetREAL team, a link person makes contact with the GetREAL team at least once during the 12-month period |
||
Receptiveness (criteria from literature review) | ||
Data from the first week’s daily diaries, first supervision notes and Initial training evaluations | ||
Organisational structure, processes and systems | ||
The management team endorses, prioritises, supports and encourages the programme | ||
Organisational structure, processes and systems facilitate change; and programme is incorporated into existing change programme, if there is one | ||
The local OT was brought on board at the pre-disposing stage or at the start of the GetREAL visit | ||
Staff | ||
All unit staff in the team have a shared understanding of the programme | ||
There is staff role flexibility and/or shift pattern flexibility | ||
Staff attitudes: understand service users with complex needs have non-linear recovery; do not think they are too hard to engage with in recovery | ||
There is a shift from custodial or protective care model to recovery-based model, and engaging service users in activities of their choice | ||
Staff do not experience stress, low job satisfaction, burnout | ||
Culture and climate | ||
The culture is of a supportive organisation with supportive colleagues (teamwork) | ||
Culture of and positive collaboration between all staff disciplines including OTs (MDT teamwork) | ||
Supportive culture encourages autonomy and positive risk-taking | ||
Service users and staff work together in a community sharing power | ||
Resources | ||
Resources are adequate: staffing capacity, time, physical space and resources | ||
Reduced administrative burdens and competing work priorities allow time to spend with service users on activities | ||
Time and resources for shift handovers | ||
Strong supportive community and family links |
LM analysed the data using Stata version 13 for Windows (StataCorp LP, College Station, TX, USA). Summary statistics were calculated for the uptake and receptiveness instrument total score, the uptake and receptiveness subscale scores and the individual instrument items. The three ratings (uptake, receptiveness and total score) were used as continuous measures and also dichotomised at their means and medians.
The outcomes entered into the model were the 12-month follow-up assessments of service user activities74 and social function75 gathered in phase 3 of the REAL programme (time-use diary and LSP scores, respectively). Individual-level data were modelled using linear mixed models to take into account the clustering by unit to see if the uptake and receptiveness rating scores, or items selected in line with the theories generated from the realistic evaluation, were associated with the outcomes. 148 These were explored first in an unadjusted model and then in three adjusted models, as follows:
-
adjusted model 1 – adjusted for unit mean baseline score of the outcome (time-use diary or LSP)
-
adjusted model 2 – adjusted for unit mean baseline score of the outcome and GetREAL team
-
adjusted model 3 – adjusted for unit mean baseline score of the outcome and interaction between uptake and receptiveness rating score or dichotomous good/poor rating as appropriate and GetREAL team.
Results
Table 39 shows the summary statistics for the 19 units that were randomised to the GetREAL intervention. The mean overall rating score for uptake and receptiveness was 0.47 (SD 0.16) and the median was 0.50. All units had a score for all uptake items (mainly because these were primarily from the fidelity checklist). Most units scored highly on uptake (e.g. in 18 out of 19 of units at least 50% of staff attended the first GetREAL training day) with the exception of item 7 (units making contact with the GetREAL team in the year following the intervention), which was endorsed by 7 out of 19 units. There were more missing data for the receptiveness items because these were scored from the GetREAL teams’ reflective diaries, supervision notes and training evaluations, which were not designed for this purpose. For example, only nine units had scores for the item ‘time and resources for shift handover’. The mean receptiveness score was lower than the mean uptake score [0.34 (SD 0.21) and 0.69 (SD 0.21), respectively].
Variable | Baseline, mean (SD) | 12 months, mean (SD) |
---|---|---|
Activity (service user rated)a | 46 (9) | 48 (11) |
Activity (staff rated)a | 45 (11) | 46 (14) |
LSPa | 124 (12) | 127 (18) |
Rating of uptake and receptiveness scoresb | ||
Overall | 0.47 (0.16) | |
Uptake | 0.69 (0.21) | |
Receptiveness | 0.34 (0.21) | |
Items from rating scores,c n/N (%) | ||
Uptake 1 | 16/19 (84) | |
Uptake 2 | 18/19 (95) | |
Uptake 3 | 9/19 (47) | |
Uptake 4 | 9/19 (47) | |
Uptake 5 | 16/19 (84) | |
Uptake 6 | 17/19 (89) | |
Uptake 7 | 7/19 (37) | |
Receptiveness 8 | 10/18 (56) | |
Receptiveness 9 | 5/18 (28) | |
Receptiveness 10 | 10/19 (53) | |
Receptiveness 11 | 7/19 (37) | |
Receptiveness 12 | 3/15 (20) | |
Receptiveness 13 | 2/15 (13) | |
Receptiveness 14 | 8/17 (47) | |
Receptiveness 15 | 1/12 (8) | |
Receptiveness 16 | 5/16 (31) | |
Receptiveness 17 | 2/17 (12) | |
Receptiveness 18 | 3/15 (20) | |
Receptiveness 19 | 5/17 (29) | |
Receptiveness 20 | 6/17 (35) | |
Receptiveness 21 | 10/13 (77) | |
Receptiveness 22 | 4/9 (44) | |
Receptiveness 23 | 7/13 (54) |
Table 40 shows the results of the regression modelling. None of the unadjusted or adjusted models was statistically significant, with the exception of the model that investigated the association between uptake item ‘at least 50% of staff attended the initial training day for staff’ and time-use diary scores (–11.08, 95% CI –21.57 to –0.59). However, this is likely to be type 1 error due to multiple testing (a large number of models were computed). Additionally, the coefficient is in the unexpected direction; that is, if a high percentage of staff attended the training one might expect a positive association. However, most of the models did show a positive association between outcome and ratings of receptiveness and uptake, albeit one that was not statistically significant.
Variable/score | Coefficient (95% CI) | |||
---|---|---|---|---|
Unadjusted | Adjusted 1a | Adjusted 2b | Adjusted 3c | |
Overall score of uptake and receptiveness | ||||
Activity (service user rated) | 0.67 (–17.82 to 19.16) | 2.40 (–9.46 to 14.26) | 2.42 (–9.87 to 14.72) | 10.75 (–12.92 to 34.41) |
Activity (staff rated) | –1.50 (–20.32 to 17.31) | –0.02 (–12.87 to 12.84) | –1.31 (–14.88 to 12.27) | –20.97 (–47.89 to 5.96) |
Social function | 11.62 (–13.46 to 36.71) | 5.62 (–14.89 to 26.13) | 4.34 (–16.93 to 25.62) | 5.62 (–36.04 to 47.28) |
Overall score ≥ meand | ||||
Activity (service user rated) | 1.42 (–4.42 to 7.25) | 1.16 (–2.52 to 4.83) | 1.17 (–2.58 to 4.93) | 1.73 (–3.96 to 7.42) |
Activity (staff rated) | 0.97 (–4.90 to 6.85) | –0.06 (–4.01 to 3.90) | –0.35 (–4.42 to 3.73) | –3.42 (–9.39 to 2.55) |
Social function | 4.36 (–3.50 to 12.23) | 1.55 (–4.96 to 8.05) | 1.22 (–5.32 to 7.77) | –0.80 (–10.30 to 8.70) |
Overall score ≥ mediand | ||||
Activity (service user rated) | 1.42 (–4.42 to 7.25) | 1.16 (–2.52 to 4.83) | 1.17 (–2.58 to 4.93) | 1.73 (–3.96 to 7.42) |
Activity (staff rated) | 0.97 (–4.90 to 6.85) | –0.06 (–4.01 to 3.90) | –0.35 (–4.42 to 3.73) | –3.42 (–9.39 to 2.55) |
Social function | 4.36 (–3.50 to 12.23) | 1.55 (–4.96 to 8.05) | 1.22 (–5.32 to 7.77) | –0.80 (–10.30 to 8.70) |
Uptake score | ||||
Activity (service user rated) | 0.95 (–13.06 to 14.96) | 2.12 (–6.65 to 10.89) | 2.22 (–7.02 to 11.46) | 3.01 (–11.47 to 17.49) |
Activity (staff rated) | 1.47 (–12.63 to 15.56) | 3.96 (–5.18 to 13.11) | 3.50 (–6.41 to 13.41) | –9.55 (–25.85 to 6.73) |
Social function | 10.98 (–7.47 to 29.43) | 10.05 (–4.03 to 24.13) | 9.81 (–5.16 to 24.77) | 3.58 (–21.20 to 28.36) |
Uptake score ≥ meand | ||||
Activity (service user rated) | 0.30 (–6.04 to 6.64) | 1.31 (–2.71 to 5.33) | 1.45 (–2.88 to 5.77) | –2.15 (–11.24 to 6.94) |
Activity (staff rated) | 0.48 (–5.97 to 6.93) | 2.12 (–2.19 to 6.43) | 1.96 (–2.77 to 6.68) | –11.10 (–21.77 to –0.44) |
Social function | 5.52 (–2.85 to 13.89) | 3.05 (–3.89 to 9.98) | 2.71 (–4.80 to 10.22) | 6.80 (–9.88 to 23.49) |
Uptake score ≥ mediand | ||||
Activity (service user rated) | –0.10 (–6.20 to 6.00) | 1.46 (–2.44 to 5.36) | 1.45 (–2.46 to 5.37) | 3.76 (–1.75 to 9.27) |
Activity (staff rated) | –0.08 (–6.20 to 6.04) | 2.54 (–1.59 to 6.67) | 2.44 (–1.72 to 6.60) | 0.74 (–5.24 to 6.72) |
Social function | 1.43 (–6.89 to 9.74) | 1.70 (–4.77 to 8.17) | 1.46 (–5.02 to 7.94) | 0.47 (–8.81 to 9.74) |
Receptiveness score | ||||
Activity (service user rated) | 1.03 (–13.53 to 15.59) | 0.84 (–8.49 to 10.17) | 0.80 (–8.60 to 10.20) | 6.09 (–10.53 to 22.70) |
Activity (staff rated) | –0.48 (–15.19 to 14.22) | –2.87 (–12.86 to 7.12) | –3.32 (–13.40 to 6.76) | –6.74 (–24.02 to 10.54) |
Social function | 5.84 (–14.10 to 25.78) | 0.99 (–15.01 to 16.99) | 0.41 (–15.49 to 16.31) | 3.65 (–23.02 to 30.32) |
Receptiveness score ≥ meand | ||||
Activity (service user rated) | 1.93 (–3.98 to 7.83) | 1.38 (–2.35 to 5.10) | 1.40 (–2.40 to 5.21) | 3.38 (–2.02 to 8.79) |
Activity (staff rated) | 0.63 (–5.36 to 6.62) | –0.36 (–4.39 to 3.67) | –0.66 (–4.80 to 3.48) | –1.67 (–7.65 to 4.32) |
Social function | 5.45 (–2.44 to 13.34) | 1.98 (–4.77 to 8.73) | 1.67 (–5.13 to 8.46) | 2.05 (–7.28 to 11.38) |
Receptiveness score ≥ mediand | ||||
Activity (service user rated) | –1.13 (–6.96 to 4.70) | 0.64 (–3.15 to 4.44) | 0.66 (–3.14 to 4.47) | 3.43 (–1.92 to 8.78) |
Activity (staff rated) | –3.22 (–8.93 to 2.50) | –2.55 (–6.45 to 1.34) | –2.47 (–6.38 to 1.44) | –1.39 (–7.37 to 4.59) |
Social function | –1.67 (–9.64 to 6.29) | –2.45 (–8.45 to 3.54) | –2.44 (–8.35 to 3.48) | 1.67 (–6.75 to 10.08) |
Attendance at initial training day for at least 50% of staff | ||||
Activity (service user rated) | –4.01 (–17.32 to 9.30) | –2.00 (–10.95 to 6.95) | –2.05 (–11.37 to 7.28) | – |
Activity (staff rated) | –11.03 (–24.75 to 2.70) | –11.08 (–21.57 to –0.59) | –10.98 (–21.76 to –0.20) | – |
Social function | –1.86 (–21.19 to 17.46) | 5.92 (–10.61 to 22.46) | 7.11 (–9.55 to 23.76) | – |
Attendance at final training day by at least 50% of staff | ||||
Activity (service user rated) | –0.29 (–6.16 to 5.58) | 1.74 (–1.87 to 5.35) | 1.82 (–1.84 to 5.47) | –0.51 (–5.88 to 4.85) |
Activity (staff rated) | 1.69 (–4.12 to 7.50) | 1.87 (–2.12 to 5.85) | 2.02 (–1.98 to 6.03) | –0.78 (–6.98 to 5.42) |
Social function | –1.43 (–9.51 to 6.66) | –1.06 (–7.39 to 5.27) | –0.86 (–7.17 to 5.44) | 3.84 (–5.65 to 13.34) |
At the end of 5-week visit by the GetREAL team, a Link person makes contact with the GetREAL team at least once during the 12-month period | ||||
Activity (service user rated) | –5.23 (–10.78 to 0.33) | –1.00 (–5.17 to 3.16) | –1.22 (–5.67 to 3.23) | –0.09 (–6.39 to 6.20) |
Activity (staff rated) | –4.38 (–10.11 to 1.35) | 0.77 (–3.62 to 5.16) | 0.34 (–4.45 to 5.13) | –1.65 (–8.45 to 5.16) |
Social function | –2.24 (–10.40 to 5.91) | –0.21 (–6.70 to 6.28) | –1.26 (–8.46 to 5.94) | –3.09 (–13.03 to 6.85) |
The management team endorses, prioritises, supports and encourages the programme | ||||
Activity (service user rated) | 0.95 (–5.10 to 7.00) | –1.19 (–5.03 to 2.65) | –1.31 (–5.21 to 2.60) | –0.34 (–6.07 to 5.39) |
Activity (staff rated) | 2.14 (–3.65 to 7.92) | 0.74 (–3.34 to 4.82) | 0.51 (–3.62 to 4.65) | –0.98 (–7.15 to 5.18) |
Social function | 2.14 (–6.12 to 10.41) | 0.90 (–5.72 to 7.52) | 0.61 (–5.92 to 7.14) | –2.42 (–12.18 to 7.35) |
Organisational structure, processes, systems facilitate change and incorporate the programme into existing change programme, if there is one | ||||
Activity (service user rated) | 0.09 (–6.43 to 6.61) | 1.72 (–2.39 to 5.84) | 2.01 (–2.42 to 6.44) | 2.17 (–3.21 to 7.55) |
Activity (staff rated) | –0.56 (–7.29 to 6.17) | 0.09 (–4.51 to 4.68) | –0.48 (–5.47 to 4.51) | –1.97 (–8.00 to 4.05) |
Social function | 2.76 (–5.84 to 11.37) | 0.51 (–6.36 to 7.38) | –0.80 (–7.86 to 6.25) | –1.37 (–9.77 to 7.03) |
The local OT was brought on board at the pre-disposing stage or at the start of the GetREAL visit | ||||
Activity (service user rated) | 1.91 (–3.91 to 7.74) | –0.50 (–4.37 to 3.37) | –0.48 (–4.42 to 3.45) | –4.71 (–10.13 to 0.70) |
Activity (staff rated) | 3.81 (–1.92 to 9.55) | 1.71 (–2.30 to 5.71) | 2.32 (–1.91 to 6.54) | 1.97 (–4.33 to 8.27) |
Social function | 4.59 (–3.27 to 12.45) | 0.60 (–6.18 to 7.38) | 1.38 (–5.66 to 8.43) | 2.87 (–6.44 to 12.18) |
All unit staff in the team have shared understanding of the programme | ||||
Activity (service user rated) | 0.56 (–5.46 to 6.58) | –2.13 (–5.86 to 1.59) | –2.13 (–5.86 to 1.60) | –5.39 (–10.92 to 0.14) |
Activity (staff rated) | 1.05 (–5.06 to 7.15) | –1.91 (–6.01 to 2.19) | –1.84 (–5.94 to 2.27) | –1.68 (–8.27 to 4.91) |
Social function | 6.45 (–1.38 to 14.29) | 4.07 (–2.25 to 10.38) | 4.27 (–1.92 to 10.47) | 5.50 (–3.71 to 14.70) |
Culture of and positive collaboration between all staff disciplines including OTs (MDT work) | ||||
Activity (service user rated) | 2.37 (–7.73 to 12.48) | 3.00 (–3.53 to 9.53) | 3.43 (–3.73 to 10.59) | – |
Activity (staff rated) | –2.78 (–12.54 to 6.98) | –2.22 (–8.99 to 4.54) | –4.08 (–11.63 to 3.46) | – |
Social function | –2.50 (–16.70 to 11.70) | –4.43 (–15.12 to 6.26) | –6.19 (–17.65 to 5.27) | – |
A further dichotomous interaction variable was added to the models to account for the GetREAL team delivering the intervention. This was done not only because there may have been differences in their approach, but also because the ratings of receptiveness and uptake might differ between teams, as the reflective diaries gave more detail about receptiveness in one team than the other. This did not alter the results to any meaningful degree.
Discussion
The rapid realist review highlighted the complexity inherent in an intervention to change the behaviour of an interdisciplinary staff team working in a complex and challenging setting. We have identified several possible mechanisms that may operate to encourage staff to increase their recovery-based practice long term, following a training intervention. The literature has also uncovered many possible contextual factors (pertaining broadly to the training/change programme itself, resources available to the staff members, organisational structures and systems, culture and climate, and the staff team) that may each trigger or block one or more of these mechanisms to help, or hinder, long-term change. Through engagement with a stakeholder (LRG) group, we identified seven priority programme theories (CMO configurations) for long-term change. We presented consolidating evidence for these theories from the literature and tested these theories using data collected during the evaluation of the GetREAL intervention. The rapid realist review investigated the broad area of recovery-based training and change programmes in the literature. In the evaluation of the GetREAL intervention we narrowed our focus to examine one aspect of this: working with service users to engage in activities of their choice.
The case study analysis demonstrated that the enabling stage of the GetREAL intervention was successfully delivered in all three selected units; however, long-term change in practice was problematic. The staff focus groups and service user interviews highlighted a range of challenges that units faced during the intervention. Leadership and ongoing support to embed changes in practice were key.
The evaluation of the GetREAL intervention was designed as a conventional cluster randomised controlled trial with a view to answering the question ‘does the intervention work?’. As such, fidelity to the intervention was important. We acknowledge, however, that this may not be the most appropriate approach for evaluating complex interventions. Complex interventions are those interventions that attempt to change social systems through influencing the behaviour of individuals and, where those systems can respond in unpredictable ways, can demonstrate emergence (complex patterns of behaviour arise out of relatively simple interactions) and non-linearity of outcomes. 103 It has been argued that for complex interventions, maintaining the integrity of the key functions provided by the key elements of the intervention is more important than maintaining the integrity of the specific actions used to achieve them. 149,150 Furthermore, within the realist framework, we can turn to CMO configurations, rather than the intervention per se as the unit of analysis. 150 Various ways of approaching a ‘realist randomised controlled trial’ have been proposed. In a ‘realist quasi-experimental design’ for an evaluation of youth mentoring, acknowledging that ‘the firing of a mechanism is completely dependent on context’, the author purposively chose experimental and control groups to have similar initial conditions, as far as possible, in terms of the contextual factors that were deemed important for firing the theorised mechanism. 151 The downside to this selective approach, however, is that allocation by the play of chance is lost, and many known and unknown influences might bias the assessment of effectiveness. Another suggestion is for a realist randomised controlled trial to focus on examining mechanisms of change, and/or the effects of intervention components separately as well as in combination, and to look for those intermediate/proximal outcomes that are hypothesised, rather than the ultimate, long-term outcomes that the intervention seeks to produce. 150 Once again, however, there may be statistical objections if not all participants, be they individuals or organisations, receive the components in approximately the same way or same setting, and the internal validity of the trial is compromised.
In some organisational settings there may be overwhelming problems that would need to be remedied before a training/change intervention would be worth undertaking. Therefore, in addition to tailoring the GetREAL intervention to the individual units, and including realistic evaluation in the methodology, we propose that it would be useful to do some initial, pre-intervention exploration of the organisation. This would serve to identify any organisational/structural/staff team issues that might present fault lines when the team is placed under the additional strain of the intervention. Of course, a full realist investigation would have been impossible before the trial as by its nature it would have been conjectural rather than actual.
The quantitative exercise undertaken to investigate the uptake and receptiveness of the GetREAL intervention did not find any conclusive associations that could be discussed in comparison with the other findings from the case studies and the literature review. The rating of uptake and receptiveness encountered weaknesses in the methods, as the data had several limitations that impacted on their validity and reliability. All of the qualitative data apart from the fidelity sheets were originally produced not for the purpose of research, but to enhance reflective practice and for training purposes with unit staff. A quantitative rating was then applied to these qualitative data, rather than a standardised method such as a questionnaire. In addition, each OT adopted a different focus and style in their daily reflective diaries, making it difficult to carry out rating in a reliable manner.
The choice of realist methodology to evaluate the GetREAL intervention has been vindicated through a demonstration of the complexity of the system. The use of a rapid realist review to generate candidate programme theories proposing the relationships between context and mechanism leading to long-term change has been instrumental to the evaluation process, particularly as we were dealing with a scarcity of programme data for evaluation. Without the rapid realist review to generate the candidate programme theories, there would have been a danger of ‘overfitting’ the data,151 and our findings would have limited generalisability even to other units within the study.
Owing to time limitations, the rapid realist review was constrained by the scope (and quantity) of the documents feeding into the candidate programme theories. Thus, for pragmatic reasons, the bibliographic database searches were tightly defined; however, this was tempered by taking a more inclusive approach to papers put forward by the stakeholder (LRG and EP) groups and discovered through grey literature, reference and citation searches. As was appropriate for maximum utility to the realistic evaluation, the candidate programme theories were rooted in the context of recovery-based training for multidisciplinary staff working in inpatient mental health rehabilitation services. Further work would be needed to make our theories generalisable outside this context. The triangulation of our theories with other published mid-range theories of individual and organisational behavioural change is also desirable: again, an activity that was not performed for this evaluation owing to time constraints. However, transparently recorded processes throughout the rapid realist review have ensured a robust platform on which to build this future work.
The LRG/EP consultation and involvement was a strength of our approach; although time constraints prevented us from performing a Delphi consensus-forming exercise, we did obtain validation of our candidate programme theories and a steer on prioritising them for further exploration. However, in some instances the papers that were originally suggested by the LRG and EP members were not of immediate relevance to the review, possibly representing a shortfall in our efforts to explain the methodology in layman’s terms.
Although the priority programme theories have been consolidated using evidence from the literature, and tested as far as possible using data gathered during the evaluation of the GetREAL intervention, more thorough theory testing from case studies drawn from wider bodies of literature is ongoing work. Additionally, we acknowledge that theoretical saturation may well not yet have been reached; there may yet be further mechanisms and/or contexts that are undiscovered. For instance, in the treatment of theory RS3 (incorporation of recovery to existing change programme) (see Priority theory RS3), we have proposed that one way of moderating the effect of staff ‘change fatigue’ might be to incorporate a new training programme into an existing change initiative. However, other strategies may also be effective.
Although acknowledging the complexity of the interactions between contexts and mechanisms, our findings suggest that the following modifications could strengthen the GetREAL intervention:
-
Pre-intervention site assessment to identify potential problems and the service’s readiness for change.
-
Initial buy-in needed by all disciplines, at all levels.
-
Attendance at training workshops should be promoted by senior staff to demonstrate managerial sign-up and to engage reluctant staff.
-
Services should ensure that they have structures and processes to facilitate and maintain service user involvement in the planning and delivery of the service (e.g. service user-led meetings and posts such as peer support workers).
-
There needs to be sufficient staff and greater flexibility in working patterns to allow staff time to engage in activities with service users.
-
The recording of service user activities needs to reflect all aspects of activity, not just those on the unit, and data need to be collated and fed back to staff and service users in a way that is meaningful and useful for them and for service managers.
-
Roles such as change agent or champion need to be integrated within the team, rather than this function being associated with an individual staff member (who may leave).
-
Existing practice improvement programmes (such as ‘Productive Ward’) need to be integrated with GetREAL to embed a combined long-term change process.
-
The OT on the unit needs to have the skills and support to engage the MDT in activities as part of everyone’s role.
-
Staff supervision needs to incorporate regular review of how the team and individual staff are working to prioritise and engage service users in activities.
Chapter 7 Summary of the main findings from the REAL programme, main limitations and recommendations for further research
Summary of the main findings from the REAL programme
Possibly the most important result of the REAL research programme is the finding that NHS inpatient mental health rehabilitation services are able to work constructively with the ‘low-volume, high-needs’ group of people with severe and complex mental health problems who have not recovered adequately to be discharged home from acute inpatient mental health services. The specialist treatment and support provided by inpatient mental health rehabilitation services facilitates successful community discharge for the majority of this group, with over half achieving this within 12 months. The figure could rise to almost three-quarters were there adequate provision of supported accommodation in the community for people to be discharged to.
We found that the quality of care provided in inpatient mental health rehabilitation facilities in England is higher than that in the rest of Europe. Higher-quality care was associated with greater service user autonomy (the main aim of rehabilitation) and with greater satisfaction with care, but not with costs of care. This is somewhat reassuring as it suggests that, even in financially challenging times, quality of care is not solely dependent on greater investment of resources. We complemented the quantitative data gathered in our national survey with qualitative interviews with staff and service users. These identified a number of facilitators of and barriers to delivering high-quality mental health rehabilitation services, including the provision of adequate resourcing and supervision of staff.
In our cohort study we found that the additional costs of care associated with the improvement in service users’ social function were small (£67 per month per service user). This appears a relatively small investment to make for such an important outcome. This finding from our health economic analysis helps to justify ongoing investment in mental health rehabilitation services, as the alternative is for people with complex mental health needs to remain for long periods on acute inpatient units without the specialist care they require, not progressing in their recovery and accruing considerable costs to the health service.
Our cohort study identified a possible association between successful discharge and a focus on recovery-orientated practice that promotes service user activities and social skills. Although we should not overinterpret this finding, as this was an uncontrolled study and the ORs were small, it makes intuitive sense and may provide a helpful indication of the more active components of the complex intervention of mental health rehabilitation that require further study.
Alongside our cohort study, we carried out a rigorous cluster randomised controlled trial in a large number of rehabilitation units in England to investigate the efficacy of a staff training intervention, which aimed to facilitate service user activities in inpatient mental health rehabilitation units. These units were chosen on the basis of their quality and all were in the lower median of scores as measured by the QuIRC assessment. The justification for a focus on activities was provided by previous research and clinical consensus of the importance of activities in mental health rehabilitation. The results of our cohort study corroborate this focus. However, we found no significant difference in clinical effectiveness or cost-effectiveness 12 months after recruitment into the trial between those units that received the training intervention and those that continued to provide usual care. Our qualitative interviews with staff and service users who received the intervention identified that although the pre-disposing and enabling stages were well received, there was a reversion to previous practice during the reinforcing stage, after the teams who had trained unit staff left. Inflexible role definition, inadequate resources and competing priorities were the main barriers to sustaining the intervention. We carried out an additional component to our original research programme proposal in the form of a ‘realist evaluation’ of the intervention. This suggested that strengthening the intervention through stronger ‘buy-in’ of senior management to support and encourage staff to focus on service user activities alongside other tasks may improve its effectiveness. In addition, it suggested that key aspects of the intervention should be embedded into routine practice and reviewed through supervision processes to enhance longer-term change.
Our finding may also have been a consequence of the nature of selection of units for phases 3 and 4 based on their QuIRC scores. It made sense to conduct a naturalistic evaluation of patients’ outcomes in the higher-quality rehabilitation units (phase 4), as this gave a measure of progress in the best possible circumstances. However, with hindsight, it possibly made less sense to include only lower-quality units in the randomised trial (phase 3), as these were units that, by definition, were functioning at a lower level, possibly in more difficult circumstances, with fewer resources and more entrenched staff attitudes. Although these were the units in most need of improvement, they may have required the most intensive training and the most sustained input after the training to maintain the changes achieved. An alternative approach might have been to select units with scores between the first and third quartiles on the QuIRC, in which the training might have had its maximum range of impact. The cohort in phase 4 would then have been made up of high- and low-functioning units in which routine outcomes could have been contrasted. It is always tempting, however, to redesign studies after they are completed and, thus, this discussion must remain conjectural. Furthermore, this alternative approach might have been threatened by insufficient power to make what are, in essence, more fine-grained comparisons between units in both phases 3 and 4.
Although the qualitative aspects of our research focused on different components of our programme, a number of common, higher-level themes emerged that are relevant to the successful implementation of changes to staff practice that aim to improve the quality of care delivered in mental health rehabilitation services. These are summarised in Figure 11.
Summary of the main limitations of the REAL programme
The national survey we carried out in phase 1 of the programme represents the first in-depth study of NHS mental health rehabilitation units in England. Although the high participation rate strengthens the generalisability of our findings, we acknowledge that there are many rehabilitation units in the independent sector that were not included in our study, and we also excluded NHS secure rehabilitation units. We have explained the justification for our focus further in the limitations section of Chapter 2, but the main reason was the lack of time and resources to expand the programme beyond non-secure NHS facilities. Some limitations to the qualitative component of phase 1 also need to be acknowledged. The findings cannot be regarded as representative of all rehabilitation units in the NHS in England. We were unable to purposively sample units, staff and service users owing to the logistical challenges of collecting data for this component concurrently with the national survey. However, the characteristics of the staff and service users interviewed were similar to those of the whole phase 1 sample. Nevertheless, we acknowledge the possibility of a sampling bias, as those with specific issues that they wished to discuss may have been more likely to agree to participate. Furthermore, interviews could not be carried out with service users who were too unwell to give informed consent to participate and therefore those interviewed were likely to be further forward in their recovery than all users of rehabilitation services.
Our cluster randomised controlled trial was conducted rigorously but our researchers were unmasked in 15% of units. Although, when this occurred, the second researcher was able to collect the 12-month follow-up data, the researchers were able to identify three-quarters of units correctly in terms of whether or not they received the GetREAL intervention. However, the fact that we found no differences in outcomes between intervention and comparison units suggests that observer bias was not a major problem. Although the trial was slightly underpowered owing to the higher observed intracluster correlation coefficient and the slight under-recruitment of service users, this was compensated for by the smaller actual cluster size (8.8 rather than the 12 we included in our sample size estimate) and through the use of a more efficient analysis at the service user level that adjusted for baseline scores. The supporting unit-level analyses concurred with the results of the primary service user-level analysis, which adds to our confidence that our results were robust.
In the qualitative component of phase 3 of our programme, we employed purposive sampling to ensure participation of staff of different seniority, different disciplines and unit types (hospital/community based) based in different settings (urban/suburban/rural). There was also a good range in the length of time since the staff of the unit had participated in the GetREAL intervention. However, logistical constraints meant that participants were mostly the staff who happened to be present on the day that the focus interviewers attended. Although the unit managers attempted to agree a date and time for the focus groups when the maximum number of staff who had experienced the GetREAL intervention were available, inevitably some of those who may have wished to participate will not have been available and, thus, some important insights may have been missed. A similar sampling bias applies to the service user participants in this component and, as in phase 1, those with specific issues they wished to discuss may have been more likely to agree to participate. Again, interviews could not be carried out with service users who were too unwell to give informed consent to participate and, therefore, those interviewed were likely to be further forward in their recovery than other service users on the unit. Nevertheless, the overlap of higher-level themes that emerged from the qualitative components in phase 1, phase 3 and the additional realistic evaluation that was undertaken provides corroborative validation of the data interpretation and some saturation of themes.
Although our analyses in the cohort study we carried out in phase 4 were exploratory, we can have some confidence in our findings. We recruited a large sample from across most of the better-performing inpatient mental health rehabilitation units in England. Our follow-up rate was excellent, with primary outcome data on successful discharge collected on 97% of our cohort. This is also a ‘hard’ dichotomous outcome, which does not rely on subjective opinion. Our decision to use staff-rated outcomes also minimised the number of missing data on service users’ social functioning at the 12-month follow-up. Nevertheless, service user-rated outcomes, such as quality of life and satisfaction with treatment and support, would have allowed us to report on a more comprehensive range of perspectives on the concept of ‘meaningful’ clinical outcome.
Recommendations for future research
The REAL research programme focused on non-secure NHS inpatient mental health rehabilitation services. These represent only one component of the whole-system mental health rehabilitation care pathway. 94 Inpatient mental health rehabilitation services of various types are provided by the independent sector (particularly forensic/secure units and so-called ‘locked’ rehabilitation units). National surveys of these facilities, similar to the survey we carried out in phase 1 of the REAL study, would provide useful information about the number and type of services being provided in the secure and independent sector, the quality of the care provided and the characteristics of the people using them. Nevertheless, our findings are likely to be relevant to all services providing care for people with longer-term and complex mental health problems.
The results of the REAL programme suggest that future research is required to develop and test the effectiveness of interventions that enhance recovery-based practice that facilitates service user activities and social skills, leading to successful community discharge. Our findings have helped to clarify the importance of taking into account the context within which services operate and the various factors that may influence the successful implementation of new, complex interventions. Although our trial focused on a staff training programme aimed at improving service user engagement in activities, our findings, particularly those from the qualitative components of the programme and the realistic evaluation of the GetREAL intervention, have relevance to the development of future trials of complex interventions delivered in mental health rehabilitation services. Many of these findings will also be of relevance in other settings. The specific factors we identified as important to consider in this regard are as follows:
-
assessing the service’s readiness for change prior to implementation
-
postponing implementation if a service is under threat of reconfiguration or closure
-
gaining ‘buy-in’ from staff at all levels early on
-
ensuring that authentic and robust support from senior staff is available to enable flexiblility and change in staff working practices
-
synergising the intervention with any other change programmes that are taking place
-
supporting staff through regular supervision that includes review of practice changes implemented as part of the intervention
-
supporting the service to enhance its approach to service user involvement, with a particular focus on how it involves service users in decisions about the choice of activities provided in the unit and the best ways to support them to access activities off the unit
-
supporting the role of a local champion long term, ensuring succession planning for this role
-
providing ongoing review of the impact of implementation through collation and feedback of appropriate process and outcome metrics to staff and service users.
We propose to use this learning to refine and re-evaluate the GetREAL staff training programme in a further research study. As well as taking into account the points above, we propose to expand the scope of the intervention to include a focus on improving service users’ social skills and improvement in the recovery-based practice of the service.
The group of service users who did not achieve successful discharge also requires further consideration. We identified a number of characteristics associated with this group in our cohort study (longer length of admission, history of arson, receipt of CBT). Our findings are consistent with those of a case–control study of 200 service users carried out in Ireland that found that those who failed to achieve successful discharge from inpatient mental health services at 18-month follow-up had more unmet needs, more challenging behaviours and more substance misuse problems than those who progressed (Dr Lavelle, St Ita’s Hospital, Dublin, Ireland, 2012, personal communication). It therefore appears that there is a particularly complex subgroup of people residing within inpatient mental health rehabilitation services, for whom appropriate interventions need to be identified to mitigate the negative consequences of lengthy admissions and the associated care costs. This group should be a focus for future research.
The majority of people with complex mental health needs are discharged from inpatient mental health rehabilitation services to supported accommodation services of various types provided by the voluntary (‘third’) sector, the independent sector or social services. We are currently undertaking a programme of research into specialist mental health supported accommodation across England, the findings of which will complement those of the REAL study. The Quality and Effectiveness of Supported Tenancies for people with mental health problems (QuEST) study is also funded by the NIHR (RP-PG-0610-10097) and will report in 2017. Further details can be found at www.ucl.ac.uk/quest.
Finally, in phase 1 of the REAL programme, we found that over half of the NHS trusts in England had a local community mental health rehabilitation team. These teams have, historically, been subject to the vagaries of mental health policy; many were rebadged as assertive community treatment teams during the era of the National Service Framework for Mental Health,2,12 and it is interesting to note that they may be having something of a renaissance. Their role in supporting successful community discharge for those with complex needs requires further research.
Study management
We established a PMG at the start of the programme that met quarterly to monitor and advise on the progress of the research. The chief investigator (HK) chaired this group and membership comprised all coapplicants, the researchers, the project manager, the study statistician (LM) and a health economist (LK). The group concerned itself with methodological and practical problems arising in the day-to-day running of the project.
Since the research programme started, Tony Johnson, the senior study statistician, retired and was succeeded in 2010 by Professor Rumana Omar, Professor of Biostatistics at University College London. The study statistician, Robyn Drake, left her post and was replaced by Dr Louise Marston. In addition, clinical collaborators Gemma Dorer (Camden and Islington NHS Foundation Trust) and Marieke Wrigley (South London and Maudsley NHS Foundation Trust), both of whom are OTs working in rehabilitation services, provided consultation for relevant aspects of the programme, particularly in relation to phase 2, the development of the GetREAL programme. In June 2011, Gemma Dorer, project collaborator, left on maternity leave and was succeeded by Louise Reynolds, senior OT at Camden and Islington NHS Foundation Trust. Louise Reynolds provided line management of the GetREAL team OTs during phase 3.
The study was conducted within the auspices of a UK Clinical Research Network fully registered clinical trials unit (PRIMENT). This clinical trials unit is formed from a collaboration of the Division of Psychiatry, the Department of Primary Care and Population Sciences and the Department of Statistical Sciences at University College London. The clinical trials unit provides an infrastructure for the conduct of large national cohort studies and randomised trials across the UK, and provides access to experienced triallists, statisticians and health economists. MK is joint director of the clinical trials unit.
A Trial Steering Committee was convened to oversee the cluster randomised controlled trial in phase 3 of the programme. This group was chaired by Professor Philippa Garety (Professor of Psychology, Institute of Psychiatry, King’s College London), and the other members were Professor Diana Rose (Professor of Service User Research, Institute of Psychiatry, King’s College London), Professor Morven Leese (Professor of Biostatistics, Institute of Psychiatry, King’s College London) and Ms Genevieve Smyth (Mental Health Lead, COT). The committee met between once and twice per year during phase 3 of the REAL programme to review the progress of the trial and to consider the findings. These meetings were attended by the chief investigator (HK), the programme manager (ML) and other members of the research team as appropriate.
Achievement of anticipated outputs and dissemination of results
The outputs of the research programme are detailed below against the deliverables that were specified in our original proposal. Deliverables five and six specify our dissemination strategy.
1. The results from phase 1 will provide a comprehensive description of contemporary English mental health rehabilitation services and the people who use them. These results will allow us to comment on current service provision in terms of the appropriateness of service resourcing in relation to clinical need. We will also be able to comment on the service factors associated with higher proportions of OAT placements and lower patient throughput.
-
Phase 1 was completed on time and the results have been published. 31
2. Each mental health rehabilitation service in England will be eligible for phase 1 and, assuming that they are willing to participate in the study, they will receive an in-depth consultancy report detailing the strengths of their service and areas for improvement. This report will be useful for their local clinical governance and can form the basis for focusing future service improvements.
-
Each service participating in phase 1 received a QuIRC report detailing their performance on the seven domains of quality. The report also gave further details about areas of poorer performance that they may have needed to address.
3. The results from phase 3 will allow us to report on the clinical effectiveness and cost-effectiveness of a relatively brief but complex intervention to improve activity for inpatient rehabilitation service users. If successful, this intervention would be potentially appropriate for implementation in other mental health services. Further manualisation of the intervention developed in phase 2 would then be undertaken to secure its consistency in implementation.
-
Phases 2 and 3 were completed on time. A paper describing the development of the GetREAL intervention has been published. 32 A protocol paper was published prior to the phase 3 data analysis. 78 The main results of the trial and its associated health economic components have been published. 33 The results of the associated qualitative component have also been published. 34 A further paper describing the specific role of the OTs as ‘agents of change’ during the intervention has been published. 35
4. The results from phase 4 will allow us to identify those service users most likely to do well and those components of care most useful in successful mental health rehabilitation. This will allow service planners, commissioners and practitioners to focus their efforts and resources on service users with the greatest needs and with greatest clinical efficiency and cost-efficiency.
-
Phase 4 was completed on time and the main results of the cohort study and the associated health economic component have been published. 36 An additional component of the programme, a realistic evaluation, was carried out, the results of which have been submitted for publication. 37,38
5. The results from all phases of the project will be submitted for publication in peer-reviewed journals with national and international readership. Agreements about authorship of these papers will be made at the PMG meetings.
-
The findings of the REAL programme were published in scientific journals in the mental health field, as described above, to maximise their dissemination nationally and internationally. Wherever possible, we published in open access journals to ensure that the findings are accessible to all those who may be interested, including service users.
6. The results from all phases of the research programme will be presented at relevant scientific, clinical and service user focused conferences and seminars. The research team believe that dissemination of the results of the programme of research is a key deliverable.
-
A project website was set up that was regularly updated by the project manager as the programme progressed to provide a description of the project aims, methods and outputs.
-
A biannual newsletter was sent to all participating services and other relevant organisations throughout the 5-year programme.
-
A dissemination event for all participating services was held on 21 March 2014 at University College London (see Appendix 12) at which the process and results of all phases of the REAL programme were presented. On average, two members of staff from participating units across England attended. In addition to the presentations about each phase of the programme, a summary of the results was given to attendees and sent to all participating units for cascading to the rest of their teams and service users.
-
As well as publishing our findings in scientific journals, our progress during the course of the programme and results from each phase were presented at scientific and non-scientific national and local seminars, meetings and conferences as follows:
-
2015 – Irish College of Psychiatry, winter meeting, Limerick, Ireland. ‘Quality and effectiveness of services for people with complex psychosis.’
-
2015 – European Network for Mental Health Services Research, Biannual Conference, Malaga, Spain. ‘The REAL Study; Rehabilitation Effectiveness for Activities for Life.’
-
2015 – EC Joint Action for Mental Health and Wellbeing. Invited expert seminar, Budapest, Hungary. ‘Transition to community based mental health services: overview from the UK.’
-
2015 – Mental Health Research Network, National Annual Scientific Conference, York. ‘Results from the REAL study; a programme of research in mental health rehabilitation services across England.’
-
2015 – Research seminar, University of Oslo, Norway. ‘Quality and effectiveness of services for people with complex mental health needs.’
-
2014 – World Association of Social Psychiatry and Royal College of Psychiatrists’ Rehabilitation Faculty Joint Conference, London. ‘Quality and effectiveness of services for people with severe and complex psychosis in the UK and Ireland.’
-
2014 – Royal College of Psychiatrists’ International Congress, London. ‘Rehabilitation Effectiveness for Activities for Life: results from national programme of research into mental health rehabilitation services in England.’
-
2014 – Symposium on recovery, Modena University, Italy. Invited plenary keynote. ‘Recovery and rehabilitation.’
-
2014 – COT Annual Conference, Brighton. ‘Rehabilitation Effectiveness for Activities for Life: results from national programme of research into mental health rehabilitation services in England.’
-
2014 – European Psychiatric Association Social Psychiatry Section meeting, Ulm, Germany. Invited keynote plenary. ‘Quality and effectiveness of services for people with complex psychosis.’
-
2014 – South Dublin mental health services, regional Continuing Professional Development event. ‘Delivering mental health rehabilitation services: what do we know about what works?’
-
2013 – Royal College of Psychiatrists’ Faculty of Rehabilitation and Social Psychiatry, annual conference, Birmingham. ‘The REAL Study: Rehabilitation Effectiveness for Activities for Life.’
-
2013 – World Association for Psychosocial Rehabilitation, Asian Section meeting, Lahore, Pakistan. ‘The ongoing need for mental health rehabilitation services.’
-
2013 – ENMESH biannual conference, Verona, Italy. ‘The REAL Study: Rehabilitation Effectiveness for Activities for Life.’
-
2013 – Kent, Surrey and Sussex Partnership NHS Trust, Continuing Professional Development event. ‘Quality and outcomes in mental health rehabilitation services.’
-
2013 – Royal College of Psychiatrists’ International Congress, Edinburgh. ‘Managing complex psychosis.’
-
2013 – World Association for Social Psychiatry, 21st World conference, Lisbon, Portugal. ‘The effectiveness of mental health rehabilitation services for people with longer term and complex needs.’
-
2013 – Clinical Research in Somerset Partnership, research seminar, Somerset. ‘The REAL Study; Rehabilitation Effectiveness for Activities for Life.’
-
2013 – Royal College of Psychiatrists in Northern Ireland, Rehabilitation Faculty meeting, Belfast. ‘The need for specialist mental health rehabilitation services.’
-
2012 – Royal College of Psychiatrists’ West Midlands Division Winter Meeting, Warwick. ‘The need for specialisation in mental health rehabilitation services.’
-
2012 – World Association for Psychosocial Rehabilitation, World Congress, Milan, Italy. Invited plenary. ‘The effectiveness of mental health rehabilitation services.’
-
2012 – New University of Lisbon, Portugal, research meeting. ‘The Rehabilitation Effectiveness for Activities for Life study.’
-
2012 – Camden and Islington NHS Foundation Trust Research Showcase. ‘Quality and effectiveness of mental health services for people with complex needs.’
-
2012 – Coventry and Warwickshire Mental Health Trusts Continuing Professional Development event. ‘Quality and Outcomes in Mental Health Rehabilitation Services.’
-
2011 – North Essex Recovery Vision Event. ‘Mental Health Rehabilitation and Recovery services.’
-
2011 – Basaglia International Summer School, Trieste, Italy. ‘Evaluating Deinstitutionalisation.’
-
2011 – Festschrift for Professor Durk Wiersma, Rob Giel Centre, University Medical Centre Groningen, Netherlands. ‘Contemporary mental health rehabilitation services in the UK.’
-
2011 – East London NHS Foundation Trust. Training event for rehabilitation services. ‘Contemporary mental health rehabilitation services in the UK.’
-
2011 – World Association for Psychosocial Rehabilitation, UK Branch meeting. ‘The renaissance of mental health rehabilitation services.’
-
2011 – Royal College of Psychiatrists in Scotland, Winter meeting. ‘Quality and effectiveness of mental health rehabilitation services.’
-
2010 – University College London Open Event: Showing it works – pathways to better health, London. ‘Quality and effectiveness of mental health rehabilitation services.’
-
2010 – Royal College of Psychiatrists’ Faculty of Rehabilitation and Social Psychiatry, annual conference, Bristol. ‘Improving the evidence base for mental health rehabilitation services.’
-
2010 – London Mental Health Research Network conference, London. ‘Rehabilitation Effectiveness for Activities for Life.’
-
2009 – Royal College of Psychiatrists Faculty of Rehabilitation and Social Psychiatry annual conference, Leeds. ‘Rehabilitation Effectiveness.’
-
Patient and public involvement
The research team consulted with service users with experience of mental health rehabilitation about this research programme through the SURF on four occasions, one more than originally proposed in our application. The first consultation was in relation to the design of the study prior to its submission for funding. The second consultation was in relation to the development of the GetREAL intervention in phase 2. The third consultation reported on the progress of the research programme and the fourth consultation reported the findings.
MA was a coinvestigator on the REAL study team and a member of the North London SURF. MA attended all Programme Management Meetings and contributed to discussions about the design and progress of the programme. MA and another member of the SURF (KB) also participated in the programme as service user expert members of the GetREAL teams in phase 3. Both have direct experience of mental health rehabilitation services and were able to give particularly helpful suggestions about the design and delivery of the intervention in phases 2 and 3. MA and KB presented their experiences of their involvement in the research programme at the Royal College of Psychiatrists’ Faculty of Rehabilitation and Social Psychiatry annual conference in Birmingham in 2013.
There were no particular challenges in managing the patient and public involvement for the REAL programme. Having MA’s and KB’s involvement, along with the ongoing input from the North London SURF, was very constructive and ensured that the programme maintained its focus on researching an area of direct clinical relevance. We are extremely grateful to MA and KB, and the other members of the SURF, for their valuable contributions to the research programme.
MA has been included as a coauthor on relevant publications from the study and KB has been acknowledged for her contribution in the relevant phase 3 publications.
Use of resources
The programme was delivered to budget with some initial underspend accounted for by:
-
The 3-month time lag between the award date (1 January 2009) and project start-up. The NIHR kindly approved a 3-month no-cost extension.
-
The employment costs of the GetREAL team members being less than estimated in the original budget owing to the discrepancy between actual and estimated salary ranges. In addition, one member was employed 2 months later than the other three.
In 2013, an application was made to the NIHR for a 12-month no-cost extension to carry out a realistic evaluation of the processes and results of the cluster randomised controlled trial carried out in phase 3 in order to understand better the specific components of the GetREAL intervention that require refinement to potentially improve its effectiveness. This no-cost extension was approved on 11 April 2014.
Acknowledgements
The REAL study was funded by the NIHR Programme Grants for Applied Research programme. See www.ucl.ac.uk/REAL for further information.
We thank our funders, the fundholders, Camden and Islington NHS Foundation Trust, and the staff, participants and local collaborators at each site for their support with the REAL study.
Contributions of authors
Helen Killaspy (Professor of Rehabilitation Psychiatry and Chief Investigator), Michael King (Professor of Primary Care Psychiatry), Frank Holloway (Emeritus Consultant Psychiatrist), Thomas J Craig (Professor of Social Psychiatry) and Maurice Arbuthnott (service user) conceived the original ideas for the research programme.
Sarah Cook (Senior Researcher in Occupational Therapy) and Tim Mundy (Senior Lecturer in Organisational Psychology) led the development of the staff training intervention.
Gerard Leavey (Professor of Social Science) led the qualitative components in phases 1 and 3.
Paul McCrone (Professor of Health Economics) led the health economics components in phases 1, 3 and 4.
Leonardo Koeser (Research Worker, Health Economics) carried out the health economic analyses under the supervision of Paul McCrone.
Rumana Omar (Professor of Biostatistics) led the statistical analysis in phases 1, 3 and 4.
Louise Marston (Statistician) carried out all quantitative data analyses in phases 1, 3 and 4 under the supervision of Rumana Omar.
Nicholas Green (Research Associate) and Isobel Harrison (Research Associate) collected all the quantitative data for all phases of the programme and the qualitative data in phase 1. Nicholas Green carried out the qualitative data analysis in phase 1 under the supervision of Gerard Leavey.
Melanie Lean (Project Manager) carried out study management activities for all phases of the programme and conducted the qualitative interviews in phase 3. She also carried out the qualitative data analysis in phase 3 under the supervision of Gerard Leavey.
Melanie Gee (Information Scientist) and Sadiq Bhanbhro (Research Fellow) carried out the realist evaluation under the supervision of Sarah Cook.
Helen Killaspy led the writing of the final report, with contributions from all coauthors.
All authors approved the final version of the report.
Publications
Killaspy H, Marston L, Omar RZ, Green N, Harrison I, Lean M, et al. Service quality and clinical outcomes: an example from mental health rehabilitation services in England. Br J Psychiatry 2013;202:28–34.
Brian H, Cook S, Taylor D, Freeman L, Mundy T, Killaspy H. Occupational therapists as change agents in multidisciplinary teams. Br J Occup Ther 2015;1:1–9.
Killaspy H, Marston L, Green N, Harrison I, Lean M, Cook S, et al. Clinical effectiveness of a staff training intervention in mental health inpatient rehabilitation units designed to increase patients’ engagement in activities (the Rehabilitation Effectiveness for Activities for Life [REAL] study): single-blind, cluster-randomised controlled trial. Lancet Psychiat 2015;2:38–48.
Lean M, Leavey G, Killaspy H, Green N, Harrison I, Cook S, et al. Barriers to the implementation and sustainability of an intervention designed to improve patient engagement within NHS mental health rehabilitation units: a qualitative study nested within a randomised controlled trial. BMC Psychiat 2015;15:209.
Bhanbro S, Gee M, Cook S, Marston L, Lean M, Killaspy H. Recovery-based training intervention within mental health rehabilitation units: a realistic evaluation. BMC Psychiatry 2016;16:292.
Cook S, Mundy T, Killaspy H, Taylor D, Freeman L, Craig T, et al. Development of a staff training intervention for inpatient mental health rehabilitation units to increase service users’ engagement in activities. Br J Occup Ther 2016;79:144–52.
Killaspy H, Marston L, Green N, Harrison I, Lean M, Holloway F, et al. Clinical outcomes and costs for people with complex psychosis: a naturalistic prospective cohort study of mental health rehabilitation service users in England. BMC Psychiatry 2016;16:95.
Gee M, Bhanbro S, Cook S, Killaspy H. Rapid realist review of the evidence: achieving lasting change when mental health rehabilitation staff undertake recovery-oriented training [published online ahead of print 10 December 2016]. J Adv Nurs 2016.
Data sharing statement
All data supporting our findings will be shared on request.
Disclaimers
This report presents independent research funded by the National Institute for Health Research (NIHR). The views and opinions expressed by authors in this publication are those of the authors and do not necessarily reflect those of the NHS, the NIHR, CCF, NETSCC, PGfAR or the Department of Health. If there are verbatim quotations included in this publication the views and opinions expressed by the interviewees are those of the interviewees and do not necessarily reflect those of the authors, those of the NHS, the NIHR, NETSCC, the PGfAR programme or the Department of Health.
References
- Killaspy H, Rambarran D, Bledin K. Mental health needs of clients of rehabilitation services: a survey in one trust. J Ment Health 2008;17:207-18. https://doi.org/10.1080/09638230701506275.
- National Service Framework for Mental Health. London: HMSO; 1999.
- Glover G, Arts G, Babu KS. Crisis resolution/home treatment teams and psychiatric admission rates in England. Br J Psychiatry 2006;189:441-5. http://dx.doi.org/10.1192/bjp.bp.105.020362.
- Killaspy H, Bebbington P, Blizard R, Johnson S, Nolan F, Pilling S, et al. The REACT study: randomised evaluation of assertive community treatment in north London. BMJ 2006;332:815-20. http://dx.doi.org/10.1136/bmj.38773.518322.7C.
- Craig TK, Garety P, Power P, Rahaman N, Colbert S, Fornells-Ambrojo M, et al. The Lambeth Early Onset (LEO) Team: randomised controlled trial of the effectiveness of specialised care for early psychosis. BMJ 2004;329. http://dx.doi.org/10.1136/bmj.38246.594873.7C.
- Holloway F. The Forgotten Need for Rehabilitation in Contemporary Mental Health Services. A Position Statement from the Executive Committee of the Faculty of Rehabilitation and Social Psychiatry. London: Royal College of Psychiatrists; 2005.
- Schizophrenia: Core Interventions in the Treatment and Management of Schizophrenia in Adults in Primary and Secondary Care. London: NICE; 2009.
- Meltzer HY. Treatment-resistant schizophrenia – the role of clozapine. Curr Med Res Opin 1997;14:1-20. http://dx.doi.org/10.1185/03007999709113338.
- Green MF. What are the functional consequences of neurocognitive deficits in schizophrenia?. Am J Psychiatry 1996;153:321-30. http://dx.doi.org/10.1176/ajp.153.3.321.
- Wykes T, Dunn G. Cognitive deficit and the prediction of rehabilitation success in a chronic psychiatric group. Psychol Med 1992;22:389-98. https://doi.org/10.1017/S0033291700030336.
- Wykes T, Katz R, Sturt E, Hemsley D. Abnormalities of response processing in a chronic psychiatric group. A possible predictor of failure in rehabilitation programmes?. Br J Psychiatry 1992;160:244-52. https://doi.org/10.1192/bjp.160.2.244.
- Mountain D, Killaspy H, Holloway F. Mental health rehabilitation services in the UK in 2007. Psychiat Bull 2009;33:215-18. https://doi.org/10.1192/pb.bp.107.018697.
- Priebe S, Turner T. Reinstitutionalisation in mental health care. BMJ 2003;326:175-6. https://doi.org/10.1136/bmj.326.7382.175.
- Poole R, Ryan T, Pearsall A. The NHS, the private sector, and the virtual asylum. BMJ 2002;325:349-50. https://doi.org/10.1136/bmj.325.7360.349.
- Ryan T, Pearsall A, Hatfield B, Poole R. Long term care for serious mental illness outside the NHS: a study of out of area placements. J Ment Health 2004;13:425-9. https://doi.org/10.1080/09638230410001729861.
- Ryan T, Hatfield B, Sharma I, Simpson V, McIntyre A. A census study of independent mental health sector usage across seven strategic health authorities. J Ment Health 2007;16:243-53. https://doi.org/10.1080/09638230701279824.
- Killaspy H, Meier R. A fair deal for mental health rehabilitation services. Psychiat Bull 2010;34:265-7. https://doi.org/10.1192/pb.bp.109.028530.
- 2009/10 National Survey of Investment in Adult Mental Health Services. London: Mental Health Strategies; 2010.
- Killaspy H, Harden C, Holloway F, King M. What do mental health rehabilitation services do and what are they for? A national survey in England. J Ment Health 2005;14:157-65. https://doi.org/10.1080/09638230500060144.
- Wing JK, Brown GW. Institutionalism and Schizophrenia. Cambridge: Cambridge University Press; 1970.
- Curson DA, Pantelis C, Ward J, Barnes TR. Institutionalism and schizophrenia 30 years on. Clinical poverty and the social environment in three British mental hospitals in 1960 compared with a fourth in 1990. Br J Psychiatry 1992;160:230-41. https://doi.org/10.1192/bjp.160.2.230.
- Oka M, Otsuka K, Yokoyama N, Mintz J, Hoshino K, Niwa S, et al. An evaluation of a hybrid occupational therapy and supported employment program in Japan for persons with schizophrenia. Am J Occup Ther 2004;58:466-75. https://doi.org/10.5014/ajot.58.4.466.
- Cook S, Howe A. Engaging people with enduring psychotic conditions in primary mental health care and occupational therapy. Br J Occup Ther 2003;66:236-46. https://doi.org/10.1177/030802260306600602.
- Buchain PC, Vizzotto AD, Henna Neto J, Elkis H. Randomized controlled trial of occupational therapy in patients with treatment-resistant schizophrenia. Rev Bras Psiquiatr 2003;25:26-30. https://doi.org/10.1590/S1516-44462003000100006.
- Patient Social Engagement and Attendance at Organised Activities on Acute Psychiatric Wards within the South London and Maudsley Trust: An Observational Study and Audit of 16 Acute Wards. London: South London and Maudsley NHS Foundation Trust Report; 2004.
- Shimitras L, Fossey E, Harvey C. Time use of people living with schizophrenia in a North London catchment area. Br J Occup Ther 2003;66:46-54. https://doi.org/10.1177/030802260306600202.
- Minato M, Zemke R. Time use of people with schizophrenia living in the community. Occup Ther Int 2004;11:177-91. https://doi.org/10.1002/oti.205.
- Bejerholm U, Eklund M. Time use and occupational performance among persons with schizophrenia. Occup Ther Ment Health 2004;20:27-46. https://doi.org/10.1300/J004v20n01_02.
- Mental Health and Social Exclusion. London: Social Exclusion Unit; 2004.
- Cook S, Birrell M. Defining and occupational therapy intervention for people with psychosis. Br J Occup Ther 2007;70:96-106. https://doi.org/10.1177/030802260707000302.
- Killaspy H, Marston L, Omar RZ, Green N, Harrison I, Lean M, et al. Service quality and clinical outcomes: an example from mental health rehabilitation services in England. Br J Psychiatry 2013;202:28-34. http://dx.doi.org/10.1192/bjp.bp.112.114421.
- Cook S, Mundy T, Killaspy H, Taylor D, Freeman L, Craig T, et al. Development of a staff training intervention for inpatient mental health rehabilitation units to increase service users’ engagement in activities. Br J Occup Ther 2016;79:144-52. https://doi.org/10.1177/0308022615600175.
- Killaspy H, Marston L, Green N, Harrison I, Lean M, Cook S, et al. Clinical effectiveness of a staff training intervention in mental health inpatient rehabilitation units designed to increase patients’ engagement in activities (the Rehabilitation Effectiveness for Activities for Life [REAL] study): single-blind, cluster-randomised controlled trial. Lancet Psychiat 2015;2:38-4. https://doi.org/10.1016/S2215-0366(14)00050-9.
- Lean M, Leavey G, Killaspy H, Green N, Harrison I, Cook S, et al. Barriers to the implementation and sustainability of an intervention designed to improve patient engagement within NHS mental health rehabilitation units: a qualitative study nested within a randomised controlled trial. BMC Psychiatry 2015;15. https://doi.org/10.1186/s12888-015-0592-9.
- Brian H, Cook S, Taylor D, Freeman L, Mundy T, Killaspy H. Occupational therapists as change agents in multidisciplinary teams. Br J Occup Ther 2015;1:1-9. https://doi.org/10.1177/0308022615586785.
- Killaspy H, Marston L, Green N, Harrison I, Lean M, Holloway F, et al. Clinical outcomes and costs for people with complex psychosis; a naturalistic prospective cohort study of mental health rehabilitation service users in England. BMC Psychiatry 2016;16. http://dx.doi.org/10.1186/s12888-016-0797-6.
- Gee M, Bhanhbro S, Cook S, Killaspy H. Rapid realist review of the evidence: achieving lasting change when mental health rehabilitation staff undertake recovery-oriented training [published online ahead of print 10 December 2016]. J Adv Nurs 2016.
- Bhanbhro S, Gee M, Cook S, Marston L, Lean M, Killaspy H. Recovery-based staff training intervention within mental health rehabilitation units: a realistic evaluation. BMC Psychiatry 2016;16. https://doi.org/10.1186/s12888-016-0999-y.
- Killaspy H, King M, Wright C, White S, McCrone P, Kallert T, et al. Study protocol for the development of a European measure of best practice for people with long term mental health problems in institutional care (DEMoBinc). BMC Psychiatry 2009;9. http://dx.doi.org/10.1186/1471-244X-9-36.
- Taylor T, Killaspy H, Wright C, Turton P, White S, Kallert T, et al. A systematic review of the international published literature relating to quality of institutional care for people with longer term mental health problems. BMC Psychiatry 2009;9. https://doi.org/10.1186/1471-244X-9-55.
- Turton P, Wright C, White S, Killaspy H. DEMoBinc Group . Promoting recovery in long-term institutional mental health care: an international Delphi study. Psychiatry Serv 2010;61:293-9. http://dx.doi.org/10.1176/appi.ps.61.3.293.
- Mental Health Act 2007. London: The Stationery Office; 2007.
- Killaspy H, White S, Wright C, Taylor TL, Turton P, Schützwohl M, et al. The development of the Quality Indicator for Rehabilitative Care (QuIRC): a measure of best practice for facilities for people with longer term mental health problems. BMC Psychiatry 2011;11. http://dx.doi.org/10.1186/1471-244X-11-35.
- Killaspy H, White S, Wright C, Taylor TL, Turton P, Kallert T, et al. Quality of longer term mental health facilities in Europe: validation of the quality indicator for rehabilitative care against service users’ views. PLOS ONE 2012;7. http://dx.doi.org/10.1371/journal.pone.0038070.
- Glover GR, Robin E, Emami J, Arabscheibani GR. A needs index for mental health care. Soc Psychiatry Psychiatr Epidemiol 1998;33:89-96. https://doi.org/10.1007/s001270050027.
- Hatton C, Emerson E, Robertson J, Gregory N, Kessissoglou S, Walsh PN. The Resident Choice Scale: a measure to assess opportunities for self-determination in residential settings. J Intellect Disabil Res 2004;48:103-13. https://doi.org/10.1111/j.1365-2788.2004.00499.x.
- Priebe S, Huxley P, Knight S, Evans S. Application and results of the Manchester Short Assessment of Quality of Life (MANSA). Int J Soc Psychiatry 1999;45:7-12. https://doi.org/10.1177/002076409904500102.
- Webb Y, Clifford P, Fowler V, Morgan C, Hanson M. Comparing patients’ experience of mental health services in England: a five-trust survey. Int J Health Care Qual Assur 2000;13:273-81. https://doi.org/10.1108/09526860010373253.
- Røssberg JI, Friis S. A suggested revision of the Ward Atmosphere Scale. Acta Psychiatr Scand 2003;108:374-80. https://doi.org/10.1034/j.1600-0447.2003.00191.x.
- Jones SH, Thornicroft G, Coffey M, Dunn G. A brief mental health outcome scale-reliability and validity of the Global Assessment of Functioning (GAF). Br J Psychiatry 1995;166:654-9. https://doi.org/10.1192/bjp.166.5.654.
- Hardin JW, Hilbe JM. Generalized Estimating Equations. Boca Raton, FL: Chapman and Hall/CRC Press; 2003.
- Royston P. Multiple imputation of missing values: update of ice. Stata J 2005;5:527-36.
- Wolfson P, Holloway F, Killaspy H. Enabling Recovery for People with Complex Mental Health Needs: A Template for Rehabilitation Services in England. London: Royal College of Psychiatrists; 2009.
- In Sight and in Mind: A Toolkit to Reduce the Use of Out of Area Mental Health Services. London: Royal College of Psychiatrists; 2010.
- Brooker C, Brabban A. Effective training in psychosocial interventions for work with people with serious mental health problems. Ment Health Rev 2006;11:7-14. https://doi.org/10.1108/13619322200600014.
- Michie S, Pilling S, Garety P, Whitty P, Eccles MP, Johnston M, et al. Difficulties implementing a mental health guideline: an exploratory investigation using psychological theory. Implement Sci 2007;2. http://dx.doi.org/10.1186/1748-5908-2-8.
- Berry K, Haddock G. The implementation of the NICE guidelines for schizophrenia: barriers to the implementation of psychological interventions and recommendations for the future. Psychol Psychother 2008;81:419-36. http://dx.doi.org/10.1348/147608308X329540.
- Curtis L. Unit Costs of Health and Social Care 2013. Canterbury: Personal Social Services Research Unit, University of Kent; 2013.
- Beecham J, Knapp M, Thornicroft G. Measuring Mental Health Needs. London: Gaskell; 2001.
- Jones J, Amaddeo F, Barbui C, Tansella M. Predicting costs of mental health care: a critical literature review. Psychol Med 2007;37:467-77. http://dx.doi.org/10.1017/S0033291706009676.
- Bowers L, Flood C. Nurse staffing, bed numbers and the cost of acute psychiatric inpatient care in England. J Psychiatr Ment Health Nurs 2008;15:630-7. http://dx.doi.org/10.1111/j.1365-2850.2008.01280.x.
- McKechnie AA, Rae D, May J. A comparison of in-patient costs of treatment and care in a Scottish psychiatric hospital. Br J Psychiatry 1982;141:602-7. https://doi.org/10.1192/bjp.141.6.602.
- Coleman JC, Paul GL. Relationship between staffing ratios and effectiveness of inpatient psychiatric units. Psychiatr Serv 2001;52:1374-9. http://dx.doi.org/10.1176/appi.ps.52.10.1374.
- West MA, Slater JA. The Effectiveness of Team Working in Primary Health Care. London: Health Education Authority; 1996.
- Johnson S, Osborn DP, Araya R, Wearn E, Paul M, Stafford M, et al. Morale in the English mental health workforce: questionnaire survey. Br J Psychiatry 2012;201:239-46. http://dx.doi.org/10.1192/bjp.bp.111.098970.
- Roberts G, Wolfson P. The rediscovery of recovery: open to all. Adv Psychiat Treat 2004;10:37-9. https://doi.org/10.1192/apt.10.1.37.
- Guidance for Commissioners of Rehabilitation Services For People With Complex Mental Health Problems. London: Royal College of Psychiatrists; 2013.
- Green LW, Eriksen MP, Schor EL. Preventive practices by physicians: behavioral determinants and potential interventions. Am J Prev Med 1988;4:101-7.
- Doumit G, Gattellari M, Grimshaw J, O’Brien MA. Local opinion leaders: effects on professional practice and health care outcomes. Cochrane Database Syst Rev 2007;1.
- O’Brien MA, Freemantle N, Oxman AD, Wolf F, Davis DA, Herrin J. Continuing education meetings and workshops: effects on professional practice and health care outcomes. Cochrane Database Syst Rev 2001;1. https://doi.org/10.1002/14651858.cd003030.
- Jamtvedt G, Young JM, Kristoffersen DT, O’Brien MA, Oxman AD. Audit and feedback: effects on professional practice and health care outcomes. Cochrane Database Syst Rev 2006;2. https://doi.org/10.1002/14651858.cd000259.pub2.
- Miller WR, Rollnick S. Motivational Interviewing: Preparing People for Change. New York, NY: Guilford Publications; 2002.
- Prochaska JO, Norcross JC. Stages of change. Psychother 2001;38:443-8. https://doi.org/10.1037/0033-3204.38.4.443.
- Jolley S, Garety PA, Ellett L, Kuipers E, Freeman D, Bebbington PE, et al. A validation of a new measure of activity in psychosis. Schizophr Res 2006;85:288-95. http://dx.doi.org/10.1016/j.schres.2006.03.012.
- Parker G, Rosen A, Emdur N, Hadzi-Pavlov D. The Life Skills Profile: psychometric properties of a measure assessing function and disability in schizophrenia. Acta Psychiatr Scand 1991;83:145-52. https://doi.org/10.1111/j.1600-0447.1991.tb07381.x.
- Drake RE, Mueser KT, McHugo GJ, Sederer LI, Dickey B. Outcomes Assessment in Clinical Practice. Baltimore, MD: Williams & Wilkins; 1996.
- Trieman N, Leff J. Difficult to place patients in a psychiatric hospital closure programme: the TAPs project 24. Psychol Med 1996;26:765-74. https://doi.org/10.1017/S0033291700037788.
- Killaspy H, Cook S, Mundy T, Craig T, Holloway F, Leavey G, et al. Study protocol: cluster randomised controlled trial to assess the clinical and cost effectiveness of a staff training intervention in inpatient mental health rehabilitation units in increasing service users’ engagement in activities. BMC Psychiatry 2013;13. https://doi.org/10.1186/1471-244X-13-216.
- Schulz KF, Altman DG, Moher D. CONSORT Group . CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials. Brit Med J 2010;340. https://doi.org/10.1136/bmj.c332.
- Campbell MK, Elbourne DR, Altman DG. CONSORT group . CONSORT statement: extension to cluster randomised trials. BMJ 2004;328:702-8. http://dx.doi.org/10.1136/bmj.328.7441.702.
- Carpenter JR, Goldstein H, Kenward MG. REALCOM-IMPUTE software for multilevel multiple imputation with mixed response types. J Stat Softw 2011;45. https://doi.org/10.18637/jss.v045.i05.
- Birkmann JC, Sperduto JS, Smith RC, Gill KJ. A collaborative rehabilitation approach to the improvement of inpatient treatment for persons with a psychiatric disability. Psychiatr Rehabil J 2006;29:157-65. https://doi.org/10.2975/29.2006.157.165.
- Jolley S, Garety P, Dunn G, White J, Aitken M, Challacombe F, et al. A pilot validation study of a new measure of activity in psychosis. Soc Psych Psych Epid 2005;40:905-11. https://doi.org/10.1007/s00127-005-0982-x.
- Killaspy H, Cardoso G, White S, Wright C, Caldas de Almeida JM, Turton P, et al. Quality of care and its determinants in longer term mental health facilities across Europe. BMC Psychiatry 2016;16. https://doi.org/10.1186/s12888-016-0737-5.
- Sabes-Figuera R, McCrone P, Sharac J, Csipke E, Craig TK, Rose DS, et al. Developing a tool for collecting and costing activity data on psychiatric inpatient wards. Epidemiol Psychiatr Sci 2012;21:393-9. http://dx.doi.org/10.1017/S2045796012000236.
- Bray J. An ethnographic study of psychiatric nursing. J Psychiatr Ment Health Nurs 1999;6:297-305. https://doi.org/10.1046/j.1365-2850.1999.00215.x.
- Michael SP. Invisible skills: how recognition and value need to be given to the ‘invisible skills’ frequently used by mental health nurses, but often unrecognized by those unfamiliar with mental health nursing. J Psychiatr Ment Health Nurs 1994;1:56-7. https://doi.org/10.1111/j.1365-2850.1994.tb00019.x.
- Totman J, Hundt GL, Wearn E, Paul M, Johnson S. Factors affecting staff morale on inpatient mental health wards in England: a qualitative investigation. BMC Psychiatry 2011;11. http://dx.doi.org/10.1186/1471-244X-11-68.
- Johnson JV, Hall EM. Job strain, work place social support, and cardiovascular disease: a cross-sectional study of a random sample of the Swedish working population. Am J Public Health 1988;78:1336-42. https://doi.org/10.2105/AJPH.78.10.1336.
- Karasek R. Job demands, job decision latitude, and mental strain: implications for job redesign. Adm Sci Q 1979;24:285-308. https://doi.org/10.2307/2392498.
- Van der Doef M, Maes S. The Job Demand-Control (-Support) Model and psychological well-being: a review of 20 years of empirical research. Work Stress 1999;13:87-114. https://doi.org/10.1080/026783799296084.
- Tansella M, Thornicroft G. Implementation science: understanding the translation of evidence into practice. Br J Psychiatry 2009;195:283-5. http://dx.doi.org/10.1192/bjp.bp.109.065565.
- Redfern S, Christian S, Norman I. Evaluating change in health care practice: lessons from three studies. J Eval Clin Pract 2003;9:239-49. https://doi.org/10.1046/j.1365-2753.2003.00387.x.
- Harrell FE. Regression Modelling Strategies: With Applications to Linear Models, Logistic Regression, and Survival Analysis. New York, NY: Springer-Verlag; 2001.
- Kurtz MM, Mueser KT. A meta-analysis of controlled research on social skills training for schizophrenia. J Consult Clin Psychol 2008;76:491-504. http://dx.doi.org/10.1037/0022-006X.76.3.491.
- Database of Abstracts of Reviews of Effects (DARE). York: University of York; 2009.
- Management of Schizophrenia. Edinburgh: SIGN; 2013.
- No Health Without Mental Health. A Cross-government Mental Health Outcomes Strategy for People of All Ages. London: HMSO; 2011.
- Shepherd G, Boardman J, Slade M. Making Recovery a Reality. London: Sainsbury Centre for Mental Health; 2008.
- Jauhar S, McKenna PJ, Radua J, Fung E, Salvador R, Laws KR. Cognitive-behavioural therapy for the symptoms of schizophrenia: systematic review and meta-analysis with examination of potential bias. Br J Psychiatry 2014;204:20-9. http://dx.doi.org/10.1192/bjp.bp.112.116285.
- Ng ESW, Grieve R, Carpenter JR. Two-stage nonparametric bootstrap sampling with shrinkage correction for clustered data. Stata J 2013;13:141-64.
- McCrone P, Johnson S, Thornicroft G. Predicting the costs of community care for individuals with severe mental illness in South London. Schizophr Bull 2001;27:653-60. https://doi.org/10.1093/oxfordjournals.schbul.a006904.
- Moore G, Audrey S, Barker M, Bond L, Bonell C, Hardeman W, et al. Process Evaluation Of Complex Interventions: UK Medical Research Council (MRC) Guidance. London: MRC Population Health Science Research Network; 2014.
- Pawson R. Evidence-based Policy: A Realist Perspective. London: Sage; 2006.
- Saul JE, Willis CD, Bitz J, Best A. A time-responsive tool for informing policy making: rapid realist review. Implement Sci 2013;8. http://dx.doi.org/10.1186/1748-5908-8-103.
- Wong G, Greenhalgh T, Westhorp G, Buckingham J, Pawson R. RAMESES publication standards: realist syntheses. BMC Med 2013;11. http://dx.doi.org/10.1186/1741-7015-11-21.
- Checkland P. Soft systems methodology: a thirty year retrospective. Syst Res Behav Sci 2000;17:S11-58. https://doi.org/10.1002/1099-1743(200011)17:1+<::AID-SRES374>3.0.CO;2-O.
- Markey K, Cochrane PA. Online Training and Practice Manual for ERIC Database Searchers. New York, NY: Syracuse University; 1981.
- Pawson R. Digging for nuggets: how ‘bad’ research can yield ‘good’ evidence. Int J Soc Res Meth 2006;9:127-42. https://doi.org/10.1080/13645570600595314.
- Pawson R. The Science of Evaluation: A Realist Manifesto. London: Sage; 2013.
- Greatorex J, Dexter T. An accessible analytical approach for investigating what happens between the rounds of a Delphi study. J Adv Nurs 2000;32:1016-24.
- Grbevski S. Health Care Professionals’ Attitudes towards Individuals Diagnosed with Severe Mental Illness. Detroit, MI: Wayne State University; 2010.
- Salyers MP, Tsai J, Stultz TA. Measuring recovery orientation in a hospital setting. Psychiatr Rehabil J 2007;31:131-7. https://doi.org/10.2975/31.2.2007.131.137.
- Chen SP. The Development of Recovery Competencies for Inpatient Mental Health Providers Working with People with Serious Mental Illness. Ontario, ON: Queen’s University; 2012.
- Corrigan PW, McCracken SG, Kommana S, Edwards M, Simpatico T. Staff perceptions about barriers to innovative behavioural rehabilitation programs. Cognitive Ther Res 1996;20:541-51. https://doi.org/10.1007/BF02227912.
- Marlowe HA, Spector PE, Bedell JR. Implementing a psychosocial rehabilitation program in a state mental hospital: a case study of organizational change. Psychosoc Rehabil J 1983;6:2-11. https://doi.org/10.1037/h0099738.
- Le Boutillier C, Slade M, Lawrence V, Bird VJ, Chandler R, Farkas M, et al. Competing priorities: staff perspectives on supporting recovery. Adm Policy Ment Health 2015;42:429-38. http://dx.doi.org/10.1007/s10488-014-0585-x.
- Way BB, Stone B, Schwager M, Wagoner D, Bassman R. Effectiveness of the New York State Office of Mental Health Core Curriculum: direct care staff training. Psychiatr Rehabil J 2002;25:398-402. https://doi.org/10.1037/h0094997.
- Bartholomew T, Kensler D. Illness management and recovery in state psychiatric hospitals. Am J Psychiat Rehabil 2010;13:105-25. https://doi.org/10.1080/15487761003756977.
- Strating MM, Broer T, van Rooijen S, Bal RA, Nieboer AP. Quality improvement in long-term mental health: results from four collaboratives. J Psychiatr Ment Health Nurs 2012;19:379-88. http://dx.doi.org/10.1111/j.1365-2850.2011.01802.x.
- Burdett C, Milne D. ‘Setting events’ as determinants of staff behaviour: an exploratory study. Behav Psychother 1985;13:300-8. https://doi.org/10.1017/S0141347300012040.
- Linhorst DM. Implementing psychosocial rehabilitation in long-term inpatient psychiatric facilities. J Ment Health Adm 1995;22:58-67. https://doi.org/10.1007/BF02519198.
- Donat DC, McKeegan GF, Neal B. Training inpatient psychiatric staff in the use of behavioral methods: a program to enhance utilization. Psychosoc Rehabil J 1991;15:69-74. https://doi.org/10.1037/h0095800.
- Corrigan PW, Holmes EP, Luchins D. Identifying staff advocates of behavioral treatment innovations in state psychiatric hospitals. J Behav Ther Exp Psychiatry 1993;24:219-25. https://doi.org/10.1016/0005-7916(93)90024-Q.
- Nancarrow SA, Booth A, Ariss S, Smith T, Enderby P, Roots A. Ten principles of good interdisciplinary team work. Hum Resour Health 2013;11. http://dx.doi.org/10.1186/1478-4491-11-19.
- Pollard L, Gelbard Y, Levy G, Gelkopf M. Examining attitudes, beliefs and knowledge of effective practices in psychiatric rehabilitation in a hospital setting. Psychiatr Rehabil J 2008;32:124-7. http://dx.doi.org/10.2975/32.2.2008.124.127.
- Narevic E, Giles GM, Rajadhyax R, Managuelod E, Monis F, Diamond F. The effects of enhanced program review and staff training on the management of aggression among clients in a long-term neurobehavioral rehabilitation program. Aging Ment Health 2011;15:103-12. https://doi.org/10.1080/13607863.2010.501070.
- Valinejad CA. Model for in-house psychological training. Ment Health Nurs 2001;21:18-21.
- Ahmed AO, Serdarevic M, Mabe PA, Buckley PF. Triumphs and challenges of transforming a state psychiatric hospital in Georgia. Int J Ment Health Prom 2013;15:68-75. https://doi.org/10.1080/14623730.2013.820575.
- Flodgren G, Parmelli E, Doumit G, Gattellari M, O’Brien MA, Grimshaw J, et al. Local opinion leaders: effects on professional practice and health care outcomes. Cochrane Database Syst Rev 2011;8. http://dx.doi.org/10.1002/14651858.CD000125.pub4.
- Vangen MD. Behavior Management Training in Inpatient Psychiatric Rehabilitation: Impact on Ward Atmosphere, Program Operation and Outcome. Lincoln, NE: University of Nebraska; 1991.
- Corrigan PW. Wanted: champions of psychiatric rehabilitation. Am Psychol 1995;50:514-21. https://doi.org/10.1037/0003-066X.50.7.514.
- Aarons GA, Hurlburt M, Horwitz SM. Advancing a conceptual model of evidence-based practice implementation in public service sectors. Adm Policy Ment Health 2011;38:4-23. http://dx.doi.org/10.1007/s10488-010-0327-7.
- Repper J, Perkins R. ImROC Briefing 6: The Team Recovery Implementation Plan: A Framework for Creating Recovery-focused Services. London: Centre for Mental Health and Mental Health Network, NHS Confederation; 2013.
- Gilburt H, Edwards N, Murray R. Transforming Mental Health: A Plan of Action for London. London: The King’s Fund; 2014.
- Drennan G, Wooldridge J. ImROC Briefing 10: Making Recovery a Reality in Forensic Settings. London: Centre for Mental Health and Mental Health Network, NHS Confederation; 2014.
- Cavendish C. The Cavendish Review: An Independent Review into Healthcare Assistants and Support Workers in the NHS and Social Care Settings. London: Department of Health; 2013.
- McCracken SG, Corrigan PW. Staff Development in Mental Health. Using Evidence in Social Work Practice: Behavioural Perspectives. Chicago, IL: Lyceum Books; 2004.
- Meaden A, Hacker D. Problematic and Risk Behaviours in Psychosis: A Shared Formulation Approach. New York, NY: Routledge/Taylor & Francis Group; 2011.
- Ritchie J, Lewis J. Qualitative Research Practice: A Guide for Social Science Students. London: Sage; 2003.
- Beyond Breaking Point? A Survey Report of RCN Members on Health, Wellbeing and Stress. London: Royal College of Nursing; 2013.
- Eklund M, Gunnarsson AB, Sandlund M, Leufstadius C. Effectiveness of an intervention to improve day centre services for people with psychiatric disabilities. Aust Occup Ther J 2014;61:268-75. http://dx.doi.org/10.1111/1440-1630.12120.
- Rethinking Risk to Others in Mental Health Services: College Report CR150. London: Royal College of Psychiatrists; 2008.
- Boardman J, Roberts G. ImROC Briefing 9: Risk, Safety and Recovery. London: Centre for Mental Health and Mental Health Network, NHS Confederation; 2014.
- Shepherd G, Boardman J, Burns M. Implementing Recovery: A Methodology for Organisational Change. London: Sainsbury Centre for Mental Health; 2010.
- Fuoco FJ, Christian WP. Behavior Analysis and Therapy in Residential Programs. New York, NY: Van Nostrand Reinhold; 1986.
- Johnstone PL. Mixed methods, mixed methodology health services research in practice. Qual Health Res 2004;14:259-71. http://dx.doi.org/10.1177/1049732303260610.
- Hedeker D, Gibbons RD, Flay BR. Random-effects regression models for clustered data with an example from smoking prevention research. J Consult Clin Psychol 1994;62:757-65. https://doi.org/10.1037/0022-006X.62.4.757.
- Hawe P, Shiell A, Riley T. Complex interventions: how ‘out of control’ can a randomised controlled trial be?. BMJ 2004;328:1561-3. http://dx.doi.org/10.1136/bmj.328.7455.1561.
- Bonell C, Fletcher A, Morton M, Lorenc T, Moore L. Realist randomised controlled trials: a new approach to evaluating complex public health interventions. Soc Sci Med 2012;75:2299-306. http://dx.doi.org/10.1016/j.socscimed.2012.08.032.
- Hawkins A. The case for experimental design in realist evaluation. Learn Commun: Int J Learn Soc Contexts 2014;14:46-59. https://doi.org/10.18793/LCJ2014.14.04.
Appendix 1 Phase 1: data analysis plan
Appendix 2 Phase 1: qualitative interview topic guide – staff
Appendix 3 Phase 1: qualitative interview topic guide – service users
Appendix 4 Phase 2: abstract for consultation workshop at College of Occupational Therapy annual conference 2010
Appendix 5 Phase 2: GetREAL fidelity assessment sheet
Appendix 6 GetREAL staff training intervention manual
Appendix 7 Phase 3: certificate of attendance at GetREAL training
Appendix 8 Phase 3: data analysis plan
Appendix 9 Phase 3: focus group topic guide – staff
REAL: topic guide for staff focus groups, phase 3 (v.1 13 January 2009)
Appendix 10 Phase 3: qualitative interview topic guide – service users
REAL: service user participant topic guide for qualitative interviews phase 3 (v.1 13 January 2009)
Appendix 11 Phase 4: data analysis plan
Appendix 12 Dissemination event agenda
REAL study dissemination seminar
Appendix 13 Realist evaluation: terms of reference for the local reference group and expert panel
Project title: Rehabilitation Effectiveness for Activities for Life: Realistic Evaluation.
Summary: The aim of the study is to identify and understand the factors associated with better uptake of the intervention (‘GetREAL’), particularly related to the reinforcing stage, and to investigate the association between uptake of the intervention and outcome. To achieve this aim a case study design using mixed methods that involve analysis of existing qualitative and quantitative data collected during the phase 3 of the REAL project and a review of existing literature using a realist synthesis approach will be applied.
The project team has planned to develop a preliminary ‘middle range’ theory from the phase 3 REAL study results concerning the social behaviour, organisation and change relationships between contexts, mechanisms and outcomes in the intervention units (mental health rehabilitation units). This will be modified by the findings of a comprehensive literature review using realist review approaches. Three inpatient mental health rehabilitation units that took part in the trial during phase 3 of the REAL study will be purposefully selected for case studies in order to explore and challenge the middle range theory. These will include those that had low, mid-range and higher scores in the trial’s primary outcome measure, service user activity as assessed using the time-use diary at the 12-month follow-up.
We are seeking the contribution and feedback from a LRG and EP members on the findings/results from both the realistic evaluation and realist review at different stages.
Expert panel
Definition: this wider group comprises researchers and practitioners who are actively engaged in conducting work in the area under review. They are engaged to ensure that the review is appropriately focused and the evidence is appropriately interpreted to produce results that have a wide validity.
Purpose: the purpose of this group is to ensure that the project findings/results have wider validity/consistency.
Membership to be invited: mental health practitioners, managers and researchers who have expertise in the field; specialists in organisational development, change management and leadership. OTs, psychiatrists, nurses, clinical psychologists and service users will be welcomed.
Contribution
-
Contribute list of most relevant papers/articles that they know about.
-
Consultation/feedback on draft of key findings (programme theories) to ensure that they have wider validity/consistency with their own experiences and to ensure nothing crucial was missed.
-
Attend consultation/feedback half-day session (to be confirmed) to provide feedback.
Terms of references for local reference group
Project Title: Rehabilitation Effectiveness for Activities for Life: Realistic Evaluation.
Summary: the aim of the study is to identify and understand the factors associated with better uptake of the intervention (‘GetREAL’), particularly related to the reinforcing stage, and to investigate the association between uptake of the intervention and outcome. To achieve this aim a case study design using mixed methods that involve analysis of existing qualitative and quantitative data collected during the phase 3 of the REAL project and a review of existing literature using a realist synthesis approach will be applied.
The project team has planned to develop a preliminary ‘middle range’ theory from the phase 3 REAL study results concerning the social behaviour, organisation and change relationships between contexts, mechanisms and outcomes in the intervention units (mental health rehabilitation units). This will be modified by the findings of a comprehensive literature review using realist review approaches. Three inpatient mental health rehabilitation units that took part in the trial during phase 3 of the REAL study will be purposefully selected for case studies in order to explore and challenge the middle range theory. These will include those that had low, mid-range and higher scores in the trial’s primary outcome measure, service user activity as assessed using the time-use diary at the 12-month follow-up.
We are seeking the contribution and feedback from LRG and EP members on the findings/results from both the realistic evaluation and realist review at different stages.
Local reference group
Definition: this small group comprises project stakeholders (typically including representatives from the project funder or commissioners and the target audience). They are engaged to ensure that the review will produce results that are directly relevant to the local context of the project.
Purpose: the purpose of this group is to ensure that the findings/results of the project are relevant to the local context of the project.
Membership to be invited: members of the REAL Steering Group.
Contribution
-
Contribute list of most relevant papers/articles/reports that they know about.
-
Provide feedback on preliminary findings from initial extractions (contextual factors and mechanisms, and patterns) to refine/focus the review scope and research questions. Provide feedback on draft of key findings (programme theories) to ensure that they are relevant to the project team.
-
Attend consultation/feedback half-day session.
Appendix 14 Realist evaluation: GetREAL documents consulted during the rapid realist review
GetREAL staff training intervention manual (see Appendix 6).
Published protocol paper for phase 3. 144
Published paper32 reporting the results phase 3.
Paper in development on OTs as change agents.
Final report (draft).
Discussion notes from 2014 dissemination event.
Conference presentations
2014: COT Annual Conference, Brighton, UK.
2013: Royal College of Psychiatrists’ Faculty of Rehabilitation and Social Psychiatry Annual Conference, Birmingham, UK.
2013: World Association for Social Psychiatry, 21st World Conference, Lisbon, Portugal.
2013: ENMESH Biannual Conference, Verona, Italy.
Appendix 15 Realist evaluation: indicative search strategy
The strategy below is for the databases MEDLINE and Cumulative Index to Nursing and Allied Health Literature Complete searched on 5 September 2014 using the EBSCOhost platform. The strategy below was replicated as closely as possible in the other databases searched.
# | Query | Results |
---|---|---|
S1 | TI ((mental* OR psychiatric) AND (ward* OR unit* OR inpatient* OR ‘in patient*’ OR residential* OR hospital*)) OR AB ((mental* OR psychiatric) AND (ward* OR unit* OR inpatient* OR ‘in patient*’ OR residential* OR hospital*)) | 102,839 |
S2 | (MH ‘Psychiatric Units’) | 1870 |
S3 | S1 OR S2 | 103,696 |
S4 | TI (rehabilitat* OR recovery) OR AB (rehabilitat* OR recovery) | 494,467 |
S5 | (MH ‘Rehabilitation, Psychosocial+’) | 3953 |
S6 | S4 OR S5 | 496,944 |
S7 | S3 AND S6 | 6822 |
S8 | TI training OR AB training | 334,279 |
S9 | TI staff n3 develop* OR AB staff n3 develop* | 6020 |
S10 | TI staff n3 learning OR AB staff n3 learning | 763 |
S11 | TI workplace n3 learning OR AB workplace n3 learning | 492 |
S12 | TI ‘research implementation’ OR AB ‘research implementation’ | 265 |
S13 | TI ‘situated learning’ OR AB ‘situated learning’ | 133 |
S14 | TI ‘peer based learning’ OR AB ‘peer based learning’ | 1 |
S15 | TI ‘hands on learning’ OR AB ‘hands on learning’ | 159 |
S16 | TI ‘action learning’ OR AB ‘action learning’ | 334 |
S17 | TI ‘learning sets’ OR AB ‘learning sets’ | 174 |
S18 | TI ‘workbased learning’ OR AB ‘workbased learning’ | 3 |
S19 | TI ‘work based learning’ OR AB ‘work based learning’ | 308 |
S20 | TI ‘practice development’ OR AB ‘practice development’ | 1645 |
S21 | TI ‘inhouse learning’ OR AB ‘inhouse learning’ | 0 |
S22 | TI ‘in house learning’ OR AB ‘in house learning’ | 2 |
S23 | (MH ‘Inservice Training+’) | 24,058 |
S24 | S8 OR S9 OR S10 OR S11 OR S12 | 340,268 |
S25 | S13 OR S14 OR S15 OR S16 OR S17 | 755 |
S26 | S18 OR S19 OR S20 OR S21 OR S22 OR S23 | 25,888 |
S27 | S24 OR S25 OR S26 | 358,917 |
S28 | S7 AND S27 | 575 |
Appendix 16 Realist evaluation: questionnaire sent to local reference group for theory prioritisation
Appendix 17 Realist evaluation: papers that informed non-prioritised candidate programme theories
Centre for Mental Health. Implementing Recovery. URL: www.centreformentalhealth.org.uk/recovery/implementing_recovery.aspx (accessed 11 May 2014).
Corrigan PW. Differences between clinical and nursing inpatient staff: implications for training in behavioral rehabilitation. J Behav Ther Exp Psychiatry 1994;25:311–16.
Corrigan et al. 115
Corrigan PW, McCracken SG, Edwards M, Kommana S, Simpatico T. Staff training to improve implementation and impact of behavioral rehabilitation programs. Psychiatr Serv 1997;48:1336–8.
Corrigan P, McCracken S. Psychiatric rehabilitation and staff development: educational and organizational models. Clin Psychol Rev 1995;15:699–719.
Davies J, Maggs RG, Lewis R. The development of a UK low secure service: philosophy, training, supervision and evaluation. Int J Forensic Ment Health 2010;9:334–42.
Eklund et al. 142
Geczy B, Sultenfuss JF. Contributions of psychologists to inpatient care of persons with chronic mental illness. Hosp Community Psychiatry 1994;45:54–7.
Grbevski. 112
Hungerford C, Kench P. The perceptions of health professionals of the implementation of recovery-oriented health services: a case study analysis. J Ment Health Training Educ Practice 2013;8:208–18.
Krupa T, Eastabrook S, Blake P, Goering P. Lessons learned: Introducing psychiatric rehabilitation in a multidisciplinary hospital setting. Psychosoc Rehabil J 1992;15:29–36.
Luxford K, Safran DG, Delbanco T. Promoting patient-centered care: a qualitative study of facilitators and barriers in healthcare organizations with a reputation for improving the patient experience. Int J Qual Health Care 2011;23:510–15.
O’Callaghan MAJ. Training the trainers (and introducing change): practical aspects of establishing and running a training system in mental health rehabilitation. J Ment Health 1992;1:141–7.
Perkins R, Repper J, Rinaldi M, Brown H. Recovery Colleges. Briefing Paper. London: Centre for Mental Health; 2012.
Plsek PE, Wilson T. Complexity, leadership, and management in healthcare organisations. BMJ 2001;323:746–9.
Pollard et al. 126
Pratt CW, Smith RC, Kazmi A, Ahmed S. Introducing psychiatric rehabilitation at a psychiatric facility in Pakistan. Am J Psychiatr Rehabil 2011;14:259–71.
Ramon S. Organisational change in the context of recovery-oriented services. J Ment Health Training Educ Practice 2011;6:38–46.
Salyers et al. 113
Schenkel LS. Characteristics of Staff–Patient Interactions on a Behavioural Treatment Unit for Patients with Serious Mental Illness, and the Effects of a Behaviour Management Training Program on Staff Performance. Lincoln, NE: University of Nebraska; 2006.
Shepherd et al. 145
The King’s Fund. Leadership and Engagement for Improvement in the NHS: Together We Can. London: The King’s Fund; 2012.
Tsai J, Salyers MP, Lobb AL. Recovery-oriented training and staff attitudes over time in two state hospitals. Psychiatr Q 2010;81:335–47.
Appendix 18 Realist evaluation: the 50 candidate programme theories for long-term change in increasing recovery-based practice
The candidate programme theories are organised by mechanism; thus, under a statement of each of the seven postulated mechanisms leading to long-term change in increasing recovery-based practice, is a statement of each of the candidate programme theories that relates to that mechanism.
Recovery is important
When staff groups of a mental health inpatient rehabilitation unit have taken part in a training programme aimed at increasing their engagement with recovery-based practice, they will make long-term changes and increase their engagement with recovery-based practice if they feel that recovery is important to them individually and to the organisation.
RI1: management endorsement and prioritisation
If the management team actively endorses and prioritises the programme, supports the staff and encourages change (e.g. gets involved, endorses the action plan, quantifies progress and incorporates external drivers), the staff will feel that recovery is important to the organisation.
RI2: performance linked to service user feedback
If the performance of the unit as a whole or of individual staff members is linked to service user feedback (either verbal or behavioural), or some other measure of patient-focused care and recovery, the staff will feel that recovery is important to the unit/organisation and to them individually.
RI3: training fits job description/continuing professional development requirements
If the training/change programme is consistent with the job descriptions of staff members or continuing professional development requirements of any professional bodies that the staff belong to, the staff members are likely to consider the training to be of professional importance.
RI4: consistent with unit mission
If the move towards greater recovery-based practice is consistent with the stated mission of the unit or the wider organisation, staff members are likely to perceive it as important to the organisation.
RI5: staff identify need for change
If the desirability of the move towards increased recovery-based practice has been identified by the staff members themselves (e.g. through the training programme), they will automatically feel that recovery is important.
RI6: training is repeated/refreshed
If the training is refreshed periodically (and appropriate systems and processes in place, e.g. ‘train the trainer’), new and existing staff members will feel that the change programme is important.
RI7: recent major negative event
If there has been a recent major negative event affecting the unit (e.g. changed location, significant loss of staff, illness or accident affecting the unit atmosphere), dealing with this will be prioritised over a change programme.
Supported change
When staff groups of a mental health inpatient rehabilitation unit have taken part in a training programme aimed at increasing their engagement with recovery-based practice, they will make long-term changes and increase their engagement with recovery-based practice if they feel encouraged/motivated/supported by management and colleagues to change.
SC1: publicly recognise, reward, incentivise programme successes
If the programme successes are shared with the staff group, recognised publicly (e.g. conferences, publications), rewarded or otherwise incentivised, the staff members will feel motivated by management and colleagues to persevere, even those formerly reticent.
SC2: regular supervisions/collaborative meetings
Regular supervisions or collaborative meetings between staff groups and the training team, and/or staff members and a local change lead (‘champion’), when these take place within a supportive organisational culture, will help staff members feel supported by their peers and managers in the change programme.
SC3: change agent/champion role
Appointing individuals from within the staff team to ‘champion’, or act as a ‘change agent’ for the programme, serves to persuade, encourage and empower (i.e. support) other staff members to change. To be effective, those individuals should have optimism, good interpersonal skills, management support and the respect of colleagues, and be influential. Ensuring that the ‘champion’ is associated with a role, rather than a single individual or individuals, ensures long-term continuity: dependence on key enthusiastic individuals runs the risk of staff members losing that support if those individuals leave or move to other departments.
SC4: management support, supported role flexibility
It is important that staff groups feel supported by their management to change. This support can be realised through active management endorsement and prioritisation of the programme, and encouraging change (e.g. getting involved themselves; endorsing an action plan for change; quantifying progress; incorporating external drivers to change). This support will help the staff feel encouraged to change even if increased engagement with recovery entails moving outside their traditional occupational role. Management support of positive risk-taking is also desirable.
SC5: organisation structures, etc., modified to support change
If the management team modifies organisational structures, processes and systems (e.g. working practices, responsibilities, policies, documentation and performance reviews) to facilitate the move towards recovery-based practice, staff members will feel supported by management in changing their practices.
SC6: involvement of relevant professional groups in programme development
If the programme is developed/facilitated by someone external to the unit who does not involve any of the same professionals within the unit, those individuals are likely to feel professionally threatened and unsupported and may disrupt the programme.
Recovery is realistic
When staff groups of a mental health inpatient rehabilitation unit have taken part in a training programme aimed at increasing their engagement with recovery-based practice, they will make long-term changes and increase their engagement with recovery-based practice if they, and the service users, feel that working collaboratively with service users towards recovery is realistic.
RR1: service user involvement in programme design/delivery
Involvement of current or former service users in the design and/or delivery of the training programme will persuade staff that both recovery and collaboration is achievable and realistic for service users, and that collaborative working with service users is achievable.
RR2: peer support workers working with staff
Peer support workers operating in tandem with the staff help to give service users a ‘voice’, and give both staff and service users a sense of hope, optimism and encouragement to work together.
RR3: staff understand recovery is non-linear
When staff members understand that recovery is non-linear they will understand how to respond flexibly, with realistic expectations, rather than becoming demotivated by fluctuations in an individual service users’ mental health status.
RR4: staff find service users with complex needs hard to engage with
When staff find service users with complex needs to be hard to engage with (e.g. owing to medication side effects, physical or mental comorbidities, impaired insight), they may have a pessimistic view about recovery, may feel they do not have the tools/skills/confidence to engage with them and may do ‘for’ rather than ‘with’ the service users.
RR5: service users and staff work together
In an environment (physical and social) on the unit that facilitates service users and staff working together as a ‘community’, and challenges power imbalances or paternalistic attitudes, service users are encouraged to become active agents in their own recovery and care becomes more individualised and patient focused.
RR6: paradigm shift from custodial/protective model to recovery-based care
Some staff members may need to undergo a paradigm shift from a ‘custodial’ or ‘protective’ model of mental health care to recovery-based, less restrictive care. Without this, they will find it hard to treat service users as partners in recovery and service users may feel threatened when faced with the possibility of recovery.
RR7: appropriate medication regime
If medication regimes are selected and regimented according to service users’ own goals, interests and aspirations, then staff and service users will feel that recovery is realistic, rather than adopting a ‘medicalised’ view of service users.
RR8: staff stress, burnout, job satisfaction
Where there is a high prevalence of stress, low job satisfaction and burnout among staff groups, those members of staff affected are more likely to perceive a threat more readily and/or make negative attributions towards the service users (i.e. recovery is not realistic).
RR9: encouraged autonomy, positive risk-taking
In an organisational culture that encourages autonomy and supports positive risk-taking in the pursuit of recovery, staff will feel that they have the autonomy and empowerment to manage risk or act beyond their traditional role descriptions.
RR10: quick wins demonstrate progress
If ‘quick wins’ in change towards increased recovery-based practice are identified, implemented and promoted, the staff groups will feel they have made progress and further change is achievable. Staff who were previously reluctant to engage may be newly motivated to engage with the programme.
Resourced for recovery
When staff groups of a mental health inpatient rehabilitation unit have taken part in a training programme aimed at increasing their engagement with recovery-based practice, they will make long-term changes and increase their engagement with recovery-based practice if they feel they have the resources to do so and/or if barriers (individual, group or organisational) have been removed.
RFR1: strong community and family links
Strong, supportive community and family links need to be in place. When there are poor links to the community (e.g. in rural/isolated units), engagement with recovery will be perceived to be difficult.
RFR2: shift/working pattern flexibility
Staff need sufficient flexibility in their shift/working pattern to enable participation in activities outside their ‘normal’ working day. A lack of flexibility impedes continuity of service user engagement/activities between one member of staff/one week and the next.
RFR3: adequate shift handovers
When adequate time and resources are devoted to shift handovers, incoming staff feel fully appraised about the individual service users’ health states and their recent/ongoing activities, facilitating appropriate patient-centred care.
RFR4: adequate staffing capacity, time, space, resources
Adequate staffing capacity, time and physical space and resources are needed. This may require reducing administrative burdens and other competing work priorities, greater role flexibility between staff and initiatives to free up time to devote on patient-focused care.
RFR5: appropriate medication regimes
Appropriate medication regimes are needed. If the service users’ own goals, aspirations and interests inform the selection and regimentation of medications, the medication regimes are more likely to be consistent with facilitating, rather than impeding, recovery.
RFR6: change built into existing organisational structure, etc.
The change towards recovery needs to be consistent with/can be built into existing organisational structures, processes and systems (e.g. working practices, responsibilities, policies, documentation and performance reviews). This will help staff members to feel that the change will not require a great amount of further effort.
RFR7: positive, collaborative culture between staff and service users
To identify and resolve individual, group and organisational barriers to change, staff and service users need to be involved in developing the programme, within a positive, collaborative culture.
Recovery is everyone’s responsibility
When staff groups of a mental health inpatient rehabilitation unit have taken part in a training programme aimed at increasing their engagement with recovery-based practice, they will make long-term changes and increase their engagement with recovery-based practice if they, and the service users, feel that recovery is everyone’s responsibility: all staff, all service users.
RER1: shared training across all staff groups
Shared training with different staff groups together, in a supportive culture, engenders understanding of different values and philosophies held and improved attitudes to service users. It enhances interstaff group relationships and a sense of shared ownership.
RER2: training ensures shared understanding of recovery
All staff needs a shared understanding of what is meant by recovery, and its relevance to all staff disciplines. Additional training time should be provided for staff who are new to the concepts, using familiar terminology and professional ideology.
RER3: staff groups reflect together
Providing opportunities for different staff groups to reflect together, obtain feedback, monitor their progress and identify areas for further change helps staff feel that recovery is a shared responsibility.
RER4: administrative burdens, competing work priorities
Administrative burdens and other competing work priorities may make some staff groups feel that recovery (being harder to quantify) is not a priority for them, especially in a culture of role inflexibility, a lack of common understanding and co-operation, and job insecurity.
RER5: action plan developed collaboratively
A clearly articulated action plan that is regularly referenced to and updated is developed with service users and builds on strengths and experiences in an organisational culture of trust and consensus will foster a common vision, effective collaboration and allow staff to challenge existing work practices.
RER6: role flexibility, common understanding and support between staff groups
A unit culture of role inflexibility and/or a lack of common understanding and co-operation between the different staff groups, staff will protect their role boundaries and resist recovery-focused role extension.
Reinforced direction
When staff groups of a mental health inpatient rehabilitation unit have taken part in a training programme aimed at increasing their engagement with recovery-based practice, they will make long-term changes and increase their engagement with recovery-based practice if they know exactly what is expected of them, and this clear direction is continually reinforced.
RD1: new staff activities reflected in organisational structures
The (new) activities expected of staff should be reflected in organisational structures, processes and systems (e.g. working practices, responsibilities, policies, documentation and performance reviews).
RD2: training is practical and specific
If the training programme is both practical (‘hands-on’) and specific (rather than generalised/inspirational), modelling desirable behaviour, staff will know what to do and will have the tools to do it.
RD3: training is repeated/refreshed
If the training is repeated and refreshed periodically (and appropriate systems and processes are in place, e.g. ‘train the trainer’), existing and new staff members will be reminded about what is expected of them.
RD4: clearly articulated action plan
The existence (and regular reference to/updating of) a clearly articulated action plan, developed collaboratively with service users, will provide clarity.
RD5: regular supervisions
Regular supervisions between staff groups and the training team, and/or staff members together with a local change lead, encourages reflection on and understanding of the change process.
Receptive staff
When staff groups of a mental health inpatient rehabilitation unit have taken part in a training programme aimed at increasing their engagement with recovery-based practice, they will be receptive to making long-term changes and increase their engagement with recovery-based practice if they feel involved, valued, enthusiastic and engaged in the programme.
RS1: action plan developed collaboratively
Staff will feel engaged, valued and involved (and hence receptive to change) if an action plan for change has been developed collaboratively between different staff groups and service users. If the action plan utilises existing strengths and experiences within the team and with service users, this will also contribute to the sense of feeling valued. Conversely, if an action plan is imposed on staff members, they will be unlikely to feel engaged, valued and involved.
RS2: high job satisfaction, low burnout
When the staff have high levels of job satisfaction and low burnout, they are likely to be engaged and motivated by the change programme, fostered by supportive organisations/colleagues and collaboration.
RS3: incorporate recovery into existing change programme
In an organisation that has undergone much recent change, staff members may feel, at best, unenthusiastic about an additional training/change programme to increase recovery-based practice or, at worst, unenthusiastic, disillusioned or downright pessimistic. Incorporating recovery into an existing change programme may help with staff engagement and enthusiasm.
RS4: climate of job uncertainty, fear
In an organisation that has a climate of uncertainty and fear (e.g. in a context of economic cutbacks and job losses), it is likely that staff members will feel disillusioned, and uninterested or pessimistic in/about a change programme. We propose that this negative organisational context therefore blocks the mechanism RS: staff will be unlikely to feel involved, engaged or valued.
RS5: all staff receive training
In an organisation lacking a culture of mutual support between and within different staff groups, and where only some staff members receive training, others may feel threatened by their own relative lack of ‘expertise’ and react defensively or resistively to the change efforts.
RS6: programme is part of research project
If the programme is part of a research project with positive collaboration between the unit/organisation and an academic body, staff members are likely to feel motivated and enthusiastic about being part of a ‘scientific’ process.
RS7: programme is tailored to staff group
If the programme has been tailored to the staff group, and its existing systems, processes and cultures, then the staff members are likely to feel that their experiences and opinions are valued.
Appendix 19 Realist evaluation: local reference group votes for the ‘top three’ candidate programme theories under each mechanism
Theory by mechanism | Number of votes in top 3a | Reviewer ranking |
---|---|---|
RI | ||
RI1: management endorsement and prioritisation | 5 | Top RI theory |
RI2: performance linked to service user feedback | 4 | |
RI3: training fits job description/continuing professional development requirements | 4 | |
RI4: consistent with unit mission | 3 | |
RI5: staff identify need for change | 0 | |
RI6: training is repeated/refreshed | 1 | |
RI7: recent major negative event | 1 | |
RI8: action planning | 0 | |
SC | ||
SC1: publicly recognise, reward, incentivise programme successes | 1 | |
SC2: regular supervisions/collaborative meetings | 4 | Top theories in top mechanisms |
SC3 : change agent/champion role | 5 | Top theories in top mechanisms |
SC4: management support, supported role flexibility | 5 | Top theories in top mechanisms |
SC5: organisation structures etc. modified to support change | 4 | Top theories in top mechanisms |
SC6: involvement of relevant professional groups in programme development | 0 | |
RR | ||
RR1: service user involvement in programme design/delivery | 3 | Equal top RR theory |
RR2: peer support workers working with staff | 1 | |
RR3: staff understand recovery is non-linear | 3 | Equal top RR theory |
RR4: staff find service users with complex needs hard to engage with | 3 | Equal top RR theory |
RR5: service users and staff work together | 2 | |
RR6: paradigm shift from custodial/protective model to recovery-based care | 2 | |
RR7: appropriate medication regime | 0 | |
RR8: staff stress, burnout, job satisfaction | 1 | |
RR9: encouraged autonomy, positive risk-taking supported | 3 | Equal top RR theory |
RR10: quick wins demonstrate progress | 2 | |
RFR | ||
RFR1: strong community and family links | 2 | |
RFR2: shift/working pattern flexibility | 4 | |
RFR3: adequate shift handovers | 0 | |
RFR4: adequate staffing capacity, time, space, resources | 6 | Top RFR theory |
RFR5: appropriate medication regime | 0 | |
RFR6: change built into existing organisational structures, etc. | 5 | |
RFR7: positive, collaborative culture between staff and service users | 1 | |
RER | ||
RER1: shared training across staff groups | 4 | |
RER2: training ensures shared understanding of recovery | 2 | |
RER3: staff groups reflect together | 1 | |
RER4: administrative burdens, competing work priorities | 3 | |
RER5: action plan developed collaboratively | 5 | Top RER theory |
RER6: role flexibility, common understanding and support between staff groups | 3 | |
RD | ||
RD1: new staff activities reflected in organisational structures | 6 | Top RD theory |
RD2: training is practical and specific | 3 | |
RD3: training is repeated/refreshed | 3 | |
RD4: clearly articulated action plan | 3 | |
RD5: regular supervisions | 4 | |
RS | ||
RS1: action plan developed collaboratively | 3 | Top theories in top mechanisms |
RS2: high job satisfaction, low burnout | 2 | |
RS3: incorporate recovery into existing change programme | 3 | Top theories in top mechanisms |
RS4: climate of job uncertainty, fear | 5 | Top theories in top mechanisms |
RS5: all staff receive training | 2 | |
RS6: programme is part of research project | 2 | |
RS7: programme is tailored to staff group | 1 |
List of abbreviations
- CBT
- cognitive–behavioural therapy
- CI
- confidence interval
- CMO
- context–mechanism–outcome
- COT
- College of Occupational Therapists
- CSRI
- Client Service Receipt Inventory
- EP
- expert panel
- GAF
- Global Assessment of Functioning
- GMI
- General Milieu Index
- IQR
- interquartile range
- LRG
- local reference group
- LSP
- Life Skills Profile
- MANSA
- Manchester Short Assessment of Quality of Life
- MDT
- multidisciplinary team
- MINI
- Mental Illness Needs Index
- NICE
- National Institute for Health and Care Excellence
- NIHR
- National Institute for Health Research
- OAT
- out-of-area treatment
- OR
- odds ratio
- OT
- occupational therapist
- PhD
- doctor of philosophy
- PMG
- Programme Management Group
- QuIRC
- Quality Indicator for Rehabilitative Care
- RAMESES
- Realist and Meta-narrative Evidence Syntheses: Evolving Standards
- RCS
- Resident Choice Scale
- RD
- reinforced direction
- REAL
- Rehabilitation Effectiveness for Activities for Life
- RER
- recovery is everyone’s responsibility
- RFR
- resourced for recovery
- RI
- recovery is important
- RR
- recovery is realistic
- RS
- receptive staff
- SC
- supported change
- SD
- standard deviation
- SPRS
- Special Problems Rating Scale
- SURF
- Service User Research Forum
- TRIP
- team recovery implementation plan
- YTC
- Your Treatment and Care