Comparing the cost-effectiveness and clinical effectiveness of a new community in-reach rehabilitation service with the cost-effectiveness and clinical effectiveness of an established hospital-based rehabilitation service for older people: a pragmatic randomised controlled trial with microcost and qualitative analysis the Community In-reach Rehabilitation And Care Transition (CIRACT) study

Article history

The research reported in this issue of the journal was funded by the HS&DR programme or one of its preceding programmes as project number 11/1023/10. The contractual start date was in February 2013. The final report began editorial review in May 2015 and was accepted for publication in November 2015. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The HS&DR editors and production house have tried to ensure the accuracy of the authors’ report and would like to thank the reviewers for their constructive comments on the final report document. However, they do not accept liability for damages or losses arising from material published in this report.

Declared competing interests of authors

none

Copyright statement

© Queen’s Printer and Controller of HMSO 2016. This work was produced by Sahota et al. under the terms of a commissioning contract issued by the Secretary of State for Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science Park, Southampton SO16 7NS, UK.

Project Better Outcomes for Older adults through Safe Transitions

Project BOOST is a transitional care programme supported by the Society of Hospital Medicine. 5,6 This quality improvement collaborative has been implemented across the USA in different hospital settings, focusing on general medicine populations, both medical and surgical. Experts in the field of quality improvement and transitions in care facilitate the development and implementation of a BOOST site-specific programme that addresses the needs of each hospital.

The BOOST toolkit includes several interventions including risk assessment, a medication review, a discharge checklist and a multidisciplinary team-based approach to the discharge process. A recent study of 30 hospitals showed modest reductions (2–6%) in 30-day readmission rates. 5

Project Re-Engineered Discharge

Project RED is focused on a multidisciplinary approach to patient care co-ordinated by a nurse discharge advocate. 7 The discharge advocate engages patients during their admission to hospital and provides clinical information and an individualised and illustrated plan post discharge. Following discharge, a pharmacist performs the telephone follow-up including a medication review with direct communication to the primary outpatient provider. In a RCT there was a non-significant 6% reduction in the 30-day readmission rate and a significant 8% reduction in the 30-day visit to the accident and emergency (A&E) department post discharge. 7

Care Transitions Intervention

The CTI is a multicomponent programme designed to facilitate patient engagement and promote direct patient and caregiver involvement in self-management following hospitalisation, including providing the necessary skills to be able to navigate the health-care system. 8–11 There are four components of the CTI: (1) medication management, (2) development of a personal health record that is carried from site to site, (3) close follow-up with a primary care provider and (4) the identification of ‘red flags’ and indications that would prompt patients to contact providers. An advanced practice nurse ‘transition coach’ performs post-discharge home visits and makes telephone calls, emphasising patient engagement and self-management in the care of chronic diseases. The programme has been studied in several different acute care settings and has shown statistically significant reductions in 30-day readmission rates (4–6%)8–9,11 and 90-day readmission rates (6–22%). 9–10

The Transitional Care Model

The TCM is another nationally recognised transitional care programme and includes a strategy focusing on hospital-based discharge planning and home follow-up. 12,13 A transitional care nurse follows patients from hospital to home, facilitates communication with outpatient providers and performs a series of home visits and telephone follow-up calls in the post-discharge period. The TCM emphasises a multidisciplinary approach to patient care, led by the transitional care nurse who remains in contact with other providers including physicians, nurses, social workers, discharge planners and pharmacists. A reduction in the 90-day readmission rate of between 13% and 48% has been reported. 12,13

Report structure

Chapter 2 provides an introduction to the study and its relationship to current NHS policy. Chapter 3 describes the design and methods of the study and Chapter 4 describes the main RCT findings. Chapter 5 describes the qualitative study design and methods, followed by a summary of the main findings. Chapter 6 details the methods and results of the health economics analysis. Chapter 7 discusses the main findings and learning points from the study.

Trial design

This trial was a single-centre pragmatic RCT (1 : 1 allocation ratio) with an integral health economic study and parallel qualitative appraisal.

Participants

The participants in the trail were older people admitted to the general medical wards as an acute medical emergency.

Eligibility criteria

Patients were eligible for the study if all of the inclusion and none of the exclusion criteria were met:

• inclusion criteria:

  • age ≥ 70 years

  • general practitioner (GP) registered within the Nottingham City CCG catchment area only (catchment population 300,000)

• exclusion criteria:

  • bed bound prior to admission or moribund on admission

  • receiving palliative care

  • previously included in the trial on an earlier admission

  • unable to be screened and recruited by the research team within 36 hours of admission to the study ward (a 36-hour deadline ensured that there was not a delay in the participant receiving therapy and enabled the recruitment of a large proportion of patients admitted over a weekend when the research team was not available)

  • nursing home residents.

Study setting

General medical elderly care wards at the Queen’s Medical Centre, Nottingham (1800-bed hospital, serving a population of 680,000), with community follow-up.

Study intervention

The trial had two arms: (1) the CIRACT service (intervention arm) and (2) THB-Rehab (standard care arm).

1. The CIRACT service provided a comprehensive assessment of each participant’s ability to perform certain tasks, which was completed within 24 hours of randomisation, enabling the formulation of a rehabilitation plan. While in hospital the participants were treated daily (7 days a week if appropriate). During the hospital stay, the team liaised with each participant and his or her carer(s) to enable a visit to the participant’s home to assess and provide recommendations for equipment and make adaptations and/or modifications as required. The CIRACT service utilised the team’s expertise in community working to form links with the appropriate services to ensure a smooth and effective discharge. In more complex cases the CIRACT team took the participant out of the hospital for a home visit prior to discharge. Following discharge, the CIRACT team visited the participant at home within 48 hours to assess the level of rehabilitation required and further follow-up visits were provided as deemed necessary.

2. The THB-Rehab service was provided by the ward therapy teams (usually a band 6 occupational therapist and a band 6 physiotherapist) on weekdays only. The team jointly conducted an assessment of each participant’s ability to perform certain tasks and provided recommendation for rehabilitation. The service referred the participants to the appropriate community-based services for provision of equipment at home, personal care and ongoing rehabilitation when appropriate at discharge. Once discharged from hospital, participants had no direct contact with the THB-Rehab service.

In either group, if a participant became medically unwell at any point to the extent that he or she was no longer able to undertake rehabilitation activities, the treating team withheld further rehabilitation until instructed by the ward doctor that it was safe to recommence rehabilitation activities. The nursing and medical care provided by the ward staff did not differ between the two groups.

Primary outcome measure

The primary outcome was hospital LOS from randomisation to discharge from the acute medical elderly care ward.

Secondary outcome measures

The secondary outcome measures were:

1. Unplanned readmission rates at day 28 and day 91.

2. Super spell bed-days (total time in NHS care including hospital care and intermediate care) from admission to 91 days’ follow-up.

3. Functional ability at 91 days as assessed by the Barthel Activities of Daily Living (ADL) index. 29 This 10-item index is scored out of 20, with a score of 20 indicating the ability to get up and down stairs unaided and in and out of the bath or shower independently.

4. Health-related quality of life as measured by the European Quality of Life-5 Dimensions three-level version (EQ-5D-3L)30 at 91 days post discharge. The EQ-5D-3L is a standardized measure of quality of life including five domains – mobility, self-care, usual activities, pain/discomfort and anxiety/depression – each with three levels.

5. Comorbidity as measured by the Charlson index31 at 91 days post discharge. The Charlson index codes a total of 22 comorbid conditions into a single score.

6. Mean cost per patient of the CIRACT and THB-Rehab services estimated using microcosting methods and cost-effectiveness analysis from a NHS and Personal Social Services (PSS) perspective, using data collected from a modified Client Service Receipt Inventory (CSRI) questionnaire,32 with quality-adjusted life-years (QALYs) at 91 days post discharge.

Data collection

The research team collected demographic data [including age, sex, Mini Mental State Examination (MMSE) score33 and living circumstances] and outcome measures at baseline at face-to-face interviews and follow-up data through established hospital and community databases and participant telephone interviews [Nottingham Information System (NOTIS) hospital database: admission/discharge date data; Community System One: contacts with other services and equipment provision data; telephone interviews: outcome measure data].

Data were collected using trial data collection forms, which were monitored by the Nottingham Clinical Trials Unit (NCTU) for consistency, validity and quality. Missing data and data queries were referred promptly back to the recruiting site for clarification. For participants who withdrew from the study, data were collected up to that point (as specified in the consent form), with no further data collected. Participants were not replaced when they withdrew from the study. All reasonable attempts were made to contact any participants lost to follow-up during the course of the trial to complete the assessments.

Data management

All trial data were entered into a trial-specific database, with participants identified only by their unique trial number, date of birth and initials. The database was developed and maintained by the NCTU. Access to the database was restricted and secure. Data quality and compliance with the protocol were assessed throughout the trial by verification of trial data against clinical records and by data checking for accuracy and internal consistency.

Sample size calculation

The primary statistical analysis was to compare LOS for those allocated to receive the CIRACT service compared with LOS for those allocated to the THB-Rehab service (see Appendix 1 for the statistical analysis plan). Pilot data28 showed the log-transformed LOS to be normally distributed with a standard deviation (SD) of 0.9. Therefore, 111 patients per arm were required to detect a clinically important effect size of 3 days (equivalent to a geometric mean ratio of 0.7) with a 5% two-sided alpha and 80% power. Allowing for 5% non-collection of primary outcome data, 250 patients in total were recruited over a 13-month recruitment period.

Recruitment and consent

All eligible patients were made aware of the trial at the time of admission to hospital and invited to participate. Written informed consent was obtained by the research team in accordance with the International Conference on Harmonisation Guidelines for Good Clinical Practice [see www.ich.org/products/guidelines/quality/article/quality-guidelines.html (accessed 4 January 2016)]. For patients who were confused (who had dementia/delirium such that they were unable to understand the nature of consenting to a research study and the study process), consent was obtained from a carer following an established framework34 used in previous ethically approved studies in older persons with dementia.

Randomisation procedure

Once the research team had gained consent, patients were allocated to either the CIRACT service or the THB-Rehab service using the web-based randomisation service provided by the NCTU. Randomisation was determined by a computer-generated pseudorandom code using random permuted blocks of randomly varying size and held on a secure server. Participants were allocated with equal probability to either arm of the study. The randomisations were requested through a PC with Internet Explorer and internet access, located on a dedicated secure server within the University of Nottingham. All communications between the user’s PC and the server were fully encrypted (secure SSL 128 bit encrypted) and used a unique username and password.

Blinding

The research team collecting data and the research team analysing the data were blinded to treatment allocation. The participants and ward staff were not blinded to treatment allocation as the treating therapists liaised closely with ward staff to ensure optimal patient care. The 3-month follow-up data were collected by the research team blinded to the intervention.

Statistical analyses

Qualitative appraisal

The parallel qualitative appraisal was concerned with understanding ‘how’ the CIRACT service:

• was implemented, to develop evidence for future roll-out

• was delivered and designed in terms of workforce configuration, to understand the barriers to and drivers of sustaining inter-occupational and inter-organisational working

• interacted with other care processes and systems, to develop knowledge on their strategic alignment with existing care models

• was experienced by clinicians, patients and families, to develop recommendations for improvement

• impacted on established roles and relationships, to understand barriers to and drivers of change manifest in distinct professional knowledge, practice and cultural domains.

The appraisal was informed by a consolidated framework, drawing attention to a range of key factors that frame and are involved in the implementation of new practices. 35 The design of the qualitative appraisal involved a number of established methods of data collection and analysis, including non-participant observations, interviews and focus groups, which are further detailed in Chapter 5.

Health economic study

The integral health economic study was designed to evaluate the microcosts of service delivery and the cost-effectiveness of the CIRACT service compared with the THB-Rehab service. To determine the microcosts of service delivery, we proposed to use a time and motion study (TMS) microcosting methodology conducted through three phases (detailed in Chapter 6).

A TMS is used to understand existing work patterns with a view to informing a more efficient model and to aid microcost analysis. It directly observes and measures the time required to deliver a service. TMS methodology has its roots within business and was devised to improve work management. TMS was initially defined by Frederick Winslow Taylor36 in the 1900s and was designed to break down work into its component parts in order to streamline or redefine them to ensure maximum efficiency. TMS is increasingly being carried out within a health-care setting to ensure that systems are running as efficiently as possible, which in turn can lead to more cost-effective models for delivery. One part of this may be ascertaining resource use and valuing such resource use using microcosting, although other costing methods also exist. The microcosting method measures resource use and costs in detail at the individual patient level.

Differing methods of data collection for TMS have been used and include either self-report or observation. The choice of data collection method would depend on how intensive a job/role is and how many activities were being worked on over what period of time. Self-report TMS has been found to result in significantly less activities being described than with observer methods37 and therefore is regarded as being less accurate, although it is cheaper and more efficient as an observer is not required. For observation TMS there are two methods: work sampling or continuous observation. In work sampling the observer is external to the workforce and randomly records instantaneous observations. Because of this, observations can be made of multiple staff members, which may be required if the unit of work being observed is being carried out by a team rather than by an individual. Continuous observation is when either the person or the unit of work being observed is ‘shadowed’ by the observer and all predefined tasks of interest are noted using a data collection tool. Although the data collected with both observer methods are not significantly different,38 work sampling is a more economical and objective method (as the observer is more remote from the subject). However the choice of method used is dependent on the research question and resources available. To determine resource use in a TMS, a flow chart is usually drafted that includes all of the necessary steps to deliver the service. 39 This then allows the development of categories to be observed and systematic data collection. 40

Study harms/adverse events

Data were collected for each individual participant with regard to any falls that occurred while an inpatient on the ward until time of discharge. A fall was classed as an ‘adverse event’ and a fall resulting in a radiologically confirmed fracture was classed as a ‘serious adverse event’. The risks of taking part in the trial were regarded as minimal as the CIRACT service was not a different type of rehabilitation but a change in delivery of the THB-Rehab service that a participant would have received as part of his or her usual care.

Ethics

The study was approved by the National Research Ethics Service West Midlands – Staffordshire Research Ethics Committee (reference number 13/WM/0050) on 27 February 2013. The trial was conducted in accordance with ethical principles that have their origin in the principles of good clinical practice41 and the Department of Health Research Governance Framework for Health and Social Care. 42

Flow of participants into the trial

Study recruitment commenced on 23 June 2013 and ended on 31 July 2014, during which 1584 patients from three elderly care medical wards were screened for eligibility, of whom 250 were randomised into the trial. The dominant reasons for exclusion were GP registered outside the Nottingham City CCG catchment area, lack of research staff capacity and unable to gain consent from the participant (Figure 1). In total, 212 participants were followed up and included in the primary analysis.

FIGURE 1.

Participant flow.

Baseline characteristics of randomised participants

The baseline characteristics of the randomised participants are shown in Table 1. The mean age at randomisation was 84.1 years (range 67–99 years) and there was a slight predominance of women (64% of the total). The mean MMSE score was 21.7 out of 30 and the mean Barthel ADL score was 10.7 out of 20. There was a high prevalence of comorbidities among the participants, with a mean Charlson index score of 7.4. The groups appeared well balanced at baseline. This also held true between participants who had primary outcome data and those who did not within each arm, except for the Barthel ADL score, which was higher among those who had primary outcome data than among those who did not.

Follow-up

In total, 212 participants were discharged from the hospital alive (106 in each arm), of whom 174 were followed up at 91 days post discharge (n = 91 from the THB-Rehab service and n = 83 from the CIRACT service) (Table 2). The main reason for not being followed up at 91 days post discharge was death post discharge.

In total, 12 participants died in hospital prior to discharge and another 18 withdrew consent prior to discharge. Eight participants were discontinued from the study prior to discharge for various post-randomisation eligibility breaches (see Table 2). These are categorised as ‘other withdrawals’ (see Figure 1).

Primary outcome

The distribution of LOS for participants discharged is shown in Figure 2. This was skewed, with a peak proportion discharged at day 8 in the CIRACT group and at day 12 in the THB-Rehab group.

FIGURE 2.

Distribution of LOS for participants discharged alive. (a) THB-Rehab; and (b) CIRACT.

There was no significant difference in LOS between the CIRACT service and the THB-Rehab service (median 8 vs. 9 days; geometric mean 7.8 vs. 8.7 days, mean ratio 0.90, 95% CI 0.74 to 1.10), which was supported by the sensitivity analyses (Table 3).

The Kaplan–Meier estimates (Figure 3) similarly showed no significant difference in LOS between the groups and there was also no significant difference in LOS between the groups with in-hospital death as a competing risk (Table 4).

FIGURE 3.

Kaplan–Meier plot of time to discharge excluding in-hospital deaths.

Secondary outcomes

There were no significant differences in any of the secondary outcomes between the two arms (Table 5). There were a median of 15 and 17 super spell bed-days for the CIRACT and THB-Rehab groups respectively (geometric mean ratio 0.96, 95% CI 0.76 to 1.21). Of participants discharged from hospital, 17% and 13% were readmitted within 28 days post discharge from the CIRACT service and the THB-Rehab service respectively (risk difference 3.8%, 95% CI –5.8% to 13.4%) and 42% and 37%, respectively, were readmitted by 91 days post discharge (risk difference 5.7%, 95% CI –7.5% to 18.8%).

There were 15 protocol deviations in the CIRACT group and eight in the THB-Rehab group (Table 6). The deviations in the CIRACT group are shown in Appendix 2.

Adverse events

There were seven non-severe falls recorded from seven participants (n = 4 CIRACT service, n = 3 THB-Rehab service) (Table 7). No safety concerns were raised by the Trial Steering Committee (TSC).

Activity 1: organisational profiling

Organisational profiling aims to understand the outer and inner context within which new interventions, such as CIRACT, are translated and implemented into practice. This can be seen as ‘setting the scene’ of enquiry or orientating the subsequent study of CIRACT ‘in action’. Accordingly, the objective of organisational profiling is to gather contextual data about local services in which new interventions or controls are in operation. This includes information on:

• the spatial configuration of service areas

• service goals, strategy and policy

• staff and resource profiles

• patient numbers, throughput and LOS

• management and governance structures and processes

• leadership roles and approaches

• team structures and processes

• financial and commissioning arrangements

• performance management, including relevant data on performance levels.

Organisational profiling was carried out across two of the three wards during the initial months of the trial. In line with the consolidated framework and building on similar research carried out by the qualitative team,43 a profiling template was developed to guide these initial enquiries reflecting the themes listed above. This was completed through a series of linked research steps.

First, research gatekeepers (local sponsors) and key informants (service leaders) were engaged in ‘fact finding’ during which they were asked about the history of the service, the general configuration and current work pressures. Second, further semistructured qualitative interviews, lasting between 30 and 90 minutes, were carried out with six service leaders and key staff: service managers, clinical leaders, sisters and research co-ordinators. Third, service leaders provided ‘guided tours’ of each service area, including walking tours and introductory meetings with staff groups and attendance at scheduling staff meetings. Fourth, the qualitative study was introduced to clinical teams and groups working in the ward areas, usually during team briefings or rest breaks, at which more informal information and insight was provided about the service through a process of questions and answers. Finally, service leaders provided access to relevant documentary sources, including local policies and procedures, and performance-related data. Through these initial activities a broad contextual picture was developed of each service from which to direct subsequent and more fine-grained analysis.

Activity 2: study of the Community In-reach Rehabilitation And Care Transition and traditional hospital-based rehabilitation services in action

The second and largest period of data collection involved an in-depth ethnographic study of how the CIRACT and THB-Rehab services were organised, delivered and experienced as a situated, social process within a given context. This focused first on mapping and then understanding the organisation and delivery of each service in terms of (i) the structure and flow of the care pathway; (ii) the allocation of roles and responsibilities across the pathway; (iii) key decision-making and communication points across the pathway; (iv) variations in periods of care, support and education; and (v) handovers between care teams and interactions with external agencies. As such, data collection ‘zoomed in’ from the broader organisation of care and division of labour to a more focused analysis of clinical practices, interactions and care-giving processes.

In line with the ethnographic approach, this involved a combination of non-participant observations (122 hours) and semistructured and ethnographic (conversational) interviewing over a period of 6 months (see Appendices 3 and 4 for details of observations). Observations were made of different activities, building on the guided tours and on the rapport developed with service providers:

• Work process observations. In-depth workplace observations were undertaken over a period of 2 months, which involved mapping the temporal and spatial organisation of daily work (schedule of ward rounds, meetings, handovers, discharge times), identifying key events and activities [multidisciplinary team (MDT) meetings, drug rounds], identifying key individuals or groups (discharge co-ordinators, clinical leads) and drawing together these data into a complex descriptive account of the social setting and care process. Observations were undertaken on a daily basis (3–4 days per week, including evenings and weekends) over 8 weeks, with additional observations undertaken in community settings with the CIRACT team.

• In-depth observations of situational activities, tasks and settings. Prolonged follow-up observations were undertaken of key activities, tasks and settings to deepen knowledge of service delivery. Each setting or activity was observed at least three times and some, such as weekly MDT meetings, were observed up to 10 times. This included observations of:

  • morning handovers with the MDT on each ward

  • home visits prior to discharge

  • patient and family consultation meetings

  • social services assessments

  • use of information communication technology (ICT) and manual records about rehabilitation progress and discharge planning

  • referrals to multiple agencies by telephone and fax

  • ordering equipment and home adaptation, including telecare monitoring and adaptive devices

  • patient education and support

  • carer education and support

  • home visits after discharge

  • end-of-life care support across locations

  • referral to community agencies for longer-term support in person, by telephone or by fax.

• Shadowing of individuals. To deepen the understanding of the roles and contributions of certain individuals or groups, shadowing observations were also undertaken with key individuals or representatives from professional groups. Observations ranged from several hours (ward clerks) to several days (discharge co-ordinators) as the individuals went about their day-to-day work. The individuals observed included:

  • CIRACT team leader (transition coach)

  • therapists

  • ward clerks and administrators

  • occupational therapists

  • physiotherapists.

All observations were recorded, first in hand-written field journals, including rich descriptions and separate interpretations, which were later typed up electronically as corresponding text and interpretations along with a summary overview of the key points. It is important to note that observations were not focused on intimate, personal or challenging patient care. All observations of patient–clinician interactions involved prior written and verbal consent and focused primarily on the activities and work of the clinical team member, not the patient.

As indicated earlier, alongside these observations members of staff were engaged in a large number of conversational-style ‘ethnographic interviews’. Ethnographic interviewing involved small conversations and interactions with clinicians and other study participants in the normal cause of their work or practice. They were usually short (i.e. 5–10 minutes) and were used to clarify observations, elaborate the reasons behind decisions or actions and gain reflective insight from participants about activities in ‘real time’. As these were informal, they were recorded only in field journals alongside observation records. It is estimated that 200 such short ethnographic interviews were carried out over the study.

More formal semistructured interviews were also carried out alongside observations. These aimed to develop more reflective accounts or narratives of the respective services (CIRACT and THB-Rehab) from the perspectives of different stakeholders. Interviews were carried out with 13 participants across the CIRACT and THB-Rehab services: two managers, six therapists, two discharge co-ordinators, one senior nurse, one social worker and one care home manager (Table 8). The interviews were arranged at the convenience of participants and, in most cases, were conducted in a private setting. They ranged from 40 to 90 minutes in length and most were digitally recorded. The interviews followed a broad guide that covered the following topics:

• the implementation and development of the CIRACT service (or THB-Rehab service), including translation into practice, training and supporting and changes in design and delivery

• the organisation and delivery of the CIRACT service (or THB-Rehab service), including service configuration, daily work planning, decision-making and communication, discharge planning and delivery of patient care

• interprofessional and interorganisational working, including the barriers to and drivers of integrated working

• the perceptions of therapy and the value of the CIRACT service from a professional viewpoint.

Activity 3: patient tracking

Patient tracking aimed to develop a highly detailed and, importantly, patient-centred understanding of care processes and experiences. It focused on the interactions and care processes of a small sample of patients as their care progressed and as they moved between care teams and settings. As such, the method aimed to place the patient at the centre of analysis with the aim of understanding the web of interconnecting care processes that contribute to care planning, delivery and transition. This approach combined first-hand observations of care activities and processes, such as patient assessment, decision-making, communication and therapy, with a series of short structured interviews with patients as they moved along the care pathway and transitioned from hospital to a community setting. As such, it can lead to a highly developed, longitudinal (time–space) understanding of the patient experience.

In the first instance, the patient/family were approached in hospital, usually when the patient had been allocated to the CIRACT service or the THB-Rehab service. In collaboration with the patient’s designated care team, the research aims and methods and the patient tracking method were explained to the patient/family. Those who consented were then involved in a series of three to four observations of patient–clinician interactions on the ward and then at home, for example assessment, care planning, education and therapy. Each patient was also asked to participate in a series of short interviews to acquire further understanding of the patients’ experiences and views about their care. These started in hospital (usually two short conversations after observations of therapy on the ward) and then continued in the community at 1 week post discharge. Although every effort was made to schedule interviews at these times, in some instances they needed to be moved by ± 1 week because of other appointments. Furthermore, not all patients were able to participate in the full series of interviews because of withdrawal (their health had worsened), readmission to hospital, transfer to another care setting (i.e. left the CIRACT service) or death. The patient tracking data were initially recorded in reflective field journals and were then summarised within a common template to enable data management and comparison (see Appendix 5).

Data analysis

All data were managed in accordance with NHS and university research governance frameworks. All interview transcripts were anonymised with pseudonyms and all identifiable information, such as contact details, was securely filed. Hand-written ethnographic notes did not include identifiable names or locations and were archived within 48 hours into locked cabinets. Electronic data were stored within encrypted and secure external drives and back-up copies were kept within a locked location within the university.

Interpretative qualitative data analysis was undertaken to develop a descriptive and contextualised understanding of the respective care services, especially the implementation, organisation and delivery of the CIRACT service. This involved an iterative process of close reading of the data, coding, constant comparison, elaboration of emerging themes and re-engaging with the wider literature. In the first instance, one member of the research team independently reviewed a sample of transcripts and observation records to develop an initial case description and coding strategy. This was presented and discussed with the wider research team and qualitative methods advisors. Following feedback from these discussions, all data were systematically coded and categorised in line with the consolidated implementation framework and research questions. With regard to the reliability of the coding process, codes and categories were reviewed on a monthly basis by the wider team to ensure the accuracy of interpretation and the internal consistency of codes. Through this iterative process a number of common themes were developed in relation to four over-riding questions:

• How was CIRACT designed, implemented and translated into practice?

• How was CIRACT (and THB-Rehab) organised and delivered in context?

• How was CIRACT (and THB-Rehab) experienced by participants and professionals?

• How did CIRACT impact on clinical roles and how might services be developed?

Results

Changing cultures of care

The majority of health and social care professionals took a positive view towards the CIRACT service and considered it a positive step towards comprehensive care for older people with complex needs. There were divergent opinions about the impact that the service made to the entire NHS service locally but generally it was recognised as a small pilot service trying to make a difference. The majority of professionals who commented on the CIRACT service could only do so from within their own working boundary, which tended to be either acute hospital or community based. With the exception of the community managers and the academic lead, professionals could form opinions only of the parts of the CIRACT service drawing mostly on their own work with the team members and their previous experiences of community care.

There were concerns about the changing grades of staff and overlaps within physiotherapy and occupational therapy by the standard service teams:

I think in principle more senior staff doing a bit of OT [occupational therapy] and OT doing a bit of physio, I think it is helpful. You can do the basics . . . in concept taking on these roles, we’re half-way there anyway.

Standard service senior physiotherapist interview, 31/01/14 (lines 454–60)

Hospital-based staff perceptions of the types of service provided by CIRACT within the community locations were more constrained than community-based staff perceptions. Hospital-based therapists tended to view the CIRACT service as providing an emotionally supportive role towards patients on discharge rather than continuing intensive therapy and securing longer-term goals of care with other agencies. There was a sense of the CIRACT service undertaking some monitoring of well-being in the home although what this consisted of remained unclear:

I’m not able to follow them [patients] up and actually hold their hand when they get in [home]. I think that if they had that reassurance, just knowing that someone cared. And we do care. It’s just our actions suggests that we’re not there.

Standard service occupational therapist interview, 02/03/14 (lines 96–9)

There was mixed evidence of professional reflection about the CIRACT service in terms of changing the cultures of organisation. Nursing professionals considered the service as non-viable in terms of fitting in with the current financing and target-driven nature of the NHS as a whole. They did consider that the service design matched patient need and that the service acted as a more proactive rather than a reactive service.

There was no evidence of the emotional labour required to sustain such an intensive therapy service. The nature of the CIRACT service entails regular contact with patients and it was inevitable that close relationships were formed. Staff were observed to be genuinely upset when informed of a patient having died after the end of therapy and were often openly moved by the conversations of suffering and loss among families. This aspect is probably worthy of further work because the team did not seek extensive emotional support from each other but seemed to draw on their spiritual beliefs.

The THB-Rehab therapists tended to take a more sceptical view of the transferability of the service across the whole directorate and were concerned about their own terms of employment rather than it challenging the organisation as a whole. The service was not regarded as meeting patient need in any more ways than the THB-Rehab service except in relation to extending therapy beyond the inpatient stay. The community health-care professionals tended to regard the service as valuable but exclusive because of its limited capacity. There were some barriers to accessing GP services by the CIRACT team because of the lack of knowledge about the service among some practices.

Cost analysis

A detailed microcost analysis of the two services was considered using a three-phase TMS. Phases 1 and 2 were used to determine the TMS methodology and phase 3 consisted of the TMS itself:

• Phase 1 – Structured literature review (see Phase 1 systematic review).

• Phase 2 – Observations and semistructured interviews. In-depth workplace observations and semistructured interviews were undertaken as part of the qualitative appraisal (see Chapter 5) to map out key events and activities in the provision of the CIRACT service and the THB-rehab service, with additional observations undertaken in community settings.

• Phase 3 – Using the information from phases 1 and 2, the microcosting analysis (TMS) was defined and then conducted to identify, measure and value resource use and costing from a NHS perspective. Eleven patients in the CIRACT arm and 11 patients in the THB-rehab arm were recruited through purposive sampling over a 6-month period. Within each intervention arm, as far as possible, half of the participants had a baseline admission Barthel ADL score of < 14 and the other half had a Barthel ADL score of > 14 to ensure that a wide range of abilities were observed.

The staff costs (midpoint pay bands) for time spent on both services were recorded. Annual gross rates were converted into rates per minute assuming 42 weeks (37.5 hours per week) a year of work after annual leave and sick leave. All costs were calculated in UK pounds sterling (2014 prices) and all staff unit costs were inflated by 15% to take account of salary on-costs faced by employers.

Cost-effectiveness analysis

A cost (utility) analysis was undertaken in line with the National Institute for Health and Care Excellence (NICE) reference case44 to compare NHS and PSS costs with QALYs using established methods. 45 The economic evaluation was based on all costs and benefits incurred during the 12 months following randomisation. The EQ-5D-3L scores were converted into QALYs using linear interpolation and area under the curve methods for the trial period. 46 These were adjusted for baseline differences between the groups. 47 The results were presented using the incremental cost-effectiveness ratio (ICER). The costs and benefits were not discounted as the study was carried out over a period of 12 months.

Primary and community care costs included those for:

• GP consultations (telephone and home visits)

• nurse consultations (practice nurse and district nurse)

• dietitians

• community health workers

• referrals to a complementary therapist

• pharmacists

• the rehabilitation team (physiotherapist/occupational therapist)

• inpatient stays

• outpatient visits (including A&E visits, NHS walk-in centre visits)

• A&E visits

• visits to a NHS walk-in centre.

Direct costs to PSS included costs for:

• social workers

• home help/community care assistants

• admissions to a residential/nursing home

• visits to a day centre

• Meals on Wheels

• community equipment and household adaptations paid by social services.

Measurement of resource use

A modified CSRI questionnaire32 was designed to obtain resource use data, with data collected through established hospital and community databases and participant telephone interviews.

Unit costs for PSS were obtained from the Unit Costs of Health and Social Care,48 NHS reference costs49 (see Appendix 8), salaries based on the NHS Agenda for Change scale50 and other published sources as appropriate. These were then combined with resource use data to produce a cost per patient. All costs were valued in UK pounds sterling at 2014 prices.

Results

Summary

In conclusion, the CIRACT service as a complex intervention did not reduce hospital LOS or short-term readmission rates, although it was highly regarded by those most involved with it compared with the THB-Rehab service. The estimated ICER appears to be cost-effective although it is subject to much uncertainty that spans all four quadrants of the cost-effectiveness plane such that caution should be used in interpreting this result. Microcosting (TMS) work-sampling observational methodology provides a useful means of estimating the cost of a service provision.

Organisational and contextual issues

• Change in one service configuration impacts on the whole system, in this case both NHS and social care and across acute and community provision, which requires strong consistent leadership across the whole care pathway.

• Readmission rates to hospital for frail older people remain high and should be targeted for further intervention.

Communications

• It important to recognise the need for a lead to act as a key cross-team and cross-location influencer with localised knowledge of services and the capacity to establish and maintain networks as a core part of their everyday work.

• Colocation of teams supports knowledge and information sharing in proactive ways among experts, although this may introduce barriers in other ways.

Costing methodology

• The method of microcosting is the most precise method available to identify and measure resource use before quantifying this into monetary units. It is particularly useful in estimating the costs of a new service for which no, or little, previous cost estimates exist.

• In addressing the research question, choice of costing method(s) is a balancing act between accuracy and feasibility. In the present study the most feasible costing approach was a mixed approach using the most precise method of microcosting for costing the services and a less accurate method to cost the wider resource implications.

Contributions of authors

Opinder Sahota (Professor of Orthogeriatrics and Consultant Physician and lead and corresponding author) designed and led the research project, managed the data analysis and wrote the final report.

Ruth Pulikottil-Jacob (Research Fellow in Health Economics) and Tracey Sach (Associate Professor in Health Economics) co-designed the health economics phase and contributed to data analysis and report writing.

Fiona Marshall (Research Fellow) conducted qualitative research, collected data from interviews, observations and patient tracking and contributed to data analysis and report writing.

Alan Montgomery (Professor of Medical Statistics) and Wei Tan (Research Fellow) designed and conducted the statistical analyses.

Pip Logan (Professor of Rehabilitation Research), Denise Kendrick (Professor of Primary Care Research) and Maria Walker (Occupational Therapist) provided valuable advice and guidance throughout the study.

Alison Watson (Research Fellow) conducted the quantitative research, collected data from interviews, observations and patient tracking and contributed to data analysis and report writing.

Justin Waring (Professor of Organisational Sociology and Improvement Science Fellow) co-designed the project, conducted qualitative research, collected data from interviews and contributed to data analysis and report writing.

Patient and public involvement

The study has benefited from strong patient and public involvement from Mrs Rosemary Clacy and Mr Peter Cass, who were involved from the inception of the trial and during conduct of the trial and data analysis. Both were able to attend regular steering group meetings and individual smaller groups meetings, which were critical at the inception stage of the trial.

Data sharing statement

Because of the nature of the study and condition of ethics approvals we are unable to share data related to this study. Please contact the lead author for further information.

Disclaimers

This report presents independent research funded by the National Institute for Health Research (NIHR). The views and opinions expressed by authors in this publication are those of the authors and do not necessarily reflect those of the NHS, the NIHR, NETSCC, the HS&DR programme or the Department of Health. If there are verbatim quotations included in this publication the views and opinions expressed by the interviewees are those of the interviewees and do not necessarily reflect those of the authors, those of the NHS, the NIHR, NETSCC, the HS&DR programme or the Department of Health.

The CIRACT (Community In-Reach And Care Transition) clinical and cost-effectiveness study Statistical Analysis Plan

Final Version 1.0 (20th Jan 2015).

Based on Protocol version 4.0 (dated 10 Mar 2014).

Informed consent deviation details

Details of other deviations