A cluster randomised controlled trial and economic evaluation of a structured training programme for caregivers of inpatients after stroke: the TRACS trial

Article history

This issue of Health Technology Assessment contains a project originally commissioned by the MRC but managed by the Efficacy and Mechanism Evaluation Programme. The EME programme was created as part of the National Institute for Health Research (NIHR) and the Medical Research Council (MRC) coordinated strategy for clinical trials. The EME programme is funded by the MRC and NIHR, with contributions from the CSO in Scotland and NISCHR in Wales and the HSC R&D, Public Health Agency in Northern Ireland. It is managed by the NIHR Evaluation, Trials and Studies Coordinating Centre (NETSCC) based at the University of Southampton.

The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The HTA editors and publisher have tried to ensure the accuracy of the authors' report and would like to thank the reviewers for their constructive comments on the draft document. However, they do not accept liability for damages or losses arising from the material published in this report.

Copyright statement

© Queen's Printer and Controller of HMSO 2013. This work was produced by Forster et al. under the terms of a commissioning contract issued by the Secretary of State for Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science Park, Southampton SO16 7NS, UK.

Previous research

A range of systematic reviews of qualitative7,12 and quantitative13 stroke literature have confirmed the diversity, complexity and frequency of problems faced by patients and caregivers during recovery from, and adjustment to, a disabling stroke. A Cochrane review has summarised the effectiveness of non-pharmacological interventions for caregivers of stroke survivors in reducing caregiver burden or enhancing caregiver well-being. 14 In eight randomised controlled trials (RCTs) involving 1007 participants, interventions evaluated were categorised as support and information provision;15–18 caregiver training programme;19 and psychoeducational. 20–24 Three of these studies were inpatient interventions;15,17,19 one was started as inpatient and crossed over into the community,21 two were conducted post discharge in the non-acute phase,18,22 and the location of delivery of two was unclear,16,20 although participants were recruited as inpatients. The comparator in six of the studies was usual care,15–19,21 one study used a crossover wait list design20 and one study used written information on stress management. 22

Only one study – the caregiver training programme19 – was seen to have a significant effect on reducing caregiver burden. The combined results of the support and information provision,15–18 and the psychoeducational20–22 interventions revealed no impact on caregiver burden.

A number of secondary outcomes were examined by the studies. Two studies (one support and information provision16 and one psychoeducational20) assessed global measures of stress or distress, and revealed no significant benefit of the intervention over usual care. One study (caregiver training programme19) assessed anxiety, and the analysis conducted by the reviewers revealed no significant effect of the intervention on caregiver anxiety. Of the five studies measuring depression,16,18–21 only the caregiver training programme19 was seen to have a significant beneficial effect. Three studies15,16,19 (the caregiver training programme19 and two support and information provision interventions15,16) assessed health-related quality of life. The caregiver training programme19 demonstrated a significant improvement in the intervention group, one support and information provision study showed no significant benefit,15 and the second support and information provision study16 showed significant improvements in 5 out of 8 SF-36 health domains. The review authors conclude that, with limited available studies the caregiver training programme19 was the most promising intervention. However, the evidence for this was from one single-centre RCT. 19

The conclusions of the Cochrane review support the findings of earlier reviews of stroke caregiver interventions:10,23,24 that previous studies have methodological limitations; further robust evaluation is required; and the effects of caregiver interventions on the patient also need to be assessed.

In the Cochrane review, the caregiver training programme,19 which included education and some practical ‘hands-on’ skills training, was identified as having the most potential to benefit both caregivers and patients. This is supported by evidence from our Cochrane review of information provision (n = 17 trials), which concluded that stroke education programmes improve patient and carer knowledge of stroke, aspects of patient satisfaction, and reduce patient depression scores. 25,26 There was also some evidence that education interventions that actively involve patients and carers and include planned follow-up for clarification and reinforcement are likely to be more successful than simply providing information. 25,26 Furthermore, caregivers have identified the information and skills training required to implement physical care as their most important pre-discharge needs. 27 Although caregiver support is a key component of stroke unit care, caregivers report that there are missed opportunities for structured skills training prior to the stroke patients discharge,28 and that support, as currently provided, is not compatible with their expressed needs, and their ability to care is not assessed. 10,27

Caregiver training programme

Literature searches conducted by the authors prior to publication of the Cochrane review on the effectiveness of caregiver interventions identified no effective early training programmes for caregivers of patients after stroke, other than the caregiver training programme evaluated by Kalra et al. 19 in an individually randomised single-centre study. This intervention – the London Stroke Carers Training Course (LSCTC) – was a systematic and structured training programme for caregivers, which included assessment in competencies in skills essential for the day-to-day management of disabled stroke survivors. The LSCTC was based on a survey among stroke caregivers, asking them to identify major problems experienced after hospital discharge. Although many reported satisfaction with involvement in discharge planning, most found themselves unprepared for the task of providing ‘hands-on’ care at home. The components of the LSCTC were therefore devised to address the knowledge and skills required to effectively care for stroke patients on discharge from hospital.

The single-centre study participants were 300 patients and caregivers admitted to a stroke rehabilitation unit in south London, UK. 19 Patients and caregivers were block randomised to receive either usual care or the LSCTC prior to discharge home. The primary outcome was the cost to health and social services during the first year of stroke. Total health and social care costs over 1 year were significantly lower in the intervention group, with a mean difference of £4043. 29 This cost difference was largely due to a shorter length of stay for patients in the intervention group than in the control group. There was no difference in quality-adjusted life-years (QALYs) in caregivers. 29 Significant secondary outcomes for caregivers receiving the intervention included a reduction in caregiver burden (as measured by the Caregiver Burden Scale; CBS30), improved quality of life [EuroQol 5-dimension health-state measure: European Quality of Life-5 Dimensions (EQ-5D) visual analogue scale31] and mood [Hospital Anxiety and Depression Scale (HADS)32]. There was no significant difference in social activity levels [Frenchay Activities Index (FAI)33]. For the patients whose caregivers had received the training, there was a significant improvement in quality of life (EQ-5D visual analogue scale) and mood (HADS). No significant differences were seen in patients' physical recovery (Barthel Index34 and modified Rankin Scale35), mortality or institutionalisation. 19

Justification for the current study

There were important limitations to the generalisability of the findings of the study of Kalra et al. 19 The LSCTC was tested in a single centre, delivered by the LSCTC development team, who might be expected to have heightened motivation and expertise, and the patient population was predominantly recruited from a middle-class suburban area, and might be more responsive to a training and education programme. In addition, having demonstrated benefit for caregivers on a range of domains, it was important to evaluate the effectiveness of the LSCTC programme on improving patient outcomes. The aim of the TRACS (Training Caregivers after Stroke) trial was to assess the effectiveness of the LSCTC on patient outcomes (once embedded in usual practice) in stroke units across the UK, thereby testing wider generalisability in settings in which the population, health and social care provision differ.

Development of the modified London Stroke Carers Training Course

The LSCTC consists of 14 core caregiver competencies that required training and testing. These competencies were important knowledge/skills that informal caregivers would need to be able to care effectively for the stroke patient on discharge home, for example demonstrating understanding of what a stroke is; knowledge of the patient-specific problems associated with stroke (which may be related to speech, mobility, memory, diet and swallowing, vision and reading, washing and dressing, transfer and walking); knowledge of how to manage and provide support for personal ADLs, including continence management if required. Modification of the LSCTC was required for the multicentre TRACS trial to allow the intervention to be implemented in different NHS settings, and by members of the stroke rehabilitation units' (SRUs') multidisciplinary teams (MDTs) with a range of skills and expertise. The LSCTC was modified and a training programme for staff was developed by the original LSCTC MDT (AM, MW and JS), based on what had worked in the single-centre study and what could be transferred to other settings. The modified LSCTC maintained the original structure of 14 core competencies; six of the training components were listed as mandatory, requiring the MDT to train all caregivers on these items, and the remaining eight components were to be completed as appropriate dependent on each individual patient's ability and their caregiver's needs. A full description of the 14 training components is provided in Table 1. To facilitate replicable delivery, a training manual was created, which described in detail the objective of each training component and recommendations on how to deliver the information/training, resources available (i.e. relevant stroke association information leaflets), and suggested ways to assess the caregivers competency (verbal/observations, etc.). A summary of the underpinning principles was also provided; for example, training should be individualised to the caregivers at the required level and should be based on the needs of the patient. Further modifications were made following pilot use of the modified LSCTC by the SRUs randomised to the intervention.

London Stroke Carers Training Course Caregiver training record

To support the training and standardisation of delivery of the LSCTC, a structured training record was created, which listed the 14 training components and provided a section to indicate whether or not that particular component was mandatory/appropriate for the individual carer. If mandatory/appropriate, the component could be ticked off once the training had been given, and the caregiver's competency could be signed off once assessed. A further section allowed documentation of the total time taken to deliver the training. On the final page, space was provided to document progress of training and so the record acted as a work in progress throughout the patient's stay and a way of communicating training progress and ongoing needs throughout the team. For each caregiver the MDT staff were asked to complete a training record. A copy of the training record can be seen in Appendix 1.

Training of the intervention

Two training sessions were provided for the SRUs randomised to deliver the LSCTC. The training days were delivered by the original LSCTC development team and were held over 2 days, 1 month apart. The same sessions were repeated twice; once in Leeds for the Yorkshire and North West centres and once in London for the London and the South East and South West Peninsula centres. The aim of the first training day was to ensure that the MDTs delivering the LSCTC were clear about what they needed to deliver to the caregivers, and to consider how best to implement the training within their local unit. The day covered, through presentations and group workshops, the background to the LSCTC, the training components and suggested delivery and use of the training records (see Appendix 2). This training day was filmed and provided the basis for a training CD to be used to cascade the training to all staff on the SRUs. Staff were provided with the training manual (see Appendix 3).

After the first session, the attendees were required to cascade the LSCTC training to the rest of the MDT on their SRUs. The LSCTC was then piloted on a small number of caregivers on each SRU, including completion of the training record. A second training day was held approximately 1 month later. This session allowed open discussion on possible refinement of delivery of the LSCTC by the MDT and modification of the training manual and records (see Appendix 2). After this meeting the LSCTC was gradually implemented as a part of standard practice on the SRUs. All centres then received a visit from the TRACS trial manager who used the completed training records as a basis for discussions on structure and process. Further local training sessions were arranged if necessary to provide feedback and support, and discuss any problems with LSCTC provision. The programme was then delivered for a duration of 24 months while the TRACS trial recruitment took place.

Training attendees

The TRACS team suggested that at least two key MDT members from each of the 18 intervention centres attended the initial training days. In practice, 1–13 members from each centre attended, with an average of three attendees per centre. The attendees came from a range of disciplines, primarily senior physiotherapists, senior occupational therapists and senior nurses, but also included Band 5 nurses, consultant physicians and senior speech and language therapists. The attendees were identified as ‘TRACS champions,’ who had responsibility to cascade the intervention within their site.

All centres were offered a local refresher course by the TRACS trial manager midway through the trial in September–October 2008. In total, 13 out of 18 centres received this refresher training; four centres said that they were completing the LSCTC successfully and did not require further training [three such centres did have good return rates of the training records, one centre (294) had not yet returned the records so compliance could not be assessed]. One centre remained non-compliant and refused further training. As the TRACS trial recruitment period was extended for an additional 10 months, a further central training day was provided in London in August 2009; staff from 11 centres attended this day.

London Stroke Carers Training Course delivery

The LSCTC was designed to be delivered to caregivers while the patient was an inpatient with one ‘follow through’ session provided in person or by telephone after hospital discharge. The training was individualised to the caregivers' required level of understanding. The timing of the sessions was not dictated, and could begin at anytime from admission, and throughout the patients stay, depending on when the team felt it appropriate for that particular caregiver, along with other factors, such as varying lengths of stay and local procedures. It was recognised that different components could be covered in a number of different ways, by different professionals depending on individual circumstances.

A key component of the LSCTC was the requirement to check each caregiver's competency on each of the training components delivered and for an appropriate member of the MDT to ‘sign off’ the competency as achieved. Competency was defined as ‘The caregiver has taken on board the knowledge/skills required to be able to deliver the support that that patient needs’. Training would continue until the caregiver was deemed competent (or until it was agreed by the MDT that the caregiver was unable to become competent). This allowed the level of training to be both individualised to the caregiver and standardised across SRUs.

Monitoring of delivery

The completed LSCTC training record for all trial participants was returned to the trial manager and was included as a standard monitoring report to the Trial Management Group (TMG) and Trial Steering Committee (TSC). This enabled monitoring of compliance with the intervention delivery in the SRUs, and the time taken and competencies achieved by each caregiver. In instances where there were concerns about SRU compliance, the trial manager directly engaged with the sites to explore difficulties. The chief investigator wrote to the local principal investigators (PIs) of two units to express concerns.

Trial design

Training Caregivers After Stroke was a pragmatic, multicentre, cluster RCT designed to evaluate whether or not a structured, competency-based training programme for caregivers (LSCTC) improved physical and psychological outcomes for patients and their caregivers after disabling stroke and to determine if such a training programme was cost-effective.

Training Caregivers After Stroke was designed as a pragmatic trial. Thus the eligibility criteria were broad and inclusive, the intervention was highly flexible in application and was delivered in a full range of SRUs by the local MDT staff. Monitoring of participant and practitioner compliance/adherence was unobtrusive with no special strategies to improve compliance. Outcomes were objective and meaningful to patients. The primary analysis was intention to treat (ITT) as a test of whether or not the treatment worked in the context of all inherent real-life noise. 36

Figure 1 summarises the study methods, and the study protocol can be viewed in Appendix 4.

Justification of a cluster randomised design

The cluster randomised trial design was purposely selected to reduce between-group treatment contamination. Within the pragmatic trial, the LSCTC intervention was incorporated into usual practice and delivered by the whole MDT. If randomisation had been at the level of individual patients, the MDT would have had to operate two approaches (usual care and the LSCTC) with an associated high risk of between-group contamination as it would not have been possible to blind members of the MDT, thus it seemed likely that the new care process would have been extended to patients in the usual care group. Randomisation was therefore at the level of the (service) stroke unit. In order to minimise selection bias, there was a clear separation between the provision of the intervention by clinical staff and the recruitment and consent of patients and caregivers by research practitioners.

Primary objectives

The primary patient objective of the trial was to determine whether or not the provision of the LSCTC improved functional independence. The primary caregiver objective was to determine whether or not the provision of the LSCTC reduced burden for caregivers.

Secondary objectives

The secondary objectives were to determine whether or not the provision of the LSCTC (1) improved physical and psychological patient outcomes in the long term, (2) improved physical and psychological caregivers outcomes and (3) was cost-effective based on (a) patient outcomes, from both health/social care and societal perspectives and (b) caregiver outcomes, from a health-care perspective.

FIGURE 1.

Study flow chart.

Stroke unit (cluster) eligibility

Stroke rehabilitation units were eligible to participate in the TRACS trial if they met four out of five key criteria used to define a stroke unit, as suggested by the Royal College of Physicians of London (RCP) for the National Sentinel Stroke Audit (NSSA) 2006. 37 The five key criteria are (1) consultant physician with responsibility for stroke, (2) formal links with patient and caregiver organisations, (3) MDT meetings at least weekly to plan patient care, (4) provision of information to patients about stroke and (5) continuing education programmes for staff. Additional eligibility criteria were that a substantial number of patients on the unit had a diagnosis of stroke, the unit was able to deliver the LSCTC and the majority of patients were discharged to a permanent place of residence.

Randomisation and stratification

Cluster randomisation of the 36 eligible SRUs was performed centrally at the Clinical Trials Research Unit (CTRU). SRUs were randomised on a 1:1 basis to either the intervention or the control group. The randomisation was stratified by geographical region (Yorkshire, the North West, the South West Peninsula, and London and the South East) and quality of care (defined as being on and above, or below, the median on the key 12-indicator score of the 2006 NSSA37). Block randomisation was used to ensure these important covariates were balanced between the arms of the trial.

Process information

Process data were collected before, during and after recruitment at each participating SRU to monitor any changes in eligibility in SRUs, and in the process of care that prepared patients and caregivers for discharge in SRUs. Data were collected on the NSSA scores completed during the trial (2006 and 2008); ward type (combined acute and rehabilitation or rehabilitation); number of stroke beds; MDT staff ratios; use of community and early supported discharge stroke teams; and usual MDT working. Senior MDT staff (where possible, therapy and nursing staff) were asked open-ended questions to describe usual ward practice and discharge preparation, and how patients and caregivers were involved with this. Responses were recorded on the first visit, and on following visits the initial responses were cross-checked with the new responses and any changes/additions/losses to service updated. Any such changes were monitored by the trial manager through visits to the centres and discussions with the researchers, PIs and MDTs, and were reported back to the TMG for discussion and decision-making.

Participant eligibility

Patients were eligible for TRACS if they had a confirmed primary diagnosis of new stroke, were medically stable, were likely to return home with residual disability, and had a caregiver available, willing and able to provide support after discharge. The caregiver was defined as the main person, other than health, social, or voluntary care provider, helping with ADL and/or advocating on behalf of the patient. Written informed patient consent/caregiver declaration and caregiver consent were obtained prior to any trial-specific procedures. Patient and caregiver dyads were excluded if the patient was in need of palliative care, if discharge was planned within 1 week of admission to the SRU, or if the patient or caregiver was previously registered to the trial.

Blinding

Participant recruitment and baseline assessments were undertaken by researchers independent of the clinical MDT. The clinical MDTs in both the intervention and control arms conducted the LSCTC/usual care with all eligible patients' caregivers whether or not they consented to study procedures. The MDTs were not informed of which patients/caregivers consented to study procedures. Participants were blinded to the SRUs allocation.

Participant recruitment

Primary outcomes

The primary patient outcome was functional independence measured at 6 months using the Nottingham Extended Activities of Daily Living (NEADL) scale. 41,42

The primary caregiver outcome was caregiver burden measured at 6 months using the CBS. 30

Secondary outcomes

Secondary patient outcomes included self-reported measures of mood (HADS32); health state (EQ-5D);31,43,44 ADLs (Barthel Index);34,45 functional ability and health-related quality of life [Stroke Impact Scale (SIS)];46–51 death; hospital readmission and institutionalisation, all measured at both 6 and 12 months after recruitment, and functional independence (NEADL) at 12 months.

Secondary caregiver outcomes included self-reported measures of social restriction (FAI);33,52 mood (HADS); health state (EQ-5D); death; hospitalisation and institutionalisation at 6 and 12 months, and caregiver burden (CBS) at 12 months.

The cost-effectiveness of the LSCTC for both patients and caregivers was also assessed. Resource use was measured using the self-completed Client Service Receipt Inventory (CSRI). 29,53 Hospital records were checked for patient hospital readmissions and caregiver hospital admissions at 6 and 12 months post registration.

Assessment instruments

Analysis methods

All data analyses were conducted to a prespecified analysis plan. All data analyses were performed using SAS version 9.2 (SAS Institute Inc., Cary, NC, USA). All hypothesis testing was performed at the 5% two-sided significance level. Analysis of health economic data was performed using SPSS (SPSS Inc., Chicago, IL, USA).

Primary analysis

For all analyses, means and 95% confidence intervals (CIs) together with values of unadjusted and adjusted ICCs are reported. Summaries from raw and predicted data from the final model are provided.

Secondary analyses

All analyses of secondary end points were conducted on the ITT population. Means, 95% CIs and values of unadjusted and adjusted ICCs are reported. Summaries from raw and predicted data from the final model are provided.

Two-level random intercept models were used, with patients being level one and stroke units being level two. Fixed parts were patient-level covariates (level one), stroke unit-level covariates (level two) and treatment. Data were assumed missing at random.

Economic evaluation

The purpose of the economic evaluation was to examine the cost-effectiveness and cost–utility of the LSCTC. There were two parallel economic evaluations: one for patients and another for caregivers. The primary economic evaluation was cost-effectiveness analyses based on the patient and caregiver primary outcome measures (NEADL and CBS, respectively). The secondary economic evaluation was cost–utility analyses based on QALYs.

Cost-effectiveness and cost–utility analyses

Cost-effectiveness/cost–utility analysis is concerned with linking costs with outcomes and comparisons between two or more alternatives.

The primary economic evaluation at 6 months, the cost-effectiveness analyses, involved examining the following four cost–outcome combinations between the two randomisation groups.

Patients:

• the additional cost per additional point improvement on the NEADL scale from the health and social care perspective

• the additional cost per additional point improvement on the NEADL scale from the societal perspective.

Caregivers:

• the additional cost per additional point improvement on the CBS from the health and social care perspective

• the additional cost per additional point improvement on the CBS from the societal perspective.

The secondary economic evaluation at 6 months, the cost–utility analyses, involved the following four further cost–outcome combinations:

Patients:

• the additional cost per additional QALY from the health and social care perspective

• the additional cost per additional QALY from the societal perspective.

Caregivers:

• the additional cost per additional QALY from the health and social care perspective

• the additional cost per additional QALY from the societal perspective.

Further considering cost-effectiveness and cost–utility over 1 year involved examining these cost–outcome combinations once more. There were thus a total of 16 cost–outcome combinations to consider.

Cost-outcome comparisons between the intervention and control groups can produce one of four outcomes:

• If costs are lower and outcomes are higher for one group, then it is considered to ‘dominate’ the other and is clearly more cost-effective.

• If outcomes are similar between groups, then the one with lower costs can be regarded as more cost-effective (or if costs are similar between groups then the one with better outcomes is regarded as more cost-effective).

• If both costs and outcomes are lower for one group, then there are value judgements involved in trading off outcomes for cost savings.

• If both costs and outcomes are higher for one group, then it falls on relevant decision-makers to decide whether or not the additional benefits are worth paying for. In this scenario, results of a cost-effectiveness/cost–utility analysis are generally expressed as incremental cost-effectiveness ratios (ICERs), which represent the additional cost associated with one additional unit of the outcome for one person; these are calculated by dividing the mean difference in cost by the mean difference in outcome.

Each of these scenarios can be represented on a cost-effectiveness plane (Figure 2), in which the vertical axis represents the additional costs of one intervention against another, and the horizontal axis represents the additional outcomes. The location of a co-ordinate representing incremental cost and incremental outcome indicates which of the four scenarios the cost-effectiveness finding falls into.

FIGURE 2.

Cost-effectiveness plane.

We constructed cost-effectiveness planes using non-parametric bootstrapped regressions (5000 replications, with replacement) of study group upon 6-month health and social care costs, patient and caregiver QALYs, total NEADL score and total CBS score, in turn, with cluster adjustment for centre and baseline values of the dependent variable (except in the case of CBS), age and sex included as covariates. The resulting coefficients of group differences were saved and plotted using scatter graphs (Stata version 10.1, StataCorp LP, College Station, TX, USA) in relevant cost–outcome combinations.

We calculated ICERs for any cost–outcome combinations where the intervention group had higher costs and better outcomes (the top-right quadrant of the cost-effectiveness plane). Although ICERs have been one of the most common ways of presenting the results of cost-effectiveness analyses, they carry two important limitations. First, they are point estimates that do not provide information about the uncertainty surrounding the estimates, a problem that is compounded with the complexity of examining CIs around ICERs. Second, their use to decision-makers is limited as they provide no information about what an ‘acceptable’ level of cost-effectiveness would be, or whether or not the likelihood of cost-effectiveness would differ according to how much decision-makers would be willing to invest. Both of these limitations can be addressed by cost-effectiveness acceptability curves (CEACs) based on the net benefit approach. 71 We use CEACs alongside cost-effectiveness planes to examine any uncertainty surrounding the cost-effectiveness and cost–utility results.

Net benefits provide a single summary monetary measure of costs and outcomes for each individual (removing the need to examine ratios) and account for the value (λ) that a decision-maker would be willing to pay for a greater net benefit. They are calculated as follows:

⁽⁴⁾ Net benefit = ( λ × outcome ) − cost

For each cost–outcome combination, a series of net benefits were calculated for a range of relevant λ values. For the QALY-based analyses, these values ranged from £0 to £50,000 (£10,000 increments), to incorporate the £20,000–30,000 per QALY gain threshold range currently specified for National Institute for Health and Care Excellence (NICE) decision-making in England and Wales. 72 For the NEADL- and CBS-based analyses, the ‘acceptable threshold’ is unclear so we examined a range that incorporated NEADL- and CBS-related ICER values at 6 months, £0–2000 (£500 increments). Net benefits were then compared at the level of randomisation group using non-parametric bootstrapped regressions (5000 replications, with replacement) of study group upon net benefit, with a cluster adjustment for centre and baseline values of costs from the relevant perspective, age and sex, baseline utility for QALY-based CEACs and baseline NEADL score for NEADL-based CEACs included as covariates. For each value of λ, the proportion of iterations indicating a higher net benefit for the LSCTC group were calculated and plotted. These plots formed the CEACs, which represent the probability of the LSCTC group being cost-effective compared with the control for a range of values that a decision-maker would be willing to pay for an additional unit of the NEADL score, CBS score and QALYs.

Definition of end of trial

The end of the trial was defined as the date that the last 12-month postal questionnaire was completed.

Adverse events and safety monitoring

Events such as patient falls and caregiver musculoskeletal injury represent an inherent consequence of an active rehabilitation process and, therefore, cannot be entirely avoided. Similarly, in this patient population, acute illness resulting in hospitalisation, new medical problems and deterioration of existing medical problems were expected. In recognition of this, events fulfilling the definition of an AE or an expected SAE were not reported in this study, with the following exceptions: patient falls with or without fracture that occurred at any time between the date of consent and date of discharge; patient and/or caregiver death (SAE); patient and/or caregiver hospital admissions and readmissions for any reason (SAE); patient and/or caregiver institutionalisation (AE); and patient and/or caregiver treatment on an emergency outpatient basis (AE). Summaries of the above expected AEs and SAEs by treatment arm were reported quarterly to the Data Monitoring and Ethics Committee (DMEC).

Data monitoring

Data were monitored for quality and completeness by the CTRU, using established verification, validation and checking processes. Missing data (except patient and caregiver completed data items collected via the baseline and postal questionnaires) were chased until received, confirmed as not available, or the trial was at analysis.

Ethical approval

The study protocol was approved by Leeds Research Ethics Committee (REC), reference 07/Q1205/12. Local REC approval and research and development (R&D) approval was obtained at each NHS trust prior to cluster randomisation.

Amendments to the study protocol following commencement of recruitment

Literature review search strategy

Literature searches were completed using the Cumulative Index to Nursing and Allied Health Literature (CINAHL), EBSCO, The Cochrane Library (Wiley Online Library), EMBASE (OVID), MEDLINE (OVID), and PsycINFO (OVID), from January 1999 to November 2011. The International Standard Randomised Controlled Trial Number (ISRCTN) registry was also searched manually. Appendix 7 provides the full search strategy used.

Recruitment and randomisation of stroke rehabilitation units

Participant flow and recruitment

Characteristics of screened participants

Table 7 summarises the characteristics of screened patients. The groups were well balanced with respect to baseline characteristics.

The mean age of screened patients was 74.4 (SD 13.39) years and the average length of hospital stay was 27.9 (SD 33.07) days. In total, there were 5967 (47.7%) males; the majority of patients were white (11,628, 92.9%); 4922 (39.3%) patients lived alone; 6227 (49.8%) patients co-habited and 691 (5.5%) were nursing or residential care home residents.

The caregiver resided with 5015 (40.1%) patients. In most cases, the caregiver was the patient's family member: partner in 5047 (40.3%) and offspring in 3587 (28.7%) cases.

TABLE 4 - Characteristics of randomised centres

Centre	Intervention (n = 18)	Control (n = 18)
Geographical region, n (%)
North West	6 (33.3)	6 (33.3)
London and the South East	4 (22.2)	4 (22.2)
South West Peninsula	3 (16.7)	3 (16.7)
Yorkshire	5 (27.8)	5 (27.8)
NSSA 2006 score
Mean (SD)	67.8 (13.09)	68.5 (13.55)
Median	68.0	68.5
No. of beds
Mean (SD)	18.6 (7.40)	18.5 (7.68)

FIGURE 3.

Flow of patients from admission to registration: by treatment arm. Percentage is calculated out of ‘assessed patients’, apart from ‘assessed’, for which percentage is calculated out of ‘admitted’.

TABLE 5 - Screened patients

For assessed patients, percentages are calculated from ‘admitted’ patients.

Percentages are calculated from ‘eligible’ patients.

Percentages are calculated from ‘assessed’ patients.

Admitted, n	Assessed/screened, n (%)a	Eligible, n (%)	Consented, n (%)	%b	Registered, n (%)
Intervention
7067	6205 (87.8)	1256 (20.2)	490 (7.9)	39.0	451 (7.3)
Control
7303	6305 (86.3)	1419 (22.5)	534 (8.5)	37.6	479 (7.6)
Total
14,370	12,510 (87.1)	2675 (21.4)	1024 (8.2)	38.3	930 (7.4)

TABLE 6 - Screened patients by centre: non-registration

Percentages are out of ‘admitted’.

Percentages are out of ‘eligible’.

Percentages are calculated out of ‘assessed’.

Centre	Admitted, n	Assessed/screened, n (%)a	Eligible, n (%)	Consented, n (%)	Consented, %b	Registered, n (%)
Intervention
1	183	149 (81.4)	45 (30.2)	23 (15.4)	51.1	20 (13.4)
2	178	173 (97.2)	52 (30.1)	21 (12.1)	40.4	21 (12.1)
3	766	637 (83.2)	71 (11.1)	24 (3.8)	33.8	24 (3.8)
4	865	831 (96.1)	75 (9.0)	33 (4.0)	44.0	30 (3.6)
5	611	579 (94.8)	60 (10.4)	35 (6.0)	58.3	33 (5.7)
6	219	211 (96.3)	85 (40.3)	38 (18.0)	44.7	32 (15.2)
7	563	445 (79.0)	84 (18.9)	33 (7.4)	39.3	27 (6.1)
8	230	222 (96.5)	53 (23.9)	25 (11.3)	47.2	23 (10.4)
9	376	360 (95.7)	115 (31.9)	30 (8.3)	26.1	29 (8.1)
10	542	362 (66.8)	67 (18.5)	25 (6.9)	37.3	25 (6.9)
11	318	194 (61.0)	30 (15.5)	16 (8.2)	53.3	16 (8.2)
12	256	227 (88.7)	106 (46.7)	38 (16.7)	35.8	35 (15.4)
13	167	162 (97.0)	70 (43.2)	16 (9.9)	22.9	14 (8.6)
14	373	362 (97.1)	107 (29.6)	38 (10.5)	35.5	35 (9.7)
15	673	645 (95.8)	69 (10.7)	24 (3.7)	34.8	19 (2.9)
16	277	227 (81.9)	55 (24.2)	17 (7.5)	30.9	17 (7.5)
17	202	171 (84.7)	44 (25.7)	27 (15.8)	61.4	25 (14.6)
18	268	248 (92.5)	68 (27.4)	27 (10.9)	39.7	26 (10.5)
Total	7067	6205 (87.8)	1256 (20.2)	490 (7.9)	39.0	451 (7.3)
Control
19	314	310 (98.7)	74 (23.9)	39 (12.6)	52.7	34 (11.0)
20	929	720 (77.5)	135 (18.8)	32 (4.4)	23.7	27 (3.8)
21	244	235 (96.3)	84 (35.7)	36 (15.3)	42.9	31 (13.2)
22	260	225 (86.5)	44 (19.6)	20 (8.9)	45.5	18 (8.0)
23	514	403 (78.4)	78 (19.4)	26 (6.5)	33.3	25 (6.2)
24	373	284 (76.1)	54 (19.0)	13 (4.6)	24.1	12 (4.2)
25	688	566 (82.3)	88 (15.5)	44 (7.8)	50.0	38 (6.7)
26	189	182 (96.3)	65 (35.7)	25 (13.7)	38.5	23 (12.6)
27	285	270 (94.7)	92 (34.1)	35 (13.0)	38.0	35 (13.0)
28	354	265 (74.9)	85 (32.1)	27 (10.2)	31.8	23 (8.7)
29	775	747 (96.4)	103 (13.8)	37 (5.0)	35.9	30 (4.0)
30	311	285 (91.6)	56 (19.6)	17 (6.0)	30.4	13 (4.6)
31	188	158 (84.0)	46 (29.1)	28 (17.7)	60.9	26 (16.5)
32	679	529 (77.9)	107 (20.2)	29 (5.5)	27.1	25 (4.7)
33	513	473 (92.2)	58 (12.3)	21 (4.4)	36.2	17 (3.6)
34	188	188 (100)	67 (35.6)	37 (19.7)	55.2	35 (18.6)
35	261	249 (95.4)	84 (33.7)	32 (12.9)	38.1	32 (12.9)
36	238	216 (90.8)	99 (45.8)	36 (16.7)	36.4	35 (16.2)
Total	7303	6305 (86.3)	1419 (22.5)	534 (8.5)	37.6	479 (7.6)
Overall total	14,370	12,510 (87.1)	2675 (21.4)	1024 (8.2)	38.3	930 (7.4)

TABLE 7 - Characteristics of screened participants

Baseline characteristics	Intervention (N = 6205)	Control (N = 6305)	Total (N = 12,510)
Age (years): mean (SD)	74.1 (13.72), n = 6154	74.7 (13.04), n = 6224	74.4 (13.39), n = 12,378
Length of hospital stay (days): mean (SD)	27.3 (34.47), n = 5860	28.5 (31.65), n = 6076	27.9 (33.07), n = 11,936
Sex (n, %), male	2931 (47.2)	3036 (48.2)	5967 (47.7)
Ethnicity (n, %), white	5742 (92.5)	5886 (93.4)	11,628 (92.9)
Living circumstances (n, %):
Lives alone	2518 (40.6)	2404 (38.1)	4922 (39.3)
Co-habits	3013 (48.6)	3214 (51.0)	6227 (49.8)
Nursing/residential care home	368 (5.9)	323 (5.1)	691 (5.5)
Missing	306 (4.9)	364 (5.8)	670 (5.4)
Co-resident caregiver (n, %)	2356 (38.0)	2659 (42.2)	5015 (40.1)
The caregiver is the patients . . . (n, %):
Partner	2528 (40.7)	2519 (40.0)	5047 (40.3)
Daughter/son	1914 (30.8)	1673 (26.5)	3587 (28.7)
Other relative	421 (6.8)	375 (5.9)	796 (6.3)
Other non-relative	144 (2.3)	157 (2.5)	301 (2.4)
No carer	887 (14.3)	1078 (17.1)	1965 (15.7)
Missing	311 (5.0)	503 (8.0)	814 (6.5)

Sample size

In total, 930 patients and their caregivers were registered between February 2008 and February 2010 (Figure 4). The number of patient and caregiver dyads recruited per SRU ranged from 13 to 38 per centre. Although, this is above the original target sample size and below the revised sample size of 950 patients and caregivers, it still provides us with over 80% power to detect a clinically relevant difference of six points on the NEADL scale.

Analysis populations

Confirmation of written informed consent could not be obtained centrally for two patients (one in each of the two randomised groups); therefore, these patients were not included in the ITT population.

The ITT population contains a total of 928 patients (intervention n = 450, control  n = 478).

Trial conduct

A CONSORT (Consolidated Standards of Reporting Trials) flow diagram of trial progress and loss to follow-up for SRUs (clusters) and individual participants (patients and caregivers) is presented in Figures 5 (patients) and 6 (caregivers).

FIGURE 4.

Accrual into TRACS trial: monthly, overall, cumulative and target accrual.

FIGURE 5.

Clusters and patients.

FIGURE 6.

Clusters and caregivers.

Baseline data

Trial outcomes

Proxy responses

Proxy responses, where the entire questionnaire was completed on patients' behalf without consulting them, were anticipated. At baseline, 63 (6.8%) of questionnaires were via a proxy response; at 6 months and 12 months, 37 (5.5%) and 41 (6.5%), respectively, were via a proxy response (Table 28).

Primary outcomes

Sensitivity analyses

Secondary outcomes

Compliance

Intervention compliance and participant outcomes

The relationship between centre-level compliance with the intervention and mean patient NEADL score at 6 months is displayed in Figure 7 and mean caregiver CBS score at 6 months is in Figure 8. The tables show no evidence of higher levels of patient independence or lower level of caregiver burden in the SRUs with better levels of intervention compliance.

FIGURE 7.

Nottingham Extended Activities of Daily Living scores at 6 months and site compliance with intervention. Plot of means with SD bars.

FIGURE 8.

Caregiver Burden Scale scores at 6 months and site compliance with intervention. Plot of means with SD bars.

Time spent with caregivers

The overall time spent with caregivers was similar in the two groups [median of 118 minutes (range 10 to 900 minutes) in the intervention group (training records) and 133 minutes (range 1 to 1130 minutes) in control group (time logs)] (Table 43; Figures 9 and 10).

TABLE 43 - Time spent with caregiver

Time spent with caregiver (minutes)	Intervention	Control
n	214	180
Mean (SD)	136.5 (118.12)	200.3 (189.12)
Median (range)	117.5 (10–900)	132.5 (1–1130)
Missing	236	298

FIGURE 9.

Overall time spent with caregiver: intervention.

FIGURE 10.

Overall time spent with carergiver: control.

Patient safety: expected adverse events/serious adverse events

Process data

A summary of the process data collected by the trial manager on the visits before (2007) and after (2009/2010) recruitment at both control and intervention centres is provided in Appendix 8, Table 85. Data collected during recruitment (2008) did not vary from that collected after recruitment had completed. The organisational structure and the process of care on the majority of stroke units did not change significantly throughout the trial. Significant changes to the standard process of care of involving patients and their carers during the inpatient stay was seen in just one centre. Four other centres experienced significant changes to their organisational structure, which did not impact upon the process of care of involving patients and their caregivers. A brief description of these centres is provided in Appendix 8, Table 86.

Client Service Receipt Inventory and European Quality of Life-5 Dimensions completion rates

Tables 46 and 47 summarise CSRI and EQ-5D completion rates, respectively, at each assessment point. Both measures had similar completion rates at each assessment point and rates were balanced between the intervention and control arms. Table 48 characterises subsamples with both cost and outcome data at 6 months, a necessary requirement for inclusion in the CEAC-based analyses. Although differences were not explored statistically, the baseline characteristics of patients and caregivers with the necessary data at 6 months appeared similar to those of the full sample. Therefore, results based on those followed up are likely to generally be representative of the full sample.

Resource use

Resource-use differences were not compared statistically, firstly because the economic evaluation was focused on costs and cost-effectiveness and, secondly, to avoid problems associated with multiple testing. Therefore, resource-use patterns are described without statistical comparisons.

Tables 49–54 show resource use at each assessment point. These tables are limited to inpatient services plus other items used by at least 10% of responders in either trial arm at that time point. Full resource-use data are provided in Appendix 9. The length of the initial stroke admission was similar in both groups (see Table 50). Other resource use also appeared broadly comparable between the two groups at baseline, 6 months and 12 months. As could be expected, patients' use of inpatient services (other than the stroke admission), outpatient services, hospital physiotherapy and hospital occupational therapy increased during the post-stroke period compared with baseline. It is also interesting to note that caregiver's use of inpatient and outpatient services increased notably during the post-stroke period. With regards to community-based services, patients most commonly used dentist, chiropodist and optician services at all three time points. Services most used by caregivers were outpatient services, GP, practice nurse and repeat prescriptions. In comparison with formal care inputs, patient informal care rates were very high. Care to patients from non-resident informal caregivers increased at each time point.

Costs and quality-adjusted life-years

The mean cost of the LSCTC training and development was £39 (Table 55). This is the mean across the whole intervention group, including those allocated zero costs for either receiving no LSCTC inputs or with missing data regarding such inputs. The mean cost among only those receiving inputs was £82.

The mean cost of the initial stroke admission was similar between groups (mean difference £1243, 95% CI –£1533 to £4019; see Table 55). Total health and social care and societal costs were broadly similar between both randomisation groups at all assessment points (Tables 56–58), except that caregivers in the intervention arm had higher health and social care costs at 6 months (+£207, 95% CI £5 to £408; see Table 58). This difference was no longer present at 12 months and was not apparent when costs from the two assessment points were combined as 1-year costs. There were also no differences in QALYs for patients or caregivers at any of the assessment points (Table 59). Although the direction of the between-group difference is opposite in the patient (positive) and caregiver (negative) evaluations the mean differences are so small that they are essentially zero. It should also be noted that comparisons of costs and outcomes do not account for the correlation between these parameters.

Sensitivity analyses

All sensitivity analyses confirmed conclusions from the base-case conclusions except for sensitivity analysis 2, adopting the replacement cost approach for informal care, which led to the between group difference in mean societal costs for caregivers changing from £99 to –£831. However, although the direction of difference changed, the difference between groups remained statistically non-significant. Substituting the base-case value with this new value for the consideration of related ICERs (Table 60) would result in the intervention group dominating the control group based on CBS scores (the only case of the intervention group's dominance at the primary end point of 6 months) and an unlikely trade-off of lower costs for fewer QALYs.

Cost-effectiveness and cost–utility

Of the 16 cost–outcome combinations examined for the cost-effectiveness and cost–utility analyses, none was based on statistically significant between-group differences for both cost and outcome elements. The intervention group ‘dominated’ in seven combinations, the control group ‘dominated’ in five, the intervention group had both higher costs and better outcomes in nine, and the remaining three involved an unlikely trade-off of lower costs for worse outcomes. Where relevant, indicative ICERs are presented for information (Table 61), but these and the term ‘dominates’ should be interpreted with caution for most combinations given the small magnitude and lack of statistical significance in differences in costs or outcomes except for caregivers having higher health and social care costs at 6 months. ICERs ranged from £96 for an additional point improvement on the CBS based on 1-year health and social care costs for caregivers to £1.18M for an additional patient QALY based on their health and social care costs at 6 months.

Cost-effectiveness planes show that although differences in patient health and social care costs, NEADL scores (Figure 11) and QALYs (Figure 12) between the two groups do vary around the point estimates, they are strongly centred around zero, i.e. no difference in either costs or outcomes. In contrast, the caregiver cost-effectiveness planes suggest that health and social care costs are higher in the intervention group and, while CBS differences are clustered around zero (i.e. no difference; Figure 13), QALYs differences are clustered to the left of zero (i.e. lower in the intervention group; Figure 14).

Figures 15–18 show probabilities that the intervention group is cost-effective compared with the control group. In the patient evaluation, probabilities of cost-effectiveness were similar from both cost perspectives. Maximum probabilities of cost-effectiveness for the threshold ranges examined for each outcome measure were 51% at £2000 for an additional point improvement on the NEADL scale (see Figure 15) and 34% for an additional QALY (this was the same across all QALY thresholds examined) (see Figure 16). In the caregiver evaluation, probabilities of cost-effectiveness were higher based on the societal perspective than for the health and social care perspective, given that the latter costs were higher in the intervention group compared with the control group. Probabilities of cost-effectiveness were reasonable for CBS point improvements (see Figure 17), up to a maximum of 62% and 68% from the health and social care and societal perspectives, respectively, at the maximum threshold examined (£2000), although it is unknown what the willingness to pay for a CBS point improvement would be in practice. However, as for the patient evaluation, probabilities of cost-effectiveness based on QALYs were low (see Figure 18), not exceeding 2% from the health and social care perspective for the range examined. Thus, the intervention is unlikely to be considered cost-effective from either patient or caregiver perspectives at current policy thresholds of £20,000–30,000 per QALY gained.

FIGURE 11.

Patients: cost-effectiveness plane of incremental total health and social care costs and point changes on the NEADL scale at 6 months.

FIGURE 12.

Patients: cost-effectiveness plane of incremental total health and social care costs and QALYs at 6 months.

FIGURE 13.

Caregivers: cost-effectiveness plane of incremental total health and social care costs and point changes on the CBS at 6 months.

FIGURE 14.

Caregivers: cost-effectiveness plane of incremental total health and social care costs and QALYs at 6 months.

FIGURE 15.

Patients: probability that the intervention is cost-effective compared with the control at 6 months, from each cost perspective, for a range of willingness-to-pay values for an additional point improvement on the NEADL scale.

FIGURE 16.

Patients: probability that the intervention is cost-effective compared with the control at 6 months, from each cost perspective, for a range of willingness-to-pay values for an additional QALY.

FIGURE 17.

Caregivers: probability that the intervention is cost-effective compared with the control at 6 months, from each cost perspective, for a range of willingness-to-pay values for an additional point improvement on the CBS.

FIGURE 18.

Caregivers: probability that the intervention is cost-effective compared with the control at 6 months, from each cost perspective, for a range of willingness-to-pay values for an additional QALY.

Key findings

The TRACS trial was a pragmatic, multicentre cluster RCT of a complex intervention. The trial was designed to evaluate the effectiveness of the LSCTC (LSCTC compared with usual care when implemented in SRUs across the UK in different health-care settings, with different patient populations). The trial evaluated whether the LSCTC improved physical and psychological outcomes for patients and their caregivers after disabling stroke, and sought to determine if such a training programme was cost-effective. Eighteen SRUs were randomised to implement the intervention as a part of their standard practice, and 18 SRUs were randomised to continue usual practice. A total of 928 patient and caregiver dyads were recruited into the trial: 450 in the intervention arm and 478 in the control arm.

Primary outcomes

Stroke patients attending intervention SRUs did not demonstrate a clinically significant improvement in functional independence compared with stroke patients attending the control SRUs at 6 months post registration. The burden for caregivers of stroke patients attending intervention SRUs was not significantly different to that of caregivers of stroke patients attending the control SRUs at 6 months. There was no difference in patient recovery or caregiver burden between the intervention and control groups.

Secondary outcomes

There were no statistically significant differences in stroke patients' mood, health state, functional ability and health-related quality of life or in the number of deaths, hospital readmissions or institutionalisations between the intervention and control arms at both 6 and 12 months post registration. Similarly, there were no statistically significant differences in caregivers' social restriction, mood, health state, or any difference in deaths, hospitalisation and institutionalisation at 6 or 12 months for those caregivers of patients attending intervention SRUs compared with those attending control SRUs. There was no difference in the outcome of patients' and caregivers' physical and psychological well-being between the intervention and control groups.

Economic evaluation

The economic evaluation suggests that from a patient perspective, health and social care costs, societal costs and outcomes are similar for the intervention and control groups at 6 months, 12 months and over 1 year. CEACs based on the net benefit approach, which accounted for uncertainty around point estimates of differences and potential willingness-to-pay thresholds of up to £2000 per point improvement on the NEADL scale and up to £50,000 for an additional QALY, suggest that the intervention group is less likely to be cost-effective than the control group.

From a caregiver perspective, societal costs and outcomes are similar between the two groups at 6 months, 12 months and over 1 year. Health and social care costs were on average £207 higher (95% CI £5 to £408) in the intervention group at 6 months, but this difference was no longer apparent at 12 months or over 1 year. Caregiver CEACs suggested that the intervention group is less likely to be cost-effective than the control group based on QALYs for thresholds up to £50,000 per QALY gain, but, based on the CBS, it has between 53% and 63% probability of cost-effectiveness for thresholds of £500 to £2000 per point improvement on this scale from the health and social care perspective and 63–68% from the societal perspective. The value of a one point change in this measure and the willingness to pay for it from a policy-making viewpoint is unclear.

Informal care costs were considerable in size and notably increased total care costs. However, conclusions were broadly similar from both health and social care and societal perspectives.

The ability of the EQ-5D to detect changes in quality of life in these patient and caregiver groups is unclear and needs further exploration.

Safety

Training of caregivers was not detrimental to patients. No unexpected SAEs occurred in the trial, and the number of falls between registration and discharge from the SRU were minimal in both control and intervention arms.

Comparison of Training Caregivers After Stroke with the single-centre study

The LSCTC was evaluated in a single-centre RCT of 300 patients by Kalra et al. 19 This study reported a significant cost reduction for those patients treated using the LSCTC, as well as a significant reduction in caregiver burden, and improved quality of life and mood for both patients and their caregivers. The TRACS trial assessed the LSCTC using a multicentre, pragmatic design, wherein the LSCTC was implemented as a part of standard practice and delivered by all members of the MDT, and its effectiveness was assessed across different health-care settings and a larger heterogeneous population. The large multicentre pragmatic RCT did not replicate the findings of the single-centre study. The LSCTC does not provide any benefit to patients' long-term recovery or psychological well-being, nor does it reduce caregivers' burden or enhance psychological well-being compared with usual care. The LSCTC was found to be no more cost-effective than standard care.

It is important to note that the recruitment of patients in the single-centre study took place almost 10 years ago. Since that time stroke guidance has highlighted the importance of involving caregivers throughout the stroke patients stay on the SRU. 39,75 Comparison of the caregiver burden scores in the single-centre study at 3 months (intervention median = 43; control median = 51) and the TRACS trial at 6 months (intervention mean = 46.3; control mean = 45.7) suggest that caregivers may receive more support, resulting in less burden. However, although some of the differences identified in the single-centre study may have been lost as general stroke care has improved, we believe that it is unlikely that standard care has improved to such a degree that caregivers needs have been successfully met. Indeed, the caregiver burden scores in both the intervention and control arms indicate a level of burden that still needs addressing, with a score of 45 relating to a high level of caregiver burden.

Strengths and limitations of the study

The TRACS trial is the largest stroke rehabilitation trial completed to date (worldwide), with 930 patient and caregiver dyads registered to the trial. It successfully recruited to target, demonstrating that large, multicentre cluster RCTs of stroke rehabilitation are feasible.

The TRACs trial was one of the first large multicentre stroke rehabilitation trials to use the new SRN. The establishment of the local SRNs helped to determine the geographical regions selected for participation in the TRACS trial. The availability of SRN researchers in hospitals across entire regions allowed TRACS centres to be recruited from an excellent spread of locations and health service types – from rural community hospitals in the South West Peninsula to large urban acute hospitals in central London. The SRN researchers assisted with the successful minimisation of selection bias in the TRACS trial, allowing recruitment by research staff that was entirely independent of the clinical MDTs. The invaluable support of the SRN no doubt ensured the successful recruitment of clusters and participants to target in the TRACS trial. In addition, the design of the TRACS trial has been a pioneer in bridging the chasm between clinical staff and research, by involving members of the MDTs in the research process. Many of the PIs in TRACS were senior therapists/nurses, and all participating MDTs have received an insight into the research process as a consequence of taking part in this pragmatic trial.

Internal validity

Generalisability

Four disparate geographical regions ensured a good representation of different health-care settings (acute/community hospitals in rural and urban settings) and different SRU set-ups (combined acute and rehabilitation, and rehabilitation). The eligibility criteria in TRACS were kept to a minimum in keeping with the pragmatic trial design, to ensure that a representative stroke patient population was recruited. Patients with language and cognitive impairment were included, and there was no ‘cut-off’ level of age or disability. The results of the TRACS trial should be generalisable to all stroke patients returning home requiring support from informal caregivers, and to these informal caregivers, in SRUs across the UK.

Implementation of a complex intervention

As a pragmatic trial, the intervention was implemented as it would have been with any service initiative within the NHS. The challenges of implementation were considered carefully in designing the delivery of the training, and included choosing a method that (1) would be acceptable and feasible to MDT staff and NHS management; (2) could easily be replicated in SRUs across the UK at the end of the trial; and (3) would allow successful implementation of the LSCTC programme. Training was provided at two national training days for each intervention site, during which practical issues of implementation were discussed. This was supported by a training manual, CD and training records. The SRUs were asked to implement and embed the intervention on the wards over a 4- to 6-month period prior to the start of patient recruitment. The intent was that the training was cascaded down by staff who attended the training day to other staff on the wards. It may be that this commonly used ‘cascade’ method was not as effective as we would have wished and that other methods for implementing such service improvements should be considered.

The lack of in-depth piloting of the modified LSCTC and its implementation may be viewed as a weakness of the present study. Although piloting was conducted on a sample of caregivers in each of the intervention centres prior to the second training day, this may not have allowed sufficient insight into the complexities involved in successfully implementing a complex intervention as a part of standard care.

Intervention compliance

The intervention was accepted well by the staff attending the training days, with recognition of the importance of each of the core competencies, demonstrating ‘face validity’ for the intervention. A component of the intervention was the completion of the TRACS training record (see Appendix 1). Completed training records were returned to the trial manager and included as a standard monitoring report to the TMG and TSC. This enabled us to monitor the intervention delivery and the compliance for each patient in terms of competencies obtained. We felt that this level of monitoring was in keeping with the pragmatic trial design. Concerns about compliance were explored by the trial manager. In two instances for which compliance was very poor, letters were sent from the chief investigator to the local PIs and these were followed up with visits from the trial manager. Higher-level monitoring, for example, by check visits to the intervention units, would be potentially threatening to participating staff, probably observe unrepresentative practice, provide only a few observations and be research resource intensive.

The completion and return of this record varied across the SRUs, with an overall compliance rate of 43.6% (range 0.0–92.9%). These data indicate that some units did not implement the LSCTC as robustly as envisaged; however, half of the participating centres had a compliance rating of over 60% (one-third > 75%). The measure of compliance, provided by the TRACS development team, was defined as completion of all six mandatory components of the training or sign-off of the training record by clinical staff. This may be a strict assessment of compliance. Compliance rates are compatible with other trials evaluating complex interventions.

To our knowledge, no other study has evaluated a novel, complex intervention delivered by the whole MDT on a stroke unit. Moreover, few studies have assessed the compliance of the interventions that they are evaluating. Four previous studies15,17,19,76 have investigated inpatient interventions delivered by a specialist individual or team on a SRU. However, only one of these studies measured and reported compliance with the intervention. In this study, completed by Larson et al. ,15 compliance (defined as patient attendance at a minimum of five out of the six education sessions offered) was 50%. It is disappointing that some centres do not appear to have implemented the intervention consistently, but it is emphasised that the compliance analysis show no evidence of higher levels of patient independence or lower level of caregiver burden in the SRUs with better levels of intervention compliance.

Interpretation of results

The observations conducted on the SRUs in the process evaluation study reflected the completion rate of training records in the main trial – the LSCTC was observed successfully in some SRUs, some of the time in other SRUs, and not at all in one SRU. The key findings of the process evaluation study were that the LSCTC was only one of a number of competing priorities for SRU teams. Members of staff were trained to deliver the LSCTC and then expected to train their teams but training did not reach all staff, particularly nurses. Therapy staff were the most actively involved with training delivery in the observed units. Different disciplines often worked separately with caregivers rather than the MDT working together as a team. Caregiver training was typically delivered very late in the inpatient stay. Staff concern with reducing risk and safe discharge often meant caregiver training involved passive observation and not active engagement and practice. For caregivers, the stroke was a shock, this impacted on their understanding of what had happened and readiness to participate in training. After discharge, caregivers' recall of information and training was very limited.

The LSCTC was implemented using cascade training. Although training did occur in all centres, the staff attending varied, and the time available to cascade the training was relatively brief. In general, nurses received less training, and for a shorter amount of time. Training tools were left on each unit to help support new members of staff, but the process evaluation study suggests that these resources were not well utilised.

Stroke survivors' abilities could change rapidly once at home, so home visits and early supported discharge schemes provided opportunities for in situ training, which was valued. Caregivers perceived the training to be important, but it only addressed one of many areas of adjustment with which to cope.

It may be speculated from process evaluation work that the intervention was more robustly implemented when key members of the MDT took ownership of the LSCTC and training records. This form of working is more similar to that used in the single-centre study, for which the training was completed by a small, highly experienced, team.

Economic evaluation

The economic evaluation from the patient perspective suggests that the intervention and control groups have similar costs and outcomes at all of the assessment points considered. From the caregiver perspective, the intervention group had higher health and social care costs at 6 months (for equivalent outcomes). Costs and outcomes over the longer term and from the societal perspective were similar, and illustrate that increasing the breadth and length of such an evaluation can provide alternative conclusions. However, even 1 year is a relatively short time horizon in the context of a lifetime and it is unclear whether or not our findings would hold over patient and caregiver lifetimes.

The cost findings are in contrast with those from the single-centre study,19,29 which found a statistically significant difference in total patient health and social care costs over 1 year of £4043 in favour of the intervention group, largely due to a lower length of stay (–12 days) for the initial stroke admission. Total patient health and social care costs over 1 year in the present study were £563 lower in the control group (although not statistically significant) and we found no difference in the average length of stroke unit stay for the initial stroke admission (45 and 42 days in the intervention and control groups, respectively), which interestingly was similar to that for the control group in the single-centre study (43 days). It is unclear whether the intervention, or the way it was implemented, simply did not impact on discharge practices in the study stroke units, or whether or not other factors (e.g. care standards) differed over time and place to reduce the potential for earlier discharge that existed in the single-centre study.

With regards to caregiver costs, we cannot discount the possibility that providing caregivers with more structured training increased their use of health and social care resources and their provision of informal care to the patient as an appropriate response to their and the patients' needs. However, if this was the case, any potential benefits of this were not evident in the outcome measures we examined.

Examining cost and outcome differences based on point estimates gave a mixed set of conclusions across the various time point and perspectives but the meaningfulness of these is unclear given the lack of statistically significant differences in costs and outcomes except for higher caregiver health and social care costs at 6 months. There were no differences in patient or caregiver QALYs at 6 months, 12 months or over 1 year, and CEACs based on patient and caregiver QALYs suggested that the intervention group was less likely to be cost-effective compared with the control group. The single-centre study on which this was based19,29 also failed to detect any QALY differences. As we used no additional health-related quality-of-life measure, it is unclear whether there truly were no differences or whether the EQ-5D was unable to detect changes in quality of life. Any such limitation of the EQ-5D may be addressed in future research with the new five-level version (EQ-5D-5L), which has been developed to improve the instrument's sensitivity and to reduce its ceiling effects.

Patient CEACS based on the NEADL score suggested equivalent probabilities of cost-effectiveness for the two groups but caregiver CEACs based on the CBS suggested 53–68% probability of cost-effectiveness for thresholds of £500–2000 per point improvement on the CBS. However, the value of a one point change in this measure and the willingness to pay for it from a policy-making viewpoint is unclear.

The cost of developing the intervention and training ward staff in its use was estimated at an average of £39 per patient/caregiver dyad across the whole intervention group. The average cost was higher at £82 when considering only those known to have received some caregiver training. This is relatively inexpensive in the context of the average cost of an acute stroke bed-day (£294) and the average stroke admission cost in this sample (£13,127 and £12,471 in the intervention group and control group, respectively). We did not include the cost of delivering the intervention to caregivers to avoid double-counting admission costs. However, this exclusion is unlikely to have affected the findings, first because these costs are likely to be small in the context of total care costs and, second, because there was little difference in the amount of time spent with caregivers in each group.

Although the difficulties of reliably assessing informal care inputs is well known, the study suggests that the informal care burden for caregivers of stroke patients is sizeable. Patient care costs were 70% higher in both intervention and control groups at 6 months when informal care costs were added in. Caregiver costs were 21% and 34% higher in the intervention and control groups, respectively, when the costs of the inputs they provided were considered. These care inputs and their costs remained high at 12 months, thus the informal care burden is both significant in size and duration. The impact of patients' strokes on caregivers' own health is further evidenced by the noticeable increase in their use of community- and secondary-based care in the post-stroke period compared with baseline.

There were two main limitations with the economic evaluation, both of which are common to evaluations of this type. First, resource-use data were collected retrospectively by self-report as this was the most efficient way of collecting data for a large sample, across a large geographical area and from a broad perspective incorporating primary health care, secondary health care, social care and informal care. Such data may be subject to recall bias, but a trade-off between reliability and scope was necessary given the trial design. Second, the cost-effectiveness and cost–utility analyses were performed on subsamples of cases with available data for both costs and relevant outcomes. However, overall follow-up rates were very good given the features of the trial (multicentre, older participants, follow-up period 6/12 months after the acute event), and sensitivity analyses that imputed missing costs and QALY data suggested that findings from the incomplete sample analyses were fairly robust.

Overall evidence

A recent Cochrane review has examined the effectiveness of non-pharmacological interventions for caregivers of stroke survivors in reducing caregiver burden or enhancing caregiver well-being. 14 Eight relevant studies were found that assessed information and support interventions, psychoeducational interventions and caregiver training. The combined sample size of the eight studies was 1007. The conclusions of this review was that it is not, at present, possible to determine the usefulness of information and support interventions, or psychoeducational interventions on reducing caregiver burden, or enhancing their psychological well-being or health-related quality of life. The results of the caregiver training programme in the single-centre study looked promising, appearing to reduce caregiver burden, depression, and improve health-related quality of life for caregivers. The purpose of TRACS was to see if this caregiver training programme – the LSCTC, continued to show benefits to caregivers and stroke patients if it was implemented as a part of standard practice in SRUs across the UK. The TRACS trial (n = 928) has almost doubled the sample size of the Cochrane Review, providing conclusive evidence that there is no difference between the LSCTC and usual care with respect to stroke patients' recovery, caregivers' burden, or other physical and psychological outcomes, nor is it cost-effective when compared with usual care. Caregivers need more than just an inpatient structured training programme to improve the patients and their own outcomes.

The findings of the process evaluation study completed in parallel with TRACS suggest that caregiver training while the stroke patient is in hospital may not be the optimal time to complete such training. The LSCTC competes with other priorities for MDT staff, and caregivers are experiencing stress, making any such training less effective at that time point. Caregiver training in the community, by a dedicated experienced team, might be more beneficial for patients and their caregivers.

Of the eight studies reviewed by Legg et al. ,14 three interventions were completed while the stroke patient was still in hospital, one was completed both as an inpatient and continued once at home, two were longer-term community interventions and the location of delivery of two were unclear, although both recruited patients in the acute inpatient setting. The timing/setting of the intervention in these studies did not have any clear influence on the effectiveness of these interventions.

Implications for clinical practice

The intervention evaluated had reported benefits in a previous single-centre evaluation but these benefits have not been replicated in this large, multicentre trial in which the intervention was evaluated in a range of settings with a greater diversity of patient populations. There was no difference between the LSCTC and usual care with respect to improving stroke patients' recovery, reducing caregivers' burden, or improving other physical and psychological outcomes, nor is it cost-effective when compared with usual care.

Compliance with the intervention varied across stroke units but analysis demonstrated no link between the degree of compliance and associated patient or caregiver outcomes, indicating that a dose effect is unlikely. The LSCTC provided a structured framework for caregiver training. It is possible that the immediate post-stroke period, when potential caregivers are coming to terms with their new situation, may not be the ideal time for the delivery of structured training. The intervention approach might be more relevant if delivered after discharge by community-based teams.

Implications for research

The TRACS trial is the world's largest completed stroke rehabilitation trial. We have demonstrated for the first time that this methodology can feasibly be implemented in stroke rehabilitation research, which represents a major step forward in research methodology for this large client group. The TRACS trial fills some of the gaps in evidence of the recent Cochrane review. Despite the promising results of the single-centre study, the results of TRACS provides conclusive evidence that there is no difference between the LSCTC and usual care on stroke patients' functional independence, caregivers' burden, or other physical and psychological outcomes, nor is it cost-effective when compared with usual care. Caregivers need more than just an inpatient structured training programme to improve the patients and their own outcomes.

Future studies should consider carefully the optimal delivery of any caregiver interventions – whether or not the staff have the time and opportunity to implement the intervention, and when is the optimal timing of the intervention for the caregiver. Caregiver interventions may be more effective in the post-acute phase, once patients are back home and caregiving has become a reality.

TRACS Trial Collaboration

Chief Investigator A Forster.

TRACS Trial Management Group A Forster, J Young, J Dickerson, A Patel, L Kalra, D Smithard, J Nixon, M Knapp, I Holloway, S Anwar and A Farrin.

Trial Steering Committee P Langhorne (Chairperson), A Drummond, K Hood and H Rodgers.

Data Monitoring and Ethics Committee D Wade (Chairperson), A Bowen and J Norrie.

LSCTC Development Team Ann Melbourn, Jayne Steadman and Margreet Wittink.

South East TRACS Coordinator Emily Jay.

Participating SRUs (no. of dyads recruited): Airedale General Hospital (34), S Mawyer, S Williamson and P Garnett; Blackpool Victoria Hospital (27), K Waywell, J Howard, H Goddard, S Preston, J McIlmoyle and M J O'Donnell; Budleigh Salterton Hospital and Crediton Hospital (31) T Ayers, L Barron and A Bowring; Calderdale and Huddersfield NHS Foundation Trust (20), G Seebass, I Shakir, C Button, J Greig, D Nicholson, M Barber, T Smith and P Finn; Camborne Redruth Community Hospital (21), S Coltman, G Courtald and S Connife-Jones; Cumberland Infirmary (24), P Davies, C Hagon and L Pearce; Darent Valley Hospital (30), S Hussein, T Daniels and J Hancock; Dewsbury and District Hospital (33), P Datta, G Bateman and K Mallinder; Ellesmere Port/Countess of Chester Hospitals (32), K Chatterjee, C Kelly, K James and C Child; Fairfield General Hospital (18), K Kawafi, A Bell, S Moulton and C Curley; Harrogate District Hospital (25), S Brotheridge, J Strover, A Norton and A Wray; Salford Royal NHS Foundation Trust (27), S Moss, J Stevens, R Marsh and E Barberan; Kent and Canterbury Hospital (12), H Baht and B Bourne; King's College Hospital (38), L Kalra, E Jay, C Potter, A Davies, A M Murtagh and M Fitzpatrick; Macclesfield District General Hospital (23), C Davison, H Rooney, B Simpson and S Bailey; Mile End Hospital (23), P Gompertz, T Sachs, H Kariuki; Mount Gould Hospital (35), C Gatehouse, B Hyams, C Brown and R Truscott; Newton Abbot Hospital (29), R Allison, N Wedge, S Thomas, A Beck and H Nott; Royal Oldham Hospital (25), E Walker and S Parnell; Pinderfields General Hospital (30), M Carpenter, A Needle, C Holland, V Newton and AM Doran; Queen Elizabeth The Queen Mother Hospital (13), G Guna, J Idris and S Jones; Rochdale Infirmary (26), S Powell, N Thomas, R Namushi and N Saravanan; Rotherham District General Hospital (26), J Okwera, K McNulty and L Strachan; Scarborough General Hospital (16), J Paterson, R Rose, S Jamieson, M Sellers, A Davidson, M Foden and L Beadle; Sheffield Teaching Hospitals NHS Foundation Trust (23), A Jones, R Palmer, S Ross and C Draper; Sussex Rehabilitation Centre, Brighton and Sussex University Hospitals (BSUH) (35), A Harper, J Breeds, G Spurling and K Stephenson; Southport and Ormskirk Hospital NHS Trust (17), H Duff, M Marshall, S Wright, R Lawrence; St Helens & Knowsley NHS Trust (14), V Gowda, S Dealing, R Brown, A Ledger, G Fletcher and R Irving; St Luke’s Hospital (35), C Patterson, L Johnston, J Stevens, D Walshaw, A Adams and S Oxley; St Thomas’ Hospital (19), A Rudd; University Hospital Aintree (35), Helen Martin, E Bacabac and V Sutton; West Cumberland Hospital (32), O Orugun and R Jolly; Weston General Hospital (17), G Saunders and H Dymond; William Harvey Hospital (25), D Smithard and A Reid; Williton Hospital (35), S Glanfield and L Caudwell; York Teaching Hospital NHS Foundation Trust (26), J Coyle, E Iveson, C Rhymes, M Keeling, S Craigie and J Heppel.

Contributions of authors

Anne Forster (Professor of Stroke Rehabilitation): the conception and design of the study, acquisition and interpretation of data and drafting of this paper.

Josie Dickerson (Trial Manager): the acquisition and interpretation of data and drafting of this paper.

John Young (Professor of Elderly Care): the design of the study, interpretation of data and commenting on the draft of this paper.

Anita Patel (Reader, Health Economics): the design of the study, analysis and interpretation of data, and drafting of this paper.

Lalit Kalra (Professor of Stroke Medicine): the conception and design of the study, and commenting on the draft of this paper.

Jane Nixon (Deputy Director, CTRU): the design of the study, acquisition of data and commenting on a draft of this paper.

David Smithard (Stroke Consultant): the design of the study and commenting on the draft of this paper.

Martin Knapp (Professor of Health Economics): the design of the study and commenting on the draft of this paper.

Ivana Holloway (Trial Statistician): the analysis of the data and drafting of this paper.

Shamaila Anwar (Senior Trial Co-ordinator): data acquisition and commenting on a draft of this paper.

Amanda Farrin (Director of Health Sciences Division, CTRU/Principal Statistician): the design of the study, acquisition, analysis and interpretation of data, drafting of this paper and statistical guarantor.

Disclaimers

This report presents independent research funded by the National Institute for Health Research (NIHR). The views and opinions expressed by authors in this publication are those of the authors and do not necessarily reflect those of the NHS, the NIHR, the MRC, NETSCC, the HTA programme, the EME programme or the Department of Health. If there are verbatim quotations included in this publication the views and opinions expressed by the interviewees are those of the interviewees and do not necessarily reflect those of the authors, those of the NHS, the NIHR, NETSCC, the HTA programme, the EME programme or the Department of Health.

Publication

Forster A, Young J, Nixon J, Kalra L, Smithard D, Patel A, et al. A cluster randomised controlled trial of a structured training programme for caregivers of inpatients after stroke (TRACS Trial). Int J Stroke 2012;7:94–9.

Forster A, Dickerson J, Young J, Patel A, Kalra L, Nixon J, et al. A structured training programme for caregivers of inpatients after stroke (TRACS): a cluster randomised controlled trial and cost–effectiveness analysis [published online ahead of print 18 September 2013]. Lancet http://dx.doi.org/10.1016/S0140-6736(13)61603-7