Goal-oriented cognitive rehabilitation for early-stage Alzheimer's and related dementias: the GREAT RCT

Concept and terminology

Most people are familiar with the concept of rehabilitation following injury or illness, aiming to return the person to a former state of functioning or, if this is not possible, to enable the person to adjust to altered capacity and function at the best possible level given the residual impairments. 18 In the acute phase during recovery, intensive rehabilitation in specialist settings may be indicated, whereas at later stages, a less intensive community-based approach may be appropriate. Rehabilitation may target physical or cognitive functioning. The concept of rehabilitation is equally relevant for people with progressive impairments, who may benefit from episodes of community-based rehabilitation at various stages or as circumstances change.

In community settings, the term ‘rehabilitation’ is now sometimes replaced by ‘reablement’, which is derived from the same root and essentially shares the same meaning, but is perhaps viewed as a more readily understandable label. Rehabilitation can also be considered as being related to the concept of ‘tertiary prevention’, which is used in public health. We will use the term ‘rehabilitation’ here. The key point is that rehabilitation (or reablement) is grounded in a philosophy of enablement, which reflects a positive approach to finding solutions and encouraging optimal functioning. This philosophy emphasises a collaborative approach in service delivery, which can be summarised as ‘doing with’ rather than ‘doing for’ or ‘doing to’21 and which translates into specific individualised interventions aimed at optimising functioning.

Application to dementia

Living with dementia means living with disability resulting from cognitive impairment. The United Nations Convention on the Rights of Persons with Disabilities sets out a range of rights, including the right to be able to attain and maintain as much independence as possible through the assistance of comprehensive rehabilitation services [Article 26 (1)]. 22 For people with dementia, rehabilitation has been proposed both as an overarching principle of care and service provision, reflecting the aim of enabling optimal functioning,3,20,23,24 and as a specific intervention approach that aims to support the attainment of practical functional goals.

The principles of rehabilitation can be applied flexibly to address different types of need at various stages of dementia. These might include needs resulting from the impact on functioning of cognitive, behavioural, emotional, communication-related, relational, social or physical changes or difficulties. CR for people with dementia focuses primarily on the effects on functioning of the cognitive, behavioural and social communication impairments that form the core symptoms of dementia and the emotional and relational impact of these. A person might have several episodes of rehabilitation over time as needs change or in response to particular circumstances, such as being discharged after a period of hospitalisation. CR is distinct from physical rehabilitation, but it is important to note that people with dementia can benefit from exercise-based interventions and should of course have access to intensive physical rehabilitation when needed following injury or illness. 25

Rehabilitation, with its focus on optimising functioning, provides a highly relevant framework for supporting people with dementia and their carers, and for designing interventions to meet their needs. However, the term ‘cognitive rehabilitation’ (or ‘neuropsychological rehabilitation’), although familiar in areas, such as brain injury research, needs to be better understood in the dementia field. ‘Rehabilitation’ signifies that the intervention aims to enable people to function optimally given any impairments they may have and ‘cognitive’ signifies that the intervention specifically addresses the impact of cognitive impairment on functional ability. This impact may be the direct result of the cognitive impairment (e.g. difficulty remembering) or may reflect secondary effects, such as loss of confidence. CR has a different focus and takes a different approach to other interventions that include the term ‘cognitive’ in their titles. 15 CT and cognitive stimulation focus on cognitive function and target specific domains or global functioning, respectively; the term CR is sometimes incorrectly used to describe these types of interventions, or as an umbrella term for them. Cognitive or cognitive–behavioural therapy targets unhelpful or self-defeating thought patterns that may underlie mental health difficulties or adjustment issues. CR is distinct from all of these other approaches, which include the term ‘cognitive’, and should not be confused with them.

Ethics

The study was reviewed by Wales Research Ethics Committee (REC) 5, which issued a favourable opinion on 25 June 2012 (reference number 12/WA/0185), and was approved by the Bangor University School of Psychology REC. Based on findings from the pilot trial, it was expected that participants and carers who were allocated to receive the CR intervention would derive some benefits, whereas people who were allocated to receive TAU would not be harmed by this allocation. As there was no existing large-scale evidence about the effects of CR, it was not considered to be unethical to withhold the treatment from those allocated to receive TAU. In addition, based on existing evidence, there were no known risks associated with CR. However, trial researchers and therapists were trained to be alert to any concerns about participants’ well-being and to refer any serious concerns to the clinician responsible for the person’s care whenever possible, with the knowledge and permission of the person and their carer.

Governance

The trial was sponsored first by Bangor University (from its start on 1 October 2012 to 28 February 2015) and then, following transfer of the co-ordinating centre, by the University of Exeter (from 1 March 2015 until its completion on 31 December 2016). The governance was overseen by a Trial Steering Committee (TSC), which included two Alzheimer’s Society research volunteers who were former carers and a sponsor’s representative, and by a Data Monitoring and Ethics Committee.

Trial registration

The trial was registered with Current Controlled Trials under reference ISRCTN21027481.

Trial protocol

The trial protocol was published in 2013. 73

Eligibility

Recruitment

Participants were recruited through NHS services, such as memory clinics and old age psychiatry teams, carer and patient support groups, led by NHS staff, support groups and networks run by the Alzheimer’s Society and Join Dementia Research. Recruitment to the trial covered a 36-month period from 1 April 2013 to 31 March 2016.

Potentially eligible individuals were initially identified by either GREAT researchers or National Institute for Health Research (NIHR) Clinical Research Network staff in England and Health and Care Research Wales staff in Wales (previously the National Institute of Social Care and Health Research Clinical Research Collaboration). GREAT researchers and research network staff visited clinics to provide information and ascertain interest in participating. Research network staff also identified possible participants through note-screening and wrote to them on behalf of the responsible clinician; they were invited to indicate their interest by sending a reply slip directly to a GREAT researcher, who then made contact by telephone, sent written information and made a further telephone call to ascertain the participant’s willingness to continue.

When a possible interest in participating was identified, a GREAT researcher visited the potential participant and carer to explain the study in detail, answer any questions, recheck eligibility and ensure that the person with dementia had the capacity to consent. Informed consent from both the person with dementia and the carer was taken at this visit or, if either person required more time to decide, at a subsequent visit. As participants were in the early stages of dementia, we expected that they would continue to have the capacity to consent throughout the period of participation. However, on entry to the trial, participants were asked whether or not, in the event that they did lose capacity, they would wish to continue to be included in the trial and to have their data used in the analysis.

Obtaining informed consent at the start of the trial was only the beginning of an ongoing process. This is particularly crucial in an intervention of this kind, which requires the participant’s active engagement. The trial researchers and therapists were trained to monitor ongoing consent and identify and respond to any indication of a possible withdrawal of consent.

Locations

The trial was conducted in eight NHS sites throughout England and Wales. These were:

• North Wales – Betsi Cadwaladr University Health Board (Bangor site)

• South Wales – Cardiff and Vale University Health Board (Cardiff site)

• London – South London and Maudsley NHS Foundation Trust, with recruitment supported by King’s College Hospital NHS Foundation Trust, Guy’s and St Thomas’ NHS Foundation Trust, St George’s Healthcare NHS Trust, and Oxleas NHS Foundation Trust (London site)

• South East England – Kent and Medway NHS and Social Care Partnership Trust (Kent site)

• South West England – RICE (Research Institute for the Care of Older People), Bath, with recruitment supported by Royal United Hospitals Bath NHS Foundation Trust and by general practitioner (GP) practices within the Wiltshire NHS Clinical Commissioning Group [(CCG) Bath site]

• West Midlands – Birmingham and Solihull Mental Health NHS Foundation Trust, with recruitment supported by Black Country Partnership NHS Foundation Trust and Heart of England NHS Foundation Trust (Birmingham site)

• North West England – Manchester Mental Health and Social Care Trust, with recruitment supported by Pennine Care NHS Foundation Trust (Manchester site)

• North East England – Northumberland, Tyne and Wear NHS Foundation Trust (Newcastle site).

Setting

All assessments and intervention sessions were conducted in participants’ own homes.

Sample size

Power calculations were based on findings from the pilot trial. Improvement in goal performance was assessed in the pilot trial, with the rating scale of the COPM,38 which is equivalent to the rating of goal attainment used as the primary outcome measure in GREAT. The effect size for improvement in goal performance was large, with a standardised effect of > 1 at the post-intervention assessment. However, it was important to be able to detect at least medium effect sizes of 0.3 for both the primary and secondary outcomes. To achieve 80% power to detect a medium effect size of 0.3, with alpha 0.05, in primary and secondary outcomes, 175 people with dementia, together with their carers, were needed to complete the trial in each treatment arm. Attrition in the pilot trial was 19% overall, but as the rate could be higher in a longer multicentre trial, we adopted a more conservative estimate of 27%. Allowing for the potential attrition of 27%, it was necessary to randomise 480 people with dementia, each with a carer.

To meet this target, we calculated that each centre would need to recruit three participants per month over 27 months, a total of 80 participants per site. Experience suggested that one in three of the people with dementia who were identified as eligible and invited to participate would be successfully recruited; thus, each month, nine potentially eligible participants would need to be approached in each centre.

Randomisation

Participants were individually randomised following consent and a baseline assessment. Randomisation was triggered by the trial researchers on completion of the baseline assessment through secure web access to the remote randomisation centre, the North Wales Organisation for Randomised Trials in Health (NWORTH) Clinical Trials Unit (CTU) at Bangor University. In this system, which was maintained and monitored independently of the trial statistician or other trial staff, the randomisation was performed by dynamic allocation76 to protect against subversion while ensuring that the trial maintained good balance to the allocation ratio of 1 : 1, both within each stratification variable and across the trial. Participants were stratified by centre, sex, age (< 75 years vs. ≥ 75 years) and MMSE score (< 24 points vs. ≥ 24 points). For validation purposes, additional information was recorded, including the participant’s trial number, initials and date of birth and the details of the person requesting the randomisation. Group allocation was notified to the trial therapists.

Blinding

The trial researchers were blind to participants’ group allocation. The importance of maintaining blinding was emphasised in the training of both the researchers and the therapists. The potential for unblinding could arise through the researchers’ contact with participants at the 3- and 9-month assessments and through day-to-day contact between the researchers and the therapists at each site.

To address the potential for unblinding through day-to-day contact between researchers and therapists at each site, we ensured that they were based in different offices and did not share telephones or printers. Arrangements for the follow-up assessment visits by the researcher were made by the therapist for all participants. As the participants and carers could not be blinded to their group allocation, they were specifically asked not to comment at post-intervention and follow-up assessments on the nature of their involvement in the study and not to reveal to the researcher whether or not they had been visited by the therapist. This was explained by the researcher during baseline visits and included in the written information given to participants, and it was reiterated by the therapists when they contacted participants to confirm the dates of the 3- and 9-month assessments.

Following each assessment at the 3- and 9-month points, the blinded researcher noted to which condition s/he thought the participant had been allocated and how certain s/he was of the allocation.

Owing to the nature of the intervention, it was not possible to blind participants and carers to group allocation.

Intervention fidelity

In line with practice recommendations,80 intervention fidelity was promoted through the provision of initial training, a therapy manual, regular centralised supervision and the recording of information about each session in therapy logs:

• Training – therapists participated in a 2-day training course to prepare them for delivering the intervention at the start of the trial, and subsequently attended a refresher training day annually. The co-investigator Jackie Pool, an OT, specialist consultant and experienced trainer with expertise in applying rehabilitation in dementia care, delivered initial training to all of the trial therapists and guided them throughout the trial in the effective and consistent application of the therapy protocol.

• Therapy manual – therapists were provided with a detailed therapists’ manual that included information about the principles and key elements of CR, as well as session-by-session overviews and references to the relevant literature.

• Supervision – therapists had monthly individual supervision via video-conferencing and 3-monthly face-to-face group supervision meetings with Jackie Pool, with ad hoc advice available between meetings if needed. Supervision meetings offered detailed guidance on the delivery of CR and enabled ongoing monitoring of fidelity to the protocol, with potential concerns discussed and resolved as they were raised. Each supervision session was documented and notes were reviewed annually to ensure the appropriate involvement of all therapists in the supervisory process. The trial manager attended the quarterly supervision meetings to review progress with therapy provision and regular updates were given to the chief investigator and the trial management group. The supervision meetings were focused on reviewing therapists’ plans for achieving individual therapy goals, resolving any specific difficulties relating to individual participants and reviewing overall progress with implementing the therapy protocol for current participants in the CR group. Advice was also given about achieving a positive therapeutic relationship with participants and managing caseloads. Group meetings provided a platform for sharing best practice and ensuring consistency across sites.

• Therapy logs – supervision was facilitated by the use of therapy logs summarising session content (with participant details anonymised). A therapy log was maintained for each participant receiving CR, with notes on session content added by the therapist after each session. The logs were submitted to the supervisor for review prior to the supervision sessions and formed a basis for discussion during the sessions.

Treatment fidelity was considered in relation to form and function. 81 While the therapy protocol was prescriptive in relation to the number and length of sessions and provided guidelines on the typical content of each session (form), a degree of flexibility was required in order to facilitate individual goal attainment, as this was a key aspect of the intervention (function). Therapists could therefore make adjustments to the content of therapy sessions in order to take account of participants’ preferences, levels of cognitive and functional ability, and social and family context.

Demographic details

At the baseline assessment, we collected demographic and background information for the person with dementia and their carer, including sex, age, relationship between the person with dementia and the carer and whether they live together, age at onset of dementia, educational level, social class and comorbid health conditions assessed using the Charlson Comorbidity Index,82 to provide both the number of conditions and the weighted comorbidity score. A weighted score of ≥ 5 is used to indicate people with a particularly high level of comorbidity translating into a high risk of mortality. This was intended to provide a profile of the sample and to allow us to examine the effects of demographic and social variables on treatment efficacy.

Primary outcome measure

The primary outcome for GREAT was participant-reported goal attainment at 3 months post randomisation (the 3-month follow-up). Participant-reported goal attainment was also assessed at 9 months post randomisation (the 9-month follow-up). Parallel carer ratings of goal attainment were obtained at both the 3-month and 9-month assessments. These ratings were obtained using the structured interview protocol of the BGSI. The trial researchers participated in a 2-day initial training course to prepare them for conducting goal-setting using the BGSI, attended annual refresher training days and participated in monthly telephone supervision with two of the investigators (from a rotating panel of four), which was focused specifically on optimising the goal-setting process.

During the initial assessment using the BGSI, participants were asked how memory and other cognitive difficulties affect (1) everyday tasks, activities and routines, (2) the possibility of engaging in pleasurable and meaningful activities and (3) social contacts and relationships. For each of these domains, participants rated how important it was to them and how ready they were to try to make changes, in each case using a scale of 1–10. This provided a basis for identifying areas in which participants would like to make changes or improvements and for setting specific goals. Participants could select up to three goals. Goals were expressed in behavioural terms using SMART principles: specific, measurable, achievable, realistic and attainable within a defined period of time. Participants described what they want to be able to do and what they are currently doing, and made ratings of their current level of goal attainment on a scale of 1–10, whereby 1 was unable to do or not currently doing and 10 was able to do well with no difficulty. Participants also rated their satisfaction with this level of attainment on the same scale of 1–10, where 1 was extremely dissatisfied and 10 was extremely satisfied. The scale was presented in a visual format as well as through verbal explanation. The mean levels of attainment and satisfaction were calculated by summing the ratings across all of the identified goals and dividing by the number of goals identified.

Carers provided their own descriptions of the person’s current functioning and made parallel ratings of attainment on the same scale of 1–10. The mean ratings for attainment were calculated by summing the ratings across all of the identified goals and dividing by the number of goals.

At the 3-month and 9-month follow-ups, participants and carers were shown the goal descriptions and baseline ratings for the originally identified goals and asked to rate the current attainment and satisfaction levels for each goal. The mean attainment and satisfaction ratings were calculated as before.

The key information that the performance and satisfaction ratings provide is an indication of the extent and direction of change. In clinical practice, it is usual for people to remember and consider previous ratings or previously obtained information when making such ratings, and this can make current ratings more informative and the process of completing the ratings more transparent. 83 However, people with dementia may find it difficult to remember their previous ratings as a result of their memory difficulties. Various approaches to obtaining follow-up ratings are used in clinical trials, and one question that arises is whether or not participants should have access to their previous ratings when completing a new set of ratings at follow-up. Evidence shows that participants prefer to be reminded of previous scores and that, in the case of healthy adults, being reminded of previous scores produces no significant differences in ratings compared with not being reminded. 84 People with dementia, because of their difficulties with memory, may benefit more than other groups from being reminded about the rating process and being given access to their earlier ratings. Furthermore, in GREAT, participants in the CR group made in-session ratings of goal attainment in the sessions prior to the 3- and 9-month assessments, and providing a reminder of previous scores to all study participants removed this source of inequity.

Secondary outcome measures for participants with dementia

The secondary outcomes for the person with dementia at the 3- and 9-month follow-ups were self-rated quality of life, self-efficacy and mood, cognitive test scores (memory, attention and executive function) and extent of service utilisation. The following measures were taken at baseline and follow-up.

Secondary outcome measures for the carer

The outcomes of interest for carers were stress, quality of life and health status, assessed with the following self-rated measures at baseline and at the 3- and 9-month follow-ups:

• The Relatives’ Stress Scale (RSS). 98 The RSS is a 15-item dementia-specific measure of caregiver stress with items rated on a 5-point Likert scale and summed (score range 0–60). A higher overall score indicates higher levels of caregiving-specific stress.

• World Health Organization’s Quality of Life Instrument – brief version (WHOQOL-BREF). 99 The WHOQOL-BREF is a 26-item scale assessing perceived quality of life, with each item scored on a 5-point Likert scale (with the lowest score being 1 and the highest score being 5) giving scores in four domains: environment (raw score range 8–40), social relationships (raw score range 3–15), psychological health (raw score range 6–30) and physical health (raw score range 7–35). Scores within each domain are summed and the mean domain score is calculated (range 1–5); this is multiplied by 4, giving a score in the range of 4–20.

• EuroQol-5 Dimensions (EQ-5D). 100 The EQ-5D is a standardised measure of health status and health outcome, applicable to a wide range of health conditions. In the first section, the respondent is asked to select one of three options for each of five dimensions: mobility, self-care, usual activities, pain/discomfort and anxiety/depression. For each dimension, the three response options are coded on a 3-point scale from 1 (no problems) to 3 (unable to perform/extreme problems). This yields a descriptive profile (e.g. 11232) across the five dimensions. Based on the EQ-5D descriptive categories, a summary index can be calculated. 101 Each level in each dimension has a weight attached, and the summary index is calculated by deducting the appropriate weights from 1. The EQ-5D, three-level version (EQ-5D-3L) has an index score from –0.594 to 1.0, with a higher index score indicating a higher quality of life and a negative index score indicating a quality of life that is considered by some respondents to be ‘worse than death’. The second part of the measure is a visual analogue scale (VAS) for the self-rating of HRQoL (‘your health state today’) on a scale of 0–100. This measure was included so that the EQ-5D score could be used to generate QALY scores using societal weights.

The goal-setting process

All participants, irrespective of allocation, identified personal goals in the baseline assessment. Participants rated the importance to them of each goal and their readiness to change. We extracted details of the goals that participants identified at baseline, together with the corresponding ratings.

The process of therapy

For participants allocated to receive the CR intervention, we evaluated the process of therapy through several means, from the perspectives of the therapists and of the participants and carers.

Primary outcome

The main analysis was an analysis of covariance (ANCOVA) adjusted for baseline score, allocation group and the stratification variables (age, sex, MMSE score and centre), which were treated as random effects. The analysis used a mixed-effects model. However, the standardised effect size estimates (Cohen’s d) and the confidence intervals (CIs) were based on a fixed-effects size model, as these parameters cannot be derived from a mixed-effects model. Effect sizes were calculated by converting eta-squared to r and then converting r to Cohen’s d, using the formula provided by McGrath and Meyer. 102

Additional regression modelling was undertaken to identify factors that could be important in attaining and maximising the observed effects. This was done separately for people with dementia and carers. For the participants with dementia, in addition to the stratification variables (age, sex, MMSE score and centre), the following factors were retained in the final model on a priori theoretical grounds: diagnosis, medication, education and comorbidity. The remaining factors [marital status, living situation, first language, ethnicity, socioeconomic status (SES) and self-rated health] were each subjected to a simple regression analysis to determine whether or not the given factor was significantly associated with outcome; if so, that factor was added to the regression model. For the carers, the following factors relating to the carer were included in the final model on a priori theoretical grounds: relationship with the person with dementia, age, sex and education. The remaining factors (first language, ethnicity, SES and self-rated health) were each subjected to a regression analysis to determine whether or not the given factor was significantly associated with outcome; if so, that factor was added to the regression model.

Secondary outcomes

The main analyses for the secondary outcomes used an ANCOVA adjusted for baseline score, allocation group and stratification variables.

We conducted exploratory analyses to examine the influence of demographic variables on key secondary outcomes at baseline and follow-up. For the person with dementia, we examined the influence of the following factors on BGSI performance ratings at 9 months and BGSI satisfaction ratings: centre, age, sex, educational level, social class, ethnicity, living situation, diagnosis, baseline MMSE score, whether or not the person was taking medication for dementia, comorbidity and effectiveness of blinding. For the carer, we examined the influence of the following factors on carer stress: age, sex, education, social class, ethnicity, relationship to the participant, whether or not the carer was co-resident with the person with dementia and self-rated health status.

Sensitivity analyses

Sensitivity analyses for the primary and main secondary outcomes were planned based on:

• Treatment received irrespective of group allocation. This analysis was planned in order to compare the outcomes of statistical analyses in which the group allocation was determined by random assignment with the outcomes of statistical analyses in which group allocation was based on the treatment received. Because the group allocation and treatment received was 100% consistent, a treatment-received analysis was not necessary.

• Complete case with no data imputation. We compared the outcomes of statistical analyses that included both imputed and complete-case data with the outcomes of statistical analyses that included complete cases only. This proposed a comparison involved appraising whether or not the results from the complete-case data analysis and the imputed data analysis are substantially different.

Adherence

For the CR group, we examined the impact of the ‘dose’ of treatment received on primary and secondary outcomes at the 3- and 9-month follow-ups. Adherence was calculated as the number of therapy sessions completed. The analysis was intended to be conducted on the basis of treatment received rather than group allocation.

Effectiveness of blinding

We examined whether or not the researcher’s ability to correctly surmise the participant’s group allocation was associated with the primary and secondary outcomes at the 3- and 9-month follow-ups, and whether this ability varied by centre or depending on any participant characteristics.

Nature of goals identified

To better understand the concerns of participants and the areas addressed by therapists in the intervention, we examined the goals set by participants in the trial and used descriptive content analysis to classify them into groups based on the content and focus of the goal.

Goal-setting and goal attainment

We compared the goal attainment ratings made by the participant, the carer and the therapist at each stage and the changes in these ratings between the point at which work on the goal was introduced and sessions 10 and 14, in order to examine the similarities and differences in perspective regarding level of functioning and change in functioning. We also compared the goal attainment ratings with the therapists’ goal attainment scaling process (which classified goals as being fully or partially achieved, with a percentage score) to determine whether or not these two procedures yielded consistent information.

Intervention fidelity

Fidelity of form was evaluated in relation to the provision of core elements of the intervention. This was indexed by the number and length of therapy sessions completed and the number of goals operationalised and addressed.

Fidelity of function was considered in relation to the therapists’ ability to deliver the intervention in a flexible, person-centred manner adapted to the needs of each individual participant. In order to evaluate the level of flexibility applied by the therapists, we reviewed the times at which additional goals were introduced and the nature and extent of modifications to goals formulated in the baseline assessment. As fidelity of function was promoted through regular supervision, with the aid of the detailed therapy logs produced by therapists, we also assessed attendance at supervision sessions and the rates of completion of therapy logs.

Therapist perspectives on the intervention and factors associated with positive outcomes

We explored what factors were thought to influence treatment outcome and the characteristics of participants who seemed most and least likely to benefit from the intervention, and why, with two sets of data: (1) data from a focus group conducted with the therapists and (2) a selection of therapy logs.

We analysed data from the focus group conducted with the therapists to identify their views about delivering the intervention and about what factors influence treatment outcome. From the existing literature, we identified a number of factors that we expected the therapists to consider; these were the level of cognitive impairment, health status, awareness of difficulties, motivation, the degree of carer involvement and family circumstances. We used a deductive approach, coding the data in relation to these factors, but also adopted an inductive perspective, which allowed new factors to emerge.

The therapy logs provided a rich source of data about the process of therapy for each individual participant and were analysed to identify significant features and processes of the intervention. We analysed therapy logs for the 25 participants with the best outcomes on the BGSI and the 25 participants with the worst outcomes on the BGSI using a matrix analysis to examine which factors identified from the focus group data were noted as being relevant in the therapy logs. This analysis was also inductive, allowing additional factors to emerge from the therapy logs. The aim was to understand more about which participants were or were not likely to benefit from the intervention.

Participant and carer experience of the intervention

We analysed the transcribed recordings of the interviews with participants and carers using a thematic analysis. The aim was to identify the level of satisfaction with the intervention and any features that contributed to greater or lesser degrees of satisfaction.

Participants’ goal attainment ratings

A linear mixed-effects model was fitted to identify whether or not any factors predicted the difference in participants’ goal attainment ratings at baseline and the 3-month follow-up. Centre was treated as a random effect and the following factors were treated as fixed effects: sex, age (stratified), MMSE score (stratified), diagnosis, medication use, education, comorbidity, SES and blinding efficiency. At the 3-month follow-up, the model was not statistically significant [χ²(18) = 41.74, R² = 0.19; p = 0.001]. Two factors within the model had a statistically significant effect in predicting a change in BGSI attainment ratings: blinding efficiency [b = 1.24, standard error (SE) = 0.30, t(216.87) = 4.12, 95% CI 0.54 to 1.76; p < 0.001] and participant SES [χ²(5) = 16.66; p = 0.005]. Greater improvement was seen when blinding was ineffective. For SES, there was a statistically significant difference in the change in BGSI goal attainment rating between professionals and non-manual skilled workers [b = –1.66, SE = 0.65, t(214.71) = –2.55, 95% CI –3.39 to –0.56; p = 0.012], professionals and manual skilled workers [b = –2.11, SE = 0.65, t(216.57) = –3.28, 95% CI –4.17 to –1.05; p = 0.001] and professionals and partly skilled workers [b = –1.93, SE = 0.75, t(216.88) = –2.58, 95% CI –3.83 to –0.67; p = 0.011]. These observed differences show that CR was more effective at improving BGSI performance ratings at the 3-month follow-up for professionals than for non-manual skilled workers, manual skilled workers or non-skilled workers.

A linear mixed-effects model was fitted to identify whether or not any factors predicted the difference between the BGSI attainment ratings at baseline and the 9-month follow-up for participants in the CR group. Centre was treated as a random effect and the following factors were treated as fixed effects: blinding effectiveness, participant’s age (stratified), sex, education, social status, ethnicity, living situation (alone or with others), diagnosis, MMSE score (stratified), medication use and comorbidity. The model was statistically significant [χ²(28) = 68.61, R² = 0.32; p < 0.001]. Factors within the model that were individually statistically significant were participant’s age, MMSE score and blinding efficiency. Greater improvement was seen in participants who were younger [b = –0.05, SE = 0.02, t(200.60) = –2.08, 95% CI –0.09 to –0.00; p = 0.038] and had higher MMSE scores [b = 0.16, SE = 0.05, t(199.31) = 3.06, 95% CI 0.06 to 0.26; p = 0.002], and when the researcher was able to correctly identify the participant’s group allocation [b = 1.35, SE = 0.32, t(199.86) = 4.21, 95% CI 0.71 to 1.99; p < 0.001].

Linear mixed-effects models were fitted to identify whether or not any carer factors predicted the difference between the participants’ BGSI attainment ratings at baseline and the 3- and 9-month follow-ups for participants in the CR group. Centre was treated as a random effect and the following factors were treated as fixed effects: carer’s sex, carer’s age, carer’s education, hours spent helping the person with dementia per day and the relationship between the carer and the participant. The models were not statistically significant and no factors within the models were statistically significant.

Carers’ ratings of participant goal attainment

Linear mixed-effects models were fitted to identify whether or not any carer factors predicted the difference between carers’ BGSI attainment ratings at baseline and at the 3- and 9-month follow-ups. The models were not statistically significant and no factors within the models were statistically significant.

Linear mixed-effects models were fitted to identify whether or not any participant characteristics predicted the difference between carers’ BGSI attainment ratings at baseline and the 3-month follow-up. The overall model included centre as a random effect and the following factors as fixed effects: MMSE score (stratified), diagnosis, medication use, education, comorbidity, SES. The model was not statistically significant [χ²(17) = 27.24, R² = 0.12; p = 0.055]. Within the model, SES was statistically significant [χ²(5) = 14.54; p = 0.013]. There was a statistically significant difference in the change in carer BGSI attainment rating between professionals and non-manual skilled workers [b = –1.83, SE = 0.68, t(214.14) = –2.70, 95% CI –3.15 to –0.50; p = 0.007], professionals and manual skilled workers [b = –1.63, SE = 0.67, t(214.99) = –2.46, 95% CI –2.94 to –0.33; p = 0.015]. These results mirror the findings for participant goal attainment ratings at the 3-month follow-up.

Linear mixed-effects models were fitted to identify whether or not any participant characteristics predicted the difference between the carers’ BGSI attainment ratings at baseline and the 9-month follow-up. The overall model included centre as a random effect and the following factors as fixed effects: MMSE score (stratified), diagnosis, medication use, education, comorbidity, SES. The model was not statistically significant [χ²(17) = 31.18, R² = 0.15; p = 0.019]. Within the model, MMSE (stratified) was a statistically significant factor [χ²(1) = 7.91; p = 0.005]. This shows that there was a statistically significant difference in carer attainment ratings for participants with MMSE scores of ≥ 24 points compared with participants with MMSE scores of < 24 points [b = 0.96, SE = 0.34, t(202.98) = 2.81, 95% CI 0.29 to 1.63; p = 0.005]. The improvement in carer attainment ratings was greater when participants had higher MMSE scores.

Participants’ ratings of satisfaction with goal attainment

A linear mixed-effects model was fitted to identify whether or not any participant characteristics predicted the difference between the BGSI satisfaction ratings at baseline and the 3-month follow-up for participants in the CR group. Centre was treated as a random effect and the following factors were treated as fixed effects: sex, ethnicity, age (stratified), MMSE score (stratified), diagnosis, medication use, education, comorbidity and SES. The model was statistically significant [χ²(28) = 49.89, R² = 0.21; p = 0.007]. Factors within the model that were individually statistically significant were participant social status [χ²(5) = 16.82; p = 0.005] and blinding efficiency [chi-squared test (1) = 10.30; p = 0.001]. There was a statistically significant difference in the change in BGSI satisfaction rating, showing that satisfaction ratings improved more in professionals than in non-manual skilled workers [b = –1.73, SE = 0.65, t(217) = –2.67, 95% CI –3.01 to –0.46; p = 0.008], manual skilled workers [b = –2.10, SE = 0.63, t(217) = –3.34, 95% CI –3.33 to –0.86; p = 0.001] and partly skilled workers [b = –1.92, SE = 0.75, t(217) = –2.57, 95% CI –2.93 to –0.26; p = 0.011]. There was greater improvement in the BGSI satisfaction ratings from baseline to the 3-month follow-up when blinding was ineffective and the researcher was able to correctly identify that the participant belonged to the CR group than when blinding was effective [b = 0.96, SE = 0.30, t(217) = 3.21, 95% CI 1.04 to 9.49; p = 0.002].

A linear mixed-effects model was fitted to predict the difference between the BGSI satisfaction ratings at baseline and the 9-month follow-up for participants in the CR group. Centre was treated as a random effect and the following factors were treated as fixed effects: blinding efficiency, participant’s age, sex, education, social status, ethnicity, living situation (living alone or with others), diagnosis, MMSE score, medication use and comorbidity. The model was statistically significant [χ²(27) = 54.26, R² = 0.25; p = 0.001]. The only effects that were statistically significant were MMSE score [χ²(1) = 15.79; p < 0.001] and blinding efficiency [χ²(1) = 14.35; p < 0.001]. Participants with higher MMSE scores at baseline showed greater improvement in BGSI satisfaction ratings from baseline to the 9-month follow-up than participants with lower MMSE scores [b = 0.20, SE = 0.05, t(199.45) = 3.97, 95% CI 0.22 to 0.27; p < 0.001]. There was greater improvement in the BGSI satisfaction ratings from baseline to the 9-month follow-up when blinding was ineffective and the researcher was able to correctly identify that the participant belonged to the CR group than when blinding was effective [b = 1.23, SE = 0.33, t(200.33) = 3.79, 95% CI 0.58 to 1.88; p < 0.001].

Ratings of importance and readiness to change

We examined whether or not participants’ BGSI ratings of importance of the functional domain addressed by each goal and ratings of readiness to change made at baseline were associated with the attainment ratings at the 3-month follow-up. Participants’ baseline ratings of the importance of the functional domain addressed by each goal did not predict improvement. Participants’ ratings of readiness to change at baseline, however, were significantly associated with improvement in attainment ratings at the 3-month follow-up [t(403) = 2.66, r = 0.13; p = 0.008].

Participants’ ratings of readiness to change remained significantly associated with improvement in their BGSI attainment ratings at the 9-month follow-up [t(379) = 2.79, r = 0.14; p = 0.005].

Goal-setting

All participants were invited to identify up to three therapy goals. Researchers were encouraged to ensure that at least two goals were identified, to ensure that the participant understood the aims of the therapy and was motivated to participate. In total, 1358 therapy goals were identified by 474 randomised participants. The majority of participants (411; 86.7%) identified three therapy goals, 62 participants (13.1%) identified two therapy goals and one participant (0.2%) identified one therapy goal. Further details are provided in Table 14.

Goal selection

The 1358 goals identified by participants were listed and the content and focus were analysed to identify the areas that were of concern to participants. This built on a preliminary analysis of 591 goals set by the first 209 participants enrolled into the trial by December 2014. 105 In the current analysis, the first 300 goals were reviewed in detail and grouped thematically to form the basis of a detailed coding system. This system was then applied to the remaining goals.

The goals were grouped into the following categories:

• Engaging in activities and personal projects (21%). This included planning and engaging in enjoyable activities, taking up new activities, restarting previously enjoyed activities, doing activities regularly or more often, doing activities independently, practising skills, keeping occupied, undertaking personal projects and doing things for others.

• Using appliances, devices and the internet (17%). This category focused on the use of household appliances, such as washing machines or microwave ovens, devices such as mobile phones, smartphones, tablets, TVs, computers, laptops, cameras, CD or DVD players, satnavs and personal safety aids, such as wristbands or pull cords. Participants wanted to learn to use, or be able to use, appliances and devices, to feel confident in using them or to use them independently. This was for a range of purposes, including completing household tasks, communication, entertainment and occupation, obtaining information and staying safe.

• Managing everyday activities, tasks and situations (16%). This included managing money, shopping, cooking and baking, correspondence and telephone calls, carrying out household tasks, going out, wayfinding, using transportation, telling time and staying safe. Participants wanted to feel confident in doing their everyday activities or to be able to do them independently or safely.

• Knowing what is happening (9%). Participants wanted to know the day and the date, the schedule for the day or the week ahead and the timing of any appointments. They were keen to know without asking others or having to check repeatedly, and sometimes mentioned particular strategies that they wished to use, such as a whiteboard, calendar or diary.

• Retaining or keeping track of information and events (8%). Participants wanted to find ways of remembering important information, information that they were recently given, information that they had told others, events and activities and messages to pass on. They wanted to be able to retain key elements of the plot of a novel or TV programme or to keep the score when watching or playing sports or games.

• Locating belongings (7%). This category was about finding personal items and items around the house, knowing where things had been left and putting things back in the right place.

• Recognising, identifying and naming (6%). Participants wanted to remember the names of family members, friends, people they met and prominent people involved in current affairs, as well as information about these people. They wanted to be able to recognise and identify people and objects.

• Engaging in conversation (5%). This category was about being able to engage in and participate in conversation, follow the thread of a conversation and retain key points, find words, keep the conversation going and speak without repetition. Participants wanted to be able to contribute confidently, in a range of settings, for example at family mealtimes or when walking with a group.

• Organising, improving and finishing (4%). Some participants emphasised a wish to organise aspects of their lives or their environment, to complete tasks they had set themselves and to improve their performance in areas like handwriting, spelling or vocabulary.

• Caring for self (3%). Participants wanted to be able to do some basic life tasks regularly or independently. These included shaving, bathing, changing dirty clothes and eating and drinking regularly.

• Keeping in contact and staying engaged with family and friends (2%). In this category, participants specifically mentioned wanting to keep in contact with family and friends through various means, to take an interest in and remember what is going on in their lives and to attend family gatherings.

• Managing emotions (2%). Participants wanted to manage anxiety or frustration, be more patient and worry less, or to cope better with changes in routine.

Goal attainment in the cognitive rehabilitation group

Participants randomised to the CR group had previously identified 679 goals at baseline. Of these, 591 goals were introduced in therapy sessions by the therapist, initial in-session goal attainment ratings were made for 590 goals and follow-up ratings were made for 563 goals. In-session goal attainment ratings showed a similar pattern to the ratings made at the baseline and follow-up assessments (details are shown in Tables 15 and 16). Correlations between the goal attainment ratings made by the participant, the carer and the therapist at each stage were large, ranging from 0.665 to 0.934, and were significant at a p-value of < 0.0001 (two-tailed), indicating a high degree of consistency among the three raters. All ratings improved, on average, by over 2 points, reflecting a clinically significant change. The change scores are shown in Table 16; correlations among the change scores for participants, carers and therapists were large, ranging from 0.679 to 0.861, and were significant at a p-value of < 0.0001 (two-tailed), again reflecting a high degree of consistency.

On introducing goals, therapists also independently established indicators to classify the extent of goal attainment in percentage terms. On completion of the therapy sessions, they rated the extent of attainment for each goal. In total, 54.8% of goals were rated as being at least 75% attained and 79.8% were rated as being at least 50% attained. Only 5% of goals showed no progress towards attainment. Figure 4 provides a summary of the therapists’ goal attainment ratings made after session 10 and Table 17 breaks this down by the order of introduction of the goals.

FIGURE 4.

Attainment of therapy goals by session 10. Proportion of therapy goals rated as being fully achieved (100%), partially achieved (25%, 50%, or 75%) and not achieved (0%) after session 10.

We compared the therapists’ goal attainment scaling with the in-session goal attainment ratings made by participants, carers and therapists in or following both sessions 10 and 14 to determine whether or not these two procedures yielded consistent information. For the session 10 goal attainment scaling, the correlations were 0.557 with participant ratings, 0.786 with carer ratings and 0.862 with therapist ratings, and for the session 14 goal attainment scaling, the correlations were 0.652 with participant ratings, 0.881 with carer ratings and 0.932 with therapist ratings. All correlations were significant at a p-value of < 0.0001, two-tailed. This indicates generally good consistency.

Focus group conducted with the therapists

Analysis of the therapy logs

Following the analysis of the focus groups, a set of therapy logs was analysed. The findings from the focus group analysis informed the analysis of the therapy log data, as described below.

Method

Three sites were able to contribute to this component of the evaluation, as they each identified an independent researcher not involved in the trial who could conduct detailed interviews. Interviews were conducted in the Bangor (21 March 2014 to 20 January 2015), Cardiff (31 July 2015 to 9 December 2015) and Manchester (1 April 2015 to 28 May 2015) sites. In each site, a consecutive series of participants and carers was approached following the 9-month follow-up assessment and invited to participate in the interview to discuss their experiences of the therapy sessions. The trial manager issued the researchers with site-specific lists of all participants in the CR group who were due to complete their final follow-up assessment in the designated recruitment period. In total, 36 couples were approached and 26 agreed to be interviewed, although, in the case of one couple, only the carer participated in the interview. We interviewed 12 carers and 11 participants at Bangor (100% of those approached), 10 carers and 10 participants at Cardiff (50% of those approached) and four carers and four participants at Manchester (100% of those approached). The demographic and clinical characteristics of these participants, and their BGSI ratings, are shown in Tables 26–30. Of note, six of the participants were also included in the therapy logs analysis, two in the poor-outcome group and four in the good-outcome group.

The interviews followed a semistructured schedule, and interviewers encouraged the participants and carers to talk freely about their experience of the intervention (see Appendix 2 for the interview schedules). The interviews covered the following topics:

• How did participants and carers experience the intervention?

• What were their overall perceptions, how useful did they find it, and what did they feel about the degree of effort required?

• What impact, if any, did the participants and carers feel that the intervention had on their everyday life?

The interviewers had an overall understanding of what the intervention involved, but no specific knowledge of the individual participants’ therapy goals or the therapy process, in order to avoid bias. Participants and carers were interviewed separately whenever possible, starting with the person with dementia. Interviewers took a photograph of the therapist on the visit to prompt the participant’s memory of the therapy sessions. If the participant was struggling to recall the therapy sessions, the interview was completed jointly with the carer. The first interview conducted by each interviewer was reviewed at the co-ordinating centre to ensure adherence to the interview schedule. Interviews were found to have been conducted satisfactorily in each case. All interviews were audio-recorded, transcribed verbatim and anonymised.

Thematic analysis started from a critical realist position and was based on an inductive approach to identifying and exploring patterns of meaning in relation to the research question. 106,108 Four researchers who were not involved in conducting the interviews analysed the interviews; three of these researchers were independent from the trial and one was the trial manager. The process was overseen by the chief investigator. Initially, two researchers read and reread the first five transcripts to familiarise themselves with the data and then identified and coded (briefly summarised and characterised) units of meaning within each transcript. Codes were listed separately, reviewed and organised into meaningful groups, representing the initial themes for each interview. The resulting lists of themes were compared and discussed by the two researchers until a consensus was reached about content and organisation, after which each researcher recoded the transcripts. Related themes were clustered together and the clusters were ordered into group-level themes and subthemes, and the two researchers worked together to integrate these into an overall thematic map. The remaining transcripts within the set were then coded by a single researcher, using the identified list of themes.

Findings

Overall, the therapy was received positively by both carers and people with dementia and, generally, there were very few criticisms of the therapy. Participants mostly said that they had nothing negative to report about the experience. Several key themes emerged, reflecting factors that influenced the experience of the intervention and whether or not it was considered to be beneficial.

Form of evaluation

The economic evaluation included cost-effectiveness and cost–utility analyses.

Effectiveness

The economic evaluation examined four outcome measures, three for participants with dementia and one for carers.

Cost-effectiveness measures:

1. incremental cost of achieving a standardised mean difference (SMD) of 1.32 points in the participant-reported goal attainment measure (the BGSI), the primary outcome for the trial

2. incremental cost of achieving a difference in effect of 0.30 in the GSES, which assesses a participant’s general sense of perceived self-efficacy.

Cost–utility measures:

1. incremental cost per QALY assessed by the participant-rated DEMQOL

2. Incremental cost per QALY for carers’ self-rated HRQoL, using the EQ-5D-3L.

A SMD for the GSES was calculated by multiplying the effect size of 0.3 by the (non-imputed) SD of the mean across sample participants at baseline following Samsa et al. 109 The second outcome in the list above thus equates to an incremental cost of achieving a SMD of 1.53 points in the GSES. These outcome measures are described in more detail in Chapter 2.

We calculated utility scores for participants with dementia from the DEMQOL instrument (the DEMQOL-U index) using published societal weights. 110 We derived QALYs from these scores using the area-under-the-curve method, with linear interpolation between the three assessment points. We calculated carers’ utility scores from the EQ-5D-3L with published societal weights. 111

Perspective

The economic evaluation first took a health and social care perspective, and second a societal perspective. Broadly speaking, the health and social care perspective took into account those costs falling to the NHS and to local authority social services departments (SSDs). Most service costs (e.g. hospital, community health care, community day care and home-based care) were considered to fall entirely to these agencies. However, only adaptations and equipment reported as being provided by the NHS or SSDs were considered within this perspective. The societal perspective was considered to encompass not only health and social care costs, but also the costs to the participant–carer dyad: lost production in terms of carer’s wages forgone because of providing care; costs of providing care in terms of hours of care provided; out-of-pocket payments (privately purchased equipment and travel costs of attending appointments related to dementia treatment).

Time horizon

The outcomes and costs considered in the economic evaluation were measured at baseline and at the 3-month and 9-month assessment points. The costs collected using the CSRI covered the 3-month period prior to each assessment point. The 9-month costs were calculated from these three data collections. The costs of services between 3 months post baseline and 6 months post baseline were assumed to be the same as the costs in the 3 months prior to the 9-month follow-up. In other words, to calculate the 9-month costs, we estimated the 3-month costs based on data from the second follow-up, multiplied this estimate by two and added this to the costs in the 3 months prior to the first follow-up. It was not necessary to apply discount rates to either costs or outcomes, because the time horizon was shorter than 1 year.

Costs

The analysis considered the comprehensive costs of care and support to the person with dementia. The costs were calculated drawing on the following collections:

• data on services used by the person with dementia, as observed and reported by carers using the CSRI112

• data on carer time spent on care and support activities and lost employment, using the CSRI

• data on time spent by professionals in delivering the intervention, using the therapy logs

• data on professionals’ labour costs, using a pro forma distributed to therapists

• costs of training (fees, materials) supplied by the project management team.

The costs of health and social care services were calculated from service use data by applying relevant, nationally generalisable unit costs taken from NHS Reference Costs 2012 to 2013113 and the Personal Social Services Research Unit (PSSRU)’s costs compendium. 114 We calculated the costs of carers’ inputs using opportunity costs (base case) and replacement costs (sensitivity) methods. 115–117 The opportunity costs approach involved attaching a value to each hour of unpaid carer time in providing care and support equal to the minimum wage. In the replacement cost approach, the cost of an hour of home care was used to value unpaid time spent providing care. The unit costs used in valuing resource use118–143 are summarised in Table 31 and reported in full in Appendix 15.

Costs are reported in the following categories: hospital services, primary and community health, mental health services, overnight respite care, community social care, day services, equipment and adaptations, mental health medication, costs of the CR intervention and unpaid care. Costs are also reported as aggregated total costs from the health and social care perspective and from the societal perspective.

The costs of the CR intervention were calculated by drawing on a number of sources. We gathered comprehensive information on the time spent by therapists delivering the CR intervention. The intervention time consisted of three elements:

• Direct (face-to-face) contact time with participants – therapists entered their contact time per visit in ‘therapy logs’ in the MACRO database system.

• Indirect contact time (planning, travel) with participants – the set-up time was estimated by the GREAT project team as 10–15 minutes. Therefore, 12.5 minutes of indirect time were allocated per visit. Therapists estimated the time taken to make a one-way journey by the participant as an average and entered this into the MACRO database system.

• Non-contact time (general training and individual training/supervision) – a pro forma collected non-contact costs (supervision and training of therapists) from the project team. These comprised the number of hours spent in general training sessions, the costs of providing general training sessions (trainers’ fees, costs of venues and materials), the travel costs of attending general training sessions, the number of hours spent in individual training sessions, the costs of providing individual training sessions (trainers’ fees, costs of venues and materials) and the travel costs of attending individual training sessions.

Therapists also completed a pro forma providing information on Agenda for Change (AfC) band, the proportion of full-time equivalent (FTE) hours worked, the start and end dates of the project, the usual mode of transport, the average number of miles travelled by car, the average parking charges and the average cost of a public transport fare.

To value the time spent by therapists in direct, indirect and travel activities, we calculated comprehensive therapy staff costs, including salary costs (by median FTE earnings per AfC band, including oncosts: superannuation at 14% of salary and national insurance contributions), administrative overheads (management and non-staff costs) and capital overheads calculated as a percentage of salary cost. Staffing costs were weighted by salary band and time contributions (FTEs at each band) to estimate a weighted cost per hour of therapy time. To value the costs of therapists’ journeys made to participants, we estimated the costs of public transport or car travel per centre using the data from each therapist-completed pro forma.

The study ran over 4 years: pro formas were issued every financial year. The costs of general training and supervision in years following the base year (2013–14) were deflated to 2013–14 prices using the Hospital and Community Health Service (HCHS) index. 144 The total across all years was divided by the number of CR participants (n = 238) multiplied by 14 sessions, to give a per-session overhead to be attached to each session attended by participants.

Missing data

Resource use data collected using the CSRI may be missing for any use of a service, for the frequency of using the service or for the duration of a service (e.g. length of a home care visit). When service use was indicated but frequency was missing, a suitable nationally applicable unit cost was used if available (e.g. cost per visit). For each case, items in each cost category were added together to give the total cost for the category. The category-level costs were then summed to give a total overall cost per case. If all costs in the category were missing, the category total (per case) was calculated as missing; if some items were missing, these were treated as zeros and the case was assigned the cost of the sum of available costs in the category.

Missing category-level costs were multiply imputed by predictive mean matching (k = 5 nearest neighbours) in a regression model that included demographic variables of the dyad, centre and stratified MMSE score and the outcome measures to be used in the cost-effectiveness analysis, using the Stata^® programme MI impute (StataCorp LP, College Station, TX, USA). 145 Whether people with dementia who had been lost to follow-up or withdrew prior to the 9-month follow-up had died during this time was unknown. The model therefore also included the survival of the person with dementia to the end of the 9-month assessment as an imputation variable. Carers who were part of a dyad that had been lost to follow-up or withdrawn were assumed to have survived. Costs and outcomes were imputed separately by allocation and depending on survival. 146 The number of imputations was guided by White et al.’s147 rule of thumb that the number of imputations should be set at the percentage of incomplete cases for variables to be used in the analyses.

Analyses

The use of services and the mean use of services within allocation groups were compared descriptively, with no tests of between-group differences, given the large numbers of potential comparisons. The descriptive statistics of costs and outcomes are presented in terms of the mean and SE in each group, and the between-group difference and the SE of the difference. All service use, cost and outcome data are summarised in terms of the sample of dyads for which unpaid carers were available for the completion of the CSRI section of the assessment at each time point. All analyses were conducted using Stata version 14. Cost outliers were identified by following the adjusted box plot technique described by Vanderviere and Huber148 and recommended for skewed data. This involved calculating the medcouple, a measure of skewness (using a user-written programme medcouple in Stata),149 establishing the upper fence of a box plot interval and defining the observations falling above the upper fence as high-cost outliers.

Cost-effectiveness

Cognitive rehabilitation was to be defined as cost-effective if it was:

• less costly and more effective than TAU or

• more costly and more effective than TAU, and society is willing to pay the additional cost in order to achieve the gain in outcome, or

• less costly and less effective than TAU, and society is willing to sacrifice some of the outcome difference in order to make a saving.

The intervention was to be defined as not cost-effective if it was both significantly more costly and less effective than TAU or if society was not willing to pay the cost of a gain in outcome. The criteria for this decision were based on the following rule:

Here ΔC represents the additional cost, ΔE is the gain in outcome associated with the treatment, and λ is the willingness to pay (WTP) for that outcome gain. 150 The incremental cost-effectiveness ratio [(ICER) ΔC/ΔE] must be below the decision-maker’s WTP (λ) to be considered cost-effective.

The ICER was calculated as the difference in the mean costs of the CR and TAU groups over the period of follow-up (9 months) divided by the difference in the mean end-point outcome measure (the BGSI and the GSES for participants with dementia) between groups. In the case of the ratio of incremental costs and QALYs (based on the DEMQOL-U for participants with dementia and the EQ-5D for carers), the denominator was the difference in mean QALYs. The ICER point estimates presented in the tables are based on the ratio of the cost results rounded to one decimal place and the outcome results rounded to two decimal places.

The cost-effectiveness decision rule can be rearranged in terms of the net monetary benefit, the monetary value of a gain in effect associated with the treatment at a given WTP and the net of the additional cost of the treatment:150

The net monetary benefit must be greater than zero if the costs associated with the intervention are not to outweigh the benefits of the intervention.

Cost-effectiveness analyses

The incremental costs and outcomes were estimated by seemingly unrelated regressions (SUR). 151 This approach was combined with non-parametric bootstrapping. The estimates were adjusted by centre, baseline outcome measures and baseline costs, as well as demographic variables (sex, age and stratified MMSE score). The regression coefficients on the allocation term were used to calculate the net monetary benefit over a range of societal WTP levels for incremental differences in the primary outcome measures and for QALY gains.

The number of bootstrap samples used in the analyses was determined by a method suggested by Gould and Pitblado. 152 This involves examining the bootstrap variance estimates for the variable of interest plotted against the number of replications. The bootstrapped SE for the allocation term in the cost regression was examined in this way. At the point when consecutive SEs produced by the bootstrap samples (increasing from a base of 1000 replications by 1000 additional replications) differed by < 1%, this was considered to be an adequate number of replications. The complete-case analyses used 60,000 replications; analyses with multiply imputed data used 3000 replications.

The proportion of bootstrap replicates in which the net benefit was greater than zero was plotted over a range of WTP values to produce cost-effectiveness acceptability curves (CEACs). These illustrate the probability of making a correct decision to fund the intervention,150 and also the sampling uncertainty around the point estimate of the ICER. 153 The cost–outcome difference pairs were plotted as points on the cost-effectiveness plane as a further means to illustrate sampling uncertainty, while providing graphical information on the joint distribution of costs and outcomes across the quadrants of the plane. For instance, cost–outcome differences that fall into the north-east quadrant of the plane indicate that CR is associated with higher costs and better outcomes than TAU. 154 Points falling into the south-east quadrant indicate that lower costs and better outcomes are produced by the new intervention relative to the old intervention (in which case the new intervention is said to ‘dominate’ the old intervention). Points falling into the north-west and south-west quadrants represent situations in which the new intervention respectively costs more (in which case, the new intervention is ‘dominated’) and in which the intervention costs less than the alternative; in either case, the new intervention produces worse outcomes.

The analyses took into account the data of those participants and carers with sufficient information to calculate both service costs and carer costs. Thus, the analyses did not consider data for which only one member of the dyad had contributed information. In addition, given the need to reflect the societal perspective, data from participants with only paid carers were not analysed. This was for two reasons: first, paid carers were not providing unpaid care; and, second, the reason that the informant was a paid carer was likely to reflect the absence of an unpaid carer who would incur any unpaid care costs.

Sample numbers

The numbers of people who formally participated in assessments were:

• at baseline, 474 people with dementia and their carers (238 in the CR arm; 236 in the TAU arm)

• at 3 months, 445 people with dementia (218 in the CR arm; 227 in the TAU arm) and 442 carers (217 in the CR arm and 225 in the TAU arm)

• at 9 months, 426 people with dementia (208 in the CR arm; 218 in the TAU arm) and 422 carers (207 in the CR arm; 215 in the TAU arm).

In terms of complete dyads who formally participated in the assessment:

• at baseline, 474 participated (238 in the CR arm; 236 in the TAU arm)

• at 3 months, 442 participated (217 in the CR arm; 225 in the TAU arm)

• at 9 months, 422 participated (207 in the CR arm; 215 in the TAU arm).

With regard to CSRI questionnaires that were partially or wholly completed by dyads consisting of a person with dementia and an unpaid carer, the numbers were:

• at baseline, 469 CSRI questionnaires (236 in the CR arm; 233 in the TAU arm)

• at 3 months, 437 CSRI questionnaires (215 in the CR arm; 222 in the TAU arm)

• at 9 months, 415 CSRI questionnaires (205 in the CR arm; 210 in the TAU arm).

Information from complete dyads (who had not withdrawn or been lost to follow-up and had an unpaid carer participating) that was sufficiently complete to calculate health and social care and societal costs was available:

• at baseline, for 469 dyads (236 in the CR arm; 233 in the TAU arm)

• at 3 months, for 435 dyads (213 in the CR arm; 222 in the TAU arm)

• at 9 months, from 414 dyads (204 in the CR arm; 210 in the TAU arm).

The outcome data available at each assessment point are described in Chapter 3. The numbers of dyads included in the cost-effectiveness analyses varied depending on the measures; the relevant valid numbers of observations associated with each measure are presented with the results of the analyses. There were four paid carers who completed the CSRI (see Chapter 3). At the end of the study, seven participants with dementia had died and one carer had died.

The complete-case sample of economic data available for analysis at 9 months (those with data from complete dyads across the three assessment points who had not withdrawn or been lost to follow-up and had an unpaid carer participating) was 412 (203 in the CR arm; 209 in the TAU arm). The numbers of cases available for the cost-effectiveness analyses varied depending on the outcome:

• For QALYs calculated using the DEMQOL-U, there were 401 cases available (196 in the CR arm; 205 in the TAU arm).

• For BGSI scores, 407 cases were available (201 in the CR arm; 206 in the TAU arm).

• For GSES scores, 389 cases were available (190 in the CR arm; 199 in the TAU arm).

• For carers’ QALYs calculated using the EQ-5D-3L, there were 390 cases (192 in the CR arm; 198 in the TAU arm).

The sample of cases available in the analysis of complete data sets as a product of the multiple imputation process was 462.

Use and costs of care and support services at each assessment point

At baseline, both the CR group and the TAU group used a wide variety of services (see Appendix 16, Tables 125–127). The groups exhibited a high use of services, such as outpatient appointments and GP and practice nursing contacts. In both groups, 11% of participants received some form of home help or home care, and roughly the same number attended a day centre. One-fifth of the dyads had cleaning services. The proportions using these services remained relatively stable over the course of the study. The proportion of participants who reported taking any mental health medications was stable over the three assessment points; approximately three-quarters of both groups took antidementia medications.

The mean number of home care contacts and hours demonstrates a problematic feature of the data. There were small numbers receiving very high levels of home care at each assessment point and these were concentrated in the TAU group. At baseline and 9 months, the average number of contacts and hours of home care in the TAU group were twice those in the CR group. However, the variation in contacts and hours was higher in the TAU group, as evidenced by larger SEs relative to the means. This complicates an assessment of the size of these differences: a unit-free measure is useful in these circumstances. The SMD (the mean CR – TAU difference divided by the SD across the groups) in home-care hours between CR and TAU was –16% at baseline, 14% at 3 months and –33% at 9 months (not presented in the table). This suggests very substantial variability in receipt, or reporting of receipt, of home care over the study period and also a large difference in home care received at 9 months in the TAU versus CR comparison. The same pattern occurred in unpaid care time of the principal carer: the number of unpaid care hours at baseline and 9 months was higher in the CR group than in the TAU group but at 3 months was higher in the TAU group.

The types of care and support provided by carers are described in Table 128 in Appendix 16. A majority of carers reported providing social support (keeping the person with dementia company) and taking the person to appointments, helping with medications, providing practical help and supervision. Fewer than one-third of carers in either group reported assisting with personal care at baseline or 3-month assessments. At the 9-month assessment point, however, the proportion of carers providing personal care was 38% in the CR group and 43% in the TAU group.

The raw costs of health and social care services, unpaid care and out-of-pocket expenses in the 3 months prior to assessment are presented in Table 32. The total health and social care and societal costs are also given. The mean differences in costs between the groups exhibited wide CIs. The pattern of apparently higher costs in the CR group at baseline and 9 months echoed that seen in the utilisation of home care and unpaid care hours; however, the unadjusted differences were not significantly different from zero at any point, as evidenced by CIs crossing zero.

The health and social care costs were, on average, relatively modest, being substantially lower than societal costs at each assessment point. These averages mask some extremely high-cost cases, particularly in the TAU group. At the 9-month follow-up, the maximum total health and social care costs over the prior 3 months reached £18,063 (vs. the mean of £1282) in the CR group and £42,504 (vs. the mean of £1647) in the TAU group. The data were highly skewed: on a test for normality, the hypothesis that the data were distributed normally was rejected (p = 0.000) for both health and social care costs and societal costs.

Unit costs and per-participant costs of the cognitive rehabilitation intervention

The cost components used to value the CR intervention are given in Table 33. CR-specific training and supervision costs per session were substantial at £33, and may not reflect the level of supervision and support that would be available in routine clinical practice. The mean number of visits in the 3 months after baseline in the economic evaluation sample (n = 215) was 9.61 (SE 0.09), as shown in Table 34; the costs in the first 3 months were £1259 (SE £18). In the period between the 3-month and the 9-month assessments in the economic evaluation sample (n = 204), the mean number of visits was 3.74 (SE 0.07) and the costs were £474 (SE £10).

Use and costs of care and support services over the period of the study

Over the 9-month period of the study, the total cost of the CR intervention (as shown in Table 35) was £1736 (SE £25) per participant. The average total estimated health and social care costs, including the costs of CR over 9 months, were £3998 (SE £539) in the CR group and £4556 (SE £815) in the TAU group. Societal costs were of the order of five times higher. Again, although the mean health and social care costs over 9 months were quite modest, there was a small number of high-cost cases in the TAU group, with the costs exceeding £130,000. A total of eight outliers were identified following the adjusted box-plot method: three in the TAU group, ranging from approximately £28,000 to £133,000, and five in the CR group, ranging from approximately £31,000 to £63,000. In contrast, no outliers were identified by this method in the societal costs.

Cost-effectiveness analyses

Incremental costs and outcomes: person with dementia

Cost-effectiveness

The ICER point estimates are given in Table 38. The cost of an increase of 1.32 points in the BGSI attainment rating was £1296 from the health and social care perspective and –£9 from the societal perspective. The probability of cost-effectiveness (Figure 5) was over 99% at a WTP of £2500 from the health and social care perspective and from the societal perspective, and over 50% at a WTP of £1300 or more. The distribution of costs and BGSI attainment rating differences on the cost-effectiveness plane is illustrated in Figure 6. Because the difference in health and social care costs was significantly greater than zero and the difference in the outcome was also significantly greater, the cloud of cost–outcome pairs lies mostly in the north-east quadrant.

TABLE 38 - Person with dementia and carer: ICERa; point estimates for CR over TAU, from the health and social care and societal perspectives

NA, not applicable.

Cost of achieving a 1.32-point difference between groups at 9 months.

Cost of achieving a 1.53-point difference between groups at 9 months.

Cost of achieving a QALY gain over 9 months; difference in QALY rounded to first non-zero decimal place.

Perspective	BGSIa (n = 407)	GSESb (n = 389)	QALY
			DEMQOL-Uc (n = 401)	EQ-5D-3Lc (n = 390)
Person with dementia: 9 months
Health and social care	1474/1.37 = 1296	1028/0.23 = 4470	1110/0.001 = 1,110,000	N/A
Societal	–13/1.37 = –9	–681/0.23 = –2961	–526/0.0005 = –1,052,000	N/A
Carer: 9 months
Health and social care	NA	NA	NA	632/0.001 = 632,000
Societal	NA	NA	NA	–902/0.001 = –902,000

FIGURE 5.

Cost-effectiveness acceptability curve: BGSI score (person with dementia).

FIGURE 6.

Cost-effectiveness plane: incremental costs and end-point difference for the BGSI score at 9 months (person with dementia).

On the GSES, the cost of attaining an increase of 1.53 points (ICER point estimate) was £4470 from the health and social care perspective and –£2961 from the societal perspective. The probability of cost-effectiveness (shown in Figure 7) was 76% at a WTP of £50,000 from the health and social care perspective and 79% at the same WTP value from the societal perspective. However, as can be seen in Figure 8, the position of the cloud of societal cost–outcome difference pairs covers all four quadrants of the cost-effectiveness plane such that in any quadrant, no line drawn from the origin could exclude 2.5% of the joint cost–outcome distribution. There is no WTP at which it would be possible to be confident that CR would be more cost-effective than TAU (or vice versa) on this measure. 153

FIGURE 7.

Cost-effectiveness acceptability curve: GSES score (person with dementia).

FIGURE 8.

Cost-effectiveness plane: incremental costs and end-point difference for the GSES score at 9 months (person with dementia).

Cost–utility

The cost per QALY derived from the DEMQOL-U (ICER point estimate) (see Table 38) was £1,110,000 from the health and social care perspective. The ICER was negative (–£1,052,000) from the societal perspective, with the cost being somewhat lower (difference of £526, 95% CI –£3108 to £1927) in the intervention group from this perspective. There were no differences between the groups in terms of QALY gain. The probability of cost-effectiveness, shown in Figure 9, was very low at all WTP values per DEMQOL-U QALY (from £0 to £50,000) from the health and social care perspective; the probability of cost-effectiveness was just at or under 65% for all values of WTP over the same range. As illustrated in Figure 10, the cloud of societal cost–outcome difference pairs covers all four quadrants of the plane in approximately equal proportions, indicating that it is not possible to be certain that either strategy is cost-effective by reference to QALY gains at any level of WTP.

FIGURE 9.

Cost-effectiveness acceptability curve: QALY [DEMQOL-U (person with dementia)].

FIGURE 10.

Cost-effectiveness plane: incremental costs and QALYs (DEMQOL-U) at 9 months (person with dementia).

Incremental costs and outcomes: carers

Cost–utility

The cost per QALY for the carer, derived from the EQ-5D-3L (see Table 38), was £632,000 from the health and social care perspective; the ICER was negative (–£902,000) from the societal perspective, with costs being somewhat lower in the CR group than in the TAU group (by £902, 95% CI –£3616 to £1705). There were no differences in terms of QALYs between the CR and TAU groups. The probability of cost-effectiveness, shown in Figure 11, was between 17% and 22% at a range of WTP per EQ-5D-3L QALY values between £0 and £50,000 from the health and social care perspective, and approximately 74% across this range from the societal perspective. The cloud of cost–outcome pairs (Figure 12) is fairly evenly distributed across all four quadrants of the plane, suggesting no certainty in the cost-effectiveness of the intervention versus TAU (or vice versa) at any WTP from this perspective.

FIGURE 11.

Cost-effectiveness acceptability curve: QALY [EQ-5D-3L (carer)].

FIGURE 12.

Cost-effectiveness plane: incremental costs and QALYs (EQ-5D-3L) at 9 months (carer).

Sensitivity analyses

We investigated the robustness of results to several key assumptions made in the base-case analyses.

Strengths and weaknesses

The data on the care and support received by participants who finished the trial were well completed, with few items missing in most cases. The evaluation was able to collect detailed information on both the number and the duration of CR visits made to participants, and thereby the per-participant costs of the CR intervention could be estimated. Cost-effectiveness analytic methods have taken account of the correlation between the cost and outcome error terms and presented information on the cost of achieving improvements in outcomes in light of sampling uncertainty.

The analyses encountered some issues. Cost data in the complete-case sample were skewed and there were a number of extreme outliers in the case of health and social care costs. The analyses therefore combined a parametric approach (SUR models, which assume normality) with non-parametric bootstrap sampling, which makes no distributional assumptions. A large number of replications were necessary to produce more efficient bias-corrected SEs for the cost difference in the complete-case analyses. Other analytic models that permit a mixture of distributions for costs and effects could be investigated, for instance, gamma-normal or gamma-beta bivariate models, estimated using Bayesian techniques. 155 The 3-month costs in the interval between 3 and 6 months post baseline were assumed to be the same as the costs between 6 and 9 months post baseline, and the total costs could be overestimated if these were in fact more similar to costs over the initial 3 months post baseline or were rising over this interval.

Concerns with the validity of the EQ-5D86,156,157 to reflect the HRQoL of people with dementia led us to choose DEMQOL, a condition-specific measure, as an outcome for the economic evaluation. The authors of the instrument recommended using the EQ-5D alongside DEMQOL,86 as they considered that the instrument required further testing to better understand its psychometric properties. However, the EQ-5D was not included in the battery of questionnaires, and so whether or not using that instrument would have changed the assessment of the cost-effectiveness of the CR intervention cannot be assessed. Emerging evidence suggests that the EQ-5D and DEMQOL capture different aspects of changes in HRQoL over time. 158 In this study, it is possible that the DEMQOL domains (social relationships, loneliness, negative emotion, positive emotion) were better suited to the types of goals identified by participants (engaging in conversation, knowing what is happening, keeping track of information).

We did not examine the budget impact of introducing the intervention at a national level as part of the economic evaluation. An implementation study following on from the trial will combine the data from the original GREAT with observational data from at least 15 participating sites to model the impact of scaling up the intervention at local and national levels.

The BGSI is a relatively new instrument; no societal WTP threshold has been established for attaining an improvement in the BGSI attainment rating, such as that associated with a QALY gain in the NHS in the deliberations of the National Institute for Health and Care Excellence. 159

Implications

The results indicate that in terms of achieving an improvement in goal attainment as rated by the participant, the intervention is cost-effective from the health and social care and societal perspectives at WTP values of ≥ £2500. This achievement does not appear to have affected parallel improvements in participants’ sense of self-efficacy, or participants’ or carers’ QALYs (assessed by HRQoL and by quality of life in dementia, respectively) or to have reduced costs from the health and social care or societal perspectives. Considering the QALY results, there are a few possible reasons for the apparent lack of effect. The intervention was not expected to affect survival, so there was no reason to expect a difference between groups in the quantity of months lived. In addition, the goals that were chosen by each individual participant would vary depending on that individual’s circumstances and interests. The goals that were set might therefore be unrelated to the domains covered by DEMQOL; furthermore, goals that were set by the participant would not necessarily impinge upon domains covered by the EQ-5D, which was chosen to measure carer quality of life (e.g. the carer’s self-care and usual activities).

Funding decisions regarding CR programmes may not be limited to cost-effectiveness in terms of the cost per QALY gained by the participant or carer (by which measures the intervention was not cost-effective). There is also a need to consider the value of goal attainment – the primary focus of this trial – in the context of other information provided in this report (see the evidence on clinical effectiveness in Chapter 3). The costs of implementing CR in the trial included the costs of substantial supervision for the therapists. In routine clinical practice, these levels of supervision and support would be lower, thus reducing the costs of CR. In turn, the different level of supervision and support could affect the outcomes achieved, although we cannot examine that possibility with the data from the present trial. A follow-on implementation study of CR delivered at scale in routine practice will provide an opportunity to examine these possibilities.

Relating the findings on clinical effectiveness to other literature

We updated the Cochrane review covering CR during the course of the trial. 15 At this time, there were no further RCTs of CR, correctly defined, that could be included, and our 2010 pilot63 trial remained the only available trial. At present, we are preparing to update the review again, and to this end a new systematic literature search using the search terms and methods outlined in the published review was conducted by the Cochrane Dementia and Cognitive Improvement Group co-ordinating centre on 9 May 2017. We used the results of this search to check that the three potentially relevant trials published since 2013 of which we were already aware appeared in the output, and to identify any additional potentially relevant trials, of which we found one, published in 2017. Of these four trials, one described what appeared to be a CR intervention, although it is also described as ‘training’,160,161 one described a mixed CR and CT intervention162 and two described the structured training of activities of daily living; one of these variously described the intervention as ‘cognitive rehabilitation’ or ‘cognitive training’,163 whereas the other described the intervention as ‘structured relearning’ or ‘training’. 164

The most important of these trials for present purposes is the ETNA3 (Évaluation de 3 Thérapies Non médicamenteuses dans la maladie d’Alzheimer trial,160,161 a large trial (n = 653 participants) conducted in France, which compared individual CR with group CT, group reminiscence and usual treatment for people with mild to moderate Alzheimer’s disease (MMSE score of 16–26 points). Participants in the CR group received individual 90-minute sessions weekly for 3 months and then 6-weekly for the next 21 months, and their carers received telephone support weekly for 3 months and 6-weekly for the next 21 months. This equates to approximately 28 90-minute sessions of CR for participants and 28 sessions of telephone support for carers. The CT and reminiscence conditions received group sessions weekly for 3 months and 6-weekly for 21 months, whereas their carers participated in parallel-group psychoeducation sessions.

The CR intervention in ETNA3, which was delivered by psychologists, was not well described. Furthermore, an unspecified proportion of participants in this group did not receive CR, but instead followed an individualised reminiscence programme; the reason for this is unclear. For those who did receive CR, the goals involved either improving an activity of daily living or maintaining a leisure activity and had to be personally meaningful, although it is not clear how this was defined or in what way participants and carers were involved in choosing goals. The goals to be addressed were defined in the first two sessions and could be changed by the treating psychologist at any time. The approach used to address goals is described as ‘training’ of particular ‘activities’, without any further explanation, so it is possible that the intervention was closer to the structured training approaches described below than to the approach used in GREAT.

The design of the CR intervention in ETNA3 appears to have drawn on some of our early feasibility studies, which are cited, but the authors do not acknowledge our pilot trial, even though they cite the updated Cochrane review,15 in which the pilot trial is discussed. In their 2013 paper,160 they incorrectly stated that there were no published RCTs of CR, and in their 2015 paper they modified this to state that there is ‘no large RCT’. Some of our early feasibility studies focused on the possible benefits of errorless learning techniques, building on work on this topic in the brain injury field, and these techniques were used in ETNA3,161 in which ‘the psychologist could rely on “errorless learning procedure” to train a particular activity’. However, by 2008 we had accumulated evidence ourselves and synthesised other available evidence, showing, first, that people with early-stage Alzheimer’s disease appear to learn equally well with errorless and trial-and-error methods and, second, that even for groups for which it does convey benefits, errorless learning is more suited to some types of task than others. 59,60 Interestingly, the most recent of the four trials identified164 also failed to take note of this evidence; the REDALI-DEM (RElearning methods on DAily LIving task performance of persons with DEMentia) trial164 assigned participants to structured relearning of activities of daily living using either errorless or trial-and-error instructional methods, and found no differences between the two groups, indicating, as we would expect, that the two types of learning strategy were equally effective.

The primary outcome in ETNA3 was survival without moderately severe to severe dementia at 2 years. This appears to suggest that the three psychological therapies were being evaluated as disease-modifying treatments expected to alter the trajectory of decline. This is an unusual and probably unrealistic expectation for interventions of this kind. Not surprisingly, in ETNA3 none of the three treatments was effective in relation to this outcome. In relation to the kinds of outcomes more typically evaluated in trials of these kinds of interventions, evidence was already available in 2003 to show that CT is not effective in short- to medium-term outcomes for people with dementia,14 and a recent definitive trial of group reminiscence failed to show any benefits in primary or secondary outcomes compared with usual treatment. 165 These findings were supported by ETNA3, as neither CT nor reminiscence yielded any benefits over usual care. With regard to CR, there was no direct assessment of functional outcomes for the CR group, and no information is provided about whether or not goals were addressed successfully or if any changes in behaviour or perceptions of behaviour were seen as a result of the intervention. However, in secondary outcomes, CR was the only one of the three interventions to show any benefits relative to usual care. Participants in the CR group had lower functional decline at 24 months, measured on the Disability Assessment for Dementia,166 a 6-month delay in institutionalisation compared with the usual-treatment group and lower rates of institutionalisation than all other conditions. There were no other significant differences in secondary outcomes, but the authors noted trends in neuropsychiatric symptoms, caregiver burden and service utilisation. The authors concluded that cognition-focused group interventions are not effective and that individualised CR interventions should be used to delay institutionalisation for people with mild to moderate Alzheimer’s disease.

Another trial combined CR with CT exercises and cognitive strategy training. Kim162 reported a small trial (n = 43 participants) conducted in South Korea, in which participants with early-stage Alzheimer’s disease (MMSE score of ≥ 18 points) were randomised to receive eight weekly sessions of either CR (n = 22) or an active control condition (n = 21) consisting of group meetings with structured conversation and health-related videos. The CR sessions consisted of 30-minute group CT exercises and cognitive strategy training and 30-minute individual CR, focused on addressing a personally meaningful goal. The primary outcome was goal performance and satisfaction rated on the COPM. 38 The CR group improved significantly in goal performance and satisfaction, and quality-of-life scores also improved, whereas participants in the control group did not show any changes. However, this was a small trial yielding evidence of limited quality; analyses were restricted to separate pre–post comparisons for each group using t-tests, so the results should be interpreted with caution.

In the other two trials identified, by Voigt-Radloff et al. 164 and Thivierge et al. ,163 the intervention involved structured training of activities of daily living.

Voigt-Radloff et al. 164 report an adequately powered trial (n = 161 participants) conducted in Germany, in which participants were randomly assigned to receive nine 1-hour sessions in which everyday tasks were trained through either errorless (n = 81) or trial-and-error (n = 80) instructional methods. Up to three sessions were devoted to choosing the tasks to work on, which were selected from a list of 43, covering household tasks, leisure activities and cognitively challenging tasks. Task performance was assessed with ratings based on observation. Performance improved in both groups with no differences between the two instructional methods. There were no changes in secondary outcomes for either group. As noted above, this trial failed to take note of available evidence, indicating that both types of learning strategy would be expected to be equally effective. 60

Thivierge et al. 163 report a small trial (n = 20 participants) conducted in Canada, using a block-randomisation wait-list control design. This tested an intervention that was variously described as ‘cognitive rehabilitation’ and ‘cognitive training’, but appeared to involve structured training of instrumental activities of daily living (IADLs). In this study, following assessment to identify problematic IADLs, participants and carers selected one to be the focus of training. Participants were trained in the chosen activity twice a week for 4 weeks, using errorless learning and expanding rehearsal methods, and they practised between sessions. Observational assessment indicated that task performance in the trained group improved and these gains were maintained at 3 months, but the groups did not differ on any other outcome measures on completion of the training or 3 months later.

These four trials taken together all focus on functional ability. However, they reflect two distinct approaches: one is goal-oriented CR addressing functioning in real-life contexts and the other is a form of CT involving structured training in and practice of everyday tasks. The appearance of a form of CT applied to everyday functional activities suggests that the concept of CT, which describes practice on abstract cognitive tasks completed with pencil and paper or via a computer, is being adapted for people with dementia to address the kinds of functional activities undertaken in everyday life. However, structured training does not address issues of transfer and generalisation of learning to the real-life setting,163,164 and the learning may never be integrated into or used in daily life. The focus on relevant everyday tasks in these studies is very positive, but the potential difficulty with generalisation and integration into daily life precisely illustrates the reason why we took a different approach in GREAT and focused on improving functioning in the real-life setting with the activities participants were actually undertaking.

In terms of effectiveness, the results from these trials confirm that the interventions produce improvements in the specific areas targeted and clearly demonstrate that it is possible to improve functional ability for people with mild to moderate dementia. However, with the exception of the functional disability measure in ETNA3, the interventions have no effect on scores on secondary outcome measures, such as quality of life or carer burden. This is consistent with our findings in GREAT. One possibility is that changes in everyday functioning or the ability to carry out specific activities, although very important in their own right and potentially very beneficial, simply are not associated with appraisals of quality of life or carer stress. The quality of life of people with dementia, for example, is associated to a small degree with many different factors, and changes in one area may have little impact overall. 167 Another possibility is that changes in functional ability do have effects in other areas of life, but we do not have outcome measures that are sensitive to these changes. This is suggested by the responses identified from in-depth interviews with GREAT participants and carers. These interviews were undertaken and analysed by researchers otherwise independent of the trial, to reduce the risk of positive response bias, and it seems unlikely that participants and carers would make strong statements about feeling more confident and better able to cope if they did not genuinely feel some benefit in these areas alongside their observations of improved functional ability.

Goal-setting

Goals identified by participants and addressed in therapy reflected the multiple ways in which cognitive impairment has an impact on everyday life for people with mild to moderate dementia. Some participants used goal-setting as a means to promote engagement in activities. Other aspects of the need for engagement were reflected in goals focusing on keeping in contact with family and friends and engaging in conversation. Some participants focused on managing everyday tasks and being able to use household appliances or devices, such as mobile phones, to help conduct their daily lives and to occupy and entertain themselves. A small number of goals focused on basic aspects of self-care, such as washing and dressing. Many goals reflected the challenges of living with memory difficulties; participants wanted to be well oriented, organised and better able to retain or keep track of information and events, locate belongings and recognise, identify and name people and objects. Managing emotions was an issue for a handful of participants.

Furthermore, a more detailed analysis of the goals participants chose will help to further develop the therapy, especially when goals can be cross-referenced with therapy logs to illustrate the different ways in which cognitive impairment affected functional ability, the problems that therapists identified and the strategies therapists used to overcome or offer solutions for these problems and help participants to achieve their goals. This will provide valuable information to support wider implementation.

It is important to note that some participants who initially expressed interest in GREAT and were assessed at baseline did not proceed to randomisation, because they felt content with their current situation and were unable to identify any areas of need in which they could formulate goals. CR does require active engagement and hence will not be appropriate for everyone. However, it is possible that carers in these cases had a different perspective and may have benefited from support in solving problems and developing strategies. In clinical practice, there may be opportunities to work directly with carers under these circumstances, even if the person with dementia does not wish to engage.

Perceptions of participants and carers

The low attrition rates and good adherence rates, together with the positive evaluations recorded in the qualitative interview data, suggest that the therapy was very acceptable to those participants and carers who opted to take part. In addition to the specific focus on improving everyday functioning and the opportunity to develop new or more effective strategies, they valued the person-centred approach, the relationship they developed with the therapist and the support this provided. The strong emphasis on the therapeutic relationship as the vehicle for change is instructive in terms of future implementation. Some people with dementia and their carers may be able to make use of information about the approach and strategies they could use to engage in self-management, but most are likely to require input from a therapist who can build trust and provide support in finding solutions to everyday challenges.

The qualitative data revealed some areas in which participant and carer perceptions might have been managed better. First, a few people were disappointed that, although the intervention improved functioning, it did not improve memory. CR was not presented as an intervention to improve memory or cognition per se, and indeed this is something that would have been made explicit from the start. Possibly this was not sufficiently explained in some cases, or perhaps it may have needed to be emphasised more throughout the therapy. Second, a few carers wondered if it was worthwhile intervening, given that dementia would progress anyway. This is a completely understandable reaction from carers who are facing, and perhaps grieving over, the gradual decline they observe in their relative, but also suggests that the aims of rehabilitation and what it can realistically achieve may have needed more explanation in some cases in order to convince carers of the value of optimising functioning and reducing excess disability at any stage of dementia. It will be important to ensure that these messages are conveyed effectively in any future implementation, so that people with dementia and their carers have realistic expectations about what CR can help them to achieve.

Challenges in delivering the intervention

Therapists were able to deliver the intervention in line with the protocol. They could successfully engage participants and carers, explain the rationale for the CR approach and conduct individually tailored interventions, addressing goals that participants and carers identified as relevant and meaningful. Therapists in GREAT faced the challenge of working with goals that had been set prior to their first meeting with the participant and carer as part of the baseline assessment. In clinical practice, and hence also in any future implementation, goals would typically be negotiated during the initial sessions with the therapist. Therapists noted the importance of being able to explain the therapy approach and the kinds of strategies used in a way that was accessible to participants and carers.

Therapists were expected to, and did, draw on their wider clinical skills and experience by developing good therapeutic relationships, managing relationship conflict between participants and carers when this emerged during sessions, providing information and, when necessary, making onward referrals, for example to social services. These non-specific elements of the therapy need consideration in preparing therapists to work in this way in any future implementation, especially when CR is delivered by less qualified staff, and need to be taken into account in training and in providing appropriate supervision arrangements.

Identifying who is most likely to benefit

Rehabilitation focuses on ‘doing with’ rather than ‘doing for’ or ‘doing to’. 21 For people in the early stages of dementia, CR requires some degree of active engagement. This implies that participants need to be able to identify something that they would like to change, improve or manage better. It is not necessary for them to acknowledge a specific dementia diagnosis or even the full range of difficulties they may be experiencing, but there needs to be something that they want to work on. This is not the case for everyone. We have conducted detailed investigations of the awareness of difficulties among people with dementia; these show that a small proportion of people with dementia are unwilling or unable to acknowledge any difficulties169 and a larger proportion underestimate the impact of memory problems,47 although there is some evidence that people with early-stage dementia are more accurate in estimating their own functional ability than their carers. 48 Even when difficulties are acknowledged, people may have reached an acceptance of these or may not wish to make the effort required to bring about changes. Approximately 5% of the people with dementia assessed at baseline said that they were content with their situation and could not identify any areas in which they felt changes were needed, and hence did not proceed to randomisation. Participants joining GREAT were able to identify areas for improvement and to evidence some motivation for change. Nevertheless, goal-setting could be challenging for some individuals, who required more time and support to identify suitable goals. As noted above, although in the trial goals were set as part of the initial assessment by the researcher, in usual practice, goals would be negotiated by the therapist and could evolve over a number of therapy sessions, making this process more accessible.

Exploratory statistical analyses revealed few predictors to indicate those participants who were likely to show the greatest gains in goal attainment. Ratings of readiness to change at baseline and the number of sessions completed were associated with greater gains. At 3 months, participants from professional occupational backgrounds had better outcomes than those from other occupational groups according to both participant ratings and carer ratings. At 9 months, the MMSE score was predictive, and participants with MMSE scores of ≥ 24 points had better outcomes according to both participant ratings and carer ratings than those with MMSE scores of ≤ 23 points, whereas participants aged < 75 years did better than those aged ≥ 75 years according to participant ratings only. Centre, sex, diagnosis, medication use and the presence of comorbid conditions were not linked to outcomes.

Data from the therapy logs analysis supplement these findings and suggest that participants with the best outcomes tended to have relatively good cognitive and functional ability, to be socially and physically active and to be highly motivated. These participants were likely to acknowledge their difficulties and express anxiety about the impact of these, and to focus on challenging goals relating to IADLs or increasing engagement in social or leisure activities, and progress was evident early in the course of therapy. Participants who made the least progress were likely to have more extensive cognitive and functional difficulties and lower motivation, and were less likely to acknowledge difficulties or express anxiety about them. For these participants, goals were more likely to relate to basic activities of daily living, and progress followed a slower trajectory.

In GREAT, participants had to have a carer who was willing to contribute, so that we could obtain collateral information, such as carer ratings of goal attainment. However, in naturalistic settings, people with dementia may not have a carer or may have a carer who is not able to support the intervention. This would not preclude them from engaging in CR, but it would be an important factor for the therapist to consider when planning the intervention.

Differing levels and types of need

In discussing their work in the focus group, the therapists seemed to distinguish different groups of participants based on levels of cognitive and functional ability and on time since diagnosis. These different groups were perceived to have different needs and to respond differently to the intervention. Participants who had received a dementia diagnosis in the previous few months, and their carers, were in the process of adjusting to dementia and were actively seeking strategies and solutions to help to manage their everyday lives. Those who had been living with dementia for longer tended to have adapted to living with the condition and to be less motivated to make changes, and in some cases carers felt that they had already tried various strategies without much success. Although participants in this latter group also benefited, working with these participants presented more of a challenge for the therapists.

This provides valuable guidance on how to target the CR intervention in future implementation. Participants in GREAT had mild to moderate dementia, but this broad grouping encompasses a wide range of cognitive and functional ability. In terms of UK NHS care pathways, it spans both the cluster 18 cognitive impairment (low need) and cluster 19 cognitive impairment (moderate need) care clusters. The principles of CR can be applied in different ways to optimise functioning, depending on need, and somewhat different approaches are likely to be required for different groups. The protocol used in GREAT, although applied effectively with both groups, appears to have been seen by therapists as more suitable for ‘cluster 18’ participants.

Cognitive rehabilitation may be a particularly valuable approach for people in the months following a dementia diagnosis and could represent an important component of post-diagnostic support within the first year of being diagnosed. For this group, the aim would be to support people in remaining active and engaged, maintaining confidence and developing a range of strategies to support practical and emotional coping. An additional element for some, although not attempted in GREAT, might be vocational rehabilitation to support continuation in employment or transition to less demanding employment or voluntary roles.

For people who have been living with dementia for longer, the aims may be more about maintaining basic skills of daily living, limiting the effects of excess disability, encouraging people to remain socially engaged and ensuring opportunities to participate in pleasurable activities. As these individuals may find it harder to identify goals, carers are likely to be more involved in the goal selection process. An additional element for some might be the use of CR to support people with dementia who are returning home after a period of hospitalisation caused by illness or injury, or who have additional physical or mental health conditions.

This highlights the need for flexibility in applying the principles of CR to support optimal functioning for people with dementia. Although GREAT followed a structured protocol, CR itself is not a fixed intervention and can be adapted to different contexts to meet a variety of needs. 3 Although the number of sessions was fixed for the trial, in practice the duration of the intervention could vary and could be tailored to participants’ needs. Some individuals may need only one or a few sessions to address specific issues, whereas others may need longer periods of support to develop and implement strategies, or perhaps to regain function after a period of illness or hospitalisation. Some may benefit from several short episodes of CR as their needs change over time. In the pilot work on implementation that we conducted in NHS sites, we trialled shorter, 6- and 8-session protocols as a starting point. However, experience with GREAT suggests that the group of people defined as having early-stage Alzheimer’s or vascular or mixed dementia could be further subdivided into two broad groups reflecting differing degrees of dementia progression. In future implementation, it will be important to prepare CR intervention protocols that meet the needs of each of these groups, to identify optimal timing and duration, and to explore outcomes accordingly.

Trial design

This was a pragmatic trial comparing CR with TAU. This design did not provide a means of controlling for the time and attention provided by the therapist, raising questions about whether treatment gains are specific to the therapy or non-specific. In this instance, however, the gains observed related specifically to the effects of CR and were demonstrated in improvements in goal attainment for goals directly targeted in the therapy. It is unlikely that these could be attributed to non-specific effects of the intervention. Furthermore, in the pilot trial, CR demonstrated effectiveness when compared with an active control condition, relaxation therapy, with an equivalent number of therapist visits as well as when compared with TAU.

In trials of behavioural interventions that involve active participation and engagement, it is not possible for participants to remain blind to their group allocation, especially when, as in this case, the intervention is compared with TAU, and this creates the potential for bias in responding on self-reported outcomes. The inclusion of parallel carer ratings may go some way towards mitigating this concern. Furthermore, if bias were present, we would expect to see it on all self-reported measures, and not just the ratings of goal attainment.

All possible precautions were taken to ensure that researchers collecting follow-up data remained blind to participants’ group allocation. However, effective blinding is extremely difficult to achieve in trials of psychosocial interventions that do not include an active control condition. Despite all precautions taken, researchers conducting assessments are likely to surmise whether or not the participant received the intervention, as was the case in GREAT. It could possibly be argued that the statistically significant association of blinding inefficiency with greater improvement in BGSI ratings at follow-up could reflect researcher bias. However, it is more plausible to consider that the ability of the researchers to correctly identify which participants had received the intervention reflects the close relationship between the CR therapy and the outcome measure. The BGSI provides a proximal measure of treatment outcome that directly assesses whether or not the intervention addressed its intended targets. Participants who received CR would be expected to be more engaged with the personal goals identified and to remember them better. These participants would also have made ratings of progress with these goals in sessions with the therapist. Therefore, it is likely that if a participant had engaged well with the CR intervention and derived benefit from working on the identified goals, this would have been evident when making the BGSI ratings in the follow-up assessment, and consequently the researcher would have been able to correctly identify the participant’s group allocation. Conversely, if it was not evident to the researcher that a participant who received CR was allocated in this way, it is likely that the participant was less engaged with the goals identified in the BGSI and that the treatment was less effective.

Outcome measures

Primary and secondary outcomes in GREAT were evaluated mainly through participant-reported outcome measures, with collateral information from carers. This was a positive choice, aiming to foreground the perspective of people with dementia and carers. It was also a practical choice, because participant numbers and the focus on behavioural change in real-life situations precluded observational assessment as a means of evaluating therapy outcomes, and because, as CR was not expected to be a disease-modifying treatment, there was no particular justification for the use of physiological markers. The pilot trial63 included a neuroimaging component, which examined whether or not participation in therapy was associated with changes in particular brain areas seen on functional MRI; the findings were difficult to interpret, especially as only 28% of participants were able or willing to undergo MRI scanning, but some differences in activation patterns between CR and control participants were found following intervention, suggesting that the intervention produced a degree of restoration of function in frontal brain areas. 66 Physiological markers, therefore, may have some relevance, but the crucial question is a behavioural one: whether or not we can improve functioning in everyday life to enable people with dementia and carers to manage everyday life more effectively and with greater satisfaction. For the present trial, therefore, the focus was on behavioural change and on the perceptions of participants and carers.

The primary outcome measure, the BGSI, proved sensitive to changes resulting from the intervention. This used a valid and reliable rating method and participant responses were corroborated by the independent carer ratings. Ratings were collected by a blinded researcher not involved in the therapy, and given that scores on secondary outcome measures showed no changes, it is unlikely that changes in BGSI ratings in the CR group reflected a positive response bias. As discussed above, it is possible that secondary outcome measures were not sensitive to other changes resulting from the therapy. One limitation in outcome measurement is the lack of inclusion of a measure of functional ability. We did not have scope to include a long-term follow-up to assess whether or not the intervention had any impact on rates of institutionalisation. In future trials, this is more likely to be feasible through the use of routine data and data linkage, but this option was not available to us when designing GREAT.

Participant selection

Because of the need for collateral information, we excluded any potential participants who did not have a carer available to contribute. This will have encompassed those who had a carer who was unwilling to contribute and those who had no carer involved and were perhaps living alone. People with dementia in either of these groups may be most in need of psychosocial support, and in clinical practice it would be important to be able to offer CR in these circumstances. Consideration would need to be given to the best way of providing CR when there is no carer involved; this could include options, such as working with volunteers or incorporating between-session contact with the therapist, as a means of supporting practice with targeted activities.

Participation was restricted to people with Alzheimer’s disease or vascular or mixed dementia. Although this accounts for the majority of dementia diagnoses,74 people with other forms of dementia also have needs that could be addressed through CR. Although GREAT was in progress, we undertook a pilot trial with people who have either Parkinson’s disease dementia or dementia with Lewy bodies. 170,171 This pilot trial replicated the methods used in our pilot trial of CR63 and yielded similar results. This indicates that CR, in a similar form to that used in GREAT, is feasible and potentially effective for people with dementias associated with Parkinson’s disease, and provides justification for a larger trial. 172 There is scope for future exploration of the adaptation of CR for people with other rare dementias.

Experiences

The researcher will explore the participant’s experiences and feelings starting with general questions:

How did you find the therapist’s visits over the past few weeks?

What was it like to work with the therapist on your goals?

More specific information will then be elicited using prompts, such as the following:

What were the more enjoyable things about your work with the therapist?

What were the less enjoyable things about your work with the therapist?

What aspects of your work with the therapist were more/less helpful?

Did you find it hard work to take part in the therapy?

What aspects of your work with the therapist were most challenging?

The interviewer will encourage the person to give specific examples, when possible.

Outcomes

The researcher will explore the impact of taking part in CR on the person’s everyday life and self-perceptions.

What difference (if any) has your work with the therapist made to your daily life?

Has the experience changed anything in the way you think about your dementia/about yourself/about the future?

Has the experience changed anything in the way you relate to your carer/family?

The researcher will draw on positive comments from the participant to end the conversation on a positive note.

Experiences

The researcher will explore the carer’s experiences and feelings starting with general questions:

How did you find the therapist’s visits over the past few weeks?

How do you think [your relative] felt about working with the therapist on his/her goals?

More specific information will then be elicited using prompts, such as the following:

What were the more enjoyable things about working with the therapist on [your relative’s] goals?

What were the less enjoyable things about working with the therapist on [your relative’s] goals?

What aspects of working with the therapist were more/less helpful?

Do you think [your relative] found it hard work to take part in the therapy?

What aspects of working with the therapist were most challenging?

The interviewer will encourage the person to give specific examples, when possible.

Outcomes

The researcher will explore the impact of taking part in CR on the person’s everyday life, and the perception of the person with memory difficulties.

What difference (if any) has the therapy made to your daily life?

What difference (if any) has the therapy made to [your relative’s] daily life?

Has the experience changed anything in the way you think about [your relative’s] memory difficulties/about [him/her]/about the future?

Has the experience changed anything in the way you relate to [your relative]?

The researcher will draw on positive comments from the carer to end the conversation on a positive note.

Bangor

Eight participants with dementia, five of whom were female, received the intervention. Five had a diagnosis of Alzheimer’s disease, one had a diagnosis of vascular dementia and two, although known to have dementia, were still awaiting a specific diagnosis from the memory service. Those with a diagnosis had received this within the previous 12 months. The mean ACE-III score was 71 points (range 62–78 points). Six participants had a carer involved; the carers were five spouses and an adult child. Five participants received a six-session intervention and three received an eight-session intervention.

Participants identified a total of 16 goals: two identified one goal, four identified two goals and two identified three goals. The goals included learning to use new technology, remembering names and activities undertaken or planned, locating items around the house, orientation to the current day and date, planning activities and engaging in social contact and community activities.

Participant and therapist pre- and post-intervention ratings of goal attainment and satisfaction with attainment were available for 13 goals. Carer ratings were available for nine goals. Mean scores improved significantly post intervention, as shown in Table 39. Scrutiny of individual goals indicated that a clinically meaningful improvement of at least 2 points was obtained for attainment of 12 out of 13 goals as rated by participants.

Occupational therapists and TIs also rated the extent of goal attainment post intervention by matching current functioning to the goal statements and goal attainment descriptors identified at the initial assessment. Nine out of 16 goals (56%) were rated as fully achieved, six were rated as partially achieved and only one was rated as not achieved.

The mean DEMQOL scores, when available (n = 7), increased from 90.29 (SD 12.55) at initial assessment to 96.14 (SD 6.47) at the end of the intervention, reflecting more positive ratings of quality of life, and the mean RSS scores (n = 6) decreased from 16.5 (SD 3.51) at initial assessment to 13.5 (SD 5.68) at the end of the intervention, reflecting lower levels of stress. However, the small numbers precluded statistical analysis.

Kent

Outcome information was available for five of the six service users included. Between them, these five service users worked on eight goals. BGSI attainment ratings were made by service users for all eight goals, by carers for seven goals and by therapists for four goals (two participants). The mean initial attainment ratings by service users were 3.13 (SD 1.46) and the mean post-intervention ratings were 7.75 (SD 0.89), reflecting a significant improvement [t(7) = –6.56; p < 0.001]. Carers’ mean initial attainment ratings were 3.43 (SD 1.27) and their mean post-intervention ratings were 7.86 (SD 1.07), which was a significant improvement [t(6) = –7.75; p < 0.001]. Therapists’ mean initial ratings were 2.75 (SD 0.96) and their post-intervention ratings were 8.25 (SD 0.96), again reflecting a significant improvement [t(3) = –8.52; p < 0.05]. Service user satisfaction ratings improved from a mean of 2.88 (SD 0.99) at initial assessment to 8.13 (SD 0.83) post intervention, also reflecting a significant change [t(7) = –9.98; p < 0.01]. Carers’ scores for stress decreased only minimally from 36.67 (SD 17.01) at initial assessment to 36.33 (SD 11.93) post intervention, and this was non-significant.

Training

Sufficient time is needed for initial training to enable staff to assimilate new information and engage with the approach. Staff prefer training spread over several days in shorter sessions rather than intensive 1- or 2-day workshops. Staff also need some study time to review training materials and consolidate their learning. Training needs to be flexible to take account of different staff backgrounds and levels of experience, and some types of staff will need more input than others. It may be helpful to provide follow-up training sessions once staff are engaged in delivering CR. With regard to the content of training, it is important for staff to understand how CR is different from their current everyday practice. Some staff may feel that they already set treatment goals and suggest strategies, and need to understand how this differs from the goal-setting and strategy application approaches used in CR, which is a more detailed and collaborative process based on careful assessment of intrinsic capacity, current ability and task demands, and which uses a problem-solving approach to identify goals and possible solutions and expresses goals in clear behavioural terms. Providing staff with a checklist to use following training, to remind them of what they should be considering in their work, may help to enhance and maintain adherence to the therapy model.

Supervision

Staff find supervision and access to advice extremely helpful and it is important that time is allocated for staff to participate in supervision, especially in the early stages of familiarisation with the approach.

Intervention

The adaptation to a six- or eight-session format worked well, and increasing the length of time between later sessions, for example by moving from weekly to fortnightly sessions, may be helpful. Telephone follow-up was helpful in some cases but not others, and should be used flexibly. The modular protocol was intended to allow for individual tailoring, but requires revision to enable staff to utilise the available guidance to tailor the intervention optimally for each person. It will clearly be important to limit ‘paperwork’ and recording requirements, and to ensure that any evaluation is not burdensome.

Service users

As with participants in the main trial, service users differed in the extent to which they were able to identify goals and in their motivation to make changes. It is important to be able to identify those service users who are likely to benefit from CR and, equally, to be able to offer other, more appropriate, kinds of support to those who are not likely to engage with the approach. It is also important that the intervention approach is sufficiently flexible to accommodate service users with different degrees of cognitive difficulty and that therapists are trained and supported to apply CR in ways that meet the needs of a range of service users.

Staff

It may be preferable to avoid staff being in the dual roles of care co-ordinator and CR therapist.

Service constraints

Unsurprisingly, time constraints, heavy workloads and staff shortages presented challenges for the implementation. The unexpected reorganisation at the Birmingham site was a classic example of the way in which wider issues can hinder implementation efforts. Future implementation work should be considered only in services for which reorganisation is not envisaged within the time frame of the work. A fuller implementation programme will need to address this through a top-down approach, by engaging service managers and key decision-makers. The consistent support of management at all levels, both senior managers and local line managers, is essential to achieve effective implementation and sustainability.