Computerised speech and language therapy or attention control added to usual care for people with long-term post-stroke aphasia: the Big CACTUS three-arm RCT

Participant identification

Participants were recruited from 21 speech and language therapy departments in 20 NHS trusts (see Appendix 1) across the UK, from both current and past patient records, speech and language therapy colleagues and contacts with longer-term voluntary support groups. Identification from past patient records and voluntary support groups was aimed at including participants who may have finished their speech and language therapy intervention based on currently available services. However, these potential participants would be eligible for additional/extended therapy if the Big CACTUS approach to providing more therapy through self-managed computer exercise was to be implemented in the future if found to be clinically effective and cost-effective. Speech and language therapy departments agreeing to participate in the project were asked to identify potential participants. Potential participants unknown to speech and language therapy departments and voluntary groups, who found the trial on the website, were able to self-present by contacting the central trial team who put them in touch with their nearest local NHS speech and language therapy departments in the trial where possible. Potential participants (those identified as having had a stroke, and a diagnosis of aphasia, at least 4 months post stroke, aged ≥ 18 years) were contacted by the research SLT in each participating NHS trust. This person was a member of the local clinical team who was appointed to take responsibility for the running of the trial and the trial intervention in their NHS trust. The participant was sent project summary information and followed up by a telephone call 1 to 2 weeks later to establish whether or not they were interested in the trial. If they were, the research SLT made an appointment to visit them at home. All trial procedures, including recruitment, intervention and outcome measures, were conducted in the participants’ own homes.

A screening log was completed by the therapist who identified potentially eligible patients from patient records, speech and language therapy colleagues or voluntary groups. Data recorded and sent back to the clinical trials research unit (CTRU) included unidentifiable information including initials, gender and age. The reason, if given, for not arranging an appointment was recorded.

Screening for eligibility

At the first visit to the potential participant, before providing detailed trial information, the research SLT determined whether or not the person was eligible. They requested verbal consent to undertake the naming test of the CAT,30 which is used in routine practice and can establish the severity of a person’s word-finding deficit. If the word-finding score was < 5 (10%) or > 43 (90%) (out of 48), an explanation was given to the patient that this type of computer therapy was not suitable for them. If they were still interested in computer-based therapies, they were directed to the aphasia software finder (www.aphasiasoftwarefinder.org; accessed 20 June 2018) developed to help patients with aphasia to identify software that would be most suitable for them. If the potential participant had eligible word-finding scores, the research SLT asked them to attempt a simple matching task on the computer to confirm ability to see the screen and perform simple tasks.

Recruitment

The level of support required to enable a person with aphasia to provide informed consent is dependent on the severity and profile of the aphasia. Considerable attention was given to the recruitment of participants with aphasia to ensure informed consent. In order to provide information in a format consistent with each individual’s language ability, the Consent Support Tool was used. 31 The research SLT at each site requested verbal consent from the potential participant to carry out part A of the Consent Support Tool (language screening test of 5–10 minutes). The result indicated which style of information would best support their understanding of the trial. Three different styles of information sheet were available to enable as many participants as possible to make their own decision regarding whether or not to consent to participation in this trial. The consent support tool, carer information sheets and consent forms can be accessed online (see www.journalslibrary.nihr.ac.uk/programmes/hta/122101/#/related-articles), as can aphasia-friendly/accessible information sheets and videos (see www.sheffield.ac.uk/scharr/sections/dts/ctru/bigcactus). Patient information sheet 1 was in large font with keywords emboldened for those who could understand written paragraphs. Patient information sheet 2 was for those who could read simple sentences but not full paragraphs. It followed standard aphasia-friendly principles with one idea presented per page in short simple sentences of large font. Keywords were emboldened and each idea was represented by a picture. Patient information sheet 3 was for those who could understand with significant support. Each idea/sentence was presented on a Microsoft PowerPoint^® (Microsoft Corporation, Redmond, WA, USA) slide with simple text, keywords emboldened and picture support. Research SLTs were instructed to present each point in turn, read aloud to the potential participant and support with gestures, objects and drawings. The next sentence was then presented. All participants were given sufficient time to consider their participation before informed consent was taken by the research SLT. Participants providing their own informed consent were asked to sign an aphasia-friendly consent form.

The Consent Support Tool also identified individuals who did not have the mental capacity to consent for themselves owing to the severity of their language difficulty (those with severe aphasia who find it difficult to understand information, even with the support of adapted/pictorial information formats). These individuals receive speech and language therapy treatment in practice and may benefit from the trial intervention; therefore, it was important to include them. These potential participants were shown a short video clip of the computer program being used and of someone being assessed to show what was involved. If potential participants with severe aphasia who lacked capacity to consent indicated an interest, a relative (in Scotland, the person’s legal representative or nearest relative) was asked to read the full information sheet and a covering letter detailing their responsibility, and to sign a carer declaration that they believed that their relative wished to take part (in Scotland, they were asked to sign a consent form). At the request of the PCPI group, all patients were given a copy of either the standard information sheet or the aphasia-friendly information booklet and a picture summary on one side of A4 paper.

For those participants with a carer, the carer was asked if they were willing to complete outcome measures related to their own quality of life and perception of their relative’s communication ability. They were provided with the carer information sheet and were asked to sign a consent form.

Usual-care control group

Usual care for people with long-standing aphasia following stroke varies across the country in terms of type, frequency and length of provision, and is dependent on available resources in each locality. To accurately describe UC provided to people with aphasia > 4 months post stroke, patients, carers and therapists were asked what therapy they had received in the 3 months before they were randomised. Therapy notes were then consulted to record the dates of therapy sessions, therapy goals, length of sessions, personnel providing therapy and the mode of therapy delivery. The UC received prior to randomisation is reported here based on the 244 participants who were at least 4 months post stroke at the start of the observation period. These data can be considered as a trial result. However, as describing UC was not a research objective, and as the data describe one of the trial interventions, the information has been reported in this section.

Self-managed computerised therapy intervention for word-finding (computerised speech and language therapy)

Attention control group

Primary outcome measures

Key secondary outcome measure

Secondary outcome measures

Staff training for delivery of outcome measures

All research SLTs attended a 1-day training course delivered by the central research team to ensure understanding of the protocol and trial procedures including how and when to administer outcome measures. It was understood that some participants may need more than one visit to complete the assessments. The flow of participants through the trial was the responsibility of the research SLTs (principal investigators) at each participating NHS trust. A flow diagram of activity was provided to assist with this (see Appendix 4).

Therapists responsible for carrying out assessments at all follow-up time points (6, 9 and 12 months) were trained in their responsibilities, the importance of blinding and how to administer outcome measures. Training was in small groups via a webinar delivered by the trial manager and the central team speech and language therapy researcher. It was recommended that they conduct each outcome measure visit within 1 month of each time point to ensure that outcome measure visits were spaced sufficiently to avoid presenting a burden to participants.

Collection of demographic data

Initial assessment was undertaken by the research SLT at each participating NHS trust. The initial assessment visit included collection of demographic data: aphasia type, age, gender, time post onset of stroke, and type and location of stroke (if known).

Recording of usual care

The research SLT at each NHS trust was asked to record UC provided to each participant throughout the trial to provide a description of UC and also to ensure that UC did not change once participants were randomised to the trial. UC included care provided by both the NHS and the voluntary sector. In addition, the research SLTs were asked to record UC provided in the 3 months prior to randomisation from participant and carer reports and from SLT notes. For each session, they were asked to report the therapy goal, length of session, mode of delivery of session, Agenda for Change band of staff delivering the session and distance travelled. This information was sought every 3 months via telephone calls with the participants and carers and from inspection of participant notes if they were made aware that therapy had been received.

Numbers recruited

The overall target for the six sites was 36 participants. The overall progression target for numbers recruited from the six sites was 30. This was equivalent to the number recruited in total in our previous pilot study and enabled comparison with previous recruitment rates to confirm whether or not our projections for the substantive study were accurate. There was also information available from other sites that had completed set-up and started to recruit; therefore, we expected at least 40 participants to have been recruited by the end of the internal pilot phase in total.

Recruitment as a percentage of the full-study recruitment targets

At the end of the internal pilot trial, progression depended on having recruited 30 participants (i.e. 10% of the total population recruited from 25% of the sites; this was midway through the recruitment phase for these sites). If we achieved this number, we would be on track to recruit only 80% of the sample size within the study period. We would then have to bring in the additional four contingency sites included in the costs to raise the recruitment to the sample size. If we did not meet this number, it would indicate that the larger study was unlikely to be feasible.

Retention to first outcome measure time point at 6 months (primary outcome)

The sample size calculation was based on an attrition rate of 15% at 90% power for the co-primary outcomes. The progression criterion for retention was set to ensure a minimum power of 80%. This would be achievable with a retention rate of 65%, which would still ensure that the results were generalisable.

Identification and retention of volunteers

Sites could provide support to patients in the intervention group of the trial from paid SLTAs or volunteers. Use of volunteers was reviewed at the end of the internal pilot phase. Progression criteria for continued use of volunteer support were set at 80% of participants having been offered a volunteer and 70% of participants continuing to be supported by the same volunteer for their 6-month treatment period. If these progression criteria were not achieved, continuation of the study would be with paid assistant support only.

Summary

In summary, 8 months after set-up of the sixth site, our progression criteria indicating feasibility of the full trial were:

• recruitment of no fewer than 30 participants (10% of the target for the full trial)

• a minimum retention rate of 65%.

Modified intention-to-treat set

The modified intention-to-treat (mITT) population includes all participants for whom consent was obtained, for whom treatment was allocated as per the randomised list regardless of circumstances after randomisation and who had primary outcome data at the 6-month assessment. This was the primary analysis set; other sets described below are for sensitivity analysis.

Complete-case set

This includes participants with outcome data at a particular assessment. The set was used for subsidiary analysis of co-primary and key secondary outcomes at 9 and 12 months, and other secondary outcomes at different assessments, and for plotting mean profile response of participants over time across interventions.

Per-protocol sets

The goal here was to explore the effectiveness of the intervention among participants who adhered to key components of the intervention. Therefore, the per-protocol sets include participants for whom key components of the intervention were adhered to, including achieving the minimum amount of practice recommended (20–30 minutes per day) and having access to support up to and including at their 6-month assessment. Per-protocol classification relating to adherence to the intervention was undertaken for only the CSLT and AC groups.

Per-protocol definitions were developed using the following rationale.

The recommended practice time was 20–30 minutes per day over 6 months. It was calculated that 26 hours provided an indication of high levels of adherence to this recommendation for the CSLT group based on a minimum of 20 minutes three times per week for at least 4 months, accounting for periods of holiday, illness and other commitments. A lower total practice time of 10 hours was also considered to identify engagement with the computer intervention even if at a lower practice amount than recommended. Participants in the AC group were recommended to complete one puzzle each day. We estimated that, on average, there were enough puzzles in a book to last for 1 month so we used six puzzle books requested/sent as a proxy for adherence to carrying out regular puzzle book activities. The intervention descriptions recommend that support should be provided once per month in both the CSLT group and the AC group. Accepting that this is not always possible, four contacts was used as an indicator that support was being provided.

For all intervention groups, participants were excluded from the per-protocol analysis if outcome measures were assessed 14 days before or 31 days after the expected 6-month assessment or if they were randomised but failed to meet at least one inclusion criterion.

The four per-protocol sets in the CSLT and AC groups were matched as follows:

1. high levels of adherence to recommended practice – practised computer therapy for a minimum total of 26 hours (CSLT) or were sent at least six puzzle books (AC) within 6 months of randomisation (referred to as PP1 CSLT26 AC6)

2. adherence to practice – practised computer therapy for a minimum total of 10 hours (CSLT) or were sent at least six puzzle books (AC) within 6 months of randomisation (PP2 CSLT10 AC6)

3. high levels of adherence to recommended practice and provided with recommended support – practised computer therapy for a minimum total of 26 hours (CSLT) or were sent at least six puzzle books (AC) and contacted at least four times (if they wished) (in both the AC group and the CSLT group) within 6 months of randomisation (PP3 CSLT26 AC6_4)

4. adherence to practice and provided with recommended support – practised computer therapy for a minimum total of 10 hours (CSLT) or were sent at least six puzzle books (AC) and contacted at least four times (if they wished) (in both the AC group and the CSLT group) within 6 months of randomisation (PP4 CSLT10 AC6_4).

On 7 March 2018, the Trial Management Group discussed the final trial results in detail. It was noted that only a small proportion of participants in the AC group met the predefined per-protocol inclusion criteria. It was agreed that the predefined per-protocol proxy of being sent six puzzle books in 6 months was overambitious and inconsistent with expectations of per protocol for CSLT. To align the puzzle book per-protocol classification in the AC group with the CSLT group, the Trial Management Group requested post hoc per-protocol classification based on being sent at least four puzzle books. This will only change the per-protocol results for comparisons involving the AC group. We therefore modified the per-protocol classifications presented above for sensitivity analysis as follows:

1. high levels of adherence to recommended practice – practised computer therapy for a minimum total of 26 hours (CSLT) or were sent at least four puzzle books (AC) within 6 months of randomisation (PP1 CSLT26 AC)

2. adherence to practice – practised computer therapy for a minimum total of 10 hours (CSLT) and were sent at least four puzzle books (AC) within 6 months of randomisation (PP2 CSLT10 AC 4)

3. high levels of adherence to recommended practice and provided with recommended support – practised computer therapy for a minimum total of 26 hours (CSLT) or were sent at least four puzzle books (AC) and contacted at least four times (if they wished) (in both the AC group and the CSLT group) within 6 months of randomisation (PP3 CSLT26 AC4_4)

4. adherence to practice and provided with recommended support – practised computer therapy for a minimum total of 10 hours (CSLT) or were sent at least four puzzle books (AC) and contacted at least four times (if they wished) (in both the AC group and the CSLT group) within 6 months of randomisation (PP4 CSLT10 AC4_4).

Multiple imputation set

This includes all randomised participants excluding those who died prior to 6 months; it is for sensitivity analysis. Multiple imputation (MI) was conducted for only the co-primary outcomes (word-finding and functional communication) and key secondary outcome (COAST). The mean value of participants with available baseline data was used to impute missing baseline data during analysis for a few participants with follow-up outcome data but who had missing related baseline data for some reason. We adopted a strategy to inform the MI statistical model for imputing missing data:

1. Potential predictors of outcomes independent of the intervention were clinically prespecified in the SAP (see www.journalslibrary.nihr.ac.uk/programmes/hta/122101/#).

2. The characteristics of completers (those meeting the mITT inclusion criteria) and all randomised participants (excluding those who died) were descriptively compared to explore predictors of missing data.

3. We graphically explored the association between baseline characteristics and outcomes of interest.

Based on these exploratory results, the intervention group appeared to be a mild predictor of missing data. Therefore, it was implausible to assume that the data were missing completely at random. Other measured predictors of missing data were unclear. As a result, we adopted the following strategy to impute missing data:

• Intervention group (UC, AC or CSLT), age, gender (male or female), the presence of a carer (yes or no), severity of word-finding (total score), severity of comprehension ability (total score) and baseline outcome measure under consideration were mandatory covariates in all MI statistical models.

• Co-primary and key secondary outcomes at baseline (covariates) and 6, 9 and 12 months were included in all MI statistical models.

• The longitudinal nature of the outcome data under consideration was accounted for in the MI statistical models using chained equations39,40 implemented via the Stata 15.1 mi command.

The multiple imputation using chained equations analysis was conducted and reported in accordance with the guidance provided by White et al. 41 We chose the number of imputations (n = 20) based on the observed proportion of missing data. 41

Linear interpolation set

For additional sensitivity analysis on the impact of missing data on the trial results, we used a linear interpolation model as a deterministic imputation approach. Where data were missing at an assessment (t_i) but valid data are available at previous (t_i – 1) and future (t_i + 1) assessments, the missing value was linearly interpolated by the formula:

Safety set

Safety analysis relates to the evaluation of the intervention effect on AEs and SAEs, and negative effects of the computer therapy. This includes all randomised participants with informed consent and treatment allocation for analysis used the actual intervention received (treatment as received principle) based on available evidence, such as the number of books sent and computer therapy practice time. We also used treatment allocation as randomised for sensitivity analysis.

Approach to dealing with deaths after randomisation

We expected some deaths in this trial population during the trial. The research team discussed implications of deaths and approaches to handle them during analysis. The influence of the trial interventions on increasing risk of mortality was unanimously viewed to be very unlikely. In addition, the interpretation of imputed missing data, such as word-finding and functional communication for participants who died, was clinically challenging. As a result, the research team agreed not to impute data that were missing because of participant deaths. Therefore, deaths that happened prior to the 6-month assessment were excluded in any clinical effectiveness analysis but included in the safety analysis. This approach is consistent with related recommendations. 42 We performed sensitivity analysis to explore the impact of data that were missing for other reasons, unrelated to death, on the bias of the results using the MI, linear interpolation and complete-case sets.

Hochberg multiple testing procedure

We used the Hochberg procedure to interpret the co-primary (word-finding and functional communication) and key secondary (COAST) outcome results at 6 months in order to control the chances of falsely declaring statistically significant results (at 5% nominal level). 43 There are two sources of multiple hypothesis testing at 6 months: multiple outcome measures (co-primary measures and a key secondary measure) and key multiple treatment comparisons (CSLT vs. UC and CSLT vs. AC). Figure 1 displays the interpretation strategy of the results in order to claim statistical significance and superiority of the intervention.

FIGURE 1.

Interpretation of the Hochberg hierarchical sequential hypotheses testing strategy. Superscript letters correspond to footnotes in Table 8.

Computation of summary outcome scores for analysis

This section details the generation of outcome scores from measurement instruments and how related missing data were handled.

Clinical effectiveness

For the co-primary and key secondary outcomes at 6 months, we used a multiple linear regression model adjusted for associated baseline outcome measures and fixed stratification factors: centre and the severity of word-finding (mild, moderate or severe). We expressed the maximum likelihood estimate of the intervention effect as the adjusted mean difference in change (MDC) between the CSLT and UC groups, and the CSLT and AC groups, with associated 95% CIs and p-values. The intervention effect between the AC and UC groups obtained via contrasts was for exploratory purposes only. We used a Hochberg procedure, as described in Hochberg multiple testing procedure, to control for the false-positive error rate for claiming the evidence of clinical effectiveness.

For sensitivity analysis, we used a multiple linear regression model adjusted for associated baseline outcome measures, fixed stratification factors (centre and the severity of word-finding), the length of time post stroke (continuous) and the location of stroke (yes or no) (middle cerebral artery, frontal lobe, parietal lobe and temporal lobe).

The long-term intervention effect at 9 and 12 months on the co-primary and key secondary outcomes, and other secondary continuous outcomes (at 6, 9 and 12 months), such as CaCOAST domains, word-finding of untreated words and word-finding of treated words used in conversation, was evaluated using a multiple linear regression model adjusted for associated baseline outcome measures and fixed stratification factors: centre and the severity of word-finding (mild, moderate and severe).

We used a multiple logistic regression model adjusted for fixed stratification factors (centre and the severity of word-finding) to explore the intervention effect on the proportion of participants achieving predefined clinical improvements of 5% and 10% in word-finding of both treated words (from personal vocabulary naming test) and untreated words (from CAT Naming Objects). The numbers and proportion of participants meeting each clinical improvement criterion are reported by intervention together with the odds ratio (OR) and associated 95% CIs and p-values. This was performed under two scenarios by considering (1) only participants with complete data and (2) all randomised participants, but assuming that those with missing data failed to achieve clinical improvement (worst-case scenario). Related results are presented in the statistical report version 1.0, which is available online as a supplementary appendix to Palmer et al. 50

Functional impact

For post hoc analysis at the request of the chief investigator following the disclosure of the results (as per predefined SAP), we explored the intervention effect on the proportion of participants who did and did not use treated words in conversation based on a 5% or 10% improvement in naming treated words (from personal vocabulary naming test) and functionally used at least 5 or 10 treated words (retrieved during videoed conversation). This analysis was requested to identify if any patients were able to retrieve newly learned words in functional contexts so that further investigation of characteristics of those who do and do not automatically use new words in context can be explored in further research to inform how others might be assisted to use new words functionally. We calculated the proportion of participants meeting each clinical improvement criterion by intervention group. The difference in proportions of participants achieving a ‘clinical improvement’ criterion between interventions was calculated, with associated 95% CIs estimated using the normal approximation to the binomial distribution without significance testing.

An additional post hoc analysis explored the proportion of participants who do or do not generalise word-finding to untreated words. A multiple logistic regression model adjusted for fixed stratification factors (centre and the severity of word-finding) and baseline measures were used to explore the intervention effect on the proportion of participants achieving clinical improvement of 5% and 10% in the generalisation of word-finding to untreated words (from CAT Naming Objects) without significance testing.

Subgroup evaluation

We prespecified the following subgroups to explore potential heterogeneity in the intervention effect on the co-primary and key secondary outcomes at 6 months (SAP version 1.2; see www.journalslibrary.nihr.ac.uk/programmes/hta/122101/#):

• the severity of word-finding difficulty – mild (31–43), moderate (18–30) and severe (5–17)

• comprehension ability – within normal limits (27–32), mild (18–26), moderate (9–17) and severe (0–8)

• the length of time post stroke – categorisation was quantile based because there was no existing literature to guide the clinical classification.

We performed subgroup analysis based on the mITT set. The number of participants and mean change in outcomes are reported stratified by the intervention received and subgroup category. We assessed effect modification between the intervention and subgroup using a multiple linear regression model that included an interaction term between the intervention and the subgroup of interest adjusted for baseline outcome measures and fixed stratification factors: centre and the severity of word-finding difficulty (mild, moderate and severe). We report the overall p-values from the interaction tests to explore the strength of evidence for heterogeneity of the intervention effect across subgroups. We use forest plots to present results and aid visual interpretation, showing the maximum likelihood estimate of the intervention effect (CSLT vs. UC and CSLT vs. AC), with associated 95% CIs stratified by subgroup category.

Safety evaluation

The primary analysis of safety outcomes was based on the safety set. We also performed sensitivity analysis on the safety outcomes using the treatment as randomised, as described in Safety set.

Screening and flow of trial participants

Between September 2014 and August 2016, 995 participants were screened and 818 were found to be appropriate for further assessment of eligibility across 20 NHS trusts in the UK (21 randomisation centres/speech and language therapy departments), predominantly via patient records (including word of mouth from speech and language therapy colleagues) and voluntary support groups. Figure 2 shows the flow of participants from screening to randomisation including trial follow-up at 6, 9 and 12 months from randomisation. Of the 995 patients screened, 288 (28.9%) eligible participants were consented and 278 (27.9%) were randomised: UC, n = 101; AC, n = 80; and CSLT, n = 97.

FIGURE 2.

Trial participant flow chart. Adapted from Palmer et al. 50 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/.

Adapted from Palmer et al. 50 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/.

Of the 278 patients randomised, eight (2.9%) died before their 6-month assessments: UC, n = 4; AC, n = 1; and CSLT, n = 3. For the remaining 270, 240 (88.9%) completed their 6-month assessments: 86, 71 and 83 in the UC, AC and CSLT groups, respectively. The proportions of participants who completed 6- and 9-month outcome assessments were very similar across interventions. However, the discontinuation rate at 12 months was slightly higher in the AC (23.8%) and the CSLT (23.7%) groups than in the UC (16.8%) group. Attrition reasons are stated in Figure 2 and detailed in Table 2. The most common reasons were personal or family issues and being unhappy with the allocated trial intervention.

In total, 240 randomised participants were eligible for inclusion in the primary mITT analysis: 86 in the UC group, 71 in the AC group and 83 in the CSLT group. As for the secondary MI analysis, 270 randomised participants were eligible for inclusion: 97 in the UC group, 79 in the AC group and 94 in the CSLT group.

Characteristics of participating speech and language therapy departments in NHS trusts

The 21 speech and language therapy departments participating in the trial served a mixture of urban and rural populations. Ten sites were described by local principal investigators as predominantly rural, four as predominantly urban and seven as mixed rural and urban. Twenty-six eligible SLTs were trained to deliver the CSLT intervention in total. Sixteen sites chose to use only SLTAs to provide the support component of the intervention, three sites used only volunteers and two sites used both SLTAs and volunteers.

Baseline demographics and characteristics of randomised participants

Tables 3 and 4 show the demographics and characteristics of randomised participants stratified by intervention; 270 met the MI inclusion criteria and 240 met the mITT inclusion criteria (‘completers’). Baseline characteristics of the eight participants who died before the 6-month outcome assessment are not reported as they were not included in the analysis because the association between the interventions and increased risk of mortality was viewed as extremely unlikely.

For the 270 randomised participants meeting the MI inclusion criteria, the recruitment across 21 speech and language therapy centres/departments (20 NHS trusts) ranged from 7 to 22 participants, with a median of 12 [interquartile range (IQR) 10–15] participants. The majority of participants (n = 164; 60.7%) were male. The overall mean age at consent was 65.4 (SD 12.9) years, ranging from 23.1 to 91.8 years. Most participants (n = 119; 44.1%) had mild word-finding difficulty; 80 participants (29.6%) had moderate and 71 (26.3%) had severe difficulty. The overall median time post stroke was approximately 2 years (IQR 1–4 years). Most participants showed no evidence of apraxia of speech (n = 174; 64.4%) and 212 (78.5%) had suffered an infarction stroke.

In summary, randomised participants and ‘completers’ were very similar on average with respect to measured characteristics and demographics (see Table 3). Furthermore, on average, the participants appeared broadly similar across interventions. However, by chance, there were a few exceptions indicating relatively small differences between interventions, such as with respect to the location of the stroke, time post stroke, the type of stroke and lateralisation, if not brain stem. There was also a chance imbalance in the number of participants randomised to AC compared with UC or CSLT.

Table 4 shows the characteristics of the randomised participants and ‘completers’ (with their available carers) with respect to continuous covariates (outcomes assessed at baseline), which are broadly similar between the two MI and mITT populations and across interventions. However, the available carers of the UC participants who agreed to take part had slightly higher CaCOAST domain scores on average than their counterparts did. It should also be noted that the participant, not the supporting carer, was the unit of randomisation.

Reported cases of unblinding of outcome assessments

Table 5 summarises cases of suspected unblinding and when the allocated intervention was guessed correctly. At the 6-month assessment, 29 (10.7%) cases of suspected unblinding were reported. Of these, the allocated intervention was incorrectly guessed in only one case. The majority of cases happened before or during the 6-month assessment, and the proportions of participants were higher in the CSLT and AC groups than in the UC group. Nevertheless, the reported cases of unblinding are negligible relative to the number of outcome assessments made during the trial.

Usual-care speech and language therapy offered (fidelity/adherence to provision of usual care)

The objective here was to assess whether or not the overall amounts of UC speech and language therapy were similar across the intervention groups throughout the trial as it was an adjunct trial aiming to compare the interventions in addition to UC with UC alone. The descriptive details of what UC speech and language therapy consisted of are reported in Chapter 2, Usual-care control group. Table 6 shows the proportion of participants who received UC speech and language therapy 3 months prior to each assessment time point. The distribution of UC speech and language therapy seems comparable across interventions; however, slightly fewer participants in the AC group than in the UC and CLST groups received UC speech and language therapy especially 3 months prior to baseline. In addition, slightly more participants in the UC group received UC speech and language therapy than their counterparts. Across interventions, the proportions of participants receiving UC speech and language therapy decreased as the trial progressed. This was to be expected as participants were further in time post stroke as the trial progressed and the pre-baseline UC descriptions (see Chapter 2, Usual-care control group) identified the fact that amounts of speech and language therapy received decreased with length of time post stroke.

The distributions of the average amounts of overall UC speech and language therapy received are displayed in Figure 3; the averages are calculated based on all participants in the denominator regardless of whether or not they received UC speech and language therapy. On average, during the 6-month intervention period (data collection points 3 and 6 months), the UC group received a mean of 3.8 hours of usual speech and language therapy, the AC group received a mean of 3.2 hours of usual speech and language therapy and the CSLT group received a mean of 3.2 hours of usual speech and language therapy. Those in receipt of therapy had a mean of 9.7 hours in UC, 11 hours in AC and 7.8 hours in CSLT. There is more variation in the average number of hours in receipt of therapy in those who received therapy than for all participants in each group as the denominator of those in receipt was small, as shown in Table 6.

FIGURE 3.

Mean amounts of overall UC speech and language therapy (all participants).

The mean amounts of overall UC speech and language therapy for only those who received it can be seen in figure 6 of the full statistical report, which is available online as a supplementary appendix to Palmer et al. 50

Fidelity to computerised speech and language therapy: adherence to practice and quality of intervention delivery

Per-protocol adherence to the attention control intervention

Only 1 of the 80 participants randomised to the AC intervention died before the 6-month assessment. Of the remaining 79, only 14 (17.7%) were sent at least six puzzle books and were contacted at least four times within 6 months. These were deemed to have adhered to the predefined key components of the AC group as detailed in Chapter 3, Per-protocol sets.

Using four puzzle books and four contacts as per protocol in the post hoc analysis to be consistent with the 4 months of computer therapy as per protocol in the CSLT group, 48 out of 79 (60.8%) participants were deemed to have adhered to the AC intervention. This post hoc approach was used only for sensitivity analysis (see Chapter 3, Per-protocol sets).

Response profiles of participants

Figure 4 shows the changes in the response of participants with respect to word-finding ability over time, stratified by intervention. There appear to be marked improvements in the CSLT group compared with the UC or AC groups. Only 10 CSLT participants failed to improve their word-finding ability at 6 months; three had no computer use recorded, four used computer therapy for < 5 hours in the first 6 months and the remaining three used computer therapy for > 20 hours in the first 6 months.

FIGURE 4.

Changes in word-finding over time stratified by the intervention. (a) UC; (b) AC; and (c) CSLT. Each line, regardless of colour, indicates a participant’s response profile.

Figure 5 shows the pattern in functional communication responses, which is unclear but appears to be similar across interventions. Only five (1.9%) participants had a TOMs rating on the ceiling (score of 5): 3 out of 94 (3.2%) in the CSLT group, 2 out of 97 (2.1%) in the UC group and none in the AC group. These participants cannot show any further improvements in functional communication during the trial because they were deemed to communicate effectively in all situations at baseline. It should be noted that the ability to communicate effectively in all situation does not preclude having word-finding difficulties as communication can be achieved by other means, such as gesture. No participants were unable to communicate in any way (with a TOMs score of 0) at baseline.

FIGURE 5.

Changes in functional communication in conversation rated by TOMs over time, stratified by intervention. (a) UC; (b) AC; and (c) CSLT. Each line, regardless of colour, indicates participant’s response profile.

The COAST response patterns over time, indicating participant perception of social participation and quality of life, stratified by intervention, are displayed in Figure 6. Participants in the AC group appear to have deteriorated at the 6-month assessment relative to their baseline. These response profile plots complement the interpretation of the main and subgroup results presented in Chapter 5, Results.

FIGURE 6.

Changes in perception of social participation and quality of life rated by COAST over time, stratified by intervention. (a) UC; (b) AC; and (c) CSLT. Each line, regardless of colour, indicates a participant’s response profile.

Effect of the intervention on the co-primary and key secondary outcome measures

The first three primary clinical effectiveness objectives were to establish whether or not self-managed CSLT intervention increases the ability of people with aphasia to use the vocabulary of personal importance, improves functional communication ability in conversation and results in perceived greater changes in social participation in daily activities and quality of life. This section presents results at the 6-month assessment to address these objectives.

Word-finding and functional communication: co-primary outcome measures

Figures 7 and 8 display the unadjusted mean responses to word-finding and functional communication over time stratified by the intervention.

FIGURE 7.

Mean word-finding ability over time stratified by the intervention. Adapted from Palmer et al. 50 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/.

Adapted from Palmer et al. 50 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/.

FIGURE 8.

Mean functional communication in conversation over time stratified by the intervention. Adapted from Palmer et al. 50 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/.

Adapted from Palmer et al. 50 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/.

For word-finding, 86, 71 and 83 participants in the UC, AC and CSLT groups, respectively, were included in the mITT analysis. The mean improvement in word-finding of personally selected words was 1.1% (SD 11.2%) in the UC group, 2.4% (SD 8.8%) in the AC group and 16.4% (SD 15.3%) in the CSLT group. This indicates an adjusted mean difference in word-finding improvement of 16.2% (95% CI 12.7% to 19.6%; p < 0.0001) in favour of CSLT compared with UC. Detailed results are presented in Table 8.

TABLE 8 - Co-primary and key secondary outcomes at 6 months (mITT)

Superscripts a, b, c, d, e and f are referenced in Figure 1 to aid interpretation of Hochberg sequential and hierarchical hypotheses testing procedure for claiming evidence.

UC as the reference group.

AC as the reference group.

Higher scores indicate improved vocabulary of personal importance.

Higher scores indicate improved functional communication ability in conversation. Seven participants had missing TOMs data (video not recorded in error, poor video sound quality, camera technical issues, participant declined to be videoed, recording failed owing to a technical issue, participant was unwell and did not want to complete the assessment and participant did not want to complete assessment).

Higher percentage score indicates improved patient perception of communication effectiveness and its impact on their quality of life. Seven participants had invalid COAST records, with > 10% of applicable items that were unclear or had no response.

Notes

Results are based on a multiple linear regression model adjusted for baseline measures and fixed stratification factors (centre and severity of word-finding difficulty).

The comparisons of interest are CSLT vs. UC and CSLT vs. AC. AC vs. UC is presented for completeness but shaded to indicate that it is not of clinical importance.

Adapted from Palmer et al. 50 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/.

Co-primary and key secondary outcomes at 6 months	UC		AC		CSLT		CSLT vs. UCg		CSLT vs. ACh		AC vs. UCg
	n	Mean (SD)	n	Mean (SD)	n	Mean (SD)	Adjusted MDC (95% CI)	p-value	Adjusted MDC (95% CI)	p-value	Adjusted MDC (95% CI)	p-value
Co-primary outcomes
Change in word-finding (%)i	86	1.1 (11.2)	71	2.4 (8.8)	83	16.4 (15.3)	16.2 (12.7 to 19.6)a	< 0.0001	14.4 (10.8 to 18.1)c	< 0.0001	1.8 (–1.9 to 5.4)	0.338
Change in functional communicationj	84	0.05 (0.59)	68	0.10 (0.61)	81	0.04 (0.58)	–0.03 (–0.21 to 0.14)b	0.709	–0.01 (–0.20 to 0.18)d	0.915	–0.02 (–0.21 to 0.17)	0.812
Key secondary outcome
Change in COAST (%)k	83	2.7 (12.6)	68	–0.3 (12.7)	82	3.3 (11.3)	0.5 (–3.1 to 4.1)e	0.772	3.8 (–0.0 to 7.5)f	0.051	–3.2 (–7.0 to 0.5)	0.089

In Figure 8, the mean functional communication barely changed at 6 months. As shown in Table 8, the mean change in functional communication in conversation on the TOMs was very similar between the CSLT and UC groups, translating to an adjusted MDC of –0.03 (95% CI –0.21 to 0.14; p = 0.709) compared with UC.

In line with the prespecified Hochberg multiple testing strategy, shown in Figure 1, we can only claim the clinical effectiveness of CSLT compared with UC in improving word-finding of personal choice at a 2.5% significance level. Because both comparisons between CSLT and UC with respect to word-finding and functional communication in conversation were not statistically significant at a 5% significance level, further statistical significance testing is prohibited. The mean improvement in word-finding of personally selected words of 14.4% (95% CI 10.8% to 18.1%) in favour of CSLT compared with AC supports that the clinical effectiveness in improving word-finding is attributed to CSLT rather than the attention provided. The mean improvement in word-finding between the AC and UC groups was similar: 1.8% (95% CI –1.9% to 5.4%). The mean changes in functional communication in conversation were very similar across interventions. Thus, CSLT did not result in improvement in functional communication ability in conversation compared with UC or AC.

The effects of CSLT on word-finding and functional communication were very similar after adjusting for additional prespecified covariates (time post stroke and location of stroke) (Tables 8 and 9).

TABLE 9 - Sensitivity analysis of the co-primary and key secondary outcomes at 6 months (mITT)

UC as the reference group.

AC as the reference group.

Higher scores indicate improved vocabulary of personal importance.

Higher scores indicate improved functional communication ability in conversation. Seven participants had missing TOMs data (video not recorded in error, poor video sound quality, camera technical issues, participant declined to be videoed, recording failed owing to a technical issue, participant was unwell and did not want to complete the assessment and participant did not want to complete assessment).

Higher percentage score indicates improved patient perception of communication effectiveness and its impact on their quality of life. Seven participants had invalid COAST records, with > 10% of applicable items that were unclear or had no response.

Notes

Sensitivity analysis results from multiple linear regression models adjusted for baseline measures, fixed stratification factors (centre and severity of word-finding) and potential confounders (length of time post stroke and location of stroke).

The comparisons of interest are CSLT vs. UC and CSLT vs. AC. AC vs. UC is presented for completeness but shaded to indicate that it is not of clinical importance.

Sensitivity analysis: co-primary and key secondary outcomes at 6 months	UC		AC		CSLT		CSLT vs. UCa		CSLT vs. ACb		AC vs. UCa
	n	Mean (SD)	n	Mean (SD)	n	Mean (SD)	Adjusted MDC (95% CI)	p-value	Adjusted MDC (95% CI)	p-value	Adjusted MDC (95% CI)	p-value
Co-primary outcomes
Change in word-finding (%)c	86	1.1 (11.2)	71	2.4 (8.8)	83	16.4 (15.3)	16.3 (12.8 to 19.8)	< 0.0001	14.7 (11.0 to 18.4)	< 0.0001	1.6 (–2.1 to 5.4)	0.385
Change in functional communicationd	84	0.05 (0.59)	68	0.10 (0.61)	81	0.04 (0.58)	–0.05 (–0.23 to 0.13)	0.596	–0.03 (–0.22 to 0.16)	0.781	–0.02 (–0.22 to 0.17)	0.830
Key secondary outcome
Change in COAST (%)e	83	2.7 (12.6)	68	–0.3 (12.7)	82	3.3 (11.3)	0.9 (–2.8 to 4.5)	0.644	3.6 (–0.2 to 7.5)	0.064	–2.8 (–6.6 to 1.1)	0.156

Patient perception of communication and its impact on their life: key secondary outcome measure

Figure 9 shows the unadjusted average profile in COAST over time. The effect of the intervention on the COAST at 6 months is presented in Table 8.

FIGURE 9.

Mean COAST score over time, stratified by the intervention. Adapted from Palmer et al. 50 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/.

Adapted from Palmer et al. 50 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/.

The average improvement in COAST was only 3.3% and 2.7% in the CSLT and UC groups, respectively, and the AC group COAST decreased slightly, by 0.3%. The adjusted MDC in COAST was only 0.5% (95% CI –3.1% to 4.1%) in the CSLT group compared with the UC group. The adjusted MDC of 3.8% (95% CI –0.0% to 7.5%) observed in the CSLT group compared with the AC group was attributable to the AC group barely changing on average at 6 months, whereas the UC group experienced a small average improvement that was comparable to that seen in the CSLT group.

In summary, there is insufficient evidence to show that CSLT improves patients’ perceptions of communication and its impact on their life.

Impact of attrition and adherence on the co-primary and key secondary outcome measures

As part of the sensitivity analysis, we present the results exploring the influence of attrition and intervention adherence on the effect of the intervention on word-finding, functional communication in conversation (TOMs) and participant perception of social participation and quality of life (COAST) at 6 months. The corresponding results for the MI set, four per-protocol sets and linear interpolation set described in Chapter 3, Analysis populations, are presented in forest plots (Figures 10–12), including the primary mITT results presented in Table 8 for comparability.

FIGURE 10.

Impact of attrition and intervention adherence on word-finding at 6 months. LI, linear interpolation; MCID, minimum clinically important difference; PP, per protocol. Per-protocol sets are defined in Chapter 3, Analysis populations.

FIGURE 11.

Impact of attrition and intervention adherence on functional communication at 6 months. LI, linear interpolation; MCID, minimum clinically important difference; PP, per protocol. Per-protocol sets are defined in Chapter 3, Analysis populations.

FIGURE 12.

Impact of attrition and intervention adherence on COAST at 6 months. LI, linear interpolation; MCID, minimum clinically important difference; PP, per protocol. Per-protocol sets are defined in Chapter 3, Analysis populations.

In general, the results are consistent across all analysis sets considered and similar to the primary analysis set (mITT). The results of per-protocol comparisons that involve the AC group should be interpreted with caution because of small sample size due to poor adherence to AC of 17.7%, based on the large number of puzzle books prespecified as per protocol (six books), as described in Chapter 3, Per-protocol sets. We further undertook sensitivity analysis based on post hoc classification of per-protocol adherence in the AC group (sent at least four puzzle books rather than six), as described in Chapter 3, Per-protocol sets. These results did not differ from the prespecified per-protocol classification to change the interpretation of findings. For additional information, these post hoc results are accessible online (see section 9.19 of the statistical report, which is available online as a supplementary appendix to Palmer et al. 50).

Subgroup influence on the effectiveness on the co-primary and key secondary outcome measures

This section presents the results exploring potential heterogeneity in the intervention effect across prespecified subgroups on word-finding, functional communication in conversation (TOMs) and participant perception of social participation/quality of life (COAST) at 6 months. The results for CSLT versus UC and CSLT versus AC comparisons are graphically displayed in forests plots together with the mITT results (Figures 13–15). In addition, the interaction tests between the intervention and subgroups are presented in Table 10.

FIGURE 13.

Subgroup influence on word-finding results at 6 months. MCID, minimum clinically important difference; Q1, 25th percentile, ≈1 year; Q2, 50th percentile, ≈2 years; Q3, 75th percentile, ≈4 years; WNL, within normal limits.

FIGURE 14.

Subgroup influence on functional communication results at 6 months. MCID, minimum clinically important difference; Q1, 25th percentile, ≈1 year; Q2, 50th percentile, ≈2 years; Q3, 75th percentile, ≈4 years; WNL, within normal limits.

FIGURE 15.

Subgroup influence on COAST results at 6 months. MCID, minimum clinically important difference; Q1, 25th percentile, ≈1 year; Q2, 50th percentile, ≈2 years; Q3, 75th percentile, ≈4 years; WNL, within normal limits.

Due consideration should be given to the clinical or biological plausibility of subgroup results when interpreting these findings. In addition, extreme caution should be taken when interpreting the intervention effect in the severe category of the CAT Comprehension of Spoken Sentences owing to very small sample sizes.

For word-finding, the results appear to be consistent and similar to the mITT results. However, the intervention effect seems to be more pronounced in patients who had mild word-finding difficulties and for those whose comprehension was within normal limits on the CAT Comprehension of Spoken Sentences. The effect of the intervention on TOMs and COAST appears to be broadly consistent across subgroups. However, for unclear clinical reasons, probably due to change, TOMs seem to have improved among the 25% of participants whose time post stroke was between the first and second quartiles of the distribution.

Long-term effects of the intervention on the co-primary and key secondary outcomes

The fourth research objective was to identify whether or not any effects of the interventions were evident 12 months after therapy had begun.

Figures 7–9 show the unadjusted average profile response of participants over time for word-finding, conversation (TOMs) and participant perception of social participation/quality of life (COAST) stratified by the intervention. The effects of the intervention (CSLT vs. UC and CSLT vs. AC) for word-finding, TOMs and COAST at 9 and 12 months are displayed in Figures 16–18, together with the primary results at 6 months (mITT) for comparability.

FIGURE 16.

Long-term intervention effect on word-finding of personal importance. CC, complete case; LI, linear interpolation; MCID, minimum clinically important difference.

FIGURE 17.

Long-term intervention effect on functional communication. CC, complete case; LI, linear interpolation; MCID, minimum clinically important difference.

FIGURE 18.

Long-term intervention effect on the COAST. CC, complete case; LI, linear interpolation; MCID, minimum clinically important difference.

In summary, the long-term effects of CSLT on word-finding of personal importance at 9 and 12 months were consistent across analysis sets but marginally lower than those observed at 6 months. However, all of the 95% CIs include the 10% improvement in word-finding of clinical relevance and excludes the zero effect of no difference in the intervention effect. Therefore, the sustained long-term intervention effect on word-finding is potentially of clinical importance. For TOMs and COAST, no changes of clinical relevance were observed and the effects are consistent with the primary results at 6 months and across the considered analysis sets.

Effect of the intervention on generalisation to untreated words

In this section, we present the results exploring whether or not there is a generalisation of word-finding from words of personal importance (treated words) to untreated words. This was assessed using the CAT Naming Objects with a possible total score from the picture-naming tasks of 24 untreated words ranging from 0 to 48.

Table 11 summarises the effect of the intervention on word-finding of untreated words at 6, 9 and 12 months from randomisation. At 6 months, the mean change in word-finding of untreated words (scores) was 3.9 (SD 7.9), 0.7 (SD 8.5) and 3.3 (SD 7.0) in the UC, AC and CSLT groups, respectively. This indicates an adjusted MDC of –0.3 (95% CI –2.7 to 2.1) in favour of UC compared with CSLT. On average, the adjusted word-finding of untreated words in the AC group was lower than those of the UC and CSLT groups across assessments.

In summary, there is insufficient evidence to support the positive effect of the intervention on improving the generalisation of word-finding to untreated words in either the short term or long term.

The literature often reports the proportion of participants who do or do not generalise word-finding to untreated words. 51 Therefore, we undertook a post hoc analysis to explore the intervention effect on the proportion of participants who generalised the word-finding of untreated words as defined by a clinical improvement of at least 5% and 10% from baseline. In summary, these results do not support that CSLT increases the proportion of participants achieving a clinical improvement of at least 5% and 10% in the generalisation of word-finding to untreated words compared with UC or AC. Detailed results are accessible online (see section 9.14.1 of the statistical report, which is available online as a supplementary appendix to Palmer et al. 50).

Effect of the intervention on generalisation of treated words used in conversation

Here, we present the results exploring the intervention effect on the use of learned vocabulary (from the word-finding treatment of personally chosen words in conversation). This is based on a possible total number of unique words retrieved in video conversations ranging from 0 to 100.

The average response profiles in the numbers of treated words used in conversation over time are shown in Figure 19. The UC and AC groups experienced an average decrease of about two personally chosen words at 6 months, whereas the CSLT group remained almost the same throughout the trial. This translated to an adjusted MDC in treated words used in conversation at 6 months of only 2.0 (95% CI 0.6 to 3.4) in favour of CSLT compared with UC and 2.9 (95% CI 1.4 to 4.4) in favour of CLST compared with AC. The effect diminished at 9 and 12 months as UC and AC participants improved slightly on average. Table 12 details results that are very consistent across analysis sets considered at 6, 9 and 12 months.

FIGURE 19.

Mean response profile in word-finding of treated words over time.

In line with the analysis of generalisation of treated words to untreated words, we performed post hoc analysis exploring the proportion of participants meeting clinical improvement in the generalisation of treated words used in conversation (words retrieved during videoed conversations) of at least five and at least 10 words. As shown in Table 13, 23 (28.4%) participants in the CSLT group, 6 (8.8%) participants in the AC group and 8 (9.5%) participants in the UC group recorded a clinical improvement of at least five words at 6 months from baseline. That is, only about 1 in 10 participants in the UC or AC groups showed a clinical improvement of at least five words, compared with approximately 3 in 10 in the CSLT group. This translates to a 18.9% or 19.6% (about 2 in 10) increase in the proportion of participants showing clinical improvement in the generalisation of treated words used in conversation in the CSLT group compared with the AC or UC groups, respectively, at 6 months. Only a handful of participants improved by > 10 words across interventions in the short and long term at 6, 9 and 12 months.

Carer-rated communication effectiveness and impact on carers’ quality of life

One of the additional research objectives is to investigate the effect of the intervention on the carer-rated communication effectiveness (using the first 15 questions from the CaCOAST) and impact on the carers’ quality of life (using the last five questions of the CaCOAST). We refer to these as ‘CaCOAST 15’ and ‘CaCOAST 5’, respectively. It should be noted that this exploratory analysis includes only available carers who agreed to take part. In addition, the unit of randomisation was the participant and not the carer.

The mean change in carer-rated communication effectiveness at 6 months was 6.8% in the CSLT group, compared with 1.0% in the UC group, translating to an adjusted MDC of 4.6% (95% CI 0.3% to 9.0%) in favour of CSLT (Table 14). This small improvement in carer-rated communication effectiveness in CSLT was similar when compared with AC: 5.1% (95% CI 0.5% to 9.7%). However, the long-term effects of the intervention on average change in the carer-rated communication effectiveness were very small: 0.6% (95% CI –4.4% to 5.7%) and 2.7% (95% CI –1.9% to 7.4%) in favour of CSLT compared with UC at 9 and 12 months, respectively.

For the carer-rated impact on their quality of life at 6 months, UC decreased by 1% and CSLT improved by 5.3%, translating to a positive adjusted MDC of 5.3% (95% CI –1.1% to 11.7%) in favour of CSLT. However, although this seems to be of potential clinical relevance, we cannot rule out the lack of benefit. The improvement in carers’ quality of life in CSLT compared with AC was close to zero: 0.3% (95% CI –6.4% to 6.9%). In other words, about 5% improvement in carers’ quality of life could be attributed the attention given rather than the computer therapy alone. The average effect at 9 months was slightly lower than at 6 months (4.0%, 95% CI –3.3% to 11.2%) in favour of CSLT compared with UC. Although CIs around observed effects include the null effect of zero (no difference between groups), clinical judgements need to be made on whether or not the observed treatment effects are of clinical importance or relevance. In addition, interpretation should be made in consideration of the observed effect between CSLT and AC.

Safety and negative effects

Model design

A Markov model was developed to estimate the long-term cost-effectiveness of the CSLT intervention for the primary analysis using Microsoft Excel^® 2010 (Microsoft Corporation, Redmond, WA, USA). The structure of the model was adapted from a previous model developed alongside the pilot CACTUS study. 18 The previously developed model consisted of three health states:

1. aphasia

2. good response

3. dead.

Owing to the additional time points at which data were collected in the Big CACTUS trial, it was possible to differentiate between responses achieved (or maintained) at the 6-, 9- and 12-month assessments. Therefore, tunnel states (i.e. states in which participants cannot remain for more than one modelled cycle) representing ‘good response (6 months)’ and ‘good response (9 months)’ were added to the model. Whether or not a patient achieved a good response was based on the co-primary outcome measures of the Big CACTUS trial: relating to change in word-finding ability and change in functional communication in conversation (see Chapter 3, Word-finding of personally selected words for treatment and Functional communication in conversation for descriptions of these measures). In the economic model, participants moved into a ‘good response’ health state if they achieved the defined minimum change for either one of these measures, where the minimum change was an increase of ≥ 10% in treated words of personal relevance named correctly for the word-finding outcome, or an increase of ≥ 0.5 points since baseline in the activity dimension of the TOMs rating scale for the functional outcome. Because we do not differentiate between good responses depending on which of the co-primary outcome measures the specified minimum improvement was achieved in, we assume that any person who meets the ‘good response’ criteria experiences the same utility gain, irrespective of which outcome measure improved for them.

The model used 3-month cycles. Figure 20 illustrates how participants could move through the model. All participants began in the ‘aphasia’ health state. Participants could remain in the ‘aphasia’ health state at 6 months, or transit to the ‘good response (6 months)’ or ‘dead’ health states. Participants in the ‘good response (6 months)’ health state could transit to the ‘good response (9 months)’ health state, relapse to the ‘aphasia’ health state or die. Participants in the ‘good response (9 months)’ health state could transit to the ‘good response (12 months and beyond)’ health state, relapse to the ‘aphasia’ health state or die. No new responses could occur beyond 12 months; participants in the ‘aphasia’ health state at this time point could either remain in that health state or die, whereas participants in the ‘good response (12 months and beyond)’ health state could remain in that health state, relapse to the ‘aphasia’ health state or die.

FIGURE 20.

Markov model structure. Navy-coloured health states represent tunnel states.

The economic model evaluated the treatment groups included in the Big CACTUS trial, and so transitions between health states included in the model were based on data from Big CACTUS, combined with information on post-stroke mortality rates. For the first 5 years of the model, mortality rates were used from a study of patients who had experienced a stroke ≥ 1 year previously. 57 After 5 years, additional mortality was applied based on Office for National Statistics life tables,58 taking into account the mean age of the Big CACTUS trial participants. We assumed that which health state people resided in did not affect the probability of death, and therefore the same probability of death was applied to all health states in the model. This means that we assumed that none of the treatment options under investigation conveyed a survival advantage.

Participants moved into a good response health state if they reported an increase of ≥ 10% in treated words of personal relevance named correctly between baseline and 6 months [or between baseline and 9 or 12 months for the ‘good response (9 months)’ and ‘good response (12 months)’ health states, respectively] or if they reported an increase of ≥ 0.5 points since baseline in the activity dimension of the TOMs rating scale). Relapse rates between 6 and 9 months and between 9 and 12 months were estimated by calculating the proportion of participants in the ‘Good response (6 months)’ and ‘Good response (9 months)’ health states who did not retain that response at the following time point. Beyond 12 months, the relapse rate was assumed to be equal to that observed between 9 and 12 months.

Health-related quality-of-life utility scores were applied to each health state, and costs were estimated for each of the treatment options under investigation (see Health-related quality of life and Resource use and costs). The utility scores applied to each health state were reduced over time to account for ageing according to multipliers estimated by Ara and Brazier,59 and QALYs were estimated for each cycle of the model by combining utility scores with life-years, allowing the total QALYs associated with each treatment strategy to be calculated. QALYs were discounted at a rate of 3.5% each year, in line with recommendations made by NICE. 52

The model structure requires that we assume that patients who reside within the same health state have the same utility score, regardless of their pathway into that health state. Hence, for example, a new responder at 12 months has the same utility score as someone who responded at 6 months and remained in the good response health state. Similarly, someone who responds and then relapses has the same utility score as someone who never responded. This is a simplification, but modelling further detail would have required additional health states, which would have resulted in estimating utility scores for some health states based on extremely small patient numbers. We believe that our model structure captures the key health states most relevant to the research question.

Health-related quality of life

Big CACTUS trial participants were asked to complete an accessible version of the EQ-5D-5L questionnaire at each of the data collection time points (baseline and 6, 9 and 12 months). An accessible version of the EQ-5D-3L questionnaire was trialled in the CACTUS pilot study. 18 Although accessible versions of the EQ-5D-3L and EQ-5D-5L questionnaires have not been validated, they represent a way in which utility (HRQoL) scores can be elicited directly from people with aphasia, whose language difficulties may make it difficult to complete standard EQ-5D-5L questionnaires. 60,61 The accessible version of the EQ-5D-5L used in Big CACTUS is available online (see www.journalslibrary.nihr.ac.uk/programmes/hta/122101/#/) and is described in Whitehurst et al. 61 Where applicable, carers of Big CACTUS trial participants were asked to complete a standard EQ-5D-5L questionnaire by proxy at each of the data collection time points. Carers were also asked to complete a standard EQ-5D-5L for themselves at each time point, to investigate potential HRQoL impacts on carers. In line with Al-Janabi et al. ,62 utility scores from the EQ-5D-5L were used to investigate carer utility.

The EQ-5D-5L represents the standard questionnaire used to calculate utilities adopted by NICE. 52 When the EQ-5D-5L was developed, it was anticipated that this would replace the EQ-5D-3L and hence the five-level version was used in the Big CACTUS trial. Indeed, NICE supports sponsors of prospective clinical studies using the five-level version. 48,52 However, the English tariff for the EQ-5D-5L published in 2016 is not currently recommended for use in a NICE economic evaluation (but may be recommended in future). 48,52,63 Instead, NICE currently recommends converting EQ-5D-5L responses to utility scores using the UK cross-walk mapping algorithm developed by van Hout et al. 49 in 2012. This algorithm maps EQ-5D-5L responses onto the EQ-5D-3L tariff in order to generate utility scores. Given the preference for patient-elicited utility scores,52 and evidence suggesting suboptimal proxy assessments of utility in the context of stroke,64 utility scores calculated using the accessible version of the EQ-5D-5L and the van Hout et al. 49 mapping algorithm were used in our ‘base-case’ analysis. Several secondary analyses were conducted using:

• the standard EQ-5D-5L questionnaire completed by proxy by carers of participants

• EQ-5D-5L scores calculated using the English EQ-5D-5L tariff,47 with no mapping

• an alternative mapping algorithm developed by Hernandez-Alava et al. ,63,65 converting EQ-5D-5L responses to EQ-5D-3L utility scores.

In addition, a secondary analysis was carried out in which QALYs associated with carers were estimated from their self-reported EQ-5D-5L scores and incorporated into the analysis.

In our base-case analysis, MI was used to impute EQ-5D-5L utility scores where data were missing. Predictive mean matching using Stata version 1538 was used to impute missing values of EQ-5D-5L scores at baseline and at the 6-, 9- and 12-month follow-up time points. 66 An estimate for the missing values was obtained for 10 imputed data sets based on recommendations made in the literature,41 and Rubin’s67 framework was used to combine these estimates across imputed data sets to produce an overall averaged estimate. The imputation models were adjusted for variables that were considered to be potentially prognostic for the participant’s health status and for which there were full data. These variables were the participant’s treatment group assignment (UC/AC/CSLT), the participant’s age and the participant’s gender. EQ-5D-5L scores for the patients who died during the course of the trial were imputed as zero values after the patient’s death. Patients who died were excluded from the analyses estimating the change in utility between responders and non-responders. A secondary ‘complete-case’ analysis was conducted, which did not impute missing EQ-5D-5L data.

Using the EQ-5D-5L data collected in the trial, we estimated utility scores associated with the ‘aphasia’, ‘good response (6 months)’, ‘good response (9 months)’ and ‘good response (12 months)’ health states. The utility score associated with the ‘aphasia’ health state was the mean score across all Big CACTUS participants at baseline. To estimate the impact of achieving a good response on utility scores, we calculated the change in utility score between baseline and the 6-, 9- and 12-month time points for responders and non-responders. By comparing changes in utility score since baseline, we were able to control for differences in baseline utilities that may have existed between participants who subsequently responded and participants who did not.

Resource use and costs

Only costs directly related to the intervention associated with each treatment option were included in the economic evaluation. The pilot CACTUS study collected data on a wide range of resource use (such as medication, primary care and hospital care) but did not show important differences associated with the CSLT intervention with respect to these. 18 This led to the decision not to collect such data in the Big CACTUS trial, and not to include these costs in the economic evaluation. Related to this, it was assumed that the CSLT and AC treatment options were provided in addition to UC, and that UC would not differ between treatment options. This is because we were evaluating therapy delivered by computer in addition to UC received, rather than instead of UC. Given this, the cost over and above UC associated with the UC-only treatment option in the economic model was zero. For further information on the UC actually received during the Big CACTUS trial period, see Chapter 2, Usual-care control group. For the CSLT and AC treatment options, the costs associated with delivering these treatments were estimated. Costing was based on the standard approach used in economic evaluations following the three-stage process: identification of resource use, measurement and valuation. 54

Addressing uncertainty

Distributions were placed around each of the uncertain parameters included in the model (i.e. transition probabilities, utility scores, resource use/costs) for use in probabilistic sensitivity analysis, which allows the uncertainty in all of the modelled parameters to be characterised and permits the estimation of cost-effectiveness acceptability curves (CEACs). The probabilistic sensitivity analysis incorporated 10,000 realisations of the model. Gamma distributions were used for resource use/cost parameters, a beta distribution was used for the ‘aphasia’ health state utility combined with normal distributions around utility changes associated with a ‘good response’, and beta distributions were used for probabilities, with dispersions based on numbers observed in the trial. We also undertook expected value of perfect information (EVPI) analysis. The value of information framework allows the maximum value of further research to be estimated, taking into account the uncertainty in the parameters included in the economic model. 70,71 EVPI was assessed using the Sheffield Accelerated Value of Information tool,72 assuming a cost-effectiveness threshold of £20,000 per QALY gained (based on NICE decision rules52) over a period of 10 years (assuming that it might take 10 years before a new treatment for these patients is developed), using a 3.5% discount rate.

Deterministic sensitivity analyses were undertaken using various alternative techniques for estimating utility scores (see Health-related quality of life) and additional scenario analyses were conducted to investigate the impact of different assumptions around the cost of the computer program used by the CSLT intervention. For ease of reference, all secondary analyses are listed in Table 21.

Subgroup analysis

Cost-effectiveness was investigated in the prespecified subgroups detailed in Chapter 3, Subgroup evaluation:

• Groups with differing severities of word-finding difficulty at baseline, identified using scores from the CAT Naming Objects. Severity was categorised as mild (31–43), moderate (18–30) or severe (5–17).

• Groups with differing impairments in their comprehension ability at baseline as assessed by the CAT Comprehension of Spoken Sentences. Severity was categorised by scores as normal (27–32), mild (18–26), moderate (9–17) or severe (0–8).

• Groups based on the time that had elapsed following the patient’s stroke, which were categorised into quantiles ranging from the 25% of patients who entered the trial closest to their stroke to the 25% who had had their stroke longest ago.

Model-based analysis

Model results: base case

The model-based long-term cost-effectiveness results (primary analysis) are presented in Table 24. The results presented are probabilistic rather than deterministic. This means that they represent the average of 10,000 runs of the model, in which values for the model parameters are randomly drawn from their specified distributions (as defined in Tables 22 and 23 and Appendix 11, Table 35, for transition probabilities, utilities and resource use, respectively). This explains the slight difference in costs estimated compared with the deterministic figures presented in Model inputs: resource use and costs. We estimate an incremental cost per patient of £732.73 (95% CrI £674.23 to £798.05) for CSLT compared with UC, and of £694.59 (95% CrI £636.46 to £760.09) for CSLT compared with AC. We estimated incremental QALY gains per patient of 0.0172 (95% CrI –0.05 to 0.10) for CSLT compared with UC, and of 0.0173 (95% CrI –0.05 to 0.10) for CSLT compared with AC. These result in estimated ICERs of £42,686 per QALY gained for CSLT compared with UC, and of £40,164 for CSLT compared with AC. We estimate that AC is dominated by UC as it is costlier and provides fewer QALYs.

TABLE 24 - Model-based results: base-case analysis

Analysis	Outcome	Intervention	Comparator	Difference in mean
CSLT vs. UC	Costs (£)	732.73	0.00	732.73 (95% CrI 674.23 to 798.05)
	QALYs	4.2164	4.1992	0.0172 (95% CrI –0.05 to 0.10)
	ICER			£42,686 per QALY gained
CSLT vs. AC	Costs (£)	732.73	38.14	694.59 (95% CrI 636.46 to 760.09)
	QALYs	4.2164	4.1991	0.0173 (95% CrI –0.05 to 0.10)
	ICER			£40,164 per QALY gained
AC vs. UC	Costs (£)	38.14	0.00	38.14 (95% CrI 34.94 to 41.50)
	QALYs	4.1991	4.1992	–0.0001 (95% CrI –0.02 to 0.02)
	ICER			Dominated

Figures 21–23 depict the cost-effectiveness planes for the comparisons of CSLT vs. UC, CSLT vs. AC and AC vs. UC, respectively. Plots are presented on the same axis scales to aid comparability. These illustrate the results of the model for each of the 10,000 realisations run in the probabilistic analysis, and therefore illustrate the degree of uncertainty associated with the results. The figures show considerable dispersion across the x-axis for all three comparisons, reflecting the high level of uncertainty associated with the estimation of incremental QALYs. It is clearly highly uncertain as to which treatment option is likely to result in the most QALYs gained; for instance, CSLT is estimated to result in more QALYs than UC in 69% of the realisations of the model. There is less uncertainty around costs, demonstrated by a much narrower dispersion across the y-axis.

FIGURE 21.

Cost-effectiveness plane: base-case analysis – CSLT vs. UC.

FIGURE 22.

Cost-effectiveness plane: base-case analysis – CSLT vs. AC.

FIGURE 23.

Cost-effectiveness plane: base-case analysis – AC vs. UC.

Figure 24 presents the CEAC for the base-case analysis, illustrating the probability of each strategy representing the most cost-effective use of resources for a range of cost-effectiveness thresholds (i.e. the threshold of willingness to pay for an additional QALY). Assuming a cost-effectiveness threshold of £20,000 per QALY gained, the treatment option with the highest probability of representing the most cost-effective strategy is UC, with a probability of 0.56. CSLT and AC both have a probability of 0.22 of representing the most cost-effective strategy at this threshold. At a cost-effectiveness threshold of £30,000 per QALY gained, the probability of UC representing the most cost-effective option is 0.45, compared with 0.32 for CSLT and 0.22 for AC. Hence, although CSLT was estimated to result in the greatest number of QALYs in the majority of the realisations of the model, it provided an ICER of below £20,000–30,000 per QALY gained in a minority of realisations.

FIGURE 24.

Cost-effectiveness acceptability curve: base-case analysis.

The results of the EVPI analysis is presented in Appendix 13. We estimated a population-level EVPI of £34.6M. The utility difference associated with a good response at 12 months was the most important parameter in the model, and obtaining perfect information on this parameter was valued at £20.4M at the population level. Hence, if further research was planned, it should be targeted at obtaining further information on this parameter.

Within-trial analysis

Inputs: health-related quality of life

The advantage of a within-trial analysis is that the uncertainty associated with extrapolating beyond the trial period is avoided. The disadvantage is that this results in an evaluation of only the short-term cost-effectiveness of the interventions, with longer-term impacts being ignored.

For the within-trial analysis, a model was not needed; QALYs were simply estimated for each treatment option according to EQ-5D-5L responses reported during the trial, adjusted for any baseline differences between treatment groups using SUR.

Figure 25 illustrates the trend of mean accessible EQ-5D-5L scores with 95% CIs at baseline and the follow-up points (6, 9 and 12 months) for the UC, AC and CSLT treatment groups, where missing scores were imputed as described in Health-related quality of life, and where utility scores were derived using the van Hout et al. 49 algorithm. Similar figures are presented in Appendix 16 for a ‘complete-case’ analysis (where missing values were not imputed) and for an analysis where only data from participants who provided responses at all follow-up time points were included. Appendix 16 also includes these results in tabular form, and presents similar results for proxy EQ-5D-5L scores and carer EQ-5D-5L scores.

FIGURE 25.

Means and 95% CIs of accessible EQ-5D-5L scores by treatment group at each time point after missing EQ-5D-5L scores are imputed.

Results suggest that imputation had a small impact on estimated utility scores, although differences were slightly larger when comparing the utility scores based on imputed data with those estimated including only patients who provided complete responses at all time points. This is not unexpected, because most data are lost when only patients who provide complete responses at all time points are included. Importantly, changes in utility over time are extremely marginal for all treatment groups. However, it appears that utility scores decline slightly in the CSLT group whereas they increase slightly in the UC group.

Impact on word-finding

The improvement in retrieval of treated words at 6 months post randomisation was substantially greater with CSLT than with UC or AC. On average, CSLT improved word-finding by 16.2% more than UC (95% CI 12.7% to 19.6%; p < 0.0001) and 14.4% more than AC (95% CI 10.8% to 18.1%), showing that the effect was largely due to speech and language therapy components of the CSLT intervention and not just extra activity and attention. The effect was in excess of the prespecified minimal clinically important difference of 10%. This effect was mostly maintained at 9 and 12 months, suggesting maintenance of the treatment effect following the 6-month intervention period. It should be noted that 57 (61%) of the participants in the CSLT group continued to use the computer therapy unsupported beyond 6 months, which may have supported this maintenance. Of the 57 who practised beyond 6 months, 33 practised between 6 and 9 months and 24 practised beyond 9 months.

Functional communication, patient perception and quality of life

There was no significant improvement in the co-primary outcome measure of functional communication in conversation in any of the groups, providing no evidence that the CSLT approach we evaluated for word-finding improves communication in conversation. Our key secondary outcome measure suggests that CSLT did not result in improvement in participants’ own perceptions of their communication and participation and the impact of these on their life, which is unsurprising given that no improvement in functional communication was seen. These findings indicate that CSLT supported participants with aphasia to make significant improvements in their ability to retrieve words of personal importance at the impairment level, but these gains did not generalise to functional communication settings, thus limiting the impact of the improvements in their lives.

Generalisation of improvement in word-finding to conversation was further explored through a secondary outcome measure specifically identifying the difference in the number of treated words used in conversations structured around topics that provided the opportunity to use those words. In keeping with the co-primary outcome measure of conversation, the groups were similar with respect to the number of words used in conversation. However, a post hoc analysis showed that 1 in 10 participants in the AC and UC groups, and 3 in 10 participants in the CSLT group used at least five more treated words in conversation at 6 months than at baseline, suggesting that there may be a small treatment effect of the use of treated words in the functional context of conversation in a few participants. The lack of carryover of improvement from retrieval of treated words in a confrontation naming task to use in functional communication situations may be a result of people being used to communicating in a different way (e.g. gesture) and therefore not remembering to use new words when they have the opportunity. In addition, retrieving words in functional communication settings is likely to be a subtly different to or a more difficult task than naming a picture. Further intervention may be required to help people with aphasia use their ‘new’ words in useful situations.

Generalisation to untreated words

A further secondary research objective of the trial was to investigate whether or not learning of treated words generalises to being able to retrieve words that were not specifically treated. There was no significant difference between the three trial groups in retrieval of untreated words between baseline and 6 months. Therefore, improved word-finding of treated words did not result in generalisation to untreated words. This suggests that people get better at saying what they practise saying > 4 months post stroke. Because individuals with long-term aphasia may learn only the words they practice, it is most useful to them if these words are of personal relevance.

Impact of aphasia profile, word-finding severity and length of time post stroke

Prespecified subgroup analyses indicated that the effect of CSLT on word-finding was slightly higher for participants with mild word-finding difficulties and for those whose verbal comprehension was within normal limits. It is likely that the brain lesions of these participants were smaller, allowing greater retained potential to relearn words with practice. Importantly, subgroup analyses showed that the treatment effect was broadly consistent regardless of the time post stroke, suggesting that people with aphasia may learn new words at any time after stroke (range of 4 months to 36 years in our trial).

Carer perception of communication effectiveness and impact on their quality of life

The carers also rated their perception of the participant’s communication effectiveness and their own quality of life. For communication effectiveness, there was a MDC of 4.6% (95% CI 0.3% to 9.0%) in favour of CSLT compared with UC at 6 months and a MDC of 5.1% (95% CI 0.5% to 9.7%) in favour of CSLT compared with AC, indicating a small improvement in the carers’ perception of communication effectiveness with CSLT. However, the long-term effects of the intervention on the average change in the carer-rated communication effectiveness were very small: 0.6% (95% CI –4.4% to 5.7%) and 2.7% (95% CI –1.9% to 7.4%) in favour of CSLT compared with UC at 9 and 12 months, respectively. This indicates that any perception of improvement in communication effectiveness is unlikely to be maintained long term. There was also a 5.3% (95% CI –1.1% to 11.7%) MDC in perception of carers’ quality of life in favour of CSLT compared with UC at 6 months. However, the improvement in carers’ reported quality of life in the CSLT group compared with AC was close to zero (0.3%, 95% CI –6.4% to 6.9%). This suggests that the small improvements seen in carers’ quality of life at 6 months may be due to the increased levels of activity their relative is engaged in and the receipt of increased amounts of attention in the CSLT and AC interventions compared with UC alone.

Safety

We investigated the safety of the CSLT intervention. Negative effects of CSLT were low, with 27% of participants in the CSLT group reporting fatigue or anxiety at some point, which translates to an average of only one event per person per year. Effects on eyes or headaches were very rarely reported. On average, the incidence of AEs per participant per person-year of follow-up was 2.18, 1.79 and 1.87 in the CSLT, AC and UC groups, respectively. This indicates a slight increase in all AEs in the CSLT group, with an IR of 1.16 (95% CI 0.83 to 1.62) and 1.22 (95% CI 0.85 to 1.77) compared with the UC and AC groups, respectively, although we cannot rule out similarity in incidences between groups as the CIs include an IRR of 1. Fits and visual difficulties were uncommon, but the incidence of fits was reported to be three times higher in the CSLT group than in the UC or AC groups, although such events were rare in all groups. Similarly, slightly more AEs were reported in the CSLT group than in the UC or AC groups; however, the CSLT group had more opportunity for reporting AEs as participants were prompted with a negative effects form each month in addition to the 3-monthly check by outcome assessors received by all groups, whereas participants in the AC and UC groups were not. Differences in the incidence of AEs between groups were insufficient to suggest differences in risk levels. The number of SAEs was 18 (15.8%) in the UC group, 11 (13.9%) in the AC group and 9 (10.6%) in the CSLT group. Although there were fewer SAEs in the CSLT group, there is insufficient evidence to suggest differences in IRs of SAEs across interventions.

Main findings

The base-case (primary) model-based cost-effectiveness analysis resulted in an ICER of £42,686 per QALY gained for CSLT compared with UC, and an ICER of £40,164 per QALY gained for CSLT compared with AC. AC was dominated by UC, meaning that the UC group cost less and had greater quality-of-life gains (measured on EQ-5D-5L for the health economic analysis) than the AC group. These ICERs are higher than the current NICE cost-effectiveness threshold of £20,000–30,000 per QALY gained,52 suggesting that in the UK the CSLT approach we evaluated is unlikely to be considered to represent a cost-effective use of health-care resources for the whole population with word-finding difficulties as a result of post-stroke aphasia.

However, cost-effectiveness estimates were highly uncertain. Although results were robust to altering assumptions around software and computer costs, they were very sensitive to the utility score estimates associated with achieving a good response to treatment, which were highly uncertain. It is not clear whether or not the intervention leads to an increase in quality of life as measured by the EQ-5D-5L. The cost-effectiveness results were sensitive to using different techniques to derive utility scores (a measure of HRQoL) from EQ-5D-5L responses. The base-case (primary) analysis used the EQ-5D-5L mapping algorithm developed by van Hout et al. 49 to calculate utility scores, as recommended in NICE guidelines for cost–utility analysis. 52 However, an unvalidated accessible version of the EQ-5D-5L61 was used so that utility scores could be derived directly from trial participants. In circumstances in which trial participants cannot complete a standard EQ-5D-5L questionnaire themselves, a typical approach is to use scores derived from standard EQ-5D-5L questionnaires completed by carers by proxy. When we conducted secondary analyses using this technique, the ICER for CSLT compared with UC fell to £28,819, potentially representing a cost-effective use of health-care resources. Using an alternative mapping algorithm (that developed by Hernandez-Alava et al. 63) to derive utility scores from the accessible version of the EQ-5D-5L also led to a reduced ICER for CSLT compared with UC, equal to £34,921 per QALY gained. These results are consistent with findings from work by the NICE Decision Support Unit comparing the mapping methods. 63 In contrast, directly using the English tariff for converting EQ-5D-5L responses to utility scores (with no mapping algorithm) led to an increase in the ICER, to £55,639 per QALY gained.

In addition, we found that important reductions in the ICER could be achieved if SLT costs could be reduced substantially. For instance, if the computer software could be set up for participants by SLTs in 1 hour instead of the average 5.77 hours observed in the Big CACTUS trial, the ICER for CSLT compared with UC would decrease to £30,181 per QALY gained. However, this ICER would remain highly uncertain owing to the uncertainty surrounding the QALY gain. In the trial, we did not evaluate the impact of the involvement of SLTs in the intervention. However, as SLT tailoring of the software incurred a significant proportion of the intervention cost, it is important to understand if and how this is related to the clinical outcomes. The relationship between SLT support and intervention effectiveness is the focus of a completed PhD.

Explanation of findings

The sensitivities on the ICERs according to the method used to derive utility scores demonstrate that the key uncertainty in this economic evaluation surrounds the size and direction of the QALY gain or loss associated with CSLT compared with UC and AC. A key message is that CSLT did lead to significantly more participants achieving a ‘good response’ in naming of words with personal relevance or conversation than UC and AC did. However, the impact on utility of achieving a good response appeared to be small and highly uncertain. Given the low cost associated with CSLT, very small increases or decreases in the estimated QALY gain can have a large impact on the ICER. In the vast majority of analyses completed (primary, scenario and subgroup analyses), the CIs for the QALY gain associated with CSLT compared with UC ranged from negative to positive. A related issue surrounds the definition of a ‘good response’. Although CSLT led to substantially more participants achieving a ‘good response’, large proportions of participants in the AC and UC groups also achieved a ‘good response’. Potentially, the definition of response (improvement of 10% in word-finding or an improvement of 0.5 on the TOMs) may have been too lenient; differences in quality of life may be more likely in participants who achieve a greater response. Related to this, an alternative modelling approach could have split the response state according to whether a response was achieved through a word-finding improvement or an improvement on TOMs (or both). In practice, as would be expected given the clinical results, the majority of responses were due to improvements in word-finding; for instance, at 6 months only 8 of the 73 responders in the CSLT group achieved a response based on TOMs and not on word-finding, and 23 achieved a response on both measures. Hence, splitting the response category would have involved estimating utility scores for different response groups based on very small patient numbers and would have been prone to substantial error.

In the base case (the primary health economic analysis), the utility score in patients who achieve a good response is estimated to be worse at 6 and 9 months than that in patients who do not achieve a good response. At 12 months, good responders are estimated to have marginally higher utility than non-responders. Given that all CIs overlap, these alternative directions in utility differences may be due to chance. Alternatively, they may suggest that it takes time for a good response to lead to utility gains as measured by the EQ-5D-5L.

Given the clinical results observed in Big CACTUS, it may not be surprising that CSLT had little impact on utility scores. Although the intervention had a significant impact on word-finding ability, it had no discernible impact on functional communication measured or on participants’ perceptions of communication effectiveness or impact on their lives. Therefore, it seems unlikely that a response would have an impact on the EQ-5D-5L domains to indicate quality of life, and indeed our within-trial analysis estimated marginally more QALYs for the UC and AC treatment groups than for the CSLT group, although this was highly uncertain. If the improvement in word-finding ability could be converted into a functional improvement, it is possible that QALY gains could be derived and, given the low cost of the intervention, this could result in CSLT being deemed cost-effective. Research to investigate this further may be valuable. In our value of information analysis, we found that the expected value of obtaining perfect information on the change in EQ-5D-5L score associated with a good response at 12 months is £20.4M. This suggests that it would be highly valuable to collect more data on this model parameter to enable it to be quantified more accurately.

Subgroup analysis

Importantly, the prespecified subgroup analyses (of the model-based health economic evaluation) suggested that the CSLT approach we evaluated may represent a cost-effective use of health-care resources for participants with mild or moderate word-finding difficulty. In these groups, CSLT was associated with ICERs of £22,371 and £28,898 per QALY gained, respectively, compared with UC, and of £30,911 and £18.855 per QALY gained compared with AC. In contrast, CSLT was associated with a higher cost and lower quality-of-life scores than AC and UC for participants with severe word-finding difficulty (CSLT was dominated by AC and UC). The proportion of CSLT participants who achieved a good response was similar in the different word-finding difficulty subgroups, but cost-effectiveness was improved in the mild and moderate groups owing to a slightly increased utility difference associated with a good response at 12 months in these groups, and a negative difference (compared with no response) in the severe subgroup. However, these analyses remain highly uncertain and, as for the base-case analysis, achieving a good response was associated with a reduction in utility at earlier time points for both the mild subgroup and the moderate subgroup. Again, the direction of the utility changes may have been due to chance. In addition, within-trial analyses in the same subgroups provided opposite results (whereby CSLT was dominated by UC in the mild and moderate word-finding difficulty subgroups). Related to this, it is important to reiterate that cost-effectiveness results based on within-trial and model-based analyses were often substantially different because the estimated higher EQ-5D-5L score associated with CSLT at the 12-month time point was extrapolated for the remaining lifetime period of the model-based analysis, but was not extrapolated in the within-trial analysis.

Cost of computerised speech and language therapy intervention

The intervention cost £733 per participant, which is relatively low. This enabled a mean of 28 hours of independent, repetitive word-finding practice. Although we do not know the relative effectiveness of 28 hours of therapy provided face to face by a SLT, providing an additional 28 hours of face-to-face speech and language therapy by a mid-grade (Agenda for Change band 6) SLT would cost £1400, almost twice as much as supporting an individual to practise independently with a computer. If the time taken by SLTs to set up the computer program could be reduced, the cost-effectiveness of the intervention could improve markedly.

Quality of computerised speech and language therapy intervention delivery

Measures used as indicators of quality of intervention delivery included training, intervention knowledge of providers, appropriate tailoring of the computer therapy and provision of support to the participants. There was high/good fidelity to these measures of intervention delivery quality. Regarding support provided by volunteers/SLTAs, 85% of the participants received support for using their newly learned words in conversation or functional contexts; however, these skills were practised with the volunteer or SLTA for a total of only 45 minutes (median) per participant across the 6-month intervention period. Increased amounts of time spent on these transfer activities may be required to assist with greater use of the newly learned words in conversation. Only 52% of feedback forms from volunteers/SLTAs were monitored by SLTs, indicating lower adherence to the role of monitoring the SLTA/volunteer support provided to the participant.

Adherence to computerised speech and language therapy practice

The CSLT participants practised for a mean of 28 hours, just above the recommended minimum for high adherence (26 hours); the median was below this at 21 hours, with just fewer than half of the participants (46%) meeting this minimum for high adherence. Sixty-four per cent of participants practised for a minimum of 10 hours, indicating some adherence to the intervention. Although we are able to report the amount of practice time, further analysis has been conducted during a PhD study regarding the content of the practice and motivation to practise.

Conclusion for computerised speech and language therapy

Overall fidelity to the CSLT intervention was judged to be high to fair.

Fidelity to attention control

The AC intervention was to complete a puzzle each day with a supportive telephone call from the research team each month for 6 months. As we expected periods of illness and holiday as with the CSLT group, we expected a minimum of four telephone calls to have taken place and four to six puzzle books to be completed. It was difficult to measure the adherence to puzzle book practice. Only 18% of participants had six books and four telephone calls but 61% of participants had a minimum of four puzzle books and four telephone calls, suggesting that adherence to the AC group was fair and therefore similar or slightly lower than the adherence to the CSLT intervention.

Fidelity to usual care

The UC received during the trial was generally similar between groups, suggesting that the adherence to the provision of UC was consistent across groups. During the 6-month intervention period, the mean amount of UC received by all participants in the UC, AC and CSLT groups, respectively, was 3.8 hours, 3.2 hours and 3.2 hours. The amounts of UC provided decreased over the trial period in all groups. This is likely to be because participants progressed along the stroke pathway of care as the trial progressed. The pre-baseline UC data indicated that patients receive reduced amounts of UC with time post stroke.

Age and gender

The age and gender profile of the participants of the Big CACTUS trial was in keeping with that seen in other aphasia studies (not all RCTs, therefore open to selective sampling), 60.7% male being consistent with 60.8% of the Rehabilitation and recovery of people with aphasia after stroke (RELEASE) data set77 of 5573 aphasic trial participants being recorded as male. The mean age of participants in Big CACTUS was 65.4 years. Again, this is similar to the median age of 63 years for 5871 aphasic participants in the RELEASE data set. 77 This is also consistent with the average age seen in stroke rehabilitation trials (64.3 years) and is almost a decade younger than those seen by physicians in daily practice. 78

Computer use

The mean amount of self-managed practice with the computer therapy in Big CACTUS was 28 hours over 6 months. This is consistent with the amount of self-managed practice conducted in the pilot study: a mean of 25 hours over 5 months. 21 In a recent systematic review of computerised aphasia therapy interventions, the protocols varied with recommended practice schedules of 10 to 11 hours over 2 months, 20 hours over 20 sessions, 24 hours over 11 to 12 weeks and 78 hours over 6 months. 17 These studies were all conducted in clinical facilities with at least some therapist supervision, unlike Big CACTUS, which was a pragmatic study of computer therapy self-managed by the person with aphasia at home, encompassing greater participant control and choice over amounts of practice. The total amount of practice carried out by the Big CACTUS participants was similar to that in two of the studies described in the systematic review but over a longer duration of time. In addition, 61% of the Big CACTUS participants continued to practise beyond the 6-month supported self-managed trial period, suggesting that participants may choose to practise less intensively for a longer duration than provided in therapist-supervised protocols studied. In the Oral Reading for Language in Aphasia (ORLA) study,10 a study of computer-based reading therapy for people with aphasia, there was also some variation in the amount of practice participants chose to carry out.

Word-finding

The Big CACTUS trial demonstrated significant improvements in word-finding at the impairment level of 16.2% (95% CI 12.7% to 19.6%; p < 0.0001) in favour of CSLT compared with UC in the chronic phase (> 4 months) post stroke. This finding is very similar to the findings of the pilot study, based on 34 participants, which showed improvements of 19.8% (95% CI 4.4% to 35.2%; p = 0.014) in favour of the computer therapy compared with usual stimulation (support groups but no therapy intervention) more than 6 months post stroke. The Cochrane review4 included three studies that compared naming therapy to social support using impairment-based naming outcome measures. On pooling the data, no differences were seen (standardised mean difference 0.14, 95% CI −0.10 to 0.38; p = 0.26). 4 However, the word-finding measures used were standardised and therefore will have been observing improvement to words that were not specifically treated in therapy. A small case series study (16 participants) by Best et al. 51 suggested that only one in four participants improved on untreated words, and then they improved by only a small amount (4%) between pre and post therapy measures. With control groups, the Big CACTUS trial showed no effect of the CSLT word-finding therapy on finding words that were not treated, in keeping with the findings in the 2012 Cochrane review of aphasia therapy79 and previous studies of word-finding therapy. 80–84

Functional communication, participation and quality of life

The CSLT for word-finding did not result in improvements at the level of communication activity and participation or quality of life in the Big CACTUS trial. A common criticism of impairment-based aphasia therapies is that improvements are not reflected in real-life, day-to-day communication. 85 The lack of generalisation of new word-finding ability to functional contexts is therefore consistent with findings from other, smaller studies of word-finding therapy. For example, Best et al. 85 demonstrated carryover of naming therapy into conversation for some individuals (as seen in Big CACTUS) but not for the group of study participants as a whole. This underlines the need for additional therapy components to be added to impairment-based intervention to aid use of new language skills.

Improvements in communication activity, participation or quality of life were not measured quantitatively in the CACTUS pilot study; however, some participants described functional improvements during qualitative interviews (e.g. use of their new words in conversation or in functional contexts, and improvements in confidence to communicate in functional communication settings). 22 The quantitative findings from the full trial therefore diverge from these qualitative findings. One explanation of the differences could be that ‘response shift’ may occur in patient-rated outcomes, whereby the internal standards against which the participants rate themselves do not stay constant between one time point and another. 86 This could lead to descriptive improvements in interviews that are not accounted for in responses on patient-rated outcome measures.

Health economics

For the heath economic evaluation, the base-case (primary) model-based cost-effectiveness analysis resulted in an ICER of £42,686 per QALY gained for CSLT compared with UC. This is much higher than the ICER estimated from the pilot study (£3127 per QALY). 18 Reasons for this difference are that incremental costs associated with the intervention were lower in the pilot than in the main study (£469 compared with £733) and incremental QALYs were higher (0.15 compared with 0.02). In the pilot study, the intervention was delivered predominantly by a SLT who was part of the research team and more familiar with the computer program than the therapists delivering the intervention in the full, pragmatic trial in routine clinical practice conditions. In addition, only 48 bespoke words were prepared by the SLT in the pilot, whereas 100 words were prepared in the full trial. The amount of therapist time taken to deliver the intervention was therefore lower in the pilot (5 hours and 20 minutes rather than 9 hours) and had a lower cost. The full study has shown QALY measurement in this population with this intervention to be highly variable, which may account for the difference in QALY gains seen in the two studies. Of particular note is that the utility gain associated with a ‘good response’ was 0.07 (95% CI –0.15 to 0.29) in the pilot study, compared with 0.02 (95% CI –0.03 to 0.07) at 12 months in Big CACTUS. Caution must be taken with this comparison because the definition of a response differed between the two studies. However, clearly the CIs around the utility change have been reduced by Big CACTUS, but they have centred around the lower end of the interval estimated in the pilot study, which is also the area that overlaps zero. For this reason, the value of the uncertainty that remains is almost as high after Big CACTUS as it was after the pilot study: a population-level EVPI of £37.0M was estimated from the pilot study, compared with £34.6M based on Big CACTUS. This is due to the increased ICER and also the finding that we still cannot be certain about whether or not the intervention results in a QALY gain.

An additional factor is that the EQ-5D-5L was used in Big CACTUS, whereas the EQ-5D-3L was used in the pilot study. Research has shown that the EQ-5D-5L is likely to reduce the utility increment or decrement associated with a quality-of-life change, compared with the EQ-5D-3L, and in fact a NICE Decision Support Unit report on this topic87 used the CACTUS pilot as a case study and estimated that the utility gain associated with a good response would have been 0.02 had the EQ-5D-5L been used. However, in line with NICE recommendations,48 in our base-case economic evaluation we used the van Hout et al. 49 cross-walk algorithm to map EQ-5D-5L responses onto the EQ-5D-3L tariff in order to generate utility scores; hence, the decrement associated with using the EQ-5D-5L should not be present. The utility gain associated with a good response was marginally lower when the EQ-5D-5L tariff was directly used, but the difference was marginal (0.017 compared with 0.02).

Strengths of the trial design

The trial compared the CSLT intervention under study with UC, but also had a third, AC, group so any effect of CSLT seen over and above UC can be attributed to the speech and language therapy components of the intervention as opposed to the receipt of extra attention. There is debate about the use of appropriate AC in the aphasia literature, suggesting that often social support AC interventions are so closely matched to speech and language therapy (in terms of materials and support) that any difference between the interventions is eroded, making it more difficult to establish the effectiveness of one intervention over another. 89 In the CSLT intervention, owing to it taking a self-management approach, the extra attention gained by the participants is limited to the monthly support with computer exercises from volunteers or SLTAs. Therefore, our third group uniquely attempted to control for both additional activity (the focus of self-managed therapy) and additional attention. The attention provided was the same frequency as for the CSLT group, but was kept distinct from the attention received in the CSLT group by focusing on different materials (puzzle books).

Our eligibility criteria were inclusive and therefore representative of the population treated in practice. For example, it is standard practice to exclude people who have had more than one stroke in many stroke studies; however, eligibility was not limited to only having had one stroke in Big CACTUS to reflect that people who have had more than one stroke do routinely receive treatment. We included practical tests to see whether or not individuals were able to see and manipulate the computer software as part of our eligibility criteria. Similar screening is likely to be carried out in clinical practice. However, assumptions that older people struggle to use computers may reduce the number of older people screened using these practical tests in practice.

A range of outcome measures were used to explore the effect of CSLT on all dimensions of the ICF (impairment, activity and participation) and on quality of life. An international core outcome set for aphasia using these dimensions has been developed. 24,90 Best et al. 85 identified that aphasia studies often measure only impairment-based improvement, stating ‘attempts to measure carryover to everyday conversation are conspicuous by their absence’. This was supported by the outcomes of the 2016 Cochrane review4 for aphasia in which only 31 out of 71 randomised comparisons used a functional communication outcome for word-finding therapy. Although the co-primary outcome measures were conducted by qualified SLTs, secondary outcome measures considered the participant’s own view and the views of carers using patient- and carer-rated outcome measures. An accessible variant of the EQ-5D-5L designed in collaboration with the PPI group enabled participants to record their own views regarding quality of life for the health economic analysis.

Random allocation to the trial groups was concealed and therefore not predictable for any trial staff. The conduct of blinded outcome measure assessments was a strength of the trial. (Our co-primary outcome measures were blinded, although the secondary patient-rated outcome measures could not be conducted blind to group allocation.) Blinding was relatively successful, with only 28 participants having at least one unblinded 6-month outcome measure conducted in total, which is negligible in terms of the numbers of assessments made in the trial. It is of note that the possibility of unblinding applied to only the co-primary word-finding measure as conversation videos were rated on the TOMs by SLTs independent of the trial, allowing blinding to group allocation and time point to be maintained. Reliability testing between SLTs scoring the conversations with the TOMs and those scoring the naming test at each site was a further strength of the design.

The trial benefited from comprehensive SAPs and health economic analysis plans being agreed before unblinding of the data for analysis (see www.journalslibrary.nihr.ac.uk/programmes/hta/122101/#). The trial was adequately designed and powered to address multiple primary and key research questions while controlling for the chances of making false-positive conclusions about the effects of the intervention. In addition, statistical analysis explored the impact of missing data on the results under a number of scenarios for sensitivity analysis, as well as the impact of adherence to the components of the CSLT and AC interventions.

The strengths of the health economic evaluation include its comprehensive investigation of many different scenarios and analyses. The use of an accessible version of the EQ-5D-5L is novel and permits the elicitation of utility scores from a patient group that is not well provided for with respect to preference-based utility measures. 60,61 The collection of EQ-5D-5L questionnaires completed by carers by proxy and also for themselves allowed two additional valuable analyses to be conducted. Given the current uncertainty around how best to use the EQ-5D-5L measure, it was important to conduct analyses using all the valuation options. The economic evaluation undertaken alongside the CACTUS pilot study was simplistic and results were highly uncertain. 18 Although this more complex evaluation has also produced highly uncertain results, we have gained important information on where the uncertainties lie. Finally, this evaluation has been conducted in line with best-practice guidelines and includes a rigorous investigation into the uncertainty associated with the decision-making problem.

Patient and public involvement

Patient and public involvement was a huge strength of this trial, meeting all four of its aims to:

1. facilitate the recruitment and inclusion of people with aphasia in the trial.

2. ensure that trial materials and processes were accessible to people with aphasia.

3. ensure that the interventions and trial procedures were appropriate and manageable for people with aphasia.

4. ensure that dissemination of trial results reached a broad audience in accessible formats.

Collaborating on the production of accessible information formats, including trial information, consent forms, wording to be used in trial procedures, computer-support guides, accessible outcome measures and monthly newsletters, has been described throughout the report. The PPI group led on the development of the study dissemination plan, considering all of the groups we need to target with the results, including people with aphasia, carers, SLTs, health professionals, guideline developers, commissioners, providers of services, voluntary sector organisations and researchers. The PPI group instigated and collaborated on the production of a film showing what aphasia is like, the CSLT intervention and our findings; a results booklet to accompany the film with key information facts and figures, and aphasia-friendly summaries of key points alongside; and flyers containing a simple summary of the key findings to send to all the participants of the trial and to display in public spaces in health and social care venues.