Developing decision support tools incorporating personalised predictions of likely visual benefit versus harm for cataract surgery: research programme

Programme structure and interlinkage of work packages

The cataract research programme consisted of four interlinking mixed-methodology work packages (WPs) (Figure 1), with work from each WP feeding forward to contribute to and inform subsequent elements. The overarching theme throughout the programme was the development and assessment of clinical tools intended to support and enhance pre-operative decision-making by patients considering cataract surgery and health-care professionals (HCPs) delivering their care.

FIGURE 1.

Research pathway diagram. Cat-PROM5, Cataract Patient-Reported Outcome Measure, five items; PROM, patient-reported outcome measure.

Work package 1 comprised the development, assessment and validation of a psychometrically robust cataract patient-reported outcome measure (PROM) for the measurement of visual difficulty from cataracts and its relief from surgical intervention. The resultant self-reported measure, Cataract Patient-Reported Outcome Measure, five items (Cat-PROM5), was taken forward to WP3, in which prediction models for the Cat-PROM5 outcome were developed to predict the likely patient self-reported post-surgery benefit. Occasionally, surgery can have adverse outcomes that are of importance to patients and HCPs. In WP2, existing prediction models for established adverse outcomes were reassessed on fresh, routinely collected NHS electronic medical record data for two important adverse outcomes: an intraoperative complication [posterior capsule rupture (PCR)] and VA loss affecting the eye that was operated on following surgery.

Using questionnaire data collected in WP3, a parallel health economics mapping exercise was undertaken to translate the Cat-PROM5 scores into established health economic utilities, thus facilitating future health economics research in cataract studies through reducing questionnaire burden on study participants. Qualitative work elements ran through each of the WPs to gain a rounded picture of the preferred formats for the development of clinical tools based on outputs of the prediction models and the information requirements of both patients and HCPs. Through this approach, a cataract decision aid (CDA) was developed, providing patients and HCPs with relevant information on cataracts and cataract surgery, including personalised predictions of both the likely post-operative self-reported visual benefit and possible adverse outcomes. In WP4, we assessed the feasibility of a future fully powered randomised controlled trial (RCT) with the CDA as an intervention.

In addition to the four main WPs, a number of additional studies were opportunistically undertaken and these have provided ‘added value’ to the cataract research programme overall.

Posterior capsule rupture

Posterior capsule rupture data were complete. In this sample, PCR arose in 3514 out of 180,114 (1.95%) eyes.

Visual acuity loss

There were some missing VA data. Of the 180,114 eyes in the sample, 147,962 had pre-operative VA readings. For 104,437 (70.6%) eyes, these were corrected VA readings; for 43,525 (29.4%) eyes, these were uncorrected readings. A further 1591 eyes had pre-operative pinhole readings only, but these were not used in this analysis. Of the 180,114 eyes in the sample, 116,038 had post-operative VA readings. For 74,887 (64.5%) eyes, these were corrected VA readings; for 28,678 (24.7%) eyes, these were uncorrected VA readings; and for 12,473 (10.8%) eyes, these were pinhole VA readings, which were accepted post operation. Of the 147,962 eyes with pre-operative VA readings, 95,561 (64.6%) had post-operative VA readings. For 60,578 (63.4%) eyes, they were corrected VA readings; for 24,460 (25.6%), they were uncorrected VA readings; and for 10,523 (11.0%), they were pinhole VA readings. In this sample of operations, undertaken between 2006 and 2010, the VA of 1455 out of 95,561 (1.52%) eyes deteriorated after cataract surgery, a proportion which has significantly reduced since these (early) data were collected. 24

Backwards and forwards logistic regression was carried out, taking account of the clustered nature of the data (the fact that left and right eyes from the same patient are unlikely to be independent). Having established the logistic regression model, the model statistics were reported (see Report Supplementary Material 1).

Posterior capsule rupture

The following variables entered into/remained in the logistic regression model: age, surgeon grade, pupil size, glaucoma, brunescent/white cataract, no fundal view/vitreous opacities, previous vitrectomy, pseudoexfoliation/phacodonesis and other ocular pathology. The area under the receiver operating characteristic (ROC) curve (C-stat) for this model was 0.64, indicating fair fit.

Visual acuity loss

The following variables entered into/remained in the logistic regression model: PCR, age, axial length, pupil size, gender and the following copathologies; glaucoma, age-related macular degeneration, diabetic retinopathy, corneal pathology, brunescent/white cataract, previous vitrectomy, other macular pathology, other retinal vascular pathology and other ocular pathology. The area under the ROC curve (C-stat) for this model was 0.71, indicating reasonable fit.

Both the PCR and VA loss models were broadly similar to our earlier work carried out on 55,678 cataract operations undertaken before and up to 2006,26 suggesting that the models were reasonably stable prior to 2010.

Aim

To review methodological options for development of the risk prediction model for PCR on extended, more up-to-date data, and to assess PCR model stability through time.

Approach

Available options for logistic regression model building were reviewed and applied to the data. In this methodological exercise we chose to analyse PCR only, as the data for this key surgical outcome were complete.

Data collection

Multicentre data were obtained from NOD1 through a data-sharing agreement with the data controller, HQIP. The data covered cataract surgery undertaken by participating centres between 2000 and 2014. The anonymised sample consisted of 602,459 operations conducted on the eyes of 404,857 patients.

Data analysis

Three approaches to candidate risk predictor selection for model building were used: (1) a clinically sound list of predictors; (2) the χ² p-value of < 0.10 was used for predictors to exclude those that, from the outset, appeared unlikely to be statistically important; and (3) a univariate effect size (ES) satisfying 0.9 > odds ratio (OR) > 1.20 was used to exclude small, and therefore clinically unimportant, effects.

Following these analyses, stability through time was considered. Early years were excluded, as the sample sizes were smaller and the number of less common risk indicators were too few, which would generate unstable results. The 10-year period from 2005 to 2014 was taken forward to assess logistic regression model stability through time. The models derived from the three options for initial selection of candidate predictors were applied to each year separately, and were assessed for stability across the 10 years.

Key findings

The three candidate predictor selection options resulted in slightly different lists of predictors, indicating that the method of selection of candidate predictors (clinically sound, or χ² p < 0.1, or 0.9 > OR > 1.20) was important. Model fit was fair to moderate for all of the models, with C-statistics near or slightly above 0.6.

The models, which were derived using the full set of data, were applied to each of the years separately and parameters examined for stability across the years. For a number of variables, statistical significance was not consistently achieved within years. Overall, for all of the initial candidate selection options, the models were stable for the period 2005–11. The year 2012 was an atypical year, not being consistent with earlier or later periods, whereas 2013 and 2014 were comparable.

These results confirm that, although the models are generally temporally stable, there are inconsistencies through time that underline the importance of reviewing risk models from time to time when these models are being used to adjust for outcomes of centres and surgeons.

Limitations

The data used in these analyses are ageing, and these analyses could be brought up to date through analysis of a fresh sample.

Because of the problem of missing VA data, it was not deemed appropriate to extend the analysis to include VA loss.

Relationship with other work packages

This ‘added value’ item in the programme provided key information about the stability of the PCR risk model and the model-based risk calculator used in the CDA (WP4).

The wider implications of these findings include confirmation that the risk model, although stable across certain periods, is prone to shifts; this emphasises the need to review and revise the models that are being used in national audits to adjust for case complexity of centres and surgeons (see Report Supplementary Material 2).

Aim

To report operative complications (PCR) and VA outcomes for cataract surgery in people aged ≥ 90 years.

Approach

We performed an analysis of elderly patients undergoing cataract surgery.

Data collection and analysis

A subgroup analysis of data obtained through a data-sharing agreement with the NOD and HQIP (as noted in Data collection) was undertaken. We analysed the outcomes of 25,856 cataract operations undertaken in 19,166 people aged ≥ 90 years between the years 2000 and 2014. 27

Key findings

A significant operative complication (PCR) occurred in 2.7% of all operations. Although this is higher than the rate found among younger patients, the surgical risk remains relatively low. 24

Post-operative VA was available for 61.8% of eyes, with VA being good enough for the patient to drive (Snellen VA 6/12) in 74.4% overall, and 84.7% reaching this level out of those without a visually significant comorbidity.

Limitations

The main limitation of this study was the reduced proportion of patients with a post-operative VA recording.

Relationship with other work packages

This work was a standalone substudy offering added value to the programme overall.

This ‘added value’ analysis has been published. 27

Aim

To undertake a proof-of-concept analysis to identify indicators of refractive outcome using complex numbers as a basis for analysis.

Approach

The measurement of refraction consists of three variables (sphere, cylinder and axis), which makes the analysis and reporting of refraction problematic; a variety of simplified measures are used clinically to choose an appropriate lens implant at the time of surgery. A comprehensive measure of refraction can, however, be achieved using complex numbers. 28 This complex-numbers approach has been used in this proof-of-concept analysis to assess its methodological feasibility for use as a comprehensive refractive outcome measure for cataract surgery that, crucially, does not approximate or discard any of the information in the refractive result.

Data collection and analysis

A fresh third set of electronic medical record data was obtained from the NOD through an updated data-sharing agreement with HQIP. Data on 1,070,601 cataract operations were received. The primary refractive outcome of interest was available for 491,414 of these operations. The primary outcome was trivariate and was modelled using a multivariate, normal, multilevel model and four multilevel structures. 29

Key findings

Despite the methodological complexity of the analytical approach, it was feasible to apply the method to refractive outcome data and rigorously model indicators of post-operative refraction. Analysis revealed that ocular comorbidities did not impact refractive outcome to a clinically important extent, but the complex-numbers methodology revealed a greater spread of refractive outcomes than expected. This method will be of interest to surgeons for outcomes assessment and refinement of refractive outcomes.

Limitations

Less than half of the full set of data contained all of the information required for the analysis, potentially reducing the generalisability of the results. However, the purpose of the analysis was to demonstrate feasibility through a proof-of-concept analysis, which was achieved.

Relationship with other work packages

This analysis is a standalone item with regard to the other WPs, but post-operative refraction is highly relevant to patients’ post-operative perception of their surgical outcome. The relationship between refractive outcome and patients’ self-reported opinion has been highlighted in the findings of the WP4 discordant outcomes study (see Work package 4, Qualitative element 2: mismatching outcomes).

Aim

To develop a benefits prediction model for personalised prediction of self-reported Cat-PROM5 benefit from cataract surgery.

Approach

We conducted a cohort study of 1500 people undergoing cataract surgery to phenotype the participants, with a view to identify pre-operative indicators of post-operative self-reported benefit from surgery in terms of final post-operative outcome and improvement from pre-operative baseline.

Data collection

The cohort study took place at two sites: Bristol and Gloucestershire. Pre-operative patients approaching cataract surgery were invited to participate. The inclusion criteria were aged ≥ 50 years; approaching either first- or second-eye cataract surgery; willingness to participate and provide informed consent; and the ability to understand and complete the Cat-PROM5, EuroQol-5 Dimensions, three-level version, (EQ-5D-3L), EuroQol-5 Dimensions, three-level version, with vision ‘bolt-on’ domain (EQ-5D-3L+VIS), EuroQol-5 Dimensions, five-level version (EQ-5D-5L) and ICEpop CAPability measure for Older people (ICECAP-O) questionnaire instruments as required. Following recovery, postal questionnaire completions were made between 6 and 8 weeks after surgery. Full recruitment of 1506 participants was achieved, with 1204 patients recruited in Bristol and 302 in Gloucestershire. It is noteworthy that encouragement from our Patient Advisory Group to introduce a self-referral option resulted in 15% of Bristol’s recruitment through this route. Following withdrawals, losses to follow-up and data cleaning, 1181 participants had complete data for analysis (Figure 2).

FIGURE 2.

The Predict-CAT flow diagram. a, Some people gave more than one reason; b, the number analysed varies between reports because of data completeness. BEH, Bristol Eye Hospital; GHNHSFT, Gloucestershire Hospitals NHS Foundation Trust. HE, health economics questionnaire.

Data analysis

Among the 1181 participants with valid data, scattered missing data items remained. To preserve the sample from further attrition, these missing data items were imputed using multivariate imputation by chained equations (MICE) in which 20 data sets were created with missing data replaced by imputed values, each entailing 10 cycles of regression switching. This method relies on the missing at random (MAR) assumption. In parallel with the multiple imputation analyses, complete-case analyses were also undertaken (with missing values ignored). The complete-case analyses were based on 1089 complete cases.

Initial descriptive analyses of candidate predictor and outcome variables were undertaken, followed by linear regression modelling of two Cat-PROM5 Rasch-calibrated outcome variables. The final outcome was modelled in two forms: first, as the post-operative score and, second, as the improvement from baseline (the difference between the pre- and post-operative scores = delta approach). Potential predictors were categorised into blocks according to timeline order, from earlier to later, and from the most general diseases to the most specific. All models included age, gender and the baseline Cat-PROM5 status as predictors, regardless of their observed ‘statistical importance’. Skewed distributions were transformed if necessary, variables were entered into the model in ordered blocks and an F-test was performed for each block as a whole. If the p-value for the block was > 0.05 then the whole block was rejected. If the p-value of the test for the block was < 0.05, the specific predictors were examined and those with small effects were iteratively removed. After each stage, all of the predictors were reviewed by an experienced ophthalmologist for whether the list and the model made clinical sense, and predictors without plausible clinical meaning were removed. Following model construction, the model diagnostics were checked and acted on if necessary. (See Report Supplementary Material 4 for further details.)

Results

The results from both approaches (ignoring missing values and applying multiple imputation) produced very similar results. The models based on analysis of the multiply imputed sets of data for the Cat-PROM5 post-operative final outcome achieved an R² of 29.1%; for improvement in Cat-PROM5 from pre to post operation, the models achieved an R² of 31.2%. The final models with and without imputation, together with model diagnostics, are available in Report Supplementary Material 4.

Key findings

Full recruitment to the cohort study was achieved. The losses to follow-up were below the level of 20% for which we had planned. Some further losses arose because of missing key variables, but the resulting sample of just under 1200 was sufficiently powerful for the intended analyses with and without multiple imputation.

Models with and without imputation produced similar results.

The benefit indicators required for the model comprise data items that are readily accessible in the context of patient care. The only indicator that is not routinely collected in pre-operative clinics is reading vision, but this is easily tested and its requirement should not present a significant barrier to implementation.

Limitations

The main limitation of this element of the work was the moderate fit of the prediction models, each with an R² ≈ 0.30.

Although the two chosen analytical approaches produced very similar models, it is possible that data were not missing at random, which may have introduced bias in the results based on multiple imputation.

The performance of these models was further assessed using fresh data in WP4.

Relationship with other work packages

The self-reported benefits prediction models were set alongside the models predicting adverse outcome risk from WP2, with all of these models taken forward into the CDA developed in WP3, qualitative element 2. The CDA was, in turn, taken forward to the WP4 feasibility RCT.

Aims

• To identify the most acceptable way of presenting risk and benefit probability information to patients as part of a CDA.

• To identify cataract surgery-specific FAQs from patients and HCPs to inform the development of a CDA.

• To explore the acceptability of Cat-PROM5 to health-care providers.

• To explore the perceived issues of shared and informed decision-making during cataract surgery patient counselling with patients and HCPs to inform the development and implementation of a CDA in routine practice.

Approach

The qualitative research elements ran alongside the quantitative research. The purpose of this work was to gather views and insights that could be used to inform the development of the decision support tools being developed in the programme. Recruitment was from patients attending the cataract services, with stratification where feasible, for example for gender. Interviews were guided by an interview schedule or topic guide, and were audio-recorded, transcribed verbatim and analysed using NVivo software. 20,22,23 Focus groups similarly followed a standard approach to methodology, data collection and analysis. 30,31

This work included conducting:

• Focus groups and interviews with patients to inform

  • The presentation format of risk information for the development of a risk and benefit probability calculator.

  • The presentation, issues of concern, supporting information and FAQs for the development of a CDA.

• Interviews with HCPs to

  • Explore the acceptability of Cat-PROM5.

  • Inform content development and explore the usefulness of decision aids in clinical practice.

  • Consider the attitudes towards and practice of shared decision-making.

Data collection

Results

Key findings

Patients preferred ‘n out of 100’, as it was the most easily understood format for risk presentation.

Issues of perceived interest, concern or importance for cataract surgery were elicited from patients and HCPs. This knowledge informed the development of the FAQs section of the CDA (WP3, qualitative element 2).

Cat-PROM5, as part of a CDA, was seen as worthwhile by HCPs in terms of standardising the assessment of self-reported difficulty suffered by patients with cataracts.

A CDA and shared decision-making in cataract surgery were mostly seen as relevant and necessary by HCPs, with risk information considered to be an important aspect of a shared discussion.

Limitations

Moderate sample sizes from a small number of centres may limit the generalisability of these findings, although recruitment continued until saturation was achieved, which will ameliorate this potential risk.

Relationship with other work packages

The format for the presentation of the personalised risk/benefit information and the information content for the FAQs were taken forward to the CDA in WP3, qualitative element 2, the Predict-CAT instrument development stage.

Cat-PROM5’s acceptability to HCPs and views on shared decision-making informed the approach to engagement with this service-critical group with regard to the set-up of the WP4 feasibility RCT.

Aims

• To develop a CDA in a FAQ format in which the likelihood of the Cat-PROM5 self-reported benefit is set alongside risks of harm (surgical complications/VA loss) to provide an integrated decision support tool with personalised prediction of outcomes.

• To develop a cataract decision quality measure (CDQM) for the cataract surgery decision.

• To conduct qualitative user testing of the personalised CDA with patients and clinicians to assess usability, acceptability, utility and expected impact.

• To refine the personalised CDA and CDQM for use in WP4 (the feasibility trial).

Approach

The International Patient Decision Aid Standards (IPDAS) Collaboration Checklist32 for the development and quality assessment of patient decision aids was used to guide the development of the CDA. We used a collaborative, multistage development process involving key stakeholders, and we conducted qualitative user testing interviews with patients and clinicians. Recruitment was from the cataract services, with purposive stratification where feasible [e.g. gender and professional group (surgeon, nurse, optometrist)]. Interviews were guided by an interview schedule or topic guide, and were audio-recorded, transcribed verbatim and analysed. 20,22,23 The focus group followed the standard approach to methodology, data collection and analysis. 30,31

Development of the prototype cataract decision aid

The multistage, iterative process used to develop and refine the CDA included a number of key activities and data sources:

• A CDA User Reference Group (URG) of HCPs and patient representatives was convened. The role of the URG was to act as an editorial team to oversee the development of the CDA (including agreeing the clinical evidence content and patients’ FAQs) and the associated CDQM.

  • A total of 16 individuals were recruited to the URG, including two patient representatives, six members of the study team and eight clinicians (working within cataract services of the four NHS trusts participating in the cataract research programme).

• A focus group was conducted with the cataract research programme PPI PAG members.

  • Five members of the PAG attended the focus group session. The audio-recorded focus group was transcribed and feedback gathered.

• To further inform the evidence-based content of the CDA, several sources of data/evidence were consulted:

  • Focus groups and patient interviews that were previously conducted as part of WP3, qualitative element 1.

  • The scoping review of published guidelines and literature.

  • Expert opinion.

  • Predicted harm models (WP2).

A prototype CDA was developed in agreement with the URG using the data gathered, as outlined above.

The prototype CDA comprised the following elements:

• Introduction page.

• Section A: FAQs.

• Section B: What matters to you? What questions do you have?

• Section C: personalised information about your likely outcomes.

This version of the CDA was used for the user testing interviews.

User testing of the prototype cataract decision aid

User testing of the CDA was conducted with patients and clinicians to assess usability, acceptability, utility and expected impact. The four NHS cataract surgical centres involved in the cataract research programme (Bristol, Gloucestershire, Torbay and Brighton) were used to identify and recruit patients and clinicians for the user testing interviews.

Patients were eligible to participate if they were post-operative patients who had undergone cataract surgery at least 6 weeks previously. User testing interviews were conducted with a total of 26 participants: 20 patient participants (10 male and 10 female) and six clinician participants (four ophthalmologists, one nurse and one optometrist). Clinician input was supplemented by the URG and the study management team.

Semistructured interview questions for the patient participants covered the following topics: views on the usefulness of the CDA; understanding of the content (including terminology, presentation of risks, etc.); views on the FAQs included, and the ranking of the candidate FAQs (to discover if any excluded FAQs should be included); and views on the different parts of the CDA (the personalised risk section, space for questions, etc.), the layout/design, readability, when/how they would like to receive the CDA and potential improvements.

Semistructured interview questions for the clinician participants covered the following topics: views on the usefulness of the CDA (for themselves and patients), the accuracy/comprehensiveness of the content, views on the different parts of the CDA (the personalised risk section, space for questions, etc.), fit within local clinical systems, potential influence on their behaviour, perceived barriers to and facilitators of use, and contextual factors that might impact future implementation.

Patient and clinician participants were sent a copy of the prototype CDA prior to the telephone interview. All patient and clinician audio-recordings were transcribed verbatim for qualitative analysis and were imported into NVivo. A framework analysis approach was used to analyse the data, based on the key topic areas covered by the interview guide (usefulness, use in clinical practice, etc.) and the different components of the CDA (FAQs, personalised information section, etc.). New emerging themes that had not been captured by the initial framework were added.

Framework analysis was conducted on 26 user testing interview transcripts. Patient and clinician data were analysed together, but any key differences in perspective were noted. The initial framework was guided by the interview schedule and consisted of five categories: (1) initial thoughts on the CDA and the purpose of the CDA, (2) perceived usefulness of the CDA, (3) CDA content, (4) CDA design/format and (5) implementation in a clinical setting. Two new categories were added to the framework during analysis: (6) information provision and preferences and (7) decision-making role and experiences. Data were summarised into a matrix for each category in the analytical framework. Verbatim quotes were extracted and entered.

Refining the prototype cataract decision aid

The results from the user testing interviews with patients and clinicians were used to refine the prototype CDA. The key sections included in the prototype CDA remained the same, but the pages in the booklet increased from seven to nine. Additional questions were added to section A (FAQs), and further clarification and details were added. Presentation changes were made to section C (personalised information about your likely outcomes), technical terms were replaced with plain English terms and an additional page for notes was added.

Cataract decision quality measure

Decision quality measures (DQMs) are condition-specific instruments that have been developed to assess patients’ understanding of available treatment options, their personal preferences, their perceived readiness to make a decision and the alignment between their preferences and preferred option. DQMs are made up of four key sections: (1) knowing your options (knowledge questions), (2) what is important to you? (preference questions), (3) thinking about the decision you face (readiness to decide, standard items that are not condition specific) and (4) choice of treatment (which assesses current treatment preference). We drew on our experience of developing DQMs for other clinical areas (breast cancer, tonsillectomy, etc.) as part of a SDM implementation programme.

We aimed to develop a CDQM, which would subsequently be used in the WP4 randomised feasibility trial. The content of the CDA was used to inform the knowledge questions used in the CDQM (section A). The candidate FAQs included in the CDA and the feedback received from patients during the user testing interviews and from the PAG focus group (about what mattered most to them) informed the preference questions (section B). The readiness-to-decide questions (section C) were standardised generic questions that have been used in other previously developed DQMs. Section D (choice of treatment) was guided by the available options.

Key findings

By involving key stakeholders in the development of the CDA, we were able to ensure that input from patients and clinicians was considered in terms of the content, format and planned use, and that the CDA best reflects the needs of the end users. Initial input from the cataract PAG highlighted the issues that matter the most to patients when making the decision about cataract surgery (including likelihood of success/benefits, pain, what happens during the surgery, side effects/risks, eyesight changes and post-surgery recovery), and these were incorporated into the CDA. Evidence syntheses ensured that the information provided in the CDA was evidence-based, for example the FAQs in section A. Incorporating the risk calculator element that was developed as part of WP2 (section C) ensured that individualised risk-information could be provided to patients, and further input from clinicians helped to ratify the accuracy of the CDA content.

The qualitative user-testing interviews found that patients and clinicians were generally positive about the CDA. Both patients and clinicians felt that the CDA would provide a trustworthy source of information, including adequate and accurate information. Clinicians felt that it would act as a framework and a reminder to cover certain information that they might otherwise not cover, and patients felt that the CDA provided them with answers to the questions that they would want answered before making a decision. Patients and clinicians felt that the CDA would reinforce the idea that a ‘choice’ exists with regard to cataract surgery, would encourage better patient involvement in cataract surgery decisions, was easy to understand and could feasibly be integrated into clinical care pathways. Some concerns were expressed by clinicians regarding the time needed to complete the personalised risk element of the CDA in the consultation and they had some reservations over how much information patients would want about their individualised risks. However, there was a disparity between what clinicians’ felt that their patients might want in terms of risk information and what patients said they wanted after reflecting on previous experiences in which they had not received this information. Overall, it was felt that the CDA would be feasible to use in routine clinical settings.

Limitations

Despite significant efforts to recruit clinicians from all four collaborating sites, it was not possible to recruit from one site. However, the clinical input was strong from the URG and the research programme team.

Relationship with other work packages

The Predict-CAT qualitative work (FAQs, etc.) fed directly into the development of the CDA. Both the CDA and the CDQM were taken forward as key instruments in the WP4 feasibility RCT.

Aims

• To evaluate the performance of existing and emerging health economic indices (the preference-based measures EQ-5D-3L, EQ-5D-3L+VIS, EQ-5D-5L and ICECAP-O) in people undergoing cataract surgery.

• To investigate how cataract-related visual disability measured using Cat-PROM5 can be calibrated or mapped against existing and emerging health economic indices of utility.

Data collection

Data for these health economic elements of the programme were collected as part of the WP3 Predict-CAT cohort study at two sites in England. The questionnaires were completed both pre and post operation for all participants, with questionnaire data available for 1315 participants. The Cat-PROM5 and ICECAP-O data were collected from all participants, with collection of the other health economic utilities (the EQ-5D-3L, EQ-5D-3L+VIS and EQ-5D-5L) undertaken on a 1 : 1 : 1 random allocation basis.

Data analysis

Results

Key findings

Performance

The EQ-5D-3L and EQ-5D-5L did not perform well across almost every measure of validity and responsiveness and they had the largest ceiling effects (Table 1); the EQ-5D-3L+VIS had a lower ceiling effect and better convergent validity with Cat-PROM5. The EQ-5D-3L+VIS was able to differentiate between patient groups who did and did not report benefit from surgery and improved quality of life after surgery; however, it also identified small positive ESs in patients who reported no benefit or improved quality of life after surgery. The ICECAP-O also had a low ceiling effect and there was some evidence of convergent validity with Cat-PROM5; it performed best on many measures of responsiveness. 38

TABLE 1 - Summary of preference-based measure performance against the criteria that were evaluated

✓, present; ✗, absent; ?, conflicting results for EuroQol-5 Dimensions randomisation groups.

EQ-5D-3L group correlation coefficient did not exceed 0.3.

Between-group differences were significant (p < 0.05) for the EQ-5D-3L group only.

There was no difference in change scores for the EQ-5D-3L group visual quality of life and patient-perceived benefit obtained from post-operative supplementary questionnaire.

Notes

Ceiling effect: > 15% scoring the maximum of 1.

First-eye or second-eye surgery: second-eye surgery patients were expected to report significantly better health-related quality of life.

Habitual near VA (usual spectacles if worn) in the operated eye (on the logMAR scale): patients with worse VA were expected to report significantly lower health-related quality of life.

Ocular comorbidities: patients with ocular comorbidities were expected to report lower health-related quality of life.

Floor effect: > 15% scoring the minimum possible score.

Criterion	Preference-based measure
	EQ-5D-3L+VIS	EQ-5D-3L	EQ-5D-5L-VSE	EQ-5D-5L-CW	ICECAP-O
Ceiling effect	✓	✗	✗	✗	✓
Floor effect	✓	✓	✓	✓	✓
Convergent validity
Cat-PROM5 correlation	✓	✗	✓	✗	?a
VA correlation	✗	✗	✗	✗	✗
Known groups validity
First-eye or second-eye surgery	✗	✗	✗	✗	✗
Habitual near VA in the operated eye (logMAR)	✓	✗	✗	✗	?b
Ocular comorbidities	✗	✗	✗	✗	✗
Responsiveness
Change scores and ESs
Visual quality of life	✗	✗	✗	✗	✓
Patient-perceived benefit of surgery	✗	✗	✗	✗	?c
Change in near VA in operated-on eye	✗	✗	✗	✗	✗

Calibration/mapping

The adjusted limited dependent variable mixture models dominated linear models on all performance criteria. The mixture models offered good to excellent fit. Three-component models that allowed component membership to be a function of covariates (sex, age and diabetic status) that conditioned on some or all of these covariates (depending on the target measure and pre- and post-operative status) performed better than models with fewer components that did not condition on covariates (Figure 3). The models for EQ-5D-3L did not converge post surgery under any mixture model, possibly because of the large number of participants reporting perfect (EQ-5D-3L index score = 1) quality of life. 39

FIGURE 3.

Calibration/mapping: comparison of actual and predicted follow-up distributions. (a) EQ-5D-5L; and (b) ICECAP-O.

Limitations

A limitation of the study was that the three versions of the EuroQol-5 Dimensions (EQ-5D) questionnaire were completed by different patient cohorts. If participants were to have completed every questionnaire, the response burden would have been excessive.

Relationship with other work packages

In addition to the links and dependencies within the Predict-CAT WP3 cohort study noted here, these health economic elements of the programme link Cat-PROM5, developed in WP1, to established and emerging health economic utilities that are preference-based measures. This work is indirectly linked to the WP4 feasibility RCT as it would inform the choice of a preference-based measure to be included in any future fully powered RCT. Furthermore, the calibration or mapping exercise would allow health economic analyses to be undertaken based on the Cat-PROM5 data without the need to increase questionnaire completion burden through adding a health economics preference-based measure to the trial protocol.

Aims

• To undertake a feasibility study for a possible future fully powered RCT of the use of a CDA, incorporating personalised risk and benefit information, to improve shared decision-making in cataract surgery.

• To estimate the sample size for a possible future fully powered RCT.

• To assess the accuracy of the self-reported outcomes and benefits prediction model developed in the programme.

Approach

This feasibility study, named ‘Involve-CAT’, took the form of a two-arm RCT with the CDA as an intervention. The intervention group was defined as the group of patients in whom the CDA was used, while patients in the standard-care control group underwent standard NHS care. The allocation of patients to groups was conducted through a 1 : 1 block randomisation process by centre. It was assumed that, within each centre, five or six participants should be allocated within each arm (receiving the CDA intervention vs. not receiving the intervention). The research process was multistage, starting with pre-screening and proceeding through assessment of patient eligibility for the study, recruitment, obtaining consent, randomisation, baseline clinical and self-reported pre-operative vision assessment with Cat-PROM5, randomisation to either the CDA intervention or defaulting to standard care, making a shared decision about surgery, and finally documenting the outcome of the operation, including a post-operative self-reported vision difficulty assessment with Cat-PROM5.

The CDQM developed as a part of WP3 was used to assess patients’ decision quality. The CDQM is a measure intended to capture patients’ knowledge about options, preferences and readiness to make a decision about the treatment. It was treated as a primary outcome in this study. The CDQM questionnaire was completed twice, first before the consultation at the baseline visit and then immediately following the consultation. A secondary quantitative outcome was Cat-PROM5, a self-reported measure of vision quality developed and validated in WP1 of the grant programme. The Cat-PROM5 questionnaire was completed by patients twice, initially at the baseline pre-operative time point and then at the post-operative follow-up visit.

Pre-operative parameters were used to predict the expected post-operative Cat-PROM5 responses based on the two ‘benefits prediction’ models developed in WP3. The predicted responses were then compared against the actual self-reported outcomes following surgery to assess the validity of the predictions.

Sample size calculations were undertaken to assess the size of a possible future fully powered RCT.

Data collection

The study assumed recruitment of 40 participants from four cataract research centres (Bristol, Torbay, Brighton and Gloucestershire), that is 10 patients per centre. During the study, however, it became clear that Gloucestershire would be unable to join the study owing to local capacity issues and Torbay would be able to join the study only later, owing to staff illness. This required over-recruitment by Bristol and Brighton; nonetheless, full recruitment of 42 completing patients was achieved (Figure 4).

FIGURE 4.

The Involve-CAT Consolidated Standards of Reporting Trials flow diagram. BEH, Bristol Eye Hospital.

Data analysis

The effects of the intervention were analysed in two ways. For the summary scores expressed by a single value [i.e. knowledge about cataract (section A), readiness for decision-making (section C) and Cat-PROM5], t-tests were undertaken for both dependent (paired) samples and independent samples. Tests for dependent samples were performed to assess the importance of the intraindividual changes in CDQM scores from before to after the consultation and in the Cat-PROM5 measures from before to after surgery. The t-tests for independent samples were used to assess the differences post consultation and post surgery, respectively, for those two measures between study arms. For the linked sections of the CDQM questionnaire that compared reported preferences with the decisions actually made (sections B and D), Spearman’s rho was calculated as a basic statistic capturing levels of concordance between what was reported as important for patients and what was actually chosen.

A power study was undertaken to inform a sample size estimate, providing calculations of the sample sizes needed for a possible future fully powered RCT to investigate the impact of the CDA on the quality of patient decisions. The magnitudes of the ESs were chosen according to Cohen’s classification of standardised ESs40 [standard deviation (SD) of unity].

Bland and Altman plots were used to compare predicted outcomes with observed outcomes for the two benefits prediction models (prediction of the final post-operative outcome and prediction of the improvement in score from pre to post operation).

Results

The level of data collection completeness was good for the feasibility study primary and secondary variables, both before and after intervention/surgery (range 100–88% complete). The intervention group and standard-care control group were similar across key variables at baseline, indicating successful randomisation. Although it is expected that, in a feasibility study, statistical power would be insufficient to arrive at any conclusions, standard analyses were undertaken on the collected data to confirm feasibility for such analyses and to gain insights into the likely future performance of the outcome measures.

Key findings

• It was feasible to achieve timely and full recruitment despite one of the study sites being unable to participate owing to lack of capacity to undertake the research, and the start of recruitment at another site being delayed owing to staff health.

• Data collection completeness for key variables was good and statistical analysis of the collected data was feasible.

• Analysis of the CDQM primary outcome revealed psychometric imperfections in the performance of the knowledge section, which would benefit from review and refinement in advance of a fully powered trial.

• For independent groups, a small ES (0.2 SD) would be detectable with 80% power using a sample size of 800 (1 : 1 allocation, 400 in each group) and a medium ES (0.5 SD) would be detectable with 90% power using a sample size of 200 (1 : 1 allocation, 100 in each group).

• The performance of the prediction model for the final Cat-PROM5 post-operative outcome was superior to the model for prediction of change in the Cat-PROM5 score from pre- to post-operative time points.

Limitations

One limitation of WP4 was related to the CDQM, the knowledge section of which would benefit from further development work prior to consideration of its use as an outcome in a fully powered RCT.

A further limitation was the fact that the prediction model (developed in the WP3 Predict-CAT cohort study) for a change in the Cat-PROM5 score did not demonstrate high predictive power in the WP4 model validation exercise. This is explained by the fact that a subtraction of two measures (post-operative Cat-PROM5 score – pre-operative Cat-PROM5 score) has greater variance and is, therefore, subject to higher uncertainty. However, the final Cat-PROM5 prediction model was confirmed to have reasonable predictive power.

Relationship with other work packages

WP4 depended on the following outputs from each of the earlier work packages:

• Cat-PROM5 from WP1.

• Risk models for PCR and VA loss from WP2.

• Benefits prediction models, qualitative elements guiding instrument development, the CDA and the CDQM from WP3.

Aims

• To explore how a CDA incorporating personalised risk and benefit information influences pre-operative SDM for cataract patients and HCPs.

• To explore how patients and HCPs perceive the CDA in the context of routine care.

Approach

All audio recordings of consultations (CDA group and standard-care control group) in which the surgeon and patient discussed the treatment options for cataract surgery were transcribed verbatim, anonymised and analysed in NVivo. 21,23 This included scoring consultations using the Observer OPTION 5 instrument,42 comparing appointments with the CDA (the intervention group) and without the CDA (the standard-care control group) of the feasibility RCT. 43,44 Sampling was both opportunistic (available HCPs) and purposive (a balance of HCP groups and genders for patients).

Recruitment

Data collection and analysis

All audio-recordings of consultations (CDA and control groups) in which the surgeon and patient discussed the treatment options for cataract surgery were transcribed verbatim. Recordings of all 42 RCT appointments were scored in relation to SDM. Audio-recordings of the follow-up interviews were also transcribed verbatim and transcripts were entered into NVivo for analysis. A framework analysis approach45 was used to analyse the data, based on the key topic areas covered by the interview guide (usefulness, use in clinical practice, experiences of the pre-operative decision-making process, etc.) and also the model of SDM in clinical practice. 46 This model outlines three key stages of the SDM process, characterised by different types of ‘talk’: choice, option and decision.

Post-consultation follow-up interviews with patients and surgeons qualitatively analysed their perceptions of the appointments and the CDA. Separate interview schedules (topic guides) were used for surgeon and patient follow-up interviews (see Report Supplementary Material 11).

A total of 15 patient follow-up interviews were undertaken: 10 in the CDA group and five in the standard-care control group (Brighton, n = 5; Bristol, n = 7; Torbay, n = 3).

Six clinician participants who participated in the CDA arm of the study participated in the post-consultation follow-up interviews across three participating sites (Brighton, n = 1; Bristol, n = 4; Torbay, n = 1). The clinician participants were surgeons (ophthalmologists), four of whom were male and two of whom were female.

Key findings

Several key issues arose that would be likely to have an impact on the effectiveness of the CDA, and the extent to which it could be easily integrated into routine care or as part of a possible future full-scale RCT:

• Overall, the Observer OPTION 5 scores revealed that there was a significant difference in mean total scores between the CDA arm and standard-care control arm, with all five items scoring higher, on average, in the CDA consultations than in the standard-care control consultations. These results indicate that when clinicians use the CDA with patients, a greater number of SDM behaviours are present and they are carried out to a greater extent.

• Analyses revealed that the key SDM tasks of introducing the choice and eliciting patient’s preferences were not always carried out, regardless of whether or not the CDA was used.

• Consultants did not consistently perceive the choices of ‘surgery, delay or decline’ as useful or legitimate and, therefore, some did not agree with the presentation of choices in the CDA.

• Many of the patients had strong prior preferences and had already decided that they wanted the surgery. Thus, it would be difficult to reintroduce the choice talk at the consultation stage. This indicated that the SDM discussion around having or declining cataract surgery might be better placed or at least initiated earlier in the clinical care pathway, before patients had formed strong prior preferences of what they wanted.

• The CDA was very effective at providing information to patients about their options, including their personalised risk, but it did little in the way of supporting the introduction of choice or the elicitation of patients’ preferences, partly because of the patients’ prior preferences.

• A number of approaches could help to rebalance the process towards SDM. For example, more work could be undertaken in the consultation to reintroduce the concept of choice, emphasising that surgery is not a foregone conclusion and providing a clear rationale for patient involvement in the decision-making process; or the introduction of choice could be initiated earlier in the care pathway, for example with a community optometrist, an ‘optician’.

• Overall, surgeons felt that the CDA could be integrated into routine clinical settings and delivered as part of a larger RCT. However, changes would need to be made to the way in which the CDA is delivered, with options including:

  • The CDA could be used as part of a two-stage process. It could be introduced to patients before the consultation, ideally being sent to patients with their appointment letters. In the consultation, the ‘what matters to me’ and ‘personalised risks and benefits’ sections could be elaborated and discussed.

  • The CDA could be introduced at an earlier point in the care pathway, such as prior to referral to the hospital.

  • More consistent and adequate clinician training in SDM could be provided to the wider team of HCPs who would deliver the CDA.

  • The risk calculators could be better integrated into the local clinical systems, with as much of the information pre-populated prior to the consultation as possible.

Limitations

Cataract patient pathways vary considerably between centres. This study involved three centres and different implementation considerations may apply in other centres. The sample sizes were not large, but saturation indicated that these were reasonable.

Relationship with other work packages

The earlier WPs all fed into this element of the work, which looks forward to a possible future RCT of the CDA that acknowledges the lessons learnt here.

Aim

To explore background and specific instances of discordance of outcomes where the perception of the health professional was at odds with that of the patient following cataract surgery.

Approach

We sought out patients with discordant or mismatching outcomes among the WP3 Predict-CAT study cohort participants, and consultants at the four collaborating centres were also separately asked to identify and recruit patients in their centre who had discordant outcomes. HCPs from the collaborating centres with experience of this relatively uncommon phenomenon were invited to join the study.

Discordance could be either positive or negative, defined as:

• Negative mismatching outcomes

  • The patient is unhappy with good surgery and VA outcomes, that is patients who are dissatisfied with the outcome of their surgery or perceive a negative outcome, despite there being no clear clinical explanation for experiencing a poor outcome. Examples might include dysphotopsia, reflections, glare or residual minor refractive error.

• Positive mismatching outcomes

  • The patient is happy with poor surgery or VA outcomes, that is patients who report satisfaction/positive outcomes when the VA or technical elements of the surgery appear to indicate that a normally symptomatic clinical problem exists, of which the patient seems to be unaware. Examples might include reduced VA, intraocular lens subluxed or mild to moderate macular dysfunction.

Data collection

Data collection for this substudy was organised in two separate strands.

The first strand included semistructured one-to-one interviews with patients identified as falling within the mismatching outcomes definition, which were carried out face to face or over the telephone. The second strand of the study involved semistructured one-to-one interviews with HCPs who were involved in cataract care delivery and had experience with the phenomenon of discordant outcomes. 20

We interviewed seven patients. Three patient participants were identified through the Predict-CAT study; four patient participants were identified by HCPs in clinics and using the mismatching outcomes definitions disseminated to each centre. Nine HCPs were interviewed, one HCP participant was an optometrist and eight were consultant ophthalmologists.

Data analysis

All interviews were, with participants’ consent, audio-recorded, fully transcribed and anonymised. Anonymised transcripts were analysed in NVivo 10, guided by principles of thematic analysis. 47

Key findings

• Discordance in most patients’ experience was the result of unexpected changes in visual ability after surgery, for example changed spectacle prescriptions and problems with peripheral vision, but two participants experienced unexpected symptoms, such as floaters and dry eyes, which they felt compromised their quality of life.

• Patient narratives highlighted the nuanced and multidimensional ‘lived experience’ of vision, whereby patients might be happy with one aspect of vision, for example distance vision, but dissatisfied with another, perhaps more valued, aspect that determined their overall judgement on the outcome of the surgery.

• The factors raised by HCPs to explain the phenomenon were linked to medical practice, doctor–patient communication and patient-specific attributes. The factors linked to medical practice were primarily the technologies used, for example the choice of lens implant and unintended optical side effects resulting from individual lenses, and the use of measurement and testing devices that are able to capture the visual experiences of the patients.

• HCPs and patients thought that the quality of the doctor–patient relationship shaped patients’ perceptions of the outcome. Both believed that there was a need for SDM when making decisions on lens choice and refractive aims of the surgery.

• Participants discussed the need for a more personalised approach to patient counselling to address discordances that were thought to exist between patients’ and HCPs’ understanding of what ‘good vision’ means; address patient preferences and expectations and realign these with a more realistic understanding of the potential outcome; and provide accurate and individualised information on what the patient should expect of their visual abilities after surgery, including use of spectacles for near and distance vision and any potential compromises in aspects of vision other than VA.

• For patients, trust in the HCPs was also important, and this trust was found to be compromised through breakdowns in the process of care delivery, for example continuity of care, ease of access to post-operative follow-up, and trust in the providers’ abilities to carry out ophthalmological examinations and procedures.

• HCPs thought that it was important to understand the ‘discordant outcomes’ case profile to identify determinants of discordance and to target patients who might be more prone to dissatisfaction with their outcome so that they could receive more intensive patient counselling.

• HCPs thought that the presence of comorbidities and a more complex clinical profile, the patients’ visual abilities before the surgery, and the patients’ personality and social characteristics explained discordant outcomes.

• The participant recruited to the study who had the only positive mismatching outcome also appeared to suffer the most significant visual disability before the surgery.

• For patients, the quality of cataract surgery counselling received prior to their surgery (trust in HCPs, the quality of their relationship with the HCPs and being engaged in informed and shared decision making) shaped their post-operative experiences.

• However, several barriers to changing practice were identified, often linked to the challenges of changing current ways of working and the time HCPs had available to engage in such in-depth personalised conversations with patients.

Limitations

As this is a relatively uncommon phenomenon, it was difficult to identify patients who had experienced a discordant outcome following their cataract surgery. There was, however, a consistency between the perceptions of affected patients and HCPs with experience of the phenomenon.

Relationship with other work packages

The WP3 Predict-CAT cohort study was used as a source of patients with discordant outcomes; HCPs from the collaborating centres who had knowledge of the programme and its aims assisted in finding the patients and undertaking interviews. This aspect of the work was, however, relatively self-standing, although it did relate to the key programme themes of patient-focused care and improved SDM.

Aim

To estimate the implementation costs and potential savings of the use of a CDA.

Approach

This study compared the additional resources incurred as a result of implementing the CDA with those of standard care in the Involve-CAT pilot RCT study. The additional resource was the time taken to collect the data needed for the CDA and administer the CDA during the SDM discussion, and the subsequent impact on discussions in the remaining appointment. There is also potential for the CDA to impact the number of patients choosing to have surgery and, if it affects the risk profile of patients having surgery, the health-care use following surgery (accident and emergency visits, etc.).

Data collection and analysis

The start and end time of each stage of the pilot RCT study appointment were recorded by the clinicians undertaking the research assessment and clinical assessment appointments. Costs were obtained from the Personal Social Services Research Unit (PSSRU). 48 The cost per minute of the clinician’s time was calculated and used to estimate the total resource use for each participant’s clinic appointment. The clinical assessment and CDA were predominantly led by consultant ophthalmologists. The CDA required an additional assessment of near vision. Secondary analysis included the cost of this additional near-vision assessment. The cost differences between study arms were assessed using two-sample t-tests.

Key findings

• The mean and median durations were longer in the CDA arm than in the standard-care control arm for all intervals reported.

• Two optometrists recorded how long the assessment of near vision took for a subset of patients in the CDA arm. The mean duration of the assessments for the two assessors was 2.6 minutes.

• The mean total cost associated with NHS resource use in the primary analysis was £52.20 for the CDA arm and £37.80 for the standard-care control arm (difference £14.40; p = 0.06; n = 40).

• Inclusion of the additional test for near vision in the secondary analysis increased this difference to £16.87; p = 0.03.

• As all patients in both arms progressed to surgery, only a small difference was observed for subsequent health-care resource use (single centre, n = 23).

• The use of a CDA was likely to moderately increase the pre-operative clinic cost as a result of a longer SDM consultation and the need for a near-vision test for the benefits calculator tool.

• To derive the quality-adjusted life-year data in a future fully powered RCT, the post-operative Cat-PROM5 scores could be mapped to health economic utilities for the CDA intervention and standard-care control groups using the mapping algorithms developed in this research programme.

Limitations

Despite the relatively small sample size in this feasibility RCT, it does appear that the introduction of a CDA would incur modest additional cost.

Relationship with other work packages

All previous elements of the programme, WPs 1–3, fed into this cost analysis.

Aim

To undertake a thematic analysis of a stakeholder meeting exploring the ethics perspectives of immediately sequential bilateral cataract surgery.

Approach

A semistructured independent stakeholder meeting was held at the Royal College of Ophthalmologists London headquarters in June 2018. This event was convened separately from the research programme by one of the programme co-applicants (CL), who approached the chief investigator with a proposal to undertake transcription and qualitative analysis of the discussion that took place at the meeting.

Data collection and analysis

In total, 29 stakeholders attended the meeting; they were invited through purposive sampling. The professions of the stakeholders included but were not limited to ophthalmologists (n = 9), patients (n = 5), religious leaders (n = 4), ophthalmic nurses (n = 3), ethicists (n = 2), lawyers (n = 2) and commissioners (n = 1). Thematic qualitative analysis47 was conducted on the resultant transcript of the discussion.

Key findings

Three overarching themes were identified, which were subdivided into eight subthemes. The themes included (1) beneficence and non-maleficence (patient benefits, patient risks, the uncertainties of risk and patient interpretation of the risk–benefit analysis), (2) autonomy (patient and surgeon choice, informed consent and the barriers to effective communication) and (3) distributive justice (the allocation of resources: the individual vs. the collective).

The stakeholders concluded that the procedure was an ethical undertaking, provided patient autonomy was appropriately considered. This requires an individual interpretation of the risk–benefit balance, which must include an understanding of the low but unquantifiable risk of severe complications, including bilateral blindness. Consequent cost savings to health care that may occur following the implementation of immediately sequential bilateral cataract surgery may be considered a secondary benefit; the primary benefit should be centred on potential (although not yet well defined) patient convenience factors.

Limitations

Although key ethics issues were identified and analysed, the 29 stakeholders who attended could not have included all of the relevant views. This analysis does, however, provide a reference point for the issues and ethics factors surrounding the practice of immediately sequential bilateral cataract surgery.

The true benefits to patients are, as of yet, ill-defined, as are the potential health-care savings. Similarly, the risks of severe bilateral complications and loss of vision remain unquantified and unquantifiable in the light of their rarity.