The Effectiveness, cost-effectiveness and acceptability of Community versus Hospital Eye Service follow-up for patients with neovascular age-related macular degeneration with quiescent disease (ECHoES): a virtual randomised balanced incomplete block trial

Recruitment

The ECHoES trial was publicised in optometry journals and forums to attract optometrists, and circulated to ophthalmologists who were members of the UK and Welsh medical retinal groups. Potential participants were directed to the ECHoES trial website, where they could read the information sheet and register their interest in the trial.

Eligibility criteria

Participants had to meet the following inclusion/exclusion criteria.

Ophthalmologists:

• have 3 years’ post-registration experience in ophthalmology

• have passed part 1 of the Royal College of Ophthalmologists examination or hold the Diploma in Ophthalmology or equivalent qualification

• working in the NHS at the time of participation in the ECHoES trial

• have experience within an age-related macular degeneration (AMD) service.

Optometrists:

• be fully qualified and registered with the General Optical Council for at least 3 years

• be practising within the General Optical Service at the time of participation in the ECHoES trial

• must not be working within AMD shared care schemes or undertaking OCT interpretation within AMD care pathways.

There were also some practical circumstances in which a potential participant was not accepted to assess the main study vignette set:

• unable to attend any of the webinar training sessions

• unable to achieve an adequate standard (75%) with respect to the assessment of lesion activity status (i.e. reactivated, suspicious or quiescent) on the training set of vignettes.

Training participants

Both ophthalmologists and optometrists are qualified to detect retinal pathology, but optometrists (and some ophthalmologists) may not have the skills to assess fundus and OCT images for reactivation of nAMD. Therefore, the training was designed to provide the key information necessary to perform this task successfully, so that all participants had a similar level of background knowledge when starting their main trial vignette assessments. The training included two parts.

Primary outcome

The primary outcome was correct classification of the activity status of the lesion by a participant, based on assessing the index images in a vignette, compared with the reference standard (see Reference standard). Activity status could be classified as ‘reactivated’, ‘suspicious’ or ‘quiescent’. For the primary outcome, a participant’s classification was scored as ‘correct’ if:

• both the participant and the reference standard lesion classification were reactivated

• both the participant and the reference standard lesion classification were quiescent

• both the participant and the reference standard lesion classification were suspicious

• either the participant or the reference standard lesion classification was suspicious, and the other classification (reference standard or participant) was quiescent.

In effect, for the primary outcome, suspicious and quiescent classifications were grouped, making the primary outcome binary (Table 3).

Secondary outcomes

1. The frequency of potentially sight-threatening ‘serious’ errors. An error of this kind was considered to have occurred when a participant’s classification of a vignette was ‘lesion quiescent’ and the reference standard classification was ‘lesion reactivated’; that is, a definitive false-negative classification by the participant. Definitive false positives were not considered sight-threatening, but were tabulated. Misclassifications involving classifications of ‘lesion suspicious’ were also not considered sight-threatening.

2. Judgements about the presence or absence of specific lesion components, for example blood and exudates in the fundus colour images, SRF, IRC, DRT and PED in the OCT images and, if present, whether or not these features had increased since baseline.

3. Participant-rated confidence in their decisions about the primary outcome, on a 5-point scale.

Adverse events

This study does not involve any risks to the participants; therefore, it was not possible for clinical adverse events to be attributed to study-specific procedures.

Implementation/management/data collection

A secure web-based application was developed to allow participants to take part in the trial remotely (see example screenshots Appendix 2). The website www.echoestrial.org/demo shows how assessors carried out assessments in the trial. Participants registered interest, entered their details, completed questionnaires (regarding opinions on ECHoES trial training and shared care), and assessed their training and main study vignettes through this application. The webinar material was also available in the web application for participants to consult if they needed to revisit it.

Additionally, the web application had tools accessible only to the trial team to help manage and monitor the conduct and progression of the trial. Further details are published elsewhere. 19

Group comparisons

All primary and secondary outcomes were analysed using mixed-effects regression models, adjusting for the order the vignettes were viewed as a fixed effect (tertiles: 1–14, 15–28, 29–42) and participant and vignette as random effects. All outcomes were binary and as such were analysed using logistic regression, with group estimates presented as odds ratios (ORs) with 95% confidence intervals (CIs).

In addition to the group comparisons, the influence of key vignette features [age, gender, smoking status, cardiac history (including angina, myocardial infarction and/or heart disease), and baseline and index BCVA (modelled as the sum and difference of BCVA at the two time points)] on the number of incorrect vignette classifications were investigated, adjusting for the reference standard classification (reactivated vs. quiescent/suspicious). This additional analysis was carried out using fixed-effects Poisson regression with the outcome of the number of incorrect classifications. The prior hypothesis was that this information would not influence the number of correct (or incorrect) classifications.

The sensitivity and specificity of the primary outcome are also presented. For these performance measures, the sensitivity is the proportion of lesions for which the reference standard is ‘reactivated’ and participants correctly classified the lesion. The specificity is the proportion of lesions for which the reference standard is either ‘suspicious’ or ‘quiescent’ and the participant’s classification is also ‘suspicious’ or ‘quiescent’.

Non-inferiority limit

For the sample size calculation, it was agreed that an absolute difference of 10% would be the maximum acceptable difference between the two groups, assuming that ophthalmologists would correctly assess 95% of their vignettes. As the group comparison of the primary outcome was analysed using logistic regression and presented as an OR, this non-inferiority margin was converted to the odds scale. Therefore, this limit was expressed as an OR of 0.298 [i.e. the odds correct for the worst acceptable performance by optometrists (85%) divided by the odds correct for worst assumed performance of ophthalmologists (95%): (0.85/0.15)/(0.95/0.05)].

Sensitivity analysis

A sensitivity analysis of the primary outcome, regrouping vignettes graded as suspicious into the ‘lesion reactivated’ group rather than ‘quiescent lesion’ group, was undertaken to assess the sensitivity of the conclusions to the classification of the vignettes graded as suspicious. This analysis was prespecified in the analysis plan but not in the trial protocol.

Post-hoc analysis

The following post-hoc analyses were prespecified in the analysis plan, but not in the trial protocol.

• Lesion classification decisions were tabulated against referral decisions for both groups.

• A descriptive analysis of the time taken to complete each vignette and how the duration of this time changed with experience in the trial (learning curve) was performed. The relationship between the duration of this time and participants’ ‘success’ in correctly classifying vignettes was also explored.

• Cross-tabulations and kappa statistics were used to compare experts’ initial classifications with the final reference standard. Similarly, cross-tabulations and kappa statistics were used to compare lesion component classifications between the three experts.

In addition, a descriptive analysis of the participants’ opinions about the training provided in the ECHoES trial, as well as their perceptions of shared care, was carried out (see ECHoES participants’ perspectives of training and shared care).

Missing data

By design, there were no missing data for the primary and secondary outcomes. However, time taken to complete vignettes was calculated as the time between when each vignette was saved on the database. It was, therefore, not possible to calculate this for the first vignette of each session. Additionally, it was assumed that times longer than 20 minutes (database timeout time) were due to interruptions, so these were set to missing. As the analysis using these times was descriptive and was not specified in the protocol, complete case analysis was performed and missing/available data described.

Statistical significance

For hypothesis tests, two-tailed p-values of < 0.05 were considered statistically significant. Likelihood ratio tests were used in preference to Wald tests for hypothesis testing.

All data management and all analyses carried out by the Clinical Trials and Evaluation Unit, Bristol, were performed in SAS version 9.3 (SAS Institute Inc., Cary, NC, USA) or Stata version 13.1 (StataCorp LP, College Station, TX, USA).

Changes since commencement of study

At the end of their participation in the trial, all participants were asked to complete a questionnaire regarding their opinions about the training provided in the ECHoES trial and their attitudes towards a shared care model. This was made available to all ECHoES trial participants.

In addition, the reference standard was originally planned to include only two categories, namely lesion reactivated or lesion quiescent. However, after a concordance exercise was carried out by the three retinal experts on the ECHoES trial team, a third category of suspicious lesion was added. This change occurred before any experts’ or participants’ lesion classifications were made.

Finally, after much consideration, it was agreed the primary analysis would be carried out using logistic regression rather than Poisson regression in order to fully account for the incomplete block design. Fitting a Poisson model would not have allowed us to include both vignette and participant as random effects in the model.

Aims and research questions

The economic evaluation component of the ECHoES trial aimed to estimate the incremental cost and incremental cost-effectiveness of community optometrists compared with hospital-based ophthalmologists performing retreatment assessments for patients with quiescent nAMD. This would enable us to determine which professional group represents the best use of scarce NHS resources in this context. The main outcome measure was a cost per correct retreatment decision, with correct meaning that both experts and trial participants judge the vignettes to be lesion reactivated, lesion quiescent or lesion suspicious.

Analysis perspective

The economic evaluation took the cost perspective of the UK NHS, personal social services and private practice optometrists, and was performed in accordance with established guidelines and recent publications. 21–23 Although it is possible that any incorrect retreatment decisions (false positives) could lead to costs being incurred by patients, their families or employers because of time away from usual activities, these wider societal costs were felt to be small compared with the implications for the NHS. Therefore, in line with the IVAN trial9, we decided not to adopt a societal perspective in the ECHoES trial.

Economic evaluation methods

The methods used in the economic evaluation are summarised in Table 4. Data on resource use and costs were collected using bespoke costing questionnaires for community optometrists. With the help of the ECHoES trial team (clinicians and optometrists), we first identified the typical components/tasks which are included in a monitoring review for nAMD and then subcategorized these tasks into resource groups such as staffing, equipment and building space. As some optometrist practices are likely to incur set-up costs associated with assessing the need for retreatment, we compiled a list of items of equipment necessary for performing each task within a monitoring review. The resource-use and cost questionnaire asked each participant which items of equipment from this list their practice currently owned and how much they had paid for the items. The questionnaire was piloted with a handful of optometrists and ophthalmologists who were able to advise on whether or not our questionnaire was straightforward to comprehend and complete. For any items which would be necessary for the review, but which the practice did not already own, we inferred that these items would have to be purchased ‘ex novo’. Once the total costs of equipment were identified, along with their predicted life span as suggested by study clinicians, we created an equivalent annual cost for all of the equipment using a 3.5% discount rate,24 and divided this cost by the number of potential patients who could undergo the monitoring reviews in the community practices. See Appendix 3 for a copy of the bespoke costing questionnaires for the optometrists in the ECHoES trial.

We estimated the costs associated with training optometrists to perform the assessments. The costs were estimated for the 2-hour duration webinar, plus time spent to revisit the webinar and time spent in checking other resources. Finally, the amount of time the ECHoES study clinicians spent preparing and delivering the webinars was also estimated.

For information on the costs of ophthalmologists performing the monitoring assessments, we used cost data from the IVAN trial, in which we had undertaken a very detailed micro-costing study. 25

In the economic evaluation, it is advised that value-added tax (VAT) should be excluded from the analysis. 21 Where possible, we used costs without VAT for our base-case analysis.

Care cost pathway decision tree

In order to generate an estimate for a cost for a correct retreatment decision, we first mapped the various care pathways that would possibly happen from the treatment assessments in the study, by comparing the reference lesion classification for each vignette to the actual classification made by the study participants (48 ophthalmologists and 48 optometrists). This gave rise to two versions of a simple decision tree: one for ophthalmologists (Figure 1) and one for optometrists (Figure 2). The starting point for the decision trees was the ‘actual’ truth. The three main branches of the trees were true active, true quiescent and true suspicious. Participants’ main trial assessments provided information about the number of participants who made correct and incorrect decisions on treatment compared with the reference standard decisions.

FIGURE 1.

Decision tree for hospital ophthalmologist review. Reproduced open access from Violato et al. 16 under the terms of the Creative Commons Attribution (CC BY 4.0) licence.

FIGURE 2.

Decision tree for community optometrist review. Reproduced open access from Violato et al. 16 under the terms of the Creative Commons Attribution (CC BY 4.0) licence.

The decisions that both groups made about the vignettes were then placed into the decision trees and the associated costs for the different pathways were calculated. This process generated an average cost for each of the alternative care pathways. Any ‘incorrect’ decisions implied that patients would have had unnecessary repeat monitoring visits at the optometrist practice or at the hospital and maybe unnecessary anti-VEGF injections if they misclassified patients as requiring treatment. All costs are reported in 2013/14 prices unless specified otherwise.

Our baseline analysis calculated the average cost and outcome on a per patient basis. From these estimates, the incremental cost-effectiveness ratios (ICERs) for the different assessment options were derived, producing an incremental cost per accurate retreatment decision. Sensitivity analyses were carried out to demonstrate the impact of variation around the key parameters in the analysis on the baseline cost-effectiveness results. The four sensitivity analyses which we conducted were:

• Sensitivity analysis 1: all patients initiating treatment were assumed to have a course of three ranibizumab given at three subsequent injection consultations, with no additional monitoring reviews. This matched the way in which discontinuous treatment was administered in the IVAN trial.

• Sensitivity analysis 2: treatment was assumed to be one aflibercept injection given during an injection consultation.

• Sensitivity analysis 3: treatment consisted of one bevacizumab injection given during an injection consultation.

• Sensitivity analysis 4: only considered the cost of a monitoring review rather than considering the cost of the whole pathway.

The economic evaluation results were expressed in terms of a cost-effectiveness acceptability curve (CEAC). This indicates the likelihood that the results fall below a given cost-effectiveness ceiling and could help decision makers to assess whether optometrists are likely to represent value for money for the NHS when compared with ophthalmologists in making decisions about retreatment in nAMD patients and also completely disparate health-care interventions. All health economics analyses were conducted in Stata version 12.1 (StataCorp LP, College Station, TX, USA), except the budget impact analysis.

Budget impact

Freeing up HES clinic time could lead to an increase in the overall capacity of the NHS both to manage the nAMD population more effectively and to manage non-nAMD eye patients (more time for ophthalmologists to spend with non-nAMD patients if they are seeing fewer nAMD patients for monitoring). Therefore, we attempted to estimate the potential time and costs that could be saved by HES clinics if some of the management of nAMD patients could be undertaken in the community. This was done by using the resource-use information collected during the IVAN trial as a basis on which to consider what proportion of an ophthalmologist’s time is spent on retreatment decisions for this group of patients relative to other aspects of their care. We brought together data from the IVAN trial (average number of patients attending a clinic) and information from the literature, and used expert opinion from the ECHoES study ophthalmologists and optometrists to try to estimate the total number of patients with quiescent or no lesion in both eyes who would be eligible for monitoring by a community optometrist in a given month and the total number of monitoring visits that could be transferred from a hospital to the community per year. We then replaced the costs of the ophthalmologist’s time with the cost of the optometrist’s time and examined the difference. The Microsoft Excel^® spreadsheet (Microsoft Corporation, Redmond, WA, USA) calculations for the budget impact are reported in Appendix 3.

Recruitment

Sampling

Basic demographic, health and professional information was collected from those who agreed to be contacted for sampling purposes. A purposeful sampling strategy was used to ensure that the feasibility and acceptability of the proposed shared model of care for nAMD was captured from a range of perspectives. Within this sampling approach, maximum variation was sought in relation to profession, age, gender and geographic location (for health professionals) and gender, age, type of AMD and time since diagnosis (for service users). Participant characteristics were assessed as the study progressed, and individuals or groups that were under-represented were targeted (i.e. commissioners or clinical advisors). Where it was felt that variation had been achieved, potential participants were thanked for their interest and informed that sufficient numbers had been recruited (i.e. optometrists).

Data collection

A favourable ethical approval for this study was granted by a UK NHS Research Ethics Committee. Written consent was obtained from participants at the start of each focus group or interview. Separate topic guides were developed for service users, optometrists/ophthalmologists (with additional questions for each professional group) and all other health professionals, to ensure that discussions within each group covered the same basic issues but with sufficient flexibility to allow new issues of importance to the informants to emerge. These were based on the study aims, relevant literature and feedback from eye-health professionals in the ECHoES study team, and consisted of open-ended questions about the current model of care for nAMD and perspectives of stable patients being monitored in the community by optometrists. They were adapted as analysis progressed to enable exploration of emerging themes.

All of the focus groups and interviews were conducted by DT; these were predominantly led by the participants themselves, with DT flexibly guiding the discussion by occasionally probing for more information, clarifying any ambiguous statements, encouraging the discussion to stay on track, and, in the focus groups, providing an opportunity for all participants to contribute to the discussion. All participants were offered a £20 gift voucher to thank them for their time (two commissioners declined the voucher).

Data analysis

Focus groups and interviews were transcribed verbatim and checked against the audio-recording for accuracy. Transcripts were imported into NVivo version 10 (QSR International, Warrington, UK), where data were systematically assigned codes and analysed thematically using constant comparison methods derived from grounded theory methodology. 26 Transcripts were repeatedly read and ongoing potential ideas and coding schemes were noted at every stage of analysis. To ensure each transcript was given equal attention in the coding process, data were analysed sentence by sentence and interesting features were coded. Clusters of related codes were then organised into potential themes, and emerging themes and codes within transcripts and across the data set were then compared with explore shared or disparate views among participants. The transcripts were reread to ensure the proposed themes adequately covered the coded extracts, and the themes were refined accordingly. Emerging themes were discussed with a second experienced social scientist (NM) with reference to the raw data. Data collection and analysis proceeded in parallel, with emerging findings informing further sampling and data collection. Data collection and analysis continued until the point of data saturation, that is, the point at which no new themes emerged.

The ECHoES trial participants’ perspectives of training and shared care

In addition to the information collected during focus groups and interviews, the opinions of all optometrists and ophthalmologists who had taken part in the ECHoES trial were sought in a short online questionnaire. The survey related to participants’ experiences of the training and their attitudes towards shared care for nAMD. Questions required a binary, a Likert scale or a free-text response.

Quantitative data were analysed in Microsoft Excel and presented as proportions. Free-text responses were imported into NVivo and coded into categories, which were derived from the main survey questions, relating to feedback on the training programme (experiences of using the web application, ease of training and additional resources used) and attitudes towards shared care (general perspectives of the proposed model, and perceived facilitators and barriers to implementation). Data were analysed thematically using the constant comparative techniques of grounded theory, whereby codes within and across the data set were compared to look for shared or disparate views among optometrists and ophthalmologists.

Serious sight-threatening errors

Serious sight-threatening errors could occur only for the vignettes which were classified as ‘reactivated’ by the reference standard. These errors occurred in 62 out of 994 (6.2%) of ophthalmologists’ classifications and 57 out of 994 (5.7%) of optometrists’ classifications; this difference was not statistically significant (OR 0.93, 95% CI 0.55 to 1.57; p = 0.789).

Each participant viewed between 15 and 27 ‘reactivated’ vignettes and the most sight-threatening errors made by a single participant was eight (out of 25) in the ophthalmologists group and five (out of 19) in the optometrists group (Table 8). Table 8 also shows the number of non-sight-threatening errors, that is, participants classifying vignettes as reactivated when the reference standard was quiescent. This type of error occurred more frequently in the optometrists group (105 out of 987; 10.6%) than in the ophthalmologists group (35 out of 987; 3.5%); this difference was not formally compared.

Lesion components

We did not attempt to achieve a consensus among the three experts for the individual features of the vignettes. Therefore, responses given by the professional groups for these individual features were formally compared with each other rather than with any reference standard. Optometrists judged lesion components to be present for all components except PED more often than ophthalmologists (Table 9 and Figure 6). This difference was particularly evident for DRT and exudates; the odds of identifying these components as present were more than three times higher in the optometrist group than in ophthalmologist group [OR 3.46, 95% CI 2.09 to 5.71 (p < 0.001), and OR 3.10, 95% CI 1.58 to 6.08 (p < 0.001), respectively]. SRF was also reported significantly more often by optometrists than ophthalmologists (OR 1.73, 95% CI 1.21 to 2.48; p = 0.002). The difference between the groups was of borderline statistical significance for blood (OR 1.56, 95% CI 1.00 to 2.44; p = 0.048). The differences between the groups for IRC and PED did not differ statistically [OR 1.00, 95% CI 0.61 to 1.65 (p = 0.985) and OR 0.91, 95% CI 0.47 to 1.79 (p = 0.786), respectively].

FIGURE 6.

Lesion components. Point estimates for different lesion components are ORs and error bars are 95% CIs. The vertical dashed line illustrates line of no difference at 1.

Confidence ratings

The confidence ratings displayed in Table 10 show that ophthalmologists were clearly more confident in their decisions than optometrists. Ophthalmologists stated that they were very confident (5 on the rating scale) in their judgements for 1175 out of 2016 (58.3%) vignettes, whereas optometrists reporting the same level of confidence in their judgements for only 575 out of 2016 (28.5%) vignettes (OR 0.15, 95% CI 0.07 to 0.32; p < 0.001). For both groups, a confidence rating of 5 resulted in a correct answer over 90% of the time, but there did not appear to be any clear relationship between confidence and correctness for lower confident ratings, especially for optometrists (see Table 10).

Vignette classifications compared with referral recommendations

It was of interest to see how lesion classification decisions related to referral decisions. These should be congruent (as referral decisions should be made based on classification decisions) but no such rules were imposed in this trial. Table 11 illustrates that, as expected, lesions classified as reactivated were usually paired with a recommendation for rapid referral to hospital (1679/1682, 99.9%) and lesions classified as quiescent were usually paired with a recommendation for review in 4 weeks (1526/1604, 95.1%). There was very little difference between the two professional groups in the extent to which lesion classifications and referral decisions were paired. Lesions classified as suspicious were often paired with a recommendation for review in 2 weeks (605/746, 81.1%), but rapid referral to hospital was not uncommon (131/746, 17.6%). Optometrists more often than ophthalmologists tended to pair suspicious classifications with review in 2 weeks (87.3% vs. 74.4%), whereas ophthalmologists tended to pair suspicious classifications with rapid referral to hospital more often than optometrists (24.8% vs. 10.9%).

Duration of vignette assessment

The assessment durations were calculated for each participant as the difference between the time an assessment was saved and the time the previous assessment was saved. Therefore, this information was not available for all 4032 assessments because many participants took breaks between assessments (see Chapter 2, Missing data). The median durations of vignette assessment for all main study assessments were 2 minutes 21 seconds (IQR 1 minute 39 seconds to 3 minutes 27 seconds; n = 1835) for ophthalmologists and 3 minutes 2 seconds (IQR 2 minutes 5 seconds to 4 minutes 57 seconds; n = 1758) for optometrists. Assessment time, on average, reduced as experience increased, especially in the optometrists group; for ophthalmologists and optometrists respectively, the median assessment durations were 2 minutes 44 seconds (IQR 2 minutes 3 seconds to 4 minutes 40 seconds; n = 45) and 5 minutes 23 seconds (IQR 3 minutes 18 seconds to 10 minutes 3 seconds; n = 41) for the second main study vignette, and 1 minute 55 seconds (IQR 1 minute 19 seconds to 2 minutes 49 seconds; n = 48) and 2 minutes 31 seconds (IQR 1 minute 52 seconds to 4 minutes 26 seconds; n = 48) for the 42nd (final) main study vignette. Figure 8 shows this relationship.

FIGURE 8.

Duration of vignette assessment. Point estimates show median assessment durations and error bars show IQRs.

Among vignette assessments for which duration was not missing, there were 1570 out of 1835 (85.6%) correct responses by ophthalmologists and 1492 out of 1758 (84.9%) correct responses by optometrists. Figure 9 illustrates the relationship between assessment duration and the percentage of correct responses; broadly speaking, shorter assessment durations were more likely to result in correct lesion assessments than longer assessment durations. This relationship was similar for both professional groups.

FIGURE 9.

Correct responses by vignette assessment duration. Point estimates show the percentage of correct assessments and errors bars show exact 95% CIs.

Expert classifications for derivation of the reference standard

Each expert individually assessed the vignettes in order to develop the reference standard (see Chapter 2, Reference standard). The lesion classifications of the three experts were congruent for these assessments for 219 out of 288 (76.0%) vignettes (comprising 103 out of 219 classifications of reactivated and 116 out of 219 classifications of quiescent). The three experts then held a consensus meeting and jointly assessed the vignettes for which there was disagreement about the lesion classification. The presence or absence of lesion components in index images of these vignettes, and their change from baseline, were discussed in detail. Consensus lesion classifications were agreed which, together with the congruent classifications, made up the final reference standard. Experts did not attempt to reach consensus about specific lesion components. A few specific vignettes were reassessed by experts who felt that they had made errors about the lesion components in their original assessments. When carrying out these reassessments, data were collected only for assessments of the lesion components.

Table 12 shows the reference standard classification against each expert’s initial individual classification; 774 out of 864 (89.6%) of the individual classifications remained unchanged, with the rest (10.4%) being amended after group discussions. The responses that did not change are shaded green. The majority of the changed classifications involved an initial classification of suspicious by one expert.

Comparing lesion component classifications across experts

Although a reference standard was established for the overall lesion classification, no such standard was established for the lesion component classifications. Therefore, it was of interest to compare lesion component classifications across experts for the six lesion components. The experts could classify each lesion component as absent, present but not increased since baseline, or present and increased since baseline. The frequencies with which each expert identified different lesion components across all vignettes are shown in Table 13. Agreement between experts with respect to lesion components is shown in Table 14. Table 14 shows that agreement was best for blood and exudates; the three experts agreed about blood classification for 266 out of 288 (92.4%) vignettes and for exudate classification for 260 out of 288 (90.3%) vignettes. Table 13 shows that experts observed no blood or exudates on the about colour images of > 90% of the vignettes. Agreement for SRF and IRC was also high, with total agreement for 259 out of 288 (89.9%) and 243 out of 288 (84.4%) of the vignettes, respectively (see Table 14). Interestingly, for both these components, expert 2 disagreed with the other two experts slightly more often than experts 1 and 3 did with each other; Table 13 shows that expert 2 identified SRF to be present more often than experts 1 and 3, and identified IRC to be present less often than experts 1 and 3. Agreement between experts was lower for DRT and PED than for the other components, suggesting that classification of these components was potentially more difficult or less clear-cut. In addition, Table 14 shows the agreement between the experts about the overall lesion classification, prior to any discussions to agree the reference standard.

Health professionals

The focus groups lasted a mean of 110 minutes, ranging from 99 to 121 minutes. The interviews lasted a mean of 48 minutes, ranging from 31 to 61 minutes. In total, 24 health professionals were recruited. This comprised eight optometrists, six ophthalmologists, two public health representatives, six NHS commissioners and two clinical eye care advisors to their local CCGs. Of these, 12 were women and 12 were men. Participants had a mean age of 39 years (range 31–64 years) and stated that they had been in their profession for a mean of 21 years (range 4–40 years). Although none had participated in shared care for nAMD, 11 participants had experience of shared care schemes for other conditions such as diabetic retinopathy screening and ocular hypertension monitoring. Table 27 shows participants’ professional background. Years in profession and location are not presented for the ‘other’ health professionals (i.e. those other than ophthalmologists and optometrists) to protect their anonymity given their unique roles. However, these participants’ locations were distributed evenly throughout England and they had been in their profession for an average of 20 years (range 8–37 years).

Service users

The focus groups lasted a mean of 71 minutes, ranging from 65 to 90 minutes. Three focus groups were conducted with 23 participants in total, with seven or eight participants in each group. The sample consisted of 15 women (65%) and eight men (35%) who described themselves as white British. The sampling strategy intended to recruit individuals from a mix of ethnicities, but all those at the supporting groups who were willing to be contacted were white British. They had a mean age of 82 years (range 72–93 years). All had nAMD and attended the same eye hospital in a major city and were diagnosed an average of 5.9 years ago (range 6 months to 20 years). Nine participants had active nAMD in one eye (39%), nine were stable in one eye (39%), four people had active disease in both eyes (18%), and one person was stable in both eyes (4%). Eight participants had dry AMD in their other eye (35%). The partners of two participants (who were also their carers) joined the focus group but spoke very little; their comments were not included in the final analysis. Table 28 provides participants’ demographic and health-related details.

Current clinic capacity: pushed to the limit

Many health professionals stated that the number of repeat visits for patients was rising exponentially, which was attributed to an increase in the number of patients who were being diagnosed with nAMD and to new government guidelines for treatment.

The numbers going through the system are higher, and they need treatment for longer.

Comm3

Hospital clinics were felt to be ‘pushed to their limit’ [Optom7, Ophthalm3] and the ophthalmologists felt frustrated that their time was mostly spent on stable patients who did not require treatment.

You are taking the potential time from the ones that actually need the care. There is a limit of work that you can do. I mean you can’t go home at 8 every day.

Ophthalm2

Both the health professionals and service users described how patients would often have to wait for long periods of time for their appointment because of how busy the eye hospital clinics were:

Well, quite honestly when you go to the eye hospital, it always seems to be packed out left, right and centre.

Harry

Sometimes we have to wait a long time, but you know it can’t be helped.

Henry

It’s just generally the issue of they have had to wait a long time [. . .] You’ve got to think also there are diabetics among there that have to regulate their meals, and they have their set routine in terms of their meals and health. It’s the same with everybody, but perhaps diabetics most, because it affects them straight away. I’ve known diabetics that have gone into a hypo [hypoglycaemia] because they were waiting around. Because they had a 9 ‘clock appointment and it got to 11. It’s just a real shame.

PH2

There was a sense that the current model would inevitably need to adapt to cope with this demand and agreement that optometrists monitoring in the community had the potential to reduce clinical workload.

It will help shift a lot of the workload out of the hospital environment where they are overrun with this and putting it into a more capable environment with local optometrists.

Optom6

[Monitoring in the community] will make less queues at the hospital. At the moment they’re choc à bloc with people.

Ruth

The optometrists, clinical advisors and commissioners also described how a shared care scheme represented a great opportunity to enhance optometrists’ professional roles by developing their skills.

It’s fantastic for the optometry profession, because it must give them much more exciting and interesting careers, and career progression, and variety within their work.

Comm3

Potential for a more patient-centred model

Most health professionals also felt that the current model of care was not appropriate for older patients with limited vision who had to regularly travel to the hospital.

The problem is that all the patients have got to go to the eye hospital all the time, which isn’t very patient-centric . . . It’s not very easy for people to get in once a month – which is obviously what Lucentis is about – for their assessment. Given that they are, almost by definition, elderly and with poor vision, it’s not an ideal centre for it.

Comm2

The service users found it stressful travelling to and from the hospital for care. Many stated that they were unable to drive because of their nAMD, which was described as a ‘massive blow’ to their independence (Pat). These service users reported difficulty seeing the bus numbers, and most needed to get multiple buses each way as there were no direct buses. Others were driven by family or friends, but described parking as ‘awful’ (Arthur).

It’s not just getting to the hospital. It’s all that time afterwards, if you’ve got to get the bus, it’s – in the winter, it’s even worse.

Ralph

One of the things that’s come out here is that everyone is, obviously, getting older. They’re stressed when they have to go out of the town because getting home when you’ve got . . .

Robert

Oh, it’s terrible.

Mandy

So if they have someone in the town who is an optician and deals with us, it’s only a short distance from home.

Robert

Monitoring in the community was described as a ‘wonderful’ idea (Elizabeth), particularly for those who lived further away from the hospital or older participants who had severe vision loss.

For me, living out of town in [small town], to get to an optician on the bus is easy, whereas it’s a day’s expedition to come into [city].

Tracey

Many rarely saw the same consultant or nurse at the hospital, and felt that staff were often impersonal as they were ‘so busy’. This was likened to ‘being on a conveyer belt’ (Pam).

If a doctor said, ‘Well, that’s alright’, that’s it. It’s reassurance. I think they’re trying to speed up time, and I know they’re very busy and they obviously look at the photographs and they can see everything, but for patients’ feel-good interest, I always like a doctor or someone to say, ‘That’s alright. You’re not doing too badly. Well, you’re in your eighties now.’ Just to talk to you properly.

George

I think one of the greatest things wrong in the eye hospital is they . . . they kick you out the door.

Arthur

In addition, most others felt that they did not receive enough information about the status of their condition.

Just be told what’s happening –

Harriett

The only criticism I would have is to try to find out how you’re doing and whether you’re getting worse or getting better, or stable, because they’re all so busy.

Ruth

These participants were therefore enthusiastic about the potential for continuity of care, which they hoped would enable them to build up a relationship with their optometrist.

I think one of the things, I think you’ll agree, has come to light this morning, is basically that many aspects of the eye hospital, it’s so impersonal. I think that probably a system like you’re suggesting would probably add a personal touch to it and a more one-on-one situation . . . That’s the big thing, seeing the same person. Like I said, the personal touch . . . The relationship would build up.

Edward

Perceptions of optometrists’ competency

The optometrists in the focus groups, who acknowledged that they had a special interest in nAMD, were very positive about the possibility of shared care and felt that their profession was more than capable of monitoring in the community. This was also echoed by the commissioners, clinical advisors and public health representatives.

They’re [optometrists] really incredible, impressive professionals, with just a huge amount of experience at looking at eyes.

Comm3

However, several health professionals (from mixed professions) commented that ophthalmologists would resist shared care as they were not convinced of optometrists’ competence.

I think it’s the misconception that optometrists won’t do as good a job as secondary care. So I think that’ll be the biggest barrier.

Comm4

This was considered to be problematic for shared care as there was uncertainty whether or not ophthalmologists would ever truly relinquish responsibility for patients.

I would not want to close the door on them, ever. I’d still want them to contact me if they noticed any change.

Ophthalm1

Honestly, I think that clinicians aren’t always very good at letting go [. . .] It will be an issue.

Comm3

There was hesitation among ophthalmologists about whether or not optometrists were capable of monitoring nAMD. The ophthalmologists referred to how they frequently received incorrect referrals from optometrists. Furthermore, they highlighted the ways in which nAMD differed to other eye diseases where shared care schemes existed.

When you work in ophthalmology for quite some time you see just the amount of work that comes to you from inappropriate referrals. Really they’re just doubling work up.

Ophthalm3

This is not like glaucoma where you notice pressure or you don’t feel OK and refer back to the hospital. This is something . . . it’s based on the scan and each patient is different. There’s only a few parameters for glaucoma, whereas here there are . . . it’s complex [. . .] So we can’t expect an optometrist to . . . [laughter].

Ophthalm5

The ophthalmologists felt that the hospital provided an environment where they had access to all previous scans and other colleagues’ expertise, which enabled them to confidently make a clinical judgement. They expressed uncertainty about whether or not optometrists would have these resources. Ultimately, owing to this perceived complexity of assessing the need for retreatment and the support and resources available in the hospital, the ophthalmologists felt that monitoring patients in the community would be a compromise.

So what we are trying to say is hospital care is the best [laughter].

Ophthalm6

Several ophthalmologists felt that patients would prefer to remain being monitored by a consultant at the hospital, whom they would inherently trust. In line with this, service users with active nAMD also tended to be apprehensive about the level of optometrist competence in the community, and commented that lengthy waiting times were secondary to receiving the best care for their condition.

If we put ourselves in their position what would we prefer to have? I’ll prefer to be seen by a doctor in a hospital.

Ophthalm1

So, you’ve got to have confidence in the person that is monitoring you [. . .] I feel that to rely on somebody that has been trained up to identify problems can’t really be as efficient as seeing an actual doctor who specialises in that subject, and because of that, I wouldn’t be happy going to an optician. It might take you longer. We have sat up there for hours, but the end result is well worth it.

Henry

The majority of service users described needing to be able to ‘have faith’ in their optometrist if they were to participate in shared care. Those whose optometrist had diagnosed their nAMD commented that this gave them a sense of confidence in their optometrist’s abilities.

I would trust my optician. He really seems to care. I would trust him. If my wet macular was stable, I’d be very happy to go to my optician because I’ve got confidence in him, because he detected it in the beginning.

Harriett

However, a few service users were apprehensive about shared care and did not wholly trust the idea of monitoring by an optometrist. Those who expressed this sentiment did not have faith in their optometrist because the optometrist had not recognised the condition initially

Personally I wouldn’t have faith in the optometrist. I would much prefer to stay with the hospital.

Henry

(Lack of) communication between optometrists and ophthalmologists

Overall, the health professionals described the relationship between optometry and ophthalmology as poor.

Collaboration optometry and ophthalmology? No way. It’s absolutely dreadful!’ [laughter]

Optom8

These participants described how ‘systems-based’ issues made communication between the two professions poor. This was described by all professions except the ophthalmologists. For instance, most of the optometrists described how it could be extremely difficult to relay information to ophthalmologists because of incompatible computer e-mail systems and variation in technology.

Our problem is we’ve got NHS.net in optometry and because everyone else, they’re all NHS.co.uk. So their end isn’t secure. We can do NHS.net to NHS.net, but we can’t do NHS.net to NHS.co.uk, which is what all hospitals give their consultants. It’s absolutely crazy.

Optom8

They’ve not embraced NHS.net at all.

Optom5

Some optometric practices don’t even have computers. Particularly in the [city] area where many of them are way behind. We actually had to buy them fax machines when this started to make it work. You’d expect most people had that sort of facility, but they didn’t. To make it work, we would do that. So I think standardisation of forms across our units, across the country, and making those forms readily available, and everybody knows that they’ve got to look for the red-topped form in the practice or whatever, could possibly aid this model of shared care.

Optom2

The other big issue is transfer. If you’re actually going to transfer the data, they are massive, massive, massive files. I was talking to an optometrist who has an OCT and sends scans to an ophthalmologist, and he literally has to do it overnight. Just for one patient, it takes so long.

Optom1

All of the optometrists also commented that technology had caused issues with their referrals to ophthalmologists in the past, in that they often got lost between the two professions. As a result, the optometrists stated that they would follow up each referral by calling the consultant to ensure it had been received or they sent referrals via multiple technology methods to ensure that one would reach the consultant.

It does happen that patients will wander back in 2 weeks later and say, ‘Oh, you told me I’d be seen within a couple of weeks. I haven’t heard a thing.’ Then we contact the hospital and we think, ‘OK, what’s happening?’.

Optom2

However, many participants (both health professionals and service users) explained that one of the key concerns of nAMD was the rapid progression of the disease if the condition became active. In particular, several service users described the devastating consequences of their vision deteriorating within days.

Last week I was all-seeing and driving and everything, and on Thursday I thought, ‘There’s something missing on that signpost’, and the glare was terrible. So then I couldn’t read the paper at lunchtime . . . To me, it’s been a disastrous week. I can no longer drive. I can no longer read. This is a week! To me, that’s a disaster. It’s very frightening.

Pat

Participants, particularly the ophthalmologists and service users, therefore expressed concerns about a potential delay between primary and secondary care if retreatment was required.

The problem is that the more steps you have in the system, the longer it takes. We don’t have the time.

Edward

The nature of the disease is such that it needs urgent attention. So it’s better if we see the patient [. . .] I can’t imagine going to the optometrist and him making a decision to refer the patient back, all that delay.

Ophthalm1

The health professionals emphasised that, in order for shared care to be delivered successfully, it would be essential that the two health-care professions were able to work collaboratively so that an efficient pathway could be developed.

I think they would like to make sure that there’s a seamless process between community and hospital, and that nothing drops through the cracks. So I think they would need to make sure that there’s a robust recall service, and that if there is an issue, that there’s a pathway back for the patient into secondary care. So those kinds of things, I think, would need to be ironed out before it goes ahead.

Comm2

The cost of shared care

A theme unique to the health professionals was the financial implications of implementing a shared care model. Both the optometrists and the ophthalmologists believed that financial considerations should be secondary to the care of the patients, although the other health professionals described how a harsh reality of health care meant that shared care would not be commissioned unless it ‘got the most out of the NHS pound’ (PH2).

I think you have to possibly take out the finance issues. How you buy something should be secondary to what the patient needs.

Optom4

You’ve got to show the CCGs that you’re saving money over sending them into the hospital. Because otherwise, they’re just not going to commission it.

PH1

The commissioner participants undertook several roles professionally and were often optometrists or GPs alongside this position. The multiple perspectives from a commissioning and clinical point of view appeared to conflict in terms of patient outcomes and financial efficiency when contemplating nAMD care.

Patients would like it [being monitored in the community] because it’s much closer to home, they don’t have to go to hospital. They don’t have to sit and queue and wait in pain, park and all that sort of stuff that patients usually tell you . . . From a commissioner’s perspective. . . in terms of saving and shifting costs across the health system for eye care services, it certainly doesn’t achieve that.

Comm4

The health professionals were divided about whether or not monitoring in the community would represent a more cost-efficient model. For instance, some perceived that NHS costs of managing sight loss could ultimately decrease.

So if you could use the monitoring to stop the wet AMD getting worse, so that it’s kind of preventative, then you would be addressing the public health indicator of dropping those numbers of people registered with AMD sight loss. [. . .] Yes, that would be the biggest advantage that I would see of it.

PH2

Many also considered that there may be an opportunity to save money with a differential fee for optometrists and ophthalmologists. Conversely, the ophthalmologists stated that the money saved from following up with patients with nAMD could be set towards resources to improve secondary care resources, rather than being ‘lost’ to the community (Ophthalm3).

It would be a cost-efficient option for the commissioners because we would be paying something like £60 for an optometrist to measure the patient’s visual acuity, rather than £100 and whatever it is for a consultant outpatient episode.

Comm4

If the optometrist where I am gets compensated, that pot of money comes from the resources from the hospital and we’re struggling as it is. We’re diverting resources to somebody else who’s not doing as much work as we are doing in the hospital, just doesn’t make sense.

Ophthalm3

The ophthalmologists stated that they took around 15 minutes to see a patient and determine the need for retreatment, while the optometrists’ estimates varied at 20, 30 or 40 minutes. As the requirements to determine reactivation were listed (including a clinical examination, administration of tropicamide drops and use of an OCT), the optometrists concluded the estimate of up to 40 minutes was more realistic to make a clinical decision and explain the results to the patient.

It has to be realistic so you can practise and sustain it.

Optom3

I’ll allow 30 and probably spend 40 [. . .] But once you’ve got an OCT sitting there and you start looking at it, and you really want to explain it to the patient and they say, ‘Oh, thank you. No one ever tells me anything like that at the hospital. It makes you feel so good at the end. You’ve actually told the patient what’s really wrong and, ‘Rest assured my dear, it’s not getting any worse, so we don’t need to send you back’. ‘Oh, thank you!’ It probably would take me 40 minutes.

Optom8

Several commissioners and one clinical advisor therefore highlighted a potential conflict between practices’ clinical and commercial interests and stated that optometrists would need to be paid a sufficient amount to ensure that shared care was economically possible.

If the current business model for eye care is that funding comes from selling glasses, then if you fill your clinic appointments up with OCTs which will be a lovely service for patients, but if the only thing that’s happening is that you are breaking even with doing an OCT, your practice isn’t going to survive.

Comm1

There was also agreement that an OCT, although considered essential for monitoring nAMD, is an expensive piece of kit that not all practices can invest in. The clinical advisors and public health representatives felt that CCGs should provide OCTs, although most of the commissioners and the ophthalmologists felt it should be self-funded to demonstrate a level of commitment to monitoring in the community or that those practices which already had an OCT would represent a more enthusiastic, clinically orientated team.

Well, it’s on the optometrists, really. Whether they’ve got the kit or whether they will want to invest in one. I think what it will be is that you’ve have a small group of practices within a particular area, who will show keenness. [. . .] They will be the more cutting edge practices.

Comm6

I think CCGs should pay for that [an OCT], the NHS. I don’t think it should be the optometrist.

PH2

A further financial issue was whether or not ophthalmologists would repeat tests, owing to difficulty relinquishing responsibility or not trusting an optometrist’s judgement.

If you’re doing injecting from a message that you’ve got from the community, then actually if something was to go wrong, you can’t just say, ‘I did it because the optician outside told me to do it.’ Do you know what I mean? [. . .] For their peace of mind.

CA1

You’d have to attend another OCT in the hospital, look at the scans and then we have to take everything, yes, you may need another angiogram, just to be certain.

Ophthalm1

Consequently, a main concern of the commissioners was that the CCGs might be charged for repeated testing.

‘From a commissioner’s point of view, we would want to make sure then we would only be being charged for that element of it, and they wouldn’t go on and repeat all the tests again.

Comm1

The importance of specialist training

The health professionals spent a considerable amount of time addressing how they felt training should be delivered, in terms of which method was most effective for learning and ensuring training was delivered in a way to reassure ophthalmologists that optometrists were being trained to a high standard.

I think the training needs to be very carefully designed.

PH2

The ophthalmologists have to believe in the competence of the optometrist. It’s important that they have belief in the quality of the accreditation.

Optom4

Virtual training was deemed appropriate for providing a ‘foundation’ level of knowledge, although most felt that this alone would not be sufficient to train the optometrists. In particular, the ophthalmologists were unconvinced by the applicability of a virtual trial.

I think it complements training, but I don’t think it should be the sole course of training. I think you still need a bit of hands-on.

Comm1

I’m sure in your studies you will find 100% virtual case studies that you give, that you will find good correlation between what the ophthalmologist would say and what the optician would say in a virtual case. You can’t just say ‘Yes, the optom [optometrist] has 100% exactly the same as the ophthalmologist, which says that now they are just as good’.

Ophthalm3

Clinical experience, whereby optometrists would gain experience of monitoring nAMD patients in a hospital setting, was viewed as an ‘essential’ component of training.

I think they need to come and see the real patient. There’s no point on sitting on the MediSoft or whatever and clicking boxes and thinking they know it all. Real life is not like that. I mean there are the OCT scans like that, sometimes they can be very devious and very challenging and very confusing . . .

Ophthalm2

The majority of participants felt that having ophthalmologists supervising this training would provide assurance of optometrists’ competence and enable greater collaboration between the two professions. However, they acknowledged that this would be time-consuming.

I would suggest that they [optometrists] would spend a certain amount of time in a consultant clinic. Most services will have a specialist clinic for macular degeneration, so they would maybe spend a session a week, or something, in such a clinic for 6 months, just getting familiar with the treatments and the monitoring, etc. Also that will help the consultants’ gain a bit of confidence in the optom [optometrist] as well. So it’s a working partnership going on there.

Comm2

I think they would be involved if they had an element of control over it. If they’d done the training and if they knew who they were sending the patients out to and they knew what protocols were being followed, the service spec. You don’t get these things to work unless you’ve got clinical buy in. You just don’t. You can set up all you want, but you’ve got to get the clinicians involved.

Optom5

The hospital departments are caving in and creaking under the weight . . . so getting a consultant to be doing local training can be difficult.

CA2

Recruitment

The staged nature of the trial and the risk of withdrawals (most, but not all, participants progressing from training to the main trial or withdrawing for other reasons) combined with the priority of maintaining the balanced incomplete blocks design made recruitment challenging, particularly given the short duration of the trial. The balanced incomplete blocks design meant that each participant had to be assigned a specified set of vignettes for the main trial assessment on completion of webinar training. This constraint prevented an extra participant from being recruited until an existing participant had definitively withdrawn. As participants could not all be recruited at the same time and progressed at different speeds, we did not know until a considerable time had elapsed whether or not additional participants would be needed. In order to expedite completion of the trial, we reopened recruitment at a later stage and deliberately over-recruited participants to the webinar training, some of whom did not progress into the main trial because the target sample size had been reached. Close to the end of the main trial, in order to increase the likelihood of completing the main trial in accordance with our revised schedule, we also assigned two participants to one set of vignettes in order to be more confident that at least one would complete the assessments promptly.

The need to over-recruit also had financial consequences for the trial budget. Participants who chose to withdraw during their participation in the trial were paid only for the time spent on webinar training and not for any training or main vignette assessments that had been carried out up to the time of withdrawal. However, those participants who were withdrawn by the trial team, either because of failure at the training stage or because the trial had reached its target, were reimbursed fully up to the point of withdrawal, as if they had completed the study.

Images used to create vignettes and constraints on viewing images

Despite the overall size of the IVAN trial image repository, the number of suitable baseline and index images available to create vignettes was much smaller than anticipated. This limitation required us to identify a different combination of (1) number of participants, (2) number of vignettes per participant and (3) number of times each vignette was assessed. The final design recruited 96 participants (48 in each professional group), each of whom assessed 42 vignettes, with each vignette being assessed seven times (by each group), requiring a total of 288 vignettes and 4032 assessments. This total was considerably smaller compared with our original plan for 96 participants each to assess 48 vignettes, with each vignette being assessed four times, which would have required 1152 vignettes and 4608 assessments.

The smaller number of available vignettes also forced us to sample sets of training vignettes from among the 288 vignettes used for the main trial assessments. This constraint meant that extra participants (recruited when recruitment reopened) could not start their training assessments until an earlier participant had withdrawn and freed up a specified set of vignettes for the main trial assessment.

All images used to create vignettes were reviewed by a senior grader from NetwORC UK (AM) and a retina specialist (UC) to identify appropriate pairs of sufficient quality. Almost all of the CF photographs in the IVAN trial repository that were potentially suitable for inclusion were considered to be of sufficient clarity and focus for constructing the vignettes. However, OCT images were subject to several limitations:

• We made the decision that vignettes could only be constructed with OCT images captured by spectral domain tomographic equipment. Spectral domain OCT systems provide images of greater clarity, resolution and definition and represent systems to which clinicians and optometrists are currently exposed. These spectral domain systems came into widespread use in 2008. When the IVAN study started in 2007/8, the standard for clinical trials was time domain OCT because the data import and export protocols and grading methods had been established and validated only for these acquisition systems. Therefore, the majority of sites in the IVAN trial used time domain systems to capture OCT images. Nonetheless, as the old time domain systems began to fail, most IVAN trial clinical sites replaced these with spectral domain systems. Validation of the spectral domain systems for trials was undertaken and, therefore, spectral domain OCT images were permitted in the IVAN trial protocol. Approximately one-third of all IVAN trial participants had OCT images acquired on one of the three types of commonly used spectral domain systems (Heidelberg Spectralis^®; Zeiss; Optovue). All the spectral domain systems yielded high-quality images. However, the lower aspect ratios of the Zeiss and Optovue made it more difficult to interpret images than with the Spectralis. This is because the appearance of the Spectralis images is more similar to that observed on a standard retinal histological preparation. The Spectralis is also in more widespread use in the UK. Both experts and participants commented that OCT images from the Zeiss and Optovue spectral domain systems were unfamiliar and, therefore, that the images were more difficult to interpret and that they were less certain about their assessments.

• Baseline and index OCT images consisted of six radial line scans which passed through the fovea. Ideally, scan lines at the same clock-face orientations should be captured at every visit and be displayed in the same order so that an assessor can compare the appearance of the retina at specific locations. For some of the scans, the orientations did not match exactly for baseline and index images.

• The web application presented ‘thumbnail’ images of all six OCT images and an assessor could ‘click’ on an image to enlarge it and to toggle backwards and forwards between the six enlarged images. (The website www.echoestrial.org/demo shows how assessors carried out assessments in the trial.) However, it was not possible to view baseline and index images for corresponding orientations simultaneously because of the limitation of providing the application on a single monitor. Paired viewing of OCT images is a desirable feature and it is usual practice for clinicians to use a paired display which allows the viewer to scroll through the different scan orientations from two visits on a single screen. One participant (Optom268, see Chapter 3, Participants’ views on ECHoES trial training) achieved this by having one set of images on a tablet and one set on a conventional computer display.

Reference standard

The original plan (described in the grant application) was for the reference standard to be derived directly from grading data collected during the IVAN trial. For various reasons, this was not considered acceptable once we started to set up the study: there are occasional errors in the grading data and the list of features/questions drawn up for the web assessment (see Table 2) did not completely match the table in the protocol or the grading data available. There were two key differences, namely (1) the distinction between presence of a feature in the index OCT and whether or not an increase in the feature had occurred from the baseline to the index OCT, and (2) the inclusion of DRT as a feature. Grading data were used to select baseline and index OCT pairs that were likely to suitable, based on the features described in the table in the protocol, but it was decided that the reference standard should be assigned by expert ophthalmologists (reading centre leads, UC, TP and SPH).

This decision was taken when the first draft of the assessment part of the website was available, in late June 2013. We accepted that training could begin while vignettes were being assessed by experts. However, there was still insufficient time for the three experts to assess each vignette by the time participants’ responses to the training vignettes had to be scored. Instead, we aimed for two experts to assess each vignette (one-third of the total number of vignettes by each pair of expert ophthalmologists), but only had one assessment for each vignette at the time that participants’ responses were scored. These assessments were carried out by two of the three experts.

As participants’ assessments on the training vignettes accrued, it became clear that there were some vignettes that participants were getting consistently wrong. Consequently, the two experts who had provided the expert assessments reviewed about 30 vignettes. Some of these were believed to be mistakes (mainly key-stroke errors when completing the web data entry) by an expert and the overall assessment was changed for 10 vignettes. These new gradings were later used in the derivation of the reference standard.

Classification of lesion activation status and lesion components

Optometrists were non-inferior to ophthalmologists with respect to their overall ability to classify lesions correctly: that is, the primary outcome. Neither group attained the level of performance expected at the outset (on which the target sample size was calculated). However, optometrists made different kinds of errors. Compared with ophthalmologists, it was less likely that they would classify a reactivated lesion as quiescent or suspicious (false-negative misclassifications) and more likely that they would classify a quiescent or suspicious as reactivated (false-positive misclassifications); better sensitivity, worse specificity. It is probable that this finding arises because optometrists tended to adopt a more cautious decision criterion, which would be consistent with their obligations under the General Optical Service contract (to refer any suspected pathology). The finding may also reflect the fact that optometrists had more difficulty in interpreting the diversity of appearance of quiescent lesions, that is, eyes with an abnormal appearance but not needing treatment, which are normally managed in the HES.

The poorer than expected overall performance may have arisen because the quality of the images was suboptimal. In a real-life scenario, any monitoring review (carried out by either an ophthalmologist or an optometrist) is likely to include examination of the patient, for example slit-lamp biomicroscopy, as well as review of CF and OCT images. Some components, such as blood and exudates, might be identified more reliably from this examination than from images. If this were the case, the combination of the vignette information with direct examination would increase the performance.

No harms could arise in the trial itself because all of the decisions being made were for anonymised vignettes. However, the health economic evaluation carefully considered the costs and consequences of the different types of error predominantly made by each group. In the context of community optometry, it seems desirable that optometrists should use a cautious decision criterion for referral, although this limits the potential benefit of the shared care model in terms of both the impact on workload in the HES (because of false-positive referrals) and its potential cost-effectiveness.

Optometrists were also non-inferior to ophthalmologists with respect to the frequency of false-negative sight-threatening errors (i.e. failure to identify a lesion as reactivated). In fact, optometrists were slightly less likely than ophthalmologists to make such errors because of their tendency to adopt a more cautious decision criterion. Conversely, optometrists made more non-sight-threatening errors (false-positive errors, i.e. failure to identify a lesion as quiescent or suspicious) than ophthalmologists.

Except for PED (which did not inform the classification of lesion activity), lesion components were identified as present more often by optometrists than by ophthalmologists, again consistent with optometrists adopting a more cautious decision criterion. This tendency was particularly evident for DRT and exudates. Ophthalmologists were much more confident in their classifications than optometrists. However, there was no association between confidence and the odds of classifying a lesion correctly. For all grades of confidence in the classification, optometrists had slightly better odds of classifying a lesion correctly. As hypothesised, other items of information included in a vignette did not influence participants’ lesion classifications.

The sensitivity analysis found a statistically significant difference, but not clinically important one, with respect to the prespecified non-inferiority margin, in favour of ophthalmologists. The alternative definition of the primary outcome used in this analysis involved reassigning four cells from Table 3. Participants’ classifications changed from incorrect to correct for two cells (suspicious reference lesions classified as reactivated by participants and reactivated reference lesions classified as suspicious by participants) and from correct to incorrect for two cells (suspicious reference lesions classified as quiescent by participants and quiescent reference lesions classified as suspicious by participants). Of these, the latter were both the most numerous (380 out of 4032 participants’ classifications) and showed the greatest difference in frequency between ophthalmologists and optometrists (88 more such classifications by optometrists considered incorrect in the sensitivity analysis; see Table 7). Reactivated reference lesions classified as suspicious by participants were second most numerous (338 out of 4032), with the difference in frequency going in the opposite direction (54 more such classifications by ophthalmologists considered correct in the sensitivity analysis). Both of these changes favoured the performance of the ophthalmologists. The large number of quiescent reference lesions classified as suspicious by optometrists almost certainly arises from their tendency to adopt a more cautious decision criterion. It is not obviously the case that it would be desirable to encourage optometrists to shift their criterion to a less cautious position, that is to ‘trade’ an improvement in specificity for a reduction in sensitivity, given the potential sight-threatening consequences of false-negative errors. The desirability of shifting the criterion would depend on the likelihoods of (1) reactivation being identified by the optometrist at the next monitoring review (when relevant lesion components are likely to have increased) and (2) irrecoverable sight loss from deterioration of the lesion.

The time taken to assess vignettes decreased as participants worked through their assessments for the main trial, and this finding was particularly marked for the optometrists. Interestingly, the time taken by optometrists more than halved and approached that taken by ophthalmologists by the time they reached their final assessment (2 minutes 31 seconds compared with 1 minute 55 seconds). This information gives some indication of how quickly optometrists can become proficient in assessing CF photographs and OCT images. We also investigated whether or not the time taken was associated with the odds of a correct assessment and found, in both professional groups, that a correct lesion assessment was more likely with a shorter assessment duration. This finding suggests that duration of assessment may have been a proxy for difficult assessments.

Data about the agreement between expert medical retina consultants represent a bonus from the study, as such data have not previously been reported. (These experts are responsible for the three sites that together make up the UK Network of Ophthalmic Reading Centres.) All three experts assigned congruent lesion classifications to over three-quarters of vignettes and disagreed completely (all three experts classified a lesion differently) for only seven vignettes. The level of agreement judged on the basis of kappa values (see Table 14) varied by lesion component; it was excellent for SRF, IRC and blood but poor for exudates and DRT. (Experts agreed about the presence or absence of exudates of > 90% of vignettes but the kappa value was low because exudates were rarely present.) These data emphasise several important points. First, the assessment task was difficult and was sometimes compromised by the images that were available (see Images used to create vignettes and constraints on viewing images). Second, signs of reactivation inevitably spanned a continuum and hence there was a need to assign a reference classification of suspicious for some vignettes. Third, the vignettes created from the IVAN trial repository represented a real-world spectrum of reactivation that was appropriate for the ECHoES trial.

There are no similar studies with which to compare our data and, consequently, no meta-analysis combining our findings with previous findings.

It is difficult to assess the extent to which the performance of participants is representative of the performance of the two professions and, hence, the applicability of the findings. Models of shared care for other eye conditions have been voluntary. Indeed, it is difficult to conceive a model in which optometrists, who are private practitioners, might be required to provide shared care. Our recruitment methods sought volunteer optometrists, presumably who were interested in shared care, albeit in the context of a research project. We cannot be sure that our recruitment methods for optometrists were generally applicable but can imagine that similar methods might be used, for example by clinical commissioners seeking to commission shared care in a particular geographic region. The motivations of the volunteer ophthalmologists were less clear, but a similar interest in the feasibility of shared care for nAMD is a possibility. The key question is whether or not volunteer ophthalmologists might have been less than averagely expert/experienced – or optometrists more than averagely expert/experienced – compared with the kinds of people who might volunteer for shared care. We think that this is unlikely but have no evidence to substantiate this.

For the findings to be applicable, the vignettes used for the study also needed to be representative of the clinical circumstances that community-based optometrists would be likely to encounter. In this respect, we are more confident that the study findings are applicable. The IVAN trial was pragmatic and had broad eligibility criteria. It recruited participants on presentation and followed their progress in the trial. Although their treatment was specified by their experimental allocation, all treatment regimens had similar effects on vision and morphology which were consistent with clinical experience and other trials of anti-VEGF drugs. Decisions about the need for retreatment had to be made in the IVAN trial just as in usual care. The availability of spectral domain tomographic equipment was limited by hospitals and clinic scheduling.

Health economics

The results of the economic evaluation show that, when we take account of downstream costs (e.g. follow-up consultations and injections), the optometrists had slightly higher costs and made slightly fewer correct retreatment decisions compared with ophthalmologists when performing the virtual monitoring review for the ECHoES trial. However, the differences were very small (an incremental cost of £13/consultation and one additional incorrect decision per 101 reviews conducted) and the differences were not significantly different.

Views of patients and health professionals about the shared care model

Overall, the findings show there was consensus that optometrists monitoring quiescent nAMD in the community have the potential to reduce clinical workload and could represent a more patient-centred option for patients. However, a number of potential barriers were identified which could limit the feasibility of a shared care scheme, including ophthalmologists’ perceptions of optometrists’ competence, the need for clinical training, whether or not optometry and ophthalmology could work more collaboratively and whether or not shared care was a financially efficient option for CCGs.

Participants felt that hospital eye clinics were pushed to the maximum capacity with the volume of patients who required care for nAMD. Research has found an insufficiency of resources to deliver nAMD secondary care from the burden of follow-up visits required in patients with inactive nAMD. 12 Therefore, the health professionals agreed that shared care would relieve the ophthalmology workload. Shared care schemes are also attractive to patients who would find it more convenient to be monitored closer to their home. 34,35 In line with previous research, the service users described how they were frustrated at the lack of support and information that they had received about their condition. 36,37 Consequently, they felt that being monitored in the community would enable them to build up a relationship with one optometrist, with whom there would be more of an opportunity to receive better support and information.

While most participants felt that motivated optometrists would be capable of monitoring patients, several ophthalmologists and service users were unsure whether or not the expertise of an optometrist in detecting reactivation of nAMD would be equal to that of an ophthalmologist. Previous research exploring perceptions of shared care for glaucoma has found that specialists are not convinced of optometrists’ expertise, even when they have received additional training. 38 Furthermore, interviews with service users who had declined to take part in a shared care scheme for a range of eye diseases (including nAMD) did so because of their familiarity with, and the reputation of, the HES. 35

Overall, the health professionals described poor collaboration between community optometrists and ophthalmologists working in the HES. Many participants, particularly the ophthalmologists and service users, voiced concerns about a potential delay between primary and secondary care if retreatment was required. This is a new finding, as previous research exploring patient perceptions of shared care for a range of eye conditions has not found this to be an issue. 34,35,39,40 Long-term research has demonstrated that recurrence of neovascular activity is common,15 and any delay beyond the recommended interval may cause patients to unnecessarily lose vision permanently. 12 Given that vision can deteriorate in a short time if nAMD is not treated and the potential impact this could have on independence and quality of life for patients, it is perhaps not surprising that receiving prompt treatment is an important priority for nAMD care.

The health professionals felt that, although virtual training could provide a foundation level of knowledge, clinical experience under the supervision of an ophthalmologist would be a more effective method of training. Shared care research for a variety of eye conditions has reported that, although web-based training is a convenient and effective option, it is not representative of clinical practice and may not be appropriate for teaching practical clinical skills which need to be developed through attendance at a training course. 35,41 Health professionals in the current study also highlighted that face-to-face training of this kind would provide reassurance to the ophthalmologists that optometrists were being trained to a high standard and further encourage interprofessional collaboration.

The health professionals considered the financial implications of moving to a shared care model, although commissioners appeared to experience conflict between what was best for the patient and what was financially efficient. There was also agreement that optometry practices may struggle to obtain appropriate equipment, but there was uncertainty as to how funding for OCTs would be provided. In line with this research, studies exploring optometrists’ perspectives of extending or enhancing their roles has highlighted a conflict between the retail and clinical sides of the optometric practice. 41,42 Amoaku et al. 12 points out that the technology involved in monitoring nAMD, particularly an OCT, is expensive and optometrists would be unlikely to receive any grants for their purchase. An additional financial consideration was that there could an opportunity to save money with a reduced fee for optometrist rather than ophthalmologist monitoring, although commissioners expressed concerns about the possibility – and subsequent cost – of repeat testing by ophthalmologists who doubted optometrists’ judgements when a patient was deemed to require retreatment.

In summary, the qualitative research demonstrates enthusiasm for shared care for nAMD. However, ophthalmologists and patients would need reassurance that greater convenience would not compromise the quality of care, in terms of both optometrist competence and the speed of the referral pathways back into secondary care if retreatment was deemed to be necessary. The research highlighted poor communication and trust between ophthalmologists and optometrists, and there was agreement that, if shared care were to be implemented, it would be essential that the two professions work more collaboratively. Training for optometrists under the supervision of ophthalmologists was deemed to be the most effective method of training and could improve the communication and trust between the disciplines.

Classification of lesion activation status and lesion components

The virtual nature of the trial element of the ECHoES study made it feasible to address the research question when a conventional trial may not have been feasible, either because of reluctance by patients or the professions to participate or because of the high cost of a conventional trial. 19 We were able to carry out the trial within a relatively short period of time and at low cost compared with a conventional trial of the research question, and the study design did not require any compromise with respect to the risk of bias. Participants engaged extremely well with the tasks that they were set, evidenced by the trial having complete data for the analysis population and the level of participation in additional tasks not communicated to participants at the outset, for example questionnaires about training and the likely resources needed to provide shared care in community optometric practices.

There are two limitations that may be perceived to be critical, namely the virtual nature of the trial itself and the adequacy of training. A critic might argue that decision-making on the basis of a vignette bears no resemblance to face-to-face decision-making in a HES clinic. However, decision-making on the basis of investigations made previously, in the absence of the patient, reflects quite well how some hospitals are managing their workload by implementing a two-step process. Patients are first recalled for a monitoring appointment to capture BCVA and retinal images (usually staffed by non-medical personnel); the information is then reviewed in an ‘offline’ assessment by an ophthalmologist or other trained member of staff, and the patient is rapidly recalled for treatment if required. In our view, the task of vignette assessment in the ECHoES study closely parallels this offline assessment in the HES.

The importance of training was highlighted by identification of concern about the competence of optometrists as a potential barrier to implementation (see Chapter 5, Perceptions of optometrists’ competency). The quality of the training was perceived to be good, very good or excellent by over 90% of ophthalmologists but by only about 70% of optometrists; 70% of ophthalmologists considered that the training was sufficient, compared with only 20% of optometrists; and almost all optometrists, but only half of the ophthalmologists, revisited the webinars (see Chapter 3, Participants’ views of ECHoES trial training). These simple responses to the questionnaire items eliciting feedback about training were supported by free-text comments. These differences in perception about training are likely to have arisen for various reasons: some ophthalmologists may have considered themselves to be already trained; the training was inevitably conceived by ophthalmologists from a HES perspective; and the relative unfamiliarity of the task for optometrists may have made them less confident about the competence that the training had in fact conferred. Both the feedback questionnaire and the qualitative research suggest that a bigger investment in training would be required if shared care were to be implemented. The nature of further training that would address the need voiced by optometrists is unclear.

The nature of some of the images available to create the vignettes and the method for viewing OCT images were also potential limitations of the study (see Images used to create vignettes and constraints on viewing images). These limitations may explain in part why the performance of ophthalmologists was worse than expected (about 85% correct compared with the expected 95%). Consequently, the performance levels observed in the trial should, perhaps, be considered to be the minimum achievable. It is possible that ophthalmologists (most of whom were already familiar with OCT images) could have been selectively disadvantaged, in that they were used to seeing images captured on spectral domain systems and displayed on very high-resolution monitors. Alternatively, they may have had the advantage of having seen older images in the HES, which optometrists may not have done. Therefore, it is unknown whether or not vignettes created from more recent information may have favoured the performance of one professional group over the other. The important point is that the trial design ensured a ‘level playing field’ in terms of the assessments that both groups, and the experts, carried out.

Finally, the final reference standard was not available by the time that participants’ training performance had to be assessed (see Reference standard). This could have been a serious limitation if the information used to assess participants’ performance at the time misclassified participants with respect to their performance and their ability to progress in the trial. However, as previously stated (see Chapter 2, Reference standard), when all experts had classified all vignettes, we checked the performance of all applicants for their training vignettes using the final reference standard classifications. No applicant was refused admission to the main trial on the basis of the interim expert classifications who would have progressed to the main trial on the basis of the final reference classifications.

Health economics

The cost-effectiveness results should be considered in the light of some limitations of the study. Because the ECHoES trial was virtual, participants were asked questions about a service not implemented yet, which posed challenges for optometrists in identifying appropriate costing information for their potential nAMD monitoring review provision. The health economics questionnaire was not a compulsory part of the trial and we wanted to avoid overburdening participants with lots of questions. Therefore, we used expert opinion to inform the economic evaluation about the types and models of different pieces of equipment, the average volume of patients using some of the equipment items and the expected working life of equipment. Furthermore, the costs used in the economic evaluation rely on the assumptions inherent within the decision trees shown in Figures 1 and 2.

Expert opinion was also used to help to map the pathways for the care cost model with respect to the likely courses of action arising from various decisions being made by optometrists and ophthalmologists. In particular, the optometrist pathway assumes that all patients classed as reactivated receive a second monitoring review in hospital (at the more expensive ophthalmologist cost, rather than the cheaper optometrist review cost), which increases costs for correct judgements of lesion reactivation in the optometrist pathway but also enables a second opportunity for any errors to be corrected before costly ranibizumab injections are given. In real-world implementations of shared care, other pathways may be devised or pathways may improve with time after they have been implemented. For example, the optometrist pathway is likely to include direct clinical examination, as well as review of CF and OCT images, and an integrated shared care pathway could allow the HES to review information obtained by community optometrists following rapid referral (as envisaged by the last item of the economic question, see Appendix 3, Q8), rather than repeating monitoring tests. Alternatively, shared care pathways may also improve with time after they have been implemented.

Views of patients and health professionals about the shared care model

A purposeful sampling approach was adopted within and across the research to ensure that the feasibility and acceptability of the proposed shared model of care for nAMD was captured from a range of perspectives. However, ophthalmologists and optometrists were recruited from specialist conferences, which may limit the extent to which their perspectives are representative of optometrists and ophthalmologists in the UK. Furthermore, service users were recruited from Macular Society support meetings and such individuals may be more proactive and informed about their condition than non-members. 37 Service users at the support groups who were willing to be contacted were white British, possibly because white people may be more susceptible to developing nAMD43 or because people from ethnic minority groups are less likely to access eye care services. 44,45 It should also be acknowledged that all service users attended the same eye hospital for their nAMD care and some aspects of their care may differ to other hospitals throughout the UK, although the themes relating to current experiences of hospital care have also been highlighted by previous research. 34,36,37

Focus groups were used to explore and understand key issues when considering barriers and facilitators to implementation. Separate rather than mixed focus groups were undertaken to capture any potential interprofessional trust issues which emerged from previous research. 35 One-to-one interviews were conducted with other health professionals because it was not possible to ascertain a convenient location, date and time to organise a third focus group. These interviews provided a rich account of the perceived feasibility and acceptability of a shared care scheme and allowed the findings from the focus groups to be followed up further and explored in-depth.

None of the participants had experience of shared care for nAMD. Research has questioned whether or not evaluations of hypothetical scenarios accurately relate to judgements in real-life situations. 46,47 However, participants gave negative as well as positive views of the shared care model, which suggests that they carefully considered the practicalities of implementation. In addition, focus groups provided an opportunity to prompt a range of issues about shared care to be discussed which participants may not have otherwise considered individually in one-to-one interviews. 48 Furthermore, the issues identified in this study about a hypothetical shared care scheme mirror many of the findings from studies in which participants gave feedback after direct experience of shared care. 35,41

Lessons for a future shared care approach to the management of neovascular age-related macular degeneration

It is clear that, given foreseeable health technologies for treating nAMD (including potential emerging therapies combining anti-VEGF drugs with stereotactic radiotherapy), the need for efficient methods of long-term review will remain an urgent priority. Some form of shared care with community-based optometrists is one approach for achieving this, notwithstanding the findings of the economic evaluation (see Future research). The ECHoES study has demonstrated that community-based optometrists can develop competency in decision-making about lesion reactivation that is equivalent to the competency of ophthalmologists working the HES. Although it was not straightforward to conduct the study in a short period of time, we were able to apply the methods we planned at the time of the grant application with few modifications. We would recommend further trials of this nature to address research questions where appropriate data repositories can be identified.

Lessons for a future economic evaluation alongside a virtual trial

The virtual trial imposed the limitation of asking participating optometrists to identify the resources and likely costs for providing a service that had not yet been implemented. This is not an intrinsic feature of virtual trials but is a likely feature, since the attraction of a virtual trial is greatest when there are obstacles to provide a service. The absence of an existing shared care pathway also contributed to uncertainty about the cost-effectiveness estimates. This was highlighted by the economic evaluation sensitivity analyses, notably the analysis which excluded rereview in the HES of a patient rapidly referred by a community-based optometrist.

Lessons for qualitative research alongside a virtual trial

This study highlights the importance of exploring the views of relevant stakeholders about the acceptability of shared care for nAMD and barriers to its implementation alongside a virtual trial. The qualitative research identified key concerns that would need to be addressed in formulating a concrete shared care model. Had the study proceeded as originally proposed by the applicants, the headline non-inferiority findings might have led to unreasonable optimism about the feasibility of shared care for nAMD.

1.1 Scope

This statistical analysis plan (SAP) details information regarding the statistical analysis of the completed ECHoES trial and covers analyses of trial data outlined in the study protocol. It does not include the health economic evaluation or analysis of the focus group discussions.

1.2 Editorial changes

Any changes made to this SAP after approval must be clearly justified and documented as an amendment at the end of this document. The SAP should then be re-approved.

1.3 SAP document approval

The trial statistician should authorise this document.

2.1 Study background

Wet, or neovascular, age-related macular degeneration (nAMD) is a condition which causes severe sight loss in older people. There is currently no evidence about the effectiveness of community follow-up by optometrists for nAMD. However, there is evidence about the acceptability of further training to optometrists, and their effectiveness in providing ‘shared care’ with the Hospital eye clinic setting (HES), for other eye diseases. Community optometrists already participate successfully in shared care management schemes for patients with glaucoma and diabetic eye disease, and evaluations comparing agreement between management by optometrists and ophthalmologists has been shown to be acceptable in the context of glaucoma and accident and emergency services.

The question of interest to the NHS is whether community optometrists can be trained to make decisions about the need for retreatment in patients with nAMD whose disease has been rendered quiescent by treatment with anti-vascular endothelial growth factor (VEGF) drugs with the same accuracy as ophthalmologists working in the HES.

2.2 Study objectives

The aim of the trial is to test the hypothesis that, compared to conventional hospital eye clinic follow-up, community follow-up by optometrists (after appropriate training) is not inferior for patients with nAMD with stable vision. This hypothesis will be tested by comparing decisions made by samples of ophthalmologists working in the HES and optometrists working in the community about the need for retreatment, following a period in which patients have not required treatment. Rather than carrying out a new (prospective) trial, optometrists and ophthalmologists participating in the trial will make decisions about vignettes composed of clinical information and colour fundus (CF) and ocular coherence tomography (OCT) images collected in the course of the IVAN trial (HTA ref: 07/36/01). Retreatment decisions made by participants in both groups will be validated against a reference standard based on the opinions of three medical retinal experts (see Appendix 5, section 4.1).

The trial has five specific objectives:

1. To compare the proportion of retreatment decisions (based on lesion classifications, see Appendix 5, section 2.3) classified as ‘correct’ (against the reference standard, ‘reactivated’ vs. ‘suspicious’ or ‘inactive lesion’, see Appendix 5, section 4.1) made by optometrists and ophthalmologists.

2. To estimate the agreement, and nature of disagreements, between retreatment decisions made by optometrists and ophthalmologists.

3. To estimate the influence of vignette clinical and demographic information on retreatment decisions.

4. To estimate the cost-effectiveness of follow-up in the community by optometrists compared to follow-up by ophthalmologists in the HES. This is not covered in this SAP.

5. To ascertain the views of patient representatives, optometrists, ophthalmologists and clinical commissioners on the proposed shared care model. This is not covered in this SAP.

2.3 Terminology

Throughout the protocol and this SAP, the terms lesion classification, retreatment decision and referral decision are used interchangeably. Unless otherwise stated, the analyses will use the lesion classification question, for which participants must classify the vignette as reactivated, suspicious or quiescent. The logic behind this is that the identification of key vignette features and the resultant classification of lesions is the difficult part; retreatment/ referral decisions can be defined by rules based on these classifications.

2.4 Primary outcome

The primary outcome is a participant’s judgement of lesion classification (“Reactivated lesion” vs. “Inactive lesion” or “Suspicious lesion”) against the reference standard (see section 4.1). The number of ‘correct’ assessments will be compared between the two participant groups (see section 4.2 for details).

2.5 Secondary outcomes

1. The secondary outcomes are:

2. The frequency of “serious” errors judged likely to be sight-threatening. This classification will be assigned to a participant’s classification of a vignette when the participant’s decision is “lesion quiescent” and the reference standard classification is “lesion reactivation”, i.e. a definitive false negative classification by the participant. Definitive false positive decisions will not be considered serious but will be tabulated separately. Misclassifications involving classifications of “suspicious lesion”, whether assigned as the reference standard or by a participant, will also not be considered serious.

3. Judgements about the presence or absence of lesion components, e.g. blood, exudates and sub retinal fluid (SRF) in the fundus colour images; SRF, intra-retinal fluid/cysts and pigment epithelial detachment (PED) in the OCT images.

4. Participant-rated confidence in their decisions about the lesion classification, on a 5-point scale.

3.1 Trial design

The participants are not randomised into the trial as by training they are either an optometrist or an ophthalmologist. However, randomisation, using a randomised block design, was used in assigning vignettes to each participant. The ‘main study’ vignettes (i.e. excluding training vignettes) were assigned to participants in a random order such that each vignette is seen seven times by an ophthalmologist and seven times by an optometrist, and each of the 48 participants in each group sees 42 vignettes. There are therefore a total of 288 (48 × 42 / 7) vignettes included in the trial. The selection of vignettes and the order they are viewed is matched for the two groups in a 1:1 ratio in order to reduce any bias that may be introduced by different assessors viewing different vignettes in different orders. Each participant is also assigned a random 24 training vignettes (and a further 24 training vignettes if they fail their first set) such that none of the vignettes in their training set(s) appear in their main study set. Again, this is matched across groups.

3.2 Flow of participants

Participants who are consented into the study must first complete two webinar training sessions. Once these have been complete they are allocated a training set of 24 vignettes, of which they must correctly assess at least 18 (75%) to pass their training. If they pass, they will then be allocated their 42 ‘main study’ vignettes. If they fail their first training vignettes, they are allocated a second training set of a further 24 vignettes. If they pass this second training set they will be allocated their 42 main study vignettes; if they fail their second set of training they will be withdrawn from the study. Only participants who assess all of their 42 main study vignettes will be included in the analysis population (see Appendix 5, section 3.5). Participants are followed up until they have assessed all of their main study vignettes or until they withdraw or are withdrawn from the study. The planned study time is approximately six months from registration until study completion.

The participant flow will be described via the flowchart.

3.3 Protocol deviations

There is no protocol deviations defined for this study.

3.4 Withdrawals

Participants can withdraw from the study at any time after registration. They may also be withdrawn by the study team if they do not correctly assess an adequate number of vignettes in their training sets (see Appendix 5, section 3.1 for details).

3.5 Analysis population

The analysis population will consist of the 48 ophthalmologists and 48 optometrists who complete their 42 main study vignettes. Participants who withdraw or are withdrawn before completion of their main study vignettes will not be included in the analysis population but their progression in the trial will be described in the flowchart.

4.1 Reference standard

All vignettes will be assessed by three experts; it is expected that for most vignettes the classification decisions of all experts will agree. For those where there is disagreement, the three experts will collectively make a final consensus decision. The agreed assessment decision for each vignette (“reactivated”, “suspicious” or “quiescent” lesion) will be known as the reference standard and will be used to calculate ‘correct’ assessment decisions of all participants.

4.2 Primary outcome

For the primary outcome, a binary ‘active lesion’ variable will be derived for both the reference standard and the participant responses, as follows:

• If a vignette is classified as “lesion reactivated” then active lesion = Yes

• If a vignette is classified as “lesion quiescent” or “lesion suspicious” then active lesion = No

• Else missing (by design there should not be any missing data)

An indicator of whether participants have made the correct classification for each vignette will then be derived as follows:

• If the reference standard and a participant have both classified a vignette as an active lesion or have both classified a vignette as not an active lesion, then classification status = correct

• If the reference standard have classified a vignette as an active lesion and a participant has not classified it as an active lesion, or vice versa, then classification status = incorrect

• Else missing (by design there should not be any missing data)

An overall count of the number of vignettes that each participant correctly classified (out of a possible 42) will also be derived.

4.3 Other variables

5.1 Participant characteristics

Participant characteristics will be described by group for participants in the analysis population.

Continuous variables will be summarised using the mean and standard deviation (SD) (or median and inter quartile range (IQR) if the distribution is skewed), and categorical data will be summarised as a number and percentage. The summary statistic headings given are those we expect to use based on a-priori knowledge of the measurements but may change.

Characteristics of the participants will be described and compared formally using t-tests, Mann-Whitney tests, chi-squared tests or Fishers exact tests as appropriate.

5.2 Primary and secondary outcome data

5.3 Safety data

There is no safety data in this study as there are no risks to participants and therefore it is not possible for clinical adverse events to be attributed to study specific procedures.