Automated imaging technologies for the diagnosis of glaucoma: a comparative diagnostic study for the evaluation of the diagnostic accuracy, performance as triage tests and cost-effectiveness (GATE study)

Article history

The research reported in this issue of the journal was funded by the HTA programme as project number 09/22/111. The contractual start date was in September 2010. The draft report began editorial review in March 2014 and was accepted for publication in November 2014. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The HTA editors and publisher have tried to ensure the accuracy of the authors’ report and would like to thank the reviewers for their constructive comments on the draft document. However, they do not accept liability for damages or losses arising from material published in this report.

Declared competing interests of authors

David Garway-Heath received grants from the National Institute of Health Research during the conduct of the study, personal fees and non-financial support from Heidelberg Engineering UK, personal fees and non-financial support from Carl Zeiss Meditec, Inc., non-financial support from OptoVue Inc. and non-financial support from Topcon outside the submitted work.

Copyright statement

© Queen’s Printer and Controller of HMSO 2016. This work was produced by Azuara-Blanco et al. under the terms of a commissioning contract issued by the Secretary of State for Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science Park, Southampton SO16 7NS, UK.

Diagnosing glaucoma

Glaucoma is diagnosed primarily by detecting glaucomatous optic neuropathy (i.e. characteristic changes of the optic nerve head – the optic disc) and a compatible visual field defect. According to current National Institute for Health and Care Excellence (NICE) guidelines,26 a definitive glaucoma diagnosis is based on the expertise of a clinician who subjectively interprets the appearance of the optic disc and the results of visual field testing. In addition to diagnosing glaucoma, the clinical examination will include a visual acuity (VA) test (to measure central vision), anterior chamber angle examination (to determine the mechanism of glaucoma, e.g. open-angle or angle-closure), and IOP measurement (which is a risk factor for glaucoma and also for disease progression).

Accurate clinical diagnosis of glaucoma is limited by subjectivity, reliance on the examiner’s experience and a wide variation of optic disc structure in the population. Imaging techniques for assessment of the structural changes at the optic nerve head and retinal nerve fibre layer (RNFL) have emerged and are in routine use in the NHS: Heidelberg Retinal Tomography (HRT)-III, scanning laser polarimetry [glaucoma diagnostics (GDx; Carl Zeiss Meditec, Dublin CA, USA)] and spectral domain optical coherence tomography (SD-OCT; Heidelberg Engineering, Heidelberg, Germany). These techniques can be easily performed by trained technicians and provide an automatic glaucoma classification index. Some clinicians now routinely incorporate the information from such imaging technologies to help make a diagnosis of glaucoma, although there is no strong evidence of their effectiveness.

Using an automated imaging quantitative test for glaucoma diagnosis may have advantages over visual field testing in that the majority of people can be imaged. 18

Comparison of glaucoma diagnostic technologies

In 1997, the Health Technology Assessment (HTA) programme funded a study entitled ‘The effectiveness of the Heidelberg Retina Tomograph and laser diagnostic glaucoma scanning system (GDx) in detecting and monitoring glaucoma’. 30 At the time, this study was the largest and most rigorous head-to-head comparison of tests for diagnosing glaucoma. However, this study used the first prototypes of the HRT and GDx, now outdated. Another serious limitation was the small study sample (250 participants), in addition to a potentially biased selection of patients, as they were not consecutively selected.

A systematic review of the performance of technologies for detecting glaucoma as both screening and diagnostic tests for glaucoma identified that the evidence is of poor quality and that no one test was clearly superior. 18 In this systematic review it was also found that populations studied were varied and biased. Furthermore, only six studies performed a direct comparison of the available diagnostic instruments (and including, on average, fewer than 300 patients), the threshold for definitions of glaucoma cases was not consistent and there were no studies reporting on the performance of GDx and optical coherence tomography (OCT) that met the inclusion criteria for this systematic review. However, the review did suggest that some diagnostic technologies perform better than others (e.g. HRT performed relatively well), but the credible intervals around the estimates were wide, reflecting considerable uncertainty, and, therefore, it recommended that the available diagnostic tests be evaluated in an appropriately powered directly comparative study.

In the published NICE guideline,26 the authors searched for evidence comparing the diagnostic performance of HRT, GDx and OCT with expert clinical examination. No studies met the inclusion criteria for the guideline review.

Triage tests in secondary care eye services

Considerable NHS resources are required to assess all patients referred to hospital eye services with glaucoma suspect. In June 2009, the chairman published on behalf of the Professional Standards Committee of the Royal College of Ophthalmology a statement that the interpretation of NICE glaucoma guidelines was putting considerable strain onto secondary care eye services through the increase in false-positive referrals from community optometrists. The statement proposed that eye departments should consider innovative and efficient clinics for the initial assessment of patients. 31

If referrals could be triaged to identify suitable referrals and discharge unsuitable referrals in an effective and cost-effective manner, the resources could be better utilised for patient eye care services. Imaging technologies are being introduced into glaucoma services in both hospital and community settings, but their role in the diagnostic pathway as triage, replacement or add-on tests has not been evaluated. The tests to be evaluated in this study are the currently available imaging technologies with characteristics that suggest that they could be valuable triage tests and that are in current use in the NHS. They do not require patient input, are user-friendly,32 provide automated quantitative classifications and potentially could reduce the need for an extensive examination by an expert glaucoma clinician. The diagnostic performance of these imaging technologies has not been evaluated in a triage setting and in a robust manner.

Aim and research objectives

Overview of the study design

An overview of the GATE study design is shown in Figure 1. The GATE study is a pragmatic within-patient comparative diagnostic evaluation of four imaging techniques for glaucoma in patients referred to hospital eye services. Specifically, this study was designed to evaluate (1) diagnostic accuracy of imaging tests for detecting glaucoma in an eye and (2) diagnostic accuracy of triage tests that consisted of a combination of an imaging test, VA and IOP measurement, for identifying patients requiring referral to hospital eye services.

FIGURE 1.

Overview of study design. GAT, Goldman applanation tonometry; VF, visual field. a, Carl Zeiss Meditec, Dublin, CA, USA.

All patients recruited to the study received four different imaging tests (using three different devices), which were compared with a reference standard (i.e. a comprehensive clinical examination). The study was co-ordinated from a central study office in the Health Services Research Unit at the University of Aberdeen.

Participants

Setting

Five NHS hospital eye services in the UK participated in this study: Aberdeen Royal Infirmary (Aberdeen), Bedford Hospital (Bedfordshire), Hinchingbrooke Hospital (Cambridgeshire), Moorfields Eye Hospital (London) and St Paul’s Eye Unit (Liverpool). The participating units consisted of three academic units of different sizes and two district general hospitals (Hinchingbrooke and Bedford).

Identification of participants and recruitment process

Consecutive eligible patients referred from community optometrists to hospital eye services with a glaucoma-related finding were identified by the research officer in each centre at the time of referral. Patients were identified from their referral letter as being referred with a possible glaucoma diagnosis or glaucoma-related finding, including high IOP, possible abnormalities in the optic disc or visual field tests, and possible narrow anterior chamber angle. To ensure that a full cross-section of referrals were identified, existing referral refinement schemes in two of the participating centres were suspended for the duration of the study in order not to introduce selection bias. In the largest centre (Moorfields Eye Hospital) only those patients booked to see a clinician trained in the study protocol to provide the reference standard were identified as eligible. Information about this study was sent to potentially eligible patients together with the date of the appointment (see Appendix 1). Patients were approached by the local research officer on their first visit to hospital eye services to discuss the study and those patients who agreed to participate and signed the consent form (see Appendix 1) were enrolled (i.e. before their consultation with the ophthalmologist). Each research centre kept a clinic log of eligible patients invited (see Appendix 2), which included patient demographics (age and sex) and, for those who declined to take part or were found to be ineligible, reason for not taking part if given.

Diagnostic technologies being assessed (index tests)

Four diagnostic tests from three imaging devices were evaluated:

1. HRT-III, confocal laser scanning imaging technology, used by the Heidelberg Retinal Tomograph (Heidelberg Engineering, Heidelberg, Germany), exploits the principle of confocal laser scanning to allow quantitative structural information of the optic disc anatomy. The topographic image is derived from multiple optical sections at consecutive focal depth planes. Each image consists of numerous pixels, with each pixel corresponding to the retinal height at its location. Images are given a measure of quality: the mean topography SD which the manufacturer recommends should be ≤ 40 µm. There are two main classification tools to define normality/outside normal limits: (1) MRA,34 which requires the user to draw a contour line to define the optic disc boundary, and (2) glaucoma probability score (GPS),35 which is fully automated and independent of operator input.

   1. The HRT-MRA produces an overall (‘global’) classification as well as by six segments (‘temporal’, ‘temporal superior’, ‘temporal inferior’, ‘nasal’, ‘nasal superior’ and ‘nasal inferior’) of the eye. Each a classification of ‘within normal limits’, ‘borderline’ and ‘outside normal limits’ is given based on whether or not the observed value is within the 95.0% prediction interval, between the 95.0% and the 99.9% prediction interval or below the 99.9% prediction interval of the preset data, respectively. The final classification is based on the most abnormal of any of the seven classifications. If any one of these is ‘outside normal limits’ then the overall classification is ‘outside normal limit’. Where there is no ‘outside normal limits’ but at least one ‘borderline’ then the final classification is ‘borderline’. Only where the global and all six segment probabilities are ‘within normal limits’ is the final classification ‘within normal limits’.

2. HRT-GPS produces an overall probability of the presence of glaucoma (‘global’) and by segment (‘temporal’, ‘temporal superior’, ‘temporal inferior’, ‘nasal’, ‘nasal superior’ and ‘nasal inferior’) for each eye. The default ‘final’ classification is based on applying cut-off to the overall and six segment probabilities: < 0.28 is ‘within normal limits’, ≥ 0.28 and < 0.65 is ‘borderline’ and ≥ 0.65 is ‘outside normal limits’. 35 If any one of these is ‘outside normal limits’ then overall classification is ‘outside normal limit’. Where there is none ‘outside normal limits’ but at least one ‘borderline’ then the final classification is ‘borderline’. Only where the global and all six segment probabilities are ‘within normal limits’ is the final classification ‘within normal limits’.

3. GDx-Enhanced Corneal Compensation (ECC) (Carl Zeiss Meditec, Dublin, CA, USA) scanning laser polarimetry measures the RNFL thickness. Measurements are based on the birefringent properties of the RNFL, which has its neurotubules disposed in an organised, parallel fashion. The software provides a discriminating classifier of glaucoma/normality, the nerve fibre indicator (NFI) value, which is fully automated and is calculated for each eye. The manufacturers’ reported cut-offs for the GDx-ECC NFI value are based on 95% and 99% coverage of the normative database population and are 1–35 (‘normal’), 36–55 (‘abnormal 95’) and ≥ 56 (‘abnormal 99’). 36 The difference between ‘abnormal 95’ and ‘abnormal 99’ may be viewed in a similar manner to the ‘borderline’ category for HRT-GPS, HRT-MRA and OCT classifications. The temporal, superior, nasal, inferior, temporal (TSNIT) parameters used in the calculation of the NFI are also produced overall and by eye segment (superior and inferior) and an inter-eye symmetry is also produced. Images are given a quality figure, which the manufacturer recommends should be ≥ 7. In this study, GDx-ECC measurements were made using either the GDx-Pro (three centres) or the GDx-VCC with updated ECC module (two centres).

4. OCT: SD-OCT (Spectralis^®, Heidelberg Engineering, Heidelberg, Germany) is an optical imaging technique capable of providing high-resolution, cross-sectional imaging of the human retina in a fashion analogous to B-scan ultrasonography but using light instead of sound. OCT uses the principles of low-coherence interferometry using light echoes from the scanned structure to determine the thickness of the tissue. The glaucoma detection software of the Spectralis^® machine used in this study produces an average RNFL thickness value for the global and six segments of the eye and automatically compares sectors of RNFL thickness with a normative database. An overall assessment of ‘within normal limits’, ‘borderline’ or ‘outside normal limits’ is produced34 based on the global classification and the six individual segments. Inter-eye symmetry is also produced for each segment. Images are given a quality figure, which the manufacturer recommends should be > 15.

Sample reports generated by each of the imaging tests are shown in Appendix 3.

Reference standards

Outcomes

For each of the four tests (HRT-MRA, HRT-GPS, GDx and OCT) the following outcomes were measured.

Delivery of interventions and data collection

Enrolled participants attended a diagnostic station for imaging (index test) and visual field measurement immediately prior to their meeting with the ophthalmologist. In three centres (Hinchingbrooke, Bedford and Liverpool), the visual field and imaging measurements took place on a separate day prior to the ophthalmologist appointment (within 2 weeks). Pupils were not routinely dilated. However, in those patients in whom adequate quality imaging could not be obtained, pupil dilatation could be used to try to improve image quality. In exceptional circumstances, where dilatation was required in centres offering split visits, some or all of the imaging tests could be delayed until the clinic appointment but always ahead of the clinical reference standard. Imaging technicians and the patient were therefore masked to the patient’s underlying condition at the time of testing. In the remaining two centres (Aberdeen and Moorfields) all measurements were undertaken on the same day. All participants in each of the centres underwent testing with the three imaging devices, in a random order (to avoid bias when collecting participant preference) in one sitting. The random test order was automatically generated for each patient from the study website.

Imaging technicians employed at each centre performed the imaging tests. One to three technicians were identified at each centre and trained in study procedures prior to recruitment (see Appendix 4). There was no restriction on the same technician performing all imaging tests on an individual. Across all centres, most technicians were experienced in performing the test prior to the study; if technicians were not already experienced, they received training from the manufacturer or local imaging lead prior to collecting study data.

With the exception of HRT-MRA, which required an experienced user to identify a contour line at the optic disc margin, all imaging tests generated the glaucoma classification automatically once an image had been acquired. The research officer kept printed copies of the images and uploaded the imaging results to the study website. Imaging reports were identified using a unique study number and date of birth.

The participant was asked to grade the tests in order of preference, or to record no preference, using a standard form (see Appendix 2). Visual field measurements were undertaken with standard perimetry Humphrey SITA 24-2 strategy for each participant after all imaging tests had been completed. In exceptional circumstances, visual field measurements were undertaken ahead of the imaging tests because of clinic demand for equipment. Participants were then examined by an experienced glaucoma clinician who performed a comprehensive ocular examination including IOP measurement with Goldmann applanation tonometry (GAT), gonioscopy and biomicroscopic examination of the optic disc (with pupil dilated in patients without narrow anterior chamber angle) and evaluated the visual field test results. The clinician provided the reference standard masked to the results of the imaging technologies and completed a clinical data collection form (see Appendix 2).

The research officer collated the results for each participant (see Appendix 2) including a copy of the visual field test, completed forms for each participant, uploaded the information onto the web page and posted original consent forms to the central office. Information uploaded onto the web page included demographics, referral IOP, refractive error, patient preference, need for pupil dilatation, and Humphrey visual field reliability and global indices mean deviation (MD), pattern standard deviation (PSD) and visual field index (VFI).

Data management

A web-based secure study database was developed for the GATE study which research staff could access remotely. Password-protected access was provided such that centres could view data only from their own centre. All data collected during the course of the research were kept strictly confidential and accessed only by members of the study team. Minimal patient details were recorded and were stored under the guidelines of the 1998 Data Protection Act. 37 Patients were allocated an individual study number and this number was used to identify study paperwork. Study data were entered and imaging reports uploaded onto the database by the research officer working in each centre. Whenever possible, drop-down boxes were employed to select appropriate responses and minimise typographical errors. Automated range checks and validation were built in to ensure that inappropriate values could not be recorded.

Staff in the study office monitored data centrally and worked closely with local research officers to ensure that the data were as complete and accurate as possible. Missing forms and primary outcome data were automatically identified on the study website and distributed to local research officers on a regular basis. Uploaded imaging reports for each participant were checked by the central office, following an agreed checklist, and errors flagged for correction to the appropriate research team on a regular basis. This resulted in a low percentage of missing primary outcome data (1% reference standard: 1–3% imaging data). The content of approximately 50 case report forms and imaging reports selected at random was checked against entered data to ensure data entry accuracy. If consistent errors or discrepancies were found, this triggered a further training session with the research officer to discuss and resolve data collection and entry issues.

The chief investigator checked a random sample of HRT-MRA imaging reports from each centre (five reports for each operator at each centre) for accurate location of the optic disc margin. A high error rate (more than two of five checked) at one centre triggered a complete check of the data at that centre: images with incorrectly placed contour lines were excluded from the default analysis and classified as artefact, as described in Chapter 4.

Statistical analyses

Patient and public involvement

Representatives from a UK-based charity for glaucoma patients, the International Glaucoma Society, were involved in the study oversight throughout the project through the steering committee. This included review and development of the study protocol and patient paperwork; monitoring the study progress; review and discussion of the final results of the study, including the care pathways and sensitivity analyses for the economic analyses, with particular reference to the patient perspective; and proposing further research priorities, particularly the acceptability of this new model of care. Additionally, a patient with glaucoma reviewed and commented on the lay summary of the report.

Study oversight and management arrangements

The University of Aberdeen sponsored the study. An independent Trial Steering Committee (TSC) was established. The TSC comprised an independent chairperson (ophthalmologist and senior academic), three further independent members (two ophthalmologists and the chief executive of a UK-based charity for glaucoma patients, the International Glaucoma Association) and the study grant holders. The TSC met approximately annually over the course of the study. A patient (IR) agreed to provide advice on certain aspects of the study, but was not a member of the TSC. No data monitoring committee was used, as there were no safety concerns; the diagnostic technologies under evaluation were non-invasive, they were routinely performed in clinical settings and patient management did not change.

The day-to-day running of the study was the responsibility of the chief investigator (AAB) supported by the research manager, research fellow and data support staff. A project management group consisting of the coapplicants provided strategic, management and content expertise to the study.

Ethical arrangements and regulatory approvals

The study and subsequent amendments were reviewed and given a favourable opinion by the North of Scotland Research Ethics Committee (reference 10/S0801/58) and local research and development departments. The study was conducted according to the principles of good clinical practice.

Protocol amendments after study initiation

A number of minor protocol revisions were made after study initiation (Box 1).

Recruitment of participants

Between April 2011 and July 2013, 2088 participants were identified as potentially eligible to take part in the study: 2013 were sent letters of invitation and patient information sheets. Of those invited, 966 (48%) agreed to take part, and 265 (13%) expressed a preference for not participating. Characteristics of non-participants are detailed in Table 2.

Following consent, 11 participants were subsequently excluded from the study: 10 were ineligible (four had pre-existing glaucoma, four were referred from secondary care and two were not referred for glaucoma) and one person withdrew from the study. Therefore, 955 participants were available for the index test comparison. Additionally, owing to administrative and research processes, imaging was not implemented for all imaging tests in 12 participants, and these participants were excluded from all analyses. The baseline measurements presented in this chapter relate to the remaining 943 participants.

Figure 2 shows a diagram of the enrollment following the STARD reporting guidelines. Full details of patient flow through the diagnostic performance analysis are described within the results (see Chapters 4 and 5).

FIGURE 2.

Diagram showing recruitment to the study.

Aberdeen and Hinchingbrooke were the highest-recruiting centres (Table 3). Over two-thirds of GATE participants were recruited from these two sites.

Baseline characteristics of participants

Demographic characteristics of participants and non-participants were similar, with an average age slightly above 60 years (Tables 2 and 4) and similar gender distribution. Among participants, nearly 90% were of white British ethnicity (self-reported ethnicity; Table 4).

Ocular characteristics recorded in the referral letter from the optometrist are detailed in Table 5. In the majority of referrals (77%), the optometrist had highlighted abnormalities in both eyes (referral eye). The average IOP at referral was 20 mmHg. Where the method of IOP measurement was reported on the referral letter (52%), the most commonly reported method of measurement was non-contact tonometry.

Data on VA and refractive error at referral are summarised in Table 5.

Reference standard diagnosis characteristics

Tables 6–14 describe the tests used to determine the reference standard and the diagnoses in the GATE population. The average clinician IOP measured with GAT was similar to the referral IOP (see Table 6) and highest among patients with OHT and glaucoma (see Table 7). Visual field testing was outside the manufacturer-recommended reliability in one-quarter of participants. The average MD among those diagnosed with glaucoma and with reliable visual field tests was –6.0 dB (SD 6.4 dB) in the right eye and –7.5 dB (SD 6.8 dB) in the left eye (see Table 7).

Table 8 displays the diagnosis of the GATE population per eye according to the agreed reference standard (see Chapter 2). The most common diagnosis (at approximately 40%) was ‘no glaucoma-related findings’. Glaucoma was diagnosed in about 11% of eyes. Comorbidities were uncommon, except for cataract, which was reported in approximately 8% of eyes (see Table 9).

Among those eyes with glaucoma, mild disease was most prevalent (above half), while severe glaucoma was diagnosed in a relatively small proportion of eyes with the disease (28 out of 219 eyes, 12.8%; see Table 10).

Over one-third of the GATE participants were discharged after the first visit (see Table 11). Table 13 describes the diagnosis by worse eye (ranked in the order shown) and by better eye. Glaucoma was diagnosed in at least one eye in 16.8% of the GATE cohort and 6.5% had glaucoma in both eyes at referral (see Table 12).

Overview

This chapter reports the results of the diagnosis analyses which aimed to assess the diagnostic performance of the four imaging tests (HRT-MRA, HRT-GPS, GDx and OCT) and the other outcomes associated with the imaging tests (indeterminacy and participant preference). The results of the triage analyses are provided in Chapter 5. The specific diagnostic performance analyses covered in this chapter are the default diagnosis analysis (Table 15, ‘Default diagnostic analysis’), six sensitivity analyses (see Table 15, ‘Diagnosis sensitivity analyses 1–6’) and the use of a combination of the imaging tests (see Table 15, ‘Combination of tests analysis’) for a list with definitions. The default analysis was defined as one where the reference standard definition of disease was a clinical diagnosis of glaucoma only. The imaging test definition of an abnormal result was ‘outside normal limits’ for the overall classification of the imaging test (see Chapter 2).

Additionally, only cases where there was a good-quality image with an overall classification available were included (see Chapter 2). The six sensitivity analyses assessed the impact of varying assumptions made in the default analysis relating to the reference standard definition of disease (including all types of glaucoma suspects as diseased), the definition of an abnormal test result (including borderline results as abnormal), and how cases where the test did not produce an overall classification were handled in the analysis. In addition to missing data, there were four test-related reasons why an overall classification may not have been available (see Table 15, ‘Handling of no results categories’). Sensitivity analyses also assessed the impact of removing the requirement of a ‘good’-quality image and using the provided assessment, along with setting other cases which did not produce an overall classification result as abnormal.

The combination of test analyses investigated using pairs of imaging tests to produce a composite imaging test result, under the same assumptions as the default analysis. Given the findings of the default and sensitivity analyses, only three pairs of test combinations were evaluated: HRT-MRA with each of the other tests. For all analyses, a STARD flow diagram33,38 was produced which shows the flow of participants. The subset of participants who received all four tests and were considered in the statistical analyses is separated out into three groups according to whether each imaging test result was ‘abnormal’, ‘normal’ or ‘no result’ (the imaging test result being not available because either the test was inconclusive or because the result was missing). For each of these three groups, the group status according to the reference standard (‘glaucoma present’ or ‘glaucoma absent’) for each participant is given or alternatively the reference standard was stated to be missing or inconclusive. The final categorisations of the imaging test result by reference standard status provides the four possible combinations (true and false positive, and false and true negative) from which the diagnostic performance can be assessed. Sensitivity, specificity, likelihood ratios and DOR are provided with associated 95% CIs summarised for each analysis.

Of the 966 (46%) who agreed to take part in GATE, 11 were excluded from the study: 10 were ineligible and one person withdrew prior to participating in the study.

Additionally, owing to administrative and research processing errors, imaging was not implemented for all four imaging tests in 12 participants and these participants were excluded from all analyses. The analyses in this chapter pertain to the remaining 943 participants. Of these, no reference standard finding was available for 11 participants, with an inconclusive finding in a further four cases.

Default diagnosis analysis

The results for the default diagnosis analysis are presented in three sections:

• diagnostic performance of the imaging tests

• paired comparisons of imaging tests

• diagnostic performance with restricted reference standard definition of disease.

Diagnostic performance of the imaging tests

For the default analysis, abnormal imaging test results were those classified as ‘outside normal limits’ and the corresponding reference standard definition of disease was a diagnosis of glaucoma in the ‘worse’ eye. Only participants with an imaging test output with an overall classification which met the manufacturer quality criteria were included in the analysis.

The flow of study participants according to the default diagnosis analysis is shown in Figure 3, with the corresponding number of abnormal, normal and no result cases by imaging test, and the corresponding reference standard finding shown. Of the 943 patients for whom all four tests were performed, 158 were classified as disease positive and 770 as disease negative. The reference standard was missing and inconclusive for 11 and four participants, respectively. The diagnostic performance for the four tests is given in Table 16. The results showed a trade-off between detection of glaucoma and correctly identifying non-glaucoma cases: HRT-MRA had the highest sensitivity (87.0%, 95% CI 80.2% to 92.1%) but the lowest specificity (63.9%, 95% CI 60.2% to 67.4%), GDx had the lowest sensitivity (35.1%, 95% CI 27.0% to 43.8%) but the highest specificity (97.2%, 95% CI 95.6% to 98.3%) and the other two tests provided intermediate results (HRT-GPS values were very similar to the HRT-MRA results and OCT had very similar sensitivity and specificity values). Likelihood ratios (and 95% CI) showed evidence of being able to both rule in and rule out the presence of glaucoma for all four imaging tests (CIs did not contain 1.0). DORs ranged from 9.24 for HRT-GPS to 18.48 for GDx.

FIGURE 3.

Flow diagram: default diagnostic analysis. FN, false negative; FP, false positive; TN, true negative; TP, true positive.

TABLE 16 - Diagnostic performance: default diagnosis analysis

Test	Diagnostic parameter	Point estimate	95% CI
HRT-MRA	Sensitivity (%)	87.0	80.2 to 92.1
	Specificity (%)	63.9	60.2 to 67.4
	Positive likelihood ratio	2.41	2.14 to 2.71
	Negative likelihood ratio	0.20	0.13 to 0.32
	DOR	11.80	7.02 to 19.81
HRT-GPS	Sensitivity (%)	81.5	73.9 to 87.6
	Specificity (%)	67.7	64.2 to 71.2
	Positive likelihood ratio	2.53	2.21 to 2.89
	Negative likelihood ratio	0.27	0.19 to 0.39
	DOR	9.24	5.82 to 14.67
GDx	Sensitivity (%)	35.1	27.0 to 43.8
	Specificity (%)	97.2	95.6 to 98.3
	Positive likelihood ratio	12.35	7.57 to 20.14
	Negative likelihood ratio	0.67	0.59 to 0.76
	DOR	18.48	10.46 to 32.63
OCT	Sensitivity (%)	76.9	69.2 to 83.4
	Specificity (%)	78.5	75.4 to 81.4
	Positive likelihood ratio	3.58	3.04 to 4.22
	Negative likelihood ratio	0.29	0.22 to 0.40
	DOR	12.16	7.97 to 18.54

Diagnosis sensitivity analysis 1

Diagnosis sensitivity analysis 1 differed from the default analysis in that a borderline finding on the imaging test was also classified as an abnormal result.

For diagnosis sensitivity analysis 1, abnormal imaging test results were those classified as ‘outside normal limits’ and ‘borderline’, and the corresponding reference standard definition of disease was a diagnosis of glaucoma in the ‘worse’ eye. Only participants with an imaging test output with an overall classification which met the manufacturer quality cut-off point were included in the analysis.

The flow of study participants according to sensitivity analysis 1 is shown in Figure 4, with the corresponding number of abnormal, normal and no result cases given by imaging test, and the corresponding reference standard finding shown. Of the 943 patients in whom all four tests were performed, 158 were classified as disease positive and 770 as disease negative. The reference standard was missing and inconclusive for 11 and four participants, respectively. The diagnostic performance for the four tests is given in Table 22. The results showed a trade-off between detection of glaucoma and correctly identifying non-glaucoma cases: HRT-MRA had the highest sensitivity (94.9%, 95% CI 89.8% to 97.9%) but the second lowest specificity (43.9%, 95% CI 40.2% to 47.6%), GDx had the lowest sensitivity (60.4%, 95% CI 51.6% to 68.8%) but the highest specificity (82.8%, 95% CI 79.8% to 85.5%), and the other two tests provided intermediate results (HRT-GPS values were very similar to the HRT-MRA results although marginally lower and OCT had a high sensitivity and moderate specificity in relation to the other tests). Sensitivity was higher for all tests than under the default analysis but with corresponding lower specificity. Likelihood ratios (and 95% CI) showed evidence of being able to both rule in and rule out the presence of glaucoma for all four imaging tests (CIs did not contain 1.0). DORs ranged from 7.36 for GDx to 14.62 for HRT-MRA.

FIGURE 4.

Flow diagram: diagnostic sensitivity analysis 1. FN, false negative; FP, false positive; TN, true negative; TP, true positive.

Diagnosis sensitivity analysis 2

Diagnosis sensitivity analysis 2 differed from the default analysis in that the reference standard definition of disease incorporated all participants with glaucoma suspect (irrespective of type). For diagnosis sensitivity analysis 2, abnormal imaging test results were those classified as ‘outside normal limits’ and the corresponding reference standard definition of disease was a diagnosis of glaucoma in the ‘worse’ eye. Only participants with an imaging test output with an overall classification which met the manufacturer quality cut-off point were included in the analysis.

The flow of study participants according to sensitivity analysis 2 is shown in Figure 5, with the corresponding number of abnormal, normal and ‘no result’ cases by imaging test, and the corresponding reference standard finding shown. Of the 943 patients in whom all four tests were performed, 400 were classified as disease positive and 528 as disease negative. The reference standard was missing and inconclusive for 11 and four participants, respectively. The diagnostic performance of the four tests is given in Table 23. The results showed a trade-off between detection of glaucoma and correctly identifying non-glaucoma cases: HRT-MRA had the highest sensitivity (74.0%, 95% CI 69.1% to 78.5%) but lowest specificity (76.5%, 95% CI 72.5% to 80.1%), GDx had the lowest sensitivity (16.5%, 95% CI 12.8% to 20.8%) but the highest specificity (98.2%, 95% CI 96.5% to 99.2%) and the other two tests provided intermediate results (HRT-GPS had lower sensitivity than HRT-MRA but a slightly higher specificity and OCT had the second lowest sensitivity but the second highest specificity values). Sensitivity was lower for all tests than under the default analysis but with correspondingly higher specificity. Likelihood ratios (and 95% CI) showed evidence of being able to both rule in and rule out the presence of glaucoma for all four imaging tests (CIs did not contain 1.0). DORs ranged from 5.44 for OCT to 10.51 for GDx.

FIGURE 5.

Flow diagram: diagnostic sensitivity analysis 2. FN, false negative; FP, false positive; TN, true negative; TP, true positive.

Diagnosis sensitivity analysis 3

Diagnosis sensitivity analysis 3 differed from the default analysis in that a borderline finding on the imaging test was classified as an abnormal test result and the reference standard definition of disease incorporated all glaucoma suspects (irrespective of type).

For diagnosis sensitivity analysis 3, abnormal imaging test results were those classified as ‘outside normal limits’ or ‘borderline’ and the corresponding reference standard definition of disease was a diagnosis of glaucoma or glaucoma suspect in the ‘worse’ eye. Only participants with an imaging test output with an overall classification which met the manufacturer quality cut-off point were included in the analysis.

The flow of study participants according to sensitivity analysis 3 is shown in Figure 6, with the corresponding number of abnormal, normal and no result cases by imaging test, and the corresponding reference standard finding shown. Of the 943 patients in whom all four tests were performed, 400 were classified as disease positive and 528 as disease negative. The reference standard was missing and inconclusive for 11 and four participants, respectively. The diagnostic performance of the four tests is given in Table 24. The results showed a trade-off between detection of glaucoma and correctly identifying non-glaucoma cases: HRT-MRA had the highest sensitivity (88.9%, 95% CI 85.1% to 92.0%) but the second lowest specificity (56.1%, 95% CI 51.6% to 60.6%), GDx had the lowest sensitivity (39.0%, 95% CI 33.9% to 44.4%) but the highest specificity (86.7%, 95% CI 83.3% to 89.5%) and the other two tests provided intermediate results (HRT-GPS values were very similar to the HRT-MRA results and OCT had very similar sensitivity and specificity values). Sensitivity was slightly higher for GDx, HRT-GPS and HRT-MRA than under the default analysis but with correspondingly lower specificity. OCT, however, had a slightly lower sensitivity and specificity than under the default analysis.

FIGURE 6.

Flow diagram: diagnostic sensitivity analysis 3. FN, false negative; FP, false positive; TN, true negative; TP, true positive.

Likelihood ratios (and 95% CI) showed evidence of being able to both rule in and rule out the presence of glaucoma for all four imaging tests (CIs did not contain 1.0). DORs ranged from 4.04 for OCT to 10.21 for HRT-MRA.

Diagnosis sensitivity analysis 4

Diagnosis sensitivity analysis 4 has the same reference standard and definition of an abnormal imaging test as sensitivity analysis 3 differing by including the imaging test-related ‘no result’ cases (the overall classification was used irrespective of the quality indicator and the types were all classified as abnormal).

For diagnosis sensitivity analysis 4, abnormal imaging test results were those classified as ‘outside normal limits’ or ‘borderline’ and the corresponding reference standard definition of disease was a diagnosis of glaucoma or glaucoma suspect in the ‘worse’ eye. The analysis included participants with a low-quality imaging output if a classification was given; other imaging test results which did not provide an overall classification were included as abnormal.

The flow of study participants according to sensitivity analysis 4 is shown in Figure 7, with the corresponding number of abnormal, normal and no result cases and the corresponding reference standard finding shown. Of the 943 patients in whom all four tests were performed, 400 were classified as disease positive and 528 as disease negative. The reference standard was missing and inconclusive for 11 and four participants, respectively. The diagnostic performance for the four tests is given in Table 25.

FIGURE 7.

Flow diagram: diagnostic sensitivity analysis 4. FN, false negative; FP, false positive; TN, true negative; TP, true positive.

The results showed a trade-off between detection of glaucoma and correctly identifying non-glaucoma cases: HRT-MRA had the highest sensitivity (89.2%, 95% CI 85.7% to 92.1%) but second lowest specificity (55.1%, 95% CI 50.7% to 59.5%), GDx had the lowest sensitivity (41.9%, 95% CI 37.0% to 47.0%) but the highest specificity (85.6%, 95% CI 82.3% to 88.5%) and the other two tests provided intermediate results (HRT-GPS values were similar to the HRT-MRA results and OCT had similar sensitivity and specificity values). Sensitivity was higher for all tests than under the default analysis but with correspondingly lower specificity. Likelihood ratios (and 95% CI) showed evidence of being able to both rule in and rule out the presence of glaucoma for all four imaging tests (CIs did not contain 1.0). DORs ranged from 3.89 for OCT to 10.19 for HRT-MRA.

Diagnosis sensitivity analysis 5

Diagnosis sensitivity analysis 5 differed from the default analysis in that the imaging test-related ‘no result’ cases were included as above for sensitivity analysis 4.

For sensitivity analysis 5, abnormal imaging test results were those classified as ‘outside normal limits’ and the corresponding reference standard definition of disease was a diagnosis of glaucoma in the ‘worse’ eye. The analysis included participants with a low-quality imaging output if a classification was given; other imaging test results which did not provide an overall classification were included as abnormal.

The flow of study participants according to sensitivity analysis 5 is shown in Figure 8, with the corresponding number of abnormal, normal and no result cases by imaging test, and the corresponding reference standard finding shown. Of the 943 patients in whom all four tests were performed, 158 were classified as disease positive and 770 as disease negative. The reference standard was missing and inconclusive for 11 and four participants, respectively. The diagnostic performance of the four tests is given in Table 26. The results showed a trade-off between detection of glaucoma and correctly identifying non-glaucoma cases: HRT-MRA had the highest sensitivity (87.3%, 95% CI 81.0% to 92.0%) but lowest specificity (61.8%, 95% CI 58.2% to 65.3%), GDx had the lowest sensitivity (37.6%, 95% CI 30.0% to 45.7%) but the highest specificity (95.4%, 95% CI 93.7% to 96.8%) and the other two tests provided intermediate results (HRT-GPS values were very similar to the HRT-MRA results and OCT had very similar sensitivity and specificity values). Likelihood ratios (and 95% CI) showed evidence of being able to both rule in and rule out the presence of glaucoma for all four imaging tests (CIs did not contain 1.0). DORs ranged from 8.96 for HRT-GPS to 12.47 for GDx.

FIGURE 8.

Flow diagram: diagnostic sensitivity analysis 5. FN, false negative; FP, false positive; TN, true negative; TP, true positive.

Diagnosis sensitivity analysis 6

Diagnosis sensitivity analysis 6 differed from the default analysis in that the diagnosis of the participants’ ‘better’ eye according to the reference standard was used. Abnormal imaging test results were those classified as ‘outside normal limits’ and the corresponding reference standard definition of disease was a diagnosis of glaucoma. Only participants with an imaging test output with an overall classification which met the manufacturer quality cut-off point were included in the analysis.

The flow of study participants according to sensitivity analysis 6 is shown in Figure 9, with the corresponding number of abnormal, normal and ‘no result’ cases by imaging test, and the corresponding reference standard finding shown. Of the 943 patients in whom all four tests were performed, 61 were classified as disease positive and 862 as disease negative. The reference standard was missing and inconclusive for 12 and 8 participants, respectively. The diagnostic performance of the four tests is given in Table 27. The results showed a trade-off between detection of glaucoma and correctly identifying non-glaucoma cases: HRT-GPS had the highest sensitivity (82.4%, 95% CI 69.1% to 91.6%) but also the second lowest specificity (67.8%, 95% CI 64.5% to 77.1%), GDx had the lowest sensitivity (26.9%, 95% CI 15.6% to 41.0%) but the highest specificity (96.7%, 95% CI 95.2% to 97.8%) and the other two tests provided intermediate results (HRT-MRA had a slightly lower sensitivity and specificity than HRT-GPS but a slightly higher specificity and OCT had the second lowest sensitivity but the second highest specificity values). Sensitivity was slightly lower for all HRT-MRA, GDx and OCT than under the default analysis but with a slightly higher specificity. HRT-GPS has very similar sensitivity analysis results to the default (primary) analysis. Likelihood ratios (and 95% CI) showed evidence of being able to both rule in and rule out the presence of glaucoma for all four imaging tests (CIs did not contain 1.0). DORs ranged from 6.85 for HRT-MRA to 10.83 for GDx.

FIGURE 9.

Flow diagram: diagnostic sensitivity analysis 6. FN, false negative; FP, false positive; TN, true negative, TP, true positive.

Combinations of imaging tests

The HRT-MRA test was combined with the other imaging tests to form three combined tests and the diagnostic performance was assessed. The reference standard and the definition of an abnormal imaging test result was the same as for the default analysis (abnormal imaging test ‘outside normal limits’; reference standard diagnosis of glaucoma in the ‘worse’ eye; and only participants with an imaging test output with an overall classification which met the manufacturer quality cut-off point were included in the analysis). The corresponding flow of study participants is shown in Figure 10, with the corresponding number of abnormal, normal and no results cases by combination imaging test and the corresponding reference standard finding shown. The diagnostic performance of the four tests is given in Table 28. The results showed a trade-off between detection of glaucoma and correctly identifying non-glaucoma cases: HRT-MRA combined with OCT had the highest sensitivity (91.7%, 95% CI 85.7% to 95.8%) but the second lowest specificity (53.8%, 95% CI 50.0% to 57.5%) and HRT-MRA combined with GDx had the lowest sensitivity (89.5%, 95% CI 82.7% to 94.3%) but the highest specificity (62.8%, 95% CI 59.0% to 66.5%). Likelihood ratios (and 95% CI) showed evidence of being able to both rule in and rule out the presence of glaucoma all three combination imaging tests (CIs did not contain 1.0). DORs ranged from 11.34 for HRT-MRA combined with HRT-GPS, to 14.43 for HRT-MRA combined with GDx.

FIGURE 10.

Flow diagram: combination of imaging tests. FN, false negative; FP, false positive; TN, true negative; TP, true positive.

Discussion

The diagnostic performance of four imaging tests (HRT-MRA, HRT-GPS, GDx and OCT) for the detection of glaucoma was compared for the GATE population of referrals to a glaucoma clinic in secondary care. The sensitivity and specificity of the four imaging tests for the default diagnosis analysis and sensitivity analyses (see Table 15 for details) are summarised in Figures 11 and 12, respectively.

FIGURE 11.

Summary of the sensitivity of imaging tests across all diagnosis analyses. SA, sensitivity analyses.

FIGURE 12.

Summary of the specificity of imaging tests across all diagnosis analyses. SA, sensitivity analyses.

All four imaging tests had some value in terms of ruling in and ruling out the presence of glaucoma. However, the diagnostic performance of the imaging tests differed in the ability to correctly diagnose glaucoma (sensitivity) and non-glaucoma cases (specificity). HRT-MRA had the highest sensitivity across analyses, except when the reference standard diagnosis was moderate and severe glaucoma only, when HRT-GPS was higher, but at a cost of lower specificity compared with other tests. In contrast, GDx consistently had the best specificity but the lowest sensitivity. HRT-GPS results were typically similar to HRT-MRA as might be expected given that their analysis is based on the same imaging machine. The sensitivity of OCT was generally of a similar magnitude to its specificity. When the reference standard definition of disease excluded mild glaucoma, OCT displayed better diagnostic performance than HRT-GPS and HRT-MRA, with GDx providing the best specificity. The choice of which imaging test is to be preferred reflects the inherent trade-off regarding diagnostic testing, when the desire not to miss glaucoma when present must be balanced again the desire to correctly identify those who are without disease.

The non-diagnostic outcomes tended to favour OCT. OCT had the lowest number of low-quality imaging results, with GDx having the highest. Average time taken to conduct the tests was lowest for OCT with the other tests taking a similar length of time. Less dilatation was required for OCT, followed by GDx then the HRT tests. Considering the time taken and need for dilatation, patient preference tended to favour OCT followed by GDx, although almost one-half of participants did not have a preference.

Glaucome Automated Test Evaluation was a large prospective paired diagnostic study and provided diagnostic tests in this desired setting. This is reflected in the precision in which the sensitivity and specificity were calculated with differences between every pair of tests identified for one if not both sensitivity and specificity. McNemar’s test38 was used to compare the sensitivity and specificity of the tests. Following the rationale of others in effectiveness studies, the paired comparisons were not adjusted for multicomparisons. Even if such a correction were to have been applied such was the strength of evidence there would still be evidence of differences in the diagnostic performance of the different imaging tests.

A number of sensitivity analyses were carried out to assess the robustness of the findings of the default analysis. Varying the test definition of an abnormal imaging result by including the borderline category was carried out; this had the anticipated impact of improving the detection of glaucoma, although at the expense of more participants without glaucoma being falsely classified as having glaucoma. This resulted in very high detection of glaucoma for HRT-MRA, HRT-GPS and OCT but with low to moderate diagnosis of non-glaucoma cases. GDx provided moderate performance for both detecting glaucoma and correctly diagnosing non-glaucoma cases. Additionally, the impact of also seeking to diagnose glaucoma suspect (based on optic disc and/or visual field findings as described in Chapter 2) was assessed both with and without classifying borderline imaging findings as abnormal. When the test definition of abnormal incorporated the borderline category, the net impact was a slight increase in sensitivity for GDx, HRT-GPS and HRT-MRA, with the sensitivity of OCT slightly reduced compared with the default analysis, suggesting that the OCT test deals less well with glaucoma suspect cases. The diagnostic performance on the better eye gave similar results, although with generally a lower sensitivity and slightly higher specificity than for the worse eye. HRT-GPS diagnostic performance for these data was remarkably similar to when the worse eye was used.

Finally, the impact of using a combination of tests was assessed. Given the findings of the default diagnosis analysis and associated sensitivity analyses, this was restricted to an assessment of whether or not using another imaging test in addition to HRT-MRA appeared to be beneficial. Although the additional use of another test led to improved detection of glaucoma, the improvement was marginal and smaller than the loss in terms of the handling of non-diseased cases and although the use of two tests in combination did have some benefit in terms of reducing the number of no result cases, the change in diagnostic performance coupled with the additional practical and cost implications in terms of training and staff time, and an additional requirement of equipment (for two of the three combinations) suggests that the use of a single test is to be preferred.

A number of assumptions underpinned the analysis and interpretation of the results. Most importantly, the reference standard was assumed to be perfect although it is widely recognised that diagnosis of glaucoma is difficult and uncertainty exists even among specialists. While consensus was sought through structured training, some assessor differences may have remained between the sites. Additionally, the diagnosis and clinical management of patients with glaucoma suspect is uncertain; in particular, the risk of conversion of such individuals is not known. Nevertheless, the findings provide evidence reflective of current clinical practice in NHS glaucoma clinics.

A number of areas for further research are clear. Further investigation of varying the results of the imaging tests beyond the standard options could be undertaken, as the recommended classification may not be the one best suited to the population that GATE recruited from. The definition and clinical management of glaucoma suspects is also an area in which further research is needed, in particular quantifying the proportion that will convert or will be discharged from clinical care over subsequent years. Finally, the diagnosis value of using an imaging test explicitly in a triage scenario with the additional use of an IOP measurement and VA to form a composite triage test requires evaluation.

Overview

This chapter reports the results of the triage analyses, which aimed to assess the diagnostic performance of the four imaging tests in a triage setting. The specific diagnostic performance analyses covered in this chapter are the default triage analysis (Table 29, Default triage analysis) along with eight sensitivity analyses (see Table 29, Triage sensitivity analyses 1–8) for a list with definitions. A further set of three analyses specifically to inform the economic model are described in Appendix 6. The default triage analysis was defined as one in which the reference standard was the person-level clinical decision (‘not discharged’ or ‘discharged’). The test was defined as categorising a patient as requiring to be referred on (‘for referral’) if any of the elements of the composite triage test (imaging, IOP and/or VA) were themselves ‘abnormal’: imaging outside normal limits on the overall classification of the imaging test (see Chapter 2), IOP > 21 mmHg or VA of 6/12 or poorer under the default triage analysis.

If the imaging test did not produce an overall classification or its quality was poor, the imaging test result was again defined as abnormal and, therefore, the patient was classified as ‘for referral’. The eight sensitivity analyses assessed the impact of varying assumptions made in the default triage analysis relating to the definition of a positive test result, modifying or removing the IOP and/or VA components of the triage test, and how cases where the test did not produce an overall classification were handled in the analysis.

The analyses in this chapter pertain to the 943 participants remaining in the study (see Chapter 4). The reference standard was available for 933 cases. For all analyses, a STARD diagram shows the flow of participants. The subset of participants who received all four tests and were considered in the statistical analyses are separated out into three groups according to whether each triage test result was ‘abnormal’, ‘normal’ or ‘no result’ (the triage test result was not available because either the test was inconclusive or the result was missing). For each of these three groups the group status according to the reference standard (‘discharged’ or ‘not discharged’) for each participant is given or alternatively the reference standard was stated to be missing or inconclusive. The final categorisations of the triage test result by reference standard status provides the four possible combinations (true and false positive, false and true negative) from which the diagnostic performance was assessed. Sensitivity, specificity, likelihood ratios and DOR are provided with associated 95% CIs for each analysis.

Default triage analysis

The results for the default triage analysis are presented in two sections:

• diagnostic performance of the triage tests, and

• paired comparisons of triage tests.

Diagnostic performance of the triage tests

For the default triage analysis, the triage test is classified as abnormal if (1) the imaging test result is classified as ‘outside normal limits’, (2) IOP is > 21 mmHg or (3) VA is 6/12 or poorer. Imaging test results that did not provide an overall classification were included as abnormal. The corresponding reference standard definition is a clinical decision not to discharge the patient.

The flow of study participants according to the default triage analysis is shown in Figure 13, with the corresponding numbers of referral, not for referral and no result cases by triage test and the corresponding reference standard finding shown. Of the 943 participants in whom all four tests were performed, 576 were not discharged and 357 were discharged and the discharge status was missing for 10 participants. The diagnostic performance of the four tests is given in Table 30. The results showed a trade-off between the detection of patients who need to be referred and the discharge of those who do not need to be referred: HRT-GPS had the highest sensitivity (86.0%, 95% CI 82.8% to 88.7%) but lowest specificity (39.1%, 95% CI 34.0% to 44.5%), GDx had the lowest sensitivity (64.7%, 95% CI 60.7% to 68.7%) but the highest specificity (53.6%, 95% CI 48.2% to 58.9%), and the other two tests provided intermediate results [HRT-MRA values were very similar to the HRT-GPS results, as might be expected given that they use the same machine, and OCT had lower sensitivity (75.4%, 95% CI 71.9 to 78.9) but higher specificity (41%, 95% CI 35.8 to 46.3) values than HRT-GPS and HRT-MRA]. Likelihood ratios (and 95% CI) showed evidence of being able to both rule in and rule out the presence of glaucoma for all four triage tests (CIs did not contain 1.0). DORs ranged from 2.12 for GDx and OCT to 3.94 for HRT-GPS.

FIGURE 13.

Flow diagram: default triage analysis. FN, false negative; FP, false positive; TN, true negative; TP, true positive.

TABLE 30 - Diagnostic performance: default triage analysis

Test	Diagnostic parameter	Value	95% CI
HRT-MRA	Sensitivity (%)	85.6	82.4 to 88.4
	Specificity (%)	34.2	29.20 to 39.5
	Positive likelihood ratio	1.3	1.20 to 1.41
	Negative likelihood ratio	0.4	0.33 to 0.54
	DOR	3.09	2.23 to 4.27
HRT-GPS	Sensitivity (%)	86.0	82.8 to 88.7
	Specificity (%)	39.1	34.0 to 44.5
	Positive likelihood ratio	1.41	1.29 to 1.55
	Negative likelihood ratio	0.36	0.28 to 0.46
	DOR	3.94	2.86 to 5.42
GDx	Sensitivity (%)	64.7	60.7 to 68.7
	Specificity (%)	53.6	48.2 to 58.9
	Positive likelihood ratio	1.39	1.23 to 1.59
	Negative likelihood ratio	0.66	0.57 to 0.76
	DOR	2.12	1.62 to 2.78
OCT	Sensitivity (%)	75.4	71.6 to 78.9
	Specificity (%)	41.0	35.8 to 46.3
	Positive likelihood ratio	1.28	1.16 to 1.41
	Negative likelihood ratio	0.60	0.50 to 0.73
	DOR	2.13	1.60 to 2.83

Triage sensitivity analysis 1

Triage sensitivity analysis 1 differed from the default triage analysis in that a borderline finding on the imaging test was also classified as an abnormal result.

For triage sensitivity analysis 1, the triage test is classified as abnormal if (1) the imaging test result is classified as ‘outside normal limits’ or ‘borderline’, (2) IOP is > 21 mmHg or (3) VA is 6/12 or poorer. Imaging test results which did not provide an overall classification were included as abnormal. The corresponding reference standard definition is a clinical decision not to discharge the patient.

The flow of study participants according to triage sensitivity analysis 1 is shown in Figure 14, with the corresponding numbers of referral, not for referral and no result cases by triage test, and the corresponding reference standard finding shown. Of the 943 participants in whom all four tests were performed, 576 were not discharged and 357 were discharged and the discharge status was missing for 10 participants. The diagnostic performance for the four tests is given in Table 32. The results generally showed a trade-off between the detection of patients who need to be referred and the discharge of those who do not need to be referred: HRT-GPS had the highest sensitivity (94.0%, 95% CI 91.8% to 95.8%) but second lowest specificity (24.9%, 95% CI 20.4% to 29.7%), GDx had the lowest sensitivity (74.9%, 95% CI 71.1% to 78.4%) but the highest specificity (45%, 39.7% CI 39% to 50.4%), and the other two tests provided intermediate results (HRT-MRA values were very similar though marginally inferior to the HRT-GPS results, and OCT had lower sensitivity (84.2%, 95% CI 80.9 to 87.1) but slightly higher specificity than HRT-GPS and HRT-MRA). Likelihood ratios (and 95% CI) showed evidence of being able to both rule in and rule out the presence of glaucoma for all four triage tests (CIs did not contain 1.0). DORs ranged from 2.04 for OCT to 5.21 for HRT-GPS.

FIGURE 14.

Flow diagram: triage sensitivity analysis 1. FN, false negative; FP, false positive; TN, true negative; TP, true positive.

Triage sensitivity analysis 2

Triage sensitivity analysis 2 has the same reference standard and definition of abnormal test result as the default analysis but did not include all no result cases (see Table 33).

For triage sensitivity analysis 2, the triage test is classified as abnormal if (1) the imaging test result is classified as ‘outside normal limits’, (2) IOP is > 21 mmHg or (3) VA is 6/12 or poorer. Poor-quality imaging test results were included, and those where an image was acquired but no classification generated were included as abnormal. All other missing imaging results were excluded. The corresponding reference standard definition is a clinical decision not to discharge the patient.

The flow of study participants according to triage sensitivity analysis 2 is shown in Figure 15, with the corresponding numbers of referral, not for referral and no result cases by triage test and the corresponding reference standard finding shown. Of the 943 participants in whom all four tests were performed, 481 were not discharged and 562 were discharged and the discharge status was missing for 10 participants. The diagnostic performance for the four tests is given in Table 33. The results generally showed a trade-off between the detection of patients who need to be referred and the discharge of those who do not need to be referred: HRT-GPS had the highest sensitivity (84.6%, 95% CI 81.4% to 87.5%) but the second lowest specificity (39.7%, 95% CI 34.6% to 45.1%), GDx had the lowest sensitivity (61.1%, 95% CI 56.9% to 65.1%) but the highest specificity (59.0%, 95% CI 53.7% to 64.2%) and the other two tests provided intermediate results [HRT-MRA values were very similar, although slightly inferior to the HRT-GPS results, and OCT had the second lowest sensitivity (75.0%, 95% CI 71.3% to 78.5%) but the second highest specificity (42.1%, 95% CI 36.9% to 47.4%) values]. Likelihood ratios (and 95% CI) showed evidence of being able to both rule in and rule out the presence of glaucoma for all four triage tests (CIs did not contain 1.0). DORs ranged from 2.19 for GDx to 3.61 for OCT.

FIGURE 15.

Flow diagram: triage sensitivity analysis 2. FN, false negative; FP, false positive; TN, true negative; TP, true positive.

Triage sensitivity analysis 3

Triage sensitivity analysis 3 was the same as triage sensitivity analysis 2 except that ‘borderline’ test results were also classified as abnormal.

For triage sensitivity analysis 3, the triage test is classified as abnormal if (1) the imaging test result is classified as ‘outside normal limits’ or ‘borderline’, (2) IOP is > 21 mmHg or (3) VA is 6/12 or poorer. Poor-quality imaging test results were included, and those where an image was acquired but no classification generated were included as abnormal. All other missing imaging results were excluded. The corresponding reference standard definition is a clinical decision not to discharge the patient.

The flow of study participants according to triage sensitivity analysis 3 is shown in Figure 16, with corresponding numbers of referral, not for referral and no result cases by triage test and the corresponding reference standard finding shown. Of the 943 participants in whom all four tests were performed, 481 were not discharged and 562 were discharged and the discharge status was missing for 10 participants. The diagnostic performance for the four tests is given in Table 34. The results generally showed a trade-off between the detection of patients who need to be referred and the discharge of those who do not need to be referred: HRT-GPS had the highest sensitivity (93.3%, 95% CI 91.0% to 95.2%) but second lowest specificity (24.9%, 95% CI 20.4% to 29.7%), GDx had the lowest sensitivity (72.3%, 95% CI 68.4% to 75.9%) but the highest specificity (49.0%, 95% CI 43.6% to 54.4%) and the other two tests provided intermediate results [HRT-MRA values were very similar to the HRT-GPS results, although slightly inferior, and OCT had the second lowest sensitivity (84.2%, 95% CI 80.9% to 87.1%) but the second highest specificity]. Likelihood ratios (and 95% CI) showed evidence of being able to both rule in and rule out the presence of glaucoma for all four triage tests (CIs did not contain 1.0). DORs ranged from 2.12 for OCT to 4.63 for HRT-GPS.

FIGURE 16.

Flow diagram: triage sensitivity analysis 3. FN, false negative; FP, false positive; TN, true negative; TP, true positive.

Triage sensitivity analysis 4

Triage sensitivity analysis 4 differed from the default triage analysis in that referral IOP > 21 mmHg rather than clinician IOP > 21 mmHg was used to identify abnormal tests. The triage test is classified as abnormal if (1) the imaging test result is classified as ‘outside normal limits’, (2) referral IOP is > 21 mmHg or (3) VA is 6/12 or poorer. Imaging test results which did not provide an overall classification were included as abnormal. The corresponding reference standard definition is a clinical decision not to discharge the patient.

The flow of study participants according to triage sensitivity analysis 4 is shown in Figure 17, with the corresponding numbers of referral, not for referral and no result cases by triage test and the corresponding reference standard finding shown. Of the 943 participants in whom all four tests were performed, 481 were not discharged and 562 were discharged and the discharge status was missing for 10 participants. The diagnostic performance for the four tests is given in Table 35. The results generally showed a trade-off between the detection of patients who need to be referred and the discharge of those who do not need to be referred: HRT-GPS had the highest sensitivity (86.5%, 95% CI 83.4% to 89.2%) but second lowest specificity (24.0%, 95% CI 19.6% to 28.8%), GDx had the lowest sensitivity (67.2%, 95% CI 63.2% to 71.0%) but the highest specificity (35.8%, 95% CI 30.8% to 41.1%) and the other two tests provided intermediate results (HRT-MRA values were very similar to the HRT-GPS results, although slightly inferior, and OCT had the second lowest sensitivity (76.8%, 95% CI 73.1% to 80.2%) but the second highest specificity (27.7%, 95% CI 23.1% to 32.7%). Likelihood ratios (and 95% CI) showed evidence of being able to both rule in and rule out the presence of glaucoma for all four triage tests (CIs did not contain 1.0). DORs ranged from 1.14 for GDx to 2.02 for HRT-GPS.

FIGURE 17.

Flow diagram: triage sensitivity analysis 4. FN, false negative; FP, false positive; TN, true negative; TP, true positive.

Triage sensitivity analysis 5

Triage sensitivity analysis 5 differed from the default triage analysis in that the IOP component was removed from the composite triage test. The triage test is classified as abnormal if the imaging test result is classified as (1) ‘outside normal limits’ or (2) VA is 6/12 or poorer. Imaging test results which did not provide an overall classification were included as abnormal. The corresponding reference standard definition is a clinical decision not to discharge the patient.

The flow of study participants according to triage sensitivity analysis 5 is shown in Figure 18, with the corresponding numbers of referral, not for referral and no result cases by triage test and the corresponding reference standard finding shown. Of the 933 participants in whom all four tests were performed, 481 were not discharged and 562 were discharged. The discharge status was missing for 10 participants. The diagnostic performance for the four tests is given in Table 36. The results generally showed a trade-off between the detection of patients who need to be referred and the discharge of those who do not need to be referred: HRT-MRA had the highest sensitivity (68.9%, 95% CI 64.9% to 72.7%) but the lowest specificity (52.3%, 95% CI 46.9% to 57.7%), GDx had the lowest sensitivity (32.8%, 95% CI 29.0% to 36.8%) but the highest specificity (81.1%, 95% CI 76.6% to 85.1%) and the other two tests provided intermediate results (HRT-GPS values were very similar to the HRT-MRA results and OCT had the second lowest sensitivity but the second highest specificity). Likelihood ratios (and 95% CI) showed evidence of being able to both rule in and rule out the presence of glaucoma for all four triage tests (CIs did not contain 1.0). DORs ranged from 1.80 for OCT to 2.91 for HRT-GPS.

FIGURE 18.

Flow diagram: triage sensitivity analysis 5. FN, false negative; FP, false positive; TN, true negative; TP, true positive.

Triage sensitivity analysis 6

Triage sensitivity analysis 6 differed from the default triage analysis in that the VA component was removed from the composite triage test. The triage test is classified as abnormal if (1) the imaging test result is classified as ‘outside normal limits’ or (2) IOP is > 21 mmHg. Imaging test results which did not provide an overall classification were included as abnormal. The corresponding reference standard definition is a clinical decision not to discharge the patient.

The flow of study participants according to triage sensitivity analysis 6 is shown in Figure 19, with corresponding numbers of referral, not for referral and no result cases by triage test and the corresponding reference standard finding shown. Of the 943 participants in whom all four tests were performed, 481 were not discharged and 562 were discharged and the discharge status was missing for 10 participants. The diagnostic performance for the four tests is given in Table 37. The results generally showed a trade-off between the detection of patients who need to be referred and the discharge of those who do not need to be referred: HRT-MRA had the highest sensitivity (84.9%, 95% CI 81.9% to 87.7%) but second lowest specificity (37.4%, 95% CI 32.3% to 42.8%), GDx had the lowest sensitivity (60.5%, 95% CI 56.4% to 64.6%) but the highest specificity (57.6%, 95% CI 52.2% to 62.8%), and the other two tests provided intermediate results (HRT-GPS values were very similar to the HRT-MRA results and OCT had the second lowest sensitivity but the second highest specificity). Likelihood ratios (and 95% CI) showed evidence of being able to both rule in and rule out the presence of glaucoma for all four triage tests (CIs did not contain 1.0). DORs ranged from 2.03 for OCT to 3.97 for HRT-GPS.

FIGURE 19.

Flow diagram: triage sensitivity analysis 6. FN, false negative; FP, false positive; TN, true negative; TP, true positive.

Triage sensitivity analysis 7

Triage sensitivity analysis 7 differed from the default triage analysis in that a higher IOP threshold of 26 mmHg rather than 21 mmHg was used to identify abnormal tests. The triage test is classified as abnormal if (1) the imaging test result is classified as ‘outside normal limits’, (2) IOP is > 26 mmHg or (3) VA is 6/12 or poorer. Imaging test results which did not provide an overall classification were included as abnormal. The corresponding reference standard definition is a clinical decision not to discharge the patient.

The flow of study participants according to triage sensitivity analysis 7 is shown in Figure 20, with the corresponding numbers of referral, not for referral and no result cases by triage test and the corresponding reference standard finding shown. Of the 943 participants in whom all four tests were performed, 481 were not discharged and 562 were discharged and the discharge status was missing for 10 participants. The diagnostic performance for the four tests is given in Table 38. The results generally showed a trade-off between the detection of patients who need to be referred and the discharge of those who do not need to be referred: HRT-MRA had the highest sensitivity (77.2%, 95% CI 73.5% to 80.6%) but second lowest specificity (51.8%, 95% CI 46.3% to 57.2%), GDx had the lowest sensitivity (47.9%, 95% CI 43.7% to 52.1%) but the highest specificity (79.1%, 95% CI 74.4% to 81.2%), and the other two tests provided intermediate results (HRT-GPS values were very similar to the HRT-MRA results and OCT had very similar sensitivity and specificity). Likelihood ratios (and 95% CI) showed evidence of being able to both rule in and rule out the presence of glaucoma for all four triage tests (CIs did not contain 1.0). DORs ranged from 2.61 for OCT to 4.03 for HRT-GPS.

FIGURE 20.

Flow diagram: triage sensitivity analysis 7. FN, false negative; FP, false positive; TN, true negative; TP, true positive.

Triage sensitivity analysis 8

Triage sensitivity analysis 8 differed from the default triage analysis in that a higher VA threshold of VA 6/18 or poorer was used to identify abnormal tests. The triage test is classified as abnormal if (1) the imaging test result is classified as ‘outside normal limits’, (2) IOP is > 21 mmHg or (3) VA is 6/18 or poorer. Imaging test results which did not provide an overall classification were included as abnormal. The corresponding reference standard definition is a clinical decision not to discharge the patient.

The flow of study participants according to triage sensitivity analysis 8 is shown in Figure 21, with the corresponding numbers of referral, not for referral and no result cases by triage test and the corresponding reference standard finding shown. Of the 943 participants in whom all four tests were performed, 481 were not discharged and 562 were discharged and the discharge status was missing for 10 participants. The diagnostic performance for the four tests is given in Table 39. The results showed a trade-off between the detection of patients who need to be referred and the discharge of those who do not need to be referred: HRT-MRA had the highest sensitivity (85.1%, 95% CI 81.8% to 87.9%) but lowest specificity (35.1%, 95% CI 30.0% to 40.4%), GDx had the lowest sensitivity (61.9%, 95% CI 57.8% to 65.9%) but the highest specificity (55.6%, 95% CI 50.2% to 60.9%) and the other two tests provided intermediate results (HRT-GPS values were very similar to the HRT-MRA results, and OCT had the second lowest sensitivity (72.9%, 95% CI 69.1% to 76.5%) but the second highest specificity (42.9%, 95% CI 37.7% to 48.3%). Likelihood ratios (and 95% CI) showed evidence of being able to both rule in and rule out the presence of glaucoma for all four triage tests (CIs did not contain 1.0). DORs ranged from 2.03 for OCT to 3.80 for HRT-GPS.

FIGURE 21.

Flow diagram: triage sensitivity analysis 8. FN, false negative; FP, false positive; TN, true negative; TP, true positive.

Discussion

Four composite triage (imaging, IOP measurement and VA assessment) tests were compared with regard to their diagnostic performance for determining who should be referred for further assessment or discharged using the GATE population of referrals to a glaucoma clinic in secondary care.

The sensitivity and specificity of the four triage tests incorporating each of the imaging technologies along with IOP and VA for the default triage analysis and sensitivity analyses (see Table 29 for details) are summarised in Figures 22 and 23, respectively.

FIGURE 22.

Summary of the sensitivity of the composite test across all triage analyses. SA, sensitivity analysis.

FIGURE 23.

Summary of the specificity of the composite test across all triage analyses. SA, sensitivity analysis.

All four triage tests had value in terms of ruling in and ruling out the need for referral on to a consultant ophthalmologist. The diagnostic performance of the triage tests differed with substantial differences in the ability to correctly detect those who need to be referred and those who do not. HRT-GPS and HRT-MRA consistently had the highest sensitivities across analyses but at a cost of lower specificity than other tests. HRT-GPS had the slightly higher specificity. In contrast, GDx consistently had the best specificity, although the lowest sensitivity. HRT-GPS results were typically similar to HRT-MRA. OCT generally had similar levels of sensitivity and specificity. The choice of which triage test is to be preferred reflects the inherent trade-off regarding diagnostic testing, where the desire to refer onwards when referral is needed must be balanced again the desire to discharge those who do not need a further assessment. A formal assessment of this trade-off and the consequences in terms of health outcome and costs is covered in Chapters 6 and 7.

The triage was formed from three components, an imaging test as evaluated in Chapter 4, a measurement of IOP and VA measurement. The elements were combined in an additive manner where an individual was referred if any one of the three components met the relevant referral criteria. A number of sensitivity analyses were carried out to assess the robustness of the findings of this default triage analysis. Varying the imaging test definition of a positive result by including the borderline category of imaging test result was carried out; this had the expected impact of improving the detection of glaucoma, although at the cost of more non-glaucoma cases being falsely identified as having glaucoma. This resulted in very high detection of glaucoma for HRT-MRA, HRT-GPS and high sensitivities for GDx and OCT but the consequence of lower specificities (GDx had a higher specificity value than the other three triage tests). Additionally, the impact of using the classification from the imaging test when the quality criterion was not met was assessed. The impact was at most a small reduction in sensitivity with an increase in specificity (only GDx had more than a nominal change in values). The added value of the IOP and VA components was assessed by dropping one of the components, varying the cut-off point used to define abnormality, and, for the IOP component, using the referral IOP measurement in place of the ophthalmologist’s. Removal of the IOP component had a noticeable impact on the diagnostic performance with exclusion leading to a reduction in sensitivity, although a gain in specificity. Modifying the IOP cut-off value changed the balance in terms of sensitivity and specificity as expected. When the referral IOP was used in place of the ophthalmologist’s IOP the specificity was reduced. Such an impact is unsurprising given the known variability in IOP measurements42 and the use of an absolute cut-off will lead to a regression to the mean effect when another measurement is taken (in this case by a different observer). Removing the VA component had very little impact on the diagnostic accuracy with a slight reduction in the sensitivity and corresponding increase in specificity. This impact may have been limited by the method of data collection (referral letter quotation) as opposed to complete data capture of a new VA measurement.

A number of assumptions underpinned the analyses and interpretation of these results in addition to those highlighted previously for diagnoses analyses. The reference standard here was the clinical decision to discharge or not, which will vary to some degree between individual clinicians and centres according to policies and practices (perhaps most noteworthy for individuals with glaucoma suspect). Components of the triage test were combined in an additive manner which reflects an implicit desire to favour sensitivity over specificity. No other options were assessed, although arguably this approach reflects clinical practice. The use of the ophthalmologist’s measurement does not reflect the reality of how a triage system would be implemented where, if a measurement was taken in hospital eye services, it would be by another individual (e.g. a technician). Using the referral IOP did have a substantial impact, although most if not all of this impact might be attributed to the inevitable variability between measurements taken at different times by different observers and the impact of regression to the mean. The finding does suggest there is value in taking a measurement upon referral to hospital eye services.

The model

The cost-effectiveness of the different imaging technologies and their subsequent care management pathways was assessed using a multistate Markov model. As glaucoma is a chronic condition, which progresses slowly over time, the model reflects the timing of both diagnostic testing and disease progression. This approach allowed modelling of the logical and temporal sequence of events (e.g. diagnosis or monitoring visits) following the initial diagnostic strategy.

Typically, Markov models have states (Markov states) in which individuals stay for a period of time called a ‘cycle’. The cycle must be a period relevant to the condition considered (e.g. 6 months, 1 year). At the end of each cycle, individuals can remain in the state in which they started the cycle or move to a different state. The probabilities of moving from one state to another are called transition probabilities. In each state, the model will assign costs and benefits for each individual according to different interventions and/or time spent in the state. In these models, there must be at least one absorbing state, typically death, from which the individual will not be able to leave. The sum of the cost in each year and the product of the utilities in each year were summed over 50 years of the simulated patient cohort to compute total cost and quality-adjusted life-year (QALY) for that cohort.

The purpose of this model was to compare and contrast different imaging technologies (used as part of a wider triage station) for the identification of patients who should be referred for a clinician-led diagnostic examination. We can thus compare and contrast these with standard care where all patients receive a clinician-led diagnosis based on clinical examination and visual field assessment (automated perimetry). The model was constructed such that different sensitivities and specificities of each diagnostic strategy would determine if glaucoma was correctly identified or not, the health state patients would move to and the associated progression of any underlying glaucoma. The consequences could then be considered in terms of the monetary costs (of testing and subsequent management of the patient’s condition) to the NHS and in terms of the effects on quality of life (by assigning utility weights). Combining these data with information of the probabilities of events occurring over time-enabled cost, patient outcomes and QALYs to be estimated for a hypothetical cohort of patients undergoing each triage strategy.

The results of the model are presented in Chapter 7 and are presented as incremental cost per QALYs and incorporate (1) costs (of testing) and diagnostic outcomes, (2) costs (of testing and subsequent management) and (3) QALYs.

Figure 24 shows the possible health states in ovals, while the arrows show the possible directions in which individuals can move at the end of each cycle, depending on the transition probabilities. The states considered in the model were those thought to reflect possible paths for individuals classified as normal, at risk of glaucoma or suffering from glaucoma at different stages (see Figure 24). Each state, other than normal and death, is divided into two categories. The treated states on the right-hand side of Figure 24 represent those individuals whose condition has been identified and is being treated, and the untreated states on the left-hand side represent those individuals whose condition has not yet been identified and thus who are not receiving treatment. The treatment health states refer to treated disease at each stage of glaucoma. The modality of treatment, IOP-lowering eye drops, laser or surgery or any combination thereof, is not specified for a glaucoma-related treatment state. A treatment state refers to any modality or combination treatment for each stage of glaucoma severity. There are three treatment states for the three stages of manifest glaucoma and a treatment state for sight impairment. The ‘at risk of glaucoma’ treatment state includes those individuals who are suspected of having glaucoma and those who have OHT and PAC. Among the ‘at risk of glaucoma’ group, we have assumed that all patients with OHT will be treated in the same way and that treatment incorporates annual outpatient appointments for observation, with all OHT individuals receiving continuous latanoprost (eye drops, once a day).

FIGURE 24.

Schematic diagram of model states and possible transitions.

Depending on their underlying condition, individuals will start in the model in a normal state, an untreated ‘at risk of glaucoma’ state or an untreated glaucoma disease state (mild, moderate, severe or sight impaired). Each individual will then enter a diagnosis process that will differ according to the compared strategies used to diagnose their condition (i.e. for current practice in the form of consultant-led diagnosis and care or a triage station including one of the imaging technologies under consideration; see Figure 24). The sensitivity and specificity of each diagnostic strategy determine the Markov state an individual will move to. In particular, it will determine if an individual enters a treated or untreated disease state and the possible transitions associated with these. In general, as time passes, the normal or ‘at risk of glaucoma’ individuals could develop glaucoma, while those with glaucoma could progress to a more severe disease state until they eventually become visually impaired.

Glaucoma is not reversible and this is reflected in the model (see Figure 24). However, individuals can return to a normal state after a number of model cycles within the ‘at risk of glaucoma’ Markov state. The absorbing state in the model is death. Any individual can move into this state from any other state in the model.

The model allows for a cohort of the population, some with glaucoma, to pass through different diagnostic strategies. The intuitive idea behind the model is to identify the strategy that leads to the largest proportion of individuals with glaucoma being correctly diagnosed and being in treatment to reduce disease progression and visual loss.

Description of the health-care diagnostic strategies and management pathways considered within the model

The care pathways modelled within the Markov model following diagnosis were developed in consultation with the study team and the independent steering committee members. The main study team for this element of the work comprised two ophthalmologists (AA-B, JB), and three health economists (RH, PM, JG), and a health services researcher (KB). Over a number of meetings, the group mapped out the sequence of events for patients potentially eligible for treatment or monitoring following the diagnostic strategies under consideration. Additional information came from our previous models in this area, notably our model comparing alternative screening strategies for OAG18 reviewed guidelines and expert opinion. These care pathways were then presented to the steering committee and revised to reflect the comments received.

Current practice care pathway

Patients enter the model as a cohort who have been identified with signs of, for example possible glaucoma or OHT by a community optometrist or GP and who have been referred to secondary care. Within hospital eye services, all individuals will see a nurse, who will perform a VA examination, and a technician, who will perform a visual field test. All individuals will then see a clinician (typically an ophthalmologist), who will measure IOP (using GAT), look at the visual field results and perform a fundus examination to examine the optic disc and the posterior retina. Figure 25 shows the care pathway.

FIGURE 25.

Care pathway: current practice. HES, hospital eye service; VF, visual field.

Considering all the clinical information, the clinician will decide on a diagnosis as described in Chapter 2. For the purpose of the model, these diagnoses have been grouped into five health states (described further in Table 40): mild glaucoma, moderate glaucoma, severe glaucoma, at risk of glaucoma and normal. Furthermore, the ‘at risk of glaucoma’ health state includes those with a diagnosis of OHT or glaucoma suspect or PAC.

Individuals who are diagnosed by the clinician to be in the normal health state are discharged from secondary care. Individuals diagnosed with glaucoma remain in secondary care under treatment and enter the relevant glaucoma-treated health state. Individuals diagnosed as ‘at risk of glaucoma’ also remain in secondary care and enter into the ‘at risk of glaucoma’ treatment state. The subset of ‘at risk of glaucoma’ patients with OHT are all assumed to be undergoing treatment.

Triage care pathway

As described in Chapter 2, the triage pathway used IOP, imaging and VA to identify patients who could be discharged from secondary care if all tests were normal. IOP and VA are routinely collected in primary and/or secondary care and used to inform the clinical decision-making process as to whether to discharge a patient or not. At hospital eye services the individuals will be seen by a nurse that will perform VA examination and IOP measurement. They will also be seen by a technician who will perform the index (imaging) test (HRT-MRA, HRT-GPS, GDx or OCT depending on triage strategy). Figure 26 shows the care pathway for the triage strategies.

FIGURE 26.

Care pathway: triage strategies. HES, hospital eye service; VF, visual field.

The results of these three examinations are combined into a composite triage test result as follows. If any of the VA or IOP or imaging test results is abnormal, then the composite test is assumed to be positive or abnormal. Only if all three tests (VA, IOP and imaging test) are normal is the composite test result negative or normal. Individuals with normal (negative) composite triage test results are discharged from secondary care. Individuals with abnormal (positive) test results are referred to the clinician to make a diagnosis. Definitions of abnormal (positive) test results for the elements of the composite test are as follows: IOP > 21 mmHg, VA 6/12 or worse, imaging technology classification abnormal or borderline.

Individuals who have been discharged with normal (negative) composite triage test results can be either truly normal (true negative) or have been incorrectly diagnosed as normal when they do in fact have disease (false negative). Individuals with an abnormal (positive) composite triage test result are then referred to the clinician, who will make a definitive diagnosis. Perfect information by the clinician is assumed in the model; therefore, individuals will be correctly identified as having glaucoma (e.g. mild, moderate, severe or visually impaired), as being at risk of glaucoma or without any of these conditions (e.g. normal). Normal individuals are discharged while all others are kept under monitoring or observation. The perfect information assumption is explored in sensitivity analysis with the possibility of misdiagnoses by the clinician (e.g. false-positive and false-negative results).

Estimation of parameters used within the model

This section summarises the parameter values used in the economic evaluation model.

Data regarding the cohort in terms of prevalence, incidence and progression are reported first, followed by diagnostic triage test performance data, with subsequent sections regarding data on cost and utilities also reported.

Base-case analysis

The base-case analysis was run for a cohort of 40-year-old males. Although the choice of this start age was arbitrary, it was felt that it covered the range over which diagnostic strategies for glaucoma might be considered, and would cover most of prevalent cases of glaucoma, which is an age-related disease. Sex-specific variables were not available for any of the model parameters except for mortality, and a decision was made to use male mortality rates in the base-case analysis, consistent with good modelling practice, as they are a conservative assumption for this enhanced case detection study. The model was run for a range of possible prevalence values and for a 50-year time horizon. Cycle length was set at 1 year. Costs are presented in 2012–13 UK pounds sterling and effectiveness in QALYs. A discount rate of 3.5% for costs and benefits was used following guidelines for technology assessment by NICE. 48 The results are presented in incremental cost-effectiveness ratios (ICERs). This measure is a ratio of the difference in costs divided by the difference in the effectiveness between two alternative strategies. These data can be interpreted as how much society would have to pay for an extra unit of effectiveness. Central to the assessment of cost-effectiveness is the value that society would put on gaining an additional QALY. NICE states that ‘Below a most plausible ICER of £20,000 per QALY, judgements about the acceptability of a technology as an effective use of NHS resources are based primarily on the cost-effectiveness estimate.’48 Between £20,000 per QALY and £30,000 per QALY, judgements about the acceptability of the technology should take into account factors such as

• the degree of uncertainty surrounding the calculation of ICERs

• the innovative nature of the technology

• the particular features of the condition and population receiving the technology

• where appropriate, the wider societal costs and benefits.

Above an ICER of £30,000 per QALY, the case for supporting the technology on these factors has to be increasingly strong. 47 In the absence of a more definitive statement this report focuses on a willingness-to-pay threshold of £30,000 for a QALY.

Sensitivity analysis

We addressed uncertainty by conducting deterministic (e.g. one-way) sensitivity analyses. In consultation with the independent advisory group, the following deterministic sensitivity analyses were considered:

1. The base-case analysis assumed that the annual probability of having an eye test is 31.2%. All patients who are discharged by the diagnosing clinician or discharged by the triage station for the triage strategies would therefore be expected to be picked up in the community and would return to the secondary care triage station approximately every 3 years. In this analysis, based on clinical opinion, the impact of changing this probability and thus the diagnostic screening interval within a range of 1–10 years inclusive was explored.

2. In the base case, the diagnostic triage strategies were micro-costed and included staff time and capital costs of the diagnostic technologies. However, owing to the relatively large cost differential of these triage strategies compared with current practice, it was deemed appropriate to explore the effects on cost-effectiveness of introducing an NHS Reference Cost for a non-consultant-led first outpatient appointment (£85) to the costs of the triage strategies. This was further varied from £10 to £85 in £5 intervals to explore if this changed either the diagnostic strategies that were deemed cost-effective or the magnitude of effect.

3. The base-case analysis included a cohort of men with an age of 40 years to be modelled for 50 years. The impact of modelling older cohorts of men was explored by varying the start age from 45 to 70 years in 10-year intervals.

4. The base-case analysis was conducted on the basis that all glaucoma patients and those at risk of glaucoma (including glaucoma suspects, OHT and PAC) would be monitored and treated depending on their definitive diagnosis. It was discussed and agreed in a meeting between the study team and the independent steering committee that there was a need to explore the effects of a hypothetical secondary care service where those patients diagnosed as ‘at risk’ would be discharged from the service, thus potentially reducing the diagnostic, monitoring and/or treatment costs.

5. The base-case analysis assumed that clinicians were 100% sensitive and specific in their diagnosis of patients. The sensitivity and specificity was varied between 0.85 and 1 to explore the impact for patients who would not always being seen in secondary care by an ophthalmologist with glaucoma expertise and thus having 100% diagnostic accuracy.

6. A threshold analysis was conducted in order to explore the impact of increasing the costs of the triage strategies and discharging those patients that are given a diagnosis of ‘at risk of glaucoma’.

7. The base-case analysis incorporated point estimates for the sensitivities and specificities of each of the imaging technologies that were estimated from the GATE study. We varied sensitivity and specificity of each triage strategy to create a best-case diagnostic scenario (+ 10% sensitivity and + 5% specificity) and a worse-case diagnostic scenario (–10% sensitivity and –5% specificity) for each of the imaging technologies as shown in Table 48 to explore the impact on the ICERs. These values were decided on by the research study team on the basis of variations in the CIs in the base-case analysis.

8. The base-case analysis assumed the prevalence of glaucoma in the referred population, which was estimated from the GATE study. However, no referral refinement schemes were in place during the GATE study. Other measures to improve the accuracy of glaucoma referrals are constantly being explored, with a reduction in false-positive rates. The impact of adding an imaging-based composite triage system to a referred population with lower false-positives rates was explored by decreasing the proportion of normal diagnoses in the cohort from 0.412 to 0.212 and increasing the glaucoma prevalence from 0.17 to 0.27 and the ‘at-risk’ group from 0.418 to 0.518.

9. The base-case analysis assumed that the utility weights for the ‘at-risk’ health state were the same as mild glaucoma in the absence of literature addressing this issue. We explored the impact of a utility weight for the ‘at-risk’ health state being the same as normal health state.

Base-case analysis

The base-case analysis was conducted for a cohort of male patients with a starting age of 40 years, who were assumed to have an eye test approximately once every 3 years, and clinicians in hospital eye services were assumed to have perfect diagnostic ability. Table 49 shows the cost-effectiveness results for the base-case analysis. All triage strategies were less costly than the current strategy, but the triage strategies resulted in fewer expected QALYs than the current strategy when a perfect diagnosis by the clinician was assumed. Triage with GDx was the strategy with lowest expected cost, followed, in order, by triage with OCT, HRT-MRA and HRT-GPS. Triage with OCT was extendedly dominated (i.e. a combination of triage with GDx or HRT-MRA could, in theory, produce more QALYs at lower expected costs than triage only with OCT alone). Triage with HRT-GPS strategy was dominated by HRT-MRA (i.e. HRT-GPS was more costly but did not produce more QALYs than HRT-MRA). This is further illustrated in Figure 27.

FIGURE 27.

Base-case cost-effectiveness analysis results.

Incremental cost-effectiveness ratios were calculated for all non-dominated strategies. The ICER reported for current practice (£156,985) represents the comparison between HRT-MRA and current practice. It should be noted that the interpretation of this ICER is slightly different from the usual case. In moving from current practice to HRT-MRA, savings would be expected, but at the expense of lost QALYs.

The usual willingness-to-pay threshold value for an additional QALY has been stated to be around £30,000 for the UK. 47 However, it is not clear what decision rule should be applied when resources are saved in exchange for fewer QALYs. One possible interpretation is that of a similar threshold (e.g. £30,000 saved at the expense of a QALY), and this has been adopted in this chapter. Therefore, with this interpretation, adopting a triage with HRT-MRA strategy would be worthwhile (e.g. resources would be freed and could be used elsewhere in the health-care system to obtain QALYs at the threshold value of £30,000 per QALY).

As shown in Figure 27, the results show that GDx is the least costly, least effective strategy and that OCT is extendedly dominated by GDx and HRT-MRA. This means that if it was possible to provide a mix of GDx and HRT-MRA, then a combination of provision of these two strategies would be dominant. Therefore, in economic evaluation we can disregard OCT from further consideration. In considering if it is worthwhile providing HRT-MRA in preference to GDx, we refer to the ICER. Relative to GDx, the ICER of HRT-MRA is £22,904 and is below the typical £30,000 value considered to be cost-effective in the UK. 48 In other words, moving from a triage strategy with HRT-MRA to GDx would save only £22,904 but at the expense of a QALY. Given the £30,000 threshold, any saved resources would not be sufficient to allow the QALY lost to be regained elsewhere.

Sensitivity analyses

A number of sensitivity analyses were performed as described in the methods (see Chapter 6).

Summary and discussion

This chapter reported the results of a cost–utility analysis of alternative composite triage strategies using alternative diagnostic imaging technologies compared with current practice for patients referred to hospital eye services for possible glaucoma.

The base-case results suggest that HRT-MRA is the most cost-effective strategy. Given that current practice represents standard care in the UK, large savings in costs (£156,985) could be made, but at the expense of a QALY. Furthermore, the ICER for current practice relative to HRT-MRA would exceed the value that is deemed to be cost-effective in the UK.

Another potential benefit is the release of clinicians’ time, which could be used to deliver other interventions.

Moreover, the sensitivity analysis results show triage strategies to be a potential cost-effective use of resources if the triage station cost does not reach £30 per triage visit. However, sensitivity analysis results were inconclusive in signalling a unique cost-effective triage strategy. HRT-GPS was often dominated by HRT-MRA, but the expected QALYs that these two strategies produce were almost identical, with the difference in total expected costs at around £10, which is not surprising since the results were obtained from the same imaging machine.

Furthermore, on a cost-effectiveness basis, GDx (or even OCT on a few occasions) could not be completely ruled out. GDx is highly specific and in a resource-constrained health economy it could be an efficient use of resources. It should be noted, however, that clinically, this strategy may not be acceptable to clinicians and/or patients because of its poor diagnostic performance (with low sensitivity). Determining a minimum level of diagnostic accuracy that is acceptable for clinical staff and patient was beyond the aims of this study and could be the subject of further research.

The QALY outcomes of all strategies depend only on the sensitivities of the tests to identify glaucoma and those at risk of glaucoma. The sensitivities of the different triage strategies for glaucoma are very close to each other, with the exception of GDx, but there is a greater difference between the strategies in their ability to identify people at risk of glaucoma. The consequences, in QALY terms, of missing a diagnosis of glaucoma are greater than those that result from missing a diagnosis of being ‘at risk of glaucoma’. For these reasons, the quality-of-life differences between triage strategies are small. The sensitivity of the triage strategies also means that the QALY differences between them and the base-case scenario are small. This was to a certain extent expected for a study in which triage strategies have similar diagnostic accuracies and a slow progression of disease. For example, this difference in the base-case analysis between current practice and HRT-MRA triage strategy was 0.0008 QALYs, representing less than 8 hours in full health. This small difference might make easier to accept a triage strategy that would result in loss of QALYs in exchange for potential savings.

Furthermore, the incremental cost-effectiveness of the triage strategies compared with current practice was very sensitive to costs included in the model. Unnecessary outpatient visits and associated treatment costs within current practice and, in particular, the costs of the actual triage strategies are model result drivers for the expected costs as well as the resulting ICERs. The cost-effectiveness of any triage strategy is heavily dependent on the unit cost of the triage station. As such, all these strategies were dominated by the current practice under the plausible assumption that an NHS provider of care would charge, for the triage station, an NHS Reference Cost tariff corresponding to an outpatient appointment. Indeed, current practice becomes dominant when the cost of an outpatient appointment increases to £61 and above.

A key assumption used in the model was that clinicians are 100% accurate in their diagnostic ability. Relaxing this assumption further increased the ICERs of current practice relative to other triage strategies above a level that would be deemed to be cost-effective in the UK. 48 Even under extreme scenarios, in which the diagnostic accuracy of the triage strategies was reduced, current practice could not be deemed the most cost-effective. Hence, in terms of diagnostic accuracy, no plausible scenarios rendered current practice the most cost-effective. A probabilistic sensitivity analysis was therefore not warranted. Only when the costs of the triage strategies increased with an NHS Reference Cost did current practice become cost-effective.

The strengths of this research are that an economic model has been developed and analysed using good modelling research practice. 51,52 The cost-effectiveness of the different imaging technologies and their subsequent care management pathways was assessed using a multistate Markov model. This modelling approach is highly relevant, as glaucoma is a chronic condition, which progresses slowly over time, allowing the model to reflect the timing of both diagnostic testing and disease progression following the initial diagnostic strategy. Furthermore, we believe that this is the first economic evaluation of these interventions to be conducted in this context.

There are limitations to this research. A key issue for the study is paucity of data regarding parameter inputs used in the model. As stated in the introduction (see Chapter 1), there is a lack of evidence regarding the diagnostic accuracy of imaging techniques in a triage setting and thus the parameter estimates regarding this have been based on the GATE study alone and not from multiple studies. Furthermore, the diagnostic accuracy of clinicians has been assumed to be perfect but explored in sensitivity analysis.

Only very limited data on the costs of diagnosis and treatment were available and, although efforts were made to identify the best data applicable to the UK, these were sparse. The model estimates would be more robust if further data were to become available and as previously stated by Burr et al. ,18 consideration should be given whether or not further primary research is needed. The model was very sensitive to the costs of the triage strategies and as stated above, adding additional costs to their unit costs renders triage not cost-effective compared with current practice.

The quality and usefulness of the economic model is dependent not only on the quality of the data, but also on the way in which the data are used. The data requirements and the use of the data were determined by the structure adopted for the model. The development of the economic model was, as described in Chapter 6, based on discussions with a number of key stakeholders. It then underwent a prolonged period of refinement during which the care pathways were critically examined and refined. The model structure applies to a UK context and may not be relevant to other country settings, although other strategies could be developed and readily added to the model.

As described in Chapter 6, the model structure was developed so that the assumptions made in the base-case analysis could be explored in future work. For example, in the base-case analysis it was assumed that the clinician would make a perfect diagnosis. The model structure has allowed for the possibility that this will not be the case and that the clinician might possibly initiate treatment when it is not required (a false positive) and fail to diagnose some cases of glaucoma (a false negative).

The model is a simplification of the care pathways that may follow. For example, the model structure does not include all possible health states that may be relevant in context, such as misdiagnosis of those at risk of glaucoma as true positives. A second simplification made in the model was the relatively small number of stages used to reflect the progression of this chronic condition. While this assumption may fail to represent the subtleties of disease progression, it was believed the health states were sufficient in number to reflect the relevant issues needed for this economic evaluation.

Estimates of the risk of progression between health states are based on data from one eye and do not necessarily represent the definition of the health states in the model, which is based on binocular visual field loss. The fellow eye may not have such advanced disease as the study eye and, therefore, the quality-of-life loss might be overestimated. While this is a limitation of the study, the alternative of using the better eye for the analysis would result in an underestimation of the risk of progressive binocular visual field loss. Furthermore, there were insufficient data to determine whether or not some of the parameter values varied between the stages of disease, for example the diagnostic performance of the diagnostic strategies. The model was, however, structured in such a way that, should such data become available in the future, the model could be readily adapted and the data incorporated.

A further simplification in the model structure was that, rather than modelling the full variety of treatments available for glaucoma, it has been assumed that the effect of treatment can be represented by a single relative effect size for treatment compared with no treatment. In addition, when interpreting the results of the economic evaluation it should be borne in mind that the estimates of cost-effectiveness relate to a male cohort. Sex-specific data were not available for any of the parameter estimates except for annual all-cause mortality.

Finally, there is no clear decision rule or willingness-to-accept threshold value to interpret cost–utility analysis results where savings are obtained at expense of QALY being lost. In this study, a similar threshold value to the one often used as willingness-to-pay for a QALY gained was assumed (i.e. £30,000). Although this is one value from many possible, in the great majority of the analyses the savings per QALY lost (ICERs) were well above this threshold. In other words, the adopted interpretation would be consistent with higher willingness-to-accept value should this become common practice.

Strengths and limitations

A number of strengths can be highlighted. GATE was a large prospective paired diagnostic study and it evaluated diagnostic tests in the desired setting. The benefit of the large sample size is reflected in the precision with which the sensitivity and specificity were calculated, with differences between every pair of tests identified for one if not both of sensitivity and specificity. McNemar’s test was used to compare the sensitivity and specificity of the tests. Following the rationale of others in effectiveness studies, the paired comparisons were not adjusted for multicomparisons. Even if such a correction had been applied, such was the strength of evidence that there would still be evidence of differences in the diagnostic performance of the different imaging tests.

The population enrolled in GATE consisted of subjects without a known history of disease, which would reflect the potential clinical application of the triage test. Other reported studies evaluating the performance of diagnostic technologies have used a population of patients already diagnosed with glaucoma, which has a risk of selection bias. This study recruited patients before diagnosis, and the population tested had a broad spectrum of disease at presentation, from early through to severe glaucoma, and included a large percentage of healthy individuals. The healthy individuals in whom the test ‘specificity’ was determined were subjects referred from primary care with a possible glaucoma-related finding (either risk factor or suspected sign). Thus, the diagnostic performance reported here refers to a secondary care setting and may be different in an unselected population.

An intentional aspect of the study’s design is the focus on both the diagnostic performance of imaging tests for the identification of individuals with glaucoma and the performance as a triage test where imaging tests would be used in conjunction with other routine measurements (IOP and VA). Both aspects are important for understanding the potential value of the imaging tests. We have also evaluated other important considerations for diagnostic technologies, such as interpretability, patient preference and time taken to perform the test.

The reference standard was provided by different ophthalmologists with glaucoma expertise. The ophthalmologists had been trained in the study protocols and agreed to a common set of criteria to define glaucoma and normality. By using different ophthalmologists working at different units, the results of the study are more likely to be generalisable than results from studies performed in a single unit. The participating units are likely to be representative of the NHS practice, including two district general hospitals and three academic units of different size: relatively small (Aberdeen), medium (Liverpool) and large (Moorfields).

The economic model was developed and analysed using good modelling research practice. 51,52 The cost-effectiveness of the different imaging technologies and their subsequent care management pathways were assessed using a multistate Markov model. This modelling approach is highly relevant as glaucoma is a chronic condition, which progresses slowly over time, allowing the model to reflect both the timing of diagnostic testing and the disease progression following the initial diagnostic strategy.

Among the limitations, we recognise that diagnosing glaucoma during the very early stage of disease is challenging, and ideally a longitudinal follow-up would provide the best possible reference standard. This was proposed by Medeiros et al. 34 who used optic nerve head progression on stereophotographic examination as the criterion for glaucoma diagnosis, but we could not contemplate this possibility in GATE, as years of follow-up would have been required. The reference standard was assumed to be perfect, although it is widely recognised that diagnosis of glaucoma is difficult in early disease, and uncertainty exists even among specialists. While consensus was sought through structured training, some assessor differences may have remained between the sites. Adding central corneal thickness information for patients referred for high IOP could potentially add valuable information and help further refine the referral pathway of such patients.

There was lack of evidence base regarding some parameter inputs used in the economic model. Only very limited data on the costs of diagnosis and treatment were available and, although efforts were made to identify the best data applicable to the UK, these were sparse. Data with respect to health utilities were available, but it is unclear whether or not the EQ-5D is sensitive enough to detect clinically significant changes in glaucoma. The model is a simplification of the care pathways that may follow, with a relatively small number of stages used to reflect the progression of this chronic condition. Estimates of the risk of progression between health states were based on data from one eye and do not necessarily represent the definition of the heath states in the model, which is based on binocular visual field loss. A further simplification in the model structure is that, rather than modelling the full variety of treatments available for OAG, it has been assumed that the effect of treatment can be represented by a single relative effect size for treatment compared with no treatment.

Uncertainties

• The diagnosis, natural history and risk of conversion to glaucoma of untreated or treated patients classified as glaucoma suspects is unknown. It is likely this is a very heterogeneous group, as reflected in the categories of glaucoma suspect defined in GATE.

• The natural history and risk of conversion to glaucoma of untreated or treated patients with OHT undergoing standard care is unclear. Although there is evidence on the efficacy of treatment of OHT from large randomised controlled trials, the generalisability of their findings to routine clinical care in the NHS is ill defined.

• It is unclear how often people attend community optometrists for regular eye examinations. If they have glaucoma that is missed by the triage, it is unknown how quickly it would be detected by the optometrist and at what severity of disease. In our model we hypothesised that all those with a false-negative diagnosis at the triage stage would return to hospital eye services within 3 years.

• The triage analysis used the IOP information provided by a consultant ophthalmologist. A triage system would rely on IOP measurements taken by a technician or a nurse, and it is uncertain whether or not such IOP measurements, possibly obtained with different tonometers, will be significantly different and what impact this would have in the performance of the triage test. The diagnostic accuracy of clinicians is uncertain. Glaucoma is diagnosed clinically, relying on the experience of the examiner, and it is likely that the relative performance of the imaging technologies may be underestimated if the reference standard comparator consists of experienced glaucoma experts, as were used in GATE. Glaucoma in the NHS is diagnosed by a variety of health-care professionals, including optometrists, specialist nurses, senior ophthalmologists with variable glaucoma expertise and trainees.

• There are other OCT instruments in the market with glaucoma diagnostic capabilities and the results of this study using the Spectralis^® device may not be fully applicable to other OCT technologies.

Implications for health care

Automated imaging technologies can be effective tests to aid in the diagnosis of glaucoma among individuals referred from the community to hospital eye services with possible glaucoma. A model of care incorporating a triage composite test for diagnosing patients referred from the community appears to be cost-effective compared with current practice. Our findings are based on a relatively non-expensive composite triage test (< £30) including an imaging technology, IOP and VA testing. The most efficient strategy would include HRT-MRA imaging. However, a triage test would be associated with reduced health, and the acceptability of this option among users and clinicians has not been evaluated.

Recommendations for research

• Acceptability to patients and health-care providers of implementing an efficient triage glaucoma diagnostic system but with reduced health should be explored. A qualitative or mixed-methods study, for example including a discrete choice experiment and also incorporating public perspectives, would be suitable.

• Further data on the glaucoma disease progression under routine care, and specifically including patients classified as having glaucoma suspect or OHT, on associated utility, on the cost of providing health-care services and on sight loss are needed. A long-term longitudinal cohort study would be ideal to address these issues.

• Further investigation of varying the thresholds for classification of the imaging tests beyond the standard options presented in the software could be undertaken, as the standard classification may not be the one best suited to the population referred from the community to hospital eye services. Further analysis of GATE data or review of data from other relevant diagnostic studies would be able to answer this question.

• The effectiveness of implementing a triage test incorporating imaging, an IOP measurement and VA requires evaluation. A longitudinal diagnostic impact study is needed.

Contributions of authors

Augusto Azuara-Blanco was the chief investigator of the study, had complete involvement and oversight of the study design, execution and data collection and provided clinical expertise, led the writing of all chapters with the exception of Chapters 4–7 and was responsible for the final report.

Katie Banister was responsible for the day-to-day management of the study, contributed to the writing of Chapter 2, commented on all chapters and was responsible for the production of the final report.

Charles Boachie conducted the statistical analysis and contributed to the writing of Chapters 4 and 5.

Peter McMeekin developed the structure of the Markov model, conducted the economic analyses and contributed to the writing of Chapters 6 and 7.

Joanne Gray led the economic analysis and led the writing of Chapters 6 and 7.

Jennifer Burr provided clinical advice and methodological support through all stages of the project and commented on the final report.

Rupert Bourne, David Garway-Heath and Mark Batterbury were clinical leads, provided expert advice on clinical aspects of the study and commented on the final report.

Rodolfo Hernández had oversight of the health economic analysis, contributed to the writing of Chapters 6 and 7 and commented on the report.

Gladys McPherson provided technical data collection expertise throughout the study and reviewed the final report.

Craig Ramsay provided methodological oversight through all stages of the project and commented on the final report.

Jonathan Cook provided methodological oversight for the whole project, led the writing of Chapters 4 and 5, contributed to Chapter 2 and commented on the final report.

Independent members of the steering committee

Colm O’Brien (chairperson), Anthony King, Anja Tuulonen, Russell Young and David Wright.

Project management group

Augusto Azuara-Blanco, Katie Banister, Jennifer Burr, Jonathan Cook, Rodolfo Hernández, Kirsty McCormack, Gladys McPherson and Craig Ramsay.

Staff who facilitated recruitment and data collection

Aberdeen: Augusto Azuara-Blanco (Principle Investigator), Jemaima Che-Hamzah, Manjula Kumarasamy, Vikki McBain, Sean Neville, Laura Park, Minimol Paulose and Patricia Peacock.

Bedford and Hinchingbrooke: Rupert Bourne (Principle Investigator), Lydia Chang, Donna Gallagher, Shazia Hussein, Wendy Newsom, Paula Turnbull, Sheila Urquhart, Shalina Begum and Charlotte Sullivan.

Moorfields Eye Hospital: David Garway-Heath (Principle Investigator) Ayse Barnes, Kanom Bibi, Jonathan Clarke, Cornelia Hirn, Poornima Rai, Gloria Roberti, Nick Strouthidis, Ananth Viswanathan and Ed White.

Royal Liverpool University Hospital: Mark Batterbury (Principle Investigator), Anshoo Choudhary, Kathryn Nutter, Paula Burke and Jerry Sharp.

The Health Services Research Unit is funded by the Chief Scientist Office of the Scottish Government Health Directorate.

Publication

Banister K, Boachie C, Bourne R, Cook J, Burr JM, Ramsay C, et al. Can automated imaging with HRT for disc and GDx-ECC and Spectralis OCT for retinal nerve fiber layer analysis aid glaucoma detection? Ophthalmology 2016; in press.

Data sharing statement

All available data can be obtained from the corresponding author.

Disclaimers

This report presents independent research funded by the National Institute for Health Research (NIHR). The views and opinions expressed by authors in this publication are those of the authors and do not necessarily reflect those of the NHS, the NIHR, NETSCC, the HTA programme or the Department of Health. If there are verbatim quotations included in this publication the views and opinions expressed by the interviewees are those of the interviewees and do not necessarily reflect those of the authors, those of the NHS, the NIHR, NETSCC, the HTA programme or the Department of Health.

Heidelburgh Retinal Tomography glaucoma probability score (HRT-GPS)

(PDF download)

Heidelbergh Retinal Tomography Moorfields regression analysis (HRP-MRA)

(PDF download)

Glaucoma diagnostics (GDx)

(PDF download)

Spectralis optical coherence topography (OCT)

(PDF download)

Overview

An additional set of two statistical analyses (see Triage sensitivity analyses 9 and 10) were carried out to specifically inform the economic modelling for GATE. These were set up to mirror the model structure in terms of population (i.e. with the simplification of ignoring the presence of non-glaucoma-related comorbidities). The first additional analysis used a reference standard definition of disease of glaucoma, glaucoma suspect, OHT and PAC; the second analysis used diagnosis of glaucoma alone as the reference standard (Table 60). The test was a composite, as previously described in Chapters 2 and 5, of the imaging test result, IOP and VA measurements (referred to throughout this appendix by the name of imaging test used within the composite test, e.g. HRT-MRA, HRT-GPS, GDx or OCT). Where a classification was not provided by the imaging test, the patient was defined as a ‘for referral’. For the first analysis, borderline imaging results were also classified as ‘for referral’, whereas for the second analysis they were classified ‘not for referral’. Triage sensitivity analyses 9 and 10 represent the analyses used to populate the diagnostic accuracy results of the base case and the sensitivity analysis scenarios, respectively (see Chapter 6 for further details). Subgroup sensitivity and specificity values were calculated for each diagnosis separately (e.g. glaucoma, ‘at risk of glaucoma’ and neither groups) breaking down the performances of the triage test to provide estimates for the economic model. ‘At risk of glaucoma’ was defined as being suspected of any type of glaucoma, or having OHT or PAC.

Diagnostic performance of the triage tests

The diagnostic accuracy results of the two analyses are given in the following two sections.

Triage sensitivity analysis 9

The flow of study participants according to triage sensitivity analysis 9 is shown in Figure 32 with corresponding numbers of referral, not for referral and no results cases by triage test. The diagnostic performance for the four tests is given in Table 61. The results showed a trade-off between the detection of patients who need to be referred and the discharge of those who do not need to be referred: HRT-MRA had the highest sensitivity (HRT-GPS was only very slightly lower) but also the second lowest specificity (HRT-GPS had the lowest), GDx had the lowest sensitivity but the highest specificity and OCT provided intermediate results. Likelihood ratios (and 95% CI) showed evidence of being able to both rule in and rule out the presence of glaucoma for all four triage tests (CIs did not contain 1.0). DORs ranged from 4.29 for GDx to 16.83 for HRT-MRA.

FIGURE 32.

Flow diagram: triage sensitivity analysis 9.

From this analysis, the sensitivity for participants with glaucoma was calculated as 99%, 99%, 88% and 97% for HRT-MRA, HRT-GPS, GDx and OCT, respectively; similarly the sensitivity for participants ‘at risk of glaucoma’ was calculated as 97%, 97%, 77% and 87%, respectively, and the specificity for participants classified as normal (not glaucoma or ‘at risk of glaucoma’) was 30%, 28%, 51% and 35% for HRT-MRA, HRT-GPS, GDx and OCT, respectively.

Triage sensitivity analysis 10

The flow of study participants according to triage sensitivity analysis 10 is shown in Figure 33, with corresponding numbers of referral, not for referral and no results cases by triage test. The diagnostic performance for the four tests is given in Table 62. The results generally showed a trade-off between the detection of patients who need to be referred and the discharge of those who do not need to be referred: HRT-MRA had the highest sensitivity (HRT-GPS was only very slightly lower) but also the second lowest specificity (HRT-GPS had the lowest), GDx had the lowest sensitivity but the highest specificity and OCT provided intermediate results. Likelihood ratios (and 95% CI) showed evidence of being able to both rule in and rule out the presence of glaucoma for all four triage tests (CIs did not contain 1.0). DORs ranged from 5.11 for GDx to 12.83 for HRT-MRA.

FIGURE 33.

Flow diagram: triage analysis sensitivity 10.

From this analysis, the sensitivity for participants with glaucoma was 93%, 89%, 49% and 83% for HRT-MRA, HRT-GPS, GDx and OCT, respectively; the sensitivity for participants ‘at risk of glaucoma’ was calculated as 61%, 59%, 17% and 36%, respectively; and the specificity for participants in the normal health state (without glaucoma or ‘at risk of glaucoma’) was calculated as 60%, 61%, 85% and 72%, respectively.