Improving the economic value of photographic screening for optical coherence tomography-detectable macular oedema: a prospective, multicentre, UK study

Inclusion criteria

• Aged ≥ 18 years.

• One or more of the following features present in at least one eye identified using retinal photography:

  • M/DHs within one disc diameter radius (DD) of the centre of the macula.

  • BHs within one DD of the centre of the macula.

  • Exudates within two DD of the centre of the macula.

• Able and willing to provide signed informed consent.

Main exclusion criteria

• Any macular or pan-retinal laser treatment in the study eye(s) or any intraocular injection, since these interventions affect disease progression and characteristics.

• Intraocular surgery (e.g. cataract surgery) within 1 year of enrolment. Cystoid MO following cataract surgery, also known as Irvine–Gass syndrome, is the most common cause of decreased vision following cataract surgery. 24

• Pregnancy. During pregnancy diabetic retinopathy can change much more rapidly. 25–27

• An inadequate OCT image or two inadequate retinal photographs. Photographs are considered inadequate if either (1) the clarity is insufficient as the macular vessels are not clearly visible or (2) the field of view does not include a circular region with a radius of at least two disc diameters centred on the fovea.

• Contraindications to pupillary dilatation should pupil dilatation be necessary. Pharmacological pupillary dilatation is necessary where the pupil is too small to allow imaging, either retinal photography or OCT. Typically the pupil diameter must exceed 4 mm to allow imaging.

Reference standard for macular oedema

Optical coherence tomography was chosen as the reference standard for identifying the presence of diabetic MO. 1 Although other technologies are available for measuring retinal thickness (see Appendix 1), OCT has the highest resolution, good repeatability, and is the only method that provides detailed anatomical cross-sections.

Sample size

A previous study found a 14% prevalence of macular lesions within a screening programme,30 the subjects making up the target cohort for this study. Of those with macular lesions, 10% are expected to have MO. Thus, the majority of the study cohort will not have MO and hence the precision of the sensitivity measurement will be the determining factor in the calculation of study power and sample size.

To detect a 3% difference in sensitivity between two diagnostic tests with 80% power using the McNemar test requires a sample of 400 positive cases. Using the assumed prevalence of positive cases above of 10% this would mean recruiting 4000 subjects.

Regulatory approval

The study was approved by the North East Scotland Research Ethics Committee on 17 December 2007 (reference 07/S0801/107). Each participating centre obtained approval from their local ethics board. The study was cosponsored by the University of Aberdeen and NHS Grampian. All participants provided signed informed consent after reading the patient information sheet and following discussion with the local study representative. A Trial Steering Committee oversaw the conduct of the study. The study was registered with the United Kingdom Clinical Research Network (UKCRN reference 9063).

Visual acuity

Visual acuity was measured using either best corrected visual acuity or a pin hole (the method used was noted). Visual acuity was recorded using the log-MAR scale. Where subjects could not resolve characters on the vision chart they were asked if they could count fingers, see hand movement or perceive light, otherwise the eye was recorded as having no perception of light. Count fingers was assigned a log-MAR value of 2, and hand movement a log-MAR value of 3. 31 Where vision is worse than hand movement the subject is unable to resolve any object and therefore the visual acuity is undefined.

Pupillary dilatation

In the English centres all patients received mydriasis, while Scottish centres only used mydriasis if the pupil size was too small for imaging. Both retinal photograph and OCT scanning require a pupil size of at least 4 mm. Studies have shown that the use of pupillary dilatation does not affect OCT thickness measurements. 32,33

Retinal photograph acquisition

All photographs were acquired by ophthalmic photographers or retinal screeners and the name of the photographer was noted for each image. A single retinal photograph was acquired for each eye meeting the following criteria:

• 45 degrees field of view

• macula centred

• colour digital photograph (between 3 and 8 megapixels)

• JPEG image compression (if used) set for high quality

• adequate field of view (the image should show a region having a radius of at least two disc diameters around the fovea)

• adequate clarity (i.e. adequate to see macular microaneurysms, if present)

• the fundus camera small pupil facility was acceptable providing a region of at least two DD was still visible around the fovea.

Optical coherence tomography reference image acquisition

All OCT scans were acquired by operators who had been accredited for the study. There was a maximum time limit of 4 weeks between the retinal photograph and OCT reference scan as the disease was unlikely to progress significantly during this period of time. Eighty-nine per cent of OCT scans were acquired on the same day as the retinal photograph.

Other recorded patient information

No patient-identifiable information left the recruiting centre. Patient identifiers were removed from images and data such as subject age were recorded with insufficient granularity to be of help in identification. In addition to the retinal photograph, OCT scan and visual acuity assessment the following subject information was collected:

• age (rounded to the nearest year)

• gender

• amblyopia.

Amblyopia is an uncorrectable decrease in vision in one eye with no apparent structural abnormality seen to explain it. It is a diagnosis of exclusion, meaning that when a decrease in vision is detected, other causes must be ruled out. There is wide variation in reported estimates for its prevalence. A review of UK amblyopia studies in children aged ≤ 5 years assumed a prevalence of 4.8%. 34 A cohort study at an English retinal screening programme recorded a 10% amblyopia prevalence. 35 Since visual acuity is used as an indicator of MO in some screening programmes, amblyopia represents a confounding factor in the detection of MO. Other non-diabetic factors affecting visual acuity, such as lenticular opacities and macular degeneration, were not recorded separately as, unlike amblyopia, they directly affect the appearance or quality of the retinal photograph:

• type of diabetes (the type of diabetes was recorded as type 1, type 2, secondary or other).

• ethnicity (the ethnicity categories were Asian, Black, Caucasian, Chinese, mixed, other, unknown)

• first half of postcode

• glitazone use within the previous 6 months.

Glitazones, or thiazolidinediones, are a group of drugs that are prescribed to increase sensitivity to insulin in people with type 2 diabetes. The most common forms are rosiglitazone and pioglitazone. Rosiglitazone had its marketing authorisation suspended during the study in September 2010 by the European Medicines Agency because of concerns regarding an increased cardiovascular risk. Oedema is a known risk factor when using glitazones and studies have suggested they are associated with an increased risk of MO,34–36 although another study found no association. 37 The recording of the use of glitazones was added to the study protocol in June 2009 because of the uncertainty surrounding their role in the development of MO.

Web submission and data validation

A website was developed to enable the transfer of both subject information and image data from each recruiting centre to the database in Aberdeen. The website also allocated the unique study identifier for each subject and printed out a reference sheet for the local study folder.

The website was designed to reduce data entry errors. Tick boxes, menus and calendar entry tools were used in preference to plain text entry (Figure 1). Checks were performed on the entered data, for instance that the subject was aged ≥ 18 years and therefore eligible for the study. Checks were also performed on the image data that were uploaded to ensure that the images were the expected size and format for the OCT scanner used at that centre.

FIGURE 1.

Web form used to upload study data.

Check sums were calculated for all images when they were uploaded. This served two purposes: first, it allowed the continuing integrity of the image data to be verified and, second, it enabled attempted duplicate image uploads to be rejected.

Retinal photographs

All the retinal photographs were graded and annotated by the same research nurse, who had 3 years' experience working as a retinal grader. A screenshot of the software used for grading is shown in Figure 2.

FIGURE 2.

Screenshot of the software used to grade images.

Image annotation

Software was developed to enable the research nurse to annotate the retinal images, indicating the size and position of the optic disc, the position of the fovea, and the location, shape and size of individual lesions within two DD of the fovea (Figure 3). All the lesions associated with maculopathy (M/DHs, BHs and exudates) were annotated, as well as non-diabetic features with similar appearance which could confound the analysis [flame haemorrhages, drusen and cotton wool spots (CWS)].

FIGURE 3.

Screenshot of software used to annotate features on the retinal photographs.

Optical coherence tomography reference standard

Cross-sectional images

The cross-sectional images were visually assessed for the presence of oedema because several pathologies, besides diabetic MO, could affect retinal thickness. Confounding features include vitreo-macular traction, epiretinal membranes, macular holes, drusen, subretinal haemorrhage, pigment epithelial detachment and choroidal neovascular membrane. Example OCT thickness map and cross-sections are shown in Figure 4.

FIGURE 4.

Example of OCT cross-sections and thickness map (report from Zeiss Cirrus OCT).

Zeiss Stratus OCT

The ‘fast macular protocol’ was used on the Stratus OCT. It acquires six intersecting 6 mm radial cross-sections centred on the fovea. Each cross-section includes 128 A-scans of 1024 axial samples over a 2 mm depth range. The total acquisition takes 1.9 seconds. The scans were considered acceptable if (1) the SD of the central foveal measurements was < 10% of the mean central measurement, (2) the recorded signal strength was at least 4 out of a possible maximum of 10, (3) there were no warnings from the instrument regarding ‘low analysis confidence’, ‘missing data’ or ‘high variance’, (4) there were no visible boundary tracking errors and (5) the fixation point was within approximately 250 µm of the centre of the foveal pit. The scanner software version was 4.0.7.

Zeiss Cirrus OCT™ (Carl Zeiss Meditec International, Jena, Germany)

The Cirrus OCT (Carl Zeiss Meditec International, Jena, Germany) acquires A-scan data almost 70 times faster than the Stratus OCT. Scans were collected covering a 6 × 6 mm square area with a 512 × 128 raster pattern. Each A-scan consisted of 1024 samples over a 2 mm depth range. The total acquisition time was 2.4 seconds. A faster protocol is available, using a 200 × 200 raster pattern, which covers the same area in 1.5 seconds. The scans were considered acceptable if (1) the signal strength was at least 6 out of 10, (2) there were no obvious signs of movement, (3) there were no visible boundary tracing errors and (4) the fixation point was within 250 µm of the centre of the foveal pit. The scanner software version was 2.0.0.54.

Topcon 3D OCT-1000™ (Topcon Corporation, Tokyo, Japan)

Scans were acquired covering a 6 × 6 mm square area to a depth of 1.68 mm using a 256 × 128 raster pattern of A-scans. Each acquisition took approximately 2 seconds. The scans were considered acceptable if (1) there were no visible boundary tracing errors, (2) there were no obvious signs of movement, (3) < 10% of B-scan lines were missing in any ETDRS region and (4) the fixation point was within 250 µm of the centre of the foveal pit. The scanner software version was 2.11.

Heidelberg Spectralis™ (Heidelberg Engineering, Heidelberg, Germany)

The Spectralis A-scan rate of 40,000 A-scans/second is the fastest in this group. Its acquisition also differs from the other scanners in two significant respects. First, it includes an eye tracking facility which pauses the acquisition whenever the subject moves or blinks (consequently, the scan time can vary depending on subject movement). Second, each B-scan cross-section is acquired several times and combined to improve the signal-to-noise ratio and reduce speckle artefacts.

Scans were acquired for a nominal 30 × 30 degree region using a 768 × 256 raster pattern, with each B-scan cross-section acquired five times. Each A-scan consisted of 512 samples covering a 1.8 mm depth range. The scans were considered acceptable if (1) the 6 mm ETDRS region fitted within the acquired region and (2) there were no boundary tracking errors. Software allowed the regions to be translated to correct for minor fixation errors. Boundary tracings were not available for all images and so the exclusion criteria were less stringent than for the other scanners. The software version used was 3.0.

Boundary measurements

Differences in how boundaries are identified by the various scanners were a significant problem. Boundary A (Figure 5) indicates reflections from the nerve fibre layer on the top surface of the retina; all the scanners begin their thickness measurements at this layer. However, there is disagreement about where to place the lower limit of the thickness measurement. Boundary B, probably reflections from the external limiting membrane, tends to be fainter than the three lower boundaries C, D and E. The Zeiss Stratus OCT takes the top of the bright reflective band (approximately layer C above) as the posterior limit (measurement M1). In contrast, the Heidelberg Spectralis takes the bottom of the bright reflective band (layer E, measurement M3). The Zeiss Cirrus OCT and Topcon 3D OCT-1000 use a limit between these two extremes, with the Topcon 3D OCT-1000 tending to find the lower edge of layer C and the Zeiss Cirrus OCT the top edge of layer D (measurement M2).

FIGURE 5.

Optical coherence tomography cross-section through the centre of the macula showing the normal fovea (pit) at the region of highest acuity vision. As shown here, darker grey levels represent stronger signals. A number of highly reflective (i.e. dark) interface boundaries are visible on this image acquired with a Heidelberg Spectralis scanner. Boundary A indicates reflections from the nerve fibre layer on the top surface of the retina, whereas boundary B, which is probably reflections from the external limiting membrane, tends to be fainter than the three lower boundaries C, D and E. Earlier scanners were unable to resolve C, D and E, which were generally lumped together as the ‘bright reflective band’.

Scans meeting inclusion criteria

Eyes were excluded if either of the repeat scans failed to meet the inclusion criteria. The number of eyes included from each scanner were: the Zeiss Stratus OCT (45/60), the Zeiss Cirrus OCT (55/66), the Topcon 3D OCT-1000 (17/26) and the Heidelberg Spectralis (27/28).

Four Zeiss Stratus OCT scans had visible boundary errors (Figure 6), two had low analysis confidence warnings, one had a fixation error, one had missing data and two had high variance warnings. On the Zeiss Cirrus OCT, fixation errors affected 10 scans and one was excluded because of movement. On the Topcon 3D OCT-1000, 13 scans showed missing data, which in nine cases was serious enough for the scans to be excluded. One Heidelberg Spectralis scan was excluded owing to a missing thickness value in one region, probably because of a boundary tracing error.

FIGURE 6.

Example of an automatic boundary detection failure. Left, Topcon 3D OCT-1000 example from a patient with MO; right, a Zeiss Stratus OCT scan from a volunteer.

Repeatability

Both eyes were included in the repeatability analysis since the repeat measurement differences are expected to be random and uncorrelated. In this study the correlation between thickness measurements in left and right eyes was strong (Pearson correlation coefficient 0.780), but there was no significant correlation in the repeat measurement differences in left and right eyes (Pearson correlation coefficient 0.014; p = 0.9).

Table 1 lists the repeatability and ICCs for the nine ETDRS regions. In general, repeatability was poorest in the smallest, central region (1) and best in the inner regions (2, 3, 4 and 5). The Zeiss Stratus OCT shows the poorest repeatability, with all regions being equal to, or worse than, the newer scanners.

Interscanner agreement

Box and whisker plots of the macular thickness in each of the nine regions are shown in Figure 7. By inspection it is clear that there were differences between all the scanners and that the sizes of these differences were similar for all regions.

FIGURE 7.

Box and whisker plots of macular thickness measurements for the nine ETDRS regions shown in Figure 29 for the four scanners. The plots show the median (central stripe), interquartile range (box) and full range (whiskers), excluding outliers (+) which are defined as observations beyond 1.5 times the interquartile range from the box. C, Zeiss Cirrus OCT; H, Heidelberg Spectralis; T, Topcon 3D OCT-1000; Z, Zeiss Stratus OCT.

The mean central thickness in volunteers was greatest on the Heidelberg Spectralis (277 µm, SD 15 µm) and least on the Zeiss Stratus OCT (201 µm, SD 19 µm). The Topcon 3D OCT-1000 (230 µm, SD 22 µm) and Zeiss Cirrus OCT (258 µm, SD 21 µm) were between these two extremes. In all scanners the average thickness in the superior region was greater than in the inferior region, and likewise greater in the nasal compared with the temporal region.

Table 2 lists the results of the mixed-model analysis. It shows the estimated differences between scanners calculated using scans from both eyes meeting the inclusion criteria. There were significant differences in thickness between all of the scanners in all nine regions (p < 0.001). In the central region the measurements were greater on the Zeiss Cirrus OCT, Topcon 3D OCT-1000 and Heidelberg Spectralis scanners than the Zeiss Stratus OCT by 58 µm, 28 µm and 78 µm, respectively. Compared with the central region differences, the other eight regional differences were slightly smaller on all scanners except the Topcon 3D OCT-1000, where the largest differences were in the inner regions.

As agreement in its technical sense is proportional to the SD of measurement differences,39 any systematic difference is disregarded. The central region macular thickness agreement between the Zeiss Stratus OCT and Zeiss Cirrus OCT was 10.5 µm (n = 30; 95% CI 7.1 to 13.9 µm), between the Zeiss Stratus OCT and Topcon 3D OCT-1000 was 6.9 µm (n = 13; 95% CI 3.26 to 10.5 µm) and between the Zeiss Stratus OCT and Heidelberg Spectralis was 12.7 µm (n = 14; 95% CI 6.3 to 19.1 µm).

Interscanner thickness conversion

Using a single correction for all nine regions and rerunning the mixed-model analysis resulted in very similar measurements from all the scanners for the central region. However, statistically significant differences still remained; in the outer regions the Zeiss Cirrus OCT and Heidelberg Spectralis were up to 16 µm lower than the Zeiss Stratus OCT and the Topcon 3D OCT-1000 up to 7 µm higher. Refining the model and using three regional constants per scanner, where correction values were calculated for the central, inner and outer regions, resulted in very few statistically significant differences remaining and the average errors were now all ≤ 3 µm, with the largest errors in the inner nasal region.

The above strategy assumes additive correction alone is sufficient. The need for possible multiplicative correction was tested by plotting the difference between measurements from two scanners against the mean thickness measured on the two scanners. Figure 8 shows the comparison of Zeiss Stratus OCT and Zeiss Cirrus OCT measurements for the central region. If an additive constant is sufficient to correct the difference then the line of regression should have a zero gradient. However, as Figure 8 shows, the regression line has a gradient of −0.17, which is significantly different from a zero gradient (p < 0.001). Although only regions 1 and 9 are significantly different from zero gradient, probably owing to the small sample size, the bar chart in Figure 9 of estimated gradients for all nine regions shows a clear trend, with negative gradients in the central and inner regions and positive gradients in the outer four regions.

FIGURE 8.

Graph of the difference between the Zeiss Stratus OCT and Zeiss Cirrus OCT measurements vs. average thickness. The dashed line shows the least squares regression.

FIGURE 9.

Bar chart showing the regression coefficients (slope) for each ETDRS region for plots of difference in Zeiss Stratus OCT and Zeiss Cirrus OCT vs. mean thickness.

Excluded subjects

The centres were not able to verify the quality of all aspects of a particular subject's information at the time of recruitment. However, checks of eligibility, and the quality and availability of data meant that 370 subjects were excluded for a variety of reasons, some for more than one reason. In 329 cases it was not possible to assess the thickness of the macula in one or both eyes and so the subject could not be classified as having MO present or absent, and in a further 41 cases retinal photographs from both eyes were of inadequate quality. The reasons, sometimes overlapping, for OCT failure included an enucleated eye (one subject), errors in identifying the boundaries of the macula, low signal, evidence of movement of the eye, positional error of the image or loss of the image. Of the 370 subjects excluded, 12 subjects were found to have had previous laser treatment in one or both eyes. Sixty-eight subjects had poor clarity in one or both photographs and nine subjects had an incorrect field of view in one or both photographs. For some subjects, both OCT and photographs were inadequate.

There were some statistically significant differences between the 3170 subjects included in further analyses and the 370 excluded. The large size of the study meant that some differences of small magnitude were found to be statistically significant. The excluded subjects tended to be older by 6 years and were slightly more likely to be female. Asian or Black subjects were slightly more likely to be excluded (14.4% and 14.0%) rather than Caucasian (9.8%). A higher percentage of subjects with type 2 diabetes were excluded (11.2%) compared with subjects with type 1 diabetes (7.7%). Those with worse visual acuity (log-MAR ≥ 0.3) were more likely to be excluded. There were clear differences between percentages of subjects excluded on the four scanners: 15.9% of subjects on the Zeiss Stratus OCT, 13.9% on the Topcon 3D OCT-1000, 7.3% on the Heidelberg Spectralis and 1.4% on the Zeiss Cirrus OCT. These findings should not be interpreted in isolation. They could have been influenced by the fact that older people are more likely to have opacity and because different scanners (with different qualities) were used in different centres.

Prevalence of thickening and macular oedema

Among the 3170 subjects included in further analyses, there were significant differences in the percentages with thickening in region 1 [greatest for Zeiss Cirrus OCT (19.1%) and least for Zeiss Stratus OCT (12.9%); comparison of all scanners p = 0.002] and also the percentages with thickness in at least one of the regions 1–5 [greatest for Zeiss Cirrus OCT (30.4%) and least for Heidelberg Spectralis (20.4%); comparison of all scanners p < 0.001]. The results are presented in Table 4. Note that the percentages sum to 100% across the page in every third column.

There were clear differences between percentages of subjects with MO in at least one eye on the four scanners: 11.8% of subjects scanned on Zeiss Cirrus OCT had MO, 8.7% on Heidelberg Spectralis, 6.5% on Topcon 3D OCT-1000 and 4.5% on Zeiss Stratus OCT (see Table 4). However, the type of scanner was almost entirely confounded with centre as most centres used a single type of scanner and some types of scanners were only used in a single centre. So the differences between scanners, as seen in the percentages with MO, could be due to differences between centres or other confounding factors.

All 243 subjects classified as having MO had thickening in at least one of the five inner regions. However, not all of those with thickness in at least one of these regions were classified as having MO after further investigation.

Counts of subjects and eye by centre

The numbers and percentages of subjects from each centre included and excluded from the study are presented in Table 5. The percentages of subjects sum to 100% across the table. Eyes may be excluded for more than one reason so percentages relating to eyes will not sum to 100% across the columns.

Demographics by macular oedema status

There were 3170 subjects with known MO status according to OCT: 243 with MO and 2927 without MO. Demographics are presented in Table 6. Where several mutually exclusive categories are presented, the percentages will sum to 100% when read down the table. If there are only two possible categories, such as gender, then the percentage in the named group is presented (e.g. 60.5% of subjects with OCT MO present are male). The percentages in Table 6 are calculated using the denominators of 243 and 2927, the total number with MO and the total number without MO. Percentages given in the text below are different and are calculated from the number with MO and a characteristic divided by the total number with that characteristic.

There were some statistically significant differences between the 243 subjects with MO and the 2927 subjects without MO. The subjects with MO tended have older median age by 8 years than those without MO. There were no differences in the percentage with MO by gender, glitazone use or amblyopia. A higher percentage of Caucasians (8.4%) had MO compared with other ethnic groups (3–4%). A higher percentage of subjects with type 2 diabetes (8.7%) had MO compared with subjects with type 1 diabetes (3.9%). Those with worse visual acuity (log-MAR ≥ 0.3) or missing visual acuity were more likely to have MO. The age, diabetes type and visual acuity results were likely to have been confounded with each other and this issue was addressed using multivariate analyses.

Counts of subjects and eyes with macular oedema by centre

There were differences in the percentages of subjects with MO by centre (Table 7). Aberdeen, Dundee and Oxford had a far higher prevalence of MO than the other centres. However, these could be due to differences in demographics, presence of lesions, different interpretations of the recruitment strategy or, as noted above, due to differences between scanners. The fact that differences between centres may be measured or unmeasured is why centre was included in the models as an adjustment variable. There were very few subjects with MO in both eyes (0–2.2%) across the seven different centres.

Presence of different types of lesion

Photographic feature information was missing for three left eyes and two right eyes so the denominator for left eyes was 3167 and for right eyes was 3168. Counts of subjects and eyes with MO and features in each eye are presented in Table 8. The same table split by centre is presented in in Appendix 3 (see Table 40). Subjects can have several different lesions or none within each eye.

Mutually exclusive groups of lesions

Mutually exclusive (non-overlapping) groups of features were identified so that every eye could be classified into one group and one group only. The groups for comparison were (1) M/DH only (not BH or exudates), (2) BH but not exudates (BH only + BH and M/DH only), and (3) exudates (regardless of any other feature being present). Prior to recruitment the three mutually exclusive groups of lesions were expected to be present in 11.3%, 1.4% and 3.5% of all scanned images with 83.8% of images having no M/DHs, BHs or exudates. Within the 16.2% of images expected to have lesions (the target population for this study), the three groups of lesions were expected to be found in 69.8%, 8.6% and 21.6% of images, respectively.

In this study, considering the left eyes only, the three groups of lesions were found in 40.3% (M/DHs only within one DD), 8.4% (BH only or BH and M/DHs within one DD) and 20.4% (exudates only or with M/DHs or BHs within one DD). In addition, 28.1% had no lesions within one DD and 2.8% had other lesions within one DD. Very similar percentages were found in the right eyes: 41.7%, 8.7% and 18.9% with 27.9% having no lesions and 2.8% having other lesions. This information is presented by centre in Appendix 3 (see Table 41).

As these percentages did not match those expected, particularly those for M/DHs only, weighting of the subjects according to the features of their worst eye was considered necessary. This will be addressed in the next chapter.

The mutually exclusive features within one DD in the left and right eyes are compared in Table 9. The percentages shown in this table are out of the total of 3165 that had lesion information from both eyes. Two hundred and twenty-two subjects (7.0%) had exudates in both eyes, 39 subjects (1.2%) had BHs in both eyes and 566 subjects (17.9%) had M/DHs in both eyes. Of the 351 subjects (11.1%) that appear to have none or other features within one DD in both eyes, 27 had exudates between one and two DD in at least one eye, some had features such as flames, drusen or CWS and 268 did not appear to have any obvious features within one DD or exudates within two DD.

Single eye analyses

In order to investigate whether or not particular distributions and combinations of lesions identified by manual screening (M/DHs, BHs and exudates) were valuable markers of MO, single eyes were considered separately. Chi-squared tests were used to identify potential risk markers and confounding factors. Logistic regression analyses were used on data from one eye only to identify features identified by manual screening that might be important for predicting the presence of MO within the relevant eye. Different representations of features including presence, count and distribution were considered.

Initially the presence of features within one DD (and for exudates within one to two DD) was investigated in relation to MO within the same eye. In these initial analyses, only single features were considered, the other features being ignored. Following that, analysis combinations of features were considered. The presence of exudates between one and two DD was considered as a separate variable. These logistic regression analyses considered the lesion variable alone (unadjusted), after adjusting for centre, and after adjusting for centre, demographic variables and visual acuity in that eye.

Counts of the different types of lesions (exudates, BHs and M/DHs) within one DD (and for exudates within one to two DD) were first considered singly and then in combination in logistic regression analyses for MO in the relevant eye. These analyses were also adjusted first for centre, and then for demographic variables and visual acuity in that eye. The ranges of counts of lesions were not the same for different types of lesions and there was concern that results of analyses of counts could be dominated by high counts. As a sensitivity analysis, counts of lesions were collapsed into zero, one, two, three, four, five and more than five, and this count variable was included in the logistic regression analysis as if it was a discrete count. This was to ensure that any relationships observed were robust to the influence of very high counts.

There is particular interest in the contribution of M/DHs to the prediction of MO as M/DHs are included in current manual grading practice in England, but not in Scotland. In order to investigate whether or not larger numbers of M/DHs were useful in predicting MO, the counts of all three types of lesions were collapsed even further so they could be included in the logistic regression models as ordered categories: zero, one, two and more than two. The opportunity to observe non-linear increases in the odds of MO with increasing numbers of lesions was provided by this alternative representation of counts of lesions as four ordered categories.

As noted earlier, the combinations of lesion types and better and worse visual acuity did not occur in the proportions expected from a cohort of subjects attending retinal screening and so data weighting was necessary. Weighting of sample data is common in sample surveys either where the sampling scheme uses unequal probabilities by design or if the data are known to be unrepresentative of the population,43 often due to disproportionate non-response. 44 If the sample is known to be substantially different to the population in the distribution of one or more key variables then an analysis of the raw data can produce biased estimates of prevalence. Reweighting can correct for bias in the estimates, but may have the effect of increasing the variances and complicating the analysis. 43 In the current study, if a statistic, such as the sensitivity of a diagnostic algorithm, differs between types of subjects, then the estimate of sensitivity based on the raw sample data could be biased towards the sensitivity within subjects who are over-represented and away from the sensitivity within those under-represented. The under-representation of subjects with only M/DHs was of particular concern. The simplest form of weight, sometimes called direct weights or post-stratification weights, was used. 44–46

The weighting of subjects was performed as follows. The proportions of subjects falling into the five categories were calculated for the study sample, the expected proportions in the five categories being known from a previous study. 30 Weights were obtained by dividing the proportions of subjects in the study sample by the proportions in the population and then multiplying by a factor of 3170/2845 to account for the zero weighting of 325 subjects with either no lesions or only very mild lesions in either eye. Where appropriate, the analyses described above were repeated after weighting. Ideally weighting would have been done to correspond to the population proportions of the five groups within each centre, but these were not known at each centre. This weighting was also used in the health economics analyses in Chapter 6.

Both eyes analysis

Information on features from both eyes was combined in order to look at the prediction of presence of MO (in one or both eyes) per subject. The most serious feature in either eye was given priority in determining the classification of the subject by their lesions. The mutually exclusive categories chosen were:

• any exudates in either eye

• any BHs (but no exudates) in either eye

• more than two M/DHs in either eye (but no exudates or BHs)

• exactly two M/DHs in the eye with most M/DHs (but no exudates or BHs)

• exactly one M/DH in the eye with most M/DHs (but no exudates or BHs)

• those with no M/DHs in either eye and those subjects with lesions which were not M/DHs, BHs or exudates in either eye.

Poor vision was classified as worse (if visual acuity was log-MAR ≥ 0.3 in either eye or if visual acuity was missing in one eye and log-MAR ≥ 0.3 in the other), better (if visual acuity was log-MAR < 0.3 in both eyes) or missing (if visual acuity was missing in one eye and better in the other or missing in both eyes).

Logistic regression was used to investigate this representation of the lesions in relation to predicting the presence of MO in at least one eye. The analysis was repeated first for centre and visual acuity, and then for centre, visual acuity and demographic factors. The analyses of both eyes together were repeated after weighting.

Presence of individual lesion types in relation to macular oedema

In Table 10 the presence or absence of a particular type of lesion in the relevant eye is considered. No adjustments were made for other lesions that might also have been present. A large difference between the percentages of subjects with MO and without MO and with a certain type of lesion present may be due to the presence of another type of lesion.

Macular oedema was roughly five times more prevalent among subjects with an exudate, a BH or a M/DH present within one DD compared with those with the same lesion absent (see Table 10). The prevalence of MO was also greater, albeit by a lesser amount, for subjects having an exudate within one to two DD, but no consideration was given whether or not there was an exudate within one DD as well. Too few eyes had CWS for a robust analysis. There were few eyes with flame haemorrhages or drusen so although there were sufficient eyes for these comparisons to be made, multivariate analyses were not considered appropriate. There did not appear to be any evidence of a relationship between the presence of drusen and the presence of MO in the relevant eye.

Subjects with worse visual acuity were about five times as likely to have MO in the relevant eye as those with better visual acuity.

Only a small number of subjects had amblyopia and, of these, very few had MO. There were no significant differences in the percentages with MO between those using and not using glitazone. In later analyses adjustment was made for glitazone use, but this was not possible for amblyopia because of the small numbers of subjects.

When mutually exclusive groups of lesions were considered, highly significant differences in the percentages with MO were found. More than 10% of subjects with exudates or with BHs (but not exudates) had MO in the same eye. The percentage with MO was just over 2% of those with just M/DHs present in the same eye, about 1% of those with lesions excluding M/DHs, BHs and exudates and < 1% with no lesions in that eye.

Visual acuity was not available for 15 left eyes, two of which had MO, and for 15 right eyes, four of which had MO.

Logistic regression analyses of single eyes

Most of the following tables in this section are for the left eye only. Analyses of the right eyes were also completed but since they showed almost identical relationships these results have not been presented. Table 11 shows the results of logistic regression analysis of a single eye with the outcome variable of MO in that eye. The odds ratio (OR) shows the increase in the odds of having MO in a subject with a characteristic compared with a subject without that characteristic. For a measured variable, such as age, the OR represents the change in the odds of having MO if that variable were to increase by one unit of measurement (1 year older) relative to the same subject without that increase. If multiple variables are included in the same logistic regression model then the OR for a single variable represents the increase in the odds of having MO if that variable were to increase by one unit of measurement, or to change to a different category, compared with a subject with all other characteristics and measurements held fixed.

In Table 11, the top category for each of the categorical variables is chosen as the reference group with OR shown as 1 and no CI displayed. The choice of the reference group is arbitrary and this does not affect statistical significance. The other ORs presented are relative to this reference group and a 95% CI is displayed in the next two columns. If this 95% CI includes 1 it shows that the OR is not significantly different to 1 (the null value) and so p > 0.05. If the 95% CI excludes 1 it indicates that this measured variable is significant for MO with p < 0.05 or that this categorical variable is significant with p < 0.05 and the relevant category is significantly different to the reference category in terms of the odds of having MO.

The first three columns of numbers show ORs and the lower and upper 95% confidence limits for each variable considered separately (unadjusted). The fourth to sixth columns show the ORs and 95% CIs for the variables adjusted for differences between centres. Ideally simultaneous adjustments would be made for centre and scanner type, but they are highly confounded with each other and so could not be included together in the same model. The feature results were similar regardless of whether they were adjusted for centre or scanner so adjustment has only been made for centre. The seventh to the ninth columns show the ORs and 95% CIs for the variables adjusted for centre, gender, age, glitazone and diabetes (type 1, type 2 and secondary/unknown). The last three columns show the ORs and 95% CIs for the variables adjusted for centre, gender, age, glitazone, diabetes (type 1, type 2 and secondary/unknown) and visual acuity.

When the mutually exclusive groups of lesions were included in the model, subjects with BHs (either on their own or BH with M/DHs) had 5.0 times the odds of having MO and subjects with exudates had 6.4 times the odds of having MO, both compared with those with only M/DHs. These ORs changed by only modest amounts as adjustments were made for centre and other characteristics to ORs of 3.6 and 6.7. Of these other variables, only centre, age and visual acuity were significant with older age being associated with higher odds of MO by 4% per additional year older (OR = 1.037; 95% CI 1.020 to 1.054). When visual acuity was included, having worse visual acuity significantly increased the odds of MO by a factor of 3.9 (95% CI 2.6 to 5.9).

An analysis of counts of lesions found that higher numbers of M/DHs, BHs and exudates within one DD significantly increased the odds of having MO when considered separately or together. The relationships were maintained after adjustment for centre, gender, age, glitazone, diabetes (type 1, type 2 and secondary/unknown) and visual acuity. Initially, counts of exudates between one to two DD appeared to be associated with greater odds of MO, but as this ceased to be significant after adjusting for exudates within one DD so it was dropped from subsequent models. It is important to note that the count of M/DHs within one DD was statistically significant after adjusting for centre, the count of BHs and the count of exudates. This suggested that having a lot of M/DHs was still useful for predicting MO.

The OR for counts of BHs was larger than for the other lesions. This does not mean BHs were more important, but that the increase in the odds of MO for each additional BH was larger than for each additional M/DH or exudate. It was thought that this could be a consequence of there being a smaller range of counts of BHs across subjects than the range of counts of M/DHs or exudates.

The maximum numbers of exudates and M/DHs within a single eye of a single subject were 59 and 50, respectively, much higher than the maximum of 16 BHs within a single eye of a single subject. A sensitivity analysis was carried out with the counts of lesions fitted as a discrete measured variable taking a maximum value of six (so zero, one, two, three, four, five, and more than five). The ORs (not shown) were larger for each extra lesion of any type. The inferences of these results were consistent with those shown for raw counts meaning that differences in ranges of counts of different lesions were not responsible for the larger ORs for counts of BHs compared with ORs for counts of exudates or M/DHs.

See Table 42 for the corresponding data for the right eye.

A simple descriptive analysis of count data was completed representing the counts of lesions of the three main types as ordered categories: zero, one, two and more than two (Table 12). The percentages of subjects with MO increased from 1% to 2% to > 3% when the number of M/DHs within one DD increased to more than two. These results suggested that this might be a useful representation when investigating the value of M/DHs in predicting MO.

This pattern was also reflected in the results presented in Table 13, which shows that the odds of having MO increased by a non-significant factor with one or two M/DHs within one DD in comparison with having no M/DHs. However, the OR of having MO dramatically increased to 6.4 when there were more than two M/DHs. This relationship was maintained (OR = 5.2; 95% CI 3.0 to 8.9) after adjusting for the counts of other lesions represented in the same way, and also centre, gender, age, glitazone, diabetes and visual acuity. The fact that the count of several M/DHs within one DD was statistically significant after adjusting for centre, the count of BHs, the count of exudates and visual acuity, suggested that while having any M/DHs might be of little value in predicting MO, having a lot of M/DHs in an eye was still useful for predicting MO in that eye. See Table 43 for the corresponding data for the right eye.

The unweighted analyses presented in Tables 11 and 13 were repeated using weighted analyses, the results are presented in Tables 14 and 15. These analyses had the effect of up-weighting data from subjects with M/DHs and down-weighting data from subjects with more serious lesions. This small inflation had no influence on the ORs, but made CIs narrower by a very small amount. See Tables 44 and 45, respectively, for the corresponding data for the right eye.

When the mutually exclusive groups of lesions were included in the model, subjects with BHs (alone or BH with M/DHs) had 4.4 times the odds of having MO and subjects with exudates had 6.0 times the odds of having MO, both compared with those with only M/DHs. After adjusting for centre and other characteristics, these ORs changed by only modest amounts to 3.1 and 5.8, respectively. Of these other variables, only centre, age and visual acuity were significant with older age being associated with higher odds of MO by 3% per additional year older (OR = 1.031; 95% CI 1.011 to 1.051). When visual acuity was included, having worse visual acuity significantly increased the odds of MO by a factor of 3.8 (95% CI 2.3 to 6.3).

An analysis of counts of lesions found that higher counts of M/DHs, BHs and exudates within one DD, considered separately and together, significantly increased the odds of having MO. The relationships were maintained after adjustment for centre, gender, age, glitazone, diabetes (type 1, type 2 and secondary/unknown) and visual acuity. Initially, counts of exudates between one to two DD appeared to be associated with greater odds of MO, but this ceased to be significant after adjusting for exudates within one DD so this count was dropped from subsequent models.

It is important to note here, as with the unweighted analyses, that the count of M/DHs within one DD is statistically significant after adjusting for centre and the count of BHs and the count of exudates. This suggests that having a lot of M/DHs is still useful for predicting MO.

The OR for counts of BHs was again larger than for the other lesions. This does not mean that BHs were more important, but that the increase in the odds of MO for each additional BH was larger than for each additional M/DH or exudate. It was thought that this could be a consequence of there being a smaller range of counts of BHs across subjects than the range of counts of M/DHs or exudates, but as with the unweighted analyses, a sensitivity analysis showed that this was not the case.

This pattern of the increasing prevalence of MO with increasing numbers of lesions was also reflected in the results presented in Table 13. The odds of having MO increased by a non-significant factor with one or two M/DHs within one DD in comparison to having no M/DHs and the OR of having MO dramatically increased to 5.2 when there were more than two M/DHs. This relationship was maintained (OR = 5.0; 95% CI 2.6 to 9.3) after adjusting for the counts of other lesions represented in the same way: centre, gender, age, glitazone, diabetes and visual acuity.

Some of the ORs presented in Table 15 were large with very wide CIs, particularly those for having several BHs. This was because weighting had the effect of reducing the apparent number of subjects with BHs and subjects with MO. This would tend to make the estimates of ORs and their CIs more volatile. After adjustment for centre, gender, age, glitazone use, diabetes type and visual acuity these estimates of the ORs for having BHs were more stable and had inferences more similar to the earlier unweighted analyses. The CIs for missing visual acuity were very wide and the width varied greatly because of the very small numbers with missing visual acuity.

Relationship of individual lesions in both eyes to macular oedema in either eye (analysis unweighted)

Single eye analyses are simple to interpret and are helpful in establishing factors which might contribute to a prediction model. However, they do not show how a strategy might work on a subject where information about the possibility of MO comes from both eyes. Mutually exclusive groups were defined taking information about lesions within one DD from both eyes (Table 16). The most serious lesions in either eye took priority in assigning subjects to six groups. Any exudates within one DD in either eye meant the subject was assigned to the exudates group. A BH within one DD in either eye, without any exudates, meant the subject was assigned to the BH group. For those with M/DHs in both eyes (and no BHs or exudates) or M/DHs (and no BHs or exudates) in one eye and no exudates or BH in the other, the group was defined by the number of M/DHs within one DD in the eye with the most M/DHs. A subject with no M/DHs, BHs or exudates within one DD was assigned to the ‘None/other both eyes’ group. Subjects were also classified into three groups according to visual acuity [better visual acuity (log-MAR < 0.3; Snellen 6/9.5 or better) in both eyes, worse visual acuity (log-MAR ≥ 0.3; Snellen 6/12 or worse) in at least one eye, missing visual acuity in both eyes or missing visual acuity in one and better visual acuity in the other].. The percentage of subjects with MO was calculated for each combination of visual acuity and lesions. There was a substantial two- to fourfold increase in the percentage with MO if visual acuity was worse rather than better in all of the lesion groups. There were very few subjects with MO among those subjects with better visual acuity and zero to two M/DHs within one DD in either eye.

Logistic regression of the relationship of individual lesions in both eyes to macular oedema in either eye (analysis unweighted)

Odds ratios for three models for predicting MO are presented in Table 17. From left to right these models include: only lesion type and the count of M/DHs (the unadjusted ORs for visual acuity are also presented here); lesion type and the count of M/DHs adjusted for centre and visual acuity; and lesion type and the count of M/DHs adjusted for centre, visual acuity and demographic variables. Exudates, BHs or more than two M/DHs within one DD in either eye were all predictive of having MO, as was worse visual acuity. Having one or two M/DHs within one DD in an eye did not appear to be any different to having none/other mild lesions. The odds of having MO were increased by factors of 11.2 for having at least one eye with an exudate, 5.9 for having at least one eye with a BH (but no exudates), 3.4 for having more than two M/DHs relative to having eyes with a maximum of one M/DH in either. Having a M/DH in one or both eyes was not significantly different to having a maximum of two M/DHs in either eye or having no lesions or mild lesions (not a M/DH, BH or exudate).

Having worse visual acuity was predictive of MO and appeared to increase the odds of having MO by a factor of 3.3. Having worse visual acuity increased the size of the odds of MO for all types of lesion and the size of this increase was not significantly different between lesions types.

The ORs for the fully adjusted model including lesions, centre, visual acuity and demographic variables are given in the final three columns. Other variables which were significant in this fully adjusted model were age and centre. For every additional year older the odds of having MO increased by a factor of 1.035 (95% CI 1.022 to 1.049) in other words by 3.5% per year older.

Subjects with one M/DH within one DD in one or both eyes were taken as the reference group so ORs for having MO in either eye are calculated compared with this group. It was not appropriate to take the group with no lesions/other lesions as the reference group for the analyses of both eyes because many of this group would be zero weighted in the weighted analyses.

The 95% CI for the OR of having MO in those with, at most, two M/DHs within one DD in either eye relative to at most one M/DH in either eye included 1. This was also true of the CI for no lesions or lesions other than M/DHs, BHs or exudates in either eye. This meant that there was no significant difference in the odds of having MO in these subjects compared with subjects with at most one M/DH within one DD in either eye. The fact that the count of several M/DHs within one DD was statistically significant after adjusting for centre, the count of BHs and the count of exudates suggested that having a lot of M/DHs within one DD was still useful for predicting MO. This remained true after adjusting for other variables including visual acuity.

If the 11 subjects with missing visual acuity were excluded, the results were almost identical to those above. With this restriction it was possible to include an interaction between visual acuity and the feature category. There was no evidence of an interaction between these variables and this suggested that both lesion category and visual acuity were important in predicting presence of MO, but the strong effect of having worse visual acuity did not vary between different lesion groups.

The receiver operating curves shown in Figure 12 correspond to the three models presented in Table 17. The model that included only the lesion categories within one DD in at least one eye [at least one M/DH, two M/DHs, more than two M/DHs, at least one BH (but no exudates), at least one exudate] had a receiver operating characteristic curve made up of straight lines between the points (sensitivity and 1-specificity) corresponding to the threshold being changed to include an extra category. For example, if only subjects with at least one exudate were referred, then the sensitivity and specificity would be 0.59 and 0.70, respectively. If subjects with at least one exudate and subjects with at least one BH were referred, then the sensitivity and specificity would be 0.80 and 0.57, respectively (1 − specificity = 0.43). The other two receiver operating characteristic curves correspond to the other two models presented in Table 17 where one model included lesion type and the count of M/DHs, centre and visual acuity, and the other model included lesion type and count of M/DHs, centre, visual acuity and demographic variables. There were gains in sensitivity and specificity from choosing more complex models as would usually be expected.

FIGURE 12.

Receiver operating characteristic curves for models that are presented in Table 17 for MO in one eye or both eyes. VA, visual acuity.

Relationship of individual lesions in both eyes to macular oedema in either eye (analysis weighted)

Similar models to those in Table 17 are presented in Table 18 for weighted data. Subjects with no lesions in either eye or only lesions other than exudates, BHs and M/DHs within one DD were removed from this analysis. For the weighted data exudates, BHs or more than two M/DHs within one DD in either eye were all predictive of having MO, as was worse visual acuity (see Table 18). Having one or two M/DHs within one DD in an eye did not appear to change the odds of MO.

There were some large ORs observed in the table, again due to the down-weighting of the subjects with more severe groups of lesions so the estimates were more volatile than for the unweighted analyses. This was less pronounced after adjustment for other variables. The odds of having MO were increased by factors of 11.0 for having at least one eye with an exudate, 5.9 for having at least one eye with a BH (but no exudates) and 3.5 for having more than two M/DHs within one DD relative to having eyes with a maximum of one M/DH within one DD in either. Having a maximum of two M/DHs within one DD in either eye was not significantly different to having a M/DH in one or both eyes.

Having worse visual acuity was predictive of MO and appeared to increase the odds of having MO by a factor of 3.3. Having worse visual acuity increased the size of the odds of MO for all types of lesion and the size of this increase was not significantly different between lesions types.

The ORs for the fully adjusted model including lesions, centre, visual acuity and demographic variables are given in the final three columns. Other variables which were significant in the fully adjusted model were age and centre. For every additional year older the odds of having MO increased by a factor of 1.029 (95% CI 1.014 to 1.044); in other words by 3.0% per year older.

As for the unweighted analysis, the 95% CI for the OR of having MO in those with at most two M/DHs within one DD in either eye relative to at most one M/DH within one DD in either eye included 1. This was also true of the CI for no lesions or lesions other than M/DHs, BHs or exudates in either eye.

This meant that there was no significant difference in the odds of having MO in for these subjects compared with subjects with at most one M/DH in either eye. The fact that the count of several M/DHs within one DD was statistically significant after adjusting for centre, the count of BHs and the count of exudates suggested that having a lot of M/DHs was still useful for predicting MO. This remained true after adjusting for other variables including visual acuity.

If the data were restricted to exclude those with missing visual acuity as well as those with no features or features other than M/DHs, BHs and exudates, the results were almost identical to those described above. With this restriction it was again possible to include an interaction between visual acuity and feature category.

There was no evidence of an interaction between these variables and this suggested that both lesion category and visual acuity were important in predicting presence of MO, but the effect of having worse visual acuity did not vary between different lesion groups.

Manual grading strategies

The manual strategies used the following photographic features:

• M/DH within one DD

• M/DH between one DD and two DD

• BH within one DD

• BH between one and two DD

• exudates within one DD

• exudates between one DD and two DD.

In addition, the following feature was used:

• visual acuity worse than or equal to Snellen 6/12 (log-MAR 0.3).

Eighteen different combinations of these features were modelled as the manual grading strategies and are listed in Table 19. Manual grading strategies 1 and 2 model the current practice in the English and the Scottish national programmes, respectively. The statistical analysis in the earlier part of this chapter showed that exudates, BHs and M/DHs within one DD, as well as reduced visual acuity, all significantly increase the risk of having MO. Furthermore, the risk increases with the number of lesions present within one DD. Lesions outside of one DD, including exudates, were not found to be predictive of MO. Manual grading strategy 4 (M/DHs within one DD only), manual grading strategy 5 (M/DHs or BHs within one DD), manual grading strategy 9 (M/DHs within two DD) and manual grading strategy 10 (M/DHs or BHs within two DD) investigate red lesions alone. Manual grading strategy 6 (exudates within one DD) and manual grading strategy 11 (exudates within two DD) examine bright lesions alone. Manual grading strategy 15 considers visual acuity alone. The remainder of the manual grading strategy 16 strategies explore combinations of red and bright lesions with visual acuity.

Computer-assisted manual annotation grading strategies

A computer-assisted manual annotation grading strategy differs from manual grading in that an automated classifier (described later) is used to combine the image information. This means that improved performance can be achieved because of the association between features. Also, it is possible to include more complex features, such as the number and area of lesions, although this would require additional grading time. As shown in Table 20, the computer-assisted manual annotation grading strategy also used computer intensity measurements within the macula (as explained below), visual acuity and other patient information.

Fully automated annotation grading strategies

Like the computer-assisted annotation grading strategies, the fully automated annotation grading strategies take advantage of richer information within the image, but with all the information derived automatically from the image without any human intervention. Clearly, as the automated lesion detection system approaches the performance of the manual observer, the performance of the fully automated annotation systems should be similar to the best computer-assisted annotation grading strategies. Three fully automated annotation grading systems were tested, two of which incorporated visual acuity and other patient information (see Table 20). The automated analysis produces technical failures (i.e. images of insufficient quality for automated analysis) and these were handled by using manual grading strategy 16 as illustrated in the flowchart in Figure 13.

FIGURE 13.

A fully automated annotation grading strategy, using manual grading to determine the outcome for technical failures from automated macular disease assessment. Manual grading strategy 16 was used because it performs best relative to the receiver operating characteristic curves shown in Figure 16 while making a reasonable compromise of sensitivity and specificity.

Automated photograph analysis

This section describes the image analysis methods used in the computer-assisted annotation manual and fully automated annotation grading strategies.

Computer intensity measurements

It was noted that in subjects with MO the macular area often appears darker than the surrounding retina (although this must be distinguished from pupillary shadows, which are a common artefact when the pupil diameter is small). This effect is even more noticeable when imaging using infrared illumination, where the wavelengths are absorbed more strongly by the oedema. To quantify this difference in the macular intensities the following measures were calculated automatically from the retinal photographs (Figure 14).

FIGURE 14.

Regions of interest centred on the fovea, used to calculate the computer intensity measurements. The first region, labelled ‘1’, is a circle with a diameter approximately equal to that of the optic disc. The second region, labelled ‘2’, is an annular region with an inner diameter of 1.5 disc diameters and outer diameter of a 2 disc diameters.

1. The ratio of the mean red and mean green colour channel values within one DD.

2. The ratio of the mean red and mean green colour channel values in the annular region between one and two DD.

3. The ratio of the mean green colour channel in the circular one DD region and the annular region between one and two DD.

4. As above, but using the red colour channel.

Note that these measures may be affected by several common features such as reflections from the internal limiting membrane in younger subjects (Figure 15c), the presence of bright exudate or drusen lesions (Figure 15d) or a shadow cast by a small pupil (Figure 15e). However, the measures may have predictive value despite these confounding factors.

FIGURE 15.

(a) Typical macular appearance with a dark region indicating the position of the centre of the fovea; (b) example retina with a darker macular region; (c) retina showing substantial reflections from the internal limiting membrane, which is often a feature in younger subjects; (d) bright and dark lesions can also affect the greyscale measurements; (e) technical problems with the image, such as pupillary shadows, will affect the greyscale measurements.

Overview

In previous chapters the sensitivity and specificity of the alternative screening strategies has been established using the OCT outcome as the reference standard. There were 329 individuals in the data set where the OCT outcome was a technical failure in at least one eye, resulting in unknown oedema status at the patient level (OCT technical failure), and a further 41 individuals with two inadequate photographs. These subjects were excluded from the analysis of sensitivity/specificity at the patient level, but the expected costs associated with OCT technical failure were incorporated in the economic models comparing alternative grading strategies involving the use of OCT within the screening programme.

The short-term costs to the NHS associated with the alternative referral criteria were estimated by applying unit costs for screening and ophthalmology referral, according to the care pathway that patients in the data set would be expected to follow under the alternative strategies. For strategies entailing the use of OCT for patients within the screening programme, prior to referral, OCT costs were also applied. The individual cost estimates were derived as described below.

Following this unadjusted analysis, a further analysis was undertaken whereby the proportions of patients with different types/patterns of surrogate markers were weighted to reflect the relative frequency with which these surrogate markers would be expected to arise in a consecutive screening cohort. In order to achieve this, four patterns with retinal pathology categories were defined based on the type and location of surrogate markers present, and these were applied in a hierarchical manner such that patients were assigned to the first category for which either of their eyes met the inclusion criteria. The sensitivity and specificity of the alternative grading strategies were determined separately for patients within each of these categories. The resultant estimates were then applied in a decision tree model (Figure 17) where the proportions of patients within each surrogate marker category were set equal to those observed in a prior screening cohort study carried out in Grampian. 33 This approach is equivalent to that used to estimate weighted sensitivities and specificities in the previous chapter.

FIGURE 17.

Structure of decision tree model for estimating cost per case detected (showing the arm for manual grading strategy 1 coupled with OCT). CSL, cost of slit-lamp; DMO, diabetic macular oedema; SL, slit-lamp.

As the cost of a referral to the eye clinic was found to be a key driver of cost-effectiveness, and the estimate for this parameter varied substantially between Scotland and England, separate analyses were conducted for the two countries.

Comparators

It was not feasible to consider all potential grading strategies in the economic analysis. The grading strategies chosen included the current English manual grading strategy (1) and the current Scottish manual grading strategy (2). Manual grading strategy 2 is one of the highest specificity simple manual grading strategies after weighting, whereas manual grading strategy 1 represents a moderate sensitivity, moderate specificity manual grading strategy (see Table 23). Based on examination of the patient-level sensitivities/specificities of alternative simple manual strategies (see Table 23), manual grading strategy 16 was also included as a potential alternative to manual grading strategy 1 (having similar sensitivity and higher specificity). In addition, consideration was given to the potential cost-effectiveness of utilising the fully automated annotation grading strategy with the inclusion of visual acuity (denoted FA2) running at an operating point on its receiver operating characteristic curve (see Figure 16), with slightly higher sensitivity and better specificity than manual grading strategy 16. Where automated grading flagged a patient's images as being of ‘insufficient quality for grading’, these patients were modelled to fall back to manual grading with manual grading strategy 16. Manual grading strategy 16 was chosen because of its similar sensitivity and superior specificity to manual grading strategy 1.

The impact of combining the selected manual grading strategies with OCT within the screening pathway prior to referral, to minimise false-positive referrals to eye clinics, was also assessed (see Figure 17). In addition, the 100% sensitive manual grading strategy 8 (any exudate or BH within one DD, or any M/DH within one DD and visual acuity ≥ 0.3 log-MAR) coupled with OCT prior to referral was considered.

The automated annotation grading strategy incorporating further patient characteristics was not included in the economic analysis as this was incomparable with the simple manual grading strategies (which did not utilise such information). Similarly, manual grading strategies relying on the full manual annotation of all retinal images were not included as these would be impractical for clinical practice. Thus, in total nine strategies were included in the analysis.

Costs

Costs were estimated using a variety of sources as described below. All costs were expressed in UK sterling for the 2009–10 financial year. Although discounting was not required for the cost-effectiveness analysis, the costs of capital items were annuitised over the useful lifespan of the item in question and a discount rate of 3.5% per annum was applied to account for the opportunity cost of the investment over time. 55 Table 24 summarises all the unit costs applied in the analysis of the cost per case detected.

Analysis of cost per case detected

The cost per case detected was assessed by determining the expected screening, referral and treatment costs, and the expected number of true cases of MO referred under the alternative grading strategies. Initially, the analysis was conducted by applying the unadjusted sensitivity and specificity estimates in the decision tree model, with the prevalence of MO set equal to that observed in the clinical data set (7.7%). To capture the expected costs associated with OCT technical failure, for grading strategies incorporating OCT, the observed OCT technical failure rate (10.5%) was applied and those experiencing a technical failure were modelled to incur the cost of both the OCT examination within the screening programme and a subsequent referral to the eye clinic. Patients assigned the outcome of photographic technical failure by a particular strategy were modelled to receive a slit-lamp examination within the screening programme. It was assumed that these patients would receive a slit-lamp examination which would ultimately result in the correct outcome being assigned (refer or recall). Thus, a number of patients correctly identified as having MO with each strategy are in fact identified following a slit-lamp examination.

A comparison of the proportions of patients in the surrogate marker categories with the proportions observed in these categories in the prior cohort study (Table 25) suggested oversampling in the current study of patients with BHs or exudates within one DD and undersampling of patients with only M/DHs in the macula, or exudate(s) between one and two DD. To adjust for this in the cost-effectiveness analysis, patients were first assigned to one of the surrogate marker categories in Table 25. Moving through the categories in descending order they were assigned to the first category that either of their eyes satisfied. The prevalence of MO and the sensitivity/specificity of the alternative referral strategies were then estimated separately for each of the patient surrogate marker categories (Table 26) and applied in the decision tree model with the categories reweighted using the frequency proportions provided in Table 25. These frequency proportions were estimated using a subset of 1099 patients with surrogate markers arising in a consecutive cohort of 6370 individuals (with complete data) screened in Grampian. 33 They are suggestive of a maculopathy incidence rate (based on an English referable maculopathy grade) of ∼ 6.3%. The process of frequency weighting patients in the decision model is equivalent to the design weighting procedure carried out for the statistical analysis.

Results were expressed for a cohort of 3170 patients with lesions within two DD centred on the fovea. Total screening and referral costs, true cases detected and false-positive referrals were tabulated for each strategy. Each grading strategy was compared incrementally in terms of its additional cost per extra case of MO detected in comparison with the most specific strategy. The results are presented both with and without the use of OCT prior to referral. The cost and effect of each grading strategy is also plotted graphically on the cost-effectiveness plane, with a line joining those grading strategies that represent potentially cost-effective options dependent on decision makers' willingness to pay per extra case of MO detected (cost-effectiveness frontier).

Sensitivity analysis

Cost per case detected

Table 26 shows the number and proportion of patients in the clinical data set who would be referred/recalled by the alternative grading strategies, by surrogate marker category and the presence/absence of diabetic MO. These data were used to populate the decision tree model so that cost-effectiveness could be estimated based on the patient category frequencies observed within the current study, and also with the more realistic frequency weights from the prior cohort study. 33

Tables 27 and 28 show the anticipated costs and consequences, from one round of screening, of using the alternative grading strategies to screen patients in the clinical data set in their unadjusted frequencies (applying English and Scottish screening and referral costs, respectively). Tables 29 and 30 show the expected costs and outcomes for a screening cohort of the same size, but with the patient categories reweighted to reflect their expected frequency within screening programmes. Table 29 presents the findings applying the English unit costs for screening and referral, whereas Table 30 shows the same results applying Scottish unit costs. Figures 18 and 19 compare all the grading strategies graphically on the cost-effectiveness plane (applying English and Scottish costs, respectively). Grading strategies falling above or behind the lines joining points on the plane represent options that are dominated (more costly and less effective than alternative options) under base-case assumptions. Those grading strategies falling on the lines offer potentially cost-effective options dependent on decision makers' willingness to pay per extra case of MO detected.

FIGURE 18.

Cost-effectiveness frontier for all strategies (applying English costs).

FIGURE 19.

Cost-effectiveness frontier for all strategies (applying Scottish costs).

The results show that, although manual grading strategy 2 (current Scottish practice) is slightly more sensitive, less specific and more costly than the manual grading strategy 1 (current English practice), based on the unweighted analysis (see Tables 27 and 28), the reweighting reverses this finding such that manual grading strategy 2 becomes substantially less sensitive, more specific and less costly than manual grading strategy 1 (see Tables 29 and 30). The results in Tables 29 and 30 also suggest that manual grading strategy 16 would dominate strategy 1. The automated grading strategy (coupled with the linear classifier and fall back to manual strategy 16 in the event of automated technical failure) would also dominate manual grading strategy 1 and manual grading strategy 16 assuming it could be implemented without increasing net grading costs to a programme. A common finding from the base-case analyses is that the addition of OCT to each grading strategy (within the screening programme) prior to referral, results in a reduction in costs to the health service, with no decrement in the number of MO cases detected.

The incremental cost per extra case of MO detected varies by grading strategy. Considering the more sensitive manual grading strategy 16 compared with manual grading strategy 2 (see Tables 29 and 30), the additional cost per extra case detected ranges from £1579 (English costs) to £985 (Scottish costs). When the manual grading strategies are coupled with OCT, these incremental costs fall to £636 and £528, respectively. Although the automated annotation grading strategy dominates manual grading strategies 1 and 16 under the base-case assumption of zero net increase in grading costs, it costs £897 per additional case detected and referred in comparison with manual grading strategy 2 (applying English screening and referral costs). This incremental cost drops to £405 when the grading strategies are coupled with OCT prior to referral. The 100% sensitive strategy 8 (OCT for all patients with any exudate or BH less than one DD, or any M/DH less than one DD and visual acuity ≥ 0.3 log-MAR) costs an additional £1510 per extra case detected in comparison with the most specific grading pathway (manual grading strategy 2 + OCT) when applying English referral costs, and £1360 per extra case detected when applying Scottish referral costs. In comparison with manual grading strategy 16 plus OCT, strategy 8 costs £1955 and £1784 per extra case detected and referred when applying English and Scottish referral costs, respectively.

Deterministic sensitivity analysis

Considering manual grading strategy 1 (current English practice), threshold analysis suggests that the cost of implementing OCT within the screening pathway (to reduce false-positive referrals) can rise to ∼ £113 per patient before it starts increasing the overall cost per true-positive referral above the level obtained in its absence (assuming English referral costs). The corresponding threshold cost for OCT coupled with manual grading strategy 2 is ∼ £110. Applying the lower Scottish referral costs, the OCT threshold costs fall to £71 and £69 for manual grading strategies 1 and 2, respectively. To put this in perspective, if an associate specialist were to perform OCT examinations within screening, and applying the same input assumptions as those used for the cost estimate based on a band 6 retinal screener (see Appendix 2), the corresponding cost would be ∼ £64. As expected, lower referral costs also improve the cost-effectiveness of the more sensitive, less specific manual grading strategies.

Further sensitivity analysis was undertaken to determine the threshold for the net incremental grading cost associated with automated annotation grading that would result in it no longer dominating manual grading strategy 1 in terms of the cost per case detected and referred. This showed that the automated classifier would remain dominant over manual grading strategy 1 up to an additional cost of ∼ £9.00 per patient graded (assuming English referral costs). The threshold cost drops to ∼ £3.00 when considering the manual grading strategies coupled with OCT. Note that an alternative operating point could also be established on the automated classifier to obtain improved specificity for the same sensitivity as any of other simple manual grading strategies.

The results of further sensitivity analyses are presented in Table 31. In general, the ordering of grading strategies was not found to be particularly sensitive to variation in key model parameters.

Probabilistic sensitivity analysis

The probabilistic analysis characterises the joint uncertainty surrounding the estimated incremental costs and cases of MO detected with each of the alternative strategies (Figures 20–22). The acceptability curves indicate the probability of each grading strategy being the preferred option given different values of decision makers' maximum willingness to pay per extra case of MO detected and referred. The preferred option changes as this threshold increases.

FIGURE 20.

Cost-effectiveness acceptability curves for the alternative grading strategies based on English costs.

FIGURE 21.

Cost-effectiveness acceptability curves for the alternative grading strategies based on Scottish costs.

FIGURE 22.

Cost-effectiveness acceptability curves excluding the automated strategies and based on English referral costs.

Applying English referral costs (see Figure 20), manual grading strategy 2 (Scottish criteria) coupled with OCT has the highest probability of being considered cost-effective up to a ‘willingness-to-pay’ threshold of ∼ £450 per extra case of MO detected. Above this ratio, automated annotation grading (coupled with OCT) has the higher probability of being the preferred option up to a ‘willingness-to-pay’ threshold of ∼ £2450. Note, however, that this is dependent on the assumption that automated annotation grading can be implemented without increasing grading costs to the screening programme. Above a ‘willingness-to-pay’ threshold of ∼ £2450, manual grading strategy 8 coupled with OCT has the highest probability of being preferred on grounds of cost-effectiveness. Figure 21 indicates that a similar pattern of results is obtained when applying the Scottish referral costs, although the thresholds for adopting the more sensitive (less specific) manual grading strategies are slightly lower because of the lower referral costs. Figure 22 indicates the choices between manual grading strategies when the automated grading procedures are taken out of the comparison (applying English costs). Under this scenario manual grading strategy 16 plus OCT has the higher probability of being the preferred option above a ‘willingness-to-pay’ threshold of ∼ £800 per case detected and referred, before manual grading strategy 8 plus OCT takes over above the ‘willingness-to-pay’ threshold of ∼ £2000 per additional case.

Model overview

In order to estimate the longer-term cost-effectiveness of the alternative referral criteria, a Markov microsimulation model was developed to simulate the progression of MO and visual loss in referred and un-referred patients over time (Figure 23). Patients with MO referred to the eye clinic were modelled to receive treatment and incur a lower risk of disease progression and visual impairment. Costs associated with screening, referral, treatment, ongoing monitoring and vision loss were incorporated in the model – as were utility weights associated with varying degrees of visual loss – allowing costs and QALYs to be estimated over a number of years.

FIGURE 23.

Diagrammatic representation of the model structure. BSE, better seeing eye; SVL, severe visual loss; VA, visual acuity.

A microsimulation approach was used whereby individual patients with characteristics matching those of the patients in the clinical data set were randomly selected into the model one at time and simulated to progress. The modelled cohort consisted of patients with any surrogate photographic markers within one DD of the fovea, and also those with exudates between one and two DD. The relative proportions of patients with different types of macular features were adjusted to reflect their expected frequency within screening programmes (using the frequency weights in Table 25).

Based on the initial visual acuity in each eye, patients were assigned to one of six health states defined by their visual acuity in the better-seeing eye; visual acuity in the better-seeing eye is generally considered to be the better determinant of health-related quality of life,60 and the health-related quality of life in patients with diabetic retinopathy is more commonly reported according to visual acuity in the better-seeing eye. These health states are represented by the nodes emanating from the Markov (strategy 5) node in Figure 23. The model was constructed to cycle on a 6-monthly basis (the average time interval between screening/monitoring appointments for patients with observable but non-referable maculopathy) with the tree to the right of the Markov health states representing the clinical pathways that simulated patients could follow within each cycle of the model. Tracker variables were used to update patient history and screening/referral decisions so that the pathway taken by patients in subsequent cycles of the model could be determined based on this prior information.

Patients were simulated to enter the model at an index screening visit. The adjusted sensitivity and specificity of the different strategies (see Table 22) were applied to patients within the alternative arms of the model. Depending on the sensitivity/specificity of the screening strategy, patients could then either be referred to the eye clinic or remain in the screening programme. The visual acuity of eyes with MO remaining in the screening programme was modelled to improve/deteriorate at the rates observed for untreated eyes in the ETDRS,61 while eyes with MO referred to the eye clinic were assumed to receive timely treatment and improve/deteriorate at the rates observes for laser-treated eyes in the ETDRS and a more recent UK-based observational study. 61,62 The change in visual acuity for each eye with MO was modelled using 6-monthly probabilities of improvement/deterioration derived from the aforementioned studies. Eyes without MO were assumed to remain stable until the development of MO. At the end of each model cycle each patient was assigned the appropriate health state for the subsequent model cycle, based on the updated visual acuity in their better-seeing eye (furthest right nodes in Figure 23).

Health state utility weights obtained from the available literature63 were used to quality adjust the time spent by patients in each visual acuity state, so as to generate QALYs. Health state utilities reflect the relative desirability of different states of health on a scale where zero represents death and one represents full health. So, for example, a year spent in the health state ‘visual acuity ≥ 74 letters’, which was assigned a utility weight of 0.83, would generate 0.83 QALYs (see Utilities for further details).

Costs associated with screening, referral, treatment, clinical observation and legal blindness (visual acuity ≤ 35 ETDRS letters) were incorporated into the model. For strategies where OCT monitoring was not available within the screening programme, referred patients with no MO were modelled to remain under 6-monthly observation in the eye clinic. Patients not referred to the eye clinic at any given screening visit were assumed to undergo screening again one year later – when they would again experience a probability of being referred to the eye clinic. In the base-case analysis it was assumed that patients with and without MO would face a constant probability of being referred each year, based on the adjusted sensitivity and specificity of the strategy in question (see Tables 29 and 30). As an alternative we conducted a sensitivity analysis where it was assumed that patients not referred on the index screening visit would remain non-referable (according to their surrogate markers) for 2 years and 5 years. During these time periods the visual acuity in eyes with MO was modelled to deteriorate at the rates observed for untreated eyes. 64

At any stage in the simulation patients could die from age-specific all-cause mortality, which was modelled based on UK life tables combined with hazard ratios for all-cause mortality associated with diabetes. 65,66 Further details of the model structure, parameters and assumptions are provided opposite.

Modelled cohort

The modelled cohort consisted of patients with any surrogate photographic markers within one DD of the fovea, and also those with exudates between one and two DD. In sampling patients for entry into the model the frequency proportions from Table 25 were applied so that simulated patients in the different surrogate marker categories would appear in the model with the same relative frequency as anticipated within screening programmes. The clinical characteristics [age, gender, type of diabetes, visual acuity (in each eye), macular surrogate marker category, retinopathy grade, and MO status (no MO/MO in right eye/MO in left eye/MO in both eyes)] of simulated patients were recorded using tracker variables, some of which were modelled to update over time (age, visual acuity, MO status).

The grading strategies

The grading strategies chosen for comparison included those examined in the cost-effectiveness analysis. This included the highly specific grading strategy 2, and the more sensitive and less specific grading strategy 1, strategy 16, and the automated annotation grading strategy FA2 (set at ∼ 75.9% sensitivity and 73.7% specificity). These grading strategies were assessed both with and without the presence of OCT screening prior to referral to the eye clinic. The maximally sensitive grading strategy 8 coupled with OCT was also assessed.

Progression to macular oedema

Eyes with no MO at baseline were modelled to progress to MO using cumulative incidence data reported by Younis et al. 67,68 for a screening cohort in Liverpool. Younis et al. reported the yearly cumulative incidence of referable maculopathy, up to 6 years, in patients with no retinopathy, background retinopathy and mild pre-proliferative retinopathy in their worst eye at baseline. The reported yearly cumulative incidence rates were used to estimate the annual risks of progression to referable maculopathy conditional on survival. These annual risks were then averaged and converted to average 6-monthly risks. Finally, these probabilities were converted into probabilities of developing OCT-positive MO, based on the positive predictive value of the English referral criteria for MO (manual strategy 1) from Table 29. For eyes with moderate/severe retinopathy at baseline, the 3-year incidence of clinically significant MO, as reported for the placebo arm in the Protein Kinase C-DRS2 trial,69 was used to estimate 6-monthly probabilities of progression. As an alternative approach to estimating rates of progression to MO in patients with type 2 diabetes, the proportion of patients (with varying degrees of retinopathy at baseline) receiving photocoagulation for MO at 6 years was estimated from the UK Prospective Diabetes Study70 and used to generate 6-monthly probabilities of progression for the eyes of modelled patients (Table 32).

Based on the observed proportion of patients recruited to the clinical study with MO in both eyes (12% of those with MO in any eye), it was assumed that 12% of incident MO cases would present with both eyes affected. It was also assumed that for patients with MO in one eye, the risk of developing MO in the other eye would be increased. In a case–control study, Bhavsar and Subramanian71 found that a prior history of ‘clinically significant’ MO in either eye significantly increased the risk of progression from subclinical MO to ‘clinically significant’ MO [unadjusted OR = 8.07 (CI 2.75 to 23.63)]. In a prospective cohort study, Varma et al. 72 found the incidence of ‘clinically significant’ MO in the second eye of patients with ‘clinically significant’ MO in one eye to be double the incidence of ‘clinically significant’ MO in a first eye. It was therefore assumed that the risk of eyes developing MO by underlying retinopathy grade (see Table 32) would increase similarly for simulated patients with MO already present in one eye.

A simplifying assumption of the model was that no eyes lose vision prior to the development of MO. This assumption should have limited impact on the comparison of alternative referral strategies, as any patient at risk of vision loss from proliferative retinopathy would in practice be referred to ophthalmology services regardless of whether or not MO was suspected. The rates of vision loss applied to eyes with MO in the model may also capture vision loss resulting from co-existing retinopathy.

Progression of visual impairment in eyes with macular oedema

The visual acuity of referred eyes with MO was modelled to deteriorate/improve at the rates observed for laser-treated eyes with ‘clinically significant’ MO, as defined by the ETDRS, whereas non-referred eyes with MO were modelled to deteriorate at the rates observed for untreated eyes with ‘clinically significant’ MO. 64

Non-referred eyes

Owing to the long-established clinical practice of laser treatment for the prevention of visual loss in eyes with ‘clinically significant’ MO,64 there is a lack of contemporaneous data on visual acuity outcomes for untreated eyes. Thus, non-referred eyes with MO were modelled to follow the visual acuity course observed for eyes with ‘clinically significant’ MO randomised to deferral of treatment in the ETDRS. 64 Although improved glucose control may reduce the rate at which patients now develop ‘clinically significant’ MO,73 the definition of MO used in the current analysis maps closely to the ETDRS definition of ‘clinically significant’ MO, and there is no evidence available to suggest that untreated eyes with ‘clinically significant’ MO currently lose vision any slower than they did when ETDRS was undertaken. The main visual acuity outcome reported by the ETDRS was MVL, defined as the loss of 15 letters (three lines) or more on the ETDRS visual acuity chart. 64 Approximately 29% of eyes with ‘clinically significant’ MO assigned to deferral of laser photocoagulation had reached this end point by 36 months, and a subsequent report showed that ∼ 40% of patients in this group had MVL at 5 years. 61 In addition, it was assumed that no untreated eyes with MO would improve by more than 10 letters (two lines) without treatment, but that a constant proportion of untreated eyes (∼ 20%) would be improved by between five and nine letters (one to two lines) over follow-up. 64 These probabilities were decomposed to generate 6-month transition probabilities using the method reported by Craig and Sendi,74 such that non-referred eyes with MO progressed as per deferred eyes with ‘clinically significant’ MO in the ETDRS (Figure 24). A simplifying assumption was that patients with deteriorating vision would not present to the eye clinic without being referred through the screening programme. This assumption works in favour of more sensitive strategies. A further assumption was that, for patients losing 15 letters or more, the average degree of visual loss would be 20 letters – this yielded a proportion of modelled eyes transiting to severe vision loss (< 24 letters) that was consistent with observations in the ETDRS. For patients with both eyes affected by MO, perfect positive correlation in visual acuity outcome was assumed between eyes.

FIGURE 24.

Modelled visual acuity outcomes for eyes with ‘clinically significant’ MO with and without referral/laser treatment.

Mortality

The model allowed the age- and gender-specific 6-monthly mortality risk to be referenced for each simulated patient during each cycle of the model. These all-cause mortality risks for the UK general population were adjusted upwards using published age- and diabetes-specific hazard ratios for all-cause mortality. 65,66 Thus, mortality was modelled based on age, gender and type of diabetes. Although this approach may overestimate mortality somewhat for the general diabetic population, it has also been noted that the mortality risk for patients with MO is increased relative to the mortality risk for diabetic patients without MO. 78

Costs and resource use

Costs associated with screening, OCT and initial referral were incorporated as shown in Table 24. In addition, costs of treatment and ongoing outpatient monitoring (for referred patients with and without MO) were incorporated in the model. All future costs were discounted at a rate of 3.5% per year, in line with the current recommendations from Her Majesty's Treasury. In relation to the capture of ongoing screening, monitoring and treatment costs, the following assumptions were applied.

Analysis

The model was analysed using Monte Carlo simulation, whereby a random number generator was used to simulate the progression of individual patients through the model one a time. Initially, the model was analysed over a 5-year period. Thereafter, further longer-term analyses were undertaken, assuming the same ongoing risks of vision loss as estimated for treated and untreated eyes in the fifth year of the model. The mean costs, years free of MVL (in either eye) and QALYs accruing to patients under the alternative referral strategies were compared to estimate incremental cost-effectiveness ratios. In order to characterise uncertainty surrounding these ratios, deterministic and probabilistic sensitivity analyses were undertaken. 80 The base-case analysis was carried out using English-specific cost data.

Probabilistic sensitivity analysis was undertaken by assigning distributions to model parameters where the data sources and approach to estimation provided sufficient information for this to be done. Beta distributions for the adjusted sensitivity and specificity estimates (of alternative grading strategies) were derived from the probabilistic simulations carried out for the analysis of the cost per case detected. Second order distributions for mean screening, OCT, referral and treatment costs were assigned by selecting variance parameters that ensured feasible low and high estimates (for each cost parameter) fell within the 2.5th and 97.5th percentile of the resultant distribution. Parameters excluded from the probabilistic sensitivity analysis included the underlying probabilities of visual loss and visual gain, and the probabilities of developing MO. The methods used to derive these probabilities precluded the estimation of uncertainty due to sampling variation, and so the impact of variation in these parameters was addressed through deterministic sensitivity analysis.

The results of the probabilistic sensitivity analysis are presented in the form of cost-effectiveness acceptability curves, a standard method for characterising the uncertainty surrounding estimates of cost-effectiveness. Deterministic sensitivity analysis was also used to assess the sensitivity of findings to various structural assumptions, the implications of using alternative sources of cost data and the incorporation of higher health and social care costs associated with legal blindness (see Table 33).

Strengths and limitations

In order to assess longer-term cost-effectiveness, we created a flexible model whereby the individual eyes of each patient were modelled separately. This enabled us to assess the impact of different assumptions about how loss of acuity in one eye or both eyes affects the health-related quality of life of patients (i.e. either assuming the patient's better-seeing eye determines health-related quality of life or assuming the worse-seeing eye determines health-related quality of life).

We tried to use the best available evidence on the effects of laser treatment compared with no treatment for those patients with MO, in order to assess the benefits of improved detection and referral. In doing so, we updated older effect profiles for laser treatment to accommodate the observation of improved outcomes being associated with its use in more contemporary studies. 68

A lack of contemporary data on progression of MO in untreated patients required the assumption that patients without treatment would progress at the rate observed in the ETDRS. 64

A weakness of the modelling related to the sample selection approach and the subsequent need to rely on surrogate marker category frequency weights (obtained from a separate cohort study) to estimate the likely sensitivity/specificity of the alternative options within routine screening. However, the overall findings and conclusions were generally insensitive to the weighting process.

A further potential weakness was the assumption that all patients with MO would be treated with laser, rather than intravitreal anti-vascular endothelial growth factor injections. This decision was taken based on current National Institute for Health and Care Excellence guidance81 as well as a survey of participating centres which suggested that laser still formed the mainstay of treatment throughout the study period.

Our model may be updated in the future to incorporate antivascular endothelial growth factor treatment. It is possible that this could improve the cost-effectiveness of more sensitive referral strategies by improving outcomes for additional patients detected and referred.

However, by the same token it might also improve outcomes for patients with MO missed by less sensitive strategies but picked up at a subsequent screening visit (or presenting with clinical symptoms).

Comparison with other studies

We are not aware of any other studies which have specifically assessed the cost-effectiveness of alternative approaches to screening for diabetic MO. Many studies have assessed the cost-effectiveness of screening for diabetic retinopathy82–85 in which detection of MO was one of the modelled benefits.

In comparison with these previous screening models, our model provides more flexibility for assessing the impact on quality of life of MVL in one or both eyes; some previous studies applied more crude annual risks of progression from MO directly to irreversible legal blindness (sometimes applying reported risks per eye with MO at the level of the patient).

Conclusions

The findings of the economic modelling would suggest that given the relatively low prevalence of MO in patients with surrogate photographic markers, coupled with the repetitive nature of screening, the adoption of more sensitive grading strategies is unlikely to be cost-effective unless this can be achieved for no (or only a very small) decrease in specificity.

A further important finding from the economic analysis is that OCT within screening pathways offers potential.

Recruitment

A total of 3540 patients were recruited from the seven participating centres, of whom 370 were excluded as either the retinal photographs or OCT images were not of adequate quality to meet the inclusion criteria in the study. The median age of the remaining 3170 subjects was 60 years. Of these subjects, 60.7% were male and 84.5% were Caucasian. There was a preponderance of people with type 2 diabetes (77.4%) compared with type 1 diabetes.

Of the 3170 patients, 243 (7.7%) were confirmed as having MO after review of the OCT images. The prevalence of MO differed greatly by centre, ranging from 3.7% to 12.2%. The prevalence of MO also differed between scanners, ranging from 4.5% to 11.8%.

Macular oedema and photographic features of retinopathy

The detection of MO was strongly related to the presence of lesions and was consistently higher in subjects with exudates or BHs (without exudates) within one DD than in those with just M/DHs within one DD.

The detection of MO was also strongly related to increasing counts of these three types of lesions. This was true of all three types of lesion within one DD (adjusted for each other) including M/DHs.

There was no evidence of a relationship between MO and the presence or count of exudates between one and two DD.

Having more than two M/DHs in an eye appeared to be of particular importance in predicting the presence of MO.

Although it would normally be impractical in manual screening to count individual lesions, it would be possible to identify an image with more than two M/DHs.

Macular oedema and patient characteristics

Having poor visual acuity was associated with increased detection of MO. Other patient characteristics associated with MO in univariate analyses were older age, Caucasian ethnicity and having type 2 diabetes rather than type 1 diabetes.

These apparent relationships between higher detection of MO with older age, poor visual acuity and type 2 diabetes were unlikely to have been entirely independent of each other.

Development of grading strategies

In order to evaluate the best combination of photographic features that would help predict the presence of MO on OCT, a number of grading strategies were explored including those currently used by screening programmes.

In addition to features of retinopathy, the inclusion of those patient characteristics that were found to be statistically significant in a univariate analysis were modelled.

We compared the performance of three broad approaches, namely manual grading strategies, computer-assisted manual annotation grading strategies and fully automated annotation grading strategies.

Because the proportion of patients recruited who had M/DHs only in the macula was lower than in the typical diabetic population attending a screening programme, it was necessary to weight subjects to bring proportions in line with those found in an earlier prospective cohort study.

The clinical effectiveness of manual grading strategies

The effectiveness of manual grading strategies is demonstrated by the trade-off between sensitivity and specificity.

The English grading system, which uses the largest number of features, is the most sensitive but the least specific, whereas the Scottish grading system is the most specific and the least sensitive.

A number of new manual grading strategies were devised with the aim of improving on existing approaches.

One of the manual grading strategies, labelled strategy 16, demonstrated an improved specificity by approximately 4% relative to the current grading practice in England, for a similar sensitivity. It differs from the current grading practice in England in that BHs within one DD are referred regardless of visual acuity status, and exudates between one DD and two DD are not included in the selection criteria. This grading strategy would appear to make grading easier and could improve appropriateness of referrals.

The current manual grading strategies that do not include visual acuity or the presence of M/DHs could be improved by taking these into account.

Current manual grading strategies that include exudates between one and two DD could be improved by removing this aspect.

An ideal grading strategy would able to take into account the presence and count of all three types of lesion within one DD and also visual acuity.

The clinical effectiveness of computer-assisted manual annotation grading strategies

The optimum grading strategy, in terms of area under its receiver operating characteristic curve, was the computer-assisted manual annotation grading strategy, CAM. This uses the results of manual annotation of the individual lesions in each image.

This is a time-consuming procedure and so is unlikely to be considered for routine screening practice. Therefore, this manual annotation grading strategy was not taken forward for economic analysis.

The clinical effectiveness of fully automated annotation grading strategies

Fully automated annotation grading strategies were developed using previously described algorithms for detection of lesions of diabetic retinopathy.

The outputs of the lesion detectors were combined using an automated classifier which was trained to detect MO. Optionally, visual acuity and other non-image data were also input to the classifier.

A grading strategy using automated analysis of the macula-centred image performs slightly better than current manual grading strategies.

If automated annotation grading data are combined with non-image data, then further improvements are obtained; a large proportion of this improvement is due to visual acuity alone. Because visual acuity is a standard measurement performed during diabetic retinopathy screening, this additional information could be added to a fully automated annotation grading strategy with little effort.

Results of the cost-effectiveness analysis for manual grading strategies including quality-adjusted life-years and long-term costs (aims b and c, Chapter 2)

The cost-effectiveness modelling clearly demonstrates how costs to the health service increase as more sensitive manual grading strategies are adopted over more specific grading strategies.

Despite the only very small differences in health outcomes between manual grading strategies, increases in sensitivity (with accompanying reductions in specificity) did result in substantial increases in costs to the health service over time, as a result of more subjects being sent for OCT or referred to eye clinics for ongoing monitoring or treatment.

While the relative difference in the adjusted sensitivity of the manual grading strategies appeared quite substantial, the repetitive nature of screening diminishes the importance of this difference since any patients missed in one round of screening have a chance of being picked up during the next round prior to any visual loss occurring.

Even assuming very favourable parameters for the more sensitive manual grading strategies, the incremental cost per QALY estimates for the more sensitive grading strategies remained unfavourable when compared with thresholds used to judge cost-effectiveness in the UK (i.e. £20,000–30,000 per QALY).

Cost-effectiveness of manual grading strategies compared with a fully automated annotation grading strategy (aims b and b, Chapter 2)

The results would suggest that if significant improvements in specificity can be achieved without reducing sensitivity – for example, by adopting the more complex classifier based on automated image grading and pattern detection – such an approach could offer a more cost-effective alternative than current manual grading strategies based on simple manual algorithms.

Costs and consequences of using optical coherence tomography within retinal screening programmes in addition to improving the photographic surrogate markers

A valuable finding is that given the relatively high costs associated with referrals to hospital outpatient eye clinics, the incorporation of OCT within the screening pathway, targeted towards those with suspected MO, could potentially reduce costs to the NHS without reducing the number of OCT-positive cases of MO detected and referred.

When looking at those grading strategies based on observing the presence of surrogate photographic markers, the addition of OCT prior to referral, as part of the screening pathway, to each of these grading strategies reduced the overall costs of detection and referral without impacting on the number of OCT-positive MO cases detected.

The OCT costs were based on the experience of the lead centre, where a band 6 screener carries out the OCT examinations under the supervision of a consultant ophthalmologist.

However, deterministic sensitive analysis suggests that if used in combination with retinal photography (on a 6-monthly basis) to monitor patients within screening, the use of OCT could remain cost-saving up to an incremental cost of ∼ £58 per patient. In the Scottish context, this threshold cost drops to ∼ £47.