Virtual chromoendoscopy for the real-time assessment of colorectal polyps in vivo: a systematic review and economic evaluation

Narrow-band imaging

Twenty-four studies20,54–78 included in the systematic review provided data on the use of NBI for VCE of colorectal polyps. From here on in the report, Kaltenbach and colleagues57,72 and Gupta and colleagues68,73 will be identified by a single study reference to the main source of data (Kaltenbach and colleagues57 and Gupta and colleagues68). Two of these studies, a prospective cohort study by Lee and colleagues77 and a RCT by Kang and colleagues,78 also reported on i-scan and FICE, respectively, and so are also included in our report in the i-scan and FICE sections.

An overview of the characteristics of the included NBI studies is presented in Table 5 (more detailed information is available in the data extraction forms presented in Appendix 3). More than half of the studies were conducted in the USA (14 studies20,54,55,57,58,61,63,64,66,68,69,74–76). Five studies were conducted in Europe (one in the UK,70 two in Italy,59,60 one in Italy and the Netherlands62 and one in Spain65). The remaining five studies were conducted in Asia: two in Japan,56,71 two in South Korea77,78 and one in Australia. 67 Seven of the studies focused on diminutive polyps,55,57,59,67,68,76,77 nine focused on small polyps (< 10 mm in size)20,56,60,62,65,70,71,75,78 and eight included polyps of any size. 54,58,61,63,64,66,69,74 The studies that included polyps larger than diminutive polyps provided at least one outcome of interest for the subgroup of diminutive polyps. One study, by Hewett and colleagues,54 was restricted to polyps in the rectosigmoid colon.

Half of the studies enrolled participants undergoing colonoscopy either for screening, surveillance or because of symptoms,20,57,59–63,65,67,69,70,74 with all but two (Hewett and colleagues20 and Patel and colleagues55) reporting the proportions of participants in each category. Five studies enrolled participants undergoing colonoscopy for either screening or surveillance reasons,54,68,75–77 but not because of symptoms, with one more study66 including participants presenting for elective screening or follow-up colonoscopy (reasons for the follow-up colonoscopy not provided). In two studies the entire sample of participants was drawn from a screening population. 71,78 In the remaining three studies the types of participants enrolled is not known because it was not reported in the publications. 56,58,64

The male-to-female ratio of participants in the included studies lay between 1 : 1 and 2 : 1 in 13 studies,54,56,59–63,65,66,69,74–76 and between 2 : 1 and 3 : 1 in three studies. 70,77,78 In the remaining four studies that reported the male-to-female ratio it was approximately 4 : 1,71 10 : 1,68 23 : 157 and, the highest reported male-to-female ratio, 35 : 1. 67 The male-to-female ratio of participants was not reported by four studies. 20,55,58,64

The mean age of participants, if it was reported, lay between 54 and 67 years (16 studies54,56,57,59–62,65,66,68,70,71,74,76–78) or the median age lay between 60 and 69 years (four studies63,67,69,75). The age of participants was not reported by the remaining four studies. 20,55,58,64

The majority of the studies were conducted in a single centre,54,56,59,60,63–65,69,70,74–78 four were conducted in two centres61,67,68,71 and one each at three centres,57 four centres55 and five centres. 62 The number of centres was not reported by three studies. 20,58,66

Study colonoscopies were undertaken by more than one endoscopist in most studies: one endoscopist in five studies,54,64,69,75,77 two in one study,20 three in one study,67 four in four studies,59,70,74,78 five in four studies,56,57,62,65 six in three studies,60,68,76 seven in three studies,63,66,71 10 in one study,61 12 in one study58 and, the largest number of endoscopists, 26 in one study. 55 In eight studies, all the endoscopists had prior experience of using NBI,54,59,60,62,67,68,71,77 and in four studies some of the endoscopists had prior experience of using NBI. 56,57,65,70 Only four studies stated that the endoscopists involved had no prior experience of using NBI to characterise colorectal polyps,55,58,61,78 but in a further eight studies it was not clear what experience of using NBI, if any, the endoscopist(s) may have had. 20,63,64,66,69,74–76 The majority of the studies included an element of training for the endoscopist(s) in the characterisation of colorectal polyps using NBI, either training all endoscopists20,55,57–67,69,74,76,78 or the non-experts. 70 In the study by Gupta and colleagues, which is a reanalysis of three earlier studies, training occurred in one of the three studies. 68 In five studies54,56,71,75,77 it was not stated if any training had taken place. In three of these, the endoscopists had prior experience of NBI. 54,71,77 In the Iwatate and colleagues study56 the five endoscopists had mixed levels of NBI experience, and it was unclear what NBI experience the single endoscopist in the Shahid and colleagues study had. 75

A variety of different systems were used to classify polyps as adenomas or hyperplastic polyps (see Table 5). The most commonly used systems were the NBI International Colorectal Endoscopic classification scheme or a version of this, which was cited by eight studies,20,56–59,63,65,66 and the criteria proposed by Rex,64 which were cited by four studies. 54,60,64,78 Two studies67,69 cited the Sano–Emura classification system, two74,75 based characterisations on modifications of the Kudo criteria and two55,68 on work by Rastogi and colleagues,73,86,87,93 with one further study76 also citing a Rastogi and colleagues publication,96 although it is not known in this case whether or not the criteria were the same. One study70 used vascular pattern intensity97 to classify polyps, one61 polyp colour, vessels and mucosal pattern,94 and one77 the author’s own system. In the final two studies either criteria were reported but not attributed to any named system62 or no criteria were reported or cited. 71

The QUADAS assessments of the NBI studies indicates that the studies were at a low risk of spectrum, verification, disease progression, incorporation, test review and clinical review biases (Table 6). Supporting information for the judgements shown in Table 6 is provided in the data extraction form for each study (see Appendix 3). Note that ‘yes’ answers to QUADAS questions 1–9 (see Table 3) imply a low risk of bias, whereas ‘yes’ answers to QUADAS questions 10 and 11 reflect adequacy of reporting and further supporting information is required to assess the risks of bias associated with these questions. For five studies55,56,58,64,66 the risk of spectrum bias (QUADAS question 1) was unclear because the reason(s) for patients having a colonoscopy were not reported. In two studies57,63 not all the polyps received verification by histopathology. In the Kaltenbach and colleagues study57 this was because, when two or more non-neoplastic polyps were identified in the rectosigmoid colon in any one patient, a ‘representative sample’ was resected for histopathological analysis. How often this circumstance arose was not reported. In the Wallace and colleagues study,63 10 polyps (from 321 polyps, therefore representing 3% of the total) were not assessed by histopathology (and whether or not one further polyp had been assessed by histopathology was unclear). Overall, it is our opinion that the risk of differential verification bias in these two studies was probably very low.

In all but four studies59,61,63,69 the risk of diagnostic review bias was rated as low (QUADAS question 7). The risk of bias was rated as unclear in the studies by Henry and colleagues,69 Paggi and colleagues,59 Pohl and colleagues61 and Wallace and colleagues63 because they did not report whether or not the histopathologist(s) were blinded to the NBI prediction for each polyp. The majority of studies did not report on uninterpretable/intermediate test results, probably because there were no uninterpretable/intermediate test results because of the nature of the NBI assessments (studies typically required a decision to be made, although this could be assigned as low confidence in some studies). In the studies by Gupta and colleagues and Iwatate and colleagues, there was evidence of uninterpretable or intermediate test results. 56,68 An optical diagnosis could not be determined for four polyps (0.3%) in the study by Gupta and colleagues,68 and Iwatate and colleagues56 excluded two patients with ‘unevaluable material’. Patel and colleagues55 reported that polyps were excluded from the analysis if a confidence level was not assigned or if histopathology was missing or ‘other’, or if the polyp could not be retrieved, so it seems likely that there were also some uninterpretable or intermediate test results in this study. The outcome for QUADAS item 10 was judged unclear for the Wallace and colleagues study because not all patients who were randomised completed the study, so it is possible that uninterpretable test results were the reason for the missing data. 63

For the final QUADAS item (question 11, attrition bias), the judgement was ‘yes’ for the majority of studies either because no withdrawals were apparent in the study20,54,56,59,60,64–67,69,71,74–77 or because withdrawals or other missing data were explained. 57,61–63,70,78 For two studies the judgement was ‘unclear’. 55,58 In the Ladabaum and colleagues study,58 the subjects of the study were endoscopists, and it was unclear whether or not any of them had dropped out of the study; there was little reporting on those undergoing colonoscopy. Patel and colleagues55 did not report the number of participants selected to take part or the number of patients included in the data analyses, so it was unclear whether or not there had been any withdrawals. For one study, by Gupta and colleagues,68 this question was not applicable because the included data were drawn from records of participants in three earlier trials that met the inclusion criteria for a retrospective analysis and, therefore, no participants were able to withdraw.

In addition to the assessment of the QUADAS items, the generalisability of each study was also briefly summarised during data extraction (the summary of reviewers’ comments can be seen in full in the data extraction forms in Appendix 3). The overall impression from the included NBI studies is that they enrolled participants likely to be representative of the types of participants who would receive colonoscopy in the UK for screening, surveillance or on account of symptoms experienced (in line with the inclusion criteria for this systematic review). However, only one study was conducted in the UK,70 and just four elsewhere in Europe,59,60,62,65 where it might reasonably be assumed that populations might be most similar to those in the UK. Most studies were conducted in a single centre,54,56,59,60,63–65,69,70,74–78 so inherently these results may not be transferable to other centres. In contrast, in most studies more than one endoscopist was involved in conducting colonoscopies and characterising polyps. 20,55–63,65–68,70,71,74,76,78 Across all the studies the experience of endoscopists covered the whole range from those who were less experienced in conducting colonoscopy generally and had little or no experience using NBI to very experienced endoscopists who also had extensive experience of using NBI. Training for endoscopists (which may have been to train those with no prior experience of NBI or to ensure that all endoscopists at a centre were characterising polyps to the same standard) formed a part of the majority of studies, but how relevant this training may have been to current UK practice is unknown. Finally, a variety of classifications systems were used to determine whether polyps were adenomas or hyperplastic. The assessment group understands that, in countries where polyp characterisation is conducted without magnification, such as the UK, the NBI International Colorectal Endoscopic classification is becoming widely accepted. It is unclear how generalisable the results obtained using other polyp classifications are to UK practice.

i-scan

Five studies77,79–82 included in the systematic review provided data on the use of i-scan for VCE of colorectal polyps. An overview of the characteristics of the included i-scan studies is presented in Table 7 (more detailed information is available in the data extraction forms presented in Appendix 3). Four of the studies were conducted in Europe (those by Basford and colleagues in the UK,79 Hoffman and colleagues80 and Rath and colleagues82 in Germany and Pigo and colleagues81 in Italy) and one, by Lee and colleagues,77 was conducted in South Korea. Basford and colleagues79 and Hoffman and colleagues80 enrolled all their participants from a screening population, whereas the other three studies77,81,82 enrolled participants receiving colonoscopy for screening or surveillance purposes, with one81 also including participants with gastrointestinal symptoms. In the three studies77,81,82 that enrolled different types of participants, the proportions of participants receiving colonoscopy for screening, surveillance or symptoms was not reported. The Pigo and colleagues study81 enrolled almost equal proportions of men and women, whereas more men than women were enrolled in the other four studies. Four studies77,80–82 reported the mean age of the participants, which ranged from 55 years to 66 years. The two studies conducted in Germany did not report data on polyp characterisation for the whole colon: Hoffman and colleagues80 reported on polyps only in the last 30 cm of colon, and Rath and colleagues82 characterised polyps in the distal colon (the descending colon, the sigmoid colon or the rectum). Three of the studies (i.e. those by Hoffman and colleagues,80 Lee and colleagues77 and Rath and colleagues82) focused on the characterisation of diminutive polyps, whereas Basford and colleagues79 focused on small polyps (< 10 mm) and Pigo and colleagues81 included polyps of all sizes (and their data on diminutive polyps were limited to the rectosigmoid colon). Consequently, for the three studies that focused on the characterisation of diminutive polyps, data are drawn from the whole patient population, whereas it is not clear what proportion of the patients contributed data on diminutive polyp characterisation in the Basford and colleagues79 and Pigo and colleagues81 studies. All the studies were conducted in single centres, and in all but one study a single endoscopist performed the study colonoscopies and characterised polyps. In the Hoffman and colleagues study,80 three endoscopists were involved. It was clearly reported in three of the five studies (i.e. by Basford and colleagues,79 Hoffman and colleagues80 and Lee and colleagues77) that the endoscopist(s) had prior experience using i-scan but, because of an absence of reported details, it is not clear whether or not study endoscopists underwent any specific training with i-scan prior to the start of the studies. Only two studies77,82 used the same system, which was developed for the Lee and colleagues study,77 to classify polyps as adenomas or hyperplastic polyps (see Table 7); the remainder all used different systems. One study81 cited the NBI International Colorectal Endoscopic classification system, one80 used surface pit pattern, citing studies by Kudo and colleagues among others, and Basford and colleagues79 developed their own system for their research.

The QUADAS assessments were conducted for each study and supporting information for the judgements shown in Table 8 is provided in the data extraction form for each study (see Appendix 3). Note that ‘yes’ answers to QUADAS questions 1–9 imply a low risk of bias whereas ‘yes’ answers to QUADAS questions 10 and 11 reflect adequacy of reporting and further supporting information is required to assess the risks of bias associated with these questions. The QUADAS assessments of the i-scan studies indicate that the studies were at a low risk of spectrum, verification, disease progression, differential verification, incorporation, diagnostic review, test review, clinical review and test classification biases (see Table 8). An exception is that, in the Hoffman and colleagues study,80 it was unclear how representative the patients were of those who would receive the test in practice because few details about the participants were reported, although it is known that they fulfilled the criteria for screening colonoscopy.

None of the studies indicated that any uninterpretable or intermediate test results had been reported. Hoffman and colleagues80 reported results for normal mucosa in addition to adenomatous and hyperplastic polyps, but there is no indication in the paper that this was as a result of any difficulty in interpreting the index test.

No withdrawals (of patients or of polyps from the analysis) were apparent in the Hoffman and colleagues80 and Lee and colleagues77 studies. The exclusion of patients screened for inclusion was explained by Basford and colleagues. 79 Pigo and colleagues81 recruited 78 patients and 150 polyps were included in the analysis, but it was not clear whether or not the 150 polyps were from the full sample of 78 recruited participants. Rath and colleagues82 recruited 224 patients to their study, but the analysis included only 77 of these (all were described as having distal diminutive polyps). It is possible that the remaining patients in these studies had larger polyps located other than in the distal colon, but this is not explicitly stated. Therefore, the Pigo and colleagues81 and the Rath and colleagues82 studies are rated as being at possible risk of attrition bias.

In addition to the assessment of the QUADAS items, the generalisability of each study was also briefly summarised during data extraction (the summary of reviewers’ comments can be seen in full in the data extraction forms in Appendix 3). The overall impression from the included i-scan studies is they enrolled participants likely to be representative of the types of participants who would receive colonoscopy in the UK for screening or surveillance or on account of symptoms experienced. However, only one study was conducted in the UK,79 with three out of the remaining four conducted in Europe (two in Germany80,82 and one in Italy81), whereas the final study was conducted in South Korea. 77 Three of the five studies were conducted by endoscopists with prior experience of i-scan,77,79,80 and all took place in single centres often described as academic or specialist centres. The results of these studies may therefore not be applicable to less experienced endoscopists working in more generalist or community settings. Only one study used the NBI International Colorectal Endoscopic classification system (which is becoming widely accepted for polyp characterisation without magnification) to determine whether polyps were adenomas or hyperplastic. 81 It is unclear how generalisable the results obtained using other polyp classifications are to UK practice.

Flexible spectral imaging colour enhancement

Three studies included in the systematic review (Kang and colleagues,78 Longcroft-Wheaton and colleagues83,84) provided data on the use of FICE for VCE of colorectal polyps (Table 9). Two of the studies were conducted in the UK83,84 and the other was conducted in South Korea. 78 In all three of these studies, all the included participants were undergoing colonoscopy for screening purposes. The Longcroft-Wheaton and colleagues83 study enrolled a slightly higher proportion of women than men, whereas the other two studies enrolled a higher proportion of men than women. All three studies reported the mean age of participants, which ranged from 5478 to 65 years. 84 All three studies focused on the real-time diagnosis of colorectal polyps sized < 10 mm and provided subgroup analyses of diminutive polyps. All the studies were conducted in single centres. In the Kang and colleagues78 study, four endoscopists carried out the colonoscopies, whereas the other two studies each involved one endoscopist. Kang and colleagues78 reported that the study endoscopists had no prior experience with FICE, whereas Longcroft-Wheaton and colleagues83,84 reported that the endoscopist in each of these studies had previous experience of in vivo diagnosis of polyps, although the authors did not specify endoscopists’ experience with FICE. Longcroft-Wheaton and colleagues83 stated that the study endoscopist had had prior training in real-time diagnosis. In the other studies,78,84 the endoscopists’ prior training in both real-time diagnosis and, more specifically, the use of FICE was unclear. Kang and colleagues78 noted, however, that the endoscopists received feedback every 2 weeks during the study about the accuracy of their endoscopic predictions compared with the histopathological diagnosis. The study by Kang and colleagues78 (which also included a NBI arm), used a classification system for polyp characterisation based on colour, vascular density and vascular pattern. 64,90,91,98 The two studies by Longcroft-Wheaton and colleagues83,84 both used a characterisation system based on vascular patterns that was developed by Teixeira and colleagues. 99

Table 10 shows the quality assessments of the three FICE studies. 78,83,84 Reviewers considered all three studies to be at a low risk of bias across most of the QUADAS items assessed. None of the studies, however, reported the number of uninterpretable test results, but reviewers believed this to be zero in two studies. 78,84 Two studies explained participant withdrawals. 78,83 Longcroft-Wheaton and colleagues84 did not state whether or not there were any withdrawals.

In addition to the assessment of the QUADAS items, the generalisability of each study was also briefly summarised during data extraction (the summary of reviewers’ comments can be seen in full in the data extraction forms in Appendix 3). Reviewers noted that two of the studies were conducted in the UK83,84 and so are likely to be representative of a UK population (although it is noted that these studies included a small number of participants – 50 and 89 participants each). It was also noted that it is unclear how representative participants in the South Korea study78 would be of the UK population and how similar the endoscopists’ training in this study would be to endoscopists’ training in the NHS in the UK. As all the studies were conducted in single centres it is unclear how the results would generalise to other centres and settings.

Narrow-band imaging

Sensitivity and specificity of narrow-band imaging for the characterisation of diminutive colorectal polyps in the whole colon

Twenty-three studies20,54–71,74,75,77,78 reported on the characterisation of diminutive polyps within the whole colon, although five of these reported data only from high-confidence characterisations. 20,57,59–61

The results for all characterisations of diminutive polyps in the whole colon (i.e. not separated by confidence level), where 2 × 2 table data were reported or calculable, are shown in Figure 5.

FIGURE 5.

Accuracy of NBI for characterising diminutive colorectal polyps as either adenomas or hyperplastic polyps. a, The values for the 2 × 2 tables of these studies were imputed. For Patel and colleagues,55 values have been imputed by the reviewer, but it was not possible to find a solution that agreed with all the 2 × 2 table outcomes reported in the paper. The imputed values for Patel and colleagues55 (which should be regarded as illustrative) produce the reported sensitivity and specificity, but produce values for PPVs and NPVs that are lower than reported and an accuracy value (proportion of correctly classified polyps among all the polyps) that is higher; and b, Rogart and colleagues74 did not report a value for specificity and it was not possible to complete the 2 × 2 table from the information reported in the published paper.

The ability of NBI to correctly identify diminutive polyps as adenomas (i.e. the sensitivity of the test) ranged from 0.55 to 0.97 (i.e. 55–97%) across the 17 studies that reported this outcome. Sensitivity was above 90% in seven studies55,64,66–68,70,71 (and in two of these it was ≥ 95%55,67) between 80% and 90% in six other studies56,58,62,69,77,78 and was < 80% in four studies. 63,65,74,75

The ability of NBI to correctly identify diminutive polyps as hyperplastic polyps (i.e. the specificity of the test) was typically lower than the sensitivity of the test, ranging from 0.62 to 0.95 (i.e. 62% to 95%) across the 16 studies that reported this outcome. Specificity was above 90% in just two studies,69,75 between 80% and 90% in seven studies62,64–66,70,71,77 and was below 80% in seven studies. 55,56,58,63,67,68,78

It was possible to run a bivariate meta-analysis (using Stata/IC14 and metandi45) for the 16 studies that reported both sensitivity and specificity. This produced a summary value for sensitivity of 0.88 (95% CI 0.83 to 0.92) and for specificity of 0.81 (95% CI 0.75 to 0.85). The parameter estimates for the bivariate model were entered into RevMan to produce the SROC plot shown in Figure 6. The 95% confidence region around the summary point indicates where we have 95% confidence that the summary point lies. The 95% prediction region illustrates the extent of statistical heterogeneity among the studies. If the bivariate model for sensitivity and specificity is correct, we have 95% confidence that the true sensitivity and specificity of a new study in the future will lie within the 95% prediction region. As can be observed from Figure 6, the 95% prediction region is large.

FIGURE 6.

Summary receiver operating characteristic curve plot from the meta-analysis of NBI for all characterisations of polyps in the whole colon.

In order to investigate the heterogeneity between studies, a covariate for endoscopist experience with NBI was added to RevMan and separate SROC curves were drawn as shown in Figure 7. Although caution must be taken when interpreting this figure, because of the small number of studies for each subgroup, it nevertheless appears to support the hypothesis that endoscopists with prior experience of using NBI to characterise diminutive colorectal polyps achieve higher sensitivity and specificity than endoscopists who have had no prior experience of using NBI to characterise diminutive colorectal polyps (other than any training that they undertook at the start of the study).

FIGURE 7.

Summary receiver operating characteristic curve plots for all characterisations of polyps in the whole colon by endoscopists’ level of experience using NBI.

The results for studies that reported results from polyp characterisations using NBI that were designated as high-confidence decisions, and where 2 × 2 table data were reported or calculable, are shown in Figure 8.

FIGURE 8.

Accuracy of NBI high-confidence decisions for characterising diminutive colorectal polyps as either adenomas or hyperplastic polyps in the whole colon. a, It was not possible for us to impute the 2 × 2 table data necessary to plot these results within this figure. Hewett and colleagues’ study20 reported a value for sensitivity of 98% (no CI provided and specificity not reported) and Ladabaum and colleagues58 reported a sensitivity of 88.4% (95% CI 82.2% to 94.7%) and a specificity of 44.1% (26.5% to 61.6%); b, the values for the 2 × 2 tables of these studies were imputed.

The ability of high-confidence characterisations made with NBI to correctly identify diminutive polyps as adenomas (i.e. the sensitivity of the test) was ≥ 0.90 (i.e. ≥ 90%) in 9 of the 13 studies20,55–57,59,60,62,64,77 (in four of these it was ≥ 95%20,55,57,64) and between 80% and 90% in three other studies. 58,61,63 The lowest sensitivity value reported was 59%, by Sola-Vera and colleagues. 65 Some studies reported the sensitivity obtained from all characterisations and the sensitivity from only the high-confidence characterisations. In all studies in which both these values were reported, the sensitivity was higher when obtained from high-confidence decisions (difference ranging from an increase of 1.5% to 5.8%).

The ability of NBI to correctly identify diminutive polyps as hyperplastic polyps (i.e. the specificity of the test) from high-confidence polyp characterisations was just > 90% (i.e. > 0.90) in three studies,64,65,77 but did not exceed 92% in any study. In just three studies, specificity lay between 80% and 90%,61–63 but in the majority of the studies it lay < 80%,55–60 with the lowest specificity just 44.1%, reported by Ladabaum and colleagues. 58 Specificity was higher when obtained from high-confidence decisions in seven of the eight studies that reported both the specificity obtained from all characterisations and the specificity from only the high-confidence characterisations, with the increase ranging from 3.5% to 7.3%. The one exception was the study by Ladabaum and colleagues58 in which the specificity calculated from high-confidence characterisations was lower than that obtained from all characterisations (44.1% vs. 64.4%, respectively).

A bivariate meta-analysis (using Stata/IC14 and metandi45) was run for the 11 studies that reported both sensitivity and specificity from polyp characterisations made with high confidence. This produced a summary value for sensitivity of 0.91 (95% CI 0.85 to 0.95) and for specificity of 0.82 (95% CI 0.76 to 0.87). The parameter estimates for the bivariate model were entered into RevMan to produce the SROC plot shown in Figure 9. The effect of reporting only on high-confidence characterisations rather than all polyp characterisations is to move the summary estimate up (increasing sensitivity) and slightly to the left (increasing specificity).

FIGURE 9.

Summary receiver operating characteristic curve plot showing the summary point on the summary curve from the meta-analysis of NBI for high-confidence characterisations of polyps in the whole colon. Note that two studies were not included in the meta-analysis: Hewett and colleagues’ study,20 with a sensitivity of 98%; and Ladabaum and colleagues’ study,58 with a sensitivity of 88.4% (95% CI 82.2% to 94.7%) and a specificity of 44.1% (26.5% to 61.6%).

The impact of restricting the analysis to high-confidence characterisations rather than including all characterisations can be observed in Figure 10, which shows both summary curves on the same plot. As already stated, the effect of reporting only on high-confidence characterisations rather than on all polyp characterisations is that the summary estimate moves up (increasing sensitivity) and slightly to the left (increasing specificity).

FIGURE 10.

Summary receiver operating characteristic curve for all NBI characterisations of polyps in the whole colon and SROC for only high-confidence NBI characterisations of polyps in the whole colon shown on the same plot. Note that for clarity the 95% prediction regions are not shown on this plot.

Seven studies55,56,58,62–65,77 reported both sensitivity and specificity from all diminutive polyp characterisations and separately for only high-confidence diminutive polyp characterisations, although for one of the these studies58 2 × 2 table data were not available for the high-confidence characterisations [which had a reported sensitivity of 88.4% (95% CI 82.2% to 94.7%) and specificity of 44.1% (95% CI 26.5% to 61.6%)]. The pairs of results from these studies are shown in Figure 11 and forest plots in Figure 12.

FIGURE 11.

Plot showing paired data from the studies that reported on all diminutive polyp characterisations and separately on high-confidence diminutive polyp characterisations.

FIGURE 12.

Accuracy of NBI in studies that reported on all diminutive polyp characterisations and separately on high-confidence diminutive polyp characterisations. a, The values for the 2 × 2 tables of these studies were imputed; b, it was not possible for us to impute the 2 × 2 table data necessary to plot these results within this figure [reported sensitivity of 88.4% (95% CI 82.2% to 94.7%) and specificity of 44.1% (26.5% to 61.6%)].

To obtain data for a scenario analysis within the economic model (see Chapter 5, Scenario analyses), a post hoc bivariate meta-analysis (using Stata/IC14 and metandi45) was run for a subgroup in which endoscopists experienced in the use of NBI characterised the polyps in the whole colon (Figure 13). Four such studies were included in this analysis. 59,60,62,77

FIGURE 13.

Accuracy of NBI high-confidence decisions for characterising diminutive colorectal polyps in the whole colon as either adenomas or hyperplastic polyps when made by endoscopists experienced in the use of NBI. a, The values for the 2 × 2 table of this study were imputed.

The meta-analysis produced a summary value for sensitivity of 0.92 (95% CI 0.89 to 0.94) and for specificity a value of 0.82 (95% CI 0.72 to 0.89). The parameter estimates for the bivariate model were entered into RevMan to produce the SROC plot shown in Figure 14. Restricting the meta-analysis from 11 studies reporting different levels of NBI experience (experienced, n = 4;59,60,62,77 mixed experience, n = 3;56,57,65 inexperienced, n = 2;55,61 and unclear, n = 263,64) to the four studies that reported endoscopists experienced in the use of NBI narrowed the 95% CI for sensitivity [11 studies with a variety of experience, 0.91 (95% CI 0.85 to 0.95); four studies with prior NBI experience, 0.91 (95% CI 0.89 to 0.94)] and widened the 95% CI for specificity [11 studies with a variety of experience, 0.82 (95% CI 0.76 to 0.87); four studies with prior NBI experience, 0.82 (95% CI 0.72 to 0.89)]. The changes in the 95% CIs are reflected in the change in the size and shape of the 95% confidence region and 95% prediction region in Figure 14 in comparison with Figure 9.

FIGURE 14.

Summary receiver operating characteristic plot showing the summary point on the summary curve from the meta-analysis of NBI for high-confidence characterisations of polyps in the whole colon when made by endoscopists experienced in the use of NBI.

Colonoscopies in one study, by Iwatate and colleagues,56 were conducted by five endoscopists. Two of the five endoscopists were described as specialists in colonoscopy and they had extensive experience in magnifying colonoscopy with NBI (> 1000 cases). The other three endoscopists were described as general endoscopists with limited experience in magnifying colonoscopy with NBI (≤ 1000 cases). As shown in Table 12, the two specialist endoscopists achieved higher sensitivity and specificity than the three general endoscopists, but the difference between the two was statistically significant only for specificity (p = 0.007).

TABLE 12 - Sensitivity and specificity according to experience with NBI of the endoscopists

Calculated by reviewer.

The differences between the specificity rates for the specialist endoscopist and the general endoscopist groups were significant at a p-value of 0.007.

Accuracy	High-confidence characterisations of polyps 1–5 mm in size
	Specialist endoscopists	General endoscopists
Sensitivity (95% CI)	93.5% (78.58% to 99.21%)a	92.9% (85.10% to 97.33%)a
Specificity (95% CI)	87.0%b (66.41% to 97.22%)a	51.7%b (32.53% to 70.55%)a

Sensitivity and specificity of narrow-band imaging for the characterisation of diminutive colorectal polyps in the rectosigmoid colon

As shown in Table 11, four studies54,55,58,63 reported sensitivity and specificity following characterisation (any level of confidence) of diminutive polyps in the rectosigmoid colon, with three of these reporting sufficient data for a 2 × 2 table to be constructed for entry into the meta-analysis. 54,58,63

Three of the four studies54,55,63 that reported results for all characterisations also reported sensitivity and specificity following high-confidence characterisations of polyps in the rectosigmoid colon, with two further studies61,62 reporting only high-confidence characterisation data. Four of the five studies reporting on high-confidence characterisations provided sufficient data for 2 × 2 tables to be constructed for entry into the meta-analysis. 54,61–63

The results from the studies that used NBI to characterise polyps in the rectosigmoid colon, where 2 × 2 table data were reported or calculable, are shown in Figure 15. The results from Patel and colleagues55 are not represented in Figure 15 because it was not possible to impute values into a 2 × 2 table that provided a solution for the reported outcomes in the paper (accuracy, sensitivity, specificity, PPV and NPV).

FIGURE 15.

Accuracy of NBI for characterising diminutive colorectal polyps as either adenomas or hyperplastic polyps in the rectosigmoid colon. (a) NBI: characterisation of diminutive polyps in the rectosigmoid colon. (b) NBI: high-confidence characterisation of diminutive polyps in the rectosigmoid colon. a, The values for the 2 × 2 tables of these studies were imputed; b, it was not possible for us to impute the 2 × 2 table data necessary to plot these results within this figure. For characterisation of all diminutive polyps in the rectosigmoid colon, Patel and colleagues55 reported a sensitivity of 88.4% (95% CI 84.8% to 92.0%) and a specificity of 78.3% (95% CI 71.8% to 84.9%). The high-confidence polyp characterisations yielded a sensitivity of 90.9% (95% CI 87.4% to 94.4%) and a specificity of 88.6% (95% CI 81.0% to 96.1%).

Bivariate meta-analyses were conducted (using Stata/IC14 and xtmelogit or using Stata/IC14 and metandi45) of the studies where 2 × 2 table data were available. For all characterisations of diminutive polyps in the rectosigmoid colon, the summary value for sensitivity is 0.85 (95% CI 0.75 to 0.91) and for specificity is 0.87 (95% CI 0.74 to 0.94). For high-confidence characterisations of diminutive polyps in the rectosigmoid colon, the summary value for sensitivity is 0.87 (95% CI 0.80 to 0.92) and for specificity is 0.95 (95% CI 0.87 to 0.98). The parameter estimates for the bivariate model from these two meta-analyses were entered into RevMan to produce the SROC plot shown in Figure 16. As seen with the results for the whole colon, the effect of reporting only high-confidence polyp characterisations rather than all polyp characterisations is to increase sensitivity and specificity (summary point moves up and to the left on the SROC plot).

FIGURE 16.

Summary receiver operating characteristic curve plot showing the summary points on the summary curves from the meta-analyses of NBI for all characterisations of polyps and for only high-confidence characterisations of polyps in the rectosigmoid colon.

Note that one study was not included in either meta-analysis, that is, Patel and colleagues,55 with all characterisations of polyps with a sensitivity of 88.4% (95% CI 84.8% to 92.0%) and a specificity of 78.3% (95% CI 71.8% to 84.9%) and high-confidence characterisations of polyps with a sensitivity of 90.9% (95% CI 87.4% to 94.4%) and a specificity of 88.6% (95% CI 81.0% to 96.1%). The large 95% confidence and a 95% prediction regions, which were generated for the high-confidence characterisation plot, are not shown on this figure and the software used to draw the SROC plot (RevMan) did not generate a 95% confidence region or a 95% prediction region for the other data set.

To obtain data for a scenario analysis within the economic model (see Chapter 5, Scenario analyses), a post hoc bivariate meta-analysis (using Stata/IC14 and xtmelogit) was run for a subgroup of studies in which the endoscopists were experienced in the use of NBI. Two such studies54,62 were included in the analysis (Figure 17).

FIGURE 17.

Accuracy of NBI high-confidence decisions, made by endoscopists with prior experience of NBI, for characterising diminutive colorectal polyps in the rectosigmoid colon as either adenomas or hyperplastic polyps. a, The values for the 2 × 2 tables of these studies were imputed.

The meta-analysis produced a summary value for sensitivity of 0.90 (95% CI 0.71 to 0.97) and for specificity of 0.98 (95% CI 0.91 to 1.00). The parameter estimates for the bivariate model were entered into RevMan to produce the SROC plot shown in Figure 18. Restricting the meta-analysis from the four studies reporting different levels of NBI experience (experienced, n = 2; inexperienced, n = 1; and unclear, n = 1) to only two studies in which endoscopists had experience in the use of NBI increased the summary value for sensitivity while widening the 95% CI [four studies with a variety of experience, 0.87 (95% CI 0.80 to 0.92); and two studies with prior NBI experience, 0.90 (95% CI 0.71 to 0.97)] and increased the summary value for specificity while narrowing the 95% CI [four studies with a variety of experience, 0.95 (95% CI 0.87 to 0.98); and two studies with prior NBI experience, 0.98 (95% CI 0.91 to 1.00)].

FIGURE 18.

Summary receiver operating characteristic curve plot showing the summary point on the summary curve from the meta-analyses of NBI for high-confidence characterisations of polyps in the rectosigmoid colon made by endoscopists with prior experience of NBI. Note that the software used to draw the SROC plot (RevMan) did not generate a 95% confidence region or a 95% prediction region for this meta-analysis. It is presumed that this is because of the small number of studies.

Negative predictive value of narrow-band imaging for the characterisation of diminutive colorectal polyps

The NPV is the probability that subjects with a negative screening test (i.e. colorectal polyp is characterised as hyperplastic) truly do not have an adenoma. However, it must be borne in mind when viewing these results that the NPV is influenced by the prevalence of disease (i.e. in this case the prevalence of adenomas in the tested populations). When prevalence is increased, the result is a decrease in the NPV. Owing to the importance of NPV within the PIVI statement (see Chapter 1, Diagnostic thresholds and requirements for use of virtual chromoendoscopy), consideration was given to meta-analysing NPVs from the included studies even though this is not advised by either the National Institute for Health and Care Excellence’s Diagnostics Assessment Programme Manual37 or the Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy Version 1.0. 36 However, because it is clear that the prevalence of adenomas and hyperplastic polyps is likely to vary between studies [e.g. because of differences in case mix (screening, surveillance and symptomatic populations) and patient characteristics (age, sex)], we chose not to pool NPVs across studies. Instead, we have provided forest plots for these outcomes and marked the 90% threshold value on each plot.

For the characterisations of diminutive polyps in the whole colon (made with any level of confidence), the NPV ranged from 43% to 96.1% (Figure 19 and Table 14). The study by Sola-Vera and colleagues65 is noteworthy because this study reported the lowest NPV – far lower than in any other study. All the other studies reported NPVs of > 70%, with five studies reporting NPVs of ≥ 90%. 55,64,66,67,69 However, it should be noted that the lower limit of the 95% CI fell below 90% in every study except Patel and colleagues. 55

FIGURE 19.

Negative predictive values of NBI for all characterisations of diminutive polyps in the whole colon (made with any level of confidence).

TABLE 14 - Negative predictive values of NBI for the characterisation of diminutive polyps in the whole colon

NR, not reported.

Calculated by reviewer.

Study	Characterisation, value (95% CI)
	All	High confidence
Aihara et al.66	96.1% (85.4% to 99.3%)	NR (NR)
Chandran et al.67	93% (80.9% to 98.5%)	NR (NR)
Gupta et al.68	87.8%a (83.6 to 91.3)a	NR (NR)
Henry et al.69	90.7% (79% to 97%)	NR (NR)
Hewett et al.20	NR (NR)	95% (NR)
Ignjatovic et al.70	82.3%a (70.5% to 90.8%)a	NR (NR)
Ikematsu et al.71	78.9% (54.4% to 94.0%)a	NR (NR)
Iwatate et al.56	71.0% (58.1 to 81.8)a	81.4% (66.6% to 91.6%)a
Kaltenbach et al.57	NR (NR)	92.0% (85.3% to 96.3%)
Kang et al.78	73.2% (66.6% to 80.5%)	NR (NR)
Ladabaum et al.58	75.9% (69.1% to 82.7%)	78.3% (69.6% to 87.0%)
Lee et al.77	88.0% (80.6% to 95.4%)	92.1%a (82.4% to 97.4%)a
Paggi et al.59	NR (NR)	83.1%a (71.7% to 91.2%)a
Paggi et al.60	NR (NR)	86.4%a (78.9% to 92.1%)a
Patel et al.55	94.2% (90.4% to 98.0%)	98.3 (95.7% to 100.0%)
Pohl et al.61	NR (NR)	82.3 (78.6% to 85.6%)
Repici et al.62	84% (78% to 88%)	89% (84% to 93%)
Rex et al.64	91.0%a (86.0% to 94.7%)a	95.5%a (90.9% to 98.2%)a
Rogart et al.74	NR (NR)	NR (NR)
Shahid et al.75	75% (62% to 84%)	NR (NR)
Sola-Vera et al.65	43% (34% to 52%)	48% (37% to 59%)
Vu et al.76	NR (NR)	NR (NR)
Wallace et al.63	80% (72.8% to 86.0%)a	82% (74.4% to 88.1%)a
Assessed by specialists in colonoscopy (whole colon)
Iwatate et al.56	NR (NR)	90.9% (70.8% to 98.9%)a
Assessed by general endoscopists (whole colon)
Iwatate et al.56	NR (NR)	71.4% (47.8% to 88.7%)a

Limiting the assessment of NPV to high-confidence polyp characterisations increased the NPVs, which ranged from 48% to 98.3% in the studies that reported this outcome (Figure 20 and Table 14). Again, the study by Sola-Vera and colleagues65 had the lowest NPV of any study by a considerable margin. All other studies reported NPVs for high-confidence assessments of > 78%, with five studies reporting NPVs of ≥ 90%. 20,55,57,64,77 Once again, however, inspection of the 95% CIs reveals that the lower limit of this fell below 90% in all but two studies. 55,64

FIGURE 20.

Negative predictive value of NBI for high-confidence characterisations of diminutive polyps in the whole colon. a, Note that no 95% CI was reported for the Hewett and colleagues study. 20

One study, by Iwatate and colleagues,56 compared differences in NPVs achieved by specialists in colonoscopy and general endoscopists. Specialists in colonoscopy achieved NPVs of > 90% (mean value 90.9%, 95% CI 70.8% to 98.9%), whereas the NPVs achieved by general endoscopists were lower, with a mean value of 71.4% (95% CI 47.8% to 88.8%); however, the difference between the groups was not statistically significant.

Seven studies54,55,58,61–63,68 reported on the NPVs for the characterisation of diminutive polyps in the rectosigmoid colon (top section, Table 15). Five of these studies54,55,58,63,68 reported data for all diminutive polyp characterisations in the rectosigmoid colon and NPVs ranged from 87.4% to 98.4%. In four54,55,63,68 of these five studies the NPVs were > 90%. Only in the study by Ladabaum and colleagues58 was the 90% threshold not reached.

TABLE 15 - Negative predictive values of NBI for the characterisation of diminutive polyps in the rectosigmoid colon and other regions of the colon

NR, not reported.

Calculated by reviewer.

There is a discrepancy in this paper between reporting in the text (which states that the NPV was for rectosigmoid colon diminutive adenomas) and in a table, which means it is possible that the reported NPVs could relate to polyps in the distal and proximal colon rather than the rectosigmoid colon.

Study	Characterisation, value (95% CI)
	All	High confidence
Rectosigmoid colon diminutive polyps
Gupta et al.68	95.4% (91.8% to 97.7%)	NR
Hewett et al.54	98.4% (95.3% to 99.7%)	99.4% (96.9% to 100.0%)
Ladabaum et al.58	87.4% (82.5% to 92.4%)	NR
Patel et al.55	93.7% (91.8% to 95.7%)	94.7% (92.6% to 96.8%)
Pohl et al.61	NR	95.1% (92.2% to 97.1%)a
Repici et al.62	NR	92% (88% to 96%)
Wallace et al.63	95% (88.8% to 98.8%)a	96% (89.3% to 99.2%)a
Diminutive polyps located on the right side of the colon
Kaltenbach et al.57	NR	87.1% (70.2% to 96.4%)
Diminutive polyps located proximal to the splenic flexure
Pohl et al.61	NR	43.4% (33.5% to 53.8%)a
Diminutive polyps located on the left side of the colon
Gupta et al.68	93.0%a (89.2% to 95.8%)a	NR
Kaltenbach et al.57	NR	93.9% (83.1% to 98.7%)
Diminutive polyps located in the distal colon
Pohl et al.61	NR	92.6% (89.4% to 95.0%)a
Rectal diminutive polyps
Kaltenbach et al.57	NR	93.8% (79.2% to 99.2%)
Diminutive polyps proximal to rectosigmoid colon
Ladabaum et al.58	57.3% (38.4% to 76.2%)	NR
Patel et al.55	65.6% (59.2% to 71.9%)	77.1% (67.9% to 86.2%)
Rectosigmoid colon diminutive polyps assessed by endoscopists with prior optical diagnosis experience in colonoscopy
bPohl et al.61	NR	96.6% (92.7% to 98.7%)
Rectosigmoid colon diminutive polyps assessed by endoscopists with no prior optical diagnosis experience in colonoscopy
bPohl et al.61	NR	93.5% (88.7% to 96.7%)

Data for high-confidence characterisations of polyps in the rectosigmoid colon were reported by five of the seven studies (Figure 21). 54,55,61–63 In three of these five studies,54,55,63 the data on high-confidence characterisations were provided in addition to data on all polyp characterisations in the rectosigmoid colon. In these studies the high-confidence results led to NPVs that remained at > 90% and were slightly increased. Two studies61,62 provided high-confidence results only for the rectosigmoid colon and in both the NPV was over the 90% threshold. It is worth noting, however, that in two62,63 of the five studies that report NPVs for high-confidence characterisations of diminutive polyps in the rectosigmoid colon, the lower limit of the 95% CI falls below 90%.

FIGURE 21.

Negative predictive values of NBI for high-confidence characterisations of diminutive polyps in the rectosigmoid colon.

The NPVs of NBI for characterisation of diminutive polyps in other regions of the colon (where reported by studies) is also presented in Table 15. Although the mean NPV was above the 90% threshold in some instances, none of the lower limits of the 95% CI was > 90%.

One study61 reported the NPV for characterisations of diminutive polyps in the rectosigmoid colon achieved by endoscopists with prior optical diagnosis experience in colonoscopy and by endoscopists without prior optical diagnosis experience. Endoscopists with prior optical diagnosis experience achieved a NPV of 96.6% (95% CI 92.7% to 98.7%), whereas the NPV achieved by endoscopists without prior optical diagnosis experience was lower at 93.5% (95% CI 88.7% to 96.7%).

i-scan

Sensitivity and specificity of i-scan for the characterisation of diminutive colorectal polyps

Five studies77,79–82 provided data on the characterisation of diminutive polyps as adenomas or hyperplastic polyps using i-scan, with the characterisation verified by histopathological assessment of the resected polyps. The way in which data were reported by the studies varied. Two studies, by Basford and colleagues79 and Lee and colleagues,77 reported on the characterisation of diminutive polyps within the whole colon. Basford and colleagues79 presented data only from the polyp characterisations that the endoscopist had high confidence were correct, whereas Lee and colleagues77 provided data for all characterisations and then separately for characterisations made with either high or low confidence (data for low-confidence characterisations are available in Appendix 3). The other three studies presented data on the characterisation of diminutive polyps from within a part of the colon: the distal colon (Rath and colleagues82), the last 30 cm of colon (Hoffman and colleagues,80 who did not present a per-polyp analysis, only an analysis per patient) and the rectosigmoid colon (Pigo and colleagues81 and Rath and colleagues,82 although it was not possible to impute the 2 × 2 table data for the latter study). Rath and colleagues82 also provided data separately for the polyp characterisations they had made with high confidence.

The results for all characterisations (i.e. not separated by confidence level) are shown in Figure 22. The ability of i-scan to correctly identify diminutive polyps as adenomas (i.e. the sensitivity of the test) was > 90% in three of the four studies that reported results for all characterisations (i.e. Lee and colleagues,77 Pigo and colleagues81 and Rath and colleagues82), whereas sensitivity was only 82% in the per-patient analysis reported by Hoffman and colleagues. 80 The ability of i-scan to correctly identify diminutive polyps as hyperplastic polyps (i.e. the specificity of the test) was more variable across the studies, ranging from 83% (Rath and colleagues,82 results for polyps in the distal colon) to 96% (Hoffman and colleagues80).

FIGURE 22.

Accuracy of i-scan for characterising diminutive colorectal polyps as either adenomas or hyperplastic polyps. (a) i-scan: polyps in the whole colon; (b) i-scan: polyps in the distal colon; (c) i-scan: polyps in the last 30 cm of colon (analysis by patient); and (d) i-scan: polyps in the rectosigmoid colon. a, 2 × 2 table data imputed; b, Rath and colleagues82 presented summary data for polyps in the rectosigmoid colon, but it was not possible for us to impute the 2 × 2 table data necessary to plot these results within this figure. The reported sensitivity was 90.3% (95% CI 73.1% to 97.5%) and specificity 87.5% (95% CI 74.1% to 94.8%).

The results for studies that reported results from polyp characterisations with i-scan that were designated as high-confidence decisions are shown in Figure 23. The ability of high-confidence characterisations made with i-scan to correctly identify diminutive polyps as adenomas (i.e. the sensitivity of the test) in the three studies that provided data was 0.94 (i.e. 94%; Lee and colleagues77), 0.97 (97%; Basford and colleagues79) and, in the Rath and colleagues’ study,82 0.98 for distal polyps and 0.96 in the analysis limited to polyps in the rectosigmoid colon. In the Lee and colleagues study,77 the sensitivity achieved from high-confidence polyp characterisations was slightly lower than that obtained from all the polyp characterisations, 0.94 (95% CI 0.84 to 0.99) versus 0.95 (95% CI 0.87 to 0.99), whereas the reverse was true for the Rath and colleagues study82 for both the data set for distal polyps and that for rectosigmoid colon polyps (distal polyps: high confidence 0.98, 95% CI 0.90 to 1.00, vs. overall 0.93, 95% CI 0.83 to 0.98; rectosigmoid colon: high confidence 0.96, 95% CI 0.80 to 1.0, vs. overall 0.90, 95% CI 0.73 to 0.98). The ability of i-scan to correctly identify diminutive polyps as hyperplastic polyps (i.e. the specificity of the test) when the characterisation was made with high confidence was ≥ 0.90 (i.e. 90%) in all three studies. Furthermore, the specificity of i-scan arising from high-confidence decisions was greater than the specificity observed when all the polyp characterisations were taken into account in the two studies that reported both sets of data (Lee and colleagues,77 92% vs. 86%; Rath and colleagues,82 distal polyps 95% vs. 83%, rectosigmoid colon polyps 95.5% vs. 87.5%). The 2005 Rath and colleagues82 study, which was conducted in Germany among patients attending for screening or surveillance colonoscopy and which reported on characterisation of distal polyps (polyps in the descending colon, the sigmoid colon or the rectum), achieved the best sensitivity (98%), which was coupled with the second highest value for specificity (95%). However, in common with the other studies providing data on i-scan, a single endoscopist working in what appears to be a specialist endoscopy centre achieved these results, so it is not clear how transferable these results would be to less experienced endoscopists working in less specialist settings.

FIGURE 23.

Accuracy of i-scan high-confidence characterisations of diminutive colorectal polyps as either adenomas or hyperplastic polyps. (a) i-scan: high-confidence characterisations of polyps in the whole colon; (b) i-scan: high-confidence characterisations of distal polyps; and (c) i-scan: high-confidence characterisations of polyps in the rectosigmoid colon. a, 2 × 2 table data imputed; b, Rath and colleagues82 presented summary data for high-confidence characterisations of polyps in the rectosigmoid colon, but it was not possible for us to impute the 2 × 2 table data necessary to plot these results within this figure. The reported sensitivity was 96.4% (95% CI 79.8% to 99.8%) and specificity 95.5% (95% CI 83.3% to 99.2%).

A bivariate meta-analysis was run (using Stata/IC14 and xtmelogit) to provide a summary estimate for the two studies that reported high-confidence characterisations of polyps in the whole colon, which could be used in the economic model. This produced a summary value for sensitivity of 0.96 (95% CI 0.92 to 0.98) and for specificity of 0.91 (95% CI 0.84 to 0.95). The parameter estimates for the bivariate model were entered into RevMan to produce the SROC plot shown in Figure 24.

FIGURE 24.

Summary receiver operating characteristic curve plot from the meta-analysis of i-scan for high-confidence characterisations of polyps in the whole colon. Note that the software used to draw the SROC plot (RevMan) did not generate a 95% confidence region or a 95% prediction region for this meta-analysis. It is presumed that this is because of the small number of studies.

Flexible spectral imaging colour enhancement

Sensitivity and specificity of flexible spectral imaging colour enhancement for the characterisation of diminutive colorectal polyps

Three studies 78,83,84 provided data on the characterisation of diminutive polyps as adenomas or hyperplastic polyps using FICE compared with characterisation verified by histopathological assessment of the resected polyps. In all three studies the characterisations were made on polyps in any part of the colon, and in all three the level of confidence with which the characterisation was made was not stated. The results of the polyp characterisations are shown in Figure 25.

FIGURE 25.

Accuracy of FICE for characterising diminutive colorectal polyps as either adenomas or hyperplastic polyps. a, 2 × 2 table data imputed.

The ability of FICE to correctly identify diminutive polyps as adenomas (i.e. the sensitivity of the test) ranged from 74% to 88% across the studies. The ability of FICE to correctly identify diminutive polyps as hyperplastic polyps (i.e. the specificity of the test) had a narrower range across the studies, from 82% to 88%.

It was possible to run a bivariate meta-analysis (using Stata/IC14 and xtmelogit) with data from the three studies. This produced a summary value for sensitivity of 0.81 (95% CI 0.73 to 0.88) and for specificity of 0.85 (95% CI 0.79 to 0.90). The parameter estimates for the bivariate model were entered into RevMan to produce the SROC plot shown in Figure 26.

FIGURE 26.

Summary receiver operating characteristic curve plot from the meta-analysis of FICE for all characterisations of polyps in the whole colon. Note that the software used to draw the SROC plot (RevMan) did not generate a 95% confidence region or a 95% prediction region for this meta-analysis. It is presumed that this is because of the small number of studies.

Post hoc pooled analysis of all virtual chromoendoscopy technologies

The appropriateness of pooling evidence from different VCE technologies together is uncertain. The technologies certainly all aim to enhance surface vessel patterns, but the technologies use different methods to achieve this. We have therefore assumed that there is a ‘class effect’ and that evidence from different VCE technologies can be meaningfully pooled.

A pooled analysis of the studies included in this assessment for which 2 × 2 data were available was undertaken in order to inform a scenario analysis using the economic model (see Chapter 5, Sensitivity analyses). Data for high-confidence assessments of polyps in the whole colon were available from 11 NBI studies and two i-scan studies (note that Lee and colleagues77 contribute data on NBI and i-scan) (Figure 27). No FICE data were available to include in this analysis because the FICE studies did not report high-confidence polyp characterisations separately.

FIGURE 27.

Accuracy of VCE high-confidence decisions for characterising diminutive colorectal polyps as either adenomas or hyperplastic polyps in the whole colon.

A bivariate meta-analysis (using Stata/IC14 and metandi45) was carried out, which produced a pooled summary estimate for sensitivity of 0.92 (95% CI 0.87 to 0.95) and for specificity of 0.83 (95% CI 0.78 to 0.87). The parameter estimates for the bivariate model were entered into RevMan to produce the SROC plot shown in Figure 28. The VCE pooled estimates for sensitivity and specificity do not differ greatly from the NBI pooled estimates (see Figure 9), which is unsurprising given that the bulk of the evidence comes from studies of NBI.

FIGURE 28.

Summary receiver operating characteristic curve plot showing the summary point on the summary curve from the meta-analysis of VCE high-confidence decisions for characterising diminutive colorectal polyps in the whole colon.

A pooled analysis of the virtual chromoendoscopy studies for high-confidence assessments of polyps in the rectosigmoid colon, equivalent to that above for the whole colon, has, in essence, already been presented earlier in this assessment. This is because the only data available for this analysis come from NBI studies and, thus, the results presented in Figures 15 and 16 represent all the available data on high-confidence assessments of polyps in the rectosigmoid colon; there are no equivalent data for i-scan or FICE.

Narrow-band imaging

Thirteen studies55,57,58,60–65,67,68,70,76 reported results on the impact that the use of NBI would have on recommended surveillance intervals (in comparison to surveillance intervals calculated following histopathology of all polyps) (Table 21). The agreement between the surveillance interval allocated using a NBI-based strategy and using the results of histopathology for all polyps ranged from 84%63,76 to 99%. 62 Eleven of the 13 studies reporting on this outcome achieved a level of agreement that was > 90%,55,57,58,61–65,67,68,70 although for three of these studies58,63,68 an agreement of > 90% was achieved by only one of the tested strategies (in two studies using a modified recommendation of colonoscopy in 10 years for patients with one or two small adenomas instead of 5 years,58,68 and in one study limiting the analysis to studies with high-confidence predictions for polyps ≤ 5 mm in size63). Where there were discrepancies between the surveillance interval assigned using the NBI-based strategy and the histopathology-only strategy, some studies reported whether the NBI strategy led to longer or shorter surveillance intervals being assigned. In the majority of studies in which a discrepancy in the surveillance interval was reported, the NBI-containing strategy led more often to shorter surveillance intervals being set (i.e. patients were recalled for a colonoscopy sooner than would have been the case with the histopathology-based surveillance interval) than to longer surveillance intervals. There were, however, some exceptions; in particular, in the study by Repici and colleagues,62 in the NBI-containing strategy, a difference between the surveillance intervals assigned was more likely to lead to the assignment of a longer interval (i.e. patients not recalled for repeat colonoscopy as early as they would have been with the histopathology-based surveillance interval) than to a shorter one.

Nine studies clearly calculated the concordance of surveillance intervals between VCE and histopathology in line with the PIVI requirements. 57–59,61–64,67,76 The criterion of the PIVI statement, that agreement should be ≥ 90%, was met by all but one study,76 with one further study meeting the PIVI criterion in one of the two tested strategies. 58 When the agreement was ≥ 90%, the lower limit of the 95% CI (when reported) fell below 90% in two instances. 55,62

i-scan

Two studies79,82 examined the effect that the use of i-scan had on recommended surveillance intervals in comparison with those that were allocated based on histopathological assessment of all polyps (Table 22). Both studies79,82 used in vivo diagnosis of diminutive polyps to guide surveillance interval decisions in accordance with the PIVI requirements. Both studies79,82 also calculated agreement in surveillance intervals between i-scan and histopathology when using two different guidelines for determining the surveillance interval. Across these two studies, a surveillance interval agreement of > 90% was achieved regardless of the guideline used, with agreement ranging from 93.2%82 to 97.2%. 79 In the study by Basford and colleagues,79 identical results (an agreement of 97.2%) were achieved when using both the guidelines assessed. Both studies reported whether using i-scan resulted in a longer or shorter surveillance interval being allocated than that allocated by histopathology. In the Basford and colleagues study,79 two patients were set a shorter interval with i-scan and one patient a longer interval. In the Rath and colleagues study,82 i-scan tended to results in longer intervals being allocated than with histopathology, except in one case.

Flexible Spectral Imaging Colour Enhancement

Two studies83,84 reported results on the impact that the use of FICE would have on recommended surveillance intervals (in comparison with surveillance intervals calculated following histopathology of all polyps), although neither assessed this in accordance with the PIVI criteria. This analysis, in both of these studies, included polyps < 10 mm in size (i.e. neither was restricted to diminutive polyps). The agreement between the surveillance interval allocated using a FICE-based strategy and using the results of histopathology was 100% in one study83 and 97% in the other study,84 regardless of whether the BSG or ASGE guidelines were used to determine the surveillance intervals. In the single study for which there was a discrepancy for two participants between the surveillance interval assigned using the FICE-based strategy and the histopathology strategy, it is not known whether the FICE-based strategy led to a longer or a shorter surveillance interval being set (Table 23).

Narrow-band imaging

None of the studies reported on the interpretability of the test; test acceptability (to patients and/or clinicians), number of outpatients appointments, HRQoL, incidence of colorectal cancer or mortality.

One study, by Lee and colleagues,77 reported on interobserver agreement, although this was the agreement between the characterisation obtained during real-time assessment and that obtained by an independent reader who reviewed all recorded endoscopic images while blind to the real-time assessment and the histopathology results. The interobserver agreement was 86.5%, with a κ value of 0.730 (95% CI 0.623 to 0.837), which represents ‘substantial’ agreement. One other study, by Rogart and colleagues,74 reported interobserver agreement for 20 test images, but, as this did not include any real-time assessment these data were not extracted. Lee and colleagues77 were also the only researchers to report on intraobserver agreement. This was the agreement between the characterisation obtained during real-time assessment and that obtained by the same endoscopist who reviewed all recorded endoscopic images 1–3 months after the colonoscopy. The intraobserver agreement was 89.7%, with a κ value of 0.795 (95% CI 0.699 to 0.890), again representing ‘substantial’ agreement.

Adverse events were not reported by most studies. 20,54–56,58–71,74,76,78 Of the three studies that did make mention of potential adverse events,57,75,77 all indicated that no events had occurred. Kaltenbach and colleagues57 reported no post-polypectomy bleeding, coagulation syndrome, perforation or optical misdiagnosis of advanced histopathology, Lee and colleagues77 stated that participants did not experience any procedure-related complications and Shahid and colleagues75 stated that none of the patients experienced any endoscopic complications.

Ignjatovic and colleagues70 reported on the number of diminutive polyps that would have been left in place if the management strategy was to leave diminutive hyperplastic polyps in situ in the rectosigmoid colon. The endoscopists in this study made a high-confidence optical diagnosis for 323 polyps (< 10 mm in this study) and, of these, 33 would have been left in situ. All 33 were correctly predicted to be hyperplastic polyps and all were located in the sigmoid colon or the rectum. Repici and colleagues62 made a statement indicating that, in their study, a discard-type strategy would have reduced the need for polypectomy by 48%.

Two studies reported on the number of polyps that would have been resected and discarded if a resect and discard type of management strategy had been in place. Gupta and colleagues68 reported a hypothetical strategy in which, if all the 884 diminutive polyps in their study (in which the total number of polyps of any size was 1254) were resected and discarded, this would represent a 70.5% reduction in histopathology. Using this strategy, 13 adenomas with advanced histopathological features would have been discarded. However, it must be noted that this study did not record whether characterisations were made with high or low confidence and did not report how many diminutive polyps were in the rectosigmoid colon. Ignjatovic and colleagues70 reported that a high-confidence optical diagnosis was made for 323 polyps (< 10 mm in size in this study) and, of these, 290 would have been resected and discarded. The Ignjatovic and colleagues study70 was the only NBI study to ask endoscopists to identify polyps that they would have sent electively to histopathology, even if a policy of optical diagnosis had been in place. These were polyps where the optical diagnosis was made with low confidence or where no optical diagnosis could be made. For the subgroup of diminutive polyps in this study, 7.5% (22 of 293 polyps) would have been sent for elective histopathology.

The length of time taken to perform the withdrawal phase of the colonoscopy was reported by three studies. Kaltenbach and colleagues57 reported a mean withdrawal time of 10.3 minutes [standard deviation (SD) 5.7 minutes, range 3.3–58.0 minutes]. A procedure time of 12 seconds is reported, but a definition of procedure time is not provided in the study publication, so it is not clear what this comprises. In the Kang and colleagues78 study, the mean withdrawal time in the NBI group was 13.5 minutes (SD 7.3 minutes), whereas in the Wallace and colleagues study63 it was 16.1 minutes (SD 7.3 minutes). A fourth study, Shahid and colleagues,75 reported that the average withdrawal time at their centre was typically 8–10 minutes, but withdrawal time was not reported specifically for their study. However, they did report that NBI inspection time was typically < 1 minute.

i-scan

None of the studies reported on the interpretability of the test, test acceptability (to patients and/or clinicians), number of polyps designated to be left in place, number of polyps designated to be resected and discarded, number of polyps designated for resection and histopathological examination, number of outpatients appointments, HRQoL, incidence of colorectal cancer or mortality.

One study, by Lee and colleagues,77 reported on interobserver agreement, although this was the agreement between the characterisation obtained during real-time assessment and that obtained by an independent reader who reviewed all recorded endoscopic images while blind to the real-time assessment and the histopathology results. The interobserver agreement was 87.9%, with a κ value of 0.751 (95% CI 0.640 to 0.861), which represents ‘substantial’ agreement. One other study, by Pigo and colleagues,81 reported interobserver agreement but this was based on endoscopists’ assessments of still images so, because this did not include any real-time assessment, these data were not extracted. Two studies, by Lee and colleagues77 and Rath and colleagues,82 reported on intraobserver agreement. In the Lee and colleagues study77 this was the agreement between the characterisation obtained during real-time assessment and that obtained by the same endoscopist who reviewed all recorded endoscopic images 1–3 months after the colonoscopy. The intraobserver agreement was 86.4%, with a κ value of 0.729 (95% CI 0.616 to 0.841), again representing ‘substantial’ agreement. In the Rath and colleagues’ study82 it is not clear how intraobserver agreement was assessed because no details are reported in the paper. The authors state that agreement was achieved in 113 out of 121 polyps (93.4%), with a κ coefficient of agreement of 0.867 (95% CI 0.799 to 0.967), which indicated almost perfect agreement. In the Pigo and colleagues study81 intraobserver agreement was assessed based on the endoscopists’ assessment of still images rather than real-time assessment. Furthermore, the intraobserver agreement for the evaluation of diminutive polyps was not reported, so these data were not extracted.

As already stated in Narrow-band imaging, Lee and colleagues77 reported that participants did not experience any procedure-related complications. The other four i-scan studies79–82 made no reports of adverse events.

The length of time taken to perform the withdrawal phase of the colonoscopy was not reported in any of the studies. Basford and colleagues,79 however, commented that the in vivo assessment was performed in the time between finding a polyp and preparing for polypectomy and so did not cause a significant delay. Hoffman and colleagues,80 who examined only the last 30 cm of colon, reported that with surface enhancement with i-scan the total examination time was 5 minutes.

Flexible Spectral Imaging Colour Enhancement

None of the studies reported on the interpretability of test, interobserver agreement, intraobserver agreement, test acceptability (to patients and/or clinicians), adverse events, number of polyps designated to be left in place, number of polyps designated to be resected and discarded, number of polyps designated for resection and histopathological examination, length of time to perform the colonoscopy, number of outpatient appointments, HRQoL, incidence of colorectal cancer or mortality.

Head-to-head comparisons

Head-to-head comparisons of NBI, i-scan and FICE were not within the scope of this assessment; nevertheless, two studies met the inclusion criteria for the systematic review that did compare two technologies against each other. When NBI was compared with i-scan in a prospective cohort study of the real-time histopathological prediction of diminutive colonic polyps, Lee and colleagues77 found no statistically significant differences between the two technologies (sensitivity: NBI, 88.8% vs. i-scan 94.6%; specificity: NBI 86.8% vs. i-scan 86.4%; and accuracy: NBI 87.8% vs. i-scan 90.7%; p > 0.05). In the RCT that compared NBI with FICE, Kang and colleagues78 found that for polyps < 5 mm in size there was no statistically significant difference (p > 0.05) in accuracy (NBI 74.9% vs. FICE 80.1%) or sensitivity (NBI 81.9% vs. FICE 74.5%), but there was a statistically significant difference in specificity (NBI 75.7% vs. FICE 88.4%; p = 0.006). The authors concluded that better results should be achieved for both technologies before either are used for real-time optical biopsy of colorectal polyps in colorectal screening of the general population. 78 It is worth noting that in the study by Lee and colleagues77 a single endoscopist with experience of both NBI and i-scan undertook the study colonoscopies, whereas the four endoscopists in the Kang and colleagues study78 had no prior experience of either NBI or FICE.

Narrow-band imaging

• A total of 23 studies reported either sensitivity (one study74) or both sensitivity and specificity (22 studies20,54–71,75,77,78).

• In the whole colon, characterisations of diminutive polyps made with any level of confidence had a sensitivity ranging from 0.55 to 0.97 (17 studies55,56,58,62–71,74,75,77,78) and a specificity ranging from 0.62 to 0.95 (16 studies55,56^. 58,62–71,75,77,78). A bivariate meta-analysis (16 studies55,56^. 58,62–71,75,77,78) produced a summary sensitivity value of 0.88 (95% CI 0.83 to 0.92) and specificity of 0.81 (95% CI 0.75 to 0.85). For characterisations in the whole colon made with high confidence, summary sensitivity and specificity (11 studies55–57,59–65,77) were slightly higher [sensitivity 0.91 (95% CI 0.85 to 0.95) and specificity 0.82 (95% CI 0.76 to 0.87)] and limiting this analysis to studies in which the endoscopists were experienced in the use of NBI (four studies59,60,62,77) did not greatly alter these results [sensitivity 0.92 (95% CI 0.89 to 0.94) and specificity 0.82 (95% CI 0.72 to 0.89)].

• In the rectosigmoid colon, characterisations of diminutive polyps made with any level of confidence (four studies54,55,58,63) had a sensitivity ranging from 0.84 to 0.90 and a specificity ranging from 0.76 to 0.95. A bivariate meta-analysis (three studies54,58,63) produced a summary estimate for sensitivity of 0.85 (95% CI 0.75 to 0.91) and for specificity of 0.87 (95% CI 0.74 to 0.94). For characterisations in the rectosigmoid colon made with high confidence (five studies54,55,61–63), sensitivity ranged from 0.83 to 0.96 and specificity from 0.88% to 0.99%. A bivariate meta-analysis (four studies54,61–63) produced a summary estimate for sensitivity of 0.87 (95% CI 0.80 to 0.92) and for specificity of 0.95 (95% CI 0.87 to 0.98). Limiting the analysis of high-confidence characterisations in the rectosigmoid colon to the two studies54,62 in which the endoscopists were experienced in the use of NBI increased the summary values for sensitivity and specificity [sensitivity 0.90 (95% CI 0.71 to 0.97) and specificity 0.98 (95% CI 0.91 to 1.00)].

• Some studies that reported sensitivity and specificity were not included in meta-analysis because it was not possible to impute the required 2 × 2 table data. In two of three instances where this occurred, the sensitivity and specificity reported by the absent study lay within the 95% CI of the summary estimates of the meta-analysis. In one case (the meta-analysis of high-confidence polyp characterisations in the whole colon) the missing study, that by Ladabaum and colleagues,58 reported a sensitivity that lay within the 95% CI of the summary estimate but a specificity that lay outside the 95% CI of the summary estimate.

• The NPV of NBI for the characterisation of diminutive polyps in the whole colon (made with any level of confidence) ranges from 43% to 96% (16 studies55,56,58,62–71,75,77,78). Five studies55,64,66,67,69 reported NPVs of ≥ 90%, but the lower limit of the 95% CI fell below 90% in every study except one. 55 When limited to high-confidence characterisations, NPVs ranged from 48% to 98% (13 studies20,55–65,77), with five studies20,55,57,64,77 reporting NPVs of ≥ 90%. However, the lower limit of the 95% CI remained above 90% in only two studies. 55,64

• The NPVs of NBI for the characterisation of diminutive polyps in the rectosigmoid colon (made with any level of confidence) ranged from 87% to 98% and was > 90% in four out of the five studies that reported this outcome54,55,63,68 (but the lower limit of the 95% CI remained > 90% in only three studies54,55,68). When limited to high-confidence characterisations in the rectosigmoid colon (five studies54,55,61–63), the NPVs ranged from 92% to 99%, but the lower limit of the 95% CI fell below 90% in two studies. 62,63

• Accuracy (the proportion of correctly classified polyps) of polyp characterisations in the whole colon was ≥ 90% in five studies and between 76% and 89% in 10 studies (16 studies reported this outcome55,56,58,62–71,75,77,78). High-confidence characterisations typically increased accuracy by 3–5% in studies reporting both overall and high-confidence data (eight studies55,56,58,62–65,77).

• Agreement between the surveillance interval allocated using a NBI-based strategy, and using the results of histopathology, was > 90% in 11 of the 13 studies that reported this outcome. 55,57,58,61–65,67,68,70 When there was a discrepancy in surveillance intervals, the NBI-containing strategy more often led to an earlier recall for colonoscopy than would have occurred with the histopathology-based surveillance interval.

• No outcome data were reported (interpretability of the test, test acceptability, number of outpatients appointments, HRQoL, incidence of colorectal cancer or mortality) or sparse outcome data (interobserver agreement, adverse events, polyps designated as ‘left in place’, polyps designated resect and discard, time taken to perform colonoscopy) were reported for other outcomes of interest to this review.

i-scan

• Five studies77,79–82 provided sensitivity and specificity outcomes for the characterisation of diminutive polyps as adenomas or hyperplastic polyps using i-scan. Often only a single study provided data for a particular combination of the region of the colon and the level of confidence assigned to the polyp characterisation.

• In the whole colon, or in regions of the colon, characterisations of diminutive polyps made with any level of confidence ranged in sensitivity from 0.82 to 0.95 and in specificity from 0.83 to 0.96. It was not possible to meta-analyse any of these results. For high-confidence characterisations in the whole colon, or in regions of the colon, sensitivity ranged from 94% to 98% and specificity from 90% to 95%. The only meta-analysis possible, which was conducted to inform the economic model, was for high-confidence characterisations of diminutive polyps in the whole colon. The summary value for sensitivity was 0.96 (95% CI 0.92 to 0.98) and for specificity was 0.91 (95% CI 0.84 to 0.95).

• NPVs were > 90% (all five studies77,79–82); however, the lower limit of the 95% CI was > 90% in only one study. 79

• Accuracy was ≥ 90% (all five studies) and higher for high-confidence polyp characterisations in the two studies that also reported accuracy for all polyp characterisations. 79,82

• Surveillance interval agreement (two studies79,82) determined by i-scan and histopathology was > 90%. When surveillance intervals differed, longer intervals were more likely to be set with i-scan than histopathology.

• No outcome data were reported (interpretability of the test, test acceptability, polyps designated as ‘left in place’, polyps designated resect and discard, number of outpatients appointments, HRQoL, incidence of colorectal cancer or mortality) or sparse outcome data (interobserver agreement, adverse events, time taken to perform colonoscopy) were reported for other outcomes of interest to this review.

Flexible Spectral Imaging Colour Enhancement

• Three studies78,83,84 provided sensitivity and specificity, with all reporting on characterisations of diminutive polyps made with any level of confidence in the whole colon. Reported values for sensitivity range from 74% to 88% and for specificity from 82% to 88%.

• None of the studies provided evidence on the high-confidence characterisation of diminutive polyps or restricted their analysis to a part of the colon (e.g. the rectosigmoid colon).

• It was possible to run a bivariate meta-analysis that produced a summary estimate for sensitivity of 0.81 (95% CI 0.73 to 0.88) and for specificity of 0.85 (95% CI 0.79 to 0.90).

• The NPVs of FICE (three studies78,83,84) ranged from 70% to 84%.

• The accuracy of FICE (three studies78,83,84) ranged from 80% to 85%.

• Surveillance interval agreement between FICE and histopathology was 100% (one study83) or 97% (one study84). When there was disagreement it was not reported whether the FICE-based strategy led to a longer or a shorter surveillance interval being set.

• None of the other outcomes of interest to this review was reported.

Pooled analysis of virtual chromoendoscopy technologies

• A pooled analysis of high-confidence decisions characterising diminutive polyps in the whole colon (NBI, 11 studies; and i-scan, two studies) was undertaken to inform a scenario analysis using the economic model. 54–57,59–62,64,65,77,79 This produced a pooled summary estimate for sensitivity of 0.92 (95% CI 0.87 to 0.95) and for specificity of 0.83 (95% CI 0.78 to 0.87).

Head-to-head comparisons

• Head-to-head comparisons of the technologies were not within the scope for this assessment, but two of the included studies compared two technologies against each other. For the real-time histopathological prediction of diminutive colonic polyps, no statistically significant differences were found when a single endoscopist with experience of NBI and i-scan compared these technologies in a prospective cohort study. 77 A RCT conducted by endoscopists without experience of either NBI or FICE78 found no statistically significant difference in accuracy or sensitivity, but a statistically significant difference in specificity.

Table 24 provides a summary of the pooled sensitivity and specificity values from our bivariate meta-analysis, when available.

Hassan and colleagues

Hassan and colleagues112 developed a cost-effectiveness model to calculate the potential savings and drawbacks of a resect and discard approach using NBI in a simulated colorectal cancer screening cohort. In the resect and discard approach, diminutive colorectal lesions (≤ 5 mm in size) classified by endoscopy with high confidence were not analysed by a pathologist. A Markov model was constructed with health states for no colorectal neoplasia, diminutive (≤ 5 mm), small (6–9 mm) or large (≥ 10 mm) adenomatous polyps; localised, regional or distant colorectal cancer; and colorectal cancer-related death. The resect and discard policy was instituted for all the cases in which a high-confidence diagnosis was achieved by NBI. All diminutive polyps in which a high-confidence diagnosis was not possible were removed and sent for formal histopathological evaluation. The model assumed a screening strategy of colonoscopy every 10 years. After colonoscopy, patients received follow-up surveillance based on the size and classification of the polyp(s).

Feasibility and accuracy of NBI without optical magnification in differentiating between diminutive adenomas and hyperplastic polyps were derived from three published series. 64,70,73 Feasibility was defined as the rate of high confidence in differentiating between polyps. An 84% feasibility was assumed. The sensitivity and specificity for adenomas was 94% and 89%, respectively.

Costs were derived from Medicare reimbursement rates. No incremental cost for NBI was included because it was stated to be a standard feature in current-generation colonoscopes. The cost of colonoscopy was US$630, the cost of colonoscopy with polypectomy was US$925 and of pathological examination was US$102. Costs were also included for colorectal cancer treatment and adverse event costs, such as perforation and bleeding. Costs and life-years were discounted at 3% per annum.

The discounted cost for the no-screening strategy was US$3390 per person over their lifetime (Table 27). The colonoscopy screening strategy reduced costs by US$168 per person and the colonoscopy with resect and discard strategy reduced costs by a further US$25 per person. Colonoscopy with or without resect and discard improved life expectancy by an average of 51 days per person compared with no screening. The study also extrapolated the results to the US population.

Kessler and colleagues

Kessler and colleagues113 developed a decision tree model to quantify the expected costs and outcomes of removing diminutive polyps with or without subsequent pathological assessment. They compared two strategies: ‘submit all’ diminutive polyps (≤ 5 mm in size) to pathological examination and no pathological examination of diminutive polyps (resect and discard). All other polyps were submitted for pathological examination for both strategies.

The decision model was populated with polyp frequencies based on a database of 10,060 consecutive patients who underwent colonoscopy for screening, surveillance or diagnostic indications. The decision model evaluated the frequency with which the surveillance follow-up (based on the most advanced polyp) matched that of the actual follow-up interval for the two strategies. Patients in the endoscopy database were distributed among four groups based on the characteristics that form the basis for follow-up. Group 1 comprised people who had only one diminutive polyp. Group 2 comprised those with two polyps, at least one of which was diminutive and the other not a large adenoma (≥ 10 mm in size). Patients in group 3 people had at least three polyps, at least one of which was diminutive and the others were not large adenomas, while those in group 4 had at least one diminutive polyp, as well as one or more large adenoma(s) and could have any number of additional polyps. For each of the four groups, each patient’s most advanced polyp type was either an advanced adenoma, a non-advanced adenoma or a non-adenoma.

The sensitivity and specificity of endoscopic and pathological assessment were based on the published literature. Costs were included for pathology, colonoscopy and colorectal cancer treatment. The cost of sending a polyp to pathology was US$103.87. Costs of colonoscopy, colonoscopic perforation and cancer treatment were obtained from the literature. The colonoscopy costs were US$1329 for diagnostic and US$2038 for therapeutic colonoscopies. The downstream costs and outcomes after the colonoscopy were obtained from a published discrete event simulation model of colorectal cancer screening and surveillance intervals. 118 Discounting was not included in the model.

The submit-all strategy resulted in an incorrect surveillance interval 1.9% of the time, whereas the resect and discard strategy did so 11.8% of the time, with over half of the patients having only non-adenomatous polyps but scheduled for a 5-year, rather than a 10-year, surveillance examination. The cost saving from forgoing pathological assessment was US$210 per colonoscopy when diminutive polyps were removed, while the additional cost as a result of the incorrect surveillance interval was US$35.92. The net saving was US$174.01. The number needed to harm because of perforation, major bleed or missed cancer was 7979 (i.e. an absolute risk of 0.0125%).

The expected additional benefit of the submit-all strategy was 0.17 days of life over the lifetime horizon and the incremental cost-effectiveness ratio (ICER) of the submit-all strategy compared with the resect and discard strategy was US$377,460 per life-year gained.

Deterministic sensitivity analyses were conducted for the accuracy of the colonoscopy to detect adenomas and the proportion of diminutive polyps with advanced histopathology. The sensitivity analyses performed indicate that the error rate in assigning post-polypectomy surveillance intervals was most sensitive to the accuracy of endoscopic assessment of histopathology and to the proportion of diminutive polyps with advanced histopathology.

The authors concluded that endoscopic diagnosis of polyp histopathology during colonoscopy and forgoing pathological examination would result in substantial upfront cost savings. Downstream consequences of the resulting incorrect surveillance intervals appear to be negligible.

Summary of published economic evaluations

The cost-effectiveness review of published economic evaluation for VCE technologies found two relevant studies that were both published in the USA. 112,113 The patient population differed between the two studies: Hassan and colleagues112 simulated a screening population (i.e. included patients who had no polyps identified by the colonoscopy) and Kessler and colleagues’ population113 had at least one diminutive polyp identified. Both studies compared a resect and discard strategy with a ‘submit-all’ (to histopathology) strategy to the whole colon, although Kessler and colleagues113 assumed that the resect and discard strategy would be used for all polyps, whereas Hassan and colleagues113 assumed that it would not be feasible to resect and discard some polyps (i.e. those characterised with low confidence). Neither studies used surveillance intervals for follow-up screening that correspond to those used in the UK.

The model structure differed between the two studies: Hassan and colleagues113 used a Markov model and Kessler and colleagues113 used a decision tree model. We consider that both approaches are appropriate. The cost saved per person varied between US$25112 and US$174 over the patient’s lifetime. 113 Kessler considered the expected benefit of histopathology to be 0.17 days of life, whereas Hassan assumed that there was no difference in life expectancy between groups over the patient’s lifetime. The cost-effectiveness of the submit-all strategy compared with resect and discard was US$377,460 per life-year gained for Kessler and colleagues,113 whereas Hassan and colleagues112 were not able to calculate a value as there was no difference in the life expectancy between the submit-all and resect and discard strategies. It is unclear how generalisable these results are to the UK NHS, as they used non-UK resource costs and did not include QALYs.

Review of information provided by Olympus to the National Institute for Health and Care Excellence: economic evaluation

A budget impact model was supplied as part of the information provided by Olympus to the National Institute for Health and Care Excellence and the Assessment Group. The model has also recently been published by Solon and colleagues. 117 This study did not meet our inclusion criteria for cost-effectiveness models of VCE because it did not include long-term health outcomes. However, we have provided a critical review of it as a supplement to our systematic review of cost-effectiveness studies, as it has some relevance to the decision problem in this assessment.

The decision tree

The decision tree model compares the VCE strategies (VCE with each of the technologies NBI, i-scan and FICE in the base case) with a histopathology strategy for a cohort of patients (the screening population in the base case). The model adopts a lifetime horizon and a NHS and Personal and Social Services perspective.

Patients enter the model at colonoscopy, having had at least one diminutive polyp, and no non-diminutive polyps, identified. The cohort is divided into four risk categories, based on the number of adenomas that they have:

1. no adenomas

2. low risk: one or two adenomas

3. intermediate risk: three or four adenomas

4. high risk: five or more adenomas.

The model then calculates the proportion of patients in each category expected to have the correct diagnosis and treatment, and the proportions expected to be diagnosed and treated incorrectly. There are essentially three types of error that can occur: patients might have one or more hyperplastic polyp misclassified as an adenoma and unnecessarily resected; they may have one or more adenoma misclassified as a hyperplastic polyp and left in situ; and/or they may be assigned to an incorrect surveillance interval – either too long or too short. The resulting permutations of diagnostic outcomes for patients are illustrated in Figure 31. It can be seen that there are six main patient outcomes, which are also defined in Table 32.

FIGURE 31.

Decision tree showing diagnostic outcomes for patient. CD, correct diagnosis; HPRC, hyperplastic polyp(s) resected correct surveillance; HPRI, hyperplastic polyp(s) resected incorrect surveillance; MAHPR, missed adenoma(s) and hyperplastic polyp(s) resected; MAI, missed adenoma(s) incorrect surveillance; P_he, probability that the patient is in the higher end of the risk classification; P_HPR, probability of hyperplastic polyp being resected; P_le, probability that the patient is in the lower end of the risk classification; P_MA, probability of missed adenoma.

The probability of these different outcomes depends on the number of polyps and adenomas that the patient has, the diagnostic accuracy of the colonoscopy technology and the policies for resecting polyps and assigning surveillance intervals.

In general, if the actual number of adenomas is at the higher end of the risk classification range, then a patient with one or more hyperplastic polyps identified incorrectly as adenomas may be given a shorter surveillance interval than is appropriate. Similarly, if the actual number of adenomas is at the lower end of the risk classification range, then if the patient has one or more adenomas identified incorrectly as hyperplastic polyps and left in situ, they may be given a longer surveillance interval than is appropriate.

Some outcomes are not possible for particular groups of patients; for example, a patient with one hyperplastic polyp and one adenoma (low risk) cannot be assigned an incorrect surveillance interval, as, even if the hyperplastic polyp is mistaken for an adenoma, they would still be placed in the low-risk group and be invited (correctly) for routine screening. Other outcomes will be very improbable for some patients; for example, a patient with nine adenomas (high risk) is very unlikely to be diagnosed with fewer than five adenomas, and so is unlikely to be assigned to a surveillance interval that is too long.

It is possible that patients could simultaneously have one or more hyperplastic polyp misdiagnosed as an adenoma (FP) and one or more adenoma misdiagnosed as a hyperplastic polyp (FN). If so, the patient would be at risk of harm from the unnecessary resection(s) and increased risk of cancer as a result of the adenoma(s) left in situ. However, it is unlikely that they would be assigned to an incorrect surveillance interval, as the errors for individual polyps would be likely to cancel out.

The mathematics behind the estimation of outcome probabilities for patients from polyp-level diagnostic accuracy estimates is explained in Estimating patient outcome probabilities from polyp-level diagnostic accuracy. First, we continue the overview of the decision tree model, and explain how it links to long-term outcomes from the SBCS model.

Under the histopathology strategy, all patients are assumed to receive the correct diagnosis (Table 33). All polyps including adenomas are resected, so no adenomas are left in situ, and patients are assigned to the correct follow-up strategy: routine invitation to screening for those with zero to two adenomas, 3-yearly surveillance for those with three or four adenomas and annual surveillance for those with five or more adenomas. The model calculates the resources required for histopathology and polypectomy and the number of adverse events that result from polypectomies, with associated treatment costs and disutilities. Long-term outcomes associated with each diagnostic outcome are taken from the SBCS model with no adenomas left in situ and all patients assigned to the correct follow-up. The SBCS model includes higher adenoma incidence rates for patients who have had adenomas resected than for patients who started without adenomas (normal epithelium), and the rate of recurrence of adenomas is higher for patients who were initially at higher risk. Cancer incidence, and hence costs and outcomes in the SBCS model, also depend on the surveillance interval assigned. A detailed description of the SBCS model is provided in The School of Health and Related Research’s bowel cancer screening Markov model.

With VCE, errors in characterisation of polyps are possible, and hence patients may be left with one or more adenomas in situ (as a result of FNs) and/or have hyperplastic polyps unnecessarily resected (as a result of FPs). Errors in polyp characterisation with VCE might also cause patients to be allocated to the wrong follow-up strategy – with either too long or too short an interval. The diagnostic outcomes for patients under the VCE strategy are shown in Table 34. Outcomes that are impossible or very unlikely are omitted from this table.

For patients without any adenomas, there are only two possible outcomes: they may have a correct diagnosis and have no polyps resected (correct diagnosis); or they may have one or more hyperplastic polyps removed unnecessarily [hyperplastic polyp(s) resected correct surveillance]. In either case, patients with no adenomas are very unlikely to be assigned the wrong follow-up: the probability of the three or more FP results that would be required for them to be incorrectly assessed as intermediate risk is very low. Costs and outcomes for this group are therefore taken from the results for patients starting in SBCS model in the ‘normal epithelium’ health state and following routine screening. There are five possible diagnostic outcomes for patients with one or two adenomas. They may be correctly diagnosed; have one or more adenomas missed, but no resections of hyperplastic polyps and be assigned correctly to routine screening [missed adenoma(s) correct surveillance]; have no adenomas missed but one or more hyperplastic polyps resected, either with the correct follow-up of routine screening [hyperplastic polyp(s) resected correct surveillance] or unnecessary 3-yearly surveillance [hyperplastic polyp(s) resected incorrect surveillance]; or they may have one or more adenomas missed and also one or more hyperplastic polyps resected with the correct follow-up [missed adenoma(s) and hyperplastic polyp(s) resected]. Patients in this group start in the SBCS model in the ‘post-polypectomy (low-risk adenomas removed)’ health state or in the ‘low-risk adenomas’ health state (one or two diminutive adenomas in situ). All patients in this group will be invited for routine screening, except those with one or more FP results who are misclassified as intermediate risk. Finally, patients with three or more adenomas (intermediate risk or high risk) have all possible outcomes illustrated in Figure 31. We assume that patients in this group with one or more missed adenomas start in the ‘low-risk adenomas’ health state in the SCBS model, with one or two adenomas in situ; however, it is possible that patients could have three or more adenomas missed, but this is very unlikely.

Estimating patient outcome probabilities from polyp-level diagnostic accuracy

The School of Health and Related Research’s bowel cancer screening Markov model

The SBCS model122 describes the development of adenomas and colorectal cancer and subsequent disease progression for the general population of England eligible for bowel cancer screening. It was developed by the School of Health and Related Research for the NHS Bowel Cancer Screening Programme. The model is a ‘Markov-type’ health state transition model, that takes a cohort approach (rather than individual-level simulation). It estimates QALYs and costs for a cohort of 65-year-olds at risk of developing colorectal cancer over a lifetime horizon and using an annual cycle length. Costs were estimated from the perspective of the English NHS, and a discount rate of 3.5% was applied to costs and QALYs. The basic model structure consists of a natural history model and a screening and surveillance pathway.

The basic natural history model is illustrated in Figure 32. This shows the expected progression of adenomas and colorectal cancer in the absence of an active screening and surveillance programme.

FIGURE 32.

The School of Health and Related Research’s bowel cancer screening model: natural history model. Adapted from Whyte and colleagues. 131

Patients start in one of the pre-cancer health states: normal epithelium (no adenomas), low-risk adenomas or intermediate-/high-risk adenomas. Over time, they may progress through the adenoma–carcinoma route: from normal epithelium to low-risk adenomas, to intermediate-/high-risk adenomas and to preclinical Dukes’ stage A colorectal cancer. It is also possible for patients to transition directly from normal epithelium to preclinical stage A colorectal cancer (de novo cancers). Preclinical cancer progresses through the stages, from A to B to C to D, but at some time it is likely to be diagnosed, through chance detection or symptomatic presentation, at which time the patient moves to the related ‘clinical’ cancer stage. Progression through the clinical cancer stages is not modelled; instead a stage-specific cancer survival rate is applied. It is also possible for patients with undiagnosed stage D cancer to die. Patients can die from other causes in any of the health states.

The SBCS model was designed to evaluate alternative active screening and surveillance programmes. The post-screening surveillance pathway is illustrated in Figure 33.

FIGURE 33.

The School of Health and Related Research’s bowel cancer screening model: surveillance colonoscopy pathway. HR, high risk; IR, intermediate risk; LR, low risk.

This shows the assumptions built in to the SBCS model about how patients would be followed up under BSG guidelines, according to findings at an initial colonoscopy after a positive screening result, which reflects the starting point from the end of our decision tree for our base-case screening population. Patients assessed to be at low risk following an initial colonoscopy (zero to two diminutive adenomas in our population) and those with no adenomas at two successive 3-year surveillance colonoscopies are assumed to be invited for routine screening. The screening pathway in the version of the SBCS model used to generate cost and QALY estimates for the VCE model was chosen to reflect the current NHS Bowel Cancer Screening Programme, with the offer of a home FOBT every 2 years for all men and women aged 60–74 years, and invitation for colonoscopy for patients with an abnormal screening test.

In the SCBS model, colonoscopy is assumed to be standard colonoscopy without VCE. However, the model does assume less than perfect sensitivity of colonoscopy for detecting adenomas: 0.77 for low-risk adenomas and 0.98 for intermediate-risk/high-risk adenomas. It also assumes that the cost of histopathology is incurred only for adenomas, a mean of 1.9 per person undergoing colonoscopy. Thus, the cost and accuracy of colonoscopy in the SCBS model is possibly more reflective of VCE than with standard colonoscopy.

The simple natural history diagram in Figure 32 does not show all transitions in the SBCS model. In particular, it omits recurrence of adenomas and cancer incidence for patients who have had adenomas removed at colonoscopy. These additional transitions are illustrated in Figure 34.

FIGURE 34.

The School of Health and Related Research’s bowel cancer screening model: adenoma recurrence following polypectomy. HR, high risk; IR, intermediate risk; LR, low risk.

Following colonoscopy, patients enter the following health states in the SBCS model: patients who started with no adenomas go to the ‘normal epithelium’ state; patients with one or two adenomas left in situ go to ‘low-risk adenomas’; those with three or more adenomas left in situ go to ‘intermediate-risk/high-risk adenomas’; and patients who have all had adenomas resected go to the low-, intermediate- or high-risk adenomas removed states, depending on their initial risk level. Subsequently, patients who have had all adenomas removed may have a recurrence of low-risk or intermediate-/high-risk adenomas, and they also have a small chance of ‘de novo’ cancer, transitioning directly to preclinical Dukes’ stage A colorectal cancer.

Thus, the costs and QALYs for the end points of our decision tree were calculated by running the SBCS model with a cohort of 65-year-old patients starting in each of the post-colonoscopy health states (normal epithelium, low-risk adenomas removed, intermediate-risk adenomas removed, high-risk adenomas removed, low-risk adenomas and intermediate-/high-risk adenomas). The model was run for each possible post-colonoscopy state three times, assuming routine screening, 3-yearly surveillance and annual surveillance in turn. Several updates were made to the SBCS model for these analyses. The input parameters are described in Model parameters. Screening and treatment costs were inflated or updated where appropriate (see Tables 41 and 42). Analyses were run assuming that the average number of adenomas present in patients with at least one adenoma was 1.9, although the SBCS model does not explicitly simulate the number of polyps. The final cost and QALY estimates from the SBCS model that were used in our decision tree analysis are shown in Table 37.

Screening population

We searched for studies that described the distribution of polyps in patients in a bowel screening population. We identified one study, by Raju and colleagues,132 that reported data for the distribution of polyps and adenomas per patient. We analysed the distribution of polyps and adenomas to derive the average number of polyps and adenomas for low-risk, intermediate-risk and high-risk patients, and the frequency of patients in each risk category, assuming that all polyps are diminutive.

The study by Raju and colleagues132 is a retrospective analysis of data from a colon cancer screening programme in the USA. Three hundred and forty-three patients underwent colonoscopy between 2009 and 2011. In the study, 46 patients had no polyps and there were 882 polyps in the remaining 297 patients (2.97 polyps per patient). Of the patients who had polyps, 206 had a total of 422 adenomas (i.e. 1.4 adenomas per patient with a polyp or 2.04 per patient with an adenoma). Thirty per cent of patients who had polyps had no adenomas.

We used a graphical data extraction programme (XY Scan, version 4.1.0; New Haven, CT, USA) to extract the data from Raju and colleagues. This extraction resulted in a slight overestimation of the number of adenomas (426 instead of the reported 422) and the number of patients with adenomas (207 instead of 206) in order to keep polyp numbers correct at 882.

In order to calculate the number of polyps per patient in each risk category, we assumed that patients with adenomas were evenly distributed across the risk categories, where people had adenomas. The risk stratification was defined in accordance with the current BSG guidelines:30 people with one or two adenomas are low risk, those with three or four adenomas are intermediate risk and those with five or more adenomas are high risk. First, we calculated the proportion of patients with the number of adenomas that corresponded with the risk classification and then we calculated a weighted average of the number of polyps and adenomas in these patients. The derivation of the polyp demographics are shown in more detail in Appendix 10. Polyp demographics are shown in Table 38.

Surveillance population

We were unable to identify any studies that reported the distribution of adenomas in a surveillance population, whereby all patients after colonoscopy had been followed up for the appropriate surveillance interval as defined by their risk classification. We found several studies that reported the distribution of adenomas at follow-up surveillance for specific subgroups. For example, Lee and colleagues133 reported the outcome of 12-month surveillance colonoscopy in high-risk patients (n = 1760) in the NHS Bowel Cancer Screening Programme. Martínez and colleagues134 reported a pooled analysis of eight prospective studies comprising 9167 people with previously resected colorectal adenomas during a median follow-up of 4 years. We found several other studies that reported the distribution of adenomas at various follow-up intervals for patients with more than one adenoma resected. 135,136 In the absence of data that fit our population group, we used these studies, together with an assumption, to calculate the distribution of adenomas in this population.

The proportion of patients with no adenomas at follow-up surveillance was similar for Lee and colleagues133 (49.2%) and Martínez and colleagues134 (53.3%). We chose the estimate from the study by Martínez and colleagues,134 as it was the larger study, and not only for high-risk patients. We stratified those patients who had low-risk, intermediate-risk and high-risk adenomas in the same proportion as for the screening population (see Table 38). The resulting distribution of adenomas for the surveillance population is shown in Table 39.

Symptomatic population

We identified one relevant study by McDonald and colleagues137 that described the proportion of people who had adenomas in a group of consecutive patients referred from primary care for colonoscopic examination in the NHS. Patients were referred for symptoms including rectal bleeding, change in bowel habits and abdominal pain. Patients who had been referred as a result of the Bowel Cancer Screening Programme were not included. The distribution of adenomas for the symptomatic population is shown in Table 39.

The study also included a small number of patients with irritable bowel syndrome and we have excluded these from our calculation of the distribution of adenomas in the symptomatic population. The study reports the number of people who have no adenomas, low-risk adenomas and high-risk adenomas. The high-risk adenoma group was split between intermediate-risk and high-risk in the same proportion as for the screening population (see Table 39).

System costs

The equipment and maintenance costs for VCE technologies are shown in Appendix 11. These costs are not included in the base-case analysis for VCE versus histopathology as all equipment and maintenance costs are included within the national reference costs for colonoscopy and polypectomy (see Table 42). There are differences in the costs between the VCE technologies and these are explored in a scenario analysis (see Sensitivity analyses).

Colorectal cancer treatment costs

The SBCS model includes colorectal cancer treatment costs by patient age and Dukes’ colorectal cancer staging score. These costs were taken from the study by Pilgrim and colleagues140 and have been inflated to 2015 prices using The Hospital and Community Health Service index124 (Table 43).

Training costs

As discussed earlier (see Chapter 1, Training in the use of virtual chromoendoscopy), endoscopists will need to receive training to accurately use VCE. This may include training programmes in the form of video packages and/or supervision from endoscopists experienced in using VCE. Several studies have evaluated training packages that were developed to train endoscopists in the use of NBI. 72,94,141,142

For example, Ignjatovic and colleagues141 conducted a prospective education study of a computer-based training module in 21 individuals (novices, trainees and experienced gastroenterologists) with varying colonoscopy experience in the UK. There was significant improvement in the accuracy in characterisation of polyps after the training. Ignjatovic and colleagues141 commented that, although the NBI learning curve is thought to be relatively short, with an improvement in diagnostic accuracy after as few as 44 polyps, it is not clear how expertise is best transferred to community gastroenterologists and to trainees. McGill and colleagues72 showed that the performance of endoscopists could be sustained over time by repeating the training module at the mid-point of the study. Meads and colleagues142 suggest that ongoing training and assessment is necessary to sustain performance.

We assumed that the number of days training would be 2 days per year per endoscopist, in common with the NBI study by Solon and colleagues. 117 Using a daily rate for endoscopists of £1104 from PSSRU124 and assuming that each endoscopist completes 150 endoscopies per year gives a training cost per patient of £14.72.

One-way deterministic sensitivity analyses

Parameters were varied across a range of lower and upper values. The parameters that were varied in one-way sensitivity analyses are reported in Tables 50 and 51. Most of the one-way sensitivity analyses use 95% CIs from data identified during our systematic review and targeted parameter searches. However, some data were taken from different ranges, for example to show the variation between studies for these data. The prevalence of adenomas were varied across the possible range for each risk classification.