Molecular testing for Lynch syndrome in people with colorectal cancer: systematic reviews and economic evaluation

Genetic testing for relatives

Understandably, there may be considerable anxiety and distress associated with genetic testing for hereditary cancer syndromes. For example, the knowledge of being at increased risk of cancer but not knowing if cancer will actually develop can be particularly concerning. Furthermore, parents of children with LS may express feelings of guilt. As people may be anxious about many aspects of genetic testing and screening or whether to have risk-reducing surgery, detailed information provided by an experienced clinical geneticist with psychosocial support is essential. Therefore, it is recommended that relatives opting for genetic testing should receive one or more individual pretest counselling sessions with psychological support throughout the whole testing procedure. 19

Genetic counselling helps explain what a positive or negative result means and what the implications are for the person and his or her extended family. It can also help people understand the importance of informing their extended family about the risk of having LS and the benefits of being tested [see www.macmillan.org, www.lynch-syndrome-uk.org, www.ihavelynchsyndrome.com (accessed 13 July 2017)]. Once people fully understand the implications of being diagnosed with LS for them and their families, the associated anxiety may substantially reduce. Studies evaluating the psychological distress associated with genetic testing for LS showed that, immediately after disclosure of the test result, distress significantly increases, but this decreases again after 6 months. 19 Long-term studies have demonstrated that post-result increases in distress return to baseline by 1–3 years. 19

Surgical management

An individual with LS who does not undergo a partial or total colectomy after the first mass is diagnosed as malignant has an estimated 30–40% risk of developing a metachronous tumour within 10 years and a 50% risk of developing a metachronous tumour within 15 years. It is also now known that the risks from age 40 to 70 years of a metachronous CRC (mCRC) in those who have had surgery for an initial primary CRC are 46% for MLH1 mutation carriers, 48% for MSH2 mutation carriers and 23% for MSH6 mutation carriers. 15 This compares with risks in the general population of 3% within 10 years and 5% within 15 years. 10 Furthermore, because of the risk of a synchronous tumour in individuals with LS, the complete colon should be visualised before resection. In view of this substantial risk, it is possible that a subtotal colectomy instead of a segmental resection might be the preferred treatment in patients from LS families with a primary tumour. Vasen et al. 19 also suggest that a colectomy with ileorectal anastomosis may be considered if CRC is detected in a young patient participating in a surveillance programme.

Prophylactic surgery may also be an option for certain extracolonic cancers, for example hysterectomy and oophorectomy after childbearing is complete nearly eliminate the risk of endometrial and ovarian cancer in women with LS,26 but the potential effectiveness and place of such prophylactic surgery is now open to reconsideration given that mortality from endometrial and ovarian cancer in LS is not as high as previously thought. 3,15

Chemotherapy

Fluorouracil (5-FU)-based regimes represent the current gold standard in adjuvant chemotherapy for bowel cancer. In contrast, in vitro studies indicate that MMR-proficient cells treated with 5-FU grow more slowly than MMR-deficient cells, suggesting that a competent MMR system is a critical condition for 5-FU cytotoxicity. 19 Clinical studies on the efficacy of 5-FU in MSI-high (MSI-H) colon cancer are, however, contradictory. 19 A systematic review and meta-analysis did not find evidence that the chemotherapy response was determined by MSI status,27 but it is understood that individuals with stage II CRC may receive MSI testing or MMR immunohistochemistry (IHC) testing to aid clinical decision-making. Stage II tumours showing MSI (i.e. that are MMR deficient) are usually not treated with (5-FU-based) chemotherapy as it is associated with toxicity in some patients and is believed to be of marginal clinical benefit.

There is also increasing evidence to show that MMR-deficient cancers respond especially well to programmed cell death protein 1 inhibitors. 28

Chemoprevention

There is evidence to suggest that LS may be susceptible to environmental manipulation, as demonstrated by the decrease in the incidence of gastric cancer and also perhaps by the apparent differences in penetrance between men and women. 19 This is supported by investigations into the role of aspirin in bowel cancer prevention. Several large studies have demonstrated that aspirin reduces the risk of bowel cancer in the general population, although the mechanisms by which this occurs are unknown. 4 One study has shown that a daily dose of aspirin reduced the incidence of CRC in carriers of LS after 56 months’ follow-up. 29 Furthermore, the Colorectal Adenoma/Carcinoma Prevention Programme 2 (CAPP2) trial of aspirin prophylaxis in LS has demonstrated that aspirin treatment for up to 3 years reduces, a decade later, the overall incidence of LS-associated cancers, including CRC, by 63%. 4,29

The development of vaccines directed at tumours with MSI are also showing promise that this may be an especially good way of preventing otherwise un-addressable LS-associated cancers with a poor prognosis, as well as potentially any cancer in the general population with MSI. 15

Tumour-based tests

Population

Studies of individuals with CRC were included. This included fresh samples taken from people who were newly diagnosed with CRC or samples that had been retained in storage.

The unit of assessment was individual patients. If results were presented according to individual cancers (e.g. when patients had multiple primary colorectal malignancies), when possible the earliest CRC tested with an index test was used as the unit of assessment.

Studies in which clinical or family history criteria were used to select CRC patients were eligible for inclusion under certain circumstances (see Study design).

Index tests

The index tests to be considered were:

• molecular MSI testing with or without BRAF V600E mutation testing and with or without MLH1 promoter methylation testing

• MMR IHC with or without BRAF V600E mutation testing and with or without MLH1 promoter methylation testing.

Studies in which BRAF V600E and/or MLH1 promoter methylation tests were performed only on certain patients according to their MSI or IHC test results were eligible for inclusion.

Studies were eligible for inclusion if one or more index test was assessed against a reference standard.

Reference standard

The reference standard was constitutional MMR mutation testing (for abnormalities that provide a genetic diagnosis of LS), which, as a minimum, included DNA sequencing and MLPA (or another appropriate technique for detecting large genomic abnormalities). Other appropriate techniques were Southern blot analysis, gene-targeted array-based comparative genomic hybridisation (aGCH) and NGS. However, NGS alone (i.e. without MLPA) was accepted as an includable technique for detecting large genomic abnormalities only if the study described or cited peer-reviewed methodology for identifying structural variants in LS based on output data. If no such methodology was described it was assumed that NGS would not detect structural variants and would not be an includable technique. Studies were eligible for inclusion if MLPA, Southern blot, aGCH or NGS (as described above) was conducted only when sequencing found no clearly pathogenic mutations. Studies in which IHC results directed the MMR genes to be tested were eligible for inclusion (e.g. if MLH1 was not tested when only MSH2 and MSH6 proteins were absent on IHC).

Unless the aim of a study was to investigate the test accuracy of an index test in individuals with mutations in a particular MMR gene, studies must have tested MLH1, MSH2 and MSH6 as a minimum (unless IHC results directed otherwise).

Outcomes

The outcomes assessed for index tests were:

• sensitivity

• specificity

• LR+

• LR–

• PPV

• NPV

• accuracy or concordance with the reference standard: the proportion of test results correctly identified by the test, that is, the rate of agreement with the reference standard:(7)Accuracy=True positive+True negativeTotal number of subjects=TP+TNTP+FP+FN+TN,

• diagnostic yield (also known as test positivity rate or apparent prevalence)

• test failure (non-informative test result) rate.

Study design

Single-gate diagnostic studies with random or consecutively recruited participants were considered the optimal design for evaluating the test accuracy of MSI and IHC testing and were therefore eligible for inclusion. Two-gate diagnostic studies were also included.

Studies were included if all participants received the index test(s) and the reference standard. Studies that recruited a representative sample of all CRC patients but did not apply the reference standard to all patients were included if the reference standard was applied to all patients testing positive for one or more index test(s) and to a representative (e.g. random) sample of patients testing negative for all index tests.

Studies that limited recruitment to high-risk populations (except by applying an age limit to an otherwise population-based sample) were included only to estimate sensitivity and only if the index test(s) and reference standard had been applied to all participants.

Single-gate studies recruiting population-based samples

As mentioned earlier, the population-based sample included in the study by Poynter et al. 34 appeared to be completely unselected. The other three population-based studies included in this review54–56 were based on CRC populations to which an age limit had been applied. For all three of these studies this limit was for age at diagnosis (rather than age at recruitment) and was < 55 years for Barnetson et al. ,54 < 50 years for Limburg et al. 55 and < 45 years for Southey et al. 56 These four studies varied in size; the studies by Barnetson et al. 54 and Poynter et al. 34 were the largest, recruiting 1259 participants and 1061 participants respectively. The studies by Limburg et al. 55 and Southey et al. 56 recruited similar numbers of participants (n = 195 and n = 131 respectively).

With regard to the population characteristics of these four studies, all provided details on the participants’ sex and for all studies the ratio of men to women was similar (Table 7). Although the study by Poynter et al. 34 included the only unselected CRC population identified by this review, the age of the participants was not reported. The age of the participants in the other three studies ranged from a median of 37.1 years (range 24–42 years) for those receiving the reference standard in the study by Southey et al. 56 to a mean of 49.0 years (± 3.9 years) for one of the subgroups in the study by Barnetson et al. 54 These low mean and median ages, which were similar across the three studies, are unsurprising given that all three applied an age limit to participants for inclusion in the study. The four studies recruiting population-based samples all reported a low proportion of participants meeting Amsterdam II criteria, ranging from 0.1% in one of the subgroups in the study by Poynter et al. 34 to 12% in the study by Southey et al. 56 Two studies34,54 also reported the proportion of participants meeting the revised Bethesda guidelines. The specific location of the CRC was reported by Barnetson et al. 54 and Poynter et al. 34 but not by Limburg et al. 55 or Southey et al. 56 None of the studies provided details on the ethnicity of the participants.

Three54–56 of the studies recruiting population-based samples assessed MSI and IHC, with the study by Limburg et al. 55 assessing only IHC (see Table 6).

Single-gate studies recruiting high-risk populations

The five single-gate studies34,57–60 that were based on high-risk populations all applied different criteria to select participants for inclusion. For two34,57 of these studies the participant inclusion criteria were unclear, although for the study by Caldes et al. 57 it was reported that participants were recruited from a familial cancer clinic and for the study by Poynter et al. 34 it was reported that participants were recruited through high-risk clinics and that some clinics selected participants with a ‘family history or early age at onset’. The remaining three single-gate studies appeared to include participants who were at high risk because of their family history (see Table 6). These five studies varied in size, with the largest study being that by Poynter et al. ,34 which recruited 172 participants, and the smallest being that by Mueller et al. ,58 which recruited 48 participants.

With regard to the population characteristics of these five studies (see Table 7), only one60 provided details on the participants’ sex (56.3% female). The age of participants was reported in two of the five studies,59,60 ranging from 29 to 51 years in the study by Overbeek et al. 59 and from 23 to 78 years in the study by Shia et al. 60 Only one60 of the five studies reported the proportion of participants meeting Amsterdam II criteria (38.2%) or the revised Bethesda guidelines (8.2%) and the specific location of the CRC was reported only by Overbeek et al. 59 None of the studies provided details on the ethnicity of the participants.

All five of the single-gate studies recruiting high-risk populations assessed both MSI and IHC (see Table 6).

Reference standard-positive studies

There were two reference standard-positive studies (i.e. studies that used a variation on a two-gate study design in which only participants who were reference standard positives were recruited). 61,62 The study by Hendriks et al. 61 recruited participants with a germ-line mutation in MLH1, MSH2 or MSH6 and assessed both MSI and IHC whereas the study by Okkels et al. 62 was focused on the identification of a germ-line mutation in MSH6 and assessed only IHC (see Table 6). It should be noted that, although the study by Okkels et al. 62 provides an assessment of the test accuracy of four proteins (MLH1, MSH2, MSH6, PMS2) and data are provided for all proteins combined, these data are not included in this review. Instead, the IHC results for MSH6 only are included.

Both of the reference standard-positive studies included in this review were quite small, with Hendriks et al. 61 recruiting 45 participants and Okkels et al. 62 recruiting 56 participants.

The study by Okkels et al. 62 did not provide details on participants’ age, sex or cancer location or the number of participants meeting Amsterdam II criteria or the revised Bethesda guidelines. The study by Hendriks et al. 61 included a similar number of men and women (44% female), participants ranged in age from 23 to 90 years and the specific locations of the CRCs were reported (see Table 7). The proportions meeting Amsterdam II criteria and the revised Bethesda guidelines were not reported. Neither of the reference standard-positive studies reported the ethnicity of the participants.

Patient selection

Four of the studies were rated as having a low risk of bias for patient selection. Three of these were population-based single-gate studies. 54–56 The other was a reference standard-positive study62 from which only sensitivity estimates could be ascertained. All four of these studies enrolled either a consecutive or a random sample of participants and avoided inappropriate exclusions.

For the remaining seven studies34,57–62 it was unclear whether patient selection could have introduced bias. In all of these studies it was unclear whether inappropriate exclusions were avoided by enrolling a consecutive or a random selection of participants.

For all studies included in the review there were no concerns about whether or not the included participants matched the review question.

Index tests

All of the studies included in the review of test accuracy evaluated IHC. With the exception of the studies by Limburg et al. 55 and Okkels et al. ,62 all studies also assessed MSI. For both of these index tests, all studies were rated as unclear with regard to whether the conduct and interpretation of the test could have introduced bias. None of the studies clearly reported whether the thresholds used were prespecified. In addition, none of the studies reported whether MSI results were interpreted without knowledge of the results of the reference standard. The study by Shia et al. 60 reported that IHC results were interpreted without knowledge of the results of the reference standard, but for the remaining studies this was not reported.

There were no concerns (in any of the studies) that the conduct or interpretation of either of the index tests was different from the review question.

Reference standard

In all of the included studies the reference standard was assessed as being likely to correctly classify the target condition. However, it should be noted that it has not been established in any study that the reference standard is 100% accurate and that there is between-study variation in the reference standard (see Summary of the reference standard in the included studies). Nevertheless, because a genetic definition of LS was used in this review (i.e. the reference standard is LS as indicated by a genetic mutation rather than, for example, LS defined by clinical criteria) and because the inclusion criteria for the reference standard were set so that only studies using the best current methods, or other similarly appropriate methods, for detecting LS-based gene defects were included (see Reference standard), the assumption remains that any specific disagreements between the reference standard and the index test are assumed to result from incorrect classification by the index test. Indeed, there were no concerns (in any of the studies) that the target condition, as defined by the reference standard, did not match the review question.

Despite this, all of the included studies apart from that by Hendriks et al. 61 were rated as unclear with regard to whether or not the conduct or interpretation of the reference standard could have introduced bias. This was because only Hendriks et al. 61 specified that the reference standard results were interpreted without knowledge of the results of the index test, with the rest of the studies not reporting this information. It is therefore unclear for the other studies34,54–60,62 whether there was any prior knowledge that could have influenced the interpretation of the reference standard.

Flow and timing

For all of the included studies it was unclear whether the flow of participants through the study could have introduced bias. In most of the included studies (including the high-risk sample reported by Poynter et al. 34), all of the participants received a reference standard,34,54,55,57–62, although only one54 of these studies provided information to indicate that all of the participants received the same reference standard. In five of these studies55,58,59,61,62 it was not clear whether all participants received the same reference standard and in three of these studies34,57,60 it was clear that not all participants received the same reference standard. Indeed, in the study by Caldes et al. ,57 only the MSI-H cases that were negative for mutations were analysed for genomic deletions, by Southern blot analysis, in MLH1 and MSH2. Similarly, in the study by Shia et al. ,60 only cases with tumours that exhibited MSI but in whom a point mutation was not detected were analysed for large deletions in MLH1 and MSH2. In the high-risk sample in Poynter et al. ,34 direct sequencing was used only to detect MSH6 mutations in cases with absent IHC staining of MSH6. However, in these three studies it was not believed that this would constitute a high risk of bias; it is acceptable for large deletions to be investigated only when a mutation is not found57,60 and for the reference standard to be directed by IHC results. 34

In the study by Southey et al. 56 and the population-based sample reported by Poynter et al. ,34 not all patients received the reference standard. In both of these samples this was because the reference standard was applied to a random sample of participants who were index test negative. In addition, in both of these samples it was clear that not all of the patients who received the reference standard received the same reference standard. In the study by Southey et al. 56 MLPA was used to detect large genomic alterations in MLH1 and MSH2 for cases with tumours lacking expression of at least one MMR protein and for whom no previous mutation had been identified by sequencing and, as with the high-risk sample in the study by Poynter et al. ,34 in the population-based sample reported by Poynter et al. ,34 direct sequencing was used to detect MSH6 mutations only in cases with absent IHC staining of MSH6. As discussed earlier, these variations in study design would not be thought to constitute a high risk of bias.

None of the studies included in the test accuracy review clearly specified the interval between the index tests and the reference standard. However, as the results for the index tests and the reference standard would be expected to be stable over time, this information is of little importance in itself; variations in timing between the index tests and the reference standard would not lead to a high risk of bias.

With regard to missing data, it was clear that in most of the studies some of the participants were excluded from the analysis,34,54–57,61,62 although details regarding the characteristics of these participants were not provided. However, the study by Mueller et al. 58 clearly specified that data were analysed from all participants who received tests. In the studies by Overbeek et al. 59 and Shia et al. 60 it was unclear whether or not all tested participants were analysed.

Quality appraisal summary

Overall, no evidence was found to indicate that any of the included studies was at a high risk of bias. Of course, the single-gate studies based on a high-risk population34,57–60 would necessarily confer a risk of bias (i.e. if the results were used to make assumptions about the general population with CRC). Indeed, when studies recruit only from high-risk populations this would obviously lead to biased estimates of PPV, NPV and yield. It would also possibly lead to biased estimates of sensitivity and specificity because of spectrum bias,64 although in a previous review this did not appear to lead to significant bias in estimates of sensitivity. 42 As a result, this was dealt with in this review by prespecifying that single-gate studies based on high-risk populations would be used only to estimate sensitivity. Ordinarily, two-gate studies would also be at risk of inflating diagnostic accuracy. 65,66 However, the two studies included in this review that were not single-gate studies were not in fact two-gate studies but reference standard-positive studies;61,62 no reference standard negatives were included in these two studies and so an unbiased estimate of sensitivity could be made.

It is important to note that an absence of evidence to suggest that the included studies were at high risk of bias does not suggest that the studies were at low risk of bias. In fact, for all studies it was unclear whether the index tests or the flow and timing of the study would have introduced bias. Similarly, in all but one study61 it was unclear whether the conduct of the reference standard would have introduced bias and only four studies54–56,62 provided sufficient information to establish that the selection of participants was unlikely to have introduced bias.

Assessment of test accuracy: microsatellite instability testing

Microsatellite instability was assessed in eight of the 10 studies (nine of the 11 samples) included in the review of test accuracy: three of the four population-based samples,34,54,56 all five high-risk samples34,57–60 and one of the reference standard-positive studies. 61

Microsatellite instability-high and microsatellite instability-low compared with microsatellite stable

When MSI-L tumours were considered to be index test positives, data were available from all three population-based samples that assessed MSI,34,54,56 two high-risk single-gate study samples34,58 and one reference standard-positive study sample. 61 As previously discussed, the other three studies assessing MSI57,59,60 already categorised MSI using a bimodal distribution (MSI positive or MSI negative in the study by Overbeek et al. 59 and Shia et al. 60; MSI-H or MSS in the study by Caldes et al. 57). The results from these three studies are included here as well as previously in Microsatellite instability-high compared with microsatellite stable and microsatellite instability-low.

Sensitivity and specificity estimates

Sensitivity was calculated for all studies that provided data in which MSI-L was considered to be index test positive (as well as the studies categorising MSI using a bimodal distribution). Specificity was calculated only for the three population-based study samples. 34,54,56 These sensitivity and specificity estimates are reported in Table 13. For the five studies that mentioned assessing unclassified variants in addition to pathogenic mutations,34,54,57,60,61 the unclassified variants were considered to be reference standard negatives.

TABLE 13 - Sensitivity and specificity for MSI (MSI-H or MSI-L vs. MSS)

Population-based sample.

MSI-L not defined.

Clinic-based sample.

Study	Sensitivity (95% CI) (%)	Specificity (95% CI) (%)
Population-based single-gate studies
aPoynter et al., 200834	100.0 (93.9 to 100.0)	29.5 (25.8 to 33.4)
Barnetson et al., 200654	93.3 (77.9 to 99.2)	84.5 (80.0 to 88.2)
Southey et al., 200556	94.4 (72.7 to 99.9)	58.5 (42.1 to 73.7)
Single-gate studies recruiting high-risk samples
bCaldes et al., 200457	79.4 (62.1 to 91.3)	–
Mueller et al., 200958	93.1 (77.2 to 99.2)	–
bOverbeek et al., 200759	90.0 (59.6 to 98.2)	–
cPoynter et al., 200834	94.7 (82.3 to 99.4)	–
bShia et al., 200560	100.0 (85.8 to 100.0)	–
Reference standard-positive study
Hendriks et al., 200361	92.0 (74.0 to 99.0)	–

The data from the study that included an unselected CRC population34 produced a sensitivity estimate of 100.0% (95% CI 93.9% to 100.0%) and therefore sensitivity was unchanged from when MSI-L cases were considered to be index test negatives (see Microsatellite instability-high compared with microsatellite stable and microsatellite instability-low). However, specificity was much lower at 29.5% (95% CI 25.8% to 33.4%), compared with 61.1% (95% CI 57.0% to 65.1%) when MSI-L cases were considered to be index test negatives, reflecting a large increase in false-positive results. The two studies based on age-limited populations that reported data in which MSI-L cases were considered to be index test positives reported sensitivities of 93.3% (95% CI 77.9% to 99.2%)54 and 94.4% (95% CI 72.7% to 99.9%)56 respectively. These sensitivities were higher than those estimated when MSI-L cases were considered to be index test negatives, with sensitivities of 66.7% (95% CI 47.2% to 82.7%) for the study by Barnetson et al. 54 and 72.2% (95% CI 46.5% to 90.3%) for the study by Southey et al. 56 (see Table 11). As would be expected in a trade-off between sensitivity and specificity, specificities were reduced for these two studies when MSI-L cases were considered to be index test positives. However, in the study by Barnetson et al. 54 this reduction was small, with specificity estimated as 84.5% (95% CI 80.0% to 88.2%) when MSI-L cases were considered to be index test positives (see Table 13) and 92.5% (95% CI 89.1% to 95.2%) when MSI-L cases were considered to be index test negatives (see Table 11). For Southey et al. ,56 specificity was estimated as 58.5% (95% CI 42.1% to 73.7%) when MSI-L was considered to be a positive index test result (see Table 13) and 87.8% (95% CI 73.8% to 95.9%) when MSI-L was considered to be a negative index test result (see Table 11). To further demonstrate the differences in test performance between MSI-L as a positive test result and MSI-L as a negative test result, sensitivities and specificities were graphically summarised using a summary receiver operating characteristic (SROC) curve (Figure 4). This visually elucidates the trade-off between sensitivity and specificity across the three population-based study samples used to assess MSI. 34,54,56 In this SROC curve, the unclassified variants mentioned in the studies by Poynter et al. 34 and Barnetson et al. 54 were assumed to be negative reference standard results.

FIGURE 4.

Summary receiver operating characteristic curve graph for MSI testing in which unclassified variants are assumed to be negative reference standard results. Solid black circles, Barnetson et al. ;54 solid green circles, Southey et al. ;56 open circles, Poynter et al. ;34 +, MSI-L is positive; –, MSI-L is negative.

The ideal diagnostic test would generate a point in the upper left corner of the ROC space, representing 100% sensitivity [no false negatives (FNs)] and 100% specificity [no false positives (FPs)]. The closest point to this is found for the study by Barnetson et al. 54 when MSI-L cases are considered to be index test positive results. When MSI-L cases are considered to be index test negative results, specificity improves slightly but sensitivity is reduced. 54 This was also the case for the study by Southey et al. ;56 however, because of the wider discrepancy between sensitivity and specificity, the points are further away from a perfect classification. With regard to the study by Poynter et al. ,34 the lack of FNs ensured that sensitivity remained constant, with only specificity altering according to the allocation of MSI-L results.

It is unsurprising that, on the whole, sensitivity was higher and specificity was lower when MSI-L cases were considered to be index test positive results rather than index test negative results; including MSI-L cases as index test positives essentially lowers the threshold for a positive index test result. Further to the results presented above, Barnetson et al. 54 also reported that MSI-H has a sensitivity of 83% for the detection of MLH1 mutations, 75% for the detection of MSH2 mutations and 17% for the detection of MSH6 mutations, whereas MSI-L has sensitivities of 17%, 25% and 50% respectively. Therefore, the usefulness of including MSI-L as a positive index test result will likely vary according to which gene is mutated. Indeed, according to Mueller et al. ,58 the vast majority of mutations detected are usually in the MSH2 or MLH1 gene, followed by the MSH6 gene and finally the PMS2 gene; however, it is not clear what thresholds were used in this study to define MSI-H, MSI-L and MSS. Similarly, Caldes et al. 57 suggest that mutations in MSH6 do not always produce instability in tumours but in this study an MSI-L categorisation is not used (see Table 10).

For the five single-gate, high-risk samples presented in Table 13,34,57–60 sensitivity estimates ranged from 79.4% (95% CI 62.1% to 91.3%) in the study by Caldes et al. 57 to 100.0% (85.8% to 100.0%) in the study by Shia et al. 60 when MSI-L cases were considered to be index test positive results. Therefore, all of these sensitivity estimates were > 79%. It is important to remember that, for three of these studies,57,59,60 MSI was already categorised bimodally and therefore the estimates presented in Table 13 are identical to those presented in Table 11 (and are based on the same data). For the other two single-gate, high-risk studies,34,58 sensitivities were very similar, albeit slightly higher when MSI-L cases were considered to be index test positive results (93.1%, 95% CI 77.2% to 99.2% in the study by Mueller et al. 58 and 94.7%, 95% CI 82.3% to 99.4% in the study by Poynter et al. 34) than when they were considered to be index test negative results (91.3%, 95% CI 72.0% to 98.9% in the study by Mueller et al. 58 and 86.8%, 95% CI 71.9% to 95.6% in the study by Poynter et al. ;34 see Table 11). The MSI data from the reference standard-positive study61 was also used to generate sensitivity estimates (see Table 13) and, similarly, sensitivity was slightly higher when MSI-L cases were considered to be index test positives (92.0%, 95% CI 74.0% to 99.0%) than when they were considered to be index test negatives (88.0%, 95% CI 68.8% to 97.5%; see Table 11).

Other test accuracy estimates

Likelihood ratios (LR+ and LR–), PPV and NPV were calculated for the three population-based studies reporting MSI data34,54,56 when MSI-L cases were assumed to be positive index test results (and the unclassified variants reported in the studies by Barnetson et al. 54 and Poynter et al. 34 were assumed to be negative reference standard results). These estimates are summarised in Table 14.

TABLE 14 - Likelihood ratios and predictive values for MSI (MSI-H + MSI-L vs. MSS)

NE, not estimable.

Population-based sample.

Study	LR+ (95% CI)	LR– (95% CI)	PPV (95% CI) (%)	NPV (95% CI) (%)
Population-based single-gate studies
aPoynter et al., 200834	1.42 (1.35 to 1.50)	0.00 (NE)	12.6 (9.8 to 16.0)	100.0 (97.9 to 100.0)
Barnetson et al., 200654	6.01 (4.58 to 7.89)	0.08 (0.02 to 0.30)	35.9 (25.3 to 47.6)	99.3 (97.4 to 99.9)
Southey et al., 200556	2.28 (1.56 to 3.33)	0.09 (0.01 to 0.65)	50.0 (32.4 to 67.6)	96.0 (79.6 to 99.9)

For all three population-based studies, the LR+ estimate was reduced when MSI-L cases were considered to be positive index test results (1.42, 95% CI 1.35 to 1.50 in the study by Poynter et al. ;34 6.01, 95% CI 4.58 to 7.89 in the study by Barnetson et al. ;54 2.28, 95% CI 1.56 to 3.33 in the study by Southey et al. 56) compared with when they were considered to be negative index test results (2.57, 95% CI 2.32 to 2.85 in the study by Poynter et al. ;34 8.94, 95% CI 5.54 to 14.20 in the study by Barnetson et al. ;54 5.92, 95% CI 2.48 to 14.10 in the study by Southey et al. 56; see Table 12). As before, the LR– could be estimated for only two of the studies54,56 because there were no false-negative MSI results in the study by Poynter et al. 34 Again, the LR– estimate was similar in both studies54,56 and was lower when MSI-L cases were considered to be positive index test results (0.08, 95% CI 0.02 to 0.30 in the study by Barnetson et al. ;54 0.09, 95% CI 0.01 to 0.65 in the study by Southey et al. 56) than when MSI-L cases were considered to be negative index test results (0.36, 95% CI 0.21 to 0.60 in the study by Barnetson et al. ;54 0.32, 95% CI 0.15 to 0.67 in the study by Southey et al. ;56 see Table 12).

As before, PPV estimates varied a lot more between the studies, with the lowest estimate still coming from the study by Poynter et al. 34 (12.6%, 95% CI 9.8% to 16.0%) and the highest estimate still coming from the study by Southey et al. 56 (50.0%, 95% CI 32.4% to 67.6%). For all three studies, PPV estimates were lower when MSI-L cases were considered to be index test positives than when they were considered to be index test negatives (see Table 12). Conversely, NPV estimates, which were consistent across the three studies (with all estimates being > 96%), were higher when MSI-L cases were considered to be index test positives than when they were considered to be index test negatives (see Table 12). However, it should be noted that, for the unselected CRC population in the study by Poynter et al. ,34 the NPV estimate was 100% regardless of whether MSI-L cases were considered to be positive or negative index test results, although the CIs were slightly wider when MSI-L cases were considered to be positive index test results (95% CI 97.9% to 100.0%) than when they were considered to be negative index test results (95% CI 99.0% to 100.0%).

Secondary analyses (unclassified variants as reference test positives)

As before, secondary analyses were conducted in which unclassified variants were considered to indicate a positive reference standard test result. Although five of the included studies34,54,57,60,61 reported unclassified variants in their assessment of MSI, only one61 provided sufficient data to conduct secondary analyses. However, because this study is a reference standard-positive study, only sensitivity estimates were made.

When unclassified variants were considered to be reference standard positives and MSI-L cases were considered to be index test positives, Hendriks et al. 61 reported a sensitivity of 93.9% (95% CI 80.3% to 98.3%). As expected, because the threshold for a MSI case is essentially lowered when MSI-L is considered to be an index test positive, this sensitivity was higher than that reported when MSI-L cases were considered to be index test negatives and unclassified variants were considered to be reference standard positives (84.8%, 95% CI 69.0% to 93.3%). When comparing the results from the study by Hendriks et al. 61 when unclassified variants were considered to be reference standard positives (MSI-L as an index test positive) with those generated when unclassified variants were considered to be reference standard negatives (MSI-L as an index test positive), sensitivity was very similar (93.9%, 95% CI 80.3% to 98.3% vs. 92.0%, 95% CI 74.0% to 99.0% respectively).

Assessment of test accuracy: immunohistochemistry

Immunohistochemistry was conducted in all 10 studies (11 samples) included in the review of test accuracy. However, not all studies provided sufficient data to be included in the analyses. Indeed, despite IHC being conducted for the high-risk sample in the study by Poynter et al. 34 and the sample in the study by Mueller et al. ,58 insufficient data were provided for these samples to be included in any of the IHC analyses. For the population-based sample in the study by Poynter et al. 34 and the samples in the studies by Barnetson et al. ,54 Southey et al. ,56 Hendriks et al. 61 and Okkels et al. ,62 the IHC results were reported according to the particular protein assessed (MLH1, MSH2, MSH6 or PMS2), enabling an assessment of IHC for at least one of these individual proteins (i.e. whether an absence of a particular protein accurately identifies a mutation in a particular gene). In seven cases54–57,59–61 an overall result was provided (i.e. whether a positive IHC result, regardless of which protein this applies to, predicts a positive reference standard result).

As with the results for MSI, in primary analyses unclassified variants were considered to be reference standard negatives. When sufficient data were available, secondary analyses were conducted in which unclassified variants were considered to be reference standard positives.

Immunohistochemical analysis of individual proteins

The analyses described in the previous section (in which overall IHC results are considered) are limited in what they can demonstrate about the loss of expression of individual proteins and how this relates to pathogenic mutations. Indeed, Overbeek et al. 59 note that tumour cells of MLH1 mutation carriers generally lacked MLH1 and PMS2 according to IHC staining, those of MSH2 mutation carriers lacked MSH2 and MSH6, those of MSH6 mutation carriers lacked MSH6 and those of PMS2 mutation carriers lacked PMS2. Furthermore, Barnetson et al. 54 suggest that ‘the absence of MSH6 protein predicted mutations in MSH2 or MSH6 . . . , as did the absence of MSH2 for mutations in MSH2 or MSH6 . . . , reflecting the biologic interaction between these proteins’ (p. 2756).

Indeed, the significance of the patterns of IHC abnormality in predicting underlying genetic causes of CRC predisposition appears to be becoming clearer, if more complex (see Table 4); it is becoming apparent that not all mutations are associated with loss or abnormality of the corresponding protein and that a specific IHC abnormality cannot be assumed to be an absolute indicator of the underlying genetic defect. 37 However, it was beyond the scope of this review to use data from the included studies to attempt an assessment of which IHC results were more or less likely to predict the presence of particular pathogenic mutations (or whether particular patterns or combinations of IHC abnormality correspond to particular defects). In any case, there were insufficient individual patient data available from the population-based samples to attempt such an analysis.

However, for the population-based sample in the study by Poynter et al. 34 and the samples in the studies by Barnetson et al. ,54 Southey et al. ,56 Hendriks et al. 61 and Okkels et al. ,62 sufficient data were available to enable an assessment of IHC for at least one individual protein, in terms of whether loss of expression of that protein was an accurate test for a pathogenic mutation in the corresponding gene (regardless of whether there was also loss of expression of additional proteins). Three of these studies54,56,61 provided sufficient data to assess whether loss of protein expression of MLH1, MSH2 or MSH6 was an accurate test of a pathogenic mutation in the same gene. Southey et al. 56 additionally provided data to enable an assessment of whether loss of protein expression of PMS2 was an accurate test of a pathogenic mutation in PMS2. The study by Okkels et al. 62 was designed to assess whether lack of protein expression of MSH6 would predict a pathogenic mutation in MSH6. For the population-based sample in Poynter et al. ,34 limited IHC data were available for individual proteins. However, sufficient data were available to assess the specificity of the loss of protein expression of MLH1. Sensitivity was not calculated for this study because these data were available only for reference standard negatives. The estimates of sensitivity and specificity generated from these studies (for IHC analysis of individual proteins) are provided in Table 17.

TABLE 17 - Sensitivity and specificity of IHC according to lack of protein expression

Study	MLH1		MSH2		MSH6		PMS2
	Sensitivity (95% CI) (%)	Specificity (95% CI) (%)	Sensitivity (95% CI) (%)	Specificity (95% CI) (%)	Sensitivity (95% CI) (%)	Specificity (95% CI) (%)	Sensitivity (95% CI) (%)	Specificity (95% CI) (%)
Population-based single-gate studies
Poynter et al., 200834	–	70.6 (66.8 to 74.2)	–	–	–	–	–	–
Barnetson et al., 200654	100.0 (73.5 to 100.0)	96.0 (93.1 to 97.9)	81.8 (48.2 to 97.7)	96.2 (93.5 to 98.0)	75.0 (19.4 to 99.4)	95.4 (92.5 to 97.4)	–	–
Southey et al., 200556	50.0 (26.0 to 74.0)	90.2 (76.9 to 97.3)	22.2 (6.4 to 47.6)	92.7 (80.1 to 98.5)	44.4 (21.5 to 69.2)	92.7 (80.1 to 98.5)	55.6 (30.8 to 78.5)	87.8 (73.8 to 95.9)
Reference standard-positive studies
Hendriks et al., 200361	85.7 (63.7 to 97.0)	–	81.8 (48.2 to 97.7)	–	75.0 (19.4 to 99.4)	–	–	–
Okkels et al., 201262	–	–	–	–	72.7 (51.8 to 86.8)	–	–	–

Of the four studies that provided data relevant to whether loss of expression of MLH1 was an accurate test result for assessing a pathogenic mutation in MLH1, three were population-based single-gate studies34,54,56 and one was a reference standard-positive study. 61 The studies by Barnetson et al. ,54 Southey et al. 56 and Hendriks et al. 61 all provided data from which sensitivities were generated. These ranged from 50.0% (95% CI 26.0% to 74.0%) for Southey et al. 56 to 100.0% (95% CI 73.5% to 100.0%) for Barnetson et al. 54 The three population-based studies34,54,56 provided data from which specificities were generated. These ranged from 70.6% (95% CI 66.8% to 74.2%) for Poynter et al. 34 to 96.0% (95% CI 93.1% to 97.9%) for Barnetson et al. 54 The results for MSH2 were even more variable. Three studies provided data relevant to whether or not loss of expression of MSH2 was an accurate test result for assessing a pathogenic mutation in MSH2. Two of these were single-gate population-based studies54,56 and one was a reference standard-positive study. 61 Sensitivities ranged from 22.2% (95% CI 6.4% to 47.6%) for Southey et al. 56 to 81.8% (95% CI 48.2% to 97.7%) for Barnetson et al. 54 and Hendriks et al. 61 The two population-based studies54,56 provided data from which specificities were generated. These were fairly consistent across the studies, with both specificity estimates being > 92% (see Table 17).

Four studies provided data relevant to whether loss of expression of MSH6 was an accurate test result for assessing a pathogenic mutation in MSH6. Two of these were single-gate population-based studies54,56 and two were reference standard-positive studies. 61,62 Again, there was more variation in the sensitivities generated than in the specificities generated: sensitivities ranged from 44.4% (95% CI 21.5% to 69.2%) for Southey et al. 56 to 75.0% (95% CI 19.4% to 99.4%) for both Barnetson et al. 54 and Hendriks et al. ,61 whereas specificities, which were generated only for the population-based studies,54,56 were similar across studies, with both studies producing an estimate of > 92% (see Table 17). Only one study56 provided data to enable an assessment of whether loss of expression of PMS2 was an accurate test result for assessing a pathogenic mutation in PMS2, producing a sensitivity estimate of 55.6% (95% CI 30.8% to 78.5%) and a specificity estimate of 87.8 (95% CI 73.8% to 95.9%) (see Table 17).

With the exception of the sensitivity estimates generated from the study by Southey et al. ,56 it appears that the data presented in Table 17 are fairly consistent across studies. The different estimates generated from the study by Southey et al. 56 appear to be the result of higher rates of false-negative results than in the other studies. It is possible that this is because of specific between-study differences in the assessment of IHC results (a positive IHC result is a somewhat subjective judgement, made by human assessors, and so inter-rater variability may have an impact on the results). Again, it is also possible that between-study differences in the reference standard could, to some extent, account for these differences in sensitivity estimates.

This is further demonstrated in Figure 5, in which sensitivities and specificities from the studies by Barnetson et al. 54 and Southey et al. 56 are graphically summarised using a SROC curve. This visually elucidates the trade-off between sensitivity and specificity across the two studies for each of the individual proteins assessed (MLH1, MSH2 and MSH6, as well as PMS2 for Southey et al. 56). In this figure, unclassified variants were assumed to be negative reference standard results.

FIGURE 5.

Summary receiver operating characteristic curve graph for IHC testing in which unclassified variants are assumed to be negative reference standard results. B, Barnetson et al. ;54 S, Southey et al. 56

Other test accuracy estimates

The other test accuracy outcomes included in this review were LRs (LR+ and LR–), PPV and NPV, accuracy or concordance with the reference standard, diagnostic yield and test failure rates. As previously mentioned, the last three outcomes (accuracy or concordance with the reference standard, diagnostic yield and test failure rates) were not reported in any of the included studies. The other four outcomes (LR+, LR–, PPV and NPV) were calculated for the two population-based studies with sufficient IHC data by protein available. 54,56

The LR+ and LR– estimates are reported in Table 18. Unclassified variants were considered to be reference standard negative results. For loss of protein expression of MLH1, the LR+ was 25.0 (95% CI 14.4 to 43.5) for Barnetson et al. 54 and 5.1 (95% CI 1.8 to 14.5) for Southey et al. 56 For loss of protein expression of MSH2, the LR+ was 21.6 (95% CI 11.6 to 40.2) for Barnetson et al. 54 and 3.0 (95% CI 0.8 to 12.2) for Southey et al. 56 For loss of protein expression in MSH6, the LR+ was 16.2 (95% CI 7.6 to 34.3) for Barnetson et al. 54 and 6.1 (95% CI 1.8 to 20.3) for Southey et al. 56 For loss of protein expression in PMS2, data were available only for Southey et al. ,56 with a LR+ of 4.6 (95% CI 1.8 to 11.4). There are several possible reasons why the LR+ estimates were higher for Barnetson et al. 54 than for Southey et al. ,56 including the fact that the study by Barnetson et al. 54 was a larger study than that by Southey et al. ,56 that the reference standard was not identical in these studies and that there is a possibility that IHC ratings may have differed across studies (inter-rater reliability).

TABLE 18 - Likelihood ratios for IHC according to loss of protein expression

NE, not estimable.

Study	Protein
	MLH1		MSH2		MSH6		PMS2
	LR+ (95% CI)	LR– (95% CI)	LR+ (95% CI)	LR– (95% CI)	LR+ (95% CI)	LR– (95% CI)	LR+ (95% CI)	LR– (95% CI)
Barnetson et al., 200654	25.0 (14.4 to 43.5)	0.0 (NE)	21.6 (11.6 to 40.2)	0.2 (0.1 to 0.7)	16.2 (7.6 to 34.3)	0.3 (0.1 to 1.4)	–	–
Southey et al., 200556	5.1 (1.8 to 14.5)	0.6 (0.4 to 0.9)	3.0 (0.8 to 12.2)	0.8 (0.7 to 1.1)	6.1 (1.8 to 20.3)	0.6 (0.4 to 0.9)	4.6 (1.8 to 11.4)	0.5 (0.3 to 0.9)

For loss of protein expression of MLH1, the LR– was estimated for only one study56 (0.6, 95% CI 0.4 to 0.9) because there were no false-negative results in the study by Barnetson et al. 54 For loss of protein expression of MSH2, the LR– was 0.2 (95% CI 0.1 to 0.7) for Barnetson et al. 54 and 0.8 (95% CI 0.7 to 1.1) for Southey et al. 56 For loss of protein expression of MSH6, the LR– was 0.3 (95% CI 0.1 to 1.4) for Barnetson et al. 54 and 0.6 (95% CI 0.4 to 0.9) for Southey et al. 56 For loss of protein expression of PMS2, data were available only from Southey et al. ,56 with the LR– estimated as 0.5 (95% CI 0.3 to 0.9).

The PPV and NPV estimates are reported in Table 19. Again, unclassified variants were considered to be reference standard negative results. For loss of protein expression of MLH1 and MSH2, the PPV and NPV results were largely consistent across the two studies providing data,54,56 although the PPV estimates were lower in the study by Barnetson et al. 54 and the NPV estimates were lower in the study by Southey et al. 56 For loss of expression of MSH6, the NPV estimates were consistent across the two studies but the PPV estimates were vastly different, with the data from Barnetson et al. 54 resulting in a PPV of 16.7% (95% CI 3.6% to 41.4%) and the data from Southey et al. 56 resulting in a PPV of 72.7% (95% CI 39.0% to 94.0%). Although the reason for this difference is not completely clear, it is likely, in part, to be because of the very low number of true positive (TP) results (n = 3) for loss of expression of MSH6 in the study by Barnetson et al. 54 Again, only Southey et al. 56 provided data for loss of protein expression of PMS2, with the PPV estimated as 66.7% (95% CI 38.4% to 88.2%) and the NPV estimated as 81.8% (95% CI 67.3% to 91.8%).

TABLE 19 - Positive predictive values and NPVs according to loss of protein expression

Study	Protein
	MLH1		MSH2		MSH6		PMS2
	PPV (95% CI) (%)	NPV (95% CI) (%)	PPV (95% CI) (%)	NPV (95% CI) (%)	PPV (95% CI) (%)	NPV (95% CI) (%)	PPV (95% CI) (%)	NPV (95% CI) (%)
Barnetson et al., 200654	50.0 (29.1 to 70.9)	100.0 (98.7 to 100.0)	42.9 (21.8 to 66.0)	99.3 (97.7 to 99.9)	16.7 (3.6 to 41.4)	99.7 (96.2 to 100.0)	–	–
Southey et al., 200556	69.2 (38.6 to 90.9)	80.4 (66.1 to 90.6)	57.1 (18.4 to 90.1)	73.1 (59.0 to 84.4)	72.7 (39.0 to 94.0)	79.2 (65.0 to 89.5)	66.7 (38.4 to 88.2)	81.8 (67.3 to 91.8)

Secondary analyses (unclassified variants as reference test positives)

Secondary analyses were conducted in which unclassified variants were considered to be positive reference standard test results. It was not possible to include all five studies34,54,56,61,62 reporting IHC data for individual proteins in these analyses because sufficient data were not always reported. Indeed, only one study61 provided sufficient data to conduct secondary analyses and, because this study was a reference standard-positive study, only sensitivity estimates were made. These sensitivity estimates were very similar to those estimated from data in which unclassified variants were considered to be reference standard negatives.

Indeed, when unclassified variants were considered to be reference standard positives, loss of protein expression of MLH1 was estimated to have a sensitivity of 80.0% (95% CI 60.8% to 91.1%) compared with the previously reported sensitivity of 81.0% (95% CI 58.1% to 94.6%) when unclassified variants were considered to be reference test negatives. Loss of protein expression of MSH2 was estimated to have a sensitivity of 83.3% (95% CI 55.1% to 95.3%) compared with the previously reported sensitivity of 81.8% (95% CI 48.2% to 97.7%) when unclassified variants were considered to be reference test negatives. Loss of protein expression in MSH6 was estimated to have a sensitivity of 80.0% (95% CI 37.6% to 96.4%) compared with the previously reported sensitivity of 75.0% (95% CI 19.4% to 99.4%) when unclassified variants were considered to be reference test negatives (see Table 17).

Summary of included studies

Ten studies met the test accuracy review inclusion criteria. One of the included studies included two distinct samples (a population-based sample and a high-risk sample) and these two samples were treated separately. Thus, although 10 studies were included, 11 populations/data sets were included. The results from all 11 populations were considered.

The 11 study samples were divided as follows: four single-gate studies with population-based samples, including one apparently unselected CRC population34 and three age-limited populations;54–56 five single-gate studies based on high-risk populations;34,57–60 and two studies that used a variation on a two-gate study design61,62 in which participants with positive reference standard results were recruited but no reference standard negatives were recruited. For this report, and for clarity, these studies are termed ‘reference standard-positive studies’.

With the exception of the studies by Limburg et al. 55 and Okkels et al. ,62 all studies assessed MSI. Although IHC was conducted in all 10 studies (11 samples) included in the review, not all studies provided sufficient data to be included in the analyses. Indeed, for the high-risk sample in the study by Poynter et al. 34 and the sample in the study by Mueller et al. ,58 despite IHC being conducted, insufficient data were provided for these samples to be included in any of the IHC analyses. None of the studies made a direct comparison between MSI and IHC. As such, the results are reported separately for these tests.

There was significant methodological and clinical heterogeneity across studies. In particular, the reference standard differed between studies, as did the index tests. With regard to the reference standard there were differences in the testing methods used (including the sequencing methods used and genes tested, the techniques used to test for large genomic alterations and deletions, the genes tested for large genomic alterations and deletions and whether unclassified variants were investigated). As a result of this, pooling of data in statistical analyses was not appropriate. In addition, there were insufficient data to conduct subgroup analyses based on most of these variables. However, test performance statistics were primarily generated with unclassified variants categorised as negative reference standard results and two studies57,61 provided sufficient data to conduct secondary analyses in which unclassified variants were categorised as positive reference standard results.

Quality appraisal of all 11 data sets was conducted using phase 3 of the QUADAS-2 tool. Four of the studies were rated as having a low risk of bias for patient selection, with three of these being population-based single-gate studies54–56 and one being a reference standard-positive study62 for which only sensitivity estimates could be made. For all studies included in the review there were no concerns about whether or not the included participants matched the review question. For both index tests, all studies were rated as having an unclear risk of bias with regard to whether the conduct and interpretation of the test could have introduced bias, but there were no concerns (in any of the studies) that the conduct or interpretation of either of the index tests was different from the review question. All of the included studies apart from that by Hendriks et al. 61 were rated as unclear with regard to whether or not the conduct or interpretation of the reference standard could have introduced bias. This was because only the study by Hendriks et al. 61 specified that the reference standard results were interpreted without knowledge of the results of the index test, with the rest of the studies not reporting this information. However, in all of the included studies the reference standard was assessed as being likely to correctly classify the target condition (because a genetic definition of LS was used in this review) and there were no concerns in any of the studies that the target condition, as defined by the reference standard, did not match the review question. For all included studies it was unclear whether the flow of participants through the study could have introduced bias.

The index tests included in this review (MSI and IHC) are highly susceptible to spectrum effects in populations that have been selected on the basis of clinical characteristics. In particular, an increased presence of MMR mutation carriers in a population would change the apparent sensitivity and specificity of the index tests. 54 However, a previous review did not find that this issue led to significant bias in estimates of sensitivity. 42 Because of this, studies recruiting high-risk populations were used only to estimate sensitivity. For the four samples included in this review that can be described as population-based samples,34,54–56 sensitivity, specificity, LR+, LR–, PPV and NPV were all estimated. However, it should be noted that three of these studies recruited age-limited populations,54–56 for which some spectrum bias may be expected.

Summary of the results for microsatellite instability

There were a variety of between-study differences in the MSI testing procedures used. In addition, potential differences between studies in MSI testing methods were not always clear because the methods used were not always reported in sufficient detail. For example, three34,58,59 of the eight studies assessing MSI did not report microdissection techniques (microdissection assists in ensuring that malignant tissue that does not contain DNA from surrounding healthy colonic tissue is analysed). The other differences between studies with regard to MSI testing methods can be categorised as differences in the panel of markers used, differences in the way in which MSI was categorised (e.g. as a bimodal or a trimodal distribution) and differences in the thresholds used to categorise MSI. Indeed, none of the population-based studies assessed the same panel of markers (differences exist in both the type and the number of markers assessed). Five studies34,54,56,58,61 defined tumours as MSI-H, MSI-L or MSS, also known as a trimodal distribution, two59,60 defined tumours as MSI positive or MSI negative, also known as a bimodal distribution, and one57 used a bimodal distribution but defined tumours as either MSI-H or MSS. Of the five studies that used the MSI-H, MSI-L and MSS categories (trimodal distribution), the thresholds used to categorise the tumours varied greatly, with one study34 using the commonly used threshold of ≥ 30% of unstable markers to define MSI-H tumours, three studies54,56,61 using differing numbers of unstable markers to define MSI-H tumours and one study58 not providing details on the thresholds used to categorise the tumours as MSI-H, MSI-L or MSS. Of the two studies that defined tumours as MSI positive or MSI negative (bimodal distribution), one reported using a threshold of ≥ 30% of unstable markers to define MSI-positive tumours60 and the other defined MSI-positive tumours as those with more than two unstable Bethesda panel markers. 59 The study by Caldes et al. 57 defined MSI-H tumours as those with two or more unstable Bethesda panel markers (or one marker in the case of BAT26) and MSS tumours as those showing no instability. It is not clear how cases with only one unstable marker (other than BAT26) were categorised, but in any case data were presented only for tumours that were categorised as MSS and MSI-H.

In primary analyses unclassified variants were categorised as negative reference standard results. Six study samples (including both samples in the study by Poynter et al. 34) provided data in which MSI-L was considered to be a negative index test result. 34,54,56,58,61 The other three samples utilised a bimodal distribution of MSI. 57,59,60 Across all nine samples, when MSI-L was considered to be a negative index test result, sensitivity ranged from 66.7% (95% CI 47.2% to 82.7%) in the population-based sample reported by Barnetson et al. 54 to 100.0% (95% CI 93.9% to 100.0%) in the population-based sample reported in Poynter et al. 34 and 100% (95% CI 85.8% to 100.0%) in the high-risk sample reported in Shia et al. 60 Sensitivity increased when MSI-L was considered to be a positive index test result (for the six study samples in which a trimodal distribution of MSI was used, with data remaining unchanged for the three samples in which a bimodal distribution of MSI was used). Indeed, across the nine study samples, the lower end of the range for sensitivity increased to 79.4% (95% CI 62.1% to 91.3%) in the high-risk sample recruited by Caldes et al. ,57 with the upper end of the range still being 100%.

In primary analyses (in which unclassified variants were categorised as negative reference standard results) three population-based study samples provided data in which MSI-L was considered to be a negative index test result. 34,54,56 Across these three samples, when MSI-L was considered to be a negative index test result, specificity ranged from 61.1% (95% CI 57.0% to 65.1%) in the study by Poynter et al. 34 to 92.5% (95% CI 89.1% to 95.2%) in the study by Barnetson et al. 54 It should be noted that the study by Barnetson et al. 54 included an age-limited sample whereas the study by Poynter et al. 34 included an unselected CRC population. Specificity decreased when MSI-L was considered to be a positive index test result. Indeed, the lower end of the range decreased to 29.5% (95% CI 25.8% to 33.4%) in the study by Poynter et al. 34 and the upper end of the range decreased to 84.5% (95% CI 80.0% to 88.2%) in the study by Barnetson et al. 54

It is unsurprising that, on the whole, sensitivity was higher and specificity was lower when MSI-L was considered to be a positive result than when MSI-L was considered to be a negative result; including MSI-L as a positive result essentially lowers the threshold for a positive index test result.

For the three studies that recruited population-based samples,34,54,56 LR+, LR–, PPV and NPV were also calculated. The LR+ was reduced when MSI-L was considered to be a positive index test result (1.42, 95% CI 1.35 to 1.50 for Poynter et al. ;3 6.01, 95% CI 4.58 to 7.89 for Barnetson et al. ;54 2.28, 95% CI 1.56 to 3.33 for Southey et al. 56) compared with when MSI-L was considered to be a negative index test result (2.57, 95% CI 2.32 to 2.85 for Poynter et al. ;34 8.94, 95% CI 5.54 to 14.20 for Barnetson et al. ;54 5.92, 95% CI 2.48 to 14.10 for Southey et al. 56) The LR– could be estimated for only two of the studies54,56 because there were no false-negative MSI results in the study by Poynter et al. 34 The LR– was similar in both studies and was lower when MSI-L was considered to be a positive index test result (0.08, 95% CI 0.02 to 0.30 for Barnetson et al. ;54 0.09, 95% CI 0.01 to 0.65 for Southey et al. 56) than when it was considered to be a negative index test result (0.36, 95% CI 0.21 to 0.60 for Barnetson et al. ;54 0.32, 95% CI 0.15 to 0.67 for Southey et al. 56). PPV estimates varied a lot more between the studies but were lower when MSI-L was considered to be a positive index test result than when it was considered to be a negative index test result. Conversely, NPV estimates were consistent across the three studies and were higher when MSI-L was considered to be a positive index test result than when it was considered to be a negative index test result. However, it should be noted that, for the unselected CRC population in the study by Poynter et al. ,34 the NPV estimate was 100% regardless of whether MSI-L was considered to be a positive or a negative index test result, although the CIs were slightly wider when MSI-L was considered to a positive result (95% CI 97.9% to 100.0%) than when it was considered to be a negative result (95% CI 99.0% to 100.0%).

Secondary analyses were conducted when data permitted in which unclassified variants were considered to be positive reference standard results. When MSI-L was considered to be a negative index test result, two studies57,61 provided sufficient data to conduct secondary analyses of the sensitivity estimates. Caldes et al. 57 reported a sensitivity of 81.6% (95% CI 65.7% to 92.3%) and Hendriks et al. 61 reported a sensitivity of 84.8% (95% CI 69.0% to 93.3%). These results were similar to those obtained when unclassified variants were considered to be negative reference standard results (79.4%, 95% CI 62.1% to 91.3% for Caldes et al. ;57 88.8%, 95% CI 68.8% to 97.5% for Hendriks et al. 61). When MSI-L was considered to be a positive index test result, only one study61 provided sufficient data to conduct a secondary analysis of the sensitivity estimate. In this study, sensitivity was 93.9% (95% CI 80.3% to 98.3%), which was similar to the estimate when unclassified variants were considered to be negative reference standard results (92.0%, 95% CI 74.0% to 99.0%).

Summary of the results for immunohistochemistry

In primary analyses unclassified variants were categorised as negative reference standard results. Seven studies54–57,59–61 provided data to assess the accuracy of an overall IHC result at identifying a positive reference standard result (i.e. whether a positive IHC result, regardless of which protein this applies to, identifies a positive reference standard result). For the population-based sample in the study by Poynter et al. 34 and the samples in the studies by Barnetson et al. 54; Southey et al. ,56 Hendriks et al. 61 and Okkels et al. ,62 IHC results were reported according to the particular protein assessed (MLH1, MSH2, MSH6 or PMS2), enabling an assessment of IHC for at least one of these individual proteins (i.e. whether an absence of a particular protein accurately identifies a mutation in that particular gene).

All of the seven studies54–57,59–61 assessing the overall test performance of IHC assessed MLH1, MSH2 and MSH6. Therefore, abnormal staining for any of these three proteins was considered to be a positive index test result. However, Southey et al. 56 and Overbeek et al. 59 also assessed PMS2. Therefore, for these two studies an abnormal PMS2 result was also included as a positive index test result. Three of these studies were population-based studies,54–56 three were single-gate high-risk studies57,59,60 and one was a reference standard-positive study (in which only reference standard positives were recruited). Sensitivity estimates ranged from 80.8% (95% CI 60.6% to 93.4%) in the study by Shia et al. 60 to 100.0% (95% CI 81.5% to 100.0%) in the study by Southey et al. 56 The study by Southey et al. 56 included an assessment of PMS2 as well as assessments of MLH1, MSH2 and MSH6 and it is possible that this accounted for the higher sensitivity estimate in this study. Nevertheless, all sensitivity estimates were > 80%. Because of the potential for spectrum bias, it would be expected that studies recruiting high-risk populations would have higher sensitivity estimates than those recruiting population-based samples. However, as discussed in a systematic review by Palomaki et al. ,42 and as with the MSI results reported earlier, we did not find great differences between the sensitivity estimates in the population-based studies and those in the high-risk studies. This could be because the three population-based studies with overall IHC data54–56 were based on age-limited populations and may also be subject to spectrum bias.

For two of the population-based studies55,56 specificity, LR+, LR–, PPV and NPV were also estimated for the overall IHC results. These analyses were not conducted for the third population-based study54 because overall IHC results were available only for reference standard-positive participants. Specificity was estimated as 91.9% (95% CI 86.3% to 95.7%) for Limburg et al. 55 and 80.5% (95% CI 65.1% to 91.2%) for Southey et al. 56 LR+ was estimated as 10.6 (95% CI 5.7 to 19.7) for Limburg et al. 55 and 5.1 (95% CI 2.8 to 9.5) for Southey et al. 56 LR– could be estimated for only one study55 (0.16, 95% CI 0.02 to 0.95) because there were no false-negative overall IHC results in the study by Southey et al. 56 PPV was lower in the study by Limburg et al. 55 (33.3%, 95% CI 13.3% to 59.0%) than in the study by Southey et al. 56 (69.2%, 95% CI 48.2% to 85.7%). NPV was high in both studies, with both estimates being > 99%. Again, when apparent differences in IHC performance exist between these two studies (e.g. in the PPV results), it should be considered that Southey et al. 56 included an assessment of PMS2 in their results whereas Limburg et al. 55 did not. Additionally, the specific techniques and methods used to perform the reference standard differed between studies and this may also impact on the apparent test performance.

Secondary overall IHC analyses were conducted in which unclassified variants were considered to be positive reference standard results. Only two studies57,61 provided sufficient data to conduct these secondary analyses and, because the study by Caldes et al. 57 included a high-risk population and that by Hendriks et al. 61 was a reference standard-positive study, only sensitivity was estimated (75.0%, 95% CI 57.8% to 87.9% for Caldes et al. ;57 88.6%, 95% CI 76.0% to 95.0% for Hendriks et al. 61). For Caldes et al. 57 this represented quite a reduction in sensitivity compared with when unclassified variants were considered to be reference test negatives (96.4%, 95% CI 81.7% to 99.9%).

For the population-based sample in the study by Poynter et al. 34 and the samples in the studies by Barnetson et al. ,54 Southey et al. ,56 Hendriks et al. 61 and Okkels et al. ,62 an assessment was made of IHC for at least one individual protein. Four studies provided data relevant to whether loss of expression of MLH1 was an accurate test result for assessing a pathogenic mutation in MLH1, of which three were population-based single-gate studies34,54,56 and one was a reference standard-positive study. 61 The studies by Barnetson et al. ,54 Southey et al. 56 and Hendriks et al. 61 all provided data from which sensitivities were generated; these ranged from 50.0% (95% CI 26.0% to 74.0%) for Southey et al. 56 to 100.0% (95% CI 73.5% to 100.0%) for Barnetson et al. 54 The three population-based studies34,54,56 provided data from which specificities were generated; these ranged from 70.6% (95% CI 66.8% to 74.2%) for Poynter et al. 34 to 96.0% (95% CI 93.1% to 97.9%) for Barnetson et al. 54 The results for MSH2 were even more variable. Three studies provided data for MSH2,54,56,61 with sensitivities ranging from 22.2% (95% CI 6.4% to 47.6%) for Southey et al. 56 to 81.8% (95% CI 48.2% to 97.7%) for both Barnetson et al. 54 and Hendriks et al. 61 The two population-based studies54,56 provided data from which specificities were generated, with both being > 92%. Four studies provided data for MSH6 and, again, there was more variation in the sensitivities generated than in the specificities generated: sensitivities ranged from 44.4% (95% CI 21.5% to 69.2%) for Southey et al. 56 to 75.0% (95% CI 19.4% to 99.4%) for both Barnetson et al. 54 and Hendriks et al. ,61 whereas specificities, which were generated only for the population-based studies,54,56 were > 92% in both studies. It was clear that, for loss of expression of MLH1, MSH2 and MSH6, the sensitivity estimates generated from the study by Southey et al. 56 were lower than those in the other studies. It is possible that this is because of specific between-study differences in the assessment of IHC (a positive IHC result is a somewhat subjective judgement, made by human assessors, and so inter-rater variability may impact on the results). Again, it is also possible that between-study differences in the reference standard could, to some extent, account for these differences in sensitivity. Only the study by Southey et al. 56 provided IHC data for PMS2, with a sensitivity estimate of 55.6% (95% CI 30.8% to 78.5%) and a specificity estimate of 87.8% (95% CI 73.8% to 95.9%).

The LR+, LR–, PPV and NPV were calculated for the two population-based studies with sufficient IHC data available by protein. 54,56 For MLH1, MSH2 and MSH6, LR+ was greater in the study by Barnetson et al. 54 than in the study by Southey et al. 56 (MLH1: 25.0, 95% CI 14.4 to 43.5 for Barnetson et al. 54 and 5.1, 95% CI 1.8 to 14.5 for Southey et al. ;56 MSH2: 21.6, 95% CI 11.6 to 40.2 for Barnetson et al. 54 and 3.0, 95% CI 0.8 to 12.2 for Southey et al. ;56 MSH6: 16.2, 95% CI 7.6 to 34.3 for Barnetson et al. 54 and 6.1, 95% CI 1.8 to 20.3 for Southey et al. 56) For PMS2, data were available only from Southey et al. ,56 with the LR+ estimated as 4.6 (95% CI 1.8 to 11.4). There are several possible reasons why the LR+ estimates were higher in the study by Barnetson et al. 54 than in the study by Southey et al. ,56 including that the study by Barnetson et al. 54 was a larger study than that by Southey et al. ,56 that the reference standard was not identical in these studies and that there is a possibility that IHC ratings may have differed across studies (inter-rater reliability). For MLH1, the LR– was estimated for only one study56 (0.6, 95% CI 0.4 to 0.9) because there were no false-negative results in the study by Barnetson et al. 54 For MSH2, the LR– was 0.2 (95% CI 0.1 to 0.7) for Barnetson et al. 54 and 0.8 (95% CI 0.7 to 1.1) for Southey et al. 56 and, for MSH6, the LR– was 0.3 (95% CI 0.1 to 1.4) for Barnetson et al. 54 and 0.6 (95% CI 0.4 to 0.9) for Southey et al. 56 For loss of protein expression of PMS2, data were available only from Southey et al. ,56 with the LR– estimated as 0.5 (95% CI 0.3 to 0.9). PPV and NPV estimates for MLH1 and MSH2 were largely consistent across the two studies. 54,56 For MSH6, NPV estimates were consistent across the two studies but PPV estimates were vastly different, with data from Barnetson et al. 54 resulting in a PPV of 16.7% (95% CI 3.6% to 41.4%) and data from Southey et al. 56 resulting in a PPV of 72.7% (95% CI 39.0% to 94.0%). Although the reason for this difference is not completely clear, in part it is likely to be because of the very low number of true-positive results (n = 3) for loss of expression of MSH6 in the study by Barnetson et al. 54 Again, only Southey et al. 56 provided data for PMS2, with the PPV estimated as 66.7% (95% CI 38.4% to 88.2%) and the NPV estimated as 81.8% (95% CI 67.3% to 91.8%).

Secondary IHC analyses for individual proteins were conducted in which unclassified variants were considered to be reference standard negatives. However, only one study61 provided sufficient data to conduct secondary analyses and, because this study was a reference standard-positive study, only sensitivity was estimated. These sensitivity estimates were very similar to those estimated from data in which unclassified variants were considered to be reference standard negatives.

Critical review and synthesis of information

We were not able to draw any conclusions on the effectiveness of screening for LS from the systematic review of end-to-end studies.

Mvundura et al.84

Mvundura et al. 84 produced a decision model of newly diagnosed CRC patients and their relatives in the USA. It compared IHC (with or without BRAF V600E testing), MSI testing, no testing and universal genetic testing. Relatives were identified through genetic testing. Although the study did include both MSI and IHC testing strategies, it did not include all of the interventions identified in the NICE scope. 88

Relatives with a confirmed mutation were offered colonoscopy every 1–2 years from the age of 20–25 years until the age of 79 years. LS-negative relatives and some of those who were LS positive and who declined the intensive LS surveillance were offered colonoscopy every 10 years from the age of 50 years.

The analysis did not include LS-related cancers other than CRC or the differences in CRC incidence between males and females. It also did not include all of the comparators identified in the NICE scope. 88

In their base case Mvundura et al. 84 reported only life-years gained (LYGs) as opposed to QALYs. All strategies with preliminary tests were found to be cost-effective compared with no testing. IHC followed by BRAF V600E testing was the optimal strategy, with an incremental cost-effectiveness ratio (ICER) of US$22,552 per LYG. To turn their analysis into a cost–utility analysis, the ICERs were scaled by a factor of 1.18 LYGs per QALY. This did not impact on the order of the strategies and increased the ICERs for all strategies. This approach required a number of assumptions to be made and therefore it is unlikely to reflect the true cost per QALY gained for each testing strategy.

The model results were sensitive to CRC risk in relatives, the number of relatives and adherence to surveillance.

Wang et al.86

Wang et al. 86 provided an update of the study by Ladabaum et al. ,85 looking at the cost–utility of strategies to identify LS in newly diagnosed CRC patients and their relatives in the USA. The difference between the two models was the inclusion of utilities in the update model.

Wang et al. 86 included strategies that started with clinical criteria (ACII and revised Bethesda or prediction models) as well as those that began with tumour-based testing (IHC and/or MSI). We do not focus on the strategies that began with clinical criteria as these are not considered as part of our analysis. As with the study by Mvundura et al. ,84 not all interventions identified by the NICE scope88 were included.

The model used a decision tree with Markov subtrees to model newly diagnosed CRC patients and their relatives. For people identified with a LS mutation or with an unconfirmed diagnosis of LS (plus their first-degree relatives), annual colonoscopy (from the age of 25 years) was offered. Women were offered gynaecological screening from the age of 35 years and prophylactic total abdominal hysterectomy and bilateral salpingo-oophorectomy (TAH-BSO) from the age of 40 years. When LS was not diagnosed, people were offered 10-yearly colonoscopy from the age of 50 years.

The strategy that used IHC (with or without BRAF V600E mutation testing) was found to be the most cost-effective strategy, with an ICER of US$59,719 per QALY gained. With the exception of MSI in combination with IHC (with or without BRAF V600E testing) and universal germ-line testing (both of which had ICERs of > US$100,000 compared with the next most cost-effective strategy), all other strategies were dominated or extendedly dominated.

The results were most sensitive to the age of relatives, the effectiveness of LS surveillance for CRC and the prevalence of LS.

Gallego et al.78

Gallego et al. 78 examined the cost-effectiveness and cost–utility of strategies to identify LS in CRC patients and their relatives. Details of the modelling are scarce as this study was reported only in abstract form. The main comparison appears to look at targeted sequencing compared with NGS. It was unclear whether this study used the same underlying model as that reported in the study by Gallego et al. ,81 but the author list suggests that the two studies are likely to be related. For relatives in whom LS was detected, CRC surveillance was offered, although, again, details are not provided. The reference strategy was a combination of IHC and MSI followed by targeted sequencing. Replacing targeted sequencing with NGS gave an ICER of US$71,000 per QALY gained and a strategy of NGS for all patients compared with the reference strategy gave an ICER of US$196,000 per QALY gained.

As with most other analyses, the most influential parameters included the number of relatives tested, the prevalence of LS in CRC and the cost of CRC surveillance in relatives.

Snowsill et al.4,79

Snowsill et al. 4,79 report the previous work by PenTAG, which assessed the cost-effectiveness of strategies to diagnose LS in newly diagnosed CRC patients aged < 50 years and their relatives. Snowsill et al. 4,79 presented a cost–utility analysis from the perspective of the NHS and Personal Social Services (PSS) and included both CRC and endometrial cancer as outcomes. They stated that, when possible, they adhered to the NICE reference case. 89 The model included a decision tree diagnostic section and an individual sampling model to calculate long-term outcomes and survival for each of the diagnoses from the diagnostic section. Probands and relatives who received a confirmed (mutation-positive) diagnosis or assumed LS diagnosis (probands and relatives who were offered but who declined germ-line testing and first-degree relatives of probands who were assumed to have LS) were offered colonoscopic surveillance, with women also offered prophylactic TAH-BSO at the age of 40 years.

In this analysis, all testing strategies had ICERs of < £10,000 per QALY gained. When compared incrementally, MSI followed by BRAF V600E testing was the most cost-effective strategy at a willingness-to-pay threshold of £20,000. Scenario analyses demonstrated that increasing the age of the population increased the ICERs for each strategy compared with no testing, although MSI plus BRAF V600E testing remained the most cost-effective strategy. The model appeared most sensitive to the disutilities for endometrial cancer and prophylactic TAH-BSO. Other sensitive parameters included the number of relatives, the prevalence of LS, the cost of colonoscopy complications, CRC incidence and the effectiveness of colonoscopy to prevent CRC.

Although we believe that this previous work was the first UK-based model to assess the cost–utility of strategies to diagnose LS and has been described by Ladabaum et al. 3 as the ‘most comprehensive decision model’, this model still does not fully answer the decision problem set out in this assessment. The strategies included in the study by Snowsill et al. 4,79 are not in line with the current scope,88 with several strategies including a combination of MSI and IHC testing, and methylation testing included only as a scenario analysis, in place of BRAF V600E testing. The parameters are also now several years old and were chosen for CRC patients aged < 50 years and so they are unlikely to represent the overall population and the current clinical experience.

Gallego et al.81

Gallego et al. 81 presented a cost–utility analysis of CRC patients and their relatives from a US perspective. They used a decision tree model with long-term outcome estimates based on the results presented by Mvundura et al. 84 As such, the concerns that we have over the cost–utility analysis by Mvundura et al. 84 are equally relevant here.

In this study the strategies were IHC followed by BRAF V600E testing prior to germ-line testing. The intervention strategies were NGS panels. The relevant intervention strategy for our analysis was panel 1, which looked at LS only. CRC surveillance was offered to both probands and relatives, although details of this surveillance were not provided. As the study by Mvundura et al. 84 was stated to be the source of the long-term outcomes, we assumed that surveillance was modelled as in this study.

Next-generation sequencing for LS compared with standard care gave an ICER of US$144,235 per QALY gained. Sensitivity analyses were not reported for this scenario, as it was not the base case.

Number of probands

To estimate the number of probands we used the same approach as in the study by Snowsill et al. ,4 that is, taking the number of probands from the most recent Office for National Statistics (ONS) cancer registration statistics for England. 21 The figures for the overall population and the age subgroups are provided in Table 26.

Number of relatives

The method for estimating the number of relatives was discussed in detail in Snowsill et al. 4 In summary, the number of relatives included in the model represents the possible group of identifiable relatives. For example, for probands in whom a LS mutation is identified, this is the possible number of relatives who both are contactable and may be identified through cascade testing.

The number of relatives in the study by Snowsill et al. 4 was set to five, based on previous data from Barrow et al. 90 (2.35 relatives per proband) and unpublished data provided by Ian M Frayling (Institute of Cancer and Genetics, University Hospital of Wales, Cardiff, UK, 18 December 2012, personal communication). Data recently collected from the Manchester Regional Lynch Syndrome Registry, as part of a PhD thesis investigating the diagnosis of LS,91 suggested an average of 9.95 relatives per index patient. As such, the base case number of relatives in the current model was increased slightly to six to better account for this new information. As in the study by Snowsill et al. ,4 this number was varied from 0 to 12 in univariate sensitivity analysis, to acknowledge the wide variation among the data sources. The number of relatives per proband was assumed not to alter with the age of the proband, given a paucity of evidence to demonstrate otherwise.

The Manchester Regional Lynch Syndrome Registry estimated an average of eight first-degree relatives per index patient,91 but this seems exceptionally high and so we have assumed that this figure actually represents the results of cascade testing to identify index patients. As such, our estimated number of first-degree relatives remained as 42% of all relatives (2.5 relatives per proband), which is based on a combination of published92,93 and unpublished (Ian M Frayling, personal communication) data, as reported in Snowsill et al. 4

The proportion of relatives who are male was taken from pooled analysis of unpublished data (Ian M Frayling, personal communication; Munaza Ahmed, Wessex Clinical Genetics Service, University Hospital Southampton NHS Foundation Trust, 23 January 2013, personal communication), giving a value of 38%, as used previously in the analysis reported in Snowsill et al. 4

Prevalence of Lynch syndrome

The prevalence of LS among CRC probands was estimated from the study by Hampel et al. ,92 as in the study by Snowsill et al. 4 The base-case value (2.8%) and the age subgroup values are presented in Table 27.

In the study by Snowsill et al. 4 it was important to subdivide the proportion of the population with LS by mutation, as several parameters relied on this information, including the sensitivity and specificity of MSI and IHC. In the current analysis, the test accuracy was assumed to apply for all mutations, as new test accuracy evidence is not available to predict the effect that these mutations may have. This is a result of restricting the test accuracy studies to those that answer the decision problem and not including studies with high-risk input populations. Mutation type does still affect some costs and therefore is still included in the model.

As in the study by Snowsill et al. ,4 the proportions for each LS mutation were taken from the supplementary evidence in the Evaluation of Genomic Applications in Practice and Prevention (EGAPP) review,42 which stated that, for all true LS-positive patients, 32% have a MLH1 mutation, 39% have a MSH2 mutation, 14% have a MSH6 mutation and 15% have a PMS2 mutation (Table 28). Reported family registry data can differ significantly from these values, particularly with respect to the proportion of PMS2 mutations identified91,94,95 (see Table 28). This may potentially be because of PMS2 testing having historically occurred less in current practice than in the trials on which the EGAPP review based its values. Therefore, if systematic testing were more common in UK practice, these values might change. We explore the possibility of the proportion of each mutation being closer in value to the reported registry data in a sensitivity analysis, using the Manchester Regional Lynch Syndrome Registry data as these are UK-based data. 91

For relatives of probands with LS, the proportion of relatives expected to test positive was estimated using a meta-analysis of studies (Table 29), as reported in Snowsill et al. ;4 this gave a value of 44%. This value falls below 50% for a number of reasons (Box 2).

Age on entry

In the base case there was no age threshold for CRC patients for screening for LS.

The age on entry for probands was based on the age distribution of CRC diagnoses. For probands without LS, the age distribution was estimated from cancer registration statistics in England from 2006 to 2014. 21,96–103 These statistics are grouped into 5-year age groups. It was assumed that registrations were uniform within these age groups.

For probands with LS, the age distribution was estimated from the parametric CRC incidence function (see Incidence rates for individuals with Lynch syndrome).

The cumulative registrations were used to estimate an empirical cumulative incidence function. The cumulative incidence function was truncated as appropriate for the different age subgroups. Figure 7 shows the age distribution of probands in the base case.

FIGURE 7.

Age distribution of simulated probands.

For relatives without LS, the age distribution was assumed to be equal to the age distribution of the general population, which was taken from the mid-2014 population estimates for England104 and mid-2014 population estimates of the very old (including centenarians) for the UK. 105

For relatives with LS, the age distribution was estimated by multiplying the population estimate for the general population by an estimate of the CRC mortality-free survival for individuals with LS (i.e. incorporating the raised incidence of CRC in relatives with LS).

As previously,4 the age distribution of relatives was right-truncated at 75 years as it is unlikely that any intervention would be offered to individuals aged > 75 years and was left-truncated at 18 years as few relatives are offered predictive testing before the age of 18 years. Figure 8 shows the age distribution of relatives in the base case.

FIGURE 8.

Age distribution of simulated relatives.

Diagnostic testing model

The section of the model that calculates diagnoses was built as a decision tree with no time component included. As in the study by Snowsill et al. ,4 this assumes that the diagnosis of LS in probands and relatives occurs instantaneously, although in reality this may take up to several months or years. As before, it was also assumed that probands’ treatment will not be influenced by a LS diagnosis, following the assumption that the results of LS testing will be unavailable prior to treatment. In most cases this treatment will be surgical resection with the possibility of chemotherapy or radiotherapy, depending on the stage of the cancer.

An overview of the diagnostic model is given in Figure 9. This is unchanged from that in Snowsill et al. 4

FIGURE 9.

Lynch syndrome diagnostic pathway. TN, true negative. Source: reproduced from figure 9 of Snowsill et al. 4 under the terms of the UK government’s non-commercial licence for public sector information. 32

Diagnostic strategies for probands

The NICE scope88 for this project specifies diagnostic strategies for probands that are significantly altered from those reported in Snowsill et al. 4

The 10 new strategies considered are:

1. No systematic testing to identify LS (all probands assumed not to have LS).

2. IHC four-panel test for MLH1, MSH2, MSH6 and PMS2 followed by genetic testing if IHC results are abnormal.

3. IHC four-panel test followed by BRAF V600E testing for abnormal MLH1 results. Genetic testing is carried out for any other (not MLH1) abnormal IHC results or after a negative BRAF V600E test.

4. IHC four-panel test followed by MLH1 promoter methylation testing for abnormal MLH1 results. Genetic testing is carried out for any other (not MLH1) abnormal IHC results or after a negative MLH1 promoter methylation test.

5. IHC four-panel test followed by BRAF V600E testing for abnormal MLH1 results. A negative BRAF V600E test is followed by MLH1 promoter methylation testing. Genetic testing is carried out for any other (not MLH1) abnormal IHC results or after a negative MLH1 promoter methylation test.

6. MSI test followed by genetic testing for MSI-H result.

7. MSI test followed by BRAF V600E testing for MSI-H result. Genetic testing is carried out after a negative BRAF V600E test.

8. MSI test followed by MLH1 promoter methylation testing for MSI-H result. Genetic testing is carried after a negative MLH1 promoter methylation test.

9. MSI test followed by BRAF V600E testing for MSI-H result. A negative BRAF V600E test is followed by MLH1 promoter methylation testing. Genetic testing is carried after a negative MLH1 promoter methylation test.

10. Universal genetic testing (i.e. as the first and only test for all probands).

These strategies are presented in Figures 10–13.

FIGURE 10.

Proband diagnostic strategy: no testing.

FIGURE 11.

Immunohistochemistry-based diagnostic strategies for probands. GC, genetic counselling; GT, genetic testing.

FIGURE 12.

Microsatellite instability-based diagnostic strategies for probands. GC, genetic counselling; GT, genetic testing.

FIGURE 13.

Universal genetic testing strategy for probands. GC, genetic counselling; GT, genetic testing.

Strategies 5 and 9 include both BRAF V600E and MLH1 promoter methylation testing. These are performed in sequence with either test able to rule out LS, for example a patient with BRAF V600E would not receive further testing. The ordering of these tests was informed by discussion at the NICE scoping workshop (11 January 2016).

In clinical practice it is expected that a number of patients with stage II CRC will undergo MSI or IHC analysis to inform treatment options. To highlight this, the model includes costs for MSI or IHC testing for all stage II CRC patients (≈27% of probands) in strategy 1 (no testing), but assumes that this does not lead to any further testing for LS.

For the purposes of the base case it was assumed that a MSI result corresponds to MSI-H. This assumption was explored in a scenario analysis, in which it was assumed that a MSI result corresponds to MSI-L; this is detailed further in Model parameters.

A certain proportion of probands in the MSI strategies were assumed to receive IHC to help interpret their genetic test results. For example, a variant of uncertain significance may be identified in MSH2 and IHC was conducted to identify whether the tumour cells were MSH2 deficient (indicating that the variant is likely to be pathogenic). Clinical opinion appears to be that this proportion can be quite low [Ottie O’Brien, Northern Molecular Genetics Service, personal communication, 7 April 2016 (value of < 5%); Samantha Butler, West Midlands Regional Genetics Laboratory, personal communication, 13 July 2016 (value of 10%)], so the value was set to 5% in the base case. The impact of this parameter was explored in univariate sensitivity analysis, with the proportion varied from 0% to 10%.

The potential results of the testing strategies for patients are as reported previously:4 LS mutation positive, LS assumed and LS negative. LS mutation positive occurs when a proband receives a positive genetic test result and LS negative occurs when a proband is ruled out by one of the tests in the strategy.

In the study by Snowsill et al. ,4 the outcome of LS assumed could occur when genetic testing either was uninformative or was simply not carried out (the proband declined testing). A negative genetic test result was assumed to be uninformative for probands, meaning that, although the test did not detect LS, it did not rule it out. The Amsterdam II criteria were used as an additional test for those probands who were LS mutation negative and those who declined genetic testing to decide whether they were LS assumed. In this update to the model, any use of the Amsterdam II criteria was removed, in line with the focus of the decision problem, which does not include clinical criteria. It was also assumed in the current model that only probands who decline testing can become LS assumed, and this was set to 10% for all patients [lower than the average proportion of probands who were LS assumed after declining testing in the 2014 model (21%)]. We adjusted this number for each age subgroup using the three age limit scenarios from Snowsill et al. 4 (21% for probands aged < 50 years, 17% for probands aged < 60 years, 13% for probands aged < 70 years) and set the value for the ≥ 70 years subgroup to a significantly lower value (5%) to reflect that, in higher age groups, few people are likely to be diagnosed with LS without genetic testing.

Unlike in the study by Snowsill et al. ,4 in which testing for PMS2 was modelled only for probands who were negative for MLH1, MSH2 and MSH6 but who had a family history that was indicative of LS, in the current model it was assumed that all patients who accept genetic testing will receive testing for all four known genes (MLH1, MSH2, MSH6 and PMS2). The exception to this was that probands who follow strategies that use IHC followed by either BRAF V600E or MLH1 promoter methylation testing receive only MLH1 and PMS2 germ-line testing. Mutations related to EPCAM were assumed to be identified through testing for MSH2. This is believed to be in line with current clinical practice; when clinical practice differs from this, the model will overestimate costs associated with tumour-based testing strategies.

As in the study by Snowsill et al. ,4 only a proportion of patients diagnosed as LS positive or LS assumed accept an offer of LS surveillance.

Diagnostic strategies for relatives

The diagnostic strategies for relatives are unchanged from those in the study by Snowsill et al. ;4 these strategies are summarised in Figures 14–16. As with probands, relatives could decline any testing or LS surveillance offered to them.

FIGURE 14.

Diagnostic strategy for relatives of probands diagnosed as LS mutation positive. Source: reproduced from figure 12 of Snowsill et al. 4 under the terms of the UK government’s non-commercial licence for public sector information. 32

FIGURE 15.

Diagnostic strategy for relatives of probands diagnosed as LS assumed. FDR, first-degree relative. Source: reproduced from figure 13 of Snowsill et al. 4 under the terms of the UK government’s non-commercial licence for public sector information. 32

FIGURE 16.

Diagnostic strategy for relatives of probands diagnosed as LS mutation negative. Source: reproduced from figure 14 of Snowsill et al. 4 under the terms of the UK government’s non-commercial licence for public sector information. 32

Long-term outcomes model

As described previously,4 long-term outcomes were modelled for all probands and relatives regardless of the diagnostic path that they follow. An individual patient sampling model was used to simulate 240,000 patients, distributed across 24 groups, representing all combinations of the variables shown in Table 30. An individual patient sampling model was justified because no patient interactions were modelled but there were too many patient states for a cohort model.

Mean long-term outcomes were then estimated for each of the 24 groups. These were then used to estimate the long-term outcomes for each of the diagnostic strategies in the decision tree.

Patients were simulated for 1 year at a time. Each patient starts each year in a particular state, which determines the events that can occur during that year. The events in turn determine the costs incurred and the state of the patient in the next year if the patient is still alive. The hazard rate for events (except elective events such as surveillance colonoscopy) was assumed to be constant during each year but could change between years.

Life-years and QALYs were calculated based on a patient’s state at the beginning of the year and any events occurring during the year, for example if a patient starts the year without cancer and develops cancer after 3 months, then the patient accrues 3 months of life with a non-cancer utility and 9 months of life with the utility decrement from cancer (assuming that the patient does die within the year).

Table 31 shows the different events included in the model. Mortality events are competing – no other events can occur after the patient dies.

Figure 17 presents a model diagram for the long-term outcomes model, indicating the mortality events and cancer incidence events, as well as prophylactic H-BSO. Gynaecological surveillance is indicated to reduce the risk of endometrial cancer mortality, but this is only if the individual is receiving surveillance prior to cancer incidence.

FIGURE 17.

Simplified model diagram for the long-term outcomes model. EC, endometrial cancer.

Patient’s gynaecological state (women only)

A patient’s gynaecological state encapsulates what risk-reducing measures she has employed and whether or not she has had endometrial cancer.

Although endometrial cancer can be staged and there are some survival estimates according to stage,113 the staging system has been changed recently and most patients are diagnosed in the early stage of cancer. Therefore, the stage of endometrial cancer is not tracked in the model; rather, the time since diagnosis is tracked.

It was assumed that patients will not get more than one primary endometrial cancer as they will undergo a hysterectomy, which is very effective at preventing endometrial cancer when used prophylactically. 26

Risk-reducing measures available to patients are gynaecological surveillance and prophylactic hysterectomy (and bilateral salpingo-oophorectomy). Patients can start and stop surveillance but prophylactic hysterectomy is irreversible and surveillance cannot be performed in patients after hysterectomy (Figure 18).

FIGURE 18.

Gynaecological state model diagram.

Gynaecological state on entry

All simulated individuals were assumed to start without endometrial cancer.

Probands and relatives who have been diagnosed with LS may be offered surveillance or prophylactic H-BSO depending on their age.

Table 38 shows the assumed distribution of risk-reducing strategies according to age at diagnosis, based on audit data from the Northern Genetics Service (Lorraine Cowley, Principal Genetic Counsellor, Newcastle upon Tyne Hospitals NHS Foundation Trust, 20 November 2012, personal communication).

TABLE 38 - Gynaecological cancer risk reduction for women with LS on entry

Age (years) at LS diagnosis	No risk reduction	Surveillance	Prophylactic H-BSO
0–34	1.000	0.000	0.000
35–44	0.200	0.600	0.200
45–59	0.167	0.458	0.375
60–69	0.000	0.143	0.857
≥ 70	0.143	0.000	0.857

Colorectal cancer

The simulation model includes four events relating to CRC:

• index CRC incidence

• index CRC mortality

• mCRC incidence

• mCRC mortality.

The incidence events transform the patient’s bowel state (e.g. index CRC incidence transforms the bowel state from ‘no CRC’ to ‘index CRC’ with a particular stage). The mortality events result in the patient dying with no further events occurring.

Figure 19 shows the 21 different bowel cancer histories included in the model.

FIGURE 19.

Colorectal cancer and mCRC incidence model diagram.

The probability that a CRC incidence event occurs within a year is dependent on the incidence rate, whereas the probability that a CRC mortality event occurs is dependent on the survival function.

Colorectal cancer incidence

Colorectal cancer incidence rates in the model are dependent on the following patient characteristics:

• age

• sex

• whether or not the patient has had a previous CRC

• time since first CRC

• LS status

• risk-reducing measures, that is, regular colonoscopies and aspirin, as described in Colorectal surgery pathways and Aspirin chemoprevention respectively.

Different annual incidence rates are provided for the eight combinations of sex, previous cancer (yes/no) and LS status and then risk-reducing measures are incorporated as HRs, which have a simple multiplicative effect on the incidence rate.

Incidence rates for individuals without Lynch syndrome

The incidence rates for males and females without previous cancer without LS were estimated from pooled registration statistics for CRC in England between 2006 and 2014 inclusive21,96–103 and the estimated population in the midpoints of those years. 114 Following the methodology adopted by the ONS,100 we calculated the age-specific rate of CRC incidence by dividing the number of CRC registrations within a time period by an estimate of the person-years lived during that period. Incidence figures were pooled across 5 years to achieve a large sample size but not further back than 2006 as such data may not reflect more recent developments in cancer detection and registration.

Cancer registration statistics are not provided for each year of age but for age groups, generally of 5 years. We assumed that within each of these age groups the incidence rate would remain constant. The resulting cumulative risk of CRC for individuals without LS is shown in Figure 20.

FIGURE 20.

Cumulative risk of CRC for individuals without LS. Note: this does not account for non-CRC mortality. Source: reproduced from figure 22 of Snowsill et al. 4 under the terms of the UK government’s non-commercial licence for public sector information. 32

We estimated the incidence of mCRC (i.e. the incidence in individuals who had had a previous CRC) for individuals without LS by adjusting the incidence of the first CRC by a HR of 1.4 for the first 3 years after the first CRC and 1.3 for the following 7 years, as in the study by Mulder et al. 115 Mulder et al. 115 studied 10,283 Dutch patients with CRC undergoing standard follow-up. After 10 years no additional hazard was applied.

Incidence rates for individuals with Lynch syndrome

Previously,4 we conducted a literature review to identify studies from which the age-dependent incidence rates for individuals with LS could be estimated.

Subsequent reviews2,16,116 have not identified any additional studies.

Møller et al. 5 have subsequently published estimates of the cancer risk for individuals with LS while undergoing colonoscopic surveillance; this does not address the need here (i.e. for the cancer risk in the absence of interventions).

In the absence of new evidence to consider, the model includes an incidence rate for CRC based on the study by Bonadona et al. 18

A logistic model for cumulative risk was fitted to data from Bonadona et al. 18 using the following parameterisation and ordinary least-squares regression:

⁽⁸⁾ F ( x ) = β 0 1 + exp ⁡ [ − β 1 ( x − β 2 ) ] .

The 95% CIs for the cumulative risk to age 70 years from the study by Bonadona et al. 18 were used in sensitivity analyses (by varying β₀ appropriately). Table 39 shows the parameters used in the base-case and sensitivity analyses and Figures 21 and 22 show graphically the fit to the data from Bonadona et al. 18

TABLE 39 - Logistic model parameters for CRC incidence in individuals with LS

F, women with LS; M, men with LS.

Source: reproduced from table 60 of Snowsill et al. 4 under the terms of the UK government’s non-commercial licence for public sector information. 32

Parameter	Base case	Sensitivity analyses (95% CI)
β0	M 0.464, F 0.435	M 0.303 to 0.715; F 0.265 to 0.697
β1	M 0.107, F 0.108
β2	M 55.5, F 61.3

FIGURE 21.

Colorectal cancer incidence for men with LS. Source: reproduced from figure 23 of Snowsill et al. 4 under the terms of the UK government’s non-commercial licence for public sector information. 32

FIGURE 22.

Colorectal cancer incidence for women with LS. Source: reproduced from figure 24 of Snowsill et al. 4 under the terms of the UK government’s non-commercial licence for public sector information. 32

Stage on diagnosis

The CRC stage on diagnosis is an important predictor of survival. Historically, the modified Dukes’ stage (A–D) has been used for cancer staging, but it is more common now to refer to stages by Roman numerals (I–IV).

We assumed that the stage on diagnosis would be independent of age, sex, LS status and whether it was the first (index) CRC or a mCRC (Fajobi et al. 117 conclude that stages for mCRC are no worse than those for index CRC and there was no consensus on whether they might be better). We also assumed that stage on diagnosis of mCRC was independent of the stage of the index CRC and the CRC site. CRC stage was assumed to depend only on whether the person was undergoing LS surveillance colonoscopies.

The stage distributions are provided in Model parameters, Stage on diagnosis.

Colorectal cancer site

As stated in Patient’s bowel state we model two sections of the bowel, the colon and the rectum. We grouped rectosigmoid cancer (International Classification of Diseases, 10th Revision code C19) into rectal cancer. The site of incident CRCs was dependent on sex, whether or not the person had LS and any previous surgery (Table 40).

TABLE 40 - Probability that incident CRC is situated in the colon

NA, not applicable.

NA as zero CRC incidence following proctocolectomy.

Source: partially based on table 52 of Snowsill et al. 4 under the terms of the UK government’s non-commercial licence for public sector information. 32

Previous surgery	With LS	Without LS
		Men	Women
None	0.94	0.63	0.72
Segmental resection	0.94	0.63	0.72
Subtotal colectomy	0.00	0.00	0.00
Anterior resection	1.00	1.00	1.00
Proctocolectomy	NAa	NAa	NAa

It was assumed that all CRCs are colon cancers following anterior resection (AR) and that all CRCs are rectal cancers following subtotal colectomy. If there is no previous surgery or a previous segmental resection the probability of the CRC being situated in the colon for a person with LS was estimated as 0.94 based on the study by Dinh et al. 107 For males and females without LS the probability of colon cancer was estimated from ONS cancer registration statistics. 100

Colorectal cancer survival

We assumed that mortality from CRC depends on the following:

• CRC stage at diagnosis

• years since diagnosis

• age at diagnosis

• LS status (see Lynch syndrome colorectal cancer survival).

We did not consider the effect of the following on mortality from CRC:

• patient’s sex

• site of CRC

• surgery for CRC.

The baseline annual rate of mortality from CRC was derived from data provided by the (UK) National Cancer Intelligence Network109 by extracting 1-, 2-, 3-, 4- and 5-year relative survival from survival curves and assuming constant rates of mortality within each year (Table 41 and Figure 23). It was assumed that the mortality rate for 4–5 years since diagnosis also applied after 5 years (Table 42).

TABLE 41 - Relative survival of patients with CRC by Dukes’ stage across all ages

Source: National Cancer Intelligence Network. 109 Reproduced from table 63 of Snowsill et al. 4 under the terms of the UK government’s non-commercial licence for public sector information. 32

Years since diagnosis	Dukes’ stage
	A	B	C	D
1	0.969	0.917	0.815	0.380
2	0.965	0.872	0.681	0.193
3	0.957	0.831	0.583	0.116
4	0.945	0.799	0.522	0.083
5	0.932	0.770	0.477	0.066

FIGURE 23.

Colorectal cancer survival in the model. Source: reproduced from figure 28 of Snowsill et al. 4 under the terms of the UK government’s non-commercial licence for public sector information. 32

TABLE 42 - Mortality rate from CRC (per 100,000 person years) by Dukes’ stage

Source: reproduced from table 64 of Snowsill et al. 4 under the terms of the UK government’s non-commercial licence for public sector information. 32

Years since diagnosis	Dukes’ stage, n
	A	B	C	D
0–1	3102	8709	20,460	96,729
1–2	419	5000	17,971	67,733
2–3	843	4761	15,465	51,116
3–4	1279	4000	11,060	32,857
> 4	1400	3667	9068	23,375

The assumption that the mortality rate after 5 years is equal to the mortality rate for 4–5 years is likely to be a slight overestimate of CRC mortality (Table 43). The result of this would be a slight improvement in the cost-effectiveness of strategies with a high yield of LS mutations.

TABLE 43 - One-, 5- and 10-year survival for CRC

Source: Cancer Research UK, https://web.archive.org/web/20130302050315/http://www.cancerresearchuk.org/cancer-info/cancerstats/types/bowel/survival/, accessed August 2017. 1 Copyright © 2013, Cancer Research UK. Adapted from table 65 of Snowsill et al. 4 under the terms of the UK government’s non-commercial licence for public sector information. 32

Years since diagnosis	Male colon cancer	Female colon cancer	Male rectal cancer	Female rectal cancer	Model CRC
1	0.730	0.722	0.788	0.788	0.757
5	0.544	0.551	0.546	0.575	0.530
10	0.501	0.508	0.473	0.521	0.421

The HRs for CRC mortality by age compared with CRC mortality across all ages were estimated using net survival statistics from the ONS118 and are shown in Table 44. Details of the calculations are given in appendix 6 of Snowsill et al. 4

TABLE 44 - Hazard ratios for CRC mortality by age at diagnosis compared with CRC mortality across all ages

Source: reproduced from table 66 of Snowsill et al. 4 under the terms of the UK government’s non-commercial licence for public sector information. 32

Age group (years)	HR for CRC mortality
	First year	Following 4 years	Thereafter
< 70	0.599	0.972	1
70–79	0.956	0.966	1
≥ 80	1.797	1.116	1

Metachronous colorectal cancer survival

As previously,4 mortality from mCRC was modelled by adding the mortality rates for both the index CRC and the mCRC as calculated above, assuming that mortality from the metachronous cancer would be no different from mortality from the index cancer for the same Dukes’ stage (as assumed by, for example, Mvundura et al. 84 and Dinh et al. 107). The same approach was used independent of the LS status of the patient. As the mortality rates are dependent on the time since diagnosis in the model, we kept track of time since diagnosis of the index cancer and the metachronous cancer.

Lynch syndrome colorectal cancer survival

As previously,4 the model assumed improved survival for individuals with LS and local CRC compared with individuals with sporadic CRC. A HR of 0.57 was applied for Dukes’ A and Dukes’ B CRC if the simulated individual had LS, based on the study by Lin et al. 119 Survival for individuals with Dukes’ C and Dukes’ D CRC was equal for patients with and without LS, based on the study by Barnetson et al. 54

Endometrial cancer

The lifetime risk of endometrial cancer in the general population is one in 41. 120 The lifetime risk in women with LS (in the absence of risk-reducing measures) is around 35%. 18 As previously,4 endometrial cancer was modelled only for women with LS.

Endometrial cancer incidence

As previously,4 and as for CRC, the incidence rates for endometrial cancer in women with LS were estimated from the study by Bonadona et al. 18

A piecewise constant hazard of endometrial cancer was used per decade of life to achieve the cumulative risk profile observed in the study by Bonadona et al. 18 and it was assumed that the incidence of endometrial cancer would be zero after age 80 years. Figure 24 shows the cumulative risk used in the model as well as the data from Bonadona et al. 18 for reference.

FIGURE 24.

Endometrial cancer incidence. Gynaecological surveillance was not assumed to affect the incidence of endometrial cancer (see Gynaecological surveillance).

Endometrial cancer survival

Survival from uterine cancer in England and Wales has recently been estimated by the London School of Hygiene and Tropical Medicine and published by Cancer Research UK (Figure 25). 121

FIGURE 25.

Net survival for uterine cancer in England and Wales. Source: based on a graphic created by Cancer Research UK. 121

Endometrial cancer cases make up the vast majority of uterine cancer cases, so uterine cancer cases are expected to closely approximate survival of endometrial cancer.

The survival curve was used to estimate a piecewise constant rate of mortality for each year since diagnosis. After 10 years the rate of mortality is zero.

Colorectal surgery pathways

As previously,4 colorectal surgical pathways were based on published guidelines46 with input from clinical experts.

Patients undergo surgical management if they are diagnosed with CRC and the cancer is deemed to be operable (this includes surgery when intent is palliative rather than curative). We made the simplifying assumption that all patients diagnosed with CRC undergo surgical management (> 75% of patients in the National Bowel Cancer Audit108 were treated surgically). In each case, surgery would remove the bowel portion affected by the cancer and, in some cases, additional portions, depending on previous surgery and whether LS had been diagnosed (Figure 26). Our clinical expert advice was that, in general, surgery for patients without LS tends to be conservative, without a risk-reducing element (Mr Ian Daniels, Consultant Colorectal Surgeon, Royal Devon and Exeter Hospital, 24 October 2012, personal communication.)

FIGURE 26.

Surgical management pathways for CRC. APER, abdominoperineal excision of the rectum. Source: reproduced from figure 19 of Snowsill et al. 4 under the terms of the UK government’s non-commercial licence for public sector information32 (adapted from Maeda et al. 106).

Clinical guidelines indicate that there is a place for more aggressive surgery, with a risk-reducing element, for patients known to have LS on CRC diagnosis; in particular, Cairns et al. 46 stated that, ‘For patients with proximal tumours, colectomy and ileorectal anastomosis is most relevant’ (p. 678). Input from our clinical expert suggested that this particular guidance would rarely be followed as it was based on a low category of evidence (evidence obtained from expert committee reports or opinions or clinical experiences of respected authorities) and colonoscopic surveillance is deemed effective enough to negate the need for aggressive surgery (Mr Ian Daniels, personal communication). To resolve this disagreement we included a parameter in the model that defines the probability that more aggressive surgery would be used for LS patients, which could be varied from 0 (ignore guidelines; surgical treatment not affected by Lynch diagnosis) to 1 (full adherence to guidelines; aggressive surgery always used). Previous analyses by Snowsill et al. 4 demonstrated that altering this parameter had a minimal impact on cost-effectiveness and therefore sensitivity analyses on this parameter have not been repeated.

When surgery removes the rectum because of cancer in the rectum there are two common operations: AR, which preserves the anus, and abdominoperineal excision of the rectum (APER), which results in permanent stoma. We grouped these operations together and assumed that they are both as effective at preventing metachronous rectal cancer. Some patients would require a permanent stoma, which would affect health-related quality of life (HRQoL) and costs. Rather than modelling this on an individual patient basis we assumed an average effect across all patients.

Any subsequent surgery depends on the location of the CRC, the nature of the previous surgery and whether the patient has been diagnosed with LS (unless the parameter described above is 0) (Tables 45 and 46).

TABLE 45 - Probability of different surgery types for colon cancer patients not diagnosed with LS

NA, not applicable.

In total, 8850 colon cancer patients underwent right hemicolectomy (n = 6627), transverse colectomy (n = 86), left hemicolectomy (n = 978) or sigmoid colectomy (n = 1159). 108

In total, 325 colon cancer patients underwent total or subtotal colectomy. 108

NA because subtotal colectomy and proctocolectomy are assumed to completely eliminate the risk of colon cancer.

Source: reproduced from table 48 of Snowsill et al. 4 under the terms of the UK government’s non-commercial licence for public sector information. 32

Previous surgery	Segmental resection (%)	Subtotal colectomy (%)	AR (%)	Proctocolectomy (%)	Source
None	96a	4b	0	0	National Bowel Cancer Audit108
Segmental resection	0	100	0	0	Assumption
Subtotal colectomy	NAc	NAc	NAc	NAc	Assumption
AR	0	0	0	100	Assumption
Proctocolectomy	NAc	NAc	NAc	NAc	Assumption

TABLE 46 - Probability of different surgery types for rectal cancer patients not diagnosed with LS

NA, not applicable.

In total, 4341 rectal cancer patients underwent AR (n = 2890), APER (n = 1139) or the Hartmann procedure (n = 312). 108

In total, 82 rectal cancer patients underwent total or subtotal colectomy. 108

NA because AR and proctocolectomy are assumed to completely eliminate the risk of rectal cancer.

Source: reproduced from table 49 of Snowsill et al. 4 under the terms of the UK government’s non-commercial licence for public sector information. 32

Previous surgery	Segmental resection (%)	Subtotal colectomy (%)	AR (%)	Proctocolectomy (%)	Source
None	0	0	98a	2b	National Bowel Cancer Audit108
Segmental resection	0	0	0	100	Assumption
Subtotal colectomy	0	0	0	100	Assumption
AR	NAc	NAc	NAc	NAc	Assumption
Proctocolectomy	NAc	NAc	NAc	NAc	Assumption

Surgery distributions for CRC patients diagnosed with LS were adjusted by the parameter representing the probability of aggressive surgery. If this probability is denoted as p, then for colon cancer patients:

⁽⁹⁾ $$\begin{gathered} Pr ( SEG | LS diagnosed ) = ( 1 − p ) Pr ( SEG | LS not diagnosed ) Pr ( SUB | LS diagnosed ) \\ = 1 − Pr ( SEG | LS diagnosed ) ) , \end{gathered}$$

where SEG is segmental resection and SUB is subtotal colectomy. For rectal cancer patients:

⁽¹⁰⁾ $$\begin{gathered} Pr ( AR | LS diagnosed ) = ( 1 − p ) Pr ( AR | LS not diagnosed ) ⁡ Pr ( IPAA | LS diagnosed ) \\ = 1 − Pr ( AR | LS diagnosed ) ) , \end{gathered}$$

where IPAA is ileal pouch anal anastomosis.

In the base case, this parameter was set to 0 for the following reasons:

• the clinical guidelines [e.g. the BSG/Association of Coloproctology of Great Britain and Ireland (ACPGBI) guidelines46] are not based on high-quality evidence

• the model does not incorporate any utility decrement for more aggressive surgery, even though function can be affected (see Colorectal cancer treatment)

• the model does not incorporate excess surgical mortality from more aggressive surgery.

Treatment effectiveness and extrapolation

Surveillance

Colonoscopy

Colonoscopy is assumed to lead to improved health outcomes by reducing the incidence of CRC (as adenomas that could have become adenocarcinomas are identified and removed) and by improving the cancer stage distribution on diagnosis (i.e. catching the cancer earlier), which leads to improved survival.

The model assumed that the majority of patients diagnosed with LS will be offered and will accept surveillance colonoscopies.

Data from 591 individuals from the Manchester Familial Colorectal Cancer Registry, as reported in Barrow,91 were used to estimate the acceptance rates for relatives diagnosed with LS. We assumed that these rates would be the same for probands as for relatives, as in the study by Snowsill et al. 4 the acceptance of surveillance in probands was the same as or higher than that in relatives. Previously, these estimates were taken from the study by Ladabaum et al. ,85 which was a study of a US population and which therefore does not reflect the acceptance rate in the UK population. The current values appear to suggest higher acceptance rates than previously modelled. However, as these values are taken from only one UK institution and acceptance of surveillance has been previously shown to be an influential parameter with regard to cost-effectiveness, we examined the impact of these rates in sensitivity analyses.

Table 49 shows the assumed acceptance rates used in the model.

TABLE 49 - Initial rates of acceptance of LS surveillance for CRC

Patient characteristic	Initial acceptance of surveillance91 (%)	Value in Snowsill et al.4 (based on Ladabaum et al.85) (%)
Proband tested LS mutation positive	97	80
Proband LS assumed	70	70
Relative tested LS mutation positive	97	80
Relative LS assumed	70	50

Colorectal cancer incidence

Previously,4 event times were extracted from figure 1 of Järvinen et al. 124 and used in a Cox proportional hazards regression to estimate the effectiveness of colonoscopy in reducing CRC incidence. The resulting HR estimate was 0.387 (95% CI 0.169 to 0.885).

Subsequently, other evidence has also been collected relating to this question.

The American Gastroenterological Association published a technical review on the diagnosis and management of LS in 2015. 3 This review identified five observational studies (including that by Järvinen et al. 124), which allowed estimation of the effectiveness of colonoscopy in reducing CRC incidence. The pooled effect estimate (odds ratio) was 0.23 (95% CI 0.13 to 0.41).

Møller et al. 5 reported on the cancer risks in a prospectively identified population of LS carriers, in whom surveillance is widespread. This study included 1942 LS carriers from across Europe (and Australia), with 195 of these from the UK. The authors found high rates of CRC in spite of regular surveillance.

Figure 27 illustrates the contradictory nature of the evidence. ‘Bonadona’ and ‘Møller’ are the estimated incidence rates in the studies, whereas ‘Bonadona + Järvinen’ is a counterfactual incidence rate profile obtained by applying the HR of 0.387 to the ‘Bonadona’ data. If all of these studies were conducted without bias and in similar populations and with enough statistical power, one would expect the ‘Møller’ data to closely align with the ‘Bonadona + Järvinen’ data rather than the ‘Bonadona’ data.

FIGURE 27.

Comparison of CRC incidence rates in the absence of surveillance (Bonadona) and with surveillance (Møller). Notes: assumes that 41% have MLH1 mutation, 44% have MSH2 mutations and 16% have MSH6 mutations (figures do not sum to 100% because of rounding). Sources: Bonadona et al. 18 and Møller et al. 5

There are a number of potential reasons why the results of Møller et al. 5 might not confirm the effectiveness of colonoscopy observed in the study by Järvinen et al. :124

• the limited statistical power in both the Bonadona et al. 18 study and the Møller et al. 5 study (i.e. random chance)

• the naive comparison of the results from the Bonadona et al. 18 study and the Møller et al. 5 study, with no adjustment for differences in populations (other than for gene distribution and age)

• the biased effect estimate in the study by Järvinen et al. 124

It is difficult to assess the similarity of the Bonadona et al. 18 and Møller et al. 5 patient populations. The population in Bonadona et al. 18 seems to have a more even balance between men and women whereas the population in Møller et al. 5 included somewhat more women. It is possible that there are differences in the proportions of patients with clearly pathogenic mutations – Bonadona et al. 18 included 7% (35/537) of families with variants of uncertain significance, whereas Møller et al. 5 included only patients with mutations judged pathogenic by their reporting centre (although 31% of patients had mutations not reported on the Leiden Open Variant Database by October 2015).

The two main sources of potential bias in the Järvinen et al. 124 study are confounding because of:

• self-selection of the intervention group (i.e. participants chose whether or not to receive the intervention) – this would be expected to exaggerate the effectiveness estimate as participants choosing to receive surveillance are more likely to engage in other healthy behaviours and to have other characteristics predictive of good outcomes

• treatment switching (i.e. participants who initially declined surveillance who later opted into surveillance) – this would be expected to attenuate the effectiveness estimate.

Furthermore, the results from Järvinen et al. 124 may not generalise to current surveillance in the NHS because of developments in technology or differences in service delivery and behaviour, which could result in a different distribution of screening intervals.

However, in the absence of compelling alternative effectiveness estimates, we continued to use the HR of 0.387 from the study by Järvinen et al. 124

In a worst-case scenario analysis it was assumed that surveillance does not reduce the incidence of CRC, that is, a HR of 1 was used.

Stage on diagnosis

As previously,4 the stage on diagnosis was assumed to be dependent only on whether an individual has been offered and accepted LS surveillance colonoscopies. Whether the individual has a LS mutation, their previous history of cancer and their sex and age were not modelled as affecting the stage distribution.

The stage on diagnosis for individuals not accepting surveillance was estimated from national data from England. In England in 2012, 11% of CRCs were not staged (or the stage was not recorded). 131 The National Cancer Intelligence Network used multiple imputation methods, based on patient and cancer characteristics that were recorded, to estimate the stage distribution for the unstaged patients (Table 50). When this stage distribution is combined with the data for patients whose cancers were staged, the resulting distribution for stages I–IV is 17.6 : 27.0 : 29.5 : 25.9 respectively. Previously,4 a distribution of 16.4 : 31.7 : 27.1 : 24.8 for stages I–IV, respectively, was estimated from 2009/10 data by excluding patients whose cancer stage was not recorded. The effect of the change is to reduce the number of patients with stage II CRC and increase the number of patients with other stages.

TABLE 50 - Stage distribution of CRCs in England

Source: calculated from tables 1 and 4 from National Cancer Intelligence Network. 131

Stage	Observed data for all patients		Imputed stage distribution of ‘unknown’		Combined data
	n	%	n	%	n	%
I	5255	15.5	734	2.16	5989	17.6
II	8402	24.7	768	2.26	9170	27.0
III	9258	27.2	778	2.29	10,036	29.5
IV	7351	21.6	1465	4.31	8816	25.9
Unknown	3745	11.0
Total	34,011	100.0	3745	11.02	34,011	100.0

For individuals accepting LS surveillance colonoscopies, a stage distribution of 68.6 : 10.5 : 12.8 : 8.1 for stages I–IV, respectively, was used, as previously,4 based on data from Mecklin et al. 132

Gynaecological surveillance

A literature review was conducted previously of the effectiveness of surveillance for endometrial and ovarian cancer. 4 Conceptually, surveillance was expected to reduce the incidence of gynaecological cancers by the detection of premalignancies, which are either individually removed or which prompt hysterectomy. Malignancies identified during surveillance were also expected to have a more favourable stage profile.

No experimental studies (e.g. RCTs) were identified, but three non-experimental studies were identified. 133–135 Two of these studies134,135 considered cohorts of women with LS eligible for gynaecological surveillance and compared patients receiving surveillance with patients refusing or not receiving surveillance during the study (Figures 28 and 29). These study designs could provide estimates of the effectiveness of surveillance in reducing the incidence of gynaecological cancer; however, they were both at high risk of bias because of potential confounding factors between the groups and neither could provide satisfactory effect size estimates because of the limited control group sizes and the limited number of events. The limited number of events also means that these studies are not informative for the stage distribution of gynaecological cancers.

FIGURE 28.

Patient flow diagram for the study by Dove-Edwin et al. 134 EC, endometrial cancer.

FIGURE 29.

Patient flow diagram for the study by Järvinen et al. 135 EC, endometrial cancer; OC, ovarian cancer.

The study by Renkonen-Sinisalo et al. ,133 in contrast, considered patients before and after institution of a surveillance programme. The outcomes of 385 women with LS mutations were compared with the outcomes of 83 women with LS mutations who were affected by endometrial cancer before surveillance was instituted (Figure 30).

FIGURE 30.

Patient flow diagram for the study by Renkonen-Sinisalo et al. 133 EC, endometrial cancer.

This study design cannot be used to estimate the effect of surveillance on cancer incidence, but it can be used to estimate the impact of surveillance on the stage of cancer at diagnosis (Figure 31).

FIGURE 31.

International Federation of Gynecology and Obstetrics (FIGO) stage distribution of endometrial cancers in the study by Renkonen-Sinisalo et al. 133 (a) Surveillance; and (b) control.

An ordered logistic regression was used to estimate the impact of surveillance. The regression coefficient for surveillance was not statistically significant (z = –0.45, p = 0.651) but was suggestive that surveillance could improve the stage distribution. Predicted stage distributions from the regression are shown in Figure 32.

FIGURE 32.

Predicted stage distribution based on the results of Renkonen-Sinisalo et al. 133 (a) Surveillance; and (b) control.

These stage distributions were not used directly but were instead used to estimate a HR for survival from endometrial cancer.

Lewin et al. 113 estimated the 5-year survival from endometrial cancer according to stage at diagnosis (Table 51). The weighted average 5-year survival for patients in the surveillance and control groups was estimated as 83.4% and 81.8%, respectively, from which a HR of 0.898 was derived.

TABLE 51 - Five-year survival from endometrial cancer according to stage at diagnosis

FIGO, International Federation of Gynecology and Obstetrics.

Source: calculated from results presented by Lewin et al. 113

FIGO (1998) stage	5-year survival (%)
I	87.8
II	76.2
III	55.3
IV	21.1

Because of the significant uncertainty in this HR, a scenario analysis is included in which there is no survival benefit (i.e. no benefit at all) from gynaecological surveillance.

It should be noted that the women in the study by Renkonen-Sinisalo et al. 133 were offered surveillance with 2- or 3-year intervals, whereas current European guidelines propose annual surveillance,125 so it is possible that a study of annual surveillance could find greater effectiveness than what is modelled here.