Risk scores to guide referral decisions for people with suspected ovarian cancer in secondary care: a systematic review and cost-effectiveness analysis

The ARCHITECT human epididymis protein 4 assay (Abbott Diagnostics)

The ARCHITECT HE4 assay (Abbott Diagnostics, Abbott Park, IL, USA) is a chemiluminescent microparticle immunoassay for the quantitative determination of HE4 levels in human serum. The assay is designed for use on an immunoassay analyser, specifically the ARCHITECT i2000SR or the ARCHITECT i1000SR analysers. Additional materials required to run the assay are the ARCHITECT HE4 assay software file, ARCHITECT HE4 calibrators, ARCHITECT HE4 controls, ARCHITECT multiassay manual diluent, ARCHITECT pre-trigger solution, ARCHITECT trigger solution, ARCHITECT wash buffer, ARCHITECT reaction vessels, ARCHITECT sample cups, ARCHITECT septum and ARCHITECT replacement caps.

The results of the assay are intended to be used in conjunction with the ARCHITECT CA125 II assay, as an aid in estimating the risk of epithelial ovarian cancer in women presenting with a pelvic mass who will undergo surgical intervention. The company recommends that the HE4 assay results are used in the calculation of the ROMA scores, using the following cut-off values for ROMA scores, based on obtaining a specificity of 75%:

• in premenopausal patients, a ROMA value of ≥ 7.4% indicates a high risk of finding epithelial ovarian cancer

• in premenopausal patients, a ROMA value of < 7.4% indicates a low risk of finding epithelial ovarian cancer

• in postmenopausal patients, a ROMA value of ≥ 25.3% indicates a high risk of finding epithelial ovarian cancer

• in postmenopausal patients, a ROMA value of < 25.3% indicates a low risk of finding epithelial ovarian cancer.

These results must be interpreted in conjunction with other methods and clinical data (e.g. symptoms and medical history), in accordance with standard clinical management guidelines. The company states that additional testing should be done if the HE4 results are inconsistent with the clinical evidence.

LUMIPULSE G human epididymis protein 4 (Fujirebio Diagnostics)

The LUMIPULSE G HE4 (Fujirebio Diagnostics, Gothenburg, Sweden) is a chemiluminescent enzyme immunoassay (CEIA) for the quantitative measurement of HE4 levels in human serum. The assay is designed for use on the LUMIPULSE G system (either the LUMIPULSE G1200 or the LUMIPULSE G600 immunoassay analysers). Samples are run using immunoreaction cartridges, which contain reagents and into which samples are added. Further materials required for the assay are LUMIPULSE G HE4 calibrators, LUMIPULSE G substrate solution, LUMIPULSE G wash solution, LUMIPULSE G specimen diluent I, sampling tips for the LUMIPULSE system, soda lime for the LUMIPULSE system and LUMIPULSE G dilution cartridges.

The assay is intended for use in conjunction with CA125 levels (measured using the LUMIPULSE G CA125 II assay) as an aid in estimating the risk of epithelial ovarian cancer in premenopausal and postmenopausal women presenting with a pelvic mass who will undergo surgical intervention.

The company recommends that the HE4 results are used in the calculation of the ROMA scores, and suggests the following cut-off values for the ROMA scores, based on obtaining a specificity of 75%:

• in premenopausal patients, a ROMA value of ≥ 13.1% indicates a high risk of finding epithelial ovarian cancer

• in premenopausal patients, a ROMA value of < 13.1% indicates a low risk of finding epithelial ovarian cancer

• in postmenopausal patients, a ROMA value of ≥ 27.7% indicates a high risk of finding epithelial ovarian cancer

• in postmenopausal patients, a ROMA value of < 27.7% indicates a low risk of finding epithelial ovarian cancer.

Results should be interpreted in conjunction with further methods and clinical data (e.g. clinical findings, age, family history and imaging results), in accordance with standard clinical management guidelines.

A further HE4 assay is also available from Fujirebio Diagnostics: the HE4 enzyme immunoassay (EIA), a manual, enzyme immunometric assay for the quantitative determination of HE4 in human serum. Clinical experts commented that manual kits would be unlikely to be used in routine practice in the NHS; therefore, this assay has not been included in the scope of this assessment.

Elecsys human epididymis protein 4 immunoassay (Roche Diagnostics)

The Elecsys HE4 (Roche Diagnostics, Rotkreuz, Switzerland) is an immunoassay that uses Roche Diagnostics’ electrochemiluminescence detection technology to quantify HE4 levels. The assay uses anti-HE4 mouse monoclonal antibodies to capture HE4 in a serum sample and label it with a ruthenium complex. The application of a voltage to the samples then induces chemiluminescent emissions, which are measured by a photomultiplier.

The assay is designed for use on an immunoassay analyser, specifically the following analysers: modular analytics E170, cobas e 411, cobas e 601/e 602 and cobas e 801. Additional materials required for the HE4 assay are the HE4 CalSet (Roche Diagnostics, Rotkreuz, Switzerland), PreciControl HE4 (Roche Diagnostics, Rotkreuz, Switzerland) and Diluent MultiAssay (Roche Diagnostics, Rotkreuz, Switzerland). Further materials are also required for the general running of analysers, such as wash and cleaning solutions and disposable consumables.

The assay is intended to be used in conjunction with the Elecsys CA125 II assay as an aid in estimating the risk of epithelial ovarian cancer in premenopausal and postmenopausal people with a pelvic mass. The company recommends that the HE4 and CA125 assay results are used in the calculation of the ROMA scores. The company suggests the following cut-off values for the ROMA scores, based on obtaining a specificity of 75%:

• in premenopausal patients, a ROMA value of ≥ 11.4% indicates a high risk of finding epithelial ovarian cancer

• in premenopausal patients, a ROMA value of < 11.4% indicates a low risk of finding epithelial ovarian cancer

• in postmenopausal patients, a ROMA value of ≥ 29.9% indicates a high risk of finding epithelial ovarian cancer

• in postmenopausal patients, a ROMA value of < 29.9% indicates a low risk of finding epithelial ovarian cancer.

The company states that additional testing should be done if the HE4 results are inconsistent with the clinical evidence.

Details of Risk of Ovarian Malignancy Algorithm studies

Sixteen diagnostic cohort studies,81–83,86,89,90,94–99,101–104 reported in 24 publications,56,81–83,85–104 provided data on the diagnostic performance of the ROMA score for identifying women who have an adnexal mass and are at a high risk of developing ovarian cancer. Nine studies81–83,86,90,96,99,101,103 used a ROMA score based on Abbott Diagnostics’ ARCHITECT tumour marker assays, of which six81,82,86,96,99,103 evaluated a decision threshold for the ROMA score that was consistent with the manufacturer’s recommendations. None of the included studies used the Fujirebio Diagnostics’ LUMIPULSE G automated CEIA system. For information, two studies94,98 that used a ROMA score based on manual Fujirebio Diagnostics’ tumour marker EIAs (see Appendix 5, Tables 41 and 42) were included, both using the manufacturer’s recommended decision threshold for the ROMA score; however, it should be noted that the manual assays are not specified interventions for this assessment. Finally, five studies89,95,97,102,104 used a ROMA score based on Roche Diagnostics’ Elecsys tumour marker assays, all of which used the manufacturer’s recommended decision threshold for the ROMA score.

None of the ROMA score studies that used Abbott Diagnostics’ ARCHITECT tumour marker assays was conducted in the UK; three studies81,83,86 were conducted in European countries, four82,90,96,99 were conducted in Asia, one101 was conducted in the USA and one103 was conducted in Oman. None of the ROMA score studies that used Roche Diagnostics’ Elecsys tumour marker assays was conducted in the UK, and only one97 was conducted in a European country. Three89,93,95 of the remaining studies were conducted in Asia and one104 was conducted in the USA.

This assessment is primarily concerned with providing a comparison between the RMI 1,78 used with a decision threshold of 250 (current standard practice in the NHS1), and the specified alternative risk-scoring methods (see Chapter 2, Intervention technologies). No studies were identified that reported a direct comparison (both tests used to assess the same patient cohort) between the ROMA score and the RMI 1, used with a decision threshold of 250. Five studies reported direct comparisons between the ROMA score and the RMI 1, used with a decision threshold of 200; three studies used Abbott Diagnostics’ ARCHITECT tumour marker assays;83,99,103 one study used Roche Diagnostics’ Elecsys assays;89 and one study used Fujirebio Diagnostics’ manual EIAs. 98 The following sections report all available data from direct comparison studies, as well as non-comparative data on the accuracy of the ROMA score, when decision thresholds that were consistent with the manufacturers’ recommendations were used. Additional accuracy data for alternative decision thresholds are reported in Appendix 5, Table 37.

The target condition for this assessment is ovarian cancer, including conditions covered by the NICE clinical guideline CG1221 (i.e. epithelial ovarian cancer, fallopian tube carcinoma, primary peritoneal carcinoma and borderline ovarian cancer). All studies in this section included women with one or more adnexal mass. The definition of reference standard positive ‘ovarian cancer’ varied between studies, with borderline tumours being most frequently classified as positive or excluded from analyses. In addition, some studies included patients with non-ovarian primary cancers/metastases to the ovary97,98,103 and germ cell tumours. 103 When the target condition was described as ‘all ovarian malignancy’, those women whose postoperative histological diagnosis was identified as non-ovarian primary were excluded from the estimates of test performance. Conversely, when the target condition was described as ‘all malignant tumours’, women with a non-ovarian primary were not excluded and were classified as being disease positive; this could potentially include women with any tumour on the ovaries that has metastasised from another primary [e.g. colorectal cancer (CRC)], and/or women with an adnexal/pelvic mass that turns out to be non-ovarian (not clearly specified by the included studies). Full details of the final histopathological diagnoses of study women who had a malignant mass are reported in Appendix 4, Table 36.

Accuracy of the Risk of Ovarian Malignancy Algorithm score using Abbott Diagnostics’ ARCHITECT tumour marker assays

Three83,99,103 of the nine81–83,86,90,96,99,101,103 ROMA score studies that used Abbott Diagnostics’ ARCHITECT tumour marker assays, reported a direct comparison of the ROMA score with the RMI 1. Only one study included all participants the analysis, regardless of their final histopathological diagnosis (target condition: all malignant tumours, including borderline) and this study used different thresholds from those recommended by the manufacturer (13.1% in premenopausal women and 27.7% in postmenopausal women, as opposed to the manufacturer’s recommendation of 7.4% and 25.3%). 103 One study was a retrospective study, which excluded women with histopathological diagnoses other than epithelial ovarian cancer. 99 A second study excluded from the analysis nine women (1%) with non-epithelial ovarian cancer, 69 women (6%) with non-ovarian cancers and 252 women (21%) with borderline tumours;83 the distribution of positive and negative ROMA score results in these women was not reported.

The sensitivity estimate for the ROMA score was highest (96.4%, 95% CI 93.6% to 98.2%) when the analyses excluded women with borderline tumours and those with malignancies other than epithelial ovarian cancer, and lowest (75.0%, 95% CI 60.4% to 86.4%) when all women were included in the analysis, regardless of their final histopathological diagnosis (Table 7). Conversely, the specificity estimate for the ROMA score was highest (87.9%, 95% CI 81.9% to 92.4%) in the study that included all participants,103 and lowest (53.3%, 95% CI 50.0% to 56.7%) when the analyses excluded women with borderline tumours and those with malignancies other than epithelial ovarian cancer (see Table 7). When women with borderline tumours and/or those with malignancies other than epithelial ovarian cancer were excluded from the analyses, the sensitivity estimates for the ROMA score were not significantly different from those for the RMI 1 (threshold 200), whereas the specificity estimates were significantly lower (see Table 7). In contrast, the study that included all participants in the analysis reported similar sensitivity and specificity estimates for the ROMA score and the RMI 1, with a sensitivity of 75% (95% CI 60.4% to 86.4%) versus 77.1% (95% CI 62.7% to 88.0%), and a specificity of 87.9% (95% CI 81.9% to 92.4%) versus 81.8% (95% CI 75.1% to 87.4%), respectively. 103 This study also reported lower sensitivity and higher specificity estimates, for both the ROMA score and the RMI 1, in premenopausal women than those in postmenopausal women (see Table 7).

One study reported test performance estimates calculated both with and without the inclusion of participants with borderline tumours. 99 Although the number of participants involved was small, these data indicated that around half of the FN risk scores were accounted for by women with borderline tumours, 3 out of 6 (50%) using the ROMA score and 7 out of 13 (54%) using the RMI 1 (threshold of 200). 99 Approximately 13% (17/128) of the women in this study had borderline tumours, whereas 39% (50/128) had malignant tumours [i.e. a higher proportion of women with borderline tumours had a negative ROMA score – 17.6% (3/17) – than was the case for women with malignant tumours – 6% (3/50)]. 99 A similar pattern was observed for the RMI 1; the proportion of women with borderline tumours who had a negative RMI 1 was approximately 41% (7/17), compared with 12% (6/50) for those with malignant ovarian tumours. 99

One additional study reported performance estimates for the ROMA score, excluding women with borderline tumours and those with non-ovarian malignancies, without a comparison with the RMI 1. 82 When data from this study were combined with the ROMA data from the two similar comparative accuracy studies,83,99 the summary estimates of sensitivity did not change significantly [95.1%, 95% CI 92.4% to 97.1%, based on three studies (Table 8), vs. 96.4%, 95% CI 93.6% to 98.2%, based on two studies; see Table 7]. The summary estimate of specificity, based on all three studies (62.5%, 95% CI 59.7% to 65.3%; see Table 8), was higher than that derived from the two comparative accuracy studies alone (53.3%, 95% CI 50% to 56.7%; see Table 7). There were no additional studies that evaluated the performance of the ROMA score alone, and included all participants in the analysis (target condition: all malignant tumours, including borderline).

One study82 assessed the variation in the performance of the ROMA score with different stages of epithelial ovarian cancer (see Table 8). The sensitivity estimate was highest (92.1%, 95% CI 78.6% to 98.3%) when the target condition was stage III/IV epithelial ovarian cancer, and women with stage I/II and borderline disease were excluded from the analysis. 82 There was a small, but non-significant, fall in sensitivity (82.6%, 95% CI 61.2% to 95.0%) when the target condition was stage I/II epithelial ovarian cancer and women with borderline and higher-stage disease were excluded from the analysis. 82 When the target condition was borderline epithelial tumours and all women with higher-stage disease were excluded from the analysis, the sensitivity estimate was significantly lower (56.3%, 95% CI 29.9% to 80.2%). 82 These data are consistent with the observation that the proportion of women with a negative ROMA score is higher among those women with borderline disease than among those with ovarian malignancies, and may also be higher among those with lower-stage epithelial ovarian cancer than those with higher-stage epithelial ovarian cancer.

Two studies81,96 reported accuracy data for ovarian malignancy, but without clarifying whether or not the definition of malignancy included borderline tumours (see Appendix 5, Table 43). Accuracy data for thresholds other than those recommended by the manufacturer (7.4% in premenopausal women and 25.3% in postmenopausal women) are reported in Appendix 5, Table 37; no study reported accuracy data at an alternative threshold for the inclusive target condition of all malignant tumours, including borderline, and no alternative threshold offered a clear performance advantage.

Accuracy of the Risk of Ovarian Malignancy Algorithm score using Fujirebio Diagnostics’ tumour marker assays

None of the included studies used the Fujirebio Diagnostics’ LUMIPULSE G automated CEIA system; hence, there were no studies of the ROMA score, using Fujirebio Diagnostics’ assays, that met the inclusion criteria for this assessment. Two studies94,98 that evaluated a ROMA score based on manual Fujirebio Diagnostics’ tumour marker EIAs have been included in this report. These studies are included for information only. Both of these studies included all women in the analysis, regardless of their final histopathological diagnosis (target condition: all malignant tumours, including borderline). One study98 reported a direct comparison of the ROMA score with the RMI 1 (threshold of 200). The results of these studies are provided in Appendix 5, Tables 41 and 42.

Accuracy of the Risk of Ovarian Malignancy Algorithm score using Roche Diagnostics’ tumour marker assays

Only one89 of the five89,95,97,102,104 ROMA score studies, which used Roche Diagnostics’ Elecsys tumour marker assays, reported a direct comparison of the ROMA score with the RMI 1 (threshold of 200). This study classified women found to have borderline ovarian tumours as disease negative and included women whose final histopathological diagnoses were epithelial ovarian cancer, non-epithelial ovarian cancer and metastases from non-ovarian primaries (target condition: all malignant tumours). 89 This study may be considered to be more applicable to clinical practice if it is considered to be preferable to manage women with borderline tumours in non-specialist settings. In these women, the sensitivity estimate for the ROMA score appeared to be slightly higher than that for the RMI 1 (83.8%, 95% CI 73.4% to 91.3%, vs. 78.4%, 95% CI 67.3% to 87.1%), and the specificity estimate for the ROMA score appeared to be slightly lower than that for the RMI 1 (68.8%, 95% CI 61.6% to 75.4%, vs. 79.6%, 95% CI, 73.1% to 85.1%), but neither difference was statistically significant. 89 A similar pattern was observed when data were stratified by menopausal status (Table 9). The same study also reported test performance data, whereby eight women (3%) with non-epithelial ovarian cancer and non-ovarian primaries were excluded from the analysis. This exclusion did not significantly change the test performance estimates for either the ROMA score or the RMI 1 (see Table 9). Although the number involved was small, it should be noted that women with malignancies other than epithelial ovarian cancer accounted for four (50%) of the FN results, using the ROMA score, and for three (37.5%) of the results, using the RMI 1. 89

The aforementioned comparative accuracy study89 also assessed the variation in the performance of the ROMA score with different stages of epithelial ovarian cancer (see Table 9). The sensitivity estimate was highest for both the ROMA score (97.2%, 95% CI 95.5% to 99.9%) and the RMI 1 score (88.9%, 95% CI 73.9% to 96.9%), for which the target condition was stages II–IV epithelial ovarian cancer; women with stage I disease were excluded from the analysis. 89 As with the ROMA score using Abbott Diagnostics’ ARCHITECT tumour marker assay, sensitivity estimates were lower for both the ROMA score (76.7%, 95% CI 57.7% to 90.1%) and the RMI 1 score (70.0%, 95% CI 50.6% to 85.3%), for which the target condition was stage I epithelial ovarian cancer; women with higher-stage disease were excluded from the analysis. 89 This indicates that the proportion of women with a negative ROMA score may be higher among those with lower-stage epithelial ovarian cancer than those with higher-stage epithelial ovarian cancer.

Two97,104 of the four95,97,102,104 additional studies that evaluated the performance of the ROMA score but did not provide a comparison with the RMI 1 score included all study participants in the analysis regardless of their final histopathological diagnoses [target condition: all malignant tumours including borderline (Table 10)]. The summary estimate of the sensitivity of the ROMA score (79.1%, 95% CI 74.2% to 83.5%), derived from these two studies, was lower than that reported by the comparative accuracy study89 described earlier, in which women with borderline tumours were classified as disease negative, and the summary specificity estimate was also lower (79.1%, 95% CI 76.3% to 81.6%), but these differences were not statistically significant. Two studies95,97 reported test performance data for the ROMA score, in which women found to have borderline tumours and those with non-ovarian primaries were excluded from the analyses. The sensitivity estimates derived from these two studies were very different (see Table 10) hence, no summary estimates were calculated. One of these studies97 reported test performance estimates calculated both with and without the inclusion of women with borderline tumours and those with non-ovarian primaries. As with the ROMA score, using Abbott Diagnostics’ ARCHITECT tumour marker assays, these data indicated that women with borderline tumours and those with non-ovarian primaries accounted for a high proportion (12/14; 86%) of the FN risk scores observed. 97

One study102 provided performance estimates for the ROMA score, using Roche Diagnostics’ Elecsys tumour marker assays, for the target condition ‘ovarian malignancy’, when it was not clear whether or not the definition of malignancy included borderline tumours (see Appendix 5, Table 44). Accuracy data for thresholds other than those recommended by the manufacturer (11.4% in premenopausal women and 29.9% in postmenopausal women) are reported in Appendix 5, Table 37; no study reported accuracy data at an alternative threshold for the inclusive target condition of all malignant tumours including borderline, and no alternative threshold offered a clear performance advantage.

Between-assay comparisons

No study assessed variation in the performance of the ROMA score with the use of different manufacturers’ tumour marker assays. However, between-study comparisons indicate that, when all study participants were included in the analyses regardless of final histopathological diagnosis (target condition: all malignant tumours including borderline), the estimates of sensitivity did not differ significantly between the two manufacturers’ assays for which data were available (Figure 3). The sensitivity estimate for the ROMA score using Abbott Diagnostics’ ARCHITECT tumour marker assay was 75.0% (95% CI 60.4% to 86.4%), derived from one study,103 compared with 79.1% (95% CI 74.2% to 83.5%) using Roche Diagnostics’ Elecsys tumour marker assay, derived from two studies. 97,104 However, the specificity estimate for Abbott Diagnostics’ ARCHITECT tumour marker assay (87.9%, 95% CI 81.9% to 92.4%) was higher than that for Roche Diagnostics’ Elecsys tumour marker assay (79.1%, 95% CI 76.3% to 81.6%). There were no studies of the ROMA score using Fujirebio Diagnostics’ tumour marker assays that met the inclusion criteria for this assessment. There were insufficient data to compare the performance of the ROMA score with the use of different manufacturers’ tumour marker assays for detecting different stages of disease.

FIGURE 3.

Comparison of the accuracy of the ROMA score using Abbott Diagnostics’ ARCHITECT vs. the ROMA score using Roche Diagnostics’ Elecsys (target condition: all malignant tumours including borderline). (a) Sensitivity estimate; and (b) specificity estimate.

Details of the Assessment of Different NEoplasias in the adneXa studies

Six published studies17,42–46 and one unpublished interim report (Frances Nixon, personal communication) provided data on the diagnostic performance of the ADNEX scores at different thresholds. All studies reported accuracy data for the validated 10% decision threshold to identify women with an adnexal mass who were at a high risk of developing ovarian cancer and used a version of the ADNEX model that included a measurement of CA125 level. Four of the six published studies did not report any details of the experience of those performing the ultrasound examinations. 42–45 One study46 reported that ultrasound examinations were performed by European Federation of Societies for Ultrasound in Medicine and Biology (EFSUMB) level 2 ultrasound examiners (non-consultant gynaecology specialist, gynaecology trainee doctors and gynaecology sonographers), and the remaining study17 used EFSUMB level 2/3 practitioners with 8–20 years’ experience in gynaecological sonography. (Confidential information has been removed.)

This section reports accuracy data for only the 10% threshold. Three studies17,43,46 provided accuracy data for additional thresholds and these are reported in Appendix 5, Table 38. All studies in this section were conducted in Europe; one study45 was conducted solely in the UK, and two were multicentre studies17,46 that included UK participants.

The target condition for this assessment is ovarian cancer, including conditions covered by the NICE clinical guideline CG1221 (i.e. epithelial ovarian cancer, fallopian tube carcinoma, primary peritoneal carcinoma and borderline ovarian cancer). All studies in this section include women with one or more adnexal mass, and all but one study45 included borderline tumours in their definition of malignancy; the study that did not was reported only as a conference abstract and it was not clear whether or not any borderline tumours were included (see Appendix 5, Table 45). Three published studies17,44,46 and the unpublished interim report (Frances Nixon, personal communication) included participants with ‘other malignancies’, metastases from non-ovarian sites and ‘non-ovarian cancers’. When the target condition was described as ‘all ovarian malignancy’, those participants whose postoperative histological diagnosis identified a non-ovarian primary were excluded from the estimates of test performance. Conversely, when the target condition was described as ‘all malignant tumours’, participants with a non-ovarian primary were not excluded and were classified as disease positive; this could potentially include participants with any tumour on the ovaries that has metastasised from another primary (e.g. CRC) and/or participants with an adnexal/pelvic mass that turns out to be non-ovarian (not clearly specified by the included studies). Full details of the final histopathological diagnoses of study participants who had a malignant mass are reported in Appendix 4, Table 36.

Accuracy of the Assessment of Different NEoplasias in the adneXa model for determining high risk of ovarian cancer

Three published studies17,44,46 and the unpublished interim report (Frances Nixon, personal communication) included all participants in the analysis, regardless of their final histopathological diagnosis (target condition: all malignant tumours including borderline). The summary estimate of sensitivity derived from these studies was 96.3% (95% CI 95.3% to 97.1%) and the summary estimate of specificity was 69.1% (95% CI 67.4% to 70.8%; Table 11). These estimates did not differ significantly from those calculated from only those studies of the ADNEX model that reported a direct comparison with the RMI 1 score (at a threshold of 200 or 250; see Table 14). Two further studies,42,43 reporting three data sets, excluded women with histopathological diagnoses other than primary ovarian cancer. The summary estimate of sensitivity (94%, 95% CI 88.6% to 97.4%) derived from these studies did not differ significantly from that derived from the studies that included all participants in their analyses. However, the summary estimate of specificity (77.6%, 95% CI 73.6% to 81.2%) was higher. One study,42 which reported results from two separate cohorts (Spain and Poland), also reported accuracy data stratified by menopausal status. Menopausal status did not significantly affect sensitivity; however, the specificity estimate was significantly higher in premenopausal women than in postmenopausal women (see Table 11).

Accuracy data for thresholds other than the 10% validated threshold (1%, 3%, 5%, 15%, 20% and 30%) are reported in Appendix 5, Table 38. As might be expected, sensitivity estimates increase and specificity estimates decrease with decreasing threshold.

Details of the International Ovarian Tumour Analysis group’s simple ultrasound rules studies

Seventeen published studies44,47–52,55,58–65,67 and the unpublished interim report (Frances Nixon, personal communication) provided data on the diagnostic performance of the IOTA group’s simple ultrasound rules, for the identification of women with an adnexal mass who are at a high risk of developing ovarian cancer. The majority (11/17) of the published studies44,48–50,52,58,60,62–64,66 were conducted in Europe; three of the studies were conducted in the UK. 60,62,66 Two further studies61,65 were worldwide, with multinational studies including UK participants. Two of the remaining studies were conducted in Thailand47,51 and one was conducted in Brazil. 55 One study did not report sufficient detail to determine the geographic location. 53 (Confidential information has been removed.)

Three published studies50,61,65 were clearly conducted by the IOTA study core group, using data from various phases of the IOTA study; only one report was included for each phase of the IOTA study. Phase 5 of the IOTA study is ongoing and an interim report was supplied to this assessment (confidential information has been removed) (Frances Nixon, personal communication).

Ten published studies,44,48–50,52,55,58,61,62,65 as well as the unpublished interim report (Frances Nixon, personal communication), included all participants in the analysis; participants with inconclusive IOTA group’s simple ultrasound rules assessments were either assumed to have malignant tumours or classified by subjective assessment of ultrasound images. This section reports data for studies in which all participants were included in the analysis. Six further studies47,51,53,63,64,66 excluded participants with inconclusive IOTA group’s simple ultrasound rules assessments from their analyses. The results of these studies are provided in Appendix 5, Table 39. One study60 did not report sufficient information to determine how participants with inconclusive IOTA group’s simple ultrasound rules assessments were handled.

Accuracy of the International Ovarian Tumour Analysis group’s simple ultrasound rules for determining a high risk of developing ovarian cancer

All studies in this section included all participants in their analyses, regardless of their final histopathological diagnosis (target condition: all malignant tumours including borderline). Eight published studies44,48–50,52,55,62,65 and the unpublished interim report (Frances Nixon, personal communication) provided accuracy data for the IOTA group’s simple ultrasound rules, whereby women with inconclusive assessments were assumed to have malignant tumours. The summary estimate of sensitivity derived from these studies was 94.2% (95% CI 93.3% to 95.1%) and the summary estimate of specificity was 76.1% (95% CI 74.9% to 77.3%). These estimates did not differ significantly from those calculated from only those studies of the IOTA group’s simple ultrasound rules, in which participants with inconclusive assessments were assumed to have malignant tumours, which reported a direct comparison with the RMI 1 score (at a threshold of 200 or 250; see Tables 14 and 15). Four studies44,49,50,62 of these studies reported accuracy data stratified by menopausal status. Menopausal status did not significantly affect sensitivity; however, the specificity estimate was significantly higher in premenopausal women than in postmenopausal women (Table 12).

Seven studies44,49,50,52,58,62,65 provided accuracy data for the IOTA group’s simple ultrasound rules, whereby participants with inconclusive assessments were classified by an expert subjective assessment. In this analysis, only studies in which the subjective assessment was done by experts or by level 2/3 examiners as per the EFSUMB classification system have been included. The summary estimates of sensitivity and specificity derived from these studies were 88.4% (95% CI 86.9% to 89.8%) and 92.5% (95% CI 91.6% to 93.4%), respectively (see Table 12). One of these studies49 also assessed the effect of the training level of examiners on the diagnostic performance of the IOTA group’s simple ultrasound rules and found no significant differences in test performance between EFSUMB level 2 examiners (see Table 12) and EFSUMB level 1 examiners (Table 13). However, it should be noted that all examiners received 1 half-day of practical training in the IOTA group’s Simple Rules before the study.

Five of these studies44,49,50,58,62 reported accuracy data stratified by menopausal status. Menopausal status did not significantly affect sensitivity; however, the specificity estimate was significantly higher in premenopausal women than in postmenopausal women (see Table 12). One study58 (see Appendix 5, Table 39) also assessed whether or not the addition of biomarkers to the IOTA group’s simple ultrasound rules could improve the diagnostic performance. When a positive index test was defined as a malignant classification by the IOTA group’s simple ultrasound rules (with subjective assessment of inconclusives) and a ROMA score of > 11.4% out of 29.9%, the sensitivity and specificity estimates were 90.5% (95% CI 82.1% to 95.8%) and 80.1% (95% CI 75.2% to 84.4%), respectively. When a positive index test was defined as a malignant classification by the IOTA group’s simple ultrasound rules (with subjective assessment of inconclusives) and a HE4 level of ≥ 70 out of 140, the sensitivity and specificity estimates were 86.9% (95% CI 77.8% to 93.3%) and 86.3% (95% CI 82% to 90%), respectively. Neither the addition of the ROMA score nor the addition of HE4 level alone significantly affected estimates of test performance. Finally, when a positive index test was defined as a malignant classification by the IOTA group’s simple ultrasound rules (with subjective assessment of inconclusives) and a CA125 level of ≥ 35, the sensitivity estimate was similar to that for the IOTA group’s simple ultrasound rules (90.5%, 95% CI 82.1% to 95.8%); however, the specificity estimate was significantly lower (68.1%, 95% CI 62.5% to 73.3%).

Comparison of the different methods of operationalising the IOTA group’s simple ultrasound rules (i.e. ‘inconclusive results treated as malignant’ vs. ‘inconclusive results were classified by an expert’) indicates that sensitivity estimates were significantly higher when inconclusive results were treated as malignant, whereas specificity was significantly higher when patients with inconclusive results were classified by an expert. Thus, as might be expected, applying the assumption that all patients with an inconclusive result have a malignant tumour is likely to result in fewer patients with ovarian cancer being missed (FNs), whereas expert reassessment of inconclusive results is likely to result in fewer unnecessary referrals (FPs).

Comparisons between the International Ovarian Tumour Analysis group’s Simple Rules, the Assessment of Different NEoplasias in the adneXa and the Risk of Malignancy Index 1

This assessment is primarily concerned with providing a comparison between the RMI 1,78 used with a decision threshold of 250 (the current standard practice in the NHS1) and the specified alternative risk-scoring methods (see Chapter 2, Intervention technologies). Our searches did not identify any studies that reported a direct comparison (both tests were used to assess the same patient cohort) between the ADNEX model or the IOTA group’s simple ultrasound rules and the RMI 1, used with a decision threshold of 250. One published study44 and the unpublished interim report (Frances Nixon, personal communication) reported direct comparisons between the ADNEX model at the 10% threshold, the IOTA group’s simple ultrasound rules (whereby patients with an inconclusive assessment were assumed to have malignant tumours) and the RMI 1, used with a decision threshold of 200 (Table 14). Both of these studies included all participants in the analysis, regardless of their final histopathological diagnosis (target condition: all malignant tumours including borderline). The summary estimates of sensitivity derived from these two studies were slightly higher for the ADNEX model (96%, 95% CI 94.5% to 97.1%) than for the IOTA group’s simple ultrasound rules (92.8%, 95% CI 90.9% to 94.3%). The summary estimates of specificity were similar (67%, 95% CI 64.2% to 69.6%, and 71.6%, 95% CI 68.9% to 74.1%) for the ADNEX model and the ultrasound Simple Rules, respectively. The summary estimate of sensitivity for the RMI 1 at a decision threshold of 200 (66%, 95% CI 62.9% to 69%) was significantly lower than that for both the ADNEX model estimate and the IOTA group’s simple ultrasound rules estimate. Conversely, the specificity estimate for the RMI 1 at a decision threshold of 200 was significantly higher (89%, 95% CI 87% to 90.7%) than that for both the ADNEX model estimate and the IOTA group’s simple ultrasound rules estimate (Figure 4). The unpublished interim report (Frances Nixon, personal communication) also reported direct comparisons between the ADNEX model at the 10% threshold, the IOTA group’s simple ultrasound rules (whereby patients with an inconclusive assessment were assumed to have malignant tumours) and the RMI 1, used with a decision threshold of 250. The comparative accuracy estimates at this threshold did not differ from those at 200 (see Table 14).

FIGURE 4.

Comparison of the accuracy of the ADNEX model, the IOTA group’s simple ultrasound rules and the RMI 1. (a) Sensitivity estimate and; (b) specificity estimate.

Only the published study44 reported accuracy data stratified by menopausal status. In premenopausal women, the ADNEX model at the 10% threshold and the IOTA group’s simple ultrasound rules had similar sensitivities of 100% (95% CI 86% to 100%) and 94% (95% CI 77% to 99%), respectively, in comparison with the overall population. The specificities were significantly lower at 71% (95% CI 61% to 80%) and 76% (95% CI 66% to 84%), respectively. In postmenopausal women, the ADNEX model at the 10% threshold and the IOTA group’s simple ultrasound rules had similar sensitivities of 98% (95% CI 91% to 100%) and 93% (95% CI 85% to 97%), respectively, in comparison with the overall population. The specificities of 54% (95% CI 44% to 63%) and 61% (95% CI 52% to 70%), respectively, were significantly lower than those of the overall population. The RMI 1, using a decision threshold of 200, had a significantly lower sensitivity of 42% (95% CI 25% to 61%) and a significantly higher specificity of 94% (95% CI 86% to 97%) in premenopausal women. Conversely, the sensitivity estimate was higher at 82% (95% CI 72% to 89%) and the specificity estimate was lower at 66% (95% CI 56% to 74%) in postmenopausal women (but this was not significant).

Four published studies44,48,50,62 and the unpublished interim report (Frances Nixon, personal communication) reported direct comparisons between the IOTA group’s simple ultrasound rules (whereby patients with an inconclusive assessment were assumed to have malignant tumours) and the RMI 1 used with a decision threshold of 200 (Table 15). All of these studies included all participants in the analysis, regardless of their final histopathological diagnosis (target condition: all malignant tumours including borderline). The summary estimate of sensitivity for the IOTA group’s simple ultrasound rules (93.9%, 95% CI 92.8% to 94.9%) was significantly higher than that for the RMI 1 (66.9%, 95% CI 64.8% to 68.9%). Conversely, the summary estimate of specificity for the IOTA group’s simple ultrasound rules (74.2%, 95% CI 72.6% to 75.8%) was significantly lower than that for the RMI 1 (90.1%, 95% CI 88.9% to 91.2%).

Three of the above studies44,50,62 also reported comparative accuracy data for the IOTA group’s simple ultrasound rules versus the RMI 1 (threshold of 200), whereby participants with inconclusive IOTA assessments were classified by an expert subjective assessment of the ultrasound images (see Table 15). The summary estimate of sensitivity for the IOTA group’s simple ultrasound rules (91.2%, 95% CI 89.4% to 92.8%) was significantly higher than that for the RMI 1 (67.8%, 95% CI 65% to 70.4%). Conversely, the summary estimates of specificity were significantly lower for the IOTA group’s simple ultrasound rules (89.6%, 95% CI 88.1% to 91%) than for the RMI 1 (98.5%, 95% CI 98.3% to 98.7%). These three studies also reported accuracy data stratified by menopausal status, and the comparative accuracy estimates for both subgroups followed the pattern observed for all participants (see Table 15).

In premenopausal women, using the IOTA group’s simple ultrasound rules whereby participants with an inconclusive assessment were assumed to have a malignant tumour, the summary estimates of sensitivity and specificity were 94.3% (95% CI 91.7% to 96.3%) and 78.2% (95% CI 75.7% to 80.5%), respectively. These estimates were not significantly different from those for postmenopausal women (95.5%, 95% CI 93.7% to 96.9%, and 72.3%, 95% CI 68.9% to 75.5%, respectively). When participants with inconclusive IOTA group’s Simple Rules assessments were classified by an expert subjective assessment, the summary estimates of sensitivity were similar (for both premenopausal women and postmenopausal women) to those obtained when inconclusive assessments were assumed to be malignant (see Table 15). However, in both premenopausal women and postmenopausal women, the use of a subjective assessment significantly increased the summary estimate of specificity to 92% (95% CI 90.3% to 93.5%) and 80.3% (95% CI 76.9% to 83.4%), respectively. In premenopausal women, the summary sensitivity estimate for the RMI 1 (52.2%, 95% CI 47.4% to 56.9%) was significantly lower than that for the IOTA group’s simple ultrasound rules, and the summary specificity estimate (94.2%, 95% CI 92.7% to 95.5%) was significantly higher. In postmenopausal women, the summary sensitivity estimate for the RMI 1 (78.8%, 95% CI 75.7% to 81.7%) was also significantly lower than that for the IOTA group’s simple ultrasound rules; however, there were no significant differences between the specificity estimates (see Table 15).

One further study,60 which was reported as only a conference abstract, did not report sufficient information to determine how participants with inconclusive IOTA group’s simple ultrasound rules assessments were handled or how the target condition was defined (whether or not non-ovarian and borderline tumours were included in the definition of disease positive). This study reported sensitivity estimates of 94% for the IOTA group’s simple ultrasound rules and 72% for the RMI 1, used at a decision threshold of 250. The corresponding specificity estimate was 80% for both tests.

Details of Overa (multivariate index assay, second-generation) studies

Three diagnostic cohort studies reported in four publications68–70,104 provided data on the diagnostic performance of the Overa (MIA2G) score, for the identification of women with an adnexal mass who are at a high risk of developing ovarian cancer. All the studies were conducted in the USA. Only one study70 was reported as a full paper; the remaining two studies were reported in the form of meeting slides104 and a conference abstract. 68

One study70 used an Overa (MIA2G) score based on Roche Diagnostics’ assays and a Roche Diagnostics analyser; the other two studies68,104 did not report assay details.

The target condition for this assessment is ovarian cancer (i.e. epithelial ovarian cancer, fallopian tube carcinoma, primary peritoneal carcinoma and borderline ovarian cancer). All studies in this section included women with one or more adnexal mass and used a definition of malignancy that included borderline cancers. Histopathology indicated that all of the studies also included some women with non-ovarian malignancies and non-ovarian metastases. Full details of the final histopathological diagnoses of study participants who had a malignant mass are reported in Appendix 4, Table 36.

Accuracy of Overa (MIA2G) for determining a high risk of developing ovarian cancer

No studies were identified that directly compared Overa (MIA2G) to the RMI 1 at either decision threshold (200 or 250).

One study104 reported comparative accuracy data for Overa (MIA2G) versus the ROMA score, using Roche Diagnostics’ Elecsys tumour marker assays (Table 16). This study included all participants in the analysis, regardless of their final histopathological diagnosis (target condition: all malignancies including borderline). At a 29.9%) was 79.2% (95% CI 73.7%/29.9%) was 79.2% (95% CI 73.7% to 83.8%) and the specificity estimate was 78.9% (95% CI 75.8% to 81.7). These data indicate that the sensitivity of the Overa (MIA2G) score was significantly higher than that of the ROMA score, whereas the specificity of the Overa (MIA2G) score was significantly lower than that of the ROMA score (Figure 5).

FIGURE 5.

Comparison of the summary estimates for Overa (MIA2G) and ROMA score using Roche Diagnostics’ tumour marker assay (all malignant tumours plus borderline). (a) Sensitivity estimate; and (b) specificity estimate.

The two remaining studies68,70 reported data on the accuracy of Overa (MIA2G) without comparison with the RMI 1 or any other risk score and analysed data for any malignant tumour plus borderline (Table 17). At a threshold of 5 units, the pooled sensitivity estimate was 90.2% (95% CI 84.6% to 94.3%) and the pooled specificity estimate was 65.8% (95% CI 61.9% to 69.5%); these estimates were similar to those reported by the comparative accuracy study. One study stratified data by menopausal status and found no significant variation in test performance (see Table 17). 70

Details of Risk of Malignancy Index 1 studies

Ten diagnostic cohort studies,71–80 reported in 10 full-paper publications, provided data comparing the diagnostic performance of the RMI 1 at multiple decision thresholds, including a decision threshold of 250, for the identification of women with an adnexal mass who were at a high risk of developing ovarian cancer.

Two studies78,79 specifically included women from the UK, two studies were European (from Italy and Norway)76,80 and six studies were from five non-European countries (Turkey, Pakistan, China, India and Japan). 71–75,77

Three studies75,76,78 used a RMI 1 score based on an Abbott Diagnostics’ CA125 assay, three studies71,72,74 used a Roche Diagnostics assay, one study77 used an IMMULITE^® assay, one study79 used CIS Bioindustries, one study80 used a commercial kit by Centocor (Malvern, PA, USA) and one study73 did not report the CA125 assay used.

This assessment is primarily concerned with providing a comparison between the RMI 1,78 used with a decision threshold of 250 (current standard practice in the NHS1) and the specified alternative risk-scoring methods (see Chapter 2, Intervention technologies). The identified studies for the RMI 1 reported test performance data for multiple thresholds, and full data are reported in Appendix 5, Table 40. All of the identified studies that provided comparative accuracy data for alternative risk-scoring methods versus the RMI 1 used a decision threshold of 200. In order to assess the applicability of these data to the stated objective of this assessment, this section therefore focuses on the comparative accuracy of the RMI 1, using decision thresholds of 200 and 250.

The target condition for this assessment is ovarian cancer (i.e. epithelial ovarian cancer, fallopian tube carcinoma, primary peritoneal carcinoma and borderline ovarian cancer), defined as those conditions covered by the NICE clinical guideline CG122. 1 All studies in this section included women with one or more adnexal mass. Seven studies72–74,76,78–80 used a definition of malignancy that included borderline tumours, two studies71,75 excluded women found to have borderline tumours from the analyses and, in the remaining study,77 it was unclear whether or not women with borderline tumours were included in the analysis (no histopathology was reported with which to confirm the tumour type). Six studies73,74,76,78–80 included all study participants in the analyses and included some women with ‘other malignancies’, metastases from non-ovarian sites and ‘non-ovarian cancers.’ Full details of the final histopathological diagnoses of study participants who had a malignant mass are reported in Appendix 4, Table 26.

Accuracy of Risk of Malignancy Index 1 for determining a high risk of developing ovarian cancer using different decision thresholds

Six studies73,74,76,78–80 included all study participants in the analyses, regardless of final histopathological diagnosis [target condition: all malignant tumours including borderline (Table 18)]. At the decision threshold of 200, the summary estimate of sensitivity derived from these studies was 70.8% (95% CI 65.6% to 75.6%) and the summary estimate of specificity was 91.2% (95% CI 88.9% to 93.1%). At the decision threshold of 250, the summary estimate of sensitivity was 69% (95% CI 63.7% to 73.9%) and the summary estimate of specificity was 91.6% (95% CI 89.3% to 93.5%). The sensitivity and specificity estimates did not differ significantly between the two decision thresholds [200 and 250 (Figure 6)]. Studies compared multiple thresholds (between 25 and 500); as would be expected, the sensitivity estimate for the RMI 1 increased and the specificity estimate decreased with a decreasing threshold (see Appendix 5, Table 40).

FIGURE 6.

Comparison of the summary estimates for the RMI 1 at thresholds of 200 and 250 (all malignant tumours plus borderline). (a) Sensitivity estimate; and (b) specificity estimate.

One study72 reported a direct comparison of the RMI 1 at decision thresholds of 250 and 200 and excluded women with a final histopathological diagnosis other than primary ovarian cancer from the analysis [target condition: ovarian malignancies including borderline (see Table 18)]. At the decision threshold of 200, the sensitivity estimate was 80% (95% CI 65.2% to 89.5%) and the specificity estimate was 86.4% (95% CI 81.8% to 89.9%). At the decision threshold of 250, the sensitivity estimate was 72.5% (95% CI 57.2% to 83.9%) and the specificity estimate was 88.7% (95% CI 84.4% to 92.0%). Although the sensitivity estimate was higher for the 200 threshold and the specificity estimate was higher for the 250 threshold, these differences were not significantly different. In addition, the sensitivity and specificity estimates from this study did not differ significantly from the summary estimates described earlier.

Two further studies71,75 excluded participants found to have borderline tumours from the analysis (target condition: all malignant tumours including borderline). At the decision threshold of 200, the summary estimate of sensitivity was 73.5% (95% CI 64.3% to 81.3%) and the summary estimate of specificity was 89.6% (95% CI 83.2% to 94.2%). At the decision threshold of 250, the summary estimate of sensitivity was 66.4% (95% CI 56.9% to 75.0%) and the summary estimate of specificity was 93.3% (95% CI 87.7% to 96.9%). The sensitivity and specificity estimates did not differ significantly between the two decision thresholds (200 and 250). In addition, these summary sensitivity and specificity estimates did not differ significantly from those derived from the six studies that included all participants in their analyses.

One study77 included participants with malignant tumours, but it was unclear whether or not borderline tumours were included (see Appendix 5, Table 43).

Havrilesky et al. (2015)

Havrilesky et al. 123 constructed a Markov model [in TreeAge Pro 2013 (TreeAge Software, Inc., Williamstown, MA, USA)] using alive and death health states. From a societal US perspective, the authors estimated the costs and outcomes of five strategies to help clinicians to decide which women with an adnexal mass requiring surgery would most benefit from subspecialist referral:

1. American Congress of Obstetricians and Gynecologists (ACOG)’s guidelines

2. multivariate index assay (MIA) algorithm

3. ROMA

4. CA125 level alone with lowered cut-off values to prioritise test sensitivity over specificity (15 U/ml for postmenopausal and 22 U/ml for premenopausal women)

5. referral of all women.

The analyses indicated that CA125 level is a cost-effective test for willingness-to-pay thresholds below US$9423 and US$10,644 per LYs gained for postmenopausal women and premenopausal women, respectively. The refer-all strategy was cost-effective above these thresholds. The other strategies are dominated. Therefore, it was concluded that referral of all women to a subspecialist is a cost-effective strategy for managing women with adnexal masses requiring surgery. However, if a test-based triage strategy is needed (e.g. because of capacity constraints), CA125 level with lowered cut-off values should be considered.

Drescher et al. (2012)

Drescher et al. 124 used an unspecified model type and structure to estimate the cost-effectiveness of first-line testing of women with an adnexal mass, TVS and CA125 level, in women aged 45–85 years (US setting, perspective not stated). The following multimodal testing strategies were considered:

• no primary care testing

• CA125 level followed by TVS

• CA125 level followed by hypothetical imaging with 50% improvement in sensitivity compared with TVS

• hypothetical biomarker with twofold greater sensitivity followed by TVS

• hypothetical biomarker and hypothetical imaging (as above).

The analysis indicated that CA125 level and TVS led to 1.68 more LYs than no primary care testing, resulting in an incremental cost-effectiveness ratio (ICER) of US$88,993 per LY gained. Moreover, it was concluded that testing outcomes are relatively insensitive to second-line test performance and costs. Identification of a first-line test that does substantially better than CA125 level and has similar costs is required for primary care testing to reduce ovarian mortality by at least 25% and be reasonably cost-effective.

Kearns et al. (2016)

Using the NHS Personal Social Services perspective, Kearns et al. 125 developed a Markov model to estimate the cost-effectiveness of different screening strategies in postmenopausal women and to estimate the value of further research. The following screening strategies were considered:

• multimodal screening (MMS): first-line screening with CA125 level interpreted with risk of ovarian cancer algorithm, followed by TVS performed by senior staff

• ultrasound screening (USS): first-line screening with TVS performed by less-experienced staff, followed by TVS performed by more-experienced staff

• no screening.

Results indicated that USS was dominated by MMS, being both more costly and less effective. Compared with no screening, MMS cost £419 more and generated 0.047 additional QALYs, resulting in an ICER of £8864 per QALY gained, but alternative mortality extrapolation methods increased the ICER. The conclusion was that MMS for ovarian cancer is both more effective and more expensive than no screening, but that substantial uncertainty remains regarding the extrapolated long-term effectiveness.

Forde et al. (2016)

From the perspective of the public payer, Forde et al. 122 developed a Markov model to evaluate the cost-effectiveness of different strategies for use in triaging women with an adnexal mass. The following triage strategies were considered:

• the MIA (Ova1; Vermillion, Inc., Austin, TX, USA) based on five biomarkers, including CA125 level

• the modified ACOG (mACOG) referral guidelines

• CA125 level testing alone.

The MIA resulted in fewer reoperations and pretreatment CT scans, and was cost-effective compared with the ACOG referral guidelines, with an ICER of US$35,094 per QALY gained. The MIA dominated CA125 level alone, by being cost-saving and QALY-increasing. The MIA is expected to increase the percentage of women with ovarian cancer referred to gynaecological oncologists, thereby improving clinical outcomes.

Ding and Hay (2010)

Ding and Hay121 assess the cost-effectiveness of annual MMS (with CA125 marker, followed by TVS for those women at an increased risk of developing ovarian cancer according to their CA125 level) versus no screening for postmenopausal females aged 65–69 years from a US societal perspective. It should be noted that the available information for this assessment is restricted to one abstract (despite efforts to contact the authors). The incremental analysis indicated that, over a lifetime, MMS was both more costly (incremental costs of US$820) and more effective (incremental QALY of 0.004), resulting in an ICER of US$221,622 per QALY gained compared with no screening.

Probabilities not related to the risk scores

An overview of the disease-related (ovarian cancer, CRC and benign mass) probabilities for both the decision tree and the Markov model is provided in Table 21. It was assumed that all patients are female (used for utility estimation).

The prevalence of malignancies (all, including borderline and non-ovarian malignancies) as well as the proportion of women diagnosed with other malignancies (assumed to be CRC) were obtained using a random-effects meta-analysis (with log-transformation) of diagnostic cohort studies, included in our systematic review, which reported data for the relevant target condition and subgroup. The following parameters were estimated as in CG122:1

• percentage of early-stage ovarian cancer versus AOC

• 30 days post-surgery ovarian cancer mortality

• 10-year overall survival and progression-free survival of ovarian cancer [using updates of the same trials: the International Collaborative Ovarian Neoplasm (ICON) Group’s study 1 was used to model progression-free survival and overall survival for early ovarian cancer;133 and the ICON Group’s study 3 was used to model these outcomes for AOC134].

The following parameters were estimated as in the most recent diagnostic appraisal review (DAR) in CRC:136

• percentage in each of the Dukes’ stages

• annual progression between Dukes’ stages

• mortality by Dukes’ stage.

The effect of SMDT treatment (i.e. with gynaecological oncologists on site) versus women treated in secondary care was estimated from Woo et al. ,135 who reported a hazard ratio (HR) of 0.90 (95% CI 0.820 to 0.990) for the overall survival of women with ovarian cancer treated in teaching hospitals versus general hospitals. This HR was also assumed for progression-free survival, as the analyses by Woo et al. 135 indicated that the HR for overall and progression-free survival for teaching hospitals versus general hospitals is very similar. This study was obtained from a focused literature search, which was pragmatic in design. For this, the following resources were searched:

• MEDLINE (via Ovid): 1946 to week 3 January 2017

• MEDLINE In-Process Citations (via Ovid): to 30 January 2017

• MEDLINE Daily Update (via Ovid): to 30 January 2017

• MEDLINE Epub Ahead of Print (via Ovid): to 30 January 2017

• EMBASE (via Ovid): 1974 to 30 January 2017

• Cochrane Database of Systematic Reviews (via Wiley Online Library): to Issue 1 of 12, January 2017

• Database of Abstracts of Reviews of Effects (via Wiley Online Library): to Issue 2 of 4, April 2015

• Cochrane Central Register of Controlled Trials (via Wiley Online Library): to Issue 11 of 12, November 2016

• NHS Economic Evaluation Database (via Wiley Online Library): to Issue 2 of 4, April 2015.

Full search strategies are presented in Appendix 1.

Finally, age-dependent mortality from the general population was used for women with a benign mass, after the 30-day post-surgery period. All input parameters for the Markov model are reported in Table 21.

Risk score accuracy parameters

The proportions of women testing positive (and thus referred to a SMDT) or negative were based on the estimated accuracy of the risk scores considered (see Chapter 3, Selection of diagnostic performance estimates for inclusion in cost-effectiveness modelling and Table 22) and the estimated prevalence of all malignancies detected in this population (21.3% with a standard error of 1.0%). The proportions of true positives (TPs), FPs, FNs and true negatives (TNs) were calculated as follows:

Subsequently, the proportions of women who are referred to a SMDT (TPs and FPs), and who are not referred to a SMDT (TNs and FNs) were calculated. The results are listed in Table 23.

After the risk assessment and referral decision, women in the decision tree were allocated to ‘early-stage ovarian cancer’, ‘AOC’, ‘benign mass’, ‘CRC’ and ‘death’. One of the main assumptions in the decision tree was that the women categorised as testing FN all had early-stage disease based on expert opinion (i.e. the value of 25% for early-stage ovarian cancer refers only to the TPs).

Health state utilities

For women with a benign mass, age-dependent general population utility estimates were used. 138 The utilities for early-stage and AOC were taken from Havrilesky et al. 139 and Grann et al. ,140 respectively. These estimates were also used in the economic model in CG122. 1 As in the latest CRC DAR, the study by Ness et al. 136,141 was used to inform utilities for the four stages of CRC. Utility estimates and sources are summarised in Table 24.

Resource use and costs related to the risk scores

Risk score costs are listed in Table 25.

To derive the costs associated with these risk scores, several assumptions were made. These pertained to the different components of the RMI 1 score, the ROMA score, Overa (MIA2G), the IOTA group’s simple ultrasound rules and the ADNEX model, and are summarised in the following sections.

Resource use and costs not related to the risk scores

All women with a high-risk test result are assumed to be referred to the SMDT. Based on expert opinion, the additional resource use for this is assumed to be the cost of a SMDT meeting, assuming no additional cost of the surgery or other investigations. The cost of this meeting (£116) is estimated to be that of the SMDT meetings, from the 2015–16 NHS reference costs. 143

The treatment of ovarian cancer may consist of surgery and/or chemotherapy or supportive care, depending on the disease stage. As assumed in CG122,1 chemotherapy consists of six cycles of carboplatin for women with early-stage ovarian cancer and six cycles of carboplatin/paclitaxel for women with advanced-stage disease. 1 Surgery costs were also based on CG122,1 and calculated as a weighted average of the relevant NHS reference costs, taking into account the probability of complications and the underlying disease (early-stage or AOC or benign mass). 143 The proportions of women receiving each type of care were based on CG122,1 and the unit costs of care from the latest PSSRU publication. 144 The frequency of follow-up costs for women with ovarian cancer was based on CG122,1 and the unit costs were based on the PSSRU publication. 144 Annual costs for the CRC states were estimated from the lifetime costs of CRC and mean survival as was done in the most recent DAR in CRC136,137 (Table 26). Women with a high risk score but a benign mass would be identified at the SMDT meeting to be FP cases, and would undergo SMDT surgery without any further costs incurred in the tertiary care setting.

Premenopausal subgroup

The premenopausal subgroup analysis used a different mean starting age (38 years) and drew on subgroup-specific accuracy data (see Appendix 7). Tables 29 and 30, as well as Figure 12, show the probabilistic results of the subgroup analysis in premenopausal women. The ROMA score using Abbott Diagnostics’ ARCHITECT was the least effective (18.108 LYs, 14.927 QALYs) and the RMI 1 with a threshold of 200 was the cheapest (£5188), followed by the IOTA group’s simple ultrasound rules (inconclusive, assumed to be malignant) at £5189. The most effective options were the IOTA group’s ADNEX model (18.137 LYs, 14.948 QALYs) followed by the IOTA group’s simple ultrasound rules (18.135 LYs, 14.946 QALYs). Consequently, the incremental analysis indicated that between thresholds of £15 and £18,304 per QALY gained, the IOTA group’s simple ultrasound rules are cost-effective, whereas the IOTA group’s ADNEX model (threshold of 10%) is cost-effective for higher thresholds.