OPTIMA prelim: a randomised feasibility study of personalised care in the treatment of women with early breast cancer

Individual multiparameter assays

Comparative performance of multiparameter assays

Very limited information on direct comparisons between multiparameter assays is available. Comparisons have been performed between Oncotype DX, IHC4, Prosigna and BCI results from approximately 1000 post-menopausal patients treated with endocrine therapy from the transATAC cohort. 43,55,56,75 These have focused on high-level statistical analysis of the prognostic value of different tests within this population. Information on the proportion of patients testing ‘positive’ (i.e. high risk) for individual tests is more opaque and, in particular, data on results at the individual patient level are lacking. Nonetheless these comparisons have been of substantial value in allowing comparisons between tests, albeit limited to comparing their overall prognostic impact. Taken collectively, the results of these pivotal studies support the conclusion that all four assays provide similar prognostic information at the population level. They also provide evidence that Prosigna and BCI provide statistically more prognostic information than Oncotype DX and IHC4, and in contrast to Oncotype DX and IHC4, they provide information about the risk of late (beyond 5 years) relapse. No conventional parameters of performance (e.g. area under the receiver operator curve) have been presented at any time for any of the currently available assays, which makes it difficult to evaluate their performance in this regard. However, the statistical improvements in performance for Prosigna and BCI are modest, but important, leaving space for further improvement.

An alternative approach is to use published assay algorithms, or to ‘reverse engineer’ assays, and to apply these to gene expression data sets, with associated outcome data as an ‘in silico’ experiment. 109,110 Such comparisons have, like the direct comparisons, shown that the results of the various multiparameter assays are moderately correlated with respect to the prognostic information provided. The major limitations of this methodology are, first, that different array platforms lack probes that match to prognostic signatures, such that ‘missing genes’ may compromise analysis, and, second, that the algorithms used by the assays are calibrated according to the method used to measure gene expression and may not translate well into semiquantitative analysis of microarray data sets used for this analyses. Signatures compared, in a similar manner using arrays and reverse engineering, in the I-SPY 1 trial include MammaPrint and PAM50. 40 There was again a modest correlation between assay performance, and most provide predictive information about response to neoadjuvant chemotherapy and outcome. Once again, the analysis of I-SPY 1 provided no information on the comparative performance of assays at patient level.

Cost-effectiveness analyses of multiparameter testing

We performed a systematic review of literature on the cost-effectiveness of multiparameter testing; the methods are described in Appendix 1. Six studies assessed the impact of Oncotype DX (one in each of Canada,111 the USA,112 the UK,74 Germany,113 Australia114 and Japan115), and one assessed the impact of MammaPrint in the Netherlands. 116 No head-to-head comparisons of the alternative genomic tests have been conducted. Four studies74,112,113 either were funded by the test manufacturer or were carried out by researchers who worked for the manufacturer. 114 All studies compared the given genomic test with current conventional management, which consisted of current national guidelines and tools based on patient clinicopathological data (such as Adjuvant! Online,17 PREDICT18 and the NPI19).

The majority of studies considered mixed populations, including patients with both lymph node-negative and lymph node-positive disease. 74,113,115,116 Two studies112,113 assessed populations that included only patients with lymph node-positive disease, both of which evaluated Oncotype DX. Four studies reported specific details of patients’ level of lymph node involvement;112,113,115,116 most restricted their populations to patients with low levels (1–3) of lymph node involvement, with only one study111 including patients with higher (1–9) lymph node involvement.

The single MammaPrint study estimated that the test would lead to a 0.21 decrease in quality-adjusted life-years (QALYs) together with a US$2882 fall in costs, from a Netherlands societal perspective. 116

Of the six studies assessing Oncotype DX,74,111,115 two estimated that the test would dominate standard care strategies, with Oncotype DX found to produce a gain in QALYs and a reduction in costs. 112,113 Both of these studies were funded by the manufacturer. Vanderlaan et al. 112 found that, from a health-care payer perspective and using a lifetime horizon, Oncotype DX was estimated to produce a gain of 0.17 QALYS per patient and reduce costs by US$384 (based on 2009 costs). Eiermann et al. 113 found that, from a German health-care payer perspective and over 30 years, Oncotype DX was estimated to produce a gain of 0.06 QALYS and reduce costs by €561 per patient.

The remaining four studies74,111,114,115 found Oncotype DX to be more costly and more effective than standard care. The highest incremental cost-effectiveness ratio (ICER) value was estimated in the only study that included patients with high levels of lymph node involvement (1–9 positive nodes). Lamond et al. 111 found that, from a Canadian third-party payer perspective and over a lifetime horizon, Oncotype DX was estimated to produce a 0.06 QALY gain at an incremental cost of CA$14,844 (based on 2011 costs) [95% confidence interval (CI) CA$3616 to CA$25,646] per QALY. They found that there was almost a 100% probability of Oncotype DX-guided treatment being cost-effective, but that the results for the lymph node-positive group were more sensitive to one-way sensitivity analyses than the results in a lymph node-negative population. For the remaining three studies (which assessed mixed lymph node populations), the ICER results were £6232/QALY,74 JP¥568,533/QALY (equivalent to US$5685)115 and AUD$9986/QALY. 116

In addition to these studies, two studies15,49 have considered the likely cost-effectiveness of multiparameter assays for use in the NHS. Hall et al.,15 who performed a cost–utility analysis of Oncotype DX in a node-positive population by modelling outcome data from the SWOG88-14 Oncotype DX study62 to the NHS, concluded that Oncotype DX is unlikely to be cost-effective for this population. Ward et al. ,49 who undertook the external evidence review for the NICE DG10 guidance117 in a node-negative population, concluded that Oncotype DX could be cost-effective if offered to a higher-risk subset of such patients. They considered that MammaPrint and IHC4 may also be cost-effective, but the evidence was not sufficiently robust to draw a firm conclusion, while limited evidence suggested that Mammostrat is unlikely to be cost-effective.

The literature review also identified two budget impact analyses,118,119 both of which were funded by or affiliated with the manufacturer of the test. Ragaz et al. 118 assessed the impact of Oncotype DX on US and Canadian health-care costs in both lymph node-positive and lymph node-negative patients, and estimated that Oncotype DX could save US$330.8M in the USA and US$46.2M in Canada. Zarca et al. 119 assessed the impact of MammaPrint on health-care costs in France on node-positive (1–3) breast cancer patients, and estimated that MammaPrint could save €9043 per 100 patients per year compared with current practice.

Availability of multiparameter testing in the UK

Four multiparameter assays have been evaluated by NICE (Oncotype DX, MammaPrint, IHC4 and Mammostrat) for potential use in the NHS. The resulting guidance117 was published in September 2013. The guidance states that Oncotype DX ‘is recommended as an option for guiding adjuvant chemotherapy decisions for people with ER-positive, lymph node-negative and HER2-negative early breast cancer’. The recommendation was limited to patients at ‘intermediate risk’, defined as a NPI score of > 3.4, and relies on a discounted price for Oncotype DX. The guidance and resulting funding decisions apply only to England. The three other technologies evaluated were recommended for use only in research at this time. Five additional technologies, including PAM50, BCI and NPI+, were removed from the scope in November 2011, almost 2 years before the final guidance was published, on the grounds that at that time there was insufficient evidence to justify their inclusion in the economic analysis performed by the external assessment group. 120 The appraisal was also limited to patients with node-negative disease, as the evidence for the use of the tests was considered less robust in the node-positive population. The NICE committee recommendation for the use of Oncotype DX was made on the basis that the test provides superior prognostic information to conventional clinicopathological risk assessment. The committee, however, did not consider the evidence that Oncotype DX is able to predict the benefit of chemotherapy above and beyond providing prognostic information to be robust and recommended further research into this question for all four technologies.

Trial objectives

The OPTIMA prelim objectives were:

• to evaluate the performance and health economics of alternative multiparameter tests to determine which will be evaluated in the main trial

• to establish the acceptability to patients and clinicians of randomisation to test-directed treatment assignment

• to establish efficient and timely sample collection and analysis, which is essential to the delivery of multiparameter test-driven treatment.

The OPTIMA main trial objectives are:

• to evaluate whether or not test-directed chemotherapy treatment and endocrine therapy is non-inferior to chemotherapy followed by endocrine treatment for all patients in terms of invasive disease-free survival (IDFS).

• to establish the cost-effectiveness of a test-guided treatment strategy compared with standard practice.

OPTIMA prelim success criteria

The prespecified criteria used to determine the success of OPTIMA prelim were recruitment of 300 patients no more than 2 years from the first centre opening to recruitment, and, for the final 150 patients:

• patient acceptance rate should be at least 40%

• recruitment should take no longer than 6 months

• chemotherapy should start within 6 weeks of signing the OPTIMA prelim consent form for no less than 85% of chemotherapy-assigned patients.

Outcome measures

Inclusion criteria

• Female, aged ≥ 40 years.

• Excised invasive breast cancer with local treatment either completed or planned according to trial guidelines.

• ER-positive [Allred score of ≥ 3 or histoscore (H-score) of ≥ 10 or as otherwise established by the reporting pathologist] as determined by the referring centre and centrally confirmed.

• HER2-negative (i.e. immunohistochemistry score of 0–1+) or fluorescence in situ hybridisation (FISH) or other in situ hybridisation non-amplified, as determined by the referring centre and centrally confirmed.

• Axillary lymph node status: (1) 1–9 involved (macrometastases, i.e. > 2 mm OR micrometastases, i.e. > 0.2–2 mm) OR (2) node negative AND tumour size ≥ 30 mm. Nodes containing isolated tumour cell clusters only, that is ≤ 0.2 mm in diameter, were considered to be uninvolved.

• Considered appropriate for adjuvant chemotherapy by treating physician.

• Patient assessed as fit to receive chemotherapy and other trial-specified treatments with no concomitant medical, psychiatric or social problems that might interfere with informed consent, treatment compliance or follow-up.

• Bilateral and multiple ipsilateral cancers were permitted provided at least one tumour fulfils the entry criteria and none meet any of the exclusion criteria. Patients with bilateral tumours where both tumours fulfil all eligibility criteria including size and nodal status were excluded. Note that for separate synchronous primary cancers, whether ipsilateral or bilateral, it was anticipated that laboratories would, as per standard good practice, assess ER and HER2 on the different lesions. Centres were requested to send a block for each separately reported tumour for central eligibility testing provided sufficient material was available. If there were multiple invasive foci deemed to derive from one main cancer (satellite foci), which had the same histological features including, for example tumour type and grade, it was not required that every focus has its receptor status reassessed.

• Written informed consent for the study.

Exclusion criteria

• Ten or more involved axillary nodes or involved internal mammary nodes.

• ER negative OR HER2 positive/amplified tumour on central eligibility testing.

• Metastatic disease. Note that formal staging according to local protocol was recommended for patients where there was a clinical suspicion of metastatic disease or for stage III disease (tumour > 50 mm with any nodal involvement OR any tumour size with four or more involved nodes).

• Previous diagnosis of malignancy unless:

  • managed by surgical treatment only and disease free for 10 years

  • previous basal cell carcinoma of skin, cervical intraepithelial neoplasia or ductal carcinoma in situ of the breast treated with surgery only.

• The use of oestrogen replacement therapy (HRT) at the time of surgery. Patients who were taking HRT at the time of diagnosis were eligible provided the HRT was stopped before surgery.

• Presurgical chemotherapy, endocrine therapy or radiotherapy for breast cancer. Treatment with endocrine agents known to be active in breast cancer including ovarian suppression was permitted provided this was completed > 1 year prior to study entry.

• Commencement of adjuvant treatment prior to trial entry. Short-term endocrine therapy initiated because of, for instance, prolonged recovery from surgery was permitted but had to be discontinued at trial entry.

• Trial entry more than 8 weeks after completion of breast cancer surgery.

• Planned further surgery for breast cancer, including axillary surgery, to take place after randomisation, except either re-excision or completion mastectomy for close or positive/involved margins, which could be undertaken following completion of chemotherapy.

• Patients with more than two involved axillary nodes (as defined in the inclusion criteria) identified by sentinel node biopsy or by axillary sampling where further axillary surgery was not planned.

Ethical and regulatory approval

The South East Coast–Surrey Research Ethics Committee approved the trial on 22 June 2012. Local research and development department approval was obtained at each participating NHS trust prior to patients being enrolled into the trial. The trial was conducted in accordance with the principles and guidelines of the International Conference on Harmonisation, Good Clinical Practice, UK legislation, Warwick Clinical Trials Unit (CTU) standard operating procedures and the Research Ethics Committee approved protocol. The trial is registered as Current Controlled Trials ISRCTN42400492.

Management of the trial

The Trial Steering Committee (TSC) provided overall supervision of the trial, with an independent chairperson and majority of independent members. The Trial Management Group (TMG) comprised a multidisciplinary team of clinicians, statisticians, a translational scientist and a patient advocate, all with considerable expertise in all aspects of trial design, running, quality assurance and analysis. The Data Monitoring and Ethics Committee carried out the independent monitoring of the trial. Details of the membership of the TSC, TMG and Data Monitoring and Ethics Committee can be found in the Acknowledgements (see Acknowledgements and Contributors).

Trial centres

A total of 35 hospitals from 31 NHS trusts and health boards in England, Scotland and Wales participated in OPTIMA prelim. As the demonstration of recruitment feasibility was a major end point for OPTIMA prelim, it was important to ensure that trial centres were broadly representative of the UK. Rather than issue a general invitation to participate in the study, individual centres were approached in five main geographical clusters: Scotland, the West Midlands, East Anglia, North London and South West/South Wales. Both district general hospitals and cancer centres were included. A list of all participating centres can be found in the Acknowledgements (see Acknowledgements and Contributors). Prior to activating a centre to recruitment, a trial initiation meeting was held, in person or via teleconference, to provide study-specific training to all staff members working on the trial. Continued support was offered to existing and new staff at participating centres to ensure that they remained fully aware of trial procedures and requirements.

Patient information and informed consent

Patients who were potentially eligible for participation in the study according to the criteria (see Eligibility criteria) were identified at the multidisciplinary breast cancer meetings held at each of the centres. Patients were invited to participate in the OPTIMA prelim study during the initial consultation with their oncologist, where treatment options were discussed. Here the local principal investigator or their designee discussed the trial with the patient and provided the patient with a copy of the patient information sheet (PIS) (see Appendix 2). In many cases a research professional would be present in this consultation and/or meet with the patient afterwards to continue discussing the trial. Patients were given sufficient time to consider participation in the trial and subsequently the opportunity to ask any further questions and to be satisfied with the responses. Patients who were willing to participate were asked to provide written informed consent (see Appendix 2 for a copy of the consent form). Consent for the study included consent for use of the participant’s tissue samples, removed at the time of surgery, as part of the OPTIMA prelim. Participants were also asked to donate the remainder of their samples for future unspecified research projects; however, this was optional.

Randomisation procedure

Trial entry was defined as the time consent was given, which started the randomisation procedure. Once written informed consent had been given and chemotherapy selected from the regimens permitted in the protocol (Table 3), the researcher registered the patient with the CTU. During registration, eligibility was confirmed according to the results of local pathology testing and the patient was allocated a unique participant registration number.

Once the participant was registered, the centre sent relevant tumour block(s) from the surgical resection to the central laboratory for confirmatory testing of ER and HER2 status as soon as possible.

In order to avoid unnecessary delays in the randomisation procedure, on receipt of the tumour block, the central laboratory prepared samples for Genomic Health in parallel with undertaking eligibility testing. The central laboratory informed the CTU of the patient’s ER and HER2 status, and those eligible were randomised in a 1 : 1 ratio to standard treatment (control arm) or to test-directed treatment. Randomisation was performed centrally by a computer using a minimisation algorithm stratified to ensure balance across trial arms by chemotherapy regimen: anthracycline–non-taxane [FEC75–80, FEC90–100, epirubicin, cyclophosphamide, methotrexate and fluorouracil (E-CMF)] vs. taxane–non-anthracycline [docetaxel and cyclophosphamide (TC)] vs. combined anthracycline and taxane [FEC-T, fluorouracil, epirubicin, cyclophosphamide and paclitaxel (FEC-Pw)], number of involved nodes (none vs. positive sentinel node biopsy without axillary surgery vs. 1–3 vs. 4–9 nodes), menopausal status (pre-/perimenopausal vs. post-menopausal). Oncotype DX testing proceeded immediately for all participants [tumour blocks from participants randomised to the control arm underwent Oncotype DX testing as part of the pathology study (see Chapter 3, Pathology research in the preliminary studies)]. Genomic Health reported the test result to the CTU, which subsequently informed the research centre, by fax and e-mail, whether or not the participant was to receive chemotherapy according to randomisation and RS for those randomised to test-directed treatment. It was anticipated that the randomisation procedure, from date of consent/registration to treatment assignment, would take between 3 and 4 weeks.

Blinding

OPTIMA prelim was a partially blinded trial. For participants who were allocated to receive endocrine therapy alone, it was not possible to blind either the participant or the clinicians at the centre. However, for participants allocated to receive chemotherapy followed by endocrine therapy, the participant and the centres were blind to the participant’s randomisation. In order to maintain blinding, for participants randomised to the standard arm, the CTU delayed informing the treating centre of the treatment allocation by a time period equivalent to that taken to perform the Oncotype DX test for those randomised to test-directed treatment. In practice this was on receipt of the Oncotype Dx test result as this was performed for all participants in OPTIMA prelim irrespective of randomisation. To assess the success of the OPTIMA prelim and perform a comparison of multiparameter tests, the randomised treatment allocation remained blinded to the analyst, as it was not needed in the analysis of the OPTIMA prelim.

Trial treatment

With the exception of mandating whether or not a participant received chemotherapy, treatment pathways for patients joining the OPTIMA prelim study were designed to be as similar as possible to those for equivalent patients who were either ineligible or chose not to join the study. The process of consent and subsequent Oncotype DX testing was expected to delay the start of systemic treatment (with either chemotherapy or endocrine therapy) by about 3 weeks compared with non-trial-treated patients. This delay is not expected to be clinically deleterious. 128 Although the protocol contained recommendations for the management of all aspects of patient treatment based on national/international guidelines (where available) and consensus views on best practice, the expectation was that participating centres already follow these and, therefore, that treatment would follow the local implementation of best practice and national guidelines.

Screening information

In order to assess patient acceptability, each participating centre maintained a screening log to document all patients considered for the trial but subsequently excluded. Where possible, the reason for non-entry to the trial was documented. Screening logs were requested by the CTU on a monthly basis. No patient-identifiable data were recorded on the screening log.

Data collection

Clinical and resource use data generated by all centres were collected on study case report forms (CRFs), which were monitored and entered into the trial database by the CTU. The participant’s details were entered onto the local centre’s patient identification log; at no time was this log forwarded to the CTU. To preserve participants’ anonymity, only their allocated trial number and initials were used on the CRFs and any correspondence. With participants’ permission, their name, date of birth, address and NHS/Community Health Index number were also provided to the CTU on the registration CRF to allow flagging with the Office for National Statistics. Participants’ confidentiality was respected at all times.

Case report form completion guidelines were provided to all centres to aid consistency of completion. Data management and monitoring were conducted in accordance with the CTU standard operating procedures, which are designed to ensure that trial data are as complete and accurate as possible, and comply with the Data Protection Act 1998. 129 Data management practice included verification, database validation and formal data checking following data entry. All missing and ambiguous data were pursued until resolved or confirmed as unavailable. Monthly reminders were sent to participating centres in order to flag any forthcoming or outstanding clinical visits or questionnaires.

Quality of life and health resource use assessment

The patient questionnaire was designed to assess quality of life and health resource use. The baseline questionnaire was provided by the centre and completed by participants after they had given written informed consent but before randomisation. Further questionnaires were administered at 3, 6, 12 and 24 months from date of consent. These further questionnaires were either given to participants at a clinic appointment or sent by post. The OPTIMA prelim study used two instruments to gather information on quality of life as the basis for evaluating both effectiveness and cost-effectiveness: EQ-5D124 and the FACT questionnaire comprising FACT – General126 and FACT – Breast cancer. 127

Follow-up

Participants are followed up annually for 10 years from trial entry. Annual follow-up appointments could be conducted in the clinic or via telephone for participants who had been discharged from clinical review. The method of follow-up is recorded on the CRF.

Withdrawals

Participants had the right to withdraw from the trial at any time and for any reason. Those participants who were deemed ineligible by central review were not randomised and did not count towards the sample size. Participants who were registered but not randomised were not classified as withdrawals. Owing to the cost, the multiparameter assay was cancelled for those who withdrew consent post randomisation but prior to treatment allocation. For the purposes of the preliminary study, randomised participants who had not yet been allocated treatment and for whom an Oncotype DX result was not obtainable were not followed up. Participants who declined to comply with their trial-allocated treatment remained on-study and are followed up in accordance with the protocol.

Analysis of the OPTIMA prelim success criteria

The acceptability of the trial was assessed as the proportion of eligible patients consenting to participate in the OPTIMA prelim. The characteristics of the patients for whom the trial and the concept of test-driven therapy were acceptable were provided. The time from signing the OPTIMA prelim consent form to starting chemotherapy was calculated and the proportion of chemotherapy-assigned patients starting treatment within 6 weeks was determined. Compliance with the test-directed treatment decision was assessed through the proportion of patients choosing not to follow the result of the randomisation.

Decision to continue to the main trial

There were no formal stopping rules within OPTIMA prelim, as the ability to continue to the main trial was dependent on a combination of patient acceptability, recruitment, number of alternative tests to consider and cost-effectiveness. The decision on whether or not the main trial is feasible was dependent on meeting the predefined success criteria for the study (see OPTIMA prelim success criteria).

Focus group design

Three focus groups were held for people who had or were living after cancer. These were held in Coventry in August 2012, in Cambridge in February 2013 and Guildford in April 2013. At least two members of the OPTIMA trial team were in attendance to offer factual advice about the trial and to give administrative support,134 but their contribution was kept to a minimum to allow free discussion at each focus group. The focus groups were facilitated by trained facilitators and discussions were moderated independently by members of the charity ICPV, of whom one or two attended each group. Participants were people who lived locally and had been invited via a support group they attended or through ICPV. Twelve self-selecting participants attended each focus group. The majority were women with primary breast cancer, but the group in Guildford was made up of six women living after breast cancer and six people with other cancers, including four men. Each focus group was audio-recorded to ensure that data were as complete and reliable as possible, and a member of the study team took notes of the proceedings and main areas of discussion.

Potential focus group participants were given a précis of the trial protocol, copies of information given to potential OPTIMA prelim trial participants (the PIS) and details of the focus group aims. Additionally, an information leaflet about taking part in the focus group was provided for participants to read before each meeting, and to advise them that they would be asked for consent to take part in the study and have their contribution recorded if they attended. Each focus group meeting started with introductions and a presentation of the trial with an opportunity to ask questions. Following this, the intended structure of the meeting was explained. Ground rules were suggested by the study team members and agreed at each meeting by attendees (Table 4). Participants were also asked for written consent to take part in the focus group and for the focus group to be recorded; all participants gave consent. It was made clear that documentation would not contain anything that would make individuals identifiable and that names should not be used during the focus group to keep data reporting as confidential as possible.

Each group then proceeded with an invitation for each member to give a brief introduction of why they had attended the focus group and any other information that they felt was pertinent to the group. The majority of people also gave details about their diagnosis and treatment at this point, if applicable.

Research aim

The research aim for the focus groups was to seek an understanding from people with personal experience of cancer of the possible difficulties and/or opportunities involved in making a decision to take part in the OPTIMA study. Owing to the complexity of the study, each focus group session started with a brief 5-minute introduction to the trial and written material was handed out in a pack to each participant including the flow diagram.

Each group discussion was loosely structured to allow participants to focus on the difficulties and opportunities that were important to them. Each was opened with the question, ‘Do you think you would take part in the OPTIMA study?’. This was clarified by questions from the interview schedule detailed in Table 5.

Analysis

The analysis of focus group data has been criticised as being subjective, with methods often not explained fully in publications. 134–137 It is usually based on a ‘constant comparison’ approach, as initially used in grounded theory research,138 and involves coding and categorising data to identify themes.

We followed a previously described approach to analysis,139 which incorporates eight key stages of interpretation (Table 6).

The three focus groups were analysed separately and then combined to create a composite. Analysis was iterative and did not follow a linear course. It involved listening to tapes, transcribing them and reading the transcripts with the notes taken during the group. The transcripts were loaded into NVivo V.9 (QSR International, Warrington, UK), which supports qualitative methods by allowing the organisation and analysis of unstructured spoken data. While scrutinising the data and highlighting content of interest, we began to code the data line by line. During this stage, developing categories and themes began to emerge. After this had been done separately for the three groups, they were amalgamated and coding and data categorisation continued until there was consensus about the themes.

Phase 1: understanding recruitment issues

The first phase of the integrated QRS sought to understand barriers to recruitment, with the intention of developing interventions to support recruiters, improving the quality of information provision where needed and optimising recruitment rates. The approaches used to identify recruitment difficulties followed principles of ethnography, with a flexible approach to data collection and analysis of interviews, observations and documentary data including PISs and screening logs.

Sampling and recruitment

The sampling frame consisted of all TMG members and OPTIMA prelim staff members from centres participating in the QRS at the start of the study (9 out of the 25 centres open to recruitment). ‘QRS centres’ comprised those which agreed to take part in this ‘optional’ element of OPTIMA prelim. Additional centres that opened in later stages of the trial were also approached to take part in the QRS (contributing audio-recordings only).

Data collection

Data analysis methods

Phase 2: development and implementation of recruitment strategies

The findings from the QRS were developed into plans for feedback and training with the CfI and TMG members. All materials used for guiding and supporting recruiters were based on findings grounded in data collected in phase 1. Recruitment difficulties recurring across centres formed the basis of ‘generic’ interventions disseminated across all centres through ‘tips and guidance’ documents and regional feedback meetings attended by multiple centres. Clinician-to-clinician meetings between clinical members of the TMG and clinicians at recruiting centres were suggested where differences in clinical opinion had implications for recruitment. Single-centre feedback sessions were also developed for those who requested this or were unable to attend regional meetings. Difficulties experienced by individual recruiters were tackled through development of individual feedback plans that were discussed in confidence with the relevant individuals.

Phase 3: impact of qualitative recruitment study interventions

The final phase of the QRS drew on multiple approaches to evaluating the impact of the QRS on recruitment. The time points at which QRS interventions were delivered were mapped out on graphs that charted recruitment figures over time. Changes in the direction (or rate of increase/decrease) in recruitment figures were assessed relative to the QRS interventions. Other approaches to analysis considered recruitment rates in centres that received certain forms of intervention, compared with centres that did not. Further qualitative analysis also featured in this phase, where recruiters’ practices were compared before and after points of intervention. The varied approaches to evaluating the QRS were considered collectively with a view to forming tentative conclusions about the QRS’s success in optimising recruitment within OPTIMA prelim.

Multiparameter assays performed

We set out to evaluate as many tests as possible through positive engagement with multiple partners. Discussions with several vendors of tests, which owing to their characteristics might be candidates for inclusion in the OPTIMA main study, were undertaken during the design phase of OPTIMA prelim to secure collaboration. Strenuous efforts were made to secure collaboration with additional test vendors during the recruitment phase. We recognised that for commercially available tests it was important that evidence of quality of test performance was ensured. The possibility of centralising tests to a single laboratory was considered but rejected because the performance of most tests was dependent on commercial laboratories (e.g. Oncotype DX, MammaPrint) or expert readers (e.g. Mammostrat).

The vendors of the following tests were approached for support:

• BCI

• Endopredict (Sividon Diagnostics, Cologne, Germany)

• Genomic Grade Index (Bordet Institute, Brussels, Belgium)

• MammaPrint/BluePrint/TargetPrint

• Mammostrat

• MammaTyper

• Molecular Grade Index^SM (bioTheranostics/Qiagen, Limburg, the Netherlands)

• NexCourse Breast by AQUA

• Prosigna (PAM50).

Five tests were not taken forward for analysis in the OPTIMA prelim cohort following discussions with the vendors/licensees: BCI, Endopredict, Genomic Grade Index, Molecular Grade Index and Mammostrat. The reasons for rejecting these tests included OPTIMA prelim population not appropriate for the test in question (n = 3); test licensing under negotiation/transfer (n = 1); and test withdrawn from study (n = 1). The decision not to offer these tests for the OPTIMA population reflected commercial concerns about prematurely transposing tests into novel applications. Tests developed by two vendors who were supportive during the design phase were not included because of changed commercial development plans.

It is noteworthy that all companies participating in these discussions were supportive of the collaborative effort being undertaken and the OPTIMA trial in particular, and sought to ensure the broadest comparison between testing approaches given commercial constraints.

The Prosigna assay was performed at Professor Bartlett’s research laboratory at the OICR, following staff training and accreditation by NanoString Technologies. MammaPrint and BluePrint assays were performed by Agendia technologies using sections provided by the University of Edinburgh. MammaTyper quantitative PCR assays were performed by Stratifyer GmBH at laboratories in Mainz, Germany, using residual ribonucleic acid (RNA) from the Prosigna assay extracted using the Roche mRNA extraction kit provided by Prosigna. The NexCourse Breast IHC4 assay using AQUA (IHC4 AQUA) was performed by Genoptix medical laboratories on replicate tissue microarrays (TMAs) provided by the University of Edinburgh.

In addition to the commercial assays, IHC4 was also performed by conventional pathology techniques [HER2 testing by in situ hybridisation at the UCL (University College London) Advanced Diagnostics Laboratory and ER, PgR and Ki67 on TMAs by quantitative image analysis (Ariol version 4.0 SL50, Leica Microsystems, Wetzlar, Germany) using standard immunohistochemistry at OICR as previously described]. 144 Quantitation of ER, PgR and Ki67 to the degree required to calculate an IHC4 is not standard practice in NHS pathology laboratories and, indeed, Ki67 is not a standardised marker for routine testing in breast cancer pathology assessment.

Results were calculated to established procedures for each assay or, in the case of IHC4, using the published algorithm. 56 Tumours were categorised into risk groups based on the predefined cut-off points for the continuous Prosigna ROR scores, IHC4 and IHC4 AQUA (see Appendix 6 for details). These risk groups were then compared with each other including those defined by Oncotype DX.

Some guidelines, including the internationally recognised St Gallen guideline,145 recommend the use of breast cancer subtyping as a means of informing treatment options. Breast cancers can be subtyped into four main molecularly defined intrinsic subtypes (see Parker et al. 28 and references therein25–27), namely luminal A, luminal B, HER2 enriched and basal-like.

Three tests (BluePrint, Prosigna and MammaTyper) provide information on molecular subtypes. BluePrint, which is performed together with MammaPrint, and Prosigna additionally provide information about recurrence risk, which is binary in the case of MammaPrint and takes the form of a categorised risk score for Prosigna, while MammaTyper is specifically focused on providing subtype information including a stratification of ‘luminal B’ cases into low and high risk based on the Ki67 result (see Appendix 6 for details).

Analytical methods

For the purpose of analysis, patients were divided by randomisation and by Oncotype Dx score (≤ 25 vs. > 25). Patients#x02019; individual predicted predicted risks of both recurrence and death were summarised using median and ranges. The kappa coefficient and associated 95% CI was used to assess agreement between tests. All statistical analyses were performed using the SAS statistical package (version 9.3; SAS Institute Inc., Cary, NC, USA).

Sample size

OPTIMA prelim required 300 patients to be recruited over the first 2 years (a 6-month set-up phase and an 18-month recruitment phase). Oncotype DX was used prospectively to decide on patient chemotherapy treatment in the research arm (using a RS cut-off point of ≤ 25 vs. > 25), while alternative tests were applied retrospectively. The true efficacy of the tests will not be known until all patients have been followed up for 5 years and IDFS is compared. Therefore, the study was powered to compare risk groups from the multiparameter tests with Oncotype DX as well as with each other.

To assess agreement between tests, the sample size calculations assumed that 70% of patients randomised to test-directed treatment would be assigned to no chemotherapy as a result of the Oncotype DX test and the true kappa value was 0.8. A sample size of 300 patients would provide stability around the kappa estimate and provide a lower 95% CI of 0.73. These numbers were also sufficient to be able to detect agreement between tests, taking into account the expected type of patients entered into the study, if the assumed proportion of patients randomised to test-directed treatment would be assigned to no chemotherapy varied from 55% to 80% (Table 7). The lower confidence limit for kappa varied from 0.74 to 0.72, when the proportion of patients assigned to having no chemotherapy varied from 55% to 80%.

Introduction to the economic analysis

There is currently interest in using decision modelling earlier in the research and development process for new health-care technology. 146 This is, in part, a response to the high costs and high failure rates of pre-regulatory Phase III trials. A decision model can be used to analyse uncertainties in the current effectiveness and cost-effectiveness evidence. The estimate can then be used to tailor a Phase 3 research design to address uncertainties that pose the most risk for clinical, regulatory and reimbursement decision-makers.

The objective of the economic analysis for OPTIMA prelim was to establish the value of conducting further research into the cost-effectiveness of Oncotype DX or alternative test-directed therapy in the UK in the setting of a main OPTIMA trial. OPTIMA prelim also assesses the feasibility of resource use measurement and the within-trial identification of appropriate health-care costs and health-related quality-of-life information to inform the calculation of QALYs.

The economic model and methods developed for this analysis are described in this section. Details on the interpretation of the results of the economic analysis are given in Chapter 5 (see Economic analysis).

Test selection for inclusion in the main economic (base-case) analysis

The factors that were considered when assessing alternative tests for possible inclusion in the main study included analytical validity or reproducibility, biological plausibility and cost-effectiveness. Evidence must exist that an assay produces reproducible results when it is run more than once on the same sample and for distributed tests, that the results are reproducible between laboratories undertaking the assay. The assignment of patients to chemotherapy or not by a test must also be plausible. For instance, if a HER2 test was used as the primary discriminator within the OPTIMA trial, no patient would be assigned chemotherapy, which may make this test appear highly cost-effective. However, although HER2 is a binary test that identifies a high-risk patient population, HER2-positive patients are excluded from the OPTIMA trial, so the test is not plausible.

A base-case model was specified as the main analysis and from which sensitivity analyses were performed. Tests were selected for inclusion in the base-case analysis if they met all three of the following criteria:

1. sufficient evidence of analytical validity in support of an achievable rollout into routine care in the NHS

2. peer-reviewed evidence published in the public domain for clinical validity in a relevant clinical setting

3. existence of either (a) a global or UK list price in the public domain for the test or (b) a credible way of estimating the cost of performing the test including a list price for all test components.

Tests meeting these criteria were Oncotype DX, MammaPrint and Prosigna. IHC4 was not included owing to concerns about the analytical validity when used as a locally implemented test in NHS laboratories. Further details about this rationale are outlined above (see Chapter 1, Individual multiparameter assays) and elaborated on in the discussion. IHC4 AQUA and MammaTyper were excluded owing to a lack of peer-reviewed published evidence for clinical validity in a relevant clinical setting. MammaTyper was also excluded, as the manufacturer put no price forward. An additional sensitivity analysis (see Sensitivity analysis number 3, all tests included) was conducted that included all tests. Where a price was not available from the manufacturer (as with MammaTyper) this was estimated by the analyst.

Clinical risk prediction

The outcome of the OPTIMA prelim patients will be unknown for 5 years and, therefore, it was necessary to forecast long-term survival outcome for the OPTIMA trial population as an assessment of the population under study and to enable the calculation of lifetime expected QALYs for the economic model. The population included in the OPTIMA prelim study are ER-positive, HER2-negative early-stage breast cancer patients. The clinical risk prediction nomograms NPI,19 Adjuvant! Online (version 8)17 and PREDICT18 were applied to each patient. Both Adjuvant! Online and PREDICT give an estimate of treatment benefit for both endocrine therapy and chemotherapy over a 10-year period. PREDICT, but not Adjuvant! Online, takes HER2 status into account, while only Adjuvant! Online provides an estimate of recurrence-free survival (RFS). Chemotherapy benefits estimated with the nomograms used the intended regimen for that patient, where FEC75–80, FEC90–100, E-CMF and TC are considered to be ‘second-generation’, and FEC-T and FEC-Pw are ‘third-generation’ regimens.

Endocrine therapy was specified as AI for post-menopausal and tamoxifen for premenopausal patients in Adjuvant! Online. Comorbidities for Adjuvant! Online were entered as ‘average for age’. No attempt was made to apply a correction for the HER2-negative status of the OPTIMA population using the inbuilt prognostic tool. Ki67 was entered as ‘unknown’ in PREDICT. Five-year RFS probability was also estimated using both a ‘constant benefit’ model (which assumes that the benefit of chemotherapy is constant across all risk scores and independent of tumour biology as determined in the Early Breast Cancer Trialists’ Collaborative Group meta-analysis)9 and a ‘variable benefit’ model (which assumes that the benefit of chemotherapy varies according to the test risk score, as determined in the SWOG88-14 trial). 62 These two models are described in more detail in sections Model parameters and Sensitivity analysis. The additional benefit that might be gained from having chemotherapy was calculated per patient as the difference in the survival estimates obtained if they were to have both endocrine therapy and chemotherapy, and that for if they only had endocrine therapy. The median and ranges of the estimates from each nomogram for all patients within the RS low group and high groups separately were obtained.

The economic model

A model-based economic evaluation of Oncotype DX was conducted during the design period of the OPTIMA prelim. This served a dual purpose: first, it allowed the model to inform the design of OPTIMA prelim, ensuring that the design and data collection was pertinent for UK NHS decision-making; and, second, it prepared a suitable model that could form the basis of an economic analysis at the end of the feasibility period to aid in the design and selection of appropriate tests for inclusion in the main trial. The original model is described in a separate publication. 15

For the OPTIMA prelim analysis, model parameters have been updated by a systematic literature search and costs have been replaced by contemporary costs, where possible from 2014, adjusted to a common base year of 2012–13 (with conversions for the list prices of tests into GBP made using www.xe.com). An additional health state representing acute myeloid leukaemia secondary to chemotherapy has also been added.

The economic model comprises a time-dependent discrete-state transition model (modified Markov model). A detailed explanation of these methods is provided by Sonnenberg and Beck. 147 The model was used to estimate mean differences in clinical effects, including life-years, QALYs and costs for a hypothetical cohort of women with ER-positive, lymph node-positive breast cancer.

Overview and model structure

The methods for the economic model followed the NICE guidance. 148 The structure of the model was developed by consensus among clinical experts, health economists and medical statisticians. The structure was developed from a previously published model. 149 The first part of the model structure is an instantaneous decision tree that allocates a patient cohort either to standard care, where all patients receive chemotherapy, or to a test that is used to allocate patients to high- or low-recurrence risk groups (Figure 2). Patients in the high-risk group, but not those in the low-risk group, are allocated chemotherapy. The second part is a modified Markov model, with annual cycles, which is used to calculate mean life-years, QALYs and costs per patient in each group (Figure 3). All patients were assumed to receive identical non-chemotherapeutic adjuvant therapies.

FIGURE 2.

Decision tree.

FIGURE 3.

State-transition model.

Patients entered the model at the initiation of adjuvant therapy and were assumed to be disease free. This time point is synchronous with randomisation into OPTIMA prelim. The model assumes that patients then move into a disease-free (follow-up) state; develop a local recurrence (which includes locoregional recurrence) or a distant recurrence; develop congestive cardiac failure, acute myeloid leukaemia; or die of unrelated causes. It was assumed that patients remained disease free until they developed a breast cancer recurrence, cardiac failure, acute myeloid leukaemia or died.

The model assumes that patients who develop a local recurrence can be treated curatively and move into a separate ‘disease-free after recurrence’ state, which carries distinct risks of further distant relapse. For simplicity, the model assumes that patients can develop only one local recurrence. Patients remain in the distant recurrence state until death from breast cancer or death from other causes.

Model parameters

Chemotherapy treatment and toxicity

The chemotherapy regimens and toxicity rates were used to estimate treatment-related costs. The proportions of patients treated with anthracycline plus taxane, anthracycline alone, or taxane alone were estimated from the OPTIMA prelim data. Chemotherapy toxicity rates150 were estimated from landmark chemotherapy clinical trials (Table 10). 151–153 Toxicity rates for FEC-Pw were assumed to be equivalent to FEC-T, and toxicity rates for epirubicin were assumed to be equivalent to FEC. Toxicity rates for FEC75 were assumed to be equivalent to two-thirds the rates for FEC100.

Test costs

Test costs were calculated on a per-sample basis using current list prices and data from manufacturers. Where a list price was not available in the public domain, the manufacturers were asked for an expected UK price. Any anticipated NHS discounts were not considered. Any assumptions used in the cost calculations were based on expert opinion. Costs were converted to 2013 pounds sterling using the following exchange rates: GBP to euros = €0.825, and USD to GBP = £0.60.

All tests were assumed to be exempt from VAT. For tests conducted within the NHS it was assumed that all NHS purchasing was operated under a managed service contract (which excludes VAT); similarly tests conducted by commercial institutions for the NHS are exempt from VAT. 170 For tests conducted within NHS laboratories, labour costs were calculated based on estimates of the overall time to run assay samples, and did not include sectioning time, pathologist time to mark areas for extraction and for reporting. A summary of the per-sample test costs of all tests is provided in Table 17.

A breakdown of the cost calculations for each of the tests is given below.

Oncotype DX: for Oncotype DX, tests are sent to the manufacturer (Genomic Health) to complete and return, with no additional costs to the NHS. The cost for Oncotype DX is based on the manufacturer list price at the time of analysis (£2580).

MammaPrint/BluePrint: tests are sent to the manufacturer to complete and return, with no additional costs to the NHS. The advertised list price, confirmed by manufacturer at the time of analysis, was €2675 excluding VAT, equivalent to £2207.

Prosigna: testing requires the purchase or lease of a NanoString instrument in addition to individual assay kits in order to process samples within the NHS. The cost per test, therefore, depends on the machine capital costs (purchase and services), the assay cost, RNA extraction/preparation and the labour costs. These are summarised below.

1. Capital costs: machine purchase, lifetime and service costs were communicated by the manufacturer at the time of the analysis:

   1. machine purchase – US$285,000 = £171,000

   2. machine expected lifetime = 5 years

   3. service cost – US$15,000/year = US$75,000/5 years = £45,000/5 years

   4. total capital cost per site = £216,000 per 5 years.

The expected number of tests required is 4376 per year (Cancer Registry data171). Assuming five sites within the NHS, 875 tests are required per year, per site. Capital cost per test (five sites) is, therefore, £216,000/5 years/875 tests = £49.37. There is uncertainty around the number of instruments that would be purchased across England. Assuming three sites there is a lower (25th) quartile value of £29.61 and assuming 10 sites there is an upper (75th) quartile value of £98.72.

1. Assay cost:

   1. Prosigna assay – £1277 (manufacturer quoted UK cost)

   2. cartridge sizes – four or 10.

Each cartridge pack includes one quality assurance sample, and an entire cartridge must be used at once; any unused cartridges from the pack are wasted. Thus, a pack of four can run a maximum of three samples plus one quality assurance test, and a pack of 10 can run up to nine samples.

Assuming a 5-day working week, 52 weeks per year, and excluding Christmas and New Year (3 days), there are ≈257 [(52 × 5) – 3] working days in 1 year. Assuming that there are five sites, to complete 875 tests per year requires completing (on average) 3.4 tests per day per site, or 17 per week per site. It is assumed that assays are batched on a weekly basis; therefore, running 17 tests per week requires two cartridges of size 10, with wastage of one sample. We therefore assume that, on average, cartridge sizes of size 10 are used with one sample wasted.

The expected assay cost is, therefore, £1596.25 per test (£1277 × 10/8).

There is uncertainty around the number of samples that would be wasted per cartridge. This is represented in the model using a log-normal distribution with a lower bound of £1418.89 per test (assuming maximum number of nine samples run using cartridge size 10, i.e. £1277 × 10/9) and an upper bound of £5108 per test (assuming minimum number of one samples run using cartridge size 4, i.e. £1277 × 4/1), assuming these bounds are 95% CIs.

1. Labour cost: biomedical scientist time – assuming ‘batching’ of tests in groups of 17 (plus two controls) it is estimated that 11 hours of hands-on time is required for the macrodissection, RNA extraction and assay set-up. This equates to 39 minutes of biomedical scientist time per test. Valued at an Agenda for Change grade 7 technician hourly rate of £22.98, this equates to £14.94 labour cost.

1. RNA extraction/preparation materials:

   1. US$500 for 25 isolations using the Roche kit (F. Hoffmann-La Roche Ltd, Basel, Switzerland) (US$20/sample = £12 per sample; communication with manufacturer).

   2. Total cost per Prosigna test: capital cost per test (£49.37) plus assay cost (£1596.25) plus RNA extraction cost (£12) plus labour cost (£14.94) = £1672.56.

   3. Incorporating uncertainties around the capital cost, assay cost and labour costs gives a log-normal distribution, with mean £1672.50 and SD £50.94 (µ = 7.422, σ = 0.030).

MammaTyper: testing requires the purchase or lease of a Roche LightCycler real-time PCR platform (F. Hoffmann-La Roche Ltd, Basel, Switzerland). The cost per test therefore depends on the machine capital costs (purchase and servicing), the assay cost, RNA extraction, preparation and the labour costs. These are summarised below.

1. Capital costs: capital costs include a Roche Diagnostics LightCycler real-time PCR machine. Purchase and service costs of the LightCycler machine were communicated by Roche at the time of the analysis:

   1. LightCycler LC/Z480 (96-well) machine purchase (F. Hoffmann-La Roche Ltd, Basel, Switzerland) – £23,500

   2. machine expected lifetime – 10 years

   3. service cost – 10% of purchase cost from year 2 = £21,150 (to 10 years)

   4. total capital cost per site = £44,650 over 10 years.

The expected number of tests required is 4376 per year (bespoke analysis by Public Health England Cancer Registry171). Assuming five sites within the NHS, then 875 tests are required per year, per site. Capital cost per test (five sites) is therefore (£44,650 × 5)/(4376 × 10) = £5.102. There is uncertainty around the number of instruments that would be purchased across England. Assuming three sites there is a lower (25th) quartile value of £3.061 and assuming 10 sites there is an upper (75th) quartile value of £10.203.

1. Assay cost:

   1. MammaTyper assay – £1277 (interquartile range £400–1400) (analyst estimated UK price)

   2. MammaTyper tests are purchased in batches of 10 (eight tests + two controls).

Two controls need to be included with each real-time PCR run. Controls can be reused/thawed up to three times.

Assuming a 5-day working week, 52 weeks per year, and excluding Christmas and New Year (3 days), there are ≈257 [(52 × 5)–3] working days in a year. Assuming five sites, to complete 875 tests per year requires completing (on average) 3.4 tests per day per site, or 17 per week per site. It is assumed that assays are batched on a weekly basis. Therefore, running 17 tests per week in a single batch with two controls.

The expected assay cost is, therefore, £1596 per test (price × 10/8) [interquantile range (IQR) £500–1750].

1. Labour cost: biomedical scientist time – assuming ‘batching’ of tests in groups of 17 (plus two controls) it is estimated that 11 hours of hands-on time are required for the macrodissection, RNA extraction and assay set-up. MammaTyper preparation, set-up of master mixes, distribution of master mixes and set-up of real-time instrument is estimated at 1 hour. This equates to 42 minutes of biomedical scientist time per test. Valued at an Agenda for Change grade 7 technician hourly rate of £22.98, this equates to £16.09 labour cost.

2. RNA extraction/preparation materials:

   1. £283 for 50 isolations using the Roche kit (£12 per sample).

Total cost per MammaTyper test: capital cost per test (£5.102) plus assay cost (£1596) plus RNA extraction cost (£12) plus labour cost (£16.09) = £1629.19 (IQR £531.15–1788.29).

Incorporating uncertainties around the capital cost, assay cost and labour costs gives a log-normal distribution, with mean £1629.19 and SD £1905.11 (µ = 6.991, σ = 0.899).

IHC4 AQUA: price provided by the manufacturer in 2014 for in-house/central commercial testing was US$1200 (£720).

IHC4: for IHC4 performed using routine staining methods, costs of staining are relatively simple to estimate; however, owing to the quantitative nature of the IHC4 score, additional pathological assessment is required to accurately estimate ER histoscores, and PgR and Ki67 percentage-positive cells within the narrow bands (30 units or 10%) required.

Within the OPTIMA prelim study, IHC4 was performed in a central laboratory, using TMAs and image analysis, which does not reflect routine diagnostic practice (where whole slides are assessed) and precludes accurate assessment of test costs as they would be in a NHS setting. No formal measurement of the time required by individual pathologists to perform the additional quantification required for this test is available to provide a cost estimate was undertaken within OPTIMA prelim. For the purposes of the economic model, the time taken for IHC4 was estimated by consultation with NHS pathologists and laboratory managers.

It was assumed that all IHC4 testing was conducted at local hospitals and laboratories, using currently available technology. Block selection and retrieval was not costed, as these are already routinely conducted within the NHS. The cost of consultant time was assumed to be £157 per hour based on Personal Social Services Research Unit costs. 163 The calculation of the IHC4 cost per test was as follows:

• ER – £26 (10 minutes) extra pathologist time

• PR – £15 consumables/laboratory costs plus £26 (10 minutes) pathologist time

• Ki67 – £20 consumables/laboratory costs plus £52 (20 minutes) extra pathologist time

• HER2 – no extra cost

• generation of IHC4 report via algorithm – £13 (5 minutes of pathologist time)

• total cost per test = £152.

As there remains uncertainty about this estimate, which is based on expert opinion, it will be represented as an uncertain parameter in the OPTIMA model, with mean £152 and IQR £116–207 (SD £69) (implying that there is a 50% chance that the true cost lies within this range). This is represented in the model by a log-normal distribution with parameters µ= 4.93 and σ= 0.429.

Utilities

A literature review was carried out to update the relevant health utility values for the OPTIMA model. 172 Full details of the literature review and data extraction for utility values can be found in Appendices 1 and 7. The utility parameters used in the model are shown in Table 18.

Sensitivity analyses

Analysis and the handling of uncertainty

The cost-effectiveness analysis was conducted in accordance with the specifications of the NICE reference case. 148 Outcomes for effects are measured in life-years, which represent the mean number of years of life per patient and are measured by the area under the survival curve. Life-years were weighted by estimates of health-related quality of life, which are represented by utility values (such that death is represented by 0 and 1 is the best possible health), to produce QALYs. Cost outcomes were measured as the mean cost per patient. The final cost-effectiveness outcome measure – the ICER – is the difference in expected costs for each cohort divided by the difference in expected effects (cost per QALY). According to NICE, the threshold ICER for an intervention to be considered cost-effective in the UK ranges from £20,000 to £30,000. 148 The analytic perspective was that of the UK NHS; thus, only direct costs to the NHS were considered. The base year for costs was 2013. In the initial model based on the SWOG88-14 trial, the starting age of the patient cohort was 60 years, which is consistent with the average age of patients in the majority of trials in ER-positive, early-stage breast cancer. This was modified in accordance with the findings of OPTIMA prelim. The time horizon was the lifetime of the patient cohort, assuming that all patients had died by age 100 years. Both costs and benefits were discounted at 3.5% as recommended by the NICE reference case. Mathematical programming was implemented using the R statistical programming language (The R Foundation for Statistical computing, Vienna, Austria). The University of Leeds Advanced Research Computing Cluster, which is part of the Yorkshire White Rose High-Performance Computing Grid, was used for the analysis.

Screening and acceptability

A total of 968 patients were reported on the screening logs, of whom 795 were deemed eligible according to local information; however, 45 of these were excluded by clinicians (Figure 6). Of the 750 patients approached by their clinicians about participating in OPTIMA prelim, 350 consented to join the study. This equates to a 47% patient acceptance rate, which was above the target 40%. The acceptance rate ranged from 0% to 100% across the 35 open centres, with 19 (54%) centres having an acceptance rate above the target 40%.

FIGURE 6.

Screening of potential participants.

Patients did not have to give a reason for declining to take part but when a reason was given it was recorded on the screening log. Recorded reasons are summarised in Table 19. Of those who declined to enter the study, 51% did so because they had a strong treatment preference (33% wanted chemotherapy and 18% did not want to have chemotherapy). Some patients who declined to enter the study (8%) chose to pay privately for the Oncotype DX test.

Registered patients who were not randomised

Of the 350 patients who consented to participate in the OPTIMA prelim, 313 had been randomised prior to 3 June 2014 and 15 were undergoing central eligibility confirmation on that date (Figure 7).

FIGURE 7.

The CONSORT diagram.

A total of 22 (6%) registered patients were not randomised (see Figure 7) for the following reasons:

• referring site realised that patient did not meet eligibility criteria (n = 2)

  • HRT was being taken at the time of breast cancer surgery (n = 1)

  • repeat local receptor testing showed tumour to be ER negative (n = 1)

• insufficient invasive tissue to perform Oncotype DX test and therefore ineligible (n = 1)

• withdrawal of patient by their clinician (n = 2)

  • underlying condition that made patient unsuitable for the trial (n = 1)

  • axillary surgery had been delayed and needed to begin chemotherapy immediately (n = 1)

• patient withdrew consent (n = 5)

  • sample submitted for central review did not contain sufficient invasive tumour for testing and patient did not want to wait for another sample to be sent and tested (n = 3)

  • decided to pay for an Oncotype DX test privately (n = 1)

  • decided did not want to receive chemotherapy (n = 1)

• deemed ineligible on central review of receptor status (n = 12).

The discrepancy between local and central determination of receptor status was low at 3.7% (95% CI 1.7% to 5.8%), with only 12 of the 325 registered patients for whom central review of receptors was performed deemed ineligible (Table 20).

Randomised participants

The characteristics of the 313 randomised patients are shown in Table 21. Patients were allocated to the partially blinded trial arms (157 on arm A and 156 on arm B) and their characteristics were well balanced across trial arms.

Withdrawal post randomisation

Four participants withdrew consent for the trial following randomisation but before receiving their treatment allocation (see Figure 7). Of these, two patients withdrew consent because of the delay in treatment allocation. One patient was withdrawn by her clinician because she had received preoperative letrozole and was therefore ineligible. One patient was withdrawn because, prior to treatment allocation, it was found that there was insufficient invasive tumour to obtain an Oncotype DX test and no additional sample was available.

A further three participants withdrew following treatment allocation. One patient was withdrawn by her clinician as a subsequent bone scan revealed metastatic disease and two patients withdrew consent as they chose not to follow the results of the randomisation; one patient decided to pay for Oncotype DX off-study and one did not accept her trial-allocated treatment (chemotherapy).

These seven patients are included in the baseline data, but no further information is available.

Sample handling logistics

The median time between patients signing the OPTIMA prelim consent form to treatment being allocated was 20 days (IQR 16–23 days, range 10–41 days). Treatment allocation took longer than 4 weeks from date of consent for 25 patients (8%). The main reasons for these delays were staffing problems and issues within the central laboratory (10 patients) and obtaining a suitable sample from centres for testing (11 patients). Furthermore, central eligibility testing took substantially longer than the target timeline for two patients and Oncotype DX testing took substantially longer for one patient.

The median time between patients signing the OPTIMA prelim consent form and starting chemotherapy for those allocated to chemotherapy was 4.6 weeks (IQR 4.1–5.3 weeks, range 2.6–11.3 weeks). The proportion of chemotherapy-assigned patients starting treatment within 6 weeks was 91% over the whole recruitment period, but 92% within the last 6 months of recruitment. The delays with allocating chemotherapy because of problems with the tissue samples led to the withdrawal of four registered patients and two randomised patients prior to treatment allocation.

Recruitment conclusions

OPTIMA prelim had three pre-defined success criteria for establishing feasibility, namely that 300 patients would be recruited in no more than 2 years from the first centre opening, and, for the final 150 patients, (1) patient acceptance rate should be at least 40%; (2) recruitment should take no longer than 6 months; and (3) chemotherapy should start within 6 weeks of signing the OPTIMA prelim consent form for no less than 85% of chemotherapy-assigned patients. All of these criteria have been met demonstrating that a large-scale randomised trial to demonstrate the effectiveness of multiparameter test-directed chemotherapy allocation in a high-risk population of patients with ER-positive HER2-negative chemotherapy is feasible in the UK.

Personalised chemotherapy

Perceptions of chemotherapy and the move towards more test-guided treatment, such as that evaluated by the OPTIMA trial, were discussed at length in all the groups. Although a small number of participants saw the benefits in receiving no or less chemotherapy, they felt most patients would want to treat breast cancer as thoroughly as possible, which meant having chemotherapy. Focus group participants acknowledged that test-guided treatment was becoming more commonplace now (e.g. those mentioned by group members included HER2 and ER status testing and tests required for treatment of haematological malignancy). However, they felt that there was not always a clear understanding if and when chemotherapy was needed, and that some patients ignored potential side effects in their enthusiasm for treatment.

Overall, it was felt that there was a need for contemporary information, both generally in the media and more specifically, such as in the OPTIMA prelim participant information leaflet, so that the expectation of chemotherapy, and the view that more chemotherapy is always better and that less is somehow deficient, will be challenged by both patients and within society. One group felt that this was a view perpetuated by health professionals and should be confronted through education. They felt that endocrine therapy, such as tamoxifen, was not, as one participant described, ‘painted in the same light’ by health professionals and was therefore seen to be less important and less effective. Participants stated that evidence-based information must be made widely available to make people aware that every person is different; that there is a multiplicity of breast cancer diagnoses and thus every treatment is different and ‘personalised’. They felt this would manage expectations more appropriately and prevent inappropriate comparisons with other women. They also felt that it needed to be explained that best practice changes over time.

Focus group participants felt that there should be more information made available in the news media and magazines about personalised, test-guided treatments to make the public aware and change perceptions generally about cancer always requiring chemotherapy.

It was agreed that the best way to introduce the study to potential OPTIMA participants would be to explain that breast cancer treatment is always tailored to the individual woman and it should be stressed that the OPTIMA trial involves personalised treatment, rather than less treatment, in any written information about the study.

There was some concern from focus group participants that people would think that less treatment, as prescribed in the OPTIMA trial, was simply a cost-saving measure by the NHS. They felt it should be made explicit to potential study participants that this was not the case.

Deciding to take part in research and the OPTIMA study

There was a consensus that the OPTIMA study seemed to be an appropriate and helpful study from a PPI perspective and focus group participants said they would take part in principle (one woman had actually been offered and consented to OPTIMA prelim). The reasons why they would take part were both general to research and specific to the OPTIMA study. For example, being invited to take part in research made one feel valued and important, and it was seen to make a difference to treatment and potentially help others. It also made individuals feel ‘looked after’ and the additional follow-up and ‘active monitoring’ that was involved, even within the control group, was highly valued. Participants felt that this was specifically the case in the OPTIMA study, in which the regular and active monitoring involved was seen as positive and information given about individual risk was seen as constructive.

The hypothetical question of making an informed choice to take part in the OPTIMA study was appreciated to be very difficult; focus group participants in all three groups wished this to be noted. Describing their own experience of being invited to take part in RCTs, some participants had felt under enormous pressure to make a quick decision. A minority felt that deciding to take part would have caused too much personal anxiety and so they would have preferred to leave the decision to, as described by one participant, ‘those that were qualified’. Another minority felt that they would be deterred from taking part because of the wait to get results and consequent delay in starting treatment. There was a common misconception among focus group participants that chemotherapy had to be instigated very quickly to have the most beneficial effect but they were reassured that, in breast cancer, adjuvant chemotherapy could be delayed for up to 12 weeks. 180 Participants felt that this should be made very clear to potential OPTIMA study participants. Knowing this, they also felt that the complexity of the OPTIMA trial and the possibility of potentially receiving less treatment would mean that the information and consent process would be likely to take longer and research staff should be made aware and of this and trained, if appropriate.

The information provided for the OPTIMA study

Focus group participants from all three groups stressed the importance of an expert multidisciplinary team to manage the OPTIMA study involvement and treatment in which potential study participants could have ready access. In order for the OPTIMA study participants to have confidence in their treatment allocation and in their decision to consent to the study, they agreed they would need confidence in the information they were given and the person giving it.

All focus groups also agreed that an audio-recording of the initial consultation would be positive for potential OPTIMA study participants to take away with them as discussions and information given in consultations often, as one participant described, ‘went in one ear and out of the other’. It was also felt that it was important for the OPTIMA study information to be made available so that it could be discussed with family members and friends away from the hospital environment. Multiple formats, which would allow patient choice and be appropriate for different people, were preferable and formats described included digital versatile discs, leaflets, websites and podcasts. Independent information was also valued and it was felt that this could be hosted by a neutral organisation such as ICPV or a cancer charity. The provision of full information was felt to be important and would encourage rather than discourage recruitment.

There was also some concern about the partial blinding in the OPTIMA study and the inherent lack of information. For example, would trial participants allocated to no chemotherapy worry that they actually had a borderline score? Focus group participants were reassured that borderline scores of the genomic test had been lowered specifically for the purposes of the trial to ensure safety. Focus group participants felt that if potential OPTIMA participants were told this, they might be more likely to consent knowing that there was less risk of a false-negative score. They also felt strongly that potential study participants should be made aware that chemotherapy could be offered at a later date if necessary, as many had been under the impression that the test would predict chemotherapy use indefinitely, which is not the case.

The additional burdens of chemotherapy

Interestingly, focus group participants felt that the important additional burdens of treatment were often ignored by health professionals and were under-reported and therefore not recognised. These were often related to employment and financial issues. As such, the health economics questionnaire that is included in the OPTIMA study was seen to be particularly useful and important. The additional burdens that were discussed in the groups are reproduced in Box 1. The quality-of-life questionnaires that are included as part of the trial were also perceived as highly relevant and valuable, although some individuals felt that certain important and impactful information would not be gathered by the measures proposed, for example the impact of tamoxifen and AIs on sexuality and sexual function.

Patient and public involvement conclusion

Data from the three focus groups and the dissemination of these results have generated invaluable in-depth information about the potential impact of taking part in the OPTIMA study from people who have a direct experience of living with cancer. It has shown us that the study is supported and, in theory, people would be keen to be involved and take part. However, it has highlighted the difficulties involved in accepting randomisation to potentially less treatment than expected. The data that have been generated by the focus group participants have been incorporated into the study design and, although they cannot be truly quantified, have probably contributed to the success of the study.

Qualitative recruitment study data set

In total, 18 of the 32 staff contacted for interview responded: 14 agreed to participate (no withdrawals), two declined outright and two declined the interview at the time, with the offer of being contacted at a later more convenient date. The 14 semistructured interviews were conducted with participants across six centres and included eight oncologists (three of whom were also members of the TMG), four research nurses, one surgeon and a non-recruiting member of the TMG. Six interviews were conducted face to face and eight were conducted via telephone. Interviews were conducted with staff from ‘mid-range’ centres by chance rather than by design. One of the centres included in the QRS had failed to recruit any patients; the remaining centres’ recruitment rates ranged from 0.32 to 1.85 patients per month compared with an overall range of 0 to 2.20 in the study. At least one site from each of the OPTIMA prelim’s geographical clusters participated in interviews.

Of the eight research nurses invited to take part in structured telephone interviews about patient pathways, all agreed to participate and none withdrew.

A total of 36 audio-recordings were obtained from 29 patients’ consultations. First and second oncology consultations were recorded for 7 of the 29 patients. First oncology consultations were available for 21 patients and the second oncology consultation was available for one patient. Twelve different recruiters (10 oncologists and two registrars) led the consultations and audio-recordings were obtained from all geographical clusters.

Phase 1: understanding recruitment issues

Detailed findings are reported in Appendix 8. In summary, the recruitment issues identified through the QRS fell under three overarching themes:

• eligibility processes and patient pathways

• missed opportunities for discussing the trial

• opportunities for improving communication and information provision.

The sources of data outlined earlier (see Qualitative recruitment study data set) were considered in tandem to identify key recruitment challenges in the OPTIMA prelim. Although there was some overlap in identified recruitment challenges, which emerged through different methods of data collection, there was also a tendency for certain methods to be better suited for uncovering certain challenges to recruitment. For instance, challenges relating to communication were largely uncovered through audio-recorded consultations, while issues relating to patient pathways were largely based on information derived from interviews. Some themes, such as recruiters’ levels of equipoise, were supported through both interview data and audio-recorded consultations.

Summary of findings from QRS phase 1 and opportunities for intervention

Phase 1 of the QRS identified a number of challenges to recruitment from the analyses of interviews and recruitment consultations, including opportunities for improvement to discuss with the CfI and TMG with a view to developing a plan of action to be implemented in phase 2. The potential opportunities included:

• Considering introducing screening logs at the stage of eligibility assessment. Screening logs could then record the recruitment process in greater detail, from the first point of identifying a potentially eligible patient, through to the patient’s decision to accept or decline the trial. This would reveal if as many potentially eligible patients as possible were being approached and informed about OPTIMA prelim, and help to identify centre-specific barriers to recruitment.

• Discussing ‘on the ground’ perceptions of equipoise – including discomforts around particular eligibility criteria – to reach a consensus about the need to change the criteria or to discuss these issues more widely.

• Considering whether or not the basis of decisions not to approach patients could be standardised or measured and whether or not they were appropriate, with a view to increasing opportunities for potentially eligible patients to decide for themselves whether or not to participate.

• Considering how to raise the profile of OPTIMA prelim at MDT meetings and among surgeons.

• Providing recruiters with support to explain the study design, justification and details more clearly; promoting early mention of OPTIMA prelim in consultations and encouraging recruiters to frame their explanations of treatment options around the OPTIMA prelim study design.

• Exploring with recruiters how best to conceptualise issues of uncertainty and discuss those with patients in the context of the OPTIMA prelim trial.

• Raising issues about the use of terminology that might not help recruitment, including the use of the words ‘standard’ and ‘experimental’, which suggest the trial arms are not equivalent.

• Exploring with recruiters how best to describe advantages and disadvantages of chemotherapy and the Oncotype DX test so that patients could understand the key issues.

• Exploring with recruiters how best to present issues of delays with Oncotype DX.

• Exploring with recruiters opportunities to discuss the origin of patient preferences for or against chemotherapy and, in appropriate cases, to provide additional information that might alleviate inappropriate concerns – all with the aim of ensuring more informed decision-making about participation in the study.

Phase 2: developing interventions to address recruitment challenges

Identification of potential barriers to recruitment was an iterative process that spanned the entire recruitment period. As themes emerged from data collection, interventions were designed in collaboration with the TMG and delivered to centres as recruitment proceeded. This section will outline the timing and nature of these interventions, and discuss their function relative to the recruitment challenges identified in the previous section. A summary of the interventions delivered is shown below in Table 22.

Phase 3: impact of qualitative recruitment study interventions

There were a number of approaches taken to evaluate the impact of the QRS, although no single method could definitively show the influence interventions had on recruitment. As a consequence, the discussions that follow should be viewed as speculative, although viewing the problem from multiple angles could contribute to a clearer picture than relying on one method alone. In the light of this, three approaches were used to explore the potential impact of the QRS interventions: analysis of how recruitment figures changed relative to interventions; analysis of recruitment figures for centres receiving group feedback versus those that did not; and a qualitative analysis of changes in practice for individual recruiters.

Qualitative recruitment study conclusions

Phase 1 of the QRS provided considerable information about the process of recruitment, as it was enacted in clinical centres participating in OPTIMA prelim. Interviews, audio-recordings and screening logs all provided insights that enabled a detailed and nuanced understanding of the challenges faced by recruiters in explaining the complex issues inherent in the OPTIMA prelim study and its design. These understandings were transmitted to the CfI, TMG and recruiters in a range of ways, including direct discussion of eligibility and information issues with the CfI and TMG, and through individual feedback, group discussions, tips and guidance documents and a revised PIS.

The methods used in this QRS were developed in a number of other RCTs. 140,181,182 Several issues identified as important barriers to recruitment in other trials were also found in OPTIMA prelim, such as recruiters’ discomfort in exploring patient preferences, and the use of problematic terminology in recruitment consultations and trial documentation. 182,183 However, there were also some original barriers to recruitment that were specific to the OPTIMA prelim trial, including recruiters’ discomfort in eligibility criteria and patients’ difficulties understanding the trial design.

The QRS was facilitated by a supportive CfI and TMG, but was hampered by clinical centres being given the opportunity to opt out of the QRS, and by recruiters’ unwillingness to audio-record consultations. These issues have also been found previously. 184 If OPTIMA prelim proceeds to a main trial, clearer support for the QRS as a fully integrated and study-wide recruitment support initiative should be considered and attention given to encouraging (or perhaps expecting) greater commitment to recording consultations. In particular, it was noted that most of the centres participating in the QRS in the current study were mid-range recruiters. Future work would benefit from targeting centres for participation in the QRS on the basis of purposeful sampling methods, where screening logs are used to select centres deemed as ‘high’ and ‘low’ recruiters. This approach depends on clear, complete screening logs that record the entire recruitment process, from initial screening to trial participation. A more detailed discussion of the limitations of the QRS methods, including ideas for optimising future integration of the QRS in the OPTIMA main study, is presented in the Chapter 6, Qualitative recruitment study and Appendix 10.

Characteristics of patients included in the pathology analysis by Oncotype DX recurrence score groups

Of the 302 patients, 247 (82%) had a RS of ≤ 25, 54 (18%) had a RS of > 25 and one patient had insufficient tumour available for Oncotype DX testing, although the sample was suitable for alternative testing. The characteristics of the 301 patients split by high and low RS are given in Table 24. Histological grade (1 or 2 vs. 3) was the only characteristic that was significantly associated with a RS of ≤ 25 or > 25 (odds ratio 12.3, 95% CI 6.3 to 24.4; p < 0.0001).

Comparison of Oncotype DX with ‘conventional’ prognostic risk assessment tools

The actual outcomes for the patients recruited into OPTIMA prelim will not be known for several years and were therefore estimated using a variety of nomograms and models as described above (see Chapter 3, Clinical risk prediction. Table 25 provides the median (range) estimates of overall survival and RFS by RS for the 301 patients with available data, predicted using the nomograms, PREDICT18 (www.predict.nhs.uk), Adjuvant! Online17 (www.adjuvantonline.com) and NPI,19 and also 5-year RFS from a constant benefit model and a variable benefit model.

As expected, both the median predicted overall survival and median RFS rates were poorer for the Oncotype DX high-score group than for the low-score group for all nomograms, although the ranges overlapped between the two groups (Table 25). The median predicted additional benefit of chemotherapy compared with that predicted from endocrine therapy alone was consistently greater for the Oncotype DX high-score group for all nomograms although, again, the ranges overlapped with the low-score group. The variable benefit model, which assumes that the benefit of chemotherapy varies with Oncotype DX scores shows, as expected, the most difference in the benefit of chemotherapy between the categories.

Risk categorisation by different molecular tests

One of the key end points for OPTIMA prelim is the selection of one (or more) multiparameter assay(s) to be used as the primary discriminator for chemotherapy allocation in the main study. Tumour specimens from the 302 patients were therefore analysed using additional tests to help inform this decision. Two groups of multiparameter assays were used. The first group of tests discussed in the section, Oncotype DX, MammaPrint, Prosigna ‘risk of recurrence’, which is weighed for proliferation score and tumour size (ROR_PT; NanoString Inc.) IHC4 and IHC4 AQUA, measure aspects of tumour biology and, based on previous clinical data sets, provide numerical scores for individual patients that reflect recurrence risk. The tests also provide predefined risk categories that divide the population into three to four groups based on the numerical risk score that is part of the test output; MammaPrint provides only a binary risk category but no numerical score. The Prosigna assay alone among the tests incorporates some tumour stage information, namely tumour size (but not nodal status). For the purposes of this analysis, the Oncotype DX scores were divided into two groups (rather than the manufacturer’s three predefined risk groups) with the same RS cut-off point of ≤ 25 versus > 25 used for chemotherapy allocation in the OPTIMA prelim. It must be emphasised that predefined risk categories provided by the tests are specific to that test. Although some have been defined using similar boundaries, the tests have been developed in different patient populations and therefore the actual definitions of the risk category boundaries may not translate between assays.

The second group of assays, BluePrint, Prosigna and MammaTyper, divide tumours into the intrinsic subtypes (see Parket et al. 28 and references therein25–27). MammaTyper further divides luminal B cancers into ‘low’ and ‘high’ risk based on proliferation. The subtype of breast cancer identified by these tests reflects the tumour biology, which in turn influences recurrence risk. These tests do not directly ‘measure’ risk but classify tumours into subtypes, which are known to have different drug sensitivities and natural histories. There is extensive literature that shows that luminal A breast cancer is associated with a significantly better prognosis than the other three intrinsic subtypes. Tumours have, therefore, mostly been divided into luminal A and non-luminal A subtype in this analysis. In practice, the great majority of non-luminal A tumours are categorised as luminal B. Again it must be emphasised that the precise definition of subtype is specific to the relevant assay. MammaTyper classified far fewer OPTIMA prelim patients as luminal A than Prosigna and BluePrint; therefore, on the grounds of clinical applicability (plausibility) we chose to combine the MammaTyper luminal A and low-risk luminal B groups [referred to hereafter as MammaTyper Int. (intermediate)].

The proportion of low-risk tumours predicted by Oncotype DX was higher (82%, 95% CI 78% to 86%) than the proportion predicted as either low or intermediate risk using Prosigna ROR_PT (66%, 95% CI 61% to 71%) or IHC4 (72%, 95% CI 67% to 77%). MammaPrint and IHC4 AQUA identified the fewest low-risk tumours, 61% (95% CI 55% to 67%) and 62% (95% CI 56% to 68%), respectively (Table 26). Prosigna ROR_PT identified 36% (95% CI 31% to 41%) as just low risk, compared with 24% (95% CI 19% to 29%) for IHC4 (see Table 26).

A comparison of kappa statistics between tests with risk categorisation (i.e. Oncotype DX, MammaPrint, ROR_PT, IHC4 and IHC4 AQUA) showed the highest agreement between IHC4 and IHC4 AQUA grouping low/intermediate (κ =0.60; 95% CI 0.50 to 0.70), but this is only slightly higher than the agreement between the classifications using MammaPrint and ROR_PT (low/intermediate, κ = 0.53; 95% CI 0.43 to 0.63) and Oncotype DX and IHC4 (low/intermediate, κ = 0.53; 95% CI 0.41 to 0.65) (Table 27).

What is clear from the data is that discordance between different tests in assigning individual tumours to risk categories is common. For the five tests [Oncotype DX, MammaPrint, Prosigna ROR_PT (low/intermediate), IHC4 (low/intermediate) and IHC4 AQUA (low/low–mid)], only 93 (31%) tumours were classified as low/intermediate risk by all five tests and only 26 (8%) tumours high risk by all five tests. Discordant results between the tests were seen in 183 (61%) tumours although there was agreement between four tests for 94 (31%) tumours. The rates of agreement with other tests were similar for all tests (Table 28). This is consistent with what is known about both (1) the similar prognostic information provided by different tests and (2) differences in which each test measures residual risk (see Chapter 6).

Intrinsic subtypes

While the above tests simply measure biological parameters in order to predict outcomes, the growing recognition that breast cancers can be categorised into discrete biological subgroups (or subtypes) has led to the development of multiparameter prognostication based on these subtypes. Table 29 shows the division of tumours into luminal A or non-luminal A for those multiparameter assays that provide intrinsic subtype information. For illustration, the results for both MammaTyper luminal A and MammaTyper Int., which extends the luminal A group to a combination luminal A and low-risk luminal B, are included. All further analysis involving MammaTyper were performed using MammaTyper Int.

Interestingly, despite all the patients in this study being screened by central testing as ER-positive, HER2-negative, a small proportion of tumours (15 in total) were classified by molecular subtyping as either ‘HER2 enriched’ or ‘basal-like’ by different testing methods. This phenomenon is well recognised,28 although its significance is unclear. Two tumours were classified as basal-like tumour by Prosigna; one of these was also identified as basal by BluePrint and as triple negative by MammaTyper. The six tumours classified as HER2 enriched by Prosigna did not include the seven categorised as HER2 type by either MammaTyper only (n = 4), BluePrint only (n = 2) or both (n = 1). The subtype tests (BluePrint, Prosigna and MammaTyper Int.) identified approximately 60% of tumours as luminal A [BluePrint: 61% (95% CI 55% to 67%), Prosigna 60% (95% CI 54% to 66%) and MammaTyper Int. 62% (95% CI 56% to 68%)].

Despite the close similarity in the proportions of tumours identified as luminal A, only 121 (40%) tumours were classified as luminal A subtype by all three tests and 58 tumours (19%) as all other subtypes. Discordant results across these three tests for subtype classification were seen in 123 (41%) tumours.

Given the data outlined above on subtyping across different test platforms, it is unsurprising that, as predicted, moderate kappa statistics between different tests were observed (Table 30). In a comparison between subtype assignment by BluePrint, Prosigna and MammaTyper Int., kappa statistics ranged from 0.39 (95% CI 0.29 to 0.50) for the comparison of BluePrint and MammaTyper Int. to 0.55 (95% CI 0.45 to 0.64) for the comparison of BluePrint and Prosigna.

Assessing the relationship between the continuous Prosigna subtyping and Prosigna risk of recurrence risk score

Prosigna is unique among the multiparameter assays evaluated in providing both a subtype (luminal A, luminal B, HER2 enriched and basal-like) and a continuous risk score (ROR_PT) with predefined risk categories. It is interesting to compare the information provided by these two measures that are derived from measurement of the expression of an identical set of genes.

All 178 tumours classified as luminal A by Prosigna had a ROR_PT score less than the predefined high-risk cut-off point (see Table 31); 108 (61%) of these lay in the low-risk group and the remainder, 70 (39%), were in the intermediate-risk group. None of the 113 luminal B tumours was classified as low risk; 97 (86%) of these lay in the high-risk group (ROR_PT ≥ 61) (see Table 31), with the remainder in the intermediate-risk group. Interestingly, eight tumours, all of which were centrally confirmed as ER-positive/HER2-negative, were grouped into either the basal-like (n = 2) or HER2-like (n = 6) subtypes by Prosigna, and were intermediate or high risk, respectively, according to ROR_PT score (Table 31).

Assessment of patients’ tests results according to Oncotype DX recurrence score group

Tumours were classified as low score or high score according to Oncotype DX for the purposes of treatment allocation. Table 32 shows how tumours were categorised by the other multiparameter assays according to this division. Of the 247 tumours with an Oncotype DX RS of ≤ 25, 177 (72%) were classified as low risk by MammaPrint, 175 (71%) were classified as luminal A by BluePrint subtype 172 (70%) tumours were categorised as luminal A by Prosigna and 168 (68%) were classified as luminal A by MammaTyper Int. (see Table 32). Conversely, of the 54 tumours with a high-risk RS, 44 (88%) were classified both as high risk by MammaPrint and as non-luminal A BluePrint subtype and 36 (66%) were classified as high-risk luminal B by MammaTyper Int.

Validation model: Oncotype DX versus standard care based on the SWOG88-14 trial

The validation model represents an update of the model, which was originally published in 2012, and which compared Oncotype DX with chemotherapy for all. 15 Results of the validation analysis comparing standard care (chemotherapy for all patients or ‘chemo-for-all’) to Oncotype DX are shown in Table 33. Two potential RS cut-off points for Oncotype DX have been considered: ≤ 25 and cut-off score of < 18, hereafter referred to as Oncotype DX_25 and Oncotype DX_18. The results indicate that Oncotype DX increases QALYs regardless of which cut-off point is used; however, Oncotype DX_25 proves more cost-effective than Oncotype DX_18, as Oncotype DX_25 produces additional QALYs at a lower cost per QALY rate than using a cut-off point of 18. That is, Oncotype DX_25 has a lower ICER than Oncotype DX_18 and, therefore, Oncotype DX_25 is preferred to Oncotype DX_18, thus supporting the cut-off point of 25 used in the OPTIMA prelim trial on health economic grounds.

Results of the analysis including only Oncotype DX_25 versus chemotherapy for all are shown in Table 34. Compared with chemotherapy for all, Oncotype DX_25 is associated with a cost per additional QALY of £2166.

Main economic analysis (base case)

The base-case comparison includes four tests (Oncotype DX, MammaPrint, Prosigna Subtype and Prosigna ROR_PT) and assumes that the relative benefit of chemotherapy is dependent on risk groups and test score. For the analysis we used cut-points dividing tumours into high- or lower-risk groups as follows: Oncotype Dx, RS > 25 vs. ≤ 25; MammaPrint, vendor pre-defined; Prosigna subtype, non-luminal A vs. luminal A; Prosigna ROR_PT, vendor pre-defined score of ≥ 60 vs. < 60. Post-recurrence survival is assumed to be constant regardless of pretreatment with adjuvant chemotherapy and the benefit from chemotherapy is assumed to depend on the test scores. The costs, QALYs and net health benefit of all tests are shown in Table 35 along with the probability of producing more QALYs or costing less than chemotherapy for all.

Considered in isolation, all tests are more cost-effective than standard care alone, since they all lead to an increase in QALYs and all except MammaPrint result in an overall reduction in costs (see Table 35). The net health benefit from all tests is higher than chemotherapy for all, although is fairly similar between tests.

Uncertainty in the cost-effectiveness is represented graphically for each test in comparison with chemotherapy for all in the scatterplot on the incremental cost-effectiveness plane (Figure 8) and the cost-effectiveness acceptability curves in Figure 9.

FIGURE 8.

Scatterplot on the incremental cost-effectiveness plane, comparing each test included in the base-case analysis with chemotherapy for all.

FIGURE 9.

Cost-effectiveness acceptability curves: tests compared with chemotherapy for all.

The results of an incremental analysis, in which tests included in the base case compete with each other, rather than just competing with chemotherapy for all, is presented in the cost-effectiveness acceptability curves in Figure 10. At a willingness-to-pay threshold of £20,000 per QALY Prosigna ROR_PT has the highest probability of being cost-effective at 39%.

FIGURE 10.

Cost-effectiveness acceptability curve for the base-case analysis: tests compared with each other.

Sensitivity analysis number 1

In sensitivity analysis number 1, it is assumed that post-recurrence survival is dependent on previous treatment with adjuvant chemotherapy. Results of sensitivity analysis number 1, including only tests included within the base-case analysis, are shown in Table 36 and in the accompanying cost-effectiveness acceptability curve (Figure 11). It should be noted that this analysis favours Oncotype DX over the other tests on the basis of expected cost-effectiveness, with Prosigna ROR_PT falling into second place in terms of both expected net health benefit and the probability of cost-effectiveness.

FIGURE 11.

Cost-effectiveness acceptability curve for sensitivity analysis number 1.

Sensitivity analysis number 2

In sensitivity analysis number 2, it is assumed that the effect of treatment (i.e. hazard ratio for RFS with chemotherapy compared with no chemotherapy) is constant across test-defined risk groups. It is also assumed that post-recurrence survival is constant across all patients, regardless of pretreatment, as in the base-case analysis. The mean costs, QALYs and net health benefit are presented in Table 37. Chemotherapy for all is more cost-effective than any of the testing options, although there is some uncertainty about this as demonstrated in the cost-effectiveness acceptability curve (Figure 12).

FIGURE 12.

Cost-effectiveness acceptability curve for sensitivity analysis number 2.

Value-of-information analysis for the base-case analysis and sensitivity analyses numbers 1 and 2

The expected value of perfect parameter information is calculated for the 5-year RFS parameters and other parameters that would be informed by the proposed OPTIMA trial (including benefit from chemotherapy and the proportion allocated to high risk). This represents a ceiling on the value of research into these parameters. Each EVPPI calculation is the EVPPI for a comparison between chemotherapy for all and chemotherapy directed by a single alternative test. Tests with higher EVPPI are, therefore, a higher priority for inclusion in further randomised research which has 5-year RFS as an outcome. The EVPPI is presented in Figure 13 and is high for all tests included in the base-case analysis, suggesting high value in further research into test-directed chemotherapy. The expected net benefit, probability of cost-effectiveness and the EVPPI all favour Prosigna (ROR_PT) as the preferred test for both current adoption and inclusion in further research. The EVPPI for MammaPrint is higher than for Oncotype DX, suggesting that this may be the second priority, but that it should not be prioritised over Prosigna. The ranking of tests is similar in sensitivity analysis number 1. For sensitivity analysis number 2 (constant benefit from chemotherapy, irrespective of test score) the EVPPI is much lower for all tests except MammaPrint (see Figure 13).

FIGURE 13.

Value-of-information analysis for the base-case comparison and sensitivity analyses numbers 1 and 2. SA, sensitivity analysis.

Sensitivity analysis number 3

Sensitivity analysis number 3 includes all tests, irrespective of eligibility for inclusion in the base-case analysis. The costs, QALYs and net health benefit are shown in Table 38. The probabilistic results are shown in Figure 14. The results of the probabilistic analysis are presented as a cost-effectiveness acceptability curve compared with chemotherapy for all in Figure 15 and with each other in Figure 16. It can be seen that all three additional tests have a net health benefit higher than that of the tests included in the base-case analysis, suggesting that they would be more cost-effective if adopted at the present time. The results of a value-of-information analysis, which includes all tests as comparators, presented in Figure 17, confirms the base-case conclusion that Prosigna ROR_PT remains the priority for further research, although it is now closely followed by IHC4.

FIGURE 14.

Scatterplot on the incremental cost-effectiveness plane, comparing each test with chemotherapy for all.

FIGURE 15.

Cost-effectiveness acceptability curves for sensitivity analysis number 3: all tests included and compared with chemotherapy for all.

FIGURE 16.

Cost-effectiveness acceptability curves for sensitivity analysis number 3: all tests included and compared with each other.

FIGURE 17.

Sensitivity analysis number 3: value-of-information analysis, all tests included.

Relationship between Adjuvant! Online survival prediction and test scores

The relationship of the Oncotype DX scores and ROR_PT scores with the Adjuvant! Online 10-year overall survival for endocrine therapy is displayed in Figures 18 and 19. These data are presented because of the reliance on outcomes forecast by Adjuvant! Online in the cost-effectiveness model. There is a slightly higher negative correlation between the Adjuvant! Online 10-year overall survival for endocrine therapy and ROR_PT scores (r = –0.39) compared with Oncotype DX RS (r = –0.19), which may be contributing to the higher QALYs in the Prosigna analysis.

FIGURE 18.

Scatterplot of the Adjuvant! Online 10-year overall survival for endocrine therapy only against risk scores: Oncotype DX.

FIGURE 19.

Scatterplot of the Adjuvant! Online 10-year overall survival for endocrine therapy only against risk scores: ROR_PT.

Recruitment and study design

OPTIMA prelim opened to recruitment in late September 2012. The first patient consented to join the study in October 2012. By 3 June 2014, 350 patients had been registered, of whom 313 had been randomised from 35 recruiting centres. The OPTIMA TMG, and the NIHR HTA programme as study funder, always considered that patient recruitment might prove to be difficult. Previous experience of clinical trials involving an experimental arm in which less treatment is given than is standard clinical practice, such as the ProTect trial,140 have experienced difficulties in recruitment. 185,186 Prespecified success targets were designed to demonstrate that recruitment into the proposed main trial was feasible once the number of recruiting centres was scaled up. Most of these applied to the final 6 months of recruitment to allow time to open the requisite number of sites and for research staff to learn how to recruit potential participants. Site selection for the OPTIMA prelim was on the basis of individual invitation rather than an open call. Five main geographical clusters were established, namely North London, East Anglia, the South West, the West Midlands and Scotland. Sites within each cluster included a mixture of cancer centres and district general hospitals. This policy was designed to ensure that feasibility could be demonstrated in a representative selection of UK centres rather than in those that are best described as enthusiasts.

All predefined feasibility conditions have been met, demonstrating that a large-scale effectiveness trial of the use of multiparameter assays as a decision tool for patients with primary ER-positive HER2-negative breast cancer in the UK is indeed achievable. This is despite approximately 80% of patients having lymph node involvement, which places them at comparatively high recurrence risk.

Two of the three other international trials of test-directed chemotherapy decisions have met their recruitment targets. 121,124 The MINDACT study estimated patient risk using information from a clinical risk prediction nomogram, Adjuvant! Online, and a biological test, MammaPrint. 122,186 Patients with discordant risk assessments were randomised to a decision based on one of the two. Eligible patients could have up to three involved axillary lymph nodes but there was no restriction on receptor status. Patients with high-risk disease, determined on the basis of test concordance or by randomisation, were treated with chemotherapy and offered an optional further randomisation between one of two chemotherapy regimens. Similarly, those who had ER-positive disease, whether or not they received chemotherapy, were eligible for optional randomisation between two endocrine treatment regimens. A formal feasibility analysis was published after 800 patients (including a small number from the UK) had been registered, at which time 25% of registered patients had discordant test results. 187 The MINDACT design is very complex, which inevitably created difficulties in explaining the study to patients. There is no published information on the acceptability of the study to patients, although the use of a risk score provided by Adjuvant! Online is likely to have been reassuring. The study recruited approximately 6600 patients. The first analysis is expected to be published in 2016.

The TAILORx trial was conducted in the USA. Eligible patients had ER-positive, HER2-negative tumours no larger than 5 cm in diameter and without lymph node involvement. Registered patients underwent Oncotype DX testing. 121,124 Those with a RS in the range 11–25 were eligible for randomisation between chemotherapy followed by endocrine therapy versus endocrine therapy alone. Non-randomised patients with low or high scores were followed up in registry arms. The study, which has a non-inferiority end point, aimed to enrol 11,248 patients and to randomise approximately 4500; no information is available on the number of eligible patients who accepted randomisation. Most of the patients randomised in the study would be considered at too low a risk to be offered chemotherapy in routine UK clinical practice. The primary analysis is planned to take place in December 2017.

The RxPONDER study is similar in design to TAILORx but includes patients with one to three involved lymph nodes; those with an Oncotype DX RS of ≤ 25 are eligible for randomisation. 123,188 In potential study participants, tumour testing may be performed either through trial registration or independently. An estimated 9400 patients will need to be screened to achieve the randomisation target of 4000. Cost-effectiveness research is an integral component of RxPONDER, which is largely funded by the USA private health insurers. The primary analysis of the study, which began recruitment in 2011, is expected in 2022. No information has been published on the acceptability of the study to patients.

The OPTIMA study has significant design differences to these three studies. Patients with up to nine involved axillary nodes are eligible for randomisation, which by virtue of tumour stage is a higher-risk group than the other studies have allowed. The study is partially blinded to minimise the likelihood of bias in clinician behaviour and to reduce the risk of non-acceptance of treatment allocation; thus, participants allocated to chemotherapy are not informed if they have been randomised to the control arm or have a high-score Oncotype DX test result. Although patients allocated to no chemotherapy are aware that they have a low-score Oncotype DX result, the actual RS is not disclosed to the site or patient. Patients who are premenopausal at diagnosis are routinely treated with ovarian suppression regardless of whether or not they receive chemotherapy, as otherwise a chemotherapy-induced menopause, which is difficult to identify reliably, would be a potential source of inequality between the treatment arms. All of these features have been identified in screening logs as reasons for potential participants choosing not to join the study. Despite this, overall patient acceptance identified from the screening log was 47%, which exceeds the protocol-defined target of 40%. Of those patients who went on to be randomised, 20% of patients had node-negative disease while 15% had four or more involved nodes. A total of 31% of participants were reported to be pre- or perimenopausal at the time of randomisation. Thus, the OPTIMA prelim has demonstrated that it is possible to conduct a study with design features that will maximise the chances that the study will deliver an unbiased result despite their difficulties for potential participants.

The reasons for the choice of an Oncotype DX RS cut-off point of ≤ 25 vs. > 25 for chemotherapy have been explained in Chapter 2, Methods. To put this into context, a RS of 25 equates to a 16% risk of developing metastatic disease over 10 years in a node-negative breast cancer population treated with tamoxifen for 5 years but not chemotherapy,41 often termed residual risk. Any estimation of chemotherapy benefit at this level of risk, however, must take into account that approximately one-third of this risk relates to chemotherapy-insensitive late (beyond 5 years) relapse. 9

Clinical decisions on chemotherapy use are influenced by residual risk and the predicted likelihood of chemotherapy benefit, meaning the prevention of distant recurrence and death, which is related to residual risk. There is no universally accepted level of predicted benefit that informs practice. Perception of clinical risk and what constitutes a meaningful benefit from medical treatment is very much a matter of individual judgement for both patients and clinicians. In the UK, most oncologists will discuss breast cancer chemotherapy with patients who have a 3–5% predicted chance of benefit and recommend its use for those with > 5% chance of benefit. At this level, although there is a significant population-wide treatment benefit, the probability that an individual patient will benefit is modest.

Risk of recurrence is influenced by clinical stage as well as by tumour biology, and adding stage information to multiparameter assay output improves the prediction of residual risk (e.g. see Tang et al. 64,70). Should the hypothesis underlying the OPTIMA study prove correct, however, then patients with tumours with a low multiparameter risk score are unlikely to benefit to a meaningful degree from chemotherapy despite a potentially high risk resulting from adverse stage, particularly lymph node involvement. This is the conclusion from the retrospective analysis of the NSABP B-20 trial. 64,70 It is difficult to make a direct estimate of the likely level of chemotherapy benefit in the largely node-positive OPTIMA population at the RS threshold of 25. Estimates based on NSABP B-2061 and SWOG88-1462 studies performed in node-negative and node-positive populations, respectively, suggest a reduction in 10-year breast cancer mortality risk of in the order 5%. If a RS of 25 is indeed close to the score at which the Oncotype DX predicts significant chemotherapy sensitivity, then nodal status and other stage information are unlikely to have much influence on this estimate. In contrast, for patients at risk by virtue of adverse stage and whose tumours have higher scores predicting chemotherapy sensitivity in addition to risk, chemotherapy is likely to offer a very substantial benefit. Similar considerations should apply to other multiparameter assays.

Patient focus groups

Three patient focus groups were held to assess the acceptability and decision-making for women who may have been offered randomisation into the OPTIMA prelim. These groups also reviewed the patient information, leaflets and consent form. The study was acceptable to the majority of participants. Some had clear preference for chemotherapy as that is what they had been offered and some discussed how they felt not being offered chemotherapy. The trial design using a ‘test’ to decide treatment was acceptable and most felt that a personalised approach using these multiparameter tests would be a preferred option. The results of the focus groups provided some triangulation on the QRS for the TMG.

Qualitative recruitment study

Characteristics of the OPTIMA prelim population

An important consideration is whether or not the population recruited represents those to whom the research question is relevant. Although the prevalence of high-grade tumours in the OPTIMA-eligible population is unknown, there seems to be an excess of histological grade 2 lesions compared with grade 3 tumours in the OPTIMA prelim; 6%, 67% and 27% of lesions were reported to be grade 1, 2 and 3, respectively. For reference, in an historic and completely unselected series of patients presenting with symptomatic early breast cancer the ratio of grade 1, 2 and 3 lesions was reported to be 2 : 3 : 5. 189 In mixed screening and symptomatic series of ER-positive tumours, lower proportions of high-grade tumours (e.g. 3 : 4 : 3190) have been described. It is likely that the proportion of high-grade tumours in the OPTIMA-eligible population lies between these two proportions. Certainly the 18% of tumours with an Oncotype DX score of ≥ 25 was lower than the predicted 30% in our population. There appears, however, to be little difference from that expected in tumour type distribution, with 70% of cancers of no special type/ductal in large series, compared190 with 71% in the OPTIMA prelim. One possible explanation for the apparently lower than expected recruitment of patients with more aggressive tumours is an element of patient selection by recruiting clinicians. It is important to ensure that the study is not skewed towards patients with very low-biological-risk tumours if it is to be relevant to the entire population. Recruiting clinicians therefore need to be reassured that multiparameter assays reliably classify tumour risk irrespective of histological grade; going forwards to the main trial, this will require education.

Central review of receptor status

We undertook routine central retesting of receptor status in the OPTIMA prelim because of concern about a potential for bias if patients with either ER-negative or HER2-positive disease based on local histopathology testing were included in the study. There are many possible causes for this scenario, including a well-recognised low incidence of HER2 gene amplification among tumours that are considered HER2-negative by immunohistochemistry in first step of the standard two-stage HER2 testing procedure recommended in UK guidelines. 191 Any participants with ER-negative or HER2-positive disease would be disadvantaged if allocated to endocrine therapy only, and such imbalance in treatment allocation would be a potential source of confounding. To minimise this risk, central review to confirm the referring pathology laboratory results for ER positivity and HER2 negativity was undertaken with immunohistochemistry for ER, and with FISH for HER2 status. The former was scored using the Allred system (% staining and average intensity; range 0–8) with a cut-off point of ≥ 3 defined as positive. HER2 positivity with FISH was defined as a ratio of HER2 to chromosome 17 centromeric probe copy numbers of > 2.0 (2.0–2.2 defined as borderline but amplified). This confirmed the eligibility of all but 12 of 325 patients (96.3%) tested on central review of receptor status.

Reassuringly, in the OPTIMA prelim, only 1.2% (n = 4) of patients who locally were thought to have ER-positive disease were found centrally to have ER-negative tumours. A total of 2.2% (n = 7) had tumours with at least borderline amplification HER2 (three borderline but amplified, four amplified). Of these, one woman had a tumour that was both ER negative and HER2 positive centrally. Two additional tumours were heterogeneous, with discrete subpopulations of ER-positive and ER-negative cells and were, for this reason, deemed ineligible (< 1%).

This level of discrepancy is lower than reported in the majority of clinical trials of early breast cancer, although many of these have documented central review of HER2-positive (rather than HER2-negative) disease. For example, among cases regarded locally as ER-positive, 4% and 16%, respectively, were centrally redefined as ER-negative in two reviewing centres in Italy and the USA in the BIG 2-06/NCCTG N063D trial. 192 Similarly, in a centrally reviewed consecutive series of node-negative breast cancer from the Netherlands, 4% and 5% disagreement was seen for ER and HER2, respectively, between local and central tests (n = 694). 193 The low level of disagreement in receptor status between local and central laboratories that we report in this study reflects improvements in standardisation of receptor testing and the high levels of quality assurance in the UK including, for example, mandatory participation in National External Quality Assessment Service for Immunohistochemistry. 194

The overall 3.7% (95% CI 1.7% to 5.8%) incidence of discrepancy between local and central laboratories is reassuringly low. Although none of these cases underwent Oncotype DX testing, it is likely that such cases would generate high-risk RS and consequently be allocated chemotherapy. Of the seven (2.2%) patients who were found to have tumours that were HER2 amplified, three were classified as borderline while four were clearly amplified. Such patients would ordinarily be treated with adjuvant trastuzumab but received this treatment only by virtue of agreeing to join the OPTIMA prelim. The evidence of benefit for adjuvant trastuzumab treatment for cases with equivocal HER2 amplification195 is controversial.

Therefore, based on the experience in 325 tumours centrally tested for ER and HER2, given the discordant rates of ≤ 2.2% for each receptor separately, our experience with central review of ER and HER2 status suggests that this is unnecessary in a large Phase 3 trial.

Multiparameter assays

Oncotype DX was chosen as the primary discriminator for decisions on treatment allocation for OPTIMA prelim as this was judged most likely among the available tests to be acceptable to patients and clinicians alike. The evidence supporting the use of Oncotype DX was and remains significantly stronger than for potential alternatives. There is widespread familiarity with the test as the result of its successful marketing in the UK private health-care sector. This has been reinforced by the NICE DG10 guidance,117 which provides it with a stamp of authority that its competitors lack. Nevertheless there remain significant uncertainties in relation to the use of Oncotype DX and other multiparameter assays that justify the OPTIMA study.

One particular issue in relation to Oncotype DX is the choice of a cut-off point above which patients should be advised to have chemotherapy treatment. The test output is calibrated as risk of distant recurrence at 10 years in patients with ER-positive disease treated with tamoxifen alone. The Oxford Overview has clearly shown that there is no evidence that the recurrence risk in breast cancer patients who remain disease free 5 years after diagnosis is any different between those who did or did not receive adjuvant chemotherapy. This is despite the observation that one-third of the recurrences during the 10 years after diagnosis occur in the second half of this period. 9 Therefore, estimation of likelihood of chemotherapy benefit in relation to test output is inherently difficult. In the absence of robust data on the effectiveness of chemotherapy in relation to Oncotype DX RS, the choice of a cut-off point is always going to be somewhat arbitrary. The rationale for selection of a cut-off point of 25 is described above (see Chapter 2, Methods).

As there is no established method for the comparison between multiparameter assay platforms being attempted in the OPTIMA prelim, the analysis has required the development of a methodology. There is much uncertainty inherent in this, which is discussed in Chapter 6, Pathology study results. A particular issue is that the assays have mostly developed to indicate residual risk of relapse after 10 years of endocrine therapy and their predefined risk groupings reflect this. The limitations of this definition in relation to decisions on chemotherapy use are described in the previous paragraph. As there is no standardised definition of risk of relapse, the equivalent cut-off points for the available assays differ. This means that the proportions of cases assigned by the assays to low-risk (or not high risk where there are multiple divisions) and high-risk groups differ, which for the purposes of OPTIMA means that proportions of patients advised chemotherapy differ. Consequently, the assumptions for treatment benefit made in the health economics analysis are uncertain. As a result of this, any conclusions that can be drawn from this analysis are, at best, an estimate and require formal testing in a prospective study.

Pathology study results

This study within the OPTIMA prelim represents, to our knowledge, one of the few and certainly the most comprehensive ‘head-to-head’ comparisons of multiple tests designed to utilise tumour biology to predict patient residual risk. Rarely has there been the opportunity to compare results of multiple tests on a patient-by-patient basis to explore differences in risk categorisation. However, it should be remembered in the interpretation of these data that this study was designed with limited and focused goals. We are aware of the intense interest around this study and wish to ensure that results are interpreted in the appropriate context.

The second objective of this study, after demonstrating feasibility, was to aid the selection of candidate biological risk stratification tools for inclusion in the main OPTIMA trial. The OPTIMA trial aims to test whether or not, for ER-positive HER2-negative patients deemed to be clinically high risk, an additional biological predictor of risk can inform treatment. Specifically, for the clinically high-risk population is it appropriate to assign biologically low-risk patients to endocrine therapy without chemotherapy while retaining chemotherapy for clinically and biologically high-risk patients?

A key challenge for the OPTIMA triallists remains: which of the several validated diagnostic assays that assess biological risk should be used to address the trial question?

In addressing this second goal, we recognise a number of important factors, which both inform and limit the interpretation of results from the OPTIMA prelim pathology analysis.

1. OPTIMA prelim was neither designed nor powered to evaluate or compare the prognostic value of the tests included in the study. The tests that have been evaluated in the study have been extensively validated prior to inclusion in this study for their ability to inform risk, mostly following endocrine treatment. In addition, a number of elegant studies from the transATAC study group have shown that each test provides broadly similar risk discrimination in luminal breast cancers. 43,55,56 However, all tests, while of significant value in informing patient choice, are only modestly predictive of the risk of relapse at the individual patient level. Within the validation studies for each test there are patients assigned to ‘low-risk’ groups who relapse and die of their disease, and in all cases the majority of patients who are ‘high-risk’ do not in fact relapse and die from their disease. These tests therefore represent a significant step towards personalised medicine but are not yet perfect. This is important since it implies that discordance in risk estimates between tests that measure risk in different ways may not be unexpected.

2. The OPTIMA prelim study sought to evaluate diagnostic tests in a high-risk patient group as defined by tumour stage; for some tests this means that results from the OPTIMA prelim are using risk scores designed for node-negative disease in a predominantly node-positive population. This is in line with current thinking that ‘biological’ risk measures are informative across, and to a degree independent of, stage of disease. This thinking is supported by the fact that only grade, in the current study, is significantly associated with biological risk defined by multiparameter assay. This association may be explained by the fact that both grade and most biological risk tools assess tumour proliferation.

The key end point required for an appropriate choice between tests for the OPTIMA trial, which was also a key objective of the current OPTIMA prelim analysis, was to identify the proportion of cases assigned as high risk (or low risk) by each test, accepting that for each test this was a ‘true result’. As regards biological risk, since we cannot assess outcome, each test must function as its own gold standard.

Each test for which data are available provides an individual patient risk assessment. If all, or indeed any, of the candidate tests were 100% accurate in predicting risk (recurrence) it would be appropriate to use concordance estimates or kappa statistics to compare tests. However, in the absence of such a ‘gold standard’, kappa comparisons between tests simply reflect agreement between multiple modestly accurate tests. The low concordance between tests reflect the fact that the tests are measuring different genes, using different technology and highlight the problems of predicting recurrence risk based on the biology and management of the tumour.

We have presented comparisons between two different types of test results: (1) molecular subtype comparisons that group tests into luminal A, luminal B, etc. and (2) risk assessments where tests have included a risk prediction (either categorical or continuous). For tests generating continuous risk scores we have grouped patients into ‘low risk’ (patients who might safely avoid chemotherapy in the OPTIMA study) or ‘high risk’ (patients for whom their biological risk would suggest that chemotherapy should be given). For example, Oncotype DX provides both a continuous risk score and assigns patients to ‘low’, ‘intermediate’ or ‘high’ relapse risk. For the purposes of the OPTIMA trial we have combined the OPTIMA prelim cases classified by Oncotype DX as ‘low‘ (54% of cases) and ‘intermediate’ risk (28% of cases) into a single category of ‘low’ risk of relapse/low probability of chemotherapy benefit (called low score for convenience). For other tests, patients were categorised on the basis of pre-established cut-off points or, in the cases of tests providing subtype information (e.g. BluePrint), on the basis of subtype (luminal A vs. all others).

Discussion of the health economic methodology and results

The economic analysis excluded IHC4 because of lack of cost information and because of concern about its analytical validity. Although MammaTyper has demonstrated analytical validity, there are no published clinical validation data and no cost information available. There are no published data on either the analytical or the clinical validity of IHC4 AQUA.

The results of the economic analysis require careful interpretation. The differences in the expected value between the alternative tests are relatively small, and the uncertainty in the evidence for all tests is substantial. Even the test with the best current clinical evidence base, Oncotype DX, is less (rather than more) likely to be a good use of limited NHS resources when considered in comparison with alternative tests, according to NICE’s standard value criteria. 148,196

MammaPrint is expected to be dominated by, that is produce fewer health benefits and cost more than, other options in all of the scenarios, and the probability of MammaPrint being cost-effective never exceeds single figures. On this basis, it seems reasonable to exclude MammaPrint from consideration for current adoption, as such a low proportion of simulations indicate that choosing MammaPrint would be a suboptional decision. One can speculate why this is the case, but it may be that because this is a dichotomous test, developed across the whole spectrum of breast cancer subtypes, it may not have been optimised for the patient cohort of those with a luminal breast cancer, which is the target population for the OPTIMA study. Despite its apparent low expected cost-effectiveness, MammaPrint is associated with considerable uncertainty and this manifests as research value in the value-of-information analysis.

The other strong conclusion is that test-directed chemotherapy appears to be cost-effective, regardless of the choice of tests. This supports the further development of an evidence base sufficient for robust adoptions decision-making.

An additional question is whether or not there is value in further research on some of these tests. The estimates of the value of further research for Oncotype, MammaPrint and Prosigna tests are sufficient to justify UK NHS investment in studies capable of reducing the test-specific uncertainty. The range of analyses presented here show that there is substantial uncertainty about the performance of each of the tests and there is also substantial uncertainty about the appropriate characterisation of the long-term outcome, as demonstrated by the variation in the results of the analyses across the different scenarios. Addressing the latter source of uncertainty within the trial may require a trial design with long-term detailed follow-up. It may be efficient to reduce the uncertainty on these factors through analysis of existing cancer registry data, in addition to within a clinical trial, as information on cancer recurrence and post-recurrence treatments starts to be captured in national data sets.

It is important to note that differing conclusions are drawn regarding the optimal use of Prosigna (ROR_PT or Subtype) depending on which sensitivity analysis is correct. This kind of model-dependent structural uncertainty is difficult to characterise in the value-of-information analysis. Given the lack of consensus between models it is difficult to recommend an optimal use of Prosigna based on purely economic grounds. Ideally, different ROR_PT cut-off points should be explored in further prospective research. If a single cut-off point needs to be chosen, as appears to be the case in light of the manufacturer’s marketing strategy, then it may be reasonable to base its choice using clinical or mechanistic justification.

The final notable observation from these results is that the value of undertaking further research around Oncotype DX, when the comparator is ‘chemotherapy for all’, is considerably lower than for the comparator technologies in all scenarios. This difference is likely to be driven by the higher uncertainties around the longer-term clinical outcomes, which is amplified in the model by discordance with Oncotype DX. The price differential will also be contributing, with Oncotype DX being more expensive per test than Prosigna. Again, further sensitivity analysis is required to fully understand what is driving the value of information and a more granular EVPPI analysis will enhance understanding.

The economic analysis conducted at the end of the OPTIMA prelim has presented a number of challenges. The use of value-of-information analysis to inform a design decision between a feasibility study and a fully powered RCT is a new application for this method, much advocated by methodologists in the UK and worldwide. 146 The need for influential assumptions within the model prior to the availability of evidence from a definitive trial necessitates extensive scenario analysis to test these assumptions. This has resulted in the need for three different model specifications and multiple scenarios, making the results complex to understand. The recommendation of Prosigna as most valuable for further research has withstood testing of the main assumptions, suggesting that it is a robust recommendation.

A particular additional challenge in the use of value-of-information analysis within a short analysis period is the computational expense. Each EVPPI calculation in this analysis, which uses 5000 Monte Carlo simulations in both the inner and outer loops, required in excess of 7 months of processing time using a fast processor. The OPTIMA prelim analysis was therefore only possible using a high-performance computing facility (White Rose Grid: www.whiterose.ac.uk/projectswhite-rose-grid/) to implement analyses in parallel.

A desirable further sensitivity analysis is exploration of the influence of Adjuvant! Online as a forecasting tool in the main economic analysis. It is possible that the reliance on this tool has biased the results in favour of tests for which the risk score correlates most highly with it. This is exemplified in Figures 18 and 19, which show that Prosigna ROR_PT is more highly correlated with Adjuvant! Online predictions than is Oncotype DX. The lack of CIs around Adjuvant! Online estimates also means that uncertainty will be underestimated in the model. The validation model is protected from this bias and further work exploring alternative tests using this model would be warranted. It is disappointing that validated alternatives to Adjuvant! Online reporting RFS are not available.

Recommendations for further research

OPTIMA prelim has succeeded in its aims of demonstrating the feasibility, within the UK, of a large-scale clinical trial to establish a method of selecting patients with moderate- or high-risk hormone-sensitive primary breast cancer who are likely to benefit or not benefit from post-operative chemotherapy and to establish the cost-effectiveness of alternative test-guided treatment strategies. Existing data that support the use of multiparameter assays as an aid to making chemotherapy decisions are based on retrospective analysis of historical data. Very few data are available on the clinical validity of multiparameter assays in patients with lymph node involvement, which is the population in whom the cost-effectiveness of test-guided treatment decisions is likely to be greatest. The NICE DG10 guidance,117 which applies only to women with lymph node-negative breast cancer, recommends that further research on the ability of multiparameter assays to predict chemotherapy benefit be performed. A prospective clinical trial designed to answer these questions would be of benefit to the NHS and should be undertaken. It should be noted that, from the perspective of the NHS, the value of information for further research into Prosigna is higher than for the other two assays that were evaluated in the base-case analysis. The discrepancies demonstrated between the tests that were evaluated in this study at an individual patient/sample level show that there is substantial scope for refinement of exiting multiparameter assays.

Implications for practice

OPTIMA prelim was not designed and is not expected to generate any information that will result in a change in clinical practice. The finding of significant disagreement between multiparameter assays at an individual patient level should give clinicians currently using these technologies in their clinical practice pause for thought. As there is no established ‘best test’, not only are there limitations to the evidence base supporting multiparameter assay use but there are also limitations to the technology that is currently available. Nevertheless, the data generated by this study do not in any way undermine existing data supporting multiparameter assays use to guide chemotherapy decisions.

Trial Management Group

Robert C Stein (Consultant Medial Oncologist and Honorary Senior Lecturer, University College London Hospitals, London, UK) is CfI and lead of clinical aspects of the trial. He made substantial contributions to the concept and design of the study, its day-to-day management and overall conduct, and to data analysis and the writing of this report.

Janet A Dunn (Professor of Clinical Trials, University of Warwick, UK) was the CTU lead and senior statistician for the study. She substantially contributed to the trial design and conduct, including day-to-day management and monitoring as well as the statistical analysis plan. She contributed to the preparation of the methods and results section of the report. She was responsible for the patient focus groups.

John MS Bartlett (Programme Director and Honorary Professor, Ontario Institute of Cancer Research, Canada) was the translational research lead for the trial. He contributed to study design and managed tissue banking, the establishment of commercial relationships for undertaking multiparameter assays, the performance of laboratory assays and data analysis. He contributed to the relevant sections of the report.

Amy F Campbell (Trial Manager, Warwick CTU University of Warwick, UK) was responsible for the day-to-day management of the trial, monitored data collection, sample collection and analysis. She contributed to the writing of the methods and results sections and collated the report.

Andrea Marshall* (Principal Research Fellow in Medical Statistics, University of Warwick, UK) is the trial statistician. She contributed to the statistical analysis plan and conducted the statistical analysis of the data, and wrote the statistical methods and results sections of this report.

Peter Hall* (NIHR Clinical Lecturer Medical Oncology and Health Economics, University of Leeds, UK) contributed to the health economics aspects of study design, produced the health economics analysis plan, undertook the health economics analysis and contributed to the writing of the health economics aspects of the methods, results and discussion sections of this report.

Leila Rooshenas* (Research Associate, University of Bristol, UK) was the researcher for the qualitative research substudy. She conducted all interviews with professionals, analysed audio-recordings and provided feedback to centres. She wrote the QRS methods and results sections of this report.

Adrienne Morgan (Patient Advocate and Chairperson of ICPV Trustees) contributed to the design and overall conduct of the study, and to the preparation of the sections of the report discussing patient focus groups.

Christopher Poole (Professor of Medical Oncology, University of Warwick, UK) contributed to the design of the trial and its overall conduct, and to the preparation of the background and discussion sections of this report.

Sarah E Pinder (Professor of Breast Pathology, King’s College London, UK) contributed to the trial design, advised on pathology aspects of trial conduct and prepared important intellectual content for the discussion section of the report.

David A Cameron (Professor of Oncology and Head of Cancer Services, University of Edinburgh, UK) contributed to the study design and its overall conduct, and advised on the health economics and clinical aspects of the trial.

Nigel Stallard (Professor of Medical Statistics, University of Warwick, UK) advised on statistical analysis of diagnostic testing for the study design, and contributed to the trials design and statistical sections of this report.

Jenny L Donovan (Professor of Social Medicine, University of Bristol, UK) designed the QRS and revised the QRS methods and results sections of this report for important intellectual content.

Christopher McCabe (Professor of Health Economics, University of Alberta, Canada) contributed to the health economics aspects of study design. He advised on the health economic analysis plan and revised the health economic sections of this report for important intellectual content.

Luke Hughes-Davies (Consultant Oncologist, Addenbrooke’s Hospital, Cambridge, UK) is co-CfI. He contributed to the concept and design of the study, its day-to-day management and overall conduct, and to the preparation of the background and discussion sections of this report.

Andreas Makris (Consultant Clinical Oncologist, Mount Vernon Hospital, Northwood, UK) is co-CfI. He contributed to the concept and design of the study, its day-to-day management and overall conduct and contributed to the preparation of the background and discussion sections of this report.

*These authors made equal contributions to the analysis of data and their presentation in this report.

Other members of the Trial Management Group

Peter Canney (Consultant Oncologist, retired) contributed to the study concept and design.

Helena Earl (Reader in Clinical Cancer Medicine, University of Cambridge, UK) contributed to the study design and its overall conduct.

Mary Falzon (Consultant Histopathologist, UCL Hospitals, London, UK) was responsible for central eligibility testing and advised on pathology aspects of trial conduct.

Adele Francis (Consultant Breast Surgeon, University Hospital Birmingham, UK) advised on surgical aspects of study design and contributed to its overall conduct.

Victoria Harmer (Breast Care Nurse, Imperial College NHS Healthcare Trust, London, UK) advised on nursing aspects of the study.

Helen Higgins (Senior Project Manager Warwick CTU, University of Warwick, UK) managed the study opening and contributed its day-to-day management.

Claire Hulme (Health Economist, Leeds Institute of Health Sciences, Leeds, UK) advised on patient questionnaires and health-care utilisation analysis.

Iain Macpherson (Clinical Senior Lecturer in Medical Oncology, Beatson West of Scotland Cancer Centre, Glasgow, UK) contributed to the overall conduct of the study.

Daniel Rea (Senior Lecturer in Medical Oncology, University of Birmingham, UK) contributed to the study design and its overall conduct.

Participating centres

The following centres and principal investigators contributed patients to the trial:

Addenbrooke’s Hospital: Dr Luke Hughes-Davies.

Alexandra Hospital, Redditch: Dr Denise Hrouda.

Ayr Hospital: Dr Graeme Lumsden.

Barnet Hospital: Dr Rob Stein.

Beatson West of Scotland Cancer Centre: Dr Iain Macpherson.

Bedford Hospital (Primrose Oncology Unit): Dr Sarah Smith.

Bristol Haematology and Oncology Centre: Dr Jeremey Braybrooke.

City Hospital, Birmingham: Dr Daniel Rea.

Dumfries and Galloway Royal Infirmary: Dr Tamsin Evans

Forth Valley Royal Hospital: Dr Judith Fraser.

Hairmyres Hospital, Lanarkshire: Dr Grainne Dunn.

Inverclyde Royal Hospital: Dr Abdulla Alhasso.

Luton and Dunstable University Hospital: Dr Mei-Lin Ah-See.

Mount Vernon Hospital: Dr Andreas Makris.

Musgrove Park Hospital: Dr John Graham.

Norfolk and Norwich University Hospital: Dr Adrian Hartnett.

Northwick Park Hospital: Dr Andreas Makris.

Peterborough City Hospital: Dr Karen McAdam.

Queen Elizabeth Hospital, Birmingham: Dr Daniel Rea.

Queen Elizabeth Hospital, King’s Lynn: Dr Margaret Daly.

Royal Alexandra Hospital: Dr Abdulla Alhasso.

Royal Devon and Exeter Hospital: Dr David Hwang.

Royal Glamorgan Hospital: Dr Jacinta Abraham.

Royal United Hospital Bath: Dr Mark Beresford.

St Bartholomew’s Hospital: Dr Rebecca Roylance.

The Christie: Dr Anne Armstrong.

The Woodlands Centre, Hinchingbrooke: Dr Cheryl Palmer.

Torbay Hospital: Dr Andrew Goodman.

University Hospital Coventry: Professor Christopher Poole.

University Hospital Crosshouse: Dr Graeme Lumsden.

Velindre Cancer Centre: Dr Annabel Borley.

Western General Hospital, Edinburgh: Dr Angela Bowman.

Wishaw General Hospital, Lanarkshire: Dr Jonathan Hicks.

Yeovil District Hospital: Dr Urmila Barthakur.

York District Hospital: Dr Andrew Proctor.

Independent members of the Trial Steering Committee

Professor Tim Maughan, Professor of Clinical Oncology, University of Oxford (chairperson); Professor Jon Deeks, Professor of Biostatistics, University of Birmingham; Professor Robert Leonard, Professor of Oncology, Imperial College, London; Dr Colin Purdie, Consultant Pathologist, Ninewells Hospital and Medical School, Dundee; Ms Julia Simister, Research Network Manager, Surrey West Sussex and Hampshire Cancer Research Network; and Professor Ian Tomlinson, Professor of Molecular and Population Genetics, Nuffield Department of Clinical Medicine, University of Oxford.

Independent Data Monitoring Committee members

Professor Lajos Pusztai, Chief of Breast Medical Oncology, Yale Cancer Centre, Yale School of Medicine (Chairperson); Professor Doug Altman, Director, Centre for Statistics in Medicine, University of Oxford; and Professor Giuseppe Viale, Director, Division of Pathology, European Institute of Oncology, University of Milan.

Other contributors

Health state utilities for breast cancer: update on previous search done at the School of Health and Related Research in 2009

EMBASE Classic and EMBASE

Date range searched: 1947 to 10 December 2012.

Google Scholar (advanced screen)

At least one of the following: HUI OR TTO OR SG OR EQ-5D OR SF-6D OR SF-12 OR VAS OR QWB OR HALex OR Euroqol

Exact phrases “breast cancer” AND “health state utilities”

Limits Material since 2009 and in subject areas: Medicine, Pharmacology, and Veterinary Science OR Social Sciences, Arts, and Humanities

Note: special interface used: http://scholar.google.co.uk/advanced_scholar_search?hl=en&as_sdt=0,5

Ovid MEDLINE(R)

Date range searched: 1946 to week 3, November 2012.

Ovid MEDLINE(R) In-Process and Other Non-Indexed Citations

Date searched: 6 December 2012.

Same as OVID MEDLINE. Eleven results.

Web of Science