Notes
Article history
The research reported in this article of the journal supplement was commissioned and funded by the HTA programme on behalf of NICE as project number 05/54/01. The assessment report began editorial review in July 2007 and was accepted for publication in November 2008. See the HTA programme website (www.hta.ac.uk) for further project information. This summary of the ERG report was compiled after the Appraisal Committee’s review. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The HTA editors and publisher have tried to ensure the accuracy of the authors’ report and would like to thank the referees for their constructive comments on the draft document. However, they do not accept liability for damages or losses arising from material published in this report.
Declared competing interests of authors
none
Permissions
Copyright statement
© 2009 Queen’s Printer and Controller of HMSO. This monograph may be freely reproduced for the purposes of private research and study and may be included in professional journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be addressed to: NETSCC, Health Technology Assessment, Alpha House, University of Southampton Science Park, Southampton SO16 7NS, UK.
2009 Queen’s Printer and Controller of HMSO
This paper presents a summary of the evidence review group (ERG) report into the clinical and cost-effectiveness of paclitaxel in the management of early stage breast cancer based upon the manufacturer’s submission to the National Institute for Health and Clinical Excellence (NICE) as part of the single technology appraisal (STA) process. The scope was not clearly defined in the manufacturer’s submission. Two of the three clinical trials included in the submission report showed that the addition of four cycles of paclitaxel to four cycles of doxorubicin and cyclophosphamide (AC-P) resulted in modest improvements in the two end points of disease-free survival (DFS) and overall survival (OS). The third unpublished study evaluating four cycles of AC followed by paclitaxel or docetaxel in breast cancer did not show any statistically significant differences in DFS or OS between any group. The economic evaluation of paclitaxel for adjuvant therapy in early breast cancer was based on two of the three trials submitted as clinical evidence and used a probabilistic Markov state-transition model. The measure of health benefit was quality-adjusted life-years (QALYs) and the model included direct costs using a UK NHS perspective. The primary analysis compared AC-P with four cycles of AC. The reported incremental cost-effectiveness ratio (ICER) for this comparison was £4726 per additional QALY for AC-P compared with four cycles of AC. The submission did not include a systematic review for clinical or cost-effectiveness evidence. As a result, potentially relevant trials and previously published studies were omitted. The main comparator used did not represent standard care in the UK NHS and a large number of relevant comparators were omitted, including docetaxel. The manufacturer did not consider potentially important patient subgroups defined by baseline risk, and the cost-effectiveness result in the average overall patient population may conceal important variation between subgroups. Overall, although the economic model may have indicated that the addition of four cycles of paclitaxel to four cycles of AC may be cost-effective compared with providing four cycles of AC only, this comparison is not informative to current clinical practice in the UK NHS. In the context of this review it is not possible for the ERG to predict the cost-effectiveness of paclitaxel compared with more appropriate, and potentially more effective, relevant comparators. The guidance issued by NICE in July 2006 as a result of the STA states that paclitaxel is not recommended as an option for the adjuvant treatment of women with early node-positive breast cancer.
Introduction
The National Institute for Health and Clinical Excellence (NICE) is an independent organisation within the NHS that is responsible for providing national guidance on the treatment and care of people using the NHS in England and Wales. One of the responsibilities of NICE is to provide guidance to the NHS on the use of selected new and established health technologies, based on an appraisal of those technologies.
The single technology appraisal (STA) process of NICE is specifically designed for the appraisal of a single product, device or other technology, with a single indication, for which most of the relevant evidence lies with one manufacturer or sponsor. 1 Typically, it is used for new pharmaceutical products close to launch. The principal evidence for an STA is derived from a submission by the manufacturer/sponsor of the technology. In addition, a report reviewing the evidence submission is submitted by the evidence review group (ERG), an external organisation independent of NICE. This paper presents a summary of the ERG report for the STA on the use of paclitaxel in the management of early stage breast cancer. 2
Description of the underlying health problem
In England and Wales breast cancer is the most common malignancy and cause of cancer mortality in women,3–5 with 39,175 new cases of breast cancer registered in 2003,4–6 representing a crude incidence rate of 74 per 100,000 population. In the same year over 11,000 women died of breast cancer. 3–6 This is a cancer that affects predominantly middle-aged to older women. The incidence of new cases in 2003 in women younger than 30 years was less than 0.4% and the incidence in men represented less than 1% of all new cases. 4–6 More than 80% of new cases are diagnosed in women aged 50 and over,3,4,6 with the peak age range for diagnosis in women being 55–59 years
(5395 out of 38,864 new cases in 2003). 4,6
The 5-year age-standardised relative survival rate up to the end of 2001 for adult female patients (15–99 years) diagnosed with breast cancer between 1996 and 1999 in England and Wales was 77.5%, with a trend towards increasing rates of survival over the years. 7
An invasive breast cancer is one in which there is dissemination of cancer cells outside the basement membrane of the ducts and lobules into the surrounding adjacent normal tissue. 8 The presence or absence of involved axillary lymph nodes is the single best predictor of survival in breast cancer, and important treatment decisions are based on it. Both the number of involved nodes and the level of nodal involvement predict survival from breast cancer. 9 When invasive breast cancer is diagnosed the extent of the disease should be assessed and the tumour staged. The two staging classifications in current use are the tumour node metastases (TNM) system and the International Union Against Cancer (UICC) system, which incorporates the TNM classification. Prognosis in breast cancer relates to the stage of the disease at presentation. 8
Data published in 200310 indicated a prevalence of early stage node-positive breast cancer (T1–3, N+, M0) in two regional UK populations (n = 559) of approximately 21% of all presenting breast cancers; the same study reported a pan-European (n = 4478) incidence rate of 31%. An earlier (1997) UK study (n = 1440) reported that 49.8% of all presenting breast cancers were node positive at the time of diagnosis. 11
When surgery is considered appropriate treatment for breast cancer, a number of options are available, with differing levels of breast tissue conservation. When chemotherapy is administered after surgery of any type it is known as adjuvant chemotherapy. When chemotherapy is administered before surgery it is known as neoadjuvant chemotherapy. 12
Ensuring that adjuvant therapy is always offered to women with primary breast cancer when appropriate could reduce recurrence and improve survival rates. 13 In 2002, NICE recommended that almost all patients with invasive breast cancer should be offered adjuvant systemic therapy (hormone therapy and/or chemotherapy). 13 Women at intermediate or high risk of recurrence, dictated by primary tumour size, extent of nodal involvement and tumour grade, who have not had neoadjuvant chemotherapy should normally be offered four to eight cycles of multiple-agent chemotherapy that includes an anthracycline. 13
Scope of the ERG report
The ERG report critically evaluated the evidence submission from Bristol-Myers Squibb Pharmaceuticals (BMS) on the clinical and cost-effectiveness of paclitaxel (Taxol®) for adjuvant treatment of early breast cancer. 14 The perceived aim of the BMS submission was to evaluate the clinical and cost-effectiveness of paclitaxel for the licensed indication of the treatment of early stage, operable, node-positive breast cancer following four cycles of anthracycline and cyclophosphamide therapy. The licensed dose is 175 mg/m2 every 3 weeks for four courses. Additionally, paclitaxel is licensed for treating ovarian cancer, advanced non-small cell lung cancer and AIDS-related Kaposi’s sarcoma. Paclitaxel is manufactured in the UK as Taxol (BMS) and is now also available generically (from Mayne Pharma). The list prices at time of writing are comparable, with prices of the generic drug being £112.20, £336.60 and £1009.80 and prices of Taxol being £116.05, £347.82 and £1043.46 for the 5-ml, 16.7-ml and 50-ml vials respectively. 15
Methods
The ERG report comprised a critical review of the evidence for the clinical and cost-effectiveness of the technology based upon the manufacturer’s/ sponsor’s submission to NICE as part of the STA process.
This report identifies the submission’s strengths and weaknesses, supplemented, where appropriate, with the ERG’s own analysis. Clinical experts were asked to advise the ERG to help inform the review.
As the scope for this STA was not clearly defined in the BMS submission, and BMS did not summarise the decision problem, the ERG made the decision to look at the scope based on the licensed indication, that is, the use of paclitaxel for the treatment of node-positive breast cancer following anthracycline and cyclophosphamide therapy.
In view of the lack of a systematic review in the manufacturer’s submission, a full detailed search needed to be undertaken as part of the ERG report. Thus, the ERG report included a detailed systematic search for studies and a critical analysis of relevant trials, regardless of whether the BMS submission had included them or not. It included a summary of the main points from any systematic reviews found, and a review of the main points from three sets of international guidelines included in the BMS submission.
The economic model included in the submission was reviewed on the basis of the manufacturer’s report and by direct examination of the electronic model. The critical appraisal was conducted with the aid of a checklist for assessing the quality of economic evaluations16 and a narrative review to highlight key assumptions and possible limitations.
This was a pilot STA and processes were not in place to give the manufacturer the opportunity to provide revised analyses to address limitations identified by the ERG during the course of the review process. Therefore, additional analyses were undertaken by the ERG to provide further information on areas that the ERG considered were not sufficiently dealt with in the manufacturer’s submission. The additional work undertaken by the ERG was intended to provide additional information on the qualitative impact of identified limitations. Given the restricted nature of these additional analyses only three areas were considered:
-
subgroup analysis
-
sensitivity analysis
-
additional comparator.
It should be noted that the analyses into these areas were selective and that the revised economic analyses were undertaken to examine the robustness of the manufacturer’s own model to alternative assumptions. These analyses are clearly subject to the same major limitations as the economic model. The results should therefore be taken only as indicative of the potential impact of these gaps in the manufacturer’s submission.
Results
Summary of submitted clinical evidence
Of the three clinical trials included in the submission report two were fully published. 17,18 These trials aimed to determine whether four cycles of paclitaxel following four cycles of doxorubicin and cyclophosphamide (AC-P) would prolong disease-free survival (DFS) and overall survival (OS). Improvements of 5% [hazard ratio (HR) 0.83, 95% CI 0.73–0.94] and 4.2% (HR 0.83, 95% CI 0.72–0.95) in DFS and 4% (HR 0.82, 95% CI 0.71–0.95) and 0.8% (HR 0.93, 95% CI 0.78–1.12) in OS were seen in the two published trials. Both showed that the addition of four cycles of paclitaxel to four cycles of AC chemotherapy resulted in modest improvements in these two end points. The unpublished study19 evaluated four cycles of AC followed by paclitaxel or docetaxel in breast cancer. This trial had insufficient data presented to fully assess the validity of the study, but it did show that there were no statistically significant differences in DFS or OS between any group.
Summary of submitted cost-effectiveness evidence
The submission included a de novo economic evaluation of paclitaxel for adjuvant therapy in early breast cancer, which the manufacturer’s stated was based on two17,19 of the three trials submitted as clinical evidence. Of the explicitly included trials, one was fully published and the other was unpublished. A probabilistic Markov state-transition model was used to compare the cost-effectiveness of the treatment strategies included in the two clinical trials (Figure 1). The measure of health benefit was quality-adjusted life-years (QALYs) and the model included direct costs using a UK NHS perspective. The primary analysis compared AC-P with four cycles of AC. The reported incremental cost-effectiveness ratio (ICER) for this comparison was £4726 per additional QALY for AC-P compared with four cycles of AC. A summary of the manufacturer’s economic evaluation is provided in Table 1.
Assumption | Source/Justification | |
---|---|---|
Model | Markov state-transition model with lifetime horizon, cycle length of 1 year | None provided |
Natural history | Equivalent to AC arm of single randomised trial. Baseline risk assumed constant after year 7 (maximum follow-up in trial) | Baseline data taken from CALGB 9344.17 Justification for constant risk after year 7 based on Bonadonna et al.,23 who compared CMF with no treatment, but no corresponding statements found in original paper |
Treatment effect on DFS | Lifetime treatment effect | Probability of recurrence based on CALGB 934417 and NABCI E1199.19 No Justification provided for lifetime treatment effect |
Treatment effect on OS | Location of recurrence based on excluded clinical trial NSABP-B28 | Mamounas et al.18 |
Risk of progression following a recurrence independent of treatment received and based on a previous economic study rather than OS in included trials | Johnston.24 Manufacturers state belief that OS from trials would overestimate survival and would not allow recognition of costs and quality of life implications associated with progression | |
Adverse events | Only considers the costs of managing neutropenia. All febrile neutropenia is hospitalised and treated with 14-day course of G-CSF. All neutropenia assumed to occur in first cycle of treatment and be prevented in subsequent cycles by G-CSF. No attempt to quantify the potential impact of side effects on quality of life | Probability of neutropenia based on CALGB 934417 and NABCI E1199.19 No Justification for inclusion or exclusion of adverse events |
Health-related quality of life | External utility estimates assigned to acute-phase period and the main health states. Utility during acute phase assumed to be the same for all chemotherapies. Utility for distant recurrence assumes that it is treated with second-line chemotherapy | Abstract by Sorensen et al.25 No Justification provided for selection of data source |
Treatment costs | Average patient weighs 70 kg with body surface area of 1.7 m2. Cost of 1-hour chemotherapy administration assumed equal to one outpatient visit. Cost of additional hours required for administration adjusted on the basis of US costs | BNF 50.15 No Justification provided for approach used to cost of administration |
Health state costs | Primary surgery based on that received in CALGB 9344. Death due to breast cancer incurs palliative care cost but death due to other causes does not | Johnston24 |
Discount rates | 3.5% for health outcomes and costs | In accordance with NICE guidance26 |
Commentary on the robustness of submitted evidence
The sections containing descriptions of individual studies did accurately reflect the data presented within the clinical trials that were considered in the manufacturer’s submission. The overall economic model structure was appropriate for the decision problem, and the data sources used to inform the model were appropriate from a UK NHS perspective.
The ERG felt that the BMS submission was generally of poor quality with key omissions. The major flaw in the submission was the absence of a systematic literature review, as instructed by NICE in the draft guidance. 20 BMS limited the clinical effectiveness review in the submission to three studies, and it was unclear without the ERG undertaking a full systematic review whether they had considered all of the relevant literature. This same selective use of available evidence was apparent in the economic evaluation. There was a tendency throughout the trials section to refer to relative risk rather than absolute risk, and relevant p-values were not quoted. This had the effect of exaggerating any possible benefits of treatment. Although the trial evidence around paclitaxel appears to show modest benefit, the trials themselves may not be directly applicable to the clinical situation that these patients are likely to face.
A further shortcoming of the submission was in not clearly defining the choice of comparator(s). This is important in determining relative efficacy and, if not clearly stated, affects the underlying discussions throughout the document. The comparators that were included in the cost-effectiveness analysis were not considered by the ERG to represent current treatment in the UK NHS or relevant licensed alternatives, and four cycles of AC may be regarded as a weak comparator in this patient population.
The submission did not consider identifiable subgroups of patients defined by prognostic factors that strongly influence the baseline risk of future events. Instead, the results were presented for the average patient recruited to the clinical trials included in the analysis, and this may conceal wide variation in the cost-effectiveness of paclitaxel according to baseline risk. The ERG attempted to highlight the potential impact of different patient characteristics on both DFS and the improvement in outcomes from different treatment options. They used data from Adjuvant! Online, a web-based decision aid that predicts 10-year breast cancer outcomes with and without adjuvant therapy. 21,22 Table 2 presents a comparison of 10-year DFS rates from the manufacturer’s model with those from Adjuvant! Online.
AC/first generation (%) | AC-P3/second generation (%) | Percentage point difference between treatment | ||
---|---|---|---|---|
Manufacturer’s model | 47 | 53 | 6 | |
Adjuvant! Online | ERG base case | 48.1 | 55.2 | 7.1 |
ER status negative | 39.9 | 47.5 | 7.6 | |
Grade 3 | 41.9 | 49.4 | 7.5 | |
Size > 5.0 cm | 35.7 | 43.5 | 7.8 | |
> Nine positive nodes | 31.2 | 39.1 | 7.9 | |
Low risk | 82.9 | 85.3 | 2.4 | |
High risk | 9.8 | 15.7 | 5.9 |
There were a number of typographical errors, minor discrepancies in data and modelling errors in the submission report and a number of statements throughout that were not supported by valid references. Overall, the submission report was not of a high quality. See Table 3 for a comparison of the submission model with a NICE reference case. Consequently, parts of the submission needed to be repeated by the ERG and a lot more time was spent on areas that should have been appropriately completed by BMS.
Element of assessment | Reference case | Manufacturers submission |
---|---|---|
Defining the decision problem | N/A for STA | Treatment of interest was the licensed form of paclitaxel. Model considers a hypothetical cohort of women aged 50 years with operable node-positive breast cancer (based on patients recruited to Henderson et al17) |
Comparator | Alternative therapies routinely used in the NHS | No. Four cycles of AC used as the comparator. This is unlikely to represent standard treatment in the UK for this high-risk patient population |
Perspective on costs | NHS and PSS | Yes. However, some relevant categories of cost are omitted from the analysis (e.g. premedication) |
Perspective on outcomes | All health effects on individuals | Yes. However, model does not include differential utility impact related to toxicity while receiving treatment |
Types of economic evaluation | Cost-effectiveness analysis | Yes |
Synthesis of evidence on outcomes | Based on a systematic review | No |
Measure of health benefits | Quality-adjusted life-years (QALYs) | Yes |
Description of health states for calculation of QALYs | Health states described using a standardised and validated generic instrument | No. Utilities based on standard gamble methodology. Health state descriptions not publicly available |
Methods of preference elicitation for health state valuation | Choice-based method, for example time trade-off, standard gamble (not rating scale) | Yes |
Source of preference data | Representative sample of the public | No. Sample consisted of patients: 67 postmenopausal women aged 55–70 years in the UK (n = 23) and US (n = 44) who had a history of stage 1 or 2 operable early breast cancer |
Discount rate | An annual rate of 3.5% on both costs and health effects | Yes |
Equity provision | An additional QALY has the same weight regardless of the other characteristics of the individuals receiving the health benefit | Yes |
Conclusions
The submission did not include a systematic review for clinical or cost-effectiveness evidence. As a result, potentially relevant trials and previously published studies were omitted. The main comparator used did not represent standard care in the UK NHS and a large number of relevant comparators were omitted, including docetaxel, another taxane, as licensed for the same indication. The manufacturer did not consider potentially important patient subgroups defined by baseline risk, and the cost-effectiveness result in the average overall patient population may conceal important variation between subgroups.
Overall, although the economic model may have indicated that the addition of four cycles of paclitaxel to four cycles of AC may be cost-effective compared with providing four cycles of AC only, this comparison is not informative to current clinical practice in the UK NHS. In the context of this review it is not possible for the ERG to predict the cost-effectiveness of paclitaxel compared with more appropriate, and potentially more effective, relevant comparators such as six cycles of FAC or the licensed indication of docetaxel. It is therefore impossible for the ERG to predict what effect including these comparators would have on the cost-effectiveness of paclitaxel for adjuvant treatment of early breast cancer.
Summary of NICE guidance issued as a result of the STA
The guidance issued by NICE in July 2006 states that:
Paclitaxel is not recommended as an option for the adjuvant treatment of women with early node-positive breast cancer. 27
Disclaimers
The views expressed in this publication are those of the authors and not necessarily those of the HTA programme or the Department of Health.
Key references
- National institute for Health and Clinical Excellence . Guide to the Single Technology (STA) Process 2006. www.nice.org.uk/page.aspx?o=STAprocessguide.
- Griffin S, Dunn G, Palmer S, Macfarlane K, Brent S, Dyker A, et al. The Use of Paclitaxel in the Management of Early Stage Breast Cancer 2006. www.nice.org.uk/guidance/index.jsp.jsp?action=download%26o=33583?action=download526o=33583.
- Office for National Statistics . Health – Breast Cancer 2006. www.statistics.gov.uk/CCI/nugget.asp?ID=575%26Pos=4%26ColRank=1%26Rank=144 (accessed 1 February 2006).
- Welsh Cancer Intelligence and Surveillance Unit . Cancer Incidence, 2003 Report 2003. www.wales.nhs.uk/sites3/docopen.cfm?orgid=242%26ID=49488%26BF360CDD-4EAD-415A-B9683BB5BA039A76.
- Welsh Cancer Intelligence and Surveillance Unit . Incidence of Male Breast Cancer in Wales, 2003 2006.
- Office for National Statistics . Registrations of Cancer Diagnosed in 2003, England 2003. www.statistics.gov.uk.
- Office for National Statistics . Cancer Survival: England and Wales, 1991–2001, Four Major Cancers 2001. www.statistics.gov.uk/StatBase/ssdataset.asp?vlnk=7091%26Pos=10%26ColRank=1%26Rank=272.
- Sainsbury JRC, Anderson TJ, Morgan DAL. ABC of breast disease: breast cancer. Br Med J 2000;321:745-50.
- Bundred NJ, Morgan DAL, Dixon JM. ABC of breast diseases: management of regional nodes in breast cancer. Br Med J 1994;309:1222-5.
- Sant M, Allemani C, Capocaccia R, Hakulinen T, Aareleid T, Coebergh JW, et al. EUROCARE Working Group . Stage at diagnosis is a key explanation of differences in breast cancer survival across Europe. Int J Cancer 2003;106:416-22.
- Fisher CJ, Egan MK, Smith P, Wicks K, Millis RR, Fentiman IS. Histopathology of breast cancer in relation to age. Br J Cancer 1997;75:593-6.
- ABPI . Target Breast Cancer 2005.
- National Institute of Clinical Excellence . Guidance on Cancer Services – Improving Outcomes in Breast Cancer – Manual Update 2002. www.nice.org.uk/.
- Bristol-Myers Squibb Pharmaceuticals . Taxol® (paclitaxel) for the Adjuvant Treatment of Early Breast Cancer 2006. www.nice.org.uk/guidance/index.jsp?action=download%26o=33584.
- Joint Formulary Committee . British National Formulary (BNF). 2006.
- Drummond MF, Sculpher MJ, Torrance GW, O’Brien BJ, Stoddart GL. Methods for the economic evaluation of health care programmes. Oxford: Oxford University Press; 2005.
- Henderson IC, Berry DA, Demetri GD, Cirrincione CT, Goldstein LJ, Martino S, et al. Improved outcomes from adding sequential Paclitaxel but not from escalating Doxorubicin dose in an adjuvant chemotherapy regimen for patients with node-positive primary breast cancer. J Clin Oncol 2003;21:976-83.
- Mamounas EP, Bryant J, Lembersky B, Fehrenbacher L, Sedlacek SM, Fisher B, et al. Paclitaxel after doxorubicin plus cyclophosphamide as adjuvant chemotherapy for node-positive breast cancer: results from NSABP B-28. J Clin Oncol 2005;23:3686-96.
- Sparano JA, Wang M, Martino S, Jones V, Perez EA, Saphner T, et al. Phase III Study of Doxorubicin-Cyclophosphamide Followed by Paclitaxel or Docetaxel Given Every 3 Weeks or Weekly in Patients With Axillary Node-Positive or High-Risk Nodenegative Breast Cancer: Results of North American Breast Cancer Intergroup Trial E1199 n.d.
- National Institute for Health and Clinical Excellence . Specification for Manufacturer Sponsor Submission for Single Technology Appraisal (STA) (draft for Consultation) 2006. www.nice.org.uk/page.aspx?o=278604 (accessed 5 January 2006).
- Olivotto IA, Bajdik CD, Ravdin PM, Speers CH, Coldman AJ, Norris BD, et al. Population-based validation of the prognostic model ADJUVANT! for early breast cancer. J Clin Oncol 2005;23:2716-25.
- Peele PB, Siminoff LA, Xu Y, Ravdin PM. Decreased use of adjuvant breast cancer therapy in a randomized controlled trial of a decision aid with individualized risk information. Med Decis Making 2005;25:301-7.
- Bonadonna G, Valagussa P, Moliterni A, Zambetti M, Brambilla C. Adjuvant cyclophosphamide, methotrexate, and fluorouracil in node-positive breast cancer: the results of 20 years follow-up. N Engl J Med 1995;332:901-6.
- Johnston K. Modelling the future costs of breast screening. Eur J Cancer 2001;37:1752-8.
- Sorenson SV, Brown R, Benedict A, Flood E, Revicki D. Patient-Rated Utilities in Postmenopausal Early Breast Cancer (EBC): A Cross-Country Comparison n.d.
- National Institute for Health and Clinical Excellence . Guide to the Methods of Technology Appraisal 2004. www.nice.org.uk/pdf/TAP_Methods.pdf.
- National Institute for Health and Clinical Excellence . Paclitaxel for the Adjuvant Treatment of Early Node-Positive Breast Cancer 2006. www.nice.org.uk/guidance/TA108.