Notes
Article history
The research reported in this article of the journal supplement was commissioned and funded by the HTA programme on behalf of NICE as project number 08/88/01. The assessment report began editorial review in April 2009 and was accepted for publication in May 2009. See the HTA programme website for further project information (www.hta.ac.uk). This summary of the ERG report was compiled after the Appraisal Committee’s review. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The HTA editors and publisher have tried to ensure the accuracy of the authors’ report and would like to thank the referees for their constructive comments on the draft document. However, they do not accept liability for damages or losses arising from material published in this report.
Declared competing interests of authors
none
Permissions
Copyright statement
© 2010 Queen’s Printer and Controller of HMSO. This journal is a member of and subscribes to the principles of the Committee on Publication Ethics (COPE) (http://www.publicationethics.org/). This journal may be freely reproduced for the purposes of private research and study and may be included in professional journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be addressed to: NETSCC, Health Technology Assessment, Alpha House, University of Southampton Science Park, Southampton SO16 7NS, UK.
2010 Queen’s Printer and Controller of HMSO
This paper presents a summary of the evidence review group (ERG) report into the clinical effectiveness and cost-effectiveness of pemetrexed for the first-line treatment of locally advanced or metastatic non-small cell lung cancer (NSCLC), in accordance with the licensed indication, based upon the evidence submission from Eli Lilly Ltd to the National Institute for Health and Clinical Excellence (NICE) as part of the single technology appraisal process. The majority of the efficacy evidence described in the manufacturer’s submission is derived from a phase III open label randomised controlled trial (RCT) known as the JMDB trial. The trial achieved its primary objective to demonstrate non-inferiority of pemetrexed/cisplatin to gemcitabine/cisplatin for overall survival in all patients with NSCLC. Because no other studies were found comparing pemetrexed/cisplatin with any other relevant comparator, additional efficacy evidence was presented from two phase III RCTs comparing gemcitabine/cisplatin with gemcitabine/carboplatin and docetaxel/cisplatin. The manufacturer’s submission reported from its indirect comparisons’ analysis that median overall survival and progression-free survival and tumour response rates were more favourable for pemetrexed/cisplatin than for any other comparator. The manufacturer did not identify any published cost-effectiveness analyses of pemetrexed for the first-line treatment of patients with NSCLC. Therefore economic evidence was derived solely from a de novo economic model developed by the manufacturer. A Markov model was developed to evaluate the cost-effectiveness of pemetrexed/cisplatin compared to gemcitabine/cisplatin, docetaxel/cisplatin and gemcitabine/carboplatin. The clinical data used in the economic evaluation were primarily generated from the JMDB trial, with additional data from the two further trials used in the indirect comparisons analysis. The ERG identified series problems with this economic model. As a result, three different versions of the model were submitted to NICE and considered by the ERG. The ICERs estimated by this final version of the model ranged from £8056 to £33,065 per QALY, depending on the comparator, the population and the application of a continuation rule. The ERG considered that the model required extensive modification and redesign, and should be subjected to thorough validation against the JMDB trial results. A full quality audit was also required as it was likely that further model inconsistencies may be present that had not yet been identified. The manufacturer subsequently included evidence in the form of three cost effectiveness analyses (two models and an ‘in-trial’ analysis), stating that a thorough validation process had been followed according to the NICE request. The very short time available to the ERG to consider the new evidence precluded a comprehensive assessment. Instead, the ERG chose to present a simple exploratory analysis combining its own survival projections with key cost estimates obtained from the JMDB trial individual patient data. Compared to gemcitabine, this resulted in ICERs ranging from £17,162 to £30,142 per QALY, depending on the patient population, the maximum number of cycles of chemotherapy and whether a cycle based efficacy adjustment was applied or not. The guidance issued by NICE in September 2009 states that pemetrexed in combination with cisplatin is recommended as an option for the first-line treatment of patients with locally advanced or metastatic NSCLC only if the histology of the tumour has been confirmed as adenocarcinoma or large-cell carcinoma.
Introduction
The National Institute for Health and Clinical Excellence (NICE) is an independent organisation within the NHS that is responsible for providing national guidance on the treatment and care of people using the NHS in England and Wales. One of the responsibilities of NICE is to provide guidance to the NHS on the use of selected new and established health technologies, based on an appraisal of those technologies.
NICE’s single technology appraisal (STA) process is specifically designed for the appraisal of a single product, device or other technology, with a single indication, where most of the relevant evidence lies with one manufacturer or sponsor. 1 Typically, it is used for new pharmaceutical products close to launch. The principal evidence for an STA is derived from a submission by the manufacturer/sponsor of the technology. In addition, a report reviewing the evidence submission is submitted by the evidence review group (ERG); an external organisation independent of NICE. This paper presents a summary of the ERG report for the STA of pemetrexed for the first-line treatment of locally advanced or metastatic non-small cell lung cancer (NSCLC). 2
Description of the underlying health problem
According to Cancer Research UK,3 over 38,000 people were diagnosed with lung cancer in the UK in 2005. Survival from lung cancer is poor with around a quarter of patients (25% men, 26% women) surviving for 1 year after diagnosis, falling to 7% for 5 years after diagnosis, and the disease was responsible for approximately 34,000 deaths in 2006. Reasons for this poor prognosis include the late identification of the disease and low active anticancer treatment rates.
The main subtypes of NSCLC are squamous cell carcinoma, adenocarcinoma and large-cell carcinoma. According to a recent audit of England and Wales,4 33% of patients had squamous NSCLC, 25% had adenocarcinoma, 4% had large-cell carcinoma with the remaining 36% defined as NSCLC ‘not-otherwise specified’ (NSCLC-NOS).
Scope of the evidence review group report
The ERG report presents the results of the assessment of the manufacturer’s (Eli Lilly Ltd) evidence submission regarding the use of pemetrexed with cisplatin compared to platinum-based chemotherapy for the first-line treatment of locally advanced or metastatic NSCLC. The report includes an assessment of both the clinical effectiveness and cost-effectiveness evidence submitted by the manufacturer. The primary clinical outcome measure was overall survival with secondary outcomes of progression-free survival (PFS), response to therapy and tolerability. The cost-effectiveness data were presented as incremental cost-effectiveness ratios (ICERs).
Pemetrexed (Alimta®) is a multitargeted anticancer antifolate agent that exerts its action by disrupting crucial folate-dependent metabolic processes essential for cell replication. It was approved by the European Commission for the first-line treatment of NSCLC (other than predominantly squamous cell histology) in combination with cisplatin on 8 April 2008. In this group of patients, it is indicated for patients with locally advanced or metastatic NSCLC.
Methods
The ERG report comprised a critical review of the evidence for the clinical evidence and cost-effectiveness of the technology based upon the manufacturer’s submission to NICE as part of the STA process. The ERG evaluated the quality of the manufacturer’s clinical effectiveness review. Searches conducted by the manufacturer were assessed for completeness, and the single trial put forward as evidence of effectiveness5 was critically appraised. With regard to cost-effectiveness evidence, the ERG assessed the manufacturer’s searches for completeness, critically appraised the submitted economic model using a standard assessment tool (Drummond and Jefferson6) and conducted an evaluation of the model.
Results
Summary of submitted clinical evidence
The majority of the efficacy evidence described in the manufacturer’s submission is derived from a phase III open label randomised controlled trial (RCT) known as the JMDB trial5 and is presented in Table 1. The trial achieved its primary objective to demonstrate non-inferiority of pemetrexed/cisplatin to gemcitabine/cisplatin for overall survival in all patients with NSCLC [median 10.3 months for both trial arms, adjusted hazard ratio (HR) = 0.94; 95% confidence interval (CI) 0.84 to 1.05]. As pemetrexed is only indicated for the first-line treatment of patients with non-squamous NSCLC, a subgroup analysis of patients with non-squamous NSCLC was presented that reported superiority of pemetrexed/cisplatin on the primary outcome of overall survival compared with gemcitabine/cisplatin (median 11.0 and 10.1 months, respectively, adjusted HR = 0.84; 95% CI 0.74 to 0.96). In the population of patients with non-squamous NSCLC, median PFS was not reported to be statistically superior and, while tumour response rates were reported to be higher for pemetrexed/cisplatin, significance tests were not reported.
Patient group | Median (months) (95% CI) or response rate (%) | Adjusted HR (95% CI) | p-value (superiority) | |
---|---|---|---|---|
Pemetrexed/Cisplatin | Gemcitabine/Cisplatin | |||
Overall survival | ||||
All randomised patients including squamous NSCLC (n = 1725) | 10.3 (9.8 to 11.2) | 10.3 (9.6 to 10.9) | 0.94 (0.84 to 1.05) |
p < 0.001a p = 0.259b |
Patients with non-squamous histology (n = 1252) | 11.0(10.1 to 12.5) | 10.1(9.3 to 10.9) | 0.84(0.74 to 0.96) | p = 0.011b |
Target patients: adenocarcinoma or large-cell carcinoma (n = 1000) | 11.8(10.4 to 13.2) | 10.4(9.6 to 11.2) | 0.81(0.70 to 0.94) | p = 0.005b |
Patients with adenocarcinoma (n = 847) | 12.6(10.7 to 13.4) | 10.9(10.1 to 11.9) | 0.84(0.71 to 0.99) | p = 0.033b |
Patients with large-cell carcinoma (n = 153) | 10.4(8.6 to 14.1) | 6.7(5.5 to 9.0) | 0.67(0.48 to 0.96) | p = 0.027b |
Patients with NSCLC-NOS (n = 252) | 8.6(6.8 to 10.2) | 9.2(8.1 to 10.6) | 1.08(0.81 to 1.45) | p = 0.586b |
Progression-free survival | ||||
All randomised patients including squamous NSCLC (n = 1725) | 4.8(4.6 to 5.3) | 5.1(4.6 to 5.5) | 1.04(0.94 to 1.15) | Not reported |
Patients with non-squamous histology (n = 1252) | 5.3(4.7 to 5.5) | 5.0(4.6 to 5.4) | 0.95(0.84 to 1.06) | Not reported |
Target patients: adenocarcinoma or large-cell carcinoma (n = 1000) | 5.3(4.8 to 5.7) | 4.7(4.4 to 5.4) | 0.90(0.79 to 1.02) | Not reported |
Patients with adenocarcinoma (n = 847) | 5.5(4.9 to 5.7) | 5.0(4.5 to 5.5) | 0.90(0.78 to 1.03) | Not reported |
Patients with large-cell carcinoma (n = 153) | 4.4(3.0 to 5.8) | 4.2(3.5 to 4.7) | 0.89(0.65 to 1.24) | Not reported |
Patients with NSCLC-NOS (n = 252) | 4.5(4.0 to 5.5) | 5.6(4.7 to 5.9) | 1.28(0.99 to 1.67) | Not reported |
Tumour response rate | ||||
All randomised patients including squamous NSCLC (n = 1725) | 27.15 | 24.68 | Not applicable | Not reported |
Patients with non-squamous histology (n = 1252) | 28.64 | 22.24 | Not applicable | Not reported |
Target patients: adenocarcinoma or large-cell carcinoma (n = 1000) | Not reported | Not reported | Not applicable | Not reported |
Patients with adenocarcinoma (n = 847) | 28.90 | 21.65 | Not applicable | Not reported |
Patients with large-cell carcinoma (n = 153) | 27.63 | 27.27 | Not applicable | Not reported |
Patients with NSCLC-NOS (n = 252) | Not reported | Not reported | Not applicable | Not reported |
The manufacturer also defined a more specific target population of patients with adenocarcinoma or large-cell carcinoma. In this target population, median overall survival was also significantly superior in the pemetrexed/cisplatin group (median 11.8 and 10.4 months, respectively, adjusted HR = 0.81; 95% CI 0.70 to 0.94). It should be noted that defining the target population in clinical practice would require more specific testing than is currently standard practice in the UK (as treatment is currently based on whether patients have squamous or non-squamous NSCLC). A study of preoperative histological classification of lung cancer7 cited by the manufacturer suggests that diagnosing adenocarcinoma may be particularly challenging.
Because no other studies were found comparing pemetrexed/cisplatin with any other relevant comparator, additional efficacy evidence was presented from two phase III RCTs comparing gemcitabine/cisplatin with gemcitabine/carboplatin8 and docetaxel/cisplatin9 (Table 2). The manufacturer’s submission reported from its indirect comparisons’ analysis that median overall survival and PFS and tumour response rates were more favourable for pemetrexed/cisplatin than for any other comparator.
Study | Treatment arm | Median (range) OS (months) | Median (range) PFS (months) | Median response rate |
---|---|---|---|---|
JMDB trial | Pemetrexed/cisplatin (n = 862) | 10.3 (9.8 to 11.2) | 4.8 (4.6 to 5.3) | 27% |
(ITT population)5 | Gemcitabine/cisplatin (n = 863) | 10.3 (9.6 to 10.9) | 5.1 (4.6 to 5.5) | 25% |
Zatloukal 20038 | Gemcitabine/cisplatin (n = 87) | 8.8 (6.7 to 10.5) | 5.9 (4.3 to 6.7) | 41% |
Gemcitabine/carboplatin (n = 89) | 8.0 (6.9 to 11.4) | 4.8 (4.0 to 5.6) | 29% | |
Schiller 20029 | Gemcitabine/cisplatin (n = 301) | 8.1 (7.2 to 9.4) | 4.2 (3.7 to 4.8) | 22% |
Docetaxel/cisplatin (n = 304) | 7.4 (6.6 to 8.8) | 3.7 (2.9 to 4.2) | 17% |
With the exception of nausea, pemetrexed/cisplatin appeared to be more tolerable than gemcitabine/cisplatin in terms of grade 3/4 toxicities. No safety issues related to pemetrexed/cisplatin arose beyond those already previously documented. No significant differences were reported for tolerability regarding the different cisplatin regimens (pemetrexed/cisplatin, gemcitabine/cisplatin and docetaxel/cisplatin). Gemcitabine/carboplatin reported less non-haematologic toxicity in terms of nausea and vomiting, and more haematoxicity in terms of an increased incidence of thrombocytopenia than gemcitabine/cisplatin.
Summary of submitted cost-effectiveness evidence
The manufacturer did not identify any published cost-effectiveness analyses of pemetrexed for the first-line treatment of patients with NSCLC. Therefore economic evidence was derived solely from a de novo economic model developed by the manufacturer.
A Markov model was developed to evaluate the cost-effectiveness of pemetrexed/cisplatin compared with gemcitabine/cisplatin, docetaxel/cisplatin and gemcitabine/carboplatin. The clinical data used in the economic evaluation were primarily generated from the JMDB trial,5 with additional data from the two further trials used in the indirect comparisons analysis. 8,9 Although the economic evaluation was trial-based, the modelling component enabled the extrapolation of health effects beyond the period of 30 months of the trial, adopting a lifetime horizon (taken as 6 years) for the consideration of costs and benefits. The perspective of the model was that of the UK NHS and Personal Social Services.
The ERG identified series problems with this economic model. As a result, three different versions of the model were submitted to NICE and considered by the ERG. The ICERs estimated by this final version of the model ranged from £8056 to £33,065, depending on the comparator, the population and the application of a continuation rule.
Commentary on the robustness of submitted evidence
The JMDB trial was a randomised controlled head-to-head clinical trial that was well-designed, used robust randomisation techniques and was suitably powered to demonstrate the primary non-inferiority objective of the trial for the total population of patients with squamous and non-squamous NSCLC. Subgroup analyses of patients with non-squamous NSCLC and the manufacturer’s own defined target population were conducted. The subgroups appeared to be clinically appropriate and confidence in the robustness of the findings was increased by the fact that these two subgroups were both pre-specified and relatively large in size.
Evidence from the indirect comparisons should be treated with caution as other comparators defined in the original scope and decision problem (vinorelbine and paclitaxel in combination with cisplatin or carboplatin and docetaxel/carboplatin) were excluded from the indirect comparisons analysis. In addition, the statistical approach employed to generate the findings is not considered to be the most optimal, as calculations were based on median survival times and individual trial arm level data from within trials were compared, thus ignoring the benefits of randomisation. Finally, data was only available for all patients with NSCLC (i.e. squamous or non-squamous NSCLC) in all but the JMDB trial. Thus, the HRs for each subgroup in the JMDB trial were used to estimate HRs for subgroups of patients given gemcitabine/carboplatin or docetaxel/cisplatin in the comparator trials. However, it was impossible to confirm from the data reported by the published papers of these trials whether the relative effects found in the JMDB trial would be consistent across subgroups for these patients.
Examination of the final version of the economic model submitted to NICE and considered by the ERG showed that, although minor modifications had been made to correct some of the problems identified by the ERG with earlier versions, the model still failed to adequately address the crucial problems at the heart of the model. These were beyond the remit of the ERG to address, and included:
-
The chosen model design was not obviously suitable for modelling the disease and treatments described in the published clinical trial, imposing as it does serious constraints on the possibility of representing the observed patterns of response to treatment and progression of disease.
-
The implementation of the model was marked by examples of basic errors with marked consequences.
-
There is little evidence of a systematic approach by the manufacturer to identifying and eliminating errors in the development of the model, or of attempting to replicate the prime source of information for the model, i.e. the JMDB trial itself.
-
The restriction of comparators to those that are relatively high cost is likely to give a misleading impression of the true cost-effectiveness of pemetrexed regimen. Furthermore, gemcitabine will be off patent in the UK from March 2009 and may soon become available in generic form at a lower price. This was not considered in the manufacturer’s model.
-
The methods used for adjusting treatment effects (positive and negative) when a scenario is used with fewer treatment cycles than in the trial evidence are not obviously robust and defensible and may tend to overestimate the outcome benefits to be expected from use of pemetrexed/cisplatin, while underestimating the additional cost.
Thus, the ERG believed that the model requires extensive modification and redesign, subjected to thorough validation against the JMDB trial results. A full quality audit was also required as it is likely that further model inconsistencies may be present that have not yet been identified.
Conclusions
Given that the JMDB trial subgroup analyses were predefined and a large number of patients were included, confidence in the robustness of the subgroup results was increased. These findings provide important evidence warranting further exploration that pemetrexed/cisplatin may be superior to gemcitabine/cisplatin in terms of prolonging overall survival in patients with non-squamous NSCLC, particularly in those with adenocarcinoma or large-cell carcinoma.
Identifying patients in the manufacturer’s target population requires more specific histological testing than is standard across all UK centres at present. In the JMDB trial, patients with adenocarcinoma represented half of all patients. The known proportion of patients with adenocarcinoma in the UK is not presented in the manufacturer’s submission which reports only recent audit data suggesting a quarter of patients with NSCLC have adenocarcinoma. 4 Thus, the accurate diagnosis for this significant group of patients may be a particular challenge.
As no other regimens recommended by NICE were compared in head-to-head clinical trials with pemetrexed/cisplatin, the manufacturer undertook an indirect comparisons’ analysis. This suggested pemetrexed/cisplatin to be the most efficacious regimen when also compared with gemcitabine/carboplatin, the most common regimen in the UK, or docetaxel/cisplatin. However, because not all relevant comparators were included in the indirect comparisons’ analysis and because of the statistical method employed to undertake this analysis, these findings should be treated with caution.
The ERG found a number of substantial problems with the economic model. Most seriously, there were underlying structural problems and logic errors which had still not been addressed in the third version of the model submitted by the manufacturer. Consequently, the model was unable to replicate the response rates arising in the JMDB trial and it was impossible to provide reliable ICERs. Thus, even in its modified form, the economic model was not able to provide estimates upon which to base a decision regarding the cost-effectiveness of pemetrexed/cisplatin.
Summary of NICE guidance issued as a result of the STA
Given the above conclusions, NICE guidance was only issued after considering additional evidence subsequently submitted by the manufacturer (two models and an ‘in-trial’ analysis) and a critique of this evidence by the ERG. The very short time available to the ERG to consider the new evidence precluded a comprehensive assessment. The ERG believed that some issues of face validity had not been appropriately addressed and thus the ERG presented a simple exploratory analysis combining its own survival projections with key cost estimates obtained from individual patient data provided by the manufacturer from the JMDB trial. Compared to gemcitabine, this resulted in ICERs ranging from £17,162 to £30,142 per QALY, depending on the patient population, the maximum number of cycles of chemotherapy and whether a cycle based efficacy adjustment was applied or not. Thus the guidance issued by NICE in September 2009 states that pemetrexed in combination with cisplatin is recommended as an option for the first-line treatment of patients with locally advanced or metastatic NSCLC only if the histology of the tumour has been confirmed as adenocarcinoma or large-cell carcinoma. People who are currently being treated with pemetrexed for NSCLC but who do not meet this criterion should have the option to continue their therapy until they and their clinicians consider it appropriate to stop.
Disclaimers
The views expressed in this publication are those of the authors and not necessarily those of the HTA programme or the Department of Health.
Key references
- NICE . Guide to the Single Technology (STA) Process 19 September 2006 2006. www.nice.org.uk/nicemedia/pdf/STA_Process_Guide.pdf (accessed 3 March 2009).
- Fleeman N, Bagust A, McLeod C, Greenhalgh J, Boland A, Dundar Y, et al. Pemetrexed for the first-line treatment of locally advanced or metastatic non-small cell lung cancer (NSCLC): A Single Technology Appraisal. Liverpool: LRiG, The University of Liverpool; 2009.
- Cancer Research UK . CancerStats Key Facts on Lung Cancer and Smoking 2008. info.cancerresearchuk.org/cancerstats/types/lung/?a=5441 (accessed January 2009).
- The Information Centre for Health and Social Care . National Lung Cancer Audit: Key Findings about the Quality of Care for People With Lung Cancer in England and Wales. Report for the Audit Period 2006 2007.
- Scagliotti GV, Parikh P, Von Pawel J, Biesma B, Vansteenkiste J, Manegold C, et al. Phase III study comparing cisplatin plus gemcitabine with cisplatin plus pemetrexed in chemotherapy-naive patients with advanced-stage non-small-cell lung cancer. J Clin Oncol 2008;26:3543-51.
- Drummond MF, Jefferson TO. Guidelines for authors and peer reviewers of economic submissions to the BMJ. The BMJ Economic Evaluation Working Party. BMJ 1996;313:275-83.
- Edwards SL, Roberts C, McKean ME, Cockburn JS, Jeffrey RR, Kerr KM. Preoperative histological classification of primary lung cancer: accuracy of diagnosis and use of the non-small cell category. J Clin Pathol 2000;53:537-40.
- Zatloukal P, Petruželka L, Zemanová M, Kolek Vt, Skřičková J, Pešek M, et al. Gemcitabine plus cisplatin vs. gemcitabine plus carboplatin in stage IIIb and IV non-small cell lung cancer: a phase III randomized trial. Lung Cancer 2003;41:321-31.
- Schiller JH, Harrington D, Belani CP, Langer C, Sandler A, Krook J, et al. Comparison of four chemotherapy regimens for advanced non-small-cell lung cancer. N Engl J Med 2002;346:92-8.