Everolimus for the second-line treatment of advanced and/or metastatic renal cell cancer: a critique of the submission from Novartis

Article history

The research reported in this article of the journal supplement was commissioned and funded by the HTA programme on behalf of NICE as project number 08/208/01. The assessment report began editorial review in February 2010 and was accepted for publication in March 2010. See the HTA programme website for further project information (www.hta.ac.uk). This summary of the ERG report was compiled after the Appraisal Committee’s review. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The HTA editors and publisher have tried to ensure the accuracy of the authors’ report and would like to thank the referees for their constructive comments on the draft document. However, they do not accept liability for damages or losses arising from material published in this report. The views expressed in this publication are those of the authors and not necessarily those of the HTA programme or the Department of Health.

Declared competing interests of authors

none

Copyright statement

© 2010 Queen’s Printer and Controller of HMSO. This journal is a member of and subscribes to the principles of the Committee on Publication Ethics (COPE) (http://www.publicationethics.org/). This journal may be freely reproduced for the purposes of private research and study and may be included in professional journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be addressed to: NETSCC, Health Technology Assessment, Alpha House, University of Southampton Science Park, Southampton SO16 7NS, UK.

2009 Queen’s Printer and Controller of HMSO

Description of the underlying health problem

Renal cell carcinoma (RCC), also called renal adenocarcinoma or hypernephroma, is a cancer usually originating in the lining of the tubules of the kidney. The stage of RCC is usually reported using the tumour, node and metastasis (TNM) classification. This is based on the extent of the primary tumour (T), whether lymph nodes are affected (N) and whether metastases are present (M). Advanced and metastatic RCC falls within stages III and IV, stage III denotes disease that is locally advanced and/or has spread to regional lymph nodes and stage IV denotes that distant metastasis has occurred.

Early, small RCC tumours are usually asymptomatic; the diagnosis of early RCC is usually incidental after abdominal scans for other indications. The most common presenting symptoms of advanced RCC are blood in the urine (haematuria), a palpable mass in the flank or abdomen, and abdominal pain. Other non-specific symptoms include fever, night sweats, malaise and weight loss.

Kidney cancer accounts for around 2% of all cancers in the UK. In 2004, 6180 new kidney cancers were diagnosed in England and Wales, of which an estimated 85–90% were RCC. RCC is nearly twice as common in men as in women, and most commonly affects adults aged 50–80 years old. In 2005, there were 3134 registered deaths from kidney cancer in England and Wales.

Approximately 25% of RCC patients present with advanced and/or metastatic disease (stage III or IV). An estimated 50% of patients who have curative resection for earlier stages will develop recurrent and/or metastatic disease. Without treatment, these patients have a median survival rate of only 6–12 months and a 2-year survival rate of 10–20%.

Surgical resection to remove the entire kidney (radical nephrectomy) or part of the kidney (partial nephrectomy) is the only accepted curative treatment for patients with non-metastatic RCC (TNM stage I–III), and the success of surgery depends on the stage of disease. Current standard treatment of metastatic RCC (stage IV) is immunotherapy with interleukin-2 (sometimes called aldesleukin) or interferon-alpha (IFN-α) which may lead to tumour shrinkage. Palliative surgery, arterial embolism or radiotherapy may also be considered in these patients. Bevacizumab plus IFN-α, sorafenib, sunitinib and temsirolimus all have UK marketing authorisations for use in the treatment of those with advanced and/or metastatic RCC who are suitable for immunotherapy and have an Eastern Cooperative Oncology Group performance status of 0 or 1.

Everolimus (Afinitor, Novartis Pharmaceuticals, Camberley, Surrey, UK) is an oral, once-daily selective inhibitor of the mammalian target of rapamycin protein, that controls tumour cell division, growth and angiogenesis. It does not have a UK marketing authorisation for use in advanced/metastatic RCC. However, in May 2009, the European Medicines Agency adopted a positive opinion, recommending everolimus for the treatment of patients with advanced RCC whose disease has progressed on, or after treatment with vascular endothelial growth factor targeted therapy. 3 Everolimus has a marketing authorisation for other indications in the European Union.

Scope of the evidence review group report

The purpose of the ERG report was to comment on the validity of the manufacturer’s submission on the technology of interest. The scope for this submission and hence the scope for the ERG report is shown in Table 1.

Summary of submitted clinical evidence

The evidence for this submission is based on one randomised controlled trial (RCT), the RECORD-1 (Renal Cell Cancer Treatment with Oral RAD001 Given Daily-1) study. 5 This was a randomised, double-blind, placebo-controlled, phase III clinical trial of 416 participants. Eligible patients were adults aged ≥ 18 years with RCC whose disease had progressed on or within 6 months of stopping treatment with sunitinib, sorafenib or both drugs, a directly relevant population consistent with the scope of the appraisal. Of 416 patients, 277 were randomised to 10-mg everolimus once daily plus best supportive care (BSC), and 139 to an identical placebo tablet plus BSC. The blinded phase became open-label upon disease progression when patients were allowed to cross over from placebo to treatment group. Of 139 participants in the BSC plus placebo arm, 112 received everolimus following disease progression.

The primary outcome was progression-free survival (PFS). RECORD-1 (final analysis) showed an improvement in this outcome which was unlikely to have occurred by chance alone [hazard ratio (HR) 0.33; 95% confidence interval (CI) 0.25 to 0.43; p < 0.001]. 5 This equated to a mean PFS of 4.9 months in the BSC plus everolimus arm and 1.9 months for BSC plus placebo. No important variation in relative PFS estimates by subgroups were observed. In addition, a non-statistically significant treatment-related difference in overall survival (OS) was detected (HR 0.82; 95% CI 0.57 to 1.17; p = 0.137). 5 This result was highly likely to have been influenced by the very high level of patients in the BSC plus placebo arm switching to everolimus treatment. Partial or stable tumour response was seen in 69% of patients with everolimus against 32% in the placebo arm, and stable quality of life (QoL)/patient-reported outcomes in the everolimus arm compared with placebo.

Summary of submitted cost-effectiveness evidence

No published economic evaluations of everolimus in acute and/or metastatic RCC were identified and so the cost-effectiveness work focused on a new model and economic evaluation undertaken by the manufacturer.

A Markov state transition cost–utility model compared treatment with everolimus plus BSC with BSC alone, mirroring the question addressed in the RECORD-1 RCT. The four states were: stable disease, stable disease with adverse events, progressive disease and death. Outputs were expressed as cost per quality-adjusted life-year (QALY). The base-case incremental cost-effectiveness ratio (ICER) was £61,330; this estimate was somewhat reduced when a patient access scheme (PAS) was applied. The base-case ICER when PAS was applied (leading to a reduction in the cost of the drug) was £51,613. PAS was formally approved by the Department of Health during the course of the appraisal, which led to cost-effectiveness estimates in the original submission no longer being commercially-in-confidence. The components of the base-case ICER (with PAS) were an incremental cost of £15,704, mostly attributable to the acquisition cost of everolimus, and 0.304 additional QALYs (a mean of 0.607 QALYs for BSC plus everolimus, compared to 0.302 QALYs for BSC plus placebo).

The Inverse Probability of Censoring Weight (IPCW) statistical approach was used to adjust for crossover bias in the trial data. This meant that the estimate of OS being used in the model was an HR of 0.55. In response to a request for clarification, the submitting manufacturer indicated that the ICER, using the unadjusted OS estimate from RECORD-1, was £91,000 (this incorporates the reduction in drug price consequent on the PAS).

In a supplementary analysis, the submitting manufacturer also used an alternative statistical approach, the Rank Preserving Structural Favouring Time (RPSFT) method, to adjust for crossover. This produced a very similar HR estimate to the IPCW of 0.52, and when incorporated into the economic model also produced an ICER of £53,128. However, on examination, the ERG found a significant error in the supplementary analysis which, when corrected, raised this ICER value considerably. This error was in addition to those uncovered in the original submission (as outlined below).

Clinical effectiveness

The searches were appropriate and included all relevant studies. The main RCT, RECORD-1, was of high quality. No directly relevant ongoing trials were identified, but there did appear to be studies in progress investigating the role of everolimus earlier in the management of advanced RCC.

Cost-effectiveness

The overall approach taken to modelling was reasonable and the sources and justification of estimates were also generally reasonable.

Weaknesses

The evidence was based on only one completed and published RCT, albeit a well-conducted and adequately powered study. The interpretation was reasonable, although the ERG would have more clearly presented the trial results on the higher frequency of adverse events, of a severity likely to have an impact on patient QoL, in the everolimus arm of the trial relative to the placebo arm. For example, 40.1% of participants experienced adverse events and serious adverse events in the BSC plus everolimus arm, compared to 22.6% in BSC plus placebo. Further data illustrating the same point were identified in the Clinical Study Report. The trial data available indicated that patient health-related QoL was identical in the early stage of the trial, despite there being a response to treatment in the everolimus arm.

Although the OS results from the RECORD-1 RCT are clear and uncontroversial, indicating a survival improvement that could have resulted from chance alone, the adjustment of the results for switching placebo patients to everolimus following disease progression is an area of genuine academic debate, particularly concerning the most appropriate analytical method. The ERG took expert statistical advice on this (but could not replicate the calculation of OS estimates correcting for crossover). There is an alternative, possibly slightly preferred, approach to the IPCW method used in the original submission, RPSFT. The submitting manufacturer provided an additional analysis offering an estimate of OS and ICER based on this method, which actually led to little change from its original submission (see Summary of submitted cost-effectiveness evidence, page 44).

More seriously, however, a number of potential errors were identified in the model:

1. Transition probabilities were not converted to rates before multiplying by the HRs in the model.

2. Introduction of a structural error in implementation of the mortality HR, with the result that that the observed HR in the model in most cycles was substantially less than the HR intended, 0.55. This had the effect of seriously biasing the result in favour of everolimus.

3. Not introducing discounting into the model from the first cycle onwards (as opposed to introducing discounting from the first year onwards).

Of these potential errors, the second was the most serious. The ERG attempted to recalibrate the model to correct for the potential errors, and the result was an increase in the base-case model ICER to £65,231 (with PAS).

A further concern was that QoL data were not based on European Quality of Life-5 Dimensions sources. The resulting lack of confidence in the utility parameters in models dealing with advanced and metastatic RCC has been commented on in NICE appraisals before. A specific concern was that the small modelled difference in utility between stable disease and progressive disease (0.76 vs 0.68) does not seem consistent with the improvement in well-being likely to be present in practice.

Key issues

The key issues were:

• The existence of errors in the model and the effects of correcting for them.

• The validity of adjusting for crossover bias in RCTs and the appropriate statistical technique required to adjust for it.

• The effect of concerns about utilities (as outlined above).

Implications for research

Concerning the estimates of cost-effectiveness in RCC, the observations in the ERG report provide strong further support for research collecting rigorous estimates of utilities associated with the main health states likely to be experienced by patients with renal cell cancer. This specific appraisal highlights the possibility that the utility values associated with stable disease/progressive disease may vary depending on the number of additional potentially effective lines of further treatment available.

Switching in clinical trials for new cancer treatments as last line is a common and recurring problem in trial analysis. This STA considered a number of statistical approaches to adjustment. However, the issues highlighted have general applicability to other topics where switching from placebo to active treatment occurs when the primary end point has been reached, and this may be further enhanced by methodological research. Such research could, for example, focus on the appropriateness of alternative approaches in this context and towards the development of coherent guidelines for both the application of these statistical methods in health technology appraisals more generally as well as their integration in cost-effectiveness modelling.

Further investigation of the role of everolimus earlier in the management of RCC appears to be in progress and would not currently seem to be a priority for further research.

Disclaimers

The views and opinions expressed therein are those of the authors and do not necessarily reflect those of the Department of Health.