PET-PANC: multicentre prospective diagnostic accuracy and health economic analysis study of the impact of combined modality 18fluorine-2-fluoro-2-deoxy-d-glucose positron emission tomography with computed tomography scanning in the diagnosis and management of pancreatic cancer

Inclusion criteria

1. Patients with suspected pancreatic malignancy as defined by one or more of:

   1. focal lesion in the pancreas/bulky pancreas/dilated pancreatic duct (± metastases) detected on MDCT scan [± MRI/EUS/ultrasound]

   2. jaundice because of distal obstruction of the common bile duct or ampulla (not because of calculi) defined as serum bilirubin > 35 µmol/l

   3. serum CA19.9 > 37 kU/l.

2. Able to attend for PET/CT scan.

3. Able to undergo MDCT scan.

4. Able to attend for 12 months of follow-up.

5. Fully informed written consent given.

Exclusion criteria

1. Patients aged < 18 years.

2. Pregnancy.

3. Patients with poorly controlled diabetes.

¹⁸Fluorine-2-fluoro-2-deoxy-d-glucose positron emission tomography/computed tomography

Patients underwent FDG PET/CT scanning within a maximum of 2 weeks following informed consent. PET/CT was performed under carefully controlled conditions to ensure maximum accuracy of results. Patients fasted for 6 hours prior to the scan. To ensure accurate SUV measurements patients’ weight was obtained without shoes and coat and using a calibrated class III device that satisfied requirements defined in the Non-Automatic Weighing Instruments Directive 2003 [see http://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32009L0023 (accessed 27 June 2016)] and blood glucose was recorded using a calibrated Boehringer Mannheim glucometer (Boehringer Ingelheim Ltd, Bracknell, UK). For diabetes mellitus patients only patients with a fasting blood glucose not exceeding 10.0 mmol/l were scanned to reduce false-negative FDG PET/CT results. Patients drank between two and three glasses of water before the PET/CT scan to ensure good hydration, which contributes to a good-quality scan. Metal denture and other metallic devices were removed whenever possible to reduce CT artefacts, which distort FDG uptake measurements including SUV measurements in the vicinity of the artefact. Tracer was injected via a butterfly cannula under quiet conditions. For two-dimensional scanning 350–530 MBq of FDG was injected. In patients requiring a larger dose because of a larger body weight Administration of Radioactive Substances Advisory Committee (ARSAC) certificate holder approval was obtained before giving the larger dose. For three-dimensional scanning 150–350 MBq was injected. Patients remained quiet and inactive during the uptake period in a warm room to avoid artefacts including skeletal muscle FDG uptake and brown adipose tissue uptake. Patients emptied their bladders just prior to positioning on the scanner bed to avoid artefacts from FDG activity in the urinary bladder. The PET/CT emission scan started at 90 minutes after FDG injection. Scanning was carried out on a standard PET/CT table top, beginning at the groin and ending at the base of the orbits and with arms up if a single whole-body scan was performed. Routine local acquisition parameters were used. Data were reconstructed using ordered subsets expectation maximisation reconstruction parameters with CT for attenuation correction. The PET/CT scan was first reported at the local participating centre.

Central positron emission tomography/computed tomography reporting

A central PET/CT reporting facility was established at the Paul Strickland Scanner Centre at Mount Vernon Hospital, Northwood, Middlesex. Data transfer was via secure FTP server from the National Cancer Research Institute (NCRI) PET Core Laboratory. All PET/CT scan reports were reviewed by an expert in clinical PET/CT who was independent of the local centre. The majority of PET/CT scans were reviewed at the Paul Strickland Scanner Centre using the agreed standard proforma. Central review of PET/CT scans was also performed at Aintree University Hospital (Liverpool), St James’s University Hospital (Leeds), the Royal Free Hospital (London) and Southampton General Hospital using the agreed standard proforma.

Primary objective

• To determine the incremental diagnostic accuracy and impact of PET/CT in addition to standard diagnostic work-up in patients with suspected pancreatic cancer.

Secondary objectives

• To evaluate changes in diagnosis, staging and associated intended patient management as a result of the addition of PET/CT.

• To determine the cost-effectiveness of the addition of PET/CT in the diagnosis, staging and management of pancreatic cancer.

• To evaluate the impact of the addition of PET/CT in differentiating pancreatic malignancy from chronic pancreatitis.

• To identify which groups of patients would most benefit from PET/CT.

• To report the incremental diagnostic value of PET/CT for particular types of pancreatic tumour.

Planned treatment and follow-up

The patient then underwent planned management (D3). All patients were followed up for at least 12 months or until death if before 12 months. The patient completed a short quality of life questionnaire [European Quality of Life-5 Dimensions (EQ-5D)] [see www.euroqol.org/about-eq-5d.html (accessed 27 June 2016) and Appendix 2] after consent and at each 3-monthly outpatient review. The EQ-5D is a standardised instrument for use as a measure of health outcome and provides a simple descriptive profile and a single index value for health status that can be used in the clinical and economic evaluation of health care.

Target conditions

The analyses of test accuracy and staging considered the following target conditions:

1. pancreatic cancer (for the primary objective)

2. stage of pancreatic cancer (UICC TNM classification for resectable, borderline resectable and unresectable disease)

3. chronic pancreatitis

4. particular types of pancreatic tumour.

Reference standard(s)

The reference standard for diagnosis was a clinical judgement made by an independent expert based on histology (either biopsy or resection) or clinical outcome at the 12-month assessment. The expert agreed on the appropriate staging for each patient with pancreatic cancer and the appropriate management, to be used as reference diagnoses (D4). The process is described below:

• Stage 1. The expert initially received a patient histology report for the target conditions (1–4) and information about the clinical status of the patient at 12 months (but excluding all information from investigations made at baseline and the PET/CT test results). This was according to the minimum data set of the Royal College of Pathologists,62 in a standard format for resection histology; this included pathological staging (pTNM) for tumours.

• Stage 2. If the expert was unable to make a firm reference diagnosis based on the above information, results of baseline investigations were released but not the PET/CT investigation. The two-stage process was planned so that the expert’s initial decision was not contaminated by the standard work-up of either set of test results and never by the PET/CT scan results and to avoid incorporation bias. Finally, the expert was asked to judge the appropriateness of management for each patient and if a change prompted by PET/CT was appropriate.

Radiology quality assurance

Central radiology review of 10% of the MDCT scans was carried out by Dr Jonathan Evans, Consultant Radiologist (Royal Liverpool and Broadgreen University Hospitals NHS Trust), for quality assurance. CT scans selected from each centre for central reporting were requested by the Liverpool Clinical Trials Unit (LCTU). Anonymised CT images in DICOM (Digital Imaging and Communications in Medicine) format were provided on disk and sent to the Department of Radiology, Royal Liverpool and Broadgreen University Hospitals NHS Trust. On receipt disks were loaded into an RA600 import workstation (GE Healthcare, Chalfont St Giles, UK). The scan was opened on the RA600 and the images exported from the RA600 to the Picture Archiving and Communication System (PACS; Carestream Health Inc., Hemel Hempstead, UK). Once all of the images had arrived on the PACS, they were post processed and the radiologist was able to report the images. MDCT scans were reported using the agreed standard proforma.

Pathology quality assurance

Central pathology review was carried out by Professor Fiona Campbell, Consultant Gastrointestinal Pathologist (Royal Liverpool and Broadgreen University Hospitals NHS Trust). The histology slides from all of the biopsies and approximately 10% of all resection specimens (with at least two specimens selected from each centre) were requested after completion of actual diagnosis and management (D3). Slides were sent directly to the Department of Pathology, Royal Liverpool and Broadgreen University Hospitals NHS Trust, and reported using the agreed standard proforma. Slides were returned to each centre when reviewed.

Positron emission tomography/computed tomography quality assurance

A PET/CT Core Laboratory facility was set up as part of the NCRI PET Research Network Clinical Trials Network at the PET imaging centre at St Thomas’s Hospital, London. PET/CT data were transferred in anonymised DICOM part 10 format. The submitted images were required to include the attenuation-corrected PET, the non-attenuation-corrected PET and the CT images. The recommended method for electronic data transfer from the NHS PET centres was via the NHS Secure File Transfer Service. All other sites with appropriate internet access used the NCRI Core Laboratory secure FTP server. The laboratory ensured that images acquired from participating centres were of comparable quality.

Interim analysis

The interim analysis used data from the 187 patients of the first 200 registered who had complete data (for D1–D3 diagnoses) by the cut-off date of 28 May 2012. Of these, 82 were deemed to have pancreatic cancer according to D3, giving a prevalence of 43.9% (which was consistent with the original assumption of 47%; p = 0.39).

Interim results (Tables 1 and 2) were used to estimate the correlation between tests and hence the estimated required total sample size, based on the formulae in the previous section from Alonzo et al. 63 From Table 1 (diseased patients), TPPR = 66/82 = 80.5% and TPR_B = 69/82 = 84.1%. Thus, both quantities are larger than originally assumed. Applying these updated estimates into the sample size formula for n₁ gives 341.6.

From Table 2 (non-diseased patients), TNNR = 69/105 = 65.7% and TNR_B = 73/105 = 69.5%. Applying these updated estimates into the sample size formula for n₂ gives 144.2.

Hence, taking the maximum of the two figures and rounding up, the estimated sample size is 342. However, it was noted that this estimated sample size was primarily driven by the numbers of discordant test results, both of which were small, and as a consequence there was considerable uncertainty in the estimate. The bootstrap 80% CI of the estimated sample size was 231 to 464. To ensure that there was a good chance that the study was adequately powered for the primary analysis, the upper value was chosen, with a specific target of 500 agreed on to account for possible dropouts (estimated at 5.5% after the first 200 registered patients).

Primary analysis

The primary outcome measure was relative sensitivity and relative specificity of the PET/CT scan (in range as per protocol) compared with the baseline MDCT scan with respect to a diagnosis of PDAC. These were obtained by comparing each diagnosis with the reference diagnosis made at D4 for the target conditions (pancreatic malignancy, chronic pancreatitis, other tumour types) with standard errors accounting for pairing obtained using the formula from Alonzo et al. 63

The analyses were repeated both using patients who were outside the range for uptake time and/or blood glucose and excluding such patients to assess the impact on estimates.

The homogeneity of patients across sites was assessed by comparing disease prevalences using a funnel plot. 64 This involved plotting prevalence against sample size and observing whether the points lay outside of 95% or 99% control limits.

Secondary analyses

Additional analyses

Protocol version 1 (12 January 2010) to protocol version 2 (1 March 2011)

• Alteration of inclusion criterion from ‘able to attend for 12 months’ follow-up’ to ‘able to attend for up to 12 months’ follow-up’ to reflect the potentially unpredictable and poor prognosis of patients with pancreatic malignancy.

• Addition of the exclusion criterion ‘patients with poorly controlled diabetes’.

• Inclusion of a random blood glucose test at baseline to the schedule of assessments.

• Clarification that the post-PET/CT diagnosis (D2) could be conducted either at the MDT meeting or by the investigator because of capacity constraints at MDT meetings.

• Confirmation that the NCRI PET Research Network PET Core Laboratory at St Thomas’s Hospital, London, would provide the core laboratory function and that central clinical reporting would be conducted at the Paul Strickland Scanner Centre.

• Clarification that only serious adverse events as defined in the protocol should be reported to the co-ordinating centre within 24 hours of a site becoming aware of an event.

Protocol version 2 (1 March 2010) to protocol version 3 (1 September 2011)

• Frequency of the Independent Safety and Data Monitoring Committee (ISDMC) meeting defined to be every 6 months to reflect the recruitment period.

• Alteration to the PET/CT protocol:

  • suitable fasting blood glucose level of patients able to undergo a PET/CT scan increased from 7 mmol/l to 10 mmol/l to more adequately accommodate patients with diabetes mellitus, following a recommendation of the Trial Steering Committee

  • specific instructions included for research sites for preparation of patients with type I and type II diabetes mellitus, following a recommendation of the Trial Steering Committee

  • redefinition of injected activity to take account of variation in scanning equipment in scanning centres and redefinition of the position of the patient for scanning to prevent unnecessary exposure to radiation-sensitive tissues: ‘begin scanning at the groin and end at the vertex’ altered to ‘begin scanning at the groin and end at the base of the orbits’.

Protocol version 3 (1 September 2011) to protocol version 4 (10 September 2012)

• Sample size redefined following the interim analysis.

• Clarification on the requirement of the PET/CT scan to be performed within 2 weeks of the baseline assessment. The time limit was recommended to ensure that patients did not suffer unnecessary delays in diagnosis and management decisions because of participation in the study.

• Clarification on the scheduling and format of follow-up visits.

Reference standard diagnosis (D4) of patients

The frequency of each confirmed diagnosis type (the D4 reference standard) is provided in Table 5. If patients had more than one diagnosis, the primary diagnosis was used. The largest single group of patients was those with pancreatic cancer. Other types of tumour included cholangiocarcinoma, periampullary carcinoma and neuroendocrine tumour. The other main groups consisted of those patients with benign cystic neoplasms and chronic pancreatitis. The overall disease frequencies represent the typical case mix found in a UK pancreatic MDT and reflect UK practice.

Reference standard staging for patients with in-range positron emission tomography/computed tomography

For patients with a tumour the reference standard stagings are detailed in Table 6. All tumour types are included in the second column and patients with PDAC only are included in the third column. Overall, 20% of patients with pancreatic cancer had locally advanced or metastatic disease.

Reference standard management for patients with an in-range positron emission tomography/computed tomography scan

The management outcomes for patients are detailed in Table 7. Patients may have had more than one management outcome, for example resection plus adjuvant therapy. The majority of patients who underwent resection had a right-sided (standard or pylorus-preserving Whipple) procedure. Twenty-six patients were found to be inoperable at the time of surgery and underwent either surgical bypass or open and shut laparotomy.

Patients with positron emission tomography/computed tomography and a D4 diagnosis (intention-to-treat population, n = 583)

Among the 583 patients with a D4 diagnosis the overall disease prevalence (pancreatic cancer) was 47.7% (95% CI 43.6% to 51.7%). Table 9 provides the 2 × 2 table for MDCT and PET/CT diagnosis by PDAC status (D4).

The sensitivity and specificity of the initial MDCT scan (D1) diagnosis were 87.4% (95% CI 83.5% to 91.3%) and 69.8% (95% CI 64.7% to 75.0%), respectively. The sensitivity and specificity of the subsequent PET/CT scan (D2) diagnosis were 90.3% (95% CI 86.8% to 93.7%) and 76.1% (95% CI 71.3% to 80.9%), respectively.

Table 10 shows the MDCT- and PET/CT-based diagnoses among patients with and without PDAC. Of main interest were the numbers of discordant patients (those whose diagnosis differed at each stage) as this drives the relative sensitivity and specificity of PET/CT compared with MDCT. In each case there was a greater number of cases in which the PET/CT diagnosis was correct than in which the MDCT diagnosis was correct, indicating that PET/CT performs better.

Table 11 provides the relative sensitivity and specificity for PET/CT compared with MDCT with the corresponding 95% CIs and two-sided p-values. Using all patients with a D4 diagnosis, including those out of range for either blood glucose or uptake time, the relative sensitivity did not attain statistical significance (1.03; p = 0.083). In contrast, the 9% improvement in specificity was statistically significant (p = 0.005). There were corresponding improvements in PPV and negative predictive value (NPV) but only the effect on PPV was statistically significant.

Patients with positron emission tomography/computed tomography within the uptake range and a blood glucose of < 10 mmol/l (per-protocol population, n = 550)

Among the 550 patients with a D4 diagnosis and within-range uptake and blood glucose, the overall disease prevalence was 47.5% (95% CI 43.3% to 51.6%). Table 12 provides the 2 × 2 table for MDCT and PET/CT diagnosis by PDAC status (D4).

The sensitivity and specificity of the initial MDCT scan (D1) diagnosis were 88.5% (95% CI 84.6% to 92.4%) and 70.6% (95% CI 65.3% to 75.8%), respectively. The sensitivity and specificity of the subsequent PET/CT scan (D2) diagnosis were 92.7% (95% CI 89.5% to 95.9%) and 75.8% (95% CI 70.8% to 80.7%), respectively.

Table 13 shows the MDCT- and PET/CT-based diagnoses among patients with and without PDAC. The numbers of discordant patients (those whose diagnoses differed at each stage) drive the relative sensitivity and specificity of PET/CT compared with MDCT. In each case there was a greater number of cases in which the PET/CT diagnosis was correct than in which the MDCT diagnosis was correct, indicating that PET/CT performs better.

Excluding patients with out-of-range uptake or blood glucose leads to an improvement in the sensitivity of PET/CT (92.7% vs. 90.3%) and the analysis based only on these per-protocol patients shows a significant improvement in both relative sensitivity (1.05; p = 0.010) and specificity (1.07; p = 0.023) (Table 14). The PPV and NPV are specific to the prevalence of the disease in the population, which has to be estimated from the data. Here, the relative NPV was statistically significant (p = 0.031) but the relative PPV did not attain statistical significance.

Assessment of incremental diagnostic benefit

The estimated incremental likelihood ratios demonstrate that the results of PET/CT significantly improve diagnostic accuracy in all scenarios (Figure 4). All results are significant with a bootstrap p-value of < 0.0002. A positive diagnosis from MDCT increased the odds of PDAC by 201% (95% CI 157% to 269%), whereas a negative diagnosis decreased the odds by 84% (95% CI 79% to 88%).

FIGURE 4.

Tree diagram of incremental likelihood ratios (LRs). The values of the LRs are given following positive or negative MDCT and then positive or negative PET/CT. Figures in parentheses represent 95% CIs.

Following a positive diagnosis from MDCT, a positive diagnosis at PET/CT further increased the odds of PDAC by 55% (95% CI 28% to 85%). The most substantial incremental effect of PET/CT was a negative diagnosis following a positive diagnosis from MDCT. In this case the odds of PDAC decreased by 95% (95% CI 90% to 99%).

Following a negative diagnosis from MDCT, a positive diagnosis on PET/CT increased the odds of PDAC by 538% (95% CI 249% to 1127%). A second negative diagnosis on PET/CT following a negative diagnosis from MDCT further decreased the odds of PDAC by 46% (95% CI 25% to 66%).

These results suggest that there is a substantial diagnostic benefit for patients of receiving a PET/CT scan after a potential PDAC is detected on MDCT. Patients with no PDAC detected on PET/CT following a positive MDCT scan are unlikely to have PDAC (NPV of 89%). In contrast, although a positive diagnosis on PET/CT following a negative MDCT scan increases the odds of PDAC, the overall diagnosis is still highly uncertain (PPV of 50%). This latter result suggests that if both tests are administered there is benefit in combining the results rather than taking the PET/CT result alone. In particular, patients who were positive on MDCT and PET/CT are much more likely to have PDAC than patients who are positive only on PET/CT (PPV of 80% vs. 50%). The result of the MDCT is less important if the PET/CT result is negative (NPV of 93% if both tests are negative vs. NPV of 89% if the PET/CT scan is negative after a positive MDCT scan).

Certainty of diagnosis

Using the quantitative scale for assessing clinicians’ perceptions of diagnostic certainty (0 = no certainty and 1 = absolute certainty), there was a significant increase in the median certainty score after the PET/CT scan [0.80, interquartile range (IQR) 0.25] compared with before (0.68, IQR 0.34) (p < 0.0001) (Table 15). A box plot of the distributions of perceived certainty before and after the PET/CT scan is shown in Figure 5.

FIGURE 5.

Distribution of perceived diagnostic certainty pre and post PET/CT.

A similar pattern was observed using a qualitative assessment of whether certainty after PET/CT was more, less or the same as that before PET/CT. In 322 (59%) cases clinicians were more certain following PET/CT and in 25 (5%) cases they were less certain. In 201 (37%) cases the level of certainty was the same before and after PET/CT. Clinicians were 12 times more likely to believe that they were more certain in their diagnosis than they were to be less certain (p < 0.0001) following PET/CT.

Assessing the accuracy of clinicians’ perceived certainty using the Brier score (Table 16) also indicated that clinicians’ predictions (interpreting their level of certainty as a probability) were significantly improved by the PET/CT scan result (reduction in Brier score, i.e. prediction error, of 0.045, 95% CI 0.024 to 0.066; p < 0.001).

Changes in diagnosis following positron emission tomography/computed tomography

The discordance of diagnoses following MDCT and then PET/CT can be seen in Table 17. This table shows the MDCT- and PET/CT-based diagnoses among patients with and without PDAC. The numbers of discordant patients (those whose diagnoses differed at each stage) drives the relative sensitivity and specificity of PET/CT compared with MDCT. In each case there was a greater number of cases in which the PET/CT diagnosis was correct than in which the MDCT diagnosis was correct, indicating that PET/CT performs better.

Positron emission tomography/computed tomography changed the diagnosis correctly for both the diagnosis of pancreatic cancer and the diagnosis of not pancreatic cancer in 46 patients (8.3%). PET/CT incorrectly changed the diagnoses in 20 patients (3.6%). Considering the diagnosis of malignancy in general, PET/CT correctly changed the diagnosis in 48 patients (8.7%) and incorrectly changed the diagnosis in 11 patients (2%) (Table 18).

Changes in staging following positron emission tomography/computed tomography

For the purposes of examining change of stage, patients were grouped into four categories: (a) no tumour/IA/IB/IIA, (b) IIB, (c) III and (d) IV (for the breakdown of changes in staging according to the groups see Appendix 3). The accuracy of staging among patients in stages IA/IB/IIA was slightly worse under PET/CT than under MDCT, with more incorrect (eight patients) than correct (six patients) changes. However, the difference was not statistically significant (p = 0.79). Significantly more patients with stage IIB were correctly staged by PET/CT than by MDCT, with 22 patients (21%) moving to the correct stage compared with five patients (5%) moving to the wrong stage (p = 0.002). PET/CT had no significant effect on the accuracy of staging for patients in stage III. Significantly more patients in staging group IV were changed to the correct stage (27 patients) than moved to an incorrect stage (one patient) (43% vs. 2%; p < 0.001). A summary of stage changes is provided in Table 19.

Overall, the effect of PET/CT was to change to the correct staging group significantly more often than to change from the correct group to an incorrect group (p < 0.001). However, the majority of the benefit was in correct changes to stage IIB or stage IV.

Changes in planned management

Using the question from the D2 (post-PET/CT scan) form, ‘Has the PET/CT scan influenced your planned management of this patient’s disease?’, the PET/CT scan was perceived to have changed the planned management in 250 (45%) patients. The proportion whose management was affected was slightly higher among patients whose final diagnosis was not pancreatic cancer [139 (48%) vs. 111 (43%) with pancreatic cancer] but the difference was not statistically significant (Table 20).

Using the questions from the D3 (confirmed diagnosis) form, ‘Did the patient follow the course of treatment recommended at D1?’ and ‘Did the patient follow the course of treatment recommended at D2?’, 70% of patients did not undergo a change in management throughout the study Table 21). In total, 15% underwent a change in management from either that considered at D1 (pre PET/CT) or that considered at D2 (post PET/CT). A significantly higher proportion of patients (11% vs. 4%; p = 0.0002) followed the management plan recommended after PET/CT (and not that recommended after MDCT) than followed the MDCT management plan (and not that recommended after PET/CT).

Clinically important changes resulting from positron emission tomography/computed tomography

Table 22 details the frequency of different types of clinically important management changes. The most common change was changing from resection to no resection, which occurred in 61 patients, representing 11% of all patients and 21% of patients scheduled for some kind of resection after MDCT. Changing from no further investigation to some form of further investigation/clinical follow-up occurred in 58 patients, representing 11% of all patients and 13% of those initially not scheduled for further investigation. In total, 13% of patients not thought to need surgical resection following MDCT were then planned for resection following PET/CT. Changes relating to the commencement or cessation of chemotherapy were less common.

Out-of-range patients

A GEE subgroup analysis (Table 26) comparing the patients within range for both uptake time and blood glucose with those out of range for either showed a statistically significant deterioration in sensitivity among out-of-range patients at just 52.9% (p < 0.0001). There was evidence that the relative sensitivity of PET/CT compared with MDCT was much lower among those patients, being worse than that of MDCT (relative sensitivity 0.75; p = 0.005). There was also some evidence of lower sensitivity of MDCT among this group. There was no evidence of a difference in PET/CT or MDCT specificity or relative specificity for this group.

Among patients who had an out-of-range uptake time (Table 27) there was some evidence of lower PET/CT sensitivity (71.4%), although this did not attain statistical significance. There was no evidence of differences in specificity.

Among patients who had a blood glucose level of > 10 mmol/l (Table 28) there was evidence of considerably lower PET/CT sensitivity (40.0%; p = 0.0001); although the sensitivity of MDCT was also impacted, there was nevertheless also strong evidence of reduced relative sensitivity for these patients, with PET/CT performing worse than MDCT (relative sensitivity 0.67; p = 0.027). Again, there was no evidence of differences in specificity.

These results confirm the necessity for PET/CT to be administered after the correct uptake time and the unsuitability of PET/CT for patients with poorly controlled blood glucose.

Further generalised estimating equation subgroup analyses

In the remaining analyses the subgroup of patients with the per-protocol uptake time and within-range blood glucose was considered. Tables 29–34 present the results of univariate GEE analyses for the effect of sex, age (dichotomised at < 65 years or ≥ 65 years), WHO performance status (0 vs. 1, 2 or 3; see Appendix 5) and eligibility criteria. There was no evidence of any differences in sensitivity or specificity for either test with respect to sex (see Table 29) or age (see Table 30). However, both MDCT and PET/CT had significantly higher sensitivity among patients with a WHO status of ≥ 1 than among those with a WHO status of 0, but conversely had significantly lower specificity (see Table 31). Both MDCT and PET/CT had significantly lower specificity among patients with jaundice because of distal obstruction of the common bile duct or ampulla (see Table 33). Moreover, there was also evidence that PET/CT was not more effective than MDCT for this group. There was no evidence of differences in sensitivity or specificity in relation to the other eligibility criteria (see Tables 32 and 34).

Malignant cystic neoplasm

The prevalence of malignant cystic neoplasm in the patient cohort was 1.5% (95% CI 0.5% to 2.5%). The small number of patients diagnosed with malignant cystic neoplasm (Table 35) means that statistical tests comparing the diagnostic tests have low power to detect small or moderate effects.

Both MDCT and PET/CT had a sensitivity of 75.0% (95% CI 45.0% to 99.9%). The specificity of PET/CT was higher than that of MDCT [96.1% (95% CI 94.5% to 97.8%) vs. 92.8% (95% CI 90.6% to 95.0%)]. The 3.6% (95% CI 1.8% to 5.4%) improvement in specificity between tests was statistically significant (p < 0.001) (Table 36). The very small number of cases of malignant cystic neoplasm meant that the significant difference in specificity did not translate into a significant difference in either PPV or NPV.

Cholangiocarcinoma

The prevalence of cholangiocarcinoma in the patient cohort was 8.0% (95% CI 5.7% to 10.3%). The small number of patients diagnosed with cholangiocarcinoma (Table 37) means that statistical tests comparing the diagnostic tests have low power to detect small or moderate effects.

Both MDCT and PET/CT had a relatively high specificity (97.8% and 98.8%, respectively) but low sensitivity (25% in each case). There were no significant differences in either sensitivity or specificity (Table 38).

Similarly, although the improvement in sensitivity (for cholangiocarcinoma) for the PET/CT scan translated into an improvement in PPV, the effect was not significant. Although a positive diagnosis of cholangiocarcinoma on PET/CT or MDCT greatly increases the odds of a patient having cholangiocarcinoma, there was still considerable uncertainty about the diagnosis (PPV 50% for MDCT and 65% for PET/CT). Moreover, the NPVs were very close to the original prevalence reflecting that the tests have poor negative likelihood ratios (0.77 and 0.76 for MDCT and PET/CT, respectively).

Neuroendocrine tumours

The prevalence of neuroendocrine tumours in the patient cohort was 4.9% (95% CI 3.1% to 6.7%). The small number of patients diagnosed with neuroendocrine tumour (Table 39) means that statistical tests comparing the diagnostic tests have low power to detect small or moderate effects.

Both MDCT and PET/CT had a very high specificity (99.4% and 98.7%, respectively) but low sensitivity (44.4% in both cases). The relative specificity and relative sensitivity of MDCT and PET/CT were not significantly different (Table 40).

The slightly lower specificity for PET/CT meant that the PPV was estimated to be lower for PET/CT. However, the difference was not statistically significant. Similarly, the NPV was slightly lower but again the difference was not significant.

Periampullary carcinoma

The prevalence of periampullary carcinoma in the patient cohort was 6.9% (95% CI 4.8% to 9.0%). The small number of patients diagnosed with periampullary carcinoma (Table 41) means that statistical tests comparing the diagnostic tests have low power to detect small or moderate effects.

Both MDCT and PET/CT had a relatively high specificity (95.9% and 97.2%, respectively) but relatively low sensitivity (71.1% and 65.8%, respectively) (Table 42). The relative specificity of PET/CT compared with MDCT was statistically significant (p = 0.034) but the practical improvements were outweighed by the observed, although non-significant, reduction in sensitivity.

Although PET/CT gave an improved PPV, the relative PPV was not significantly different from 1. Moreover, the estimated NPV was worse for PET/CT than for MDCT, although this differences was also non-significant.

Malignant compared with benign disease

The prevalence of any malignant disease among the patient population was 69.8% (95% CI 66.0% to 73.7%). Table 43 provides the 2 × 2 table for MDCT and PET/CT diagnosis by malignant or benign disease status (D4).

Both MDCT and PET/CT scans had high sensitivity with respect to detecting any malignancy (97.4% and 97.7%, respectively) but there was no evidence that PET/CT had better sensitivity (p = 0.763) (Table 44).

In contrast, the specificity of PET/CT was considerably higher than that for MDCT (68.7% vs. 47.0%, relative sensitivity 1.46; p < 0.0001). The improvement in specificity also corresponded with a statistically significant improvement in PPV, increasing from 80.9% for MDCT to 87.8% for PET/CT (p = 0.0002). The NPV also improved but by a non-significant margin (relative NPV 1.05; p = 0.24).

Incremental diagnostic benefit for malignancy versus benign

Using the Knottnerus method, estimates of the incremental likelihood ratios corresponding to a positive or negative PET/CT diagnosis with respect to whether the tumour was malignant or benign were obtained. In each case, the results of the PET/CT scan had a statistically significant impact on the odds of malignancy. The results suggest particular benefits of administering PET/CT after a positive MDCT scan. A positive PET/CT assessment following a positive MDCT assessment was estimated to increase the odds of malignancy by a further 88.7% (95% CI 55.4% to 134.8%) whereas a negative PET/CT assessment following a positive MDCT assessment reduced the odds of malignancy by 97.2% (95% CI 94.4% to 99.5%), meaning that malignancy is very unlikely for these patients (NPV 89.4%). A positive PET/CT assessment following a negative MDCT assessment increased the odds of malignancy by 680% (95% CI 106% to 2735%). However, there was still considerable uncertainty in the diagnosis of these patients (PPV 50.0%). A negative PET/CT assessment following a negative MDCT assessment decreased the odds of malignancy by 60.6% (95% CI 25.6% to 90.3%) and these patients were very unlikely to have a malignancy (NPV 94.7%) (Figure 6).

FIGURE 6.

Tree diagram of incremental likelihood ratios (LRs). The values of the LRs are given following positive or negative MDCT and then positive or negative PET/CT. Figures in parentheses represent 95% CIs.

Analysis of maximum standardised uptake value

A SUV_max. measurement for the primary tumour was available only if a primary tumour was identified. In addition, in a further 49 cases no SUV_max. was recorded despite identification of a primary tumour (Table 45).

The following analyses use only the data from the 362 patients for whom a primary tumour was identified and the SUV_max. was recorded.

Diagnostic accuracy of the maximum standardised uptake value for pancreatic ductal adenocarcinoma

Although the level of SUV_max. provided statistically significant discrimination between patients with and without pancreatic cancer, the level of discrimination achieved was quite poor. Figure 7 demonstrates that the area under the curve (AUC) is 0.64 (95% CI 0.57 to 0.70). The optimal cut-off point, if the aim is to minimise the sum of the error probabilities of the test, is 6.2, meaning that all tumours with a SUV_max. > 6.2 would lead to a positive diagnosis of PDAC. However, such a test has an estimated sensitivity of only 70.3% and specificity of only 59.4%.

FIGURE 7.

Receiver operating characteristic curve for the diagnosis of PDAC based on SUV_max.

Diagnostic accuracy of the maximum standardised uptake value for chronic pancreatitis

Patients with chronic pancreatitis for whom a primary tumour was identified on the PET/CT scan tended to have a lower SUV_max. than those without chronic pancreatitis. However, note that only 20 of the 41 patients with chronic pancreatitis had an identified primary tumour with a measured SUV_max.

The SUV_max. level was a statistically significant indicator of the presence of chronic pancreatitis; however, the level of discrimination was relatively modest. The estimated AUC was 0.75 (95% CI 0.64 to 0.85) (Figure 8). The optimal cut-off point, if the aim is to minimise the sum of the error probabilities of the test, is 6.4, meaning that all tumours with a SUV_max. < 6.4 would lead to a positive diagnosis of chronic pancreatitis. Such a test has an estimated sensitivity of 85% and specificity of 60%. This cut-off value is also very similar to the optimal value for assessing PDAC, suggesting that an approach of assuming PDAC for a SUV_max. of > 6.3 and chronic pancreatitis for a SUV_max. of < 6.3 has some merit.

FIGURE 8.

Receiver operating characteristic curve for the diagnosis of chronic pancreatitis based on SUV_max.

Survival analysis

In total, 186 patients (among the 550 with a per-protocol PET/CT scan and a D4 diagnosis) were observed to die. Assuming non-informative dropout, estimated 6-month survival was 82.8% (95% CI 79.7% to 86.0%) whereas 12-month survival was 69.0% (95% CI 65.1% to 73.1%). Survival was substantially lower in patients with a confirmed diagnosis of PDAC, with 6-month survival being 71.4% (95% CI 66.0% to 77.2%) and 12-month survival being 50.9% (95% CI 44.9% to 57.6%) compared with 92.6% (95% CI 89.6% to 95.7%) and 84.9% (95% CI 80.8% to 89.2%), respectively, for patients without PDAC. Survival was lower in patients with any malignant disease than in patients with benign disease (Table 47 and Figures 9 and 10).

FIGURE 9.

Kaplan–Meier estimate of survival for all patients.

FIGURE 10.

Kaplan–Meier estimates of survival for patients with and without PDAC.

Survival by stage of disease

Among patients with a D4 diagnosis of PDAC, 95% had staging information at D2. Patients assessed to be in stage IA/IB at D2 or not assigned a stage had similar 12-month survival to those assessed to be in stage IIA or IIB (68.3% vs. 65.5%) (Table 48 and Figure 11). Patients in stages III and IV had poorer 12-month survival (44.8% and 21.0%, respectively).

TABLE 48 - Estimated 12-month survival among PDAC patients by diagnosed stage at D2

Stage (determined at D2)	No. of patients	12-month survival (95% CI) (%)
IA/IB/no tumour identified	87	68.3 (59.9 to 79.2)
IIA/IIB	66	65.5 (54.5 to 78.7)
III	29	44.8 (29.3 to 68.7)
IV	79	21.0 (13.4 to 33.1)

FIGURE 11.

Estimated survival among PDAC patients grouped by diagnosed stage at D2.

Existing economic evidence

Evidence on the cost-effectiveness of PET/CT in oncology generally is mixed but largely skewed towards the procedure not being cost-effective. Some sensitivity analysis brings the incremental cost-effectiveness ratio (ICER) below National Institute for Health and Care Excellence (NICE) thresholds of £20,000–30,000 per quality-adjusted life-year (QALY), but largely it is found not to be cost-effective in a UK context. This seems to depend on the type of cancer screened for and the availability of reliable data. More research is needed as and when reliable data become available.

A rapid review electronic database search for relevant economic evaluations concerning the use of PET/CT imaging in oncology was undertaken, with searches undertaken in The Cochrane Library [Database of Abstracts of Reviews of Effects (DARE), Health Technology Assessment (HTA) database and NHS Economic Evaluation Database (NHS EED)], EMBASE, PubMed and Web of Science. Studies were eligible for inclusion if they included full health economic evaluations (cost-effectiveness, cost–utility, cost–benefit analyses) comparing PET/CT (i.e. integrated PET and MDCT scan) with different invasive and non-invasive diagnostic strategies in the clinical work-up of cancer patients. Databases were searched between March 2010 and May 2012 (see Appendix 12 for an example of the search strategies).

Our inclusion criteria identified four studies66–69 (see Appendix 6). Two of these were from the UK,68,69 one was from Denmark66 and one was from Germany. 67 One study was a cost-effectiveness analysis,66 one was a cost-effectiveness and cost–utility analysis67 and two were cost–utility analyses. 68,69 Two studies used decision-analytical models68,69 and the other two used Cox proportional hazards and generated cost-effectiveness acceptability curves (CEACs). 66,67 Cost-effectiveness was measured in terms of cost per avoided thoracotomy66 and cost per QALY. 67–69 The evidence on the cost-effectiveness of PET/CT technology in oncology was mixed. Cost-effectiveness varied according to the type of cancer screened for and varied further with the different assumptions made about the relevant costs and benefits.

Outcome measures

The outcome measure for the PET-PANC economic analysis was the QALY. Both NICE and the National Coordinating Centre for Health Technology Assessment (NCCHTA) support the use of QALYs as an outcome measure in technology assessment. 70,71 QALYs are an index of health gain, combining survival and HRQoL. 72 We calculated QALYs using the AUC approach by weighting survival by HRQoL weights (HRQoL utilities values) over the 12-month follow-up period of the PET-PANC study. When possible, the AUC was calculated in four parts representing each 3-month follow-up period. For each participant the area under the HRQoL curve gives the total QALYs. HRQoL utilities were generated using European Quality of Life-5 Dimensions three-level version (EQ-5D-3L)73 data (n = 452) collected in the PET-PANC study. The EQ-5D is a validated generic health-related preference-based measure developed by the European Quality of Life group. The EQ-5D provides a single index (utility) value for health status for each patient. In our model, we calculated the difference in mean QALYs associated with the change in patient management resulting from the use of PET/CT (over and above those for MDCT alone).

Life-years gained

When patients were alive at 12 months post baseline, life-days were assumed as 365; otherwise, life-days were calculated by subtracting date of death from date of entry into the study. Life-years were calculated as life-days divided by 365.

Resource use and costs in secondary and primary care

In the PET-PANC study we measured costs and outcomes from a NHS perspective. We recorded the type and frequency of patient contacts with NHS secondary and primary care. These included investigations, treatment and palliation and other elements of secondary, primary and pharmaceutical care. Within cancer care, we focused on the costs of PET/CT, surgery, chemotherapy, radiotherapy, other drugs, outpatient appointments and day care and inpatient stays. We collected this information as part of the case report form (CRF). We took special care to check and maintain the quality of these data. This led to a complete set of secondary and primary care use data for the 279 participants for whom cost data were available for use in our economic model.

Sources of unit costs

We drew costs from a number of sources including the Unit Costs of Health and Social Care 201374 and NHS reference costs for 2012–13. 75 The Prescription Cost Analysis (PCA) 201376 was used for costing drugs. Estimates of the costs of PET/CT were obtained from NHS reference costs. All costs are in 2012–13 pounds sterling unless stated otherwise. A table referencing unit costs for each resource use category for secondary and primary care is provided in Appendix 7.

Missing data

No formal imputation was carried out on missing data; however, assumptions were made following the death of a patient. For patients who died during the course of the study, the follow-up period in which death occurred was calculated based on each follow-up period being 365/4 days long. EQ-5D utility data were assumed to be equal to zero from the end of the follow-up period in which the patient died. No assumption was made about costs for the follow-up period during which the patient died; however, costs for subsequent follow-ups were assumed to be zero.

Cost-effectiveness analysis

Our cost-effectiveness analysis was in three parts: model 1, in which we calculated the marginal cost per additional correct diagnosis of PDAC using PET/CT and MDCT alone; model 2, in which we calculated the budget impact of use of PET/CT; and model 3, in which we modelled the change in management of patients as a result of the use of PET/CT in diagnosis. All health economic modelling was performed in Microsoft Excel^® 2013 (Microsoft Corporation, Redmond, WA, USA).

Model 3: change in management of patients as a result of use of positron emission tomography/computed tomography in diagnosis – incremental decision model

For each management strategy (e.g. resection, chemotherapy), patients in the PET-PANC study were prescribed either strategy X or strategy no X following CT scanning at time point D1. In some cases this management strategy was changed following PET/CT scanning (D2). Figure 12 presents a schematic of this change for the management strategy of resection, where a, b, c and d represent the numbers of patients following each branch of the schematic.

FIGURE 12.

Study schematic (example of resection).

We mapped the data from diagnoses at D1 and D2 into two arms of a decision model, representing hypothetical management if diagnosis was via CT alone or PET/CT. This is represented in Figure 13. To assess the cost-effectiveness of the addition of PET/CT, only incremental costs and effects require consideration. From Figure 13 it can be seen that these incremental effects are represented entirely by the patients who experienced a change of management following D2.

FIGURE 13.

Decision-analytical model (example of resection).

The impact of change in management strategies on costs, QALYs and life-years was modelled with regression models using data gathered in the PET-PANC study. Regression inputs reflect the categories of data collected on the CRF. To account for multiple treatments, QALYs and life-years gained (LYG) were modelled as decrements (1 – QALY) and (1 – LYG) and combined additively. Costs were also combined additively. When patients were moved from resection to no resection, this change indicated that the cancer had progressed beyond a state where resection would be beneficial, and bypass surgery or an open and shut laparotomy may have been performed. To adjust for this in the primary model, resection data were entered into the regression model as ‘resection: no metastases’ and ‘resection: with metastases’, the latter reflecting the case in which resection was not the optimal management.

In-house data indicated that bypass surgery is associated with increased mortality and a longer recovery phase than resection. A structural sensitivity analysis adjusted for this. Patients who were indicated a change of ‘stop resection’ were clinically assessed based on D3 and D4 and categorised whether the planned surgery would have been a bypass, open and shut laparotomy or resection, with costs and utilities adjusted accordingly in the model. Cost and utility adjustments for bypass and open and shut laparotomy were extracted from the study by a further breakdown of patient notes in the ‘resection’ and ‘drainage procedure’ categories of the CRF. For this model, ‘resection’ was entered as a single variable in the regression to avoid double counting.

Cost-effectiveness was assessed according to the ICER for the intervention, calculated as:

⁽⁴⁾ ICER = Incremental cost Incremental QALY .

An intervention with a lower ICER is deemed to be more cost-effective than one with a higher ICER. NICE considers an ICER of < £20,000–30,000 per QALY to mean that a health technology represents good value for money for the NHS in the UK.

Primary analysis was based on QALYs for the within-range cohort of patients (n = 550). As there are varying costs for PET/CT by department (nuclear medicine, clinical oncology or medical oncology), sensitivity analysis was carried out using alternative costing assumptions. Analyses are also presented for LYG. Secondary subgroup analyses were also carried out on all patients (n = 583), patients with a pancreatic cancer diagnosis at D1 (n = 316) and patients with a pancreatic cancer diagnosis at D1 who were suitable for resection (n = 217). For all models to retain power, costs, QALYs and LYG were based on those for the within-range patients. For the ‘within-range’ and ‘all patient’ groups, the cost of PET/CT was presumed to be instead of the cost of MDCT. For the PDAC and PDAC with resection groups, the cost of PET/CT was presumed to be in addition to the cost of MDCT.

A probabilistic sensitivity analysis (PSA) assessing the simultaneous uncertainty of model parameters is more informative than univariate deterministic sensitivity analysis. Correlation between regression coefficients is reflected in the PSA by use of Cholesky decomposition on the covariance matrix. Uncertainty in the cost of the PET/CT scan was represented by a normal distribution, with an assumed standard deviation of £50 to reflect uncertainty. For the PSA, when no patients experienced a given change in management, 0.1 patients were modelled as experiencing the change, to generate a distribution. This is to reflect changes in management that may occur but did not necessarily occur in the PET-PANC study. PSA was accomplished by drawing iteratively (10,000 times) from the generated distributions. CIs for incremental costs and QALYs were generated from the bootstrap simulations. A CEAC was constructed to depict the probability of the intervention being cost-effective for a given cost-effectiveness or payer threshold.

Quality-adjusted-life-years

Figure 14 shows the progression of the EQ-5D data over time based on diagnosis. Health state is seen to be worse for patients with PDAC, double cancer (pancreatic cancer plus another primary cancer) and malignant disease than for patients with benign cysts. We note that the numbers with double cancer are small (n = 15, 7, 9, 8 and 10 at the respective time points) and so we interpret these results, in particular the ‘peak’ at 3 months, with extreme caution. For all other diagnoses and time points, n > 40. These progressions are in line with what would be expected in the different disease areas and support the conclusion that patients within the study are representative of the wider population.

FIGURE 14.

European Quality of Life-5 Dimensions utility data over time by disease status.

Table 50 shows the QALY breakdown by age as calculated by the EQ-5D AUC. It seems that younger people have higher QALYs; however, care must be taken when interpreting these data as the numbers of younger people in the sample are very low.

Unit costs

In a UK NHS setting the cost of a standard CT scan as required to diagnose pancreatic cancer is £86 (lower quartile £79, upper quartile £95). 75 The corresponding cost for a PET/CT scan is £795 (lower quartile £793, upper quartile £795). This cost was sourced from nuclear medicine service costs and was the most conservative estimate (most costly). Costs pertaining to the different service descriptions are summarised in Table 51.

A cost of £709 can therefore be derived as the additional cost per patient of scanning with PET/CT in place of scanning with CT alone. If PET/CT is to be used for diagnosis subsequent to CT, then the additional cost per patient is equivalent to the cost of the PET/CT scan, that is, £795. For all analyses, when PET/CT and CT costs are considered alongside each other, both costs are taken from the same departmental source for consistency.

Comparing the PET/CT lower quartile with the CT upper quartile gives a lower bound on the marginal cost of £698. Comparing the PET/CT upper quartile with the CT lower quartile gives an upper bound on the marginal cost of £714.

Frequency of resource use

The mean frequency of use of primary and secondary care services over the 12 months of the study is shown in Table 52. The table in Appendix 7 provides the 3-monthly breakdown showing the number of patients in each follow-up period. It can be seen that the most frequent resource use in primary care is for community nurses, followed by practice nurses and general practitioners (GPs). For secondary care the most frequent resource use is for outpatient visits.

Costs over the 12-month study period

Table 53 shows the sources of unit costs used in the economic model. Appendix 8 provides further information on resource use data and unit costs.

The resultant mean total costs of service use by patients in the PET-PANC cohort are shown in Table 54, with the full table including the 3-month breakdown provided in Appendix 10. These costs are in UK pounds sterling for the year 2012–13. It can be seen that for primary care the highest cost was for GPs, followed by community nurses and practice nurses. For secondary care the highest cost was for inpatient stays.

Cost-effectiveness analysis

Probabilistic sensitivity analysis results

Probabilistic sensitivity analysis was performed as described in the methods section. The results of the 10,000 simulations for both nuclear medicine PET/CT costs and clinical oncology PET/CT costs for both model structures are plotted on cost-effectiveness planes in Figure 15. The planes illustrate the joint distributions of the incremental costs and QALYs. Blue and black lines indicate £20,000 per QALY and £30,000 per QALY thresholds, respectively. Points below the lines are considered cost-effective at the given threshold. An ‘×’ marks the point estimate of the ICER. For the conservative estimate associated with nuclear medicine in the primary model, 24% of simulations are in the north-east quadrant (higher cost, higher QALYs), 3% are in the north-west quadrant (higher cost, lower QALYs), 64% are in the south-east quadrant (lower cost, higher QALYs) and 9% are in the south-west quadrant (lower cost, lower QALYs). For clinical oncology costs in the primary model, 17% of simulations are in the north-east quadrant, 2% are in the north-west quadrant, 71% are in the south-east quadrant and 10% are in the south-west quadrant.

FIGURE 15.

Cost-effectiveness planes. (a) Nuclear medicine costs in the primary model; (b) clinical oncology costs in the primary model; (c) nuclear medicine costs in the bypass and open and shut laparotomy model structure; and (d) clinical oncology costs in the bypass and open and shut laparotomy model structure.

Modelling bypass and open and shut laparotomy associated with nuclear medicine costs, 90% of simulations are in the north-east quadrant, 4% are in the north-west quadrant, 6.19% are in the south-east quadrant and 0.3% are in the south-west quadrant. For clinical oncology costs, 68% of simulations are in the north-east quadrant, 4% are in the north-west quadrant, 27% are in the south-east quadrant and 1% are in the south-west quadrant.

Cost-effectiveness acceptability curves were generated, indicating the probability that PET/CT is cost-effective over a variety of cost-effectiveness thresholds. The CEACs are illustrated in Figure 16 and show that, for the primary model, for nuclear medicine costs the probability of being cost-effective at the £20,000 per QALY threshold is 82% and at the £30,000 per QALY threshold is 85%. For clinical oncology costs the probabilities are 88% at the £20,000 per QALY threshold and 90% at the £30,000 per QALY threshold; it is likely from this model that PET/CT would be cost-effective for the UK NHS. For the structural sensitivity analysis and conservative nuclear medicine costs, the probability of PET/CT being cost-effective at the £30,000 per QALY threshold is 28% and at the £20,000 per QALY threshold is 18%. At these costs it is unlikely that PET/CT would be cost-effective from the perspective of the UK NHS. Using clinical oncology as the source of PET/CT costs, the probability of PET/CT being cost-effective at the £30,000 per QALY threshold is 60% and at the £20,000 per QALY threshold is 50%. Based on these costs, it is likely that PET/CT would be cost-effective from the perspective of the UK NHS.

FIGURE 16.

Cost-effectiveness acceptability curves. (a) Nuclear medicine costs in the primary model; (b) clinical oncology costs in the primary model; (c) nuclear medicine costs in the bypass and open and shut laparotomy model structure; and (d) clinical oncology costs in the bypass and open and shut laparotomy model structure.

Diagnostic accuracy and incremental benefit of positron emission tomography/computed tomography

Between 2011 and 2013, 583 patients with suspected pancreatic cancer from 18 UK sites underwent PET/CT scans, of whom 550 had PET/CT scans completed according to protocol. The primary outcome measure was diagnostic accuracy and incremental benefit of PET/CT for pancreatic cancer. Adherence to the PET/CT protocol was essential to ensure that scans were at an accepted level for quality assurance. This would mean that the results would be applicable across multiple sites. There were 33 scans that were out of range, because of either abnormal blood glucose levels or PET uptake times; therefore, analyses were undertaken on the whole population with PET/CT scans and the per-protocol population. Sensitivity and specificity for the ITT and per-protocol populations were 90.3% and 76.1% and 92.7% and 75.8%, respectively. The relative sensitivity and specificity of PET/CT compared with MDCT was significantly higher in the per-protocol group, confirming that PET/CT was more accurate than standard MDCT. For the whole population, only the relative sensitivity (not the specificity) of PET/CT compared with MDCT was significantly higher. This highlights how the PET/CT protocol can affect accuracy in terms of the specificity of PET/CT. Out-of-range scans in this study would lead to higher false-negative rates. Part of the reason for out-of-range scans was high blood glucose levels at the time of scanning. In this study 17% of patients were diabetic and this prevalence would very likely be replicated in pancreatic MDT populations across the country. Good diabetic control is therefore essential for accurate PET/CT scans in this population. Overall, these results confirm the significant superiority of PET/CT compared with MDCT in the diagnosis of pancreatic cancer.

The incremental diagnostic benefit of PET/CT in addition to MDCT was quantified using incremental likelihood ratios. These demonstrated that PET/CT significantly increased diagnostic accuracy in all scenarios. Following a positive diagnosis of pancreatic cancer on MDCT, a positive PET/CT scan increased the odds of pancreatic cancer by 55% and a negative PET/CT scan decreased the odds of pancreatic cancer by 95%. Following a negative diagnosis of pancreatic cancer on MDCT, a positive PET/CT scan increased the odds of pancreatic cancer by 538% and a negative PET/CT scan decreased the odds of pancreatic cancer by 46%.

Patients with no pancreatic cancer detected on PET/CT following a positive MDCT scan were unlikely to have pancreatic cancer (NPV of 89%). In contrast, although a positive diagnosis on PET/CT following a negative MDCT scan increased the odds of PDAC, the overall diagnosis was still highly uncertain (PPV of 50%). This latter result suggests that if both tests are performed there is benefit in combining the results rather than taking the PET/CT result alone. In particular, patients who were positive on MDCT and PET/CT were much more likely to have PDAC than patients who were positive only for PET/CT (PPV of 80% vs. 50%). The result of MDCT is less important if the PET/CT result is negative (NPV of 93% if both tests are negative vs. NPV of 89% if the PET/CT scan is negative after a positive MDCT scan). These results demonstrate that there is a substantial diagnostic benefit for patients of receiving a PET/CT scan after a potential pancreatic cancer is detected on MDCT.

We also assessed clinician certainty following PET/CT using both quantitative and qualitative measurements. With both methods there was a significant increase in clinicians’ perceived certainty of diagnosis after PET/CT (p = 0.0001). Clinicians were 12 times more likely to believe that they were more certain in their diagnosis than they were to be less certain. Assessing the accuracy of the clinicians’ perceived certainty also indicated that the clinicians’ predictions (interpreting their level of certainty as a probability) were significantly improved by the PET/CT scan result (p < 0.001). These results suggest that the use of PET/CT increases the level of certainty and the predictions of the clinicians, which may contribute to better planned management overall.

Impact of positron emission tomography/computed tomography on diagnosis, staging and management

The difference in diagnoses following MDCT and then PET/CT drives the relative sensitivity and specificity of PET/CT compared with MDCT. For pancreatic cancer diagnosis, there was a greater number of cases in which the PET/CT diagnosis was correct than in which the MDCT diagnosis was correct, indicating that PET/CT performs better. This was considered for patients with pancreatic cancer and patients with a diagnosis of malignancy. PET/CT changed the diagnosis correctly for diagnoses of both pancreatic cancer and not pancreatic cancer in 46 patients (8.3%). PET/CT incorrectly changed the diagnosis in 20 patients (3.6%). Considering the diagnosis of malignancy in general, PET/CT correctly changed the diagnosis in 48 patients (8.7%) and incorrectly changed the diagnosis in 11 patients (2%). Therefore, in terms of relative sensitivity and specificity, PET/CT has a significant impact on diagnosis compared with MDCT.

The accuracy of staging among patients with early-stage disease (lymph node-negative disease) was slightly worse under PET/CT than under MDCT, with more incorrect (eight patients) than correct (six patients) changes. However, the difference was not statistically significant (p = 0.79). Significantly more patients with stage IIB disease (lymph node-positive disease) were correctly staged by PET/CT than by MDCT, with 22 patients (21%) moving to the correct stage compared with five patients (5%) moving to the wrong stage (p = 0.002). PET/CT had no significant effect on the accuracy of staging for patients in stage III. Significantly more patients in staging group IV (metastatic disease) were changed to the correct stage (27 patients) than moved to an incorrect stage (one patient) (43% vs. 2%; p < 0.001). The effect of PET/CT was to change to the correct staging group significantly more often than to change from the correct group to an incorrect group (p < 0.001). This change in stage was limited to stage IIB and IV patients; there was no significant impact of PET/CT in staging larger tumours invading surrounding structures (stage III). Overall, PET/CT more accurately staged patients in terms of identifying lymph node disease in smaller tumours and metastatic disease.

The changes in diagnosis and staging also have implications for changes in the management of patients. Anticipated management was documented after MDCT and PET/CT and actual management was then recorded. Clinicians documented that PET/CT had influenced their management decisions for 45% of patients. Overall, 30% of patients underwent a management change throughout the study. PET/CT changed the management of 11% of patients in the overall study population.

The types of management changes resulting from the use of PET/CT were then analysed. The most common alteration in management was the change from resection to no resection, which occurred in 60 patients, representing 11% of all patients and 21% of patients scheduled for surgery after MDCT. In total, 13% of patients not thought to need surgical resection following MDCT were then planned for resection following PET/CT. Changes relating to the commencement or cessation of chemotherapy were less common. The biggest impact of PET/CT was on those patients who were identified as having resectable tumours on MDCT and who were then deemed not suitable for resection following PET/CT. This would place PET/CT as an integral component of the staging process for patients who are thought to have resectable tumours at MDCT, to identify patients who would not benefit from surgery because of more advanced disease.

Impact of positron emission tomography/computed tomography on the diagnosis of chronic pancreatitis

The prevalence of chronic pancreatitis was quite low in the patient population (41 patients; 7.5%). Therefore, the power to assess the diagnostic technologies was limited. Both MDCT and PET/CT had relatively high specificity (98.4% in each case) but low sensitivity for the diagnosis of chronic pancreatitis. The sensitivity of PET/CT was higher than that for MDCT (45.2% vs. 36.6%); however, this effect did not attain statistical significance (relative sensitivity 1.27; p = 0.066). Although a positive diagnosis of chronic pancreatitis from the PET/CT or MDCT scan result greatly increased the odds of a patient having chronic pancreatitis, there was still considerable uncertainty about the diagnosis. The main diagnostic dilemma facing clinicians is when a patient already has chronic pancreatitis and pancreatic cancer is suspected. In this population only one patient had chronic pancreatitis and pancreatic cancer. The sensitivity and specificity of PET/CT in this study do not support the routine use of PET/CT in the diagnosis and assessment of patients with chronic pancreatitis. The use of PET/CT to differentiate pancreatic cancer in patients with chronic pancreatitis cannot be supported based on the findings of this study.

Benefits of positron emission tomography/computed tomography for different subgroups

A GEE subgroup analysis was performed for all those patients who had out-of-range PET/CT scans either because of an out-of-range uptake time or elevated blood glucose. The biggest effect was on the sensitivity of PET/CT, which was reduced to 52.9% and was significantly worse than the sensitivity of MDCT (70.6%) (p = 0.005). Taking each factor individually, patients with an out-of-range uptake time had a PET/CT sensitivity of 71.4% and those with elevated blood glucose (> 10 mmol/l) had a PET/CT sensitivity of 40%. These values confirm that it is essential that PET/CT be performed after the correct uptake time and that PET/CT is unsuitable for patients with poorly controlled blood glucose.

Further subgroup analyses were undertaken for age, sex and WHO performance status. There was no evidence of any differences in the sensitivity or specificity of either test with respect to sex or age. However, both MDCT and PET/CT had a significantly higher sensitivity among patients with a WHO status of ≥ 1 than among those with a WHO status of 0, but conversely had significantly lower specificity. In terms of the eligibility criteria (pancreatic mass, jaundice and serum CA19.9 > 37 kU/l), both MDCT and PET/CT had a significantly lower specificity among patients with jaundice because of distal obstruction of the common bile duct or ampulla. Moreover, there was also evidence that PET/CT was not more effective than MDCT in this group. There was no evidence of differences in sensitivity or specificity for the other eligibility criteria. It can be noted that, based on pancreatic mass on CT, PET/CT sensitivity and specificity were 93% and 76%, respectively, for diagnosing pancreatic cancer; based on jaundice, PET/CT sensitivity and specificity were 95.6% and 52.6%, respectively; and based on elevated CA19.9, PET/CT sensitivity and specificity were 92.2% and 72.5%, respectively. There were no major differences between these criteria (apart from the specificity in jaundiced patients).

Impact of positron emission tomography/computed tomography on the diagnosis of pancreatic tumours

The value of PET/CT in other pancreatic tumours was evaluated. The most frequent tumour types were assessed.

Additional analyses

Health economic evaluation

Health economic analysis was conducted from a NHS perspective. Our cost-effectiveness analysis was in three parts: model 1, in which we calculated the marginal cost per additional correct diagnosis of PDAC using PET/CT and MDCT alone; model 2, in which we calculated the budget impact of the use of PET/CT; and model 3, in which we modelled the change in management of patients over a 1-year time horizon as a result of the use of PET/CT for diagnosis. We undertook sensitivity analysis to explore uncertainty in costs (univariate) and model structure (structural). PSA assessed the likelihood that PET/CT is cost-effective at thresholds of £20,000 per QALY and £30,000 per QALY. We found evidence for the cost-effectiveness of PET/CT in terms of change in management for patients with suspected pancreatic cancer. In the change of management model, the primary model dominated management based on MDCT alone. This is largely because of the number of unnecessary surgeries avoided. Structural sensitivity analysis involved varying our base-case assumption that all patients received a resection so that some patients who were clinically indicated received bypass or open and shut laparotomy. Alongside our main finding from the base-case analysis that PET/CT dominated, the ICER in our structural sensitivity analysis ranged from £19,445 per QALY to £53,557 per QALY depending on the source of PET/CT costs chosen. For both change of management models, PET/CT was seen to be most cost-effective for the PDAC with resection subgroup. Overall, our base-case analysis showed that PET/CT dominated MDCT alone, in particular for patients who were suspected of having pancreatic cancer after standard diagnostic work-up and who were planned for surgery. However, we note that QALY gains were small and our analysis was sensitive to our source of published costs and to structural assumptions in the model.