TOMMY trial: A comparison of TOMosynthesis with digital MammographY in the UK NHS Breast Screening Programme

Article history

The research reported in this issue of the journal was funded by the HTA programme as project number 09/22/182. The contractual start date was in December 2010. The draft report began editorial review in February 2014 and was accepted for publication in August 2014. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The HTA editors and publisher have tried to ensure the accuracy of the authors’ report and would like to thank the reviewers for their constructive comments on the draft document. However, they do not accept liability for damages or losses arising from material published in this report.

Declared competing interests of authors

Dr Michell reports personal fees, non-financial support and grants from Hologic, the supplier of the mammographic and tomographic equipment, outside the submitted work. Dr Astley reports grants from NIHR during the conduct of the study; non-financial support from Matakina; and workshop training sessions and non-financial support from Hologic outside the submitted work. No others declared.

Copyright statement

© Queen’s Printer and Controller of HMSO 2015. This work was produced by Gilbert et al. under the terms of a commissioning contract issued by the Secretary of State for Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science Park, Southampton SO16 7NS, UK.

Background

Breast screening with mammography is recognised as the most effective method of detecting early-stage breast cancer and reducing breast cancer mortality. A recent meta-analysis of 11 randomised trials concluded that there was a 20% relative risk reduction in breast cancer mortality in women invited to screening. 1 However, one of the primary limitations of standard two-dimensional (2D) mammography is that overlapping dense fibroglandular tissue within the breast can decrease the visibility of malignant abnormalities or simulate the appearance of an abnormality. This reduces the sensitivity of screening and increases the number of false-positive recalls. 2 It has been shown that 15–30% of cancers are not detected by standard screening,3 and this percentage is higher in women aged under 50 years4 and in women with dense breasts. 5–7 Women with dense breasts have reduced screening programme sensitivity8–11 and tend to have larger screen-detected and interval cancers. 8,12,13 These issues are of concern for the UK NHS Breast Screening Programme (NHSBSP) as it extends the screening age range to include younger, pre- or perimenopausal women who are known to have a higher proportion of dense breast tissue14,15 and is also potentially problematic for women at moderate or high risk of developing familial breast cancer who attend annual mammography. 14

Digital breast tomosynthesis

Digital breast tomosynthesis (DBT) is a newly developed three-dimensional (3D) imaging technique that has the potential to improve the accuracy of mammography by reducing overlapping shadows from breast tissue that degrade the image quality in standard 2D projection imaging. This should improve the visibility of cancers and facilitate the differentiation between malignant and non-malignant features. The expectation is that small cancers, which may be obscured by normal fibroglandular tissue in standard 2D projection imaging, could be more readily detected using DBT, particularly in women with radiologically dense breasts. In addition, by facilitating the analysis of superimposed breast structures, DBT may enable the reader to identify features such as asymmetrical density (ASD) on 2D imaging as normal composite shadows and thereby decrease the number of false-positive recalls16–19 and the associated health-care costs20 and patient anxiety. 21,22

Development and process

The fundamentals of tomographic imaging were established in the 1930s, but clinical applications of tomosynthesis in mammography did not evolve until several decades later, following the development of flat-panel digital display detectors, rapid computer processing and advances in reconstruction and post-processing algorithms. 23 In DBT, a sequence of projection images is obtained by moving the position of the X-ray tube and making exposures at regular intervals/angles. The angular range used varies from one manufacturer to another. In the Hologic Selenia Dimensions System (Hologic Inc., Bedford, MA, USA) used in this study, 11 exposures are taken over an angular range of 15 degrees. The exposure used for each projection image is relatively small so that the overall mean glandular dose (MGD) for DBT is comparable with that of conventional 2D imaging. The projection images acquired by the detector are processed by reconstruction algorithms to produce a pseudo-3D tomographic image of the breast in which each reconstructed image (or slice) shows the tissues sharply for that plane and blurs out details in higher and lower planes. Typically, this reconstruction is done to produce images with a 1-mm slice thickness although increased slice thickness can be used. Image quality of DBT is highly dependent on system geometry and the choice of optimal image acquisition, reconstruction and display parameters. 24–27 Some manufacturers employ a larger angular range, which would theoretically improve the depth resolution between planes, but at the expense of in-plane resolution. 25,26 A viewing workstation provides tools that enable the reader to scroll vertically through the tomographic images as well as to compare them with the corresponding 2D images.

Diagnostic accuracy

The superiority of DBT over standard 2D projection imaging in terms of lesion visibility and margin detection was first demonstrated in experimental studies using phantoms and mastectomy specimens. 28–30 Improved lesion visibility, size and classification compared with standard film or digital mammography have been reported31–38 and the possibility that DBT could reduce the need for additional mammographic views at least for non-calcified lesions39–42 has been suggested. There are mixed reports of the sensitivity of DBT for the detection of microcalcifications32,43–51 that may be partly as a result of the different techniques used for image reconstruction and the need to combine image slices into thicker slabs for optimal visualisation of microcalcification clusters. These observations suggested that DBT was unlikely to be used as a stand-alone imaging modality if a 2D mammogram was required for optimal microcalcification assessment. 25,52,53

Cancer detection and recall rate

Higher sensitivity of DBT, either alone or in combination with 2D mammography, has been reported16,19,45,48,54,55 although no change56 or reductions in sensitivity of between 4% and 9% were also noted in some comparison studies of one-view and two-view DBT versus 2D or DBT added to 2D. 33,44,57 Several studies have shown increased specificity of DBT compared with 2D16,33,45 and in combination with 2D mammography. 16,20,43,46,58 In a multireader multicentre trial using receiver operating characteristic (ROC) analysis, Rafferty et al. 17 reported increased diagnostic accuracy of 2D + DBT compared with 2D alone, particularly in the detection of invasive cancers, and a reduction in the false-positive recall rate and a large retrospective evaluation of screening mammography in 13,158 women59 reported a significantly lower recall rate with 2D + DBT compared with 2D alone, especially for women aged < 50 years and those with dense breasts. However, the latter study was underpowered to demonstrate any significant increase in cancer detection rate. Rose et al. 60 conducted an observational study to assess changes in screening performance measures after the introduction of DBT. Six radiologists interpreted 13,586 screening mammography studies without DBT and 9499 studies with the addition of DBT. Routine use of 2D + DBT resulted in a significant reduction in recall rate and, although changes in other performance measures, for example cancer detection rate (in particular, earlier detection of invasive cancers), biopsy rate and positive predictive value, did not reach statistical significance, this was attributed to study design limitations.

In general, published studies have demonstrated the potential for DBT to decrease recall rates and possibly increase cancer detection rates. 61,62 Some of the conflicting results and uncertainties from early studies may be attributed to differences in study methodology, case composition and the use of prototype DBT systems from different manufacturers with different configurations. 63–65 The Houssami and Skaane review64 did conclude that the available data did support investment of new large-scale randomised population screening trials.

Several population-based screening trials are now in progress, although none is randomised:

Reader performance

The need for substantial reader training with DBT has been acknowledged. 16,67,68 Multireader studies have demonstrated improved performance, using DBT in combination with 2D mammography compared with 2D mammography alone, by radiologists with a range of experience, as measured by reduction in recall rate and ROC analysis. 17,58,69 However, Wallis et al. ,46 using a photon-counting DBT system, reported that the addition of DBT improved the performance only of inexperienced readers. Reading and reporting times are considerably longer (almost twofold) using DBT,16,46,68,70 with a reading time of 91 seconds for 2D + DBT compared with 45 seconds for 2D alone in a screen-reading setting19 because of the number of images to be reviewed. This has resource implications if DBT were to be introduced into screening practice, although the impact on workflow could be mitigated if recall rates were shown to be decreased and unnecessary diagnostic assessments avoided. 71

A number of publications have summarised and reviewed both the technical and the clinical aspects of DBT and have speculated on its potential application in breast imaging. 23,25–27,61,63,64,72,73 The AETNA policy report,74 a combined recommendation from the American College of Radiologists, the American Cancer Society and the American College of Obstetrics and Gynaecologists, considers that breast tomosynthesis imaging is experimental and investigational because of insufficient evidence of its effectiveness. An Australian health technology review acknowledged that DBT systems were likely to replace 2D in screening when existing 2D equipment is due for replacement75 and an overview of the evidence and issues to be considered in relation DBT and breast cancer screening raised concerns regarding additional radiation exposure and the learning curve for radiologists to accurately interpret DBT results. 65

The use of DBT in combination with 2D requires an approximate doubling of radiation exposure. Hologic have developed image-processing software [C-View™ (Hologic Inc., MA, USA, 2011)] to simulate a conventional 2D mammogram by generating a synthetic 2D image from each set of tomosynthesis slices,70,76 thus eliminating the need for the acquisition of additional 2D exposures. One published study70 has reported lower sensitivity using synthetic 2D + DBT than using 2D + DBT but the authors acknowledged that greater diagnostic accuracy may be achieved with newer versions of the software. Synthetic 2D is currently being evaluated within the Oslo trial. 77,78

Rationale for study

The need for robust studies using clinically relevant methodology to further inform the optimal role, if any, for DBT in the diagnosis and assessment of breast cancer has been highlighted. 63–65,79 Of primary importance is further research to address whether DBT should be used as an adjunct to standard 2D or as a stand-alone technology.

To assist in the choice of DBT system to be used in the study, a working party was set up at the start of the project in 2010 by some of the grant applicants to review the status of DBT system technology. When the working party report was published,80 seven companies were involved in the development of commercial DBT systems, and the group concluded that a multivendor evaluation of DBT systems in the study would be premature at this stage, as some of vendors’ DBT technology was still being developed. The Hologic DBT system was relatively well established, was technically stable and was the only commercially available system with US Food and Drug Administration (FDA) approval at the start of the trial. The Hologic DBT system allows both 2D and DBT images to be acquired in a single examination with the same breast position and compression (combination mode), enabling direct comparison of the performance of the two imaging modalities. This reduces any bias that could occur as a result of different positioning of the breast when taking the second image. The rapid acquisition time (< 10 seconds) minimises patient discomfort and patient movement, which could result in blurring of the acquired images, and a fast image-processing time (3–4 seconds) minimises the impact on clinic workflow. In addition, the system coregisters the 2D and DBT images, thus facilitating image interpretation and lesion localisation.

Research objectives

The purpose of the study was to test the diagnostic accuracy of DBT as an adjunct to standard 2D mammography as a primary screening tool.

Study design

This study is a multicentre retrospective reading study comparing the diagnostic performance of 2D + DBT versus 2D and synthetic 2D + DBT versus 2D. The overall study design is shown in Figures 1 and 2.

FIGURE 1.

Case collection.

FIGURE 2.

Reading study.

Study settings and population

Participants were recruited from six NHSBSP centres in the UK and from the symptomatic breast service in Aberdeen. They comprised women aged 47–73 years recalled to an assessment clinic for a mammographic abnormality detected at routine breast screening and also women below 50 years of age with a FH of breast cancer who attended annual mammography screening. 81 The purpose of targeting this group of women was to compile a data set with a relatively high proportion of cancer cases (estimated to be approximately 18%). It was expected there would be around 50% of cases with overlapping tissues on standard 2D mammography that simulated suspicious features but were actually normal breast tissue.

Recruitment

A participant information leaflet (see Appendices 1 and 2) outlining the potential benefits and risks of the study was sent to women who would be suitable for inclusion in the study, and written informed consent (see Appendix 3) was obtained on attendance at the assessment clinic or screening mammography appointment.

Intervention

All participants underwent standard two-view [mediolateral oblique (MLO) and craniocaudal (CC)] DBT imaging and two-view (MLO and CC) 2D mammography of both breasts. For participants recruited in assessment clinics, imaging was conducted prior to any additional investigations deemed necessary in the assessment clinic. Following DBT and 2D imaging, women resumed the normal pathway through the assessment or screening clinic. The DBT images were available for management of the women during the clinic or shortly afterwards. Any subsequent management followed standard assessment clinic or screening centre procedures.

Image acquisition

Both the DBT and the standard 2D imaging were performed as a single procedure at the same breast compression on a Hologic Selenia Dimensions Digital Mammography Unit (Hologic Inc., Bedford, MA, USA). This important feature of the system eliminated any bias that could have occurred as a result of different positioning of the breast when taking the second image. Differences in compression could significantly influence the detectability of cancers as a result of the resulting differences in the superimposition of tissues. The rapid acquisition sequence also minimises patient discomfort and patient movement, which could result in blurring of the acquired image. Radiographers were experienced specialist mammography radiographers, fully trained in accordance with NHSBSP standards, who had received additional specific training on the DBT equipment used in the study.

Radiation dose

The radiation dose for DBT was additional to normal procedures and some of the dose from the 2D imaging may have been additional if local protocol was to take fewer initial images at assessment (some of this dose may have been offset by not having to acquire supplementary, coned or magnification images).

The additional lifetime risk of inducing a breast cancer as a result of a single two-view mammography examination is estimated to be approximately 1 in 20,000 between the ages of 50 and 70 years. 82,83 For this trial, the total MGD was estimated at 7 mGy (Table 1), giving rise to an estimated 1 in 10,000 risk of cancer induction (assuming an induction rate of 14 per million per mGy). In practice, some of this dose would have been received during normal assessment procedures (estimated at 1.5 to 3.0 mGy depending on local practice); therefore the additional dose ranged from 4.0 to 5.5 mGy. The total dose for the trial falls just within the diagnostic reference level for standard two-view 2D mammography.

Some of the trial participants were women aged 40–49 years with FH of breast cancer who were attending annual surveillance mammography. 81 The radiation risk implications of cancer screening in this cohort was reviewed, with benefits expected to substantially exceed risks down to at least the age of 40. 82 In the trial, the total dose including DBT was approximately 7 mGy. Overall, the additional radiation dose involved was very low and within the range currently accepted for routine screening.

Reader experience

Readers from each participating centre were a mixture of radiologists, advanced practitioner radiographers and breast clinicians, representative of current reading practice in the NHSBSP. All readers had a proven track record of film reading in the NHSBSP, including:

• mammographic film reading for a minimum of 2 years

• reading a minimum of 5000 mammograms per annum

• annual participation in PERsonal PerFORmance in Mammographic Screening (PERFORMS) self-assessment test

• attendance at assessment clinics and multidisciplinary team meetings.

Readers who are indicated in Table 2 with a superscript ‘a’ contributed only to prospective reading and did not participate in the retrospective reading study.

The number of readers available varied from week to week owing to other commitments, annual leave, etc.; the average number per week was 16.

Prior to the start of the trial, reader training consisted of 2 days of applications training from the DBT system manufacturer (Hologic Inc.) and attendance at a 1-day DBT reading course presented by staff from Breast Radiology at King’s College Hospital, London. Readers were also asked to read a test set of 80 cases. These same cases were read again at the end of the trial in order to evaluate improvement in reader performance over time. An account of this evaluation is given later in this report (see Chapter 3). Over the first 12 months of the study recruitment period readers also gained experience of DBT by reviewing 2D and DBT images acquired at their own site. All readers used Hologic SecurView DW workstations (Hologic Inc., Bedford, MA), optimised to read both 2D and DBT images.

Image management

Images acquired on the Hologic Selenia Dimensions system were exported to a number of output devices:

• SecurView DX workstation (Hologic Inc, Bedford, MA, USA): Processed 2D and DBT image files were exported to the workstation for review by readers.

• Picture archiving and communications system (PACS): Processed image files that formed part of a patient’s screening episode were exported to PACS. Individual centres determined whether or not to store the processed DBT image files on their NHS PACS system. Archiving was initiated following processing of all images collected for the study.

• Hologic SecurXchange (Hologic Inc, Bedford, MA, USA)/Hologic Image Collection System (HICS) (Hologic Inc, Bedford, MA, USA): An output set containing both the raw and the processed DBT and 2D image files was exported to the SecurXchange archiver unit and HICS for anonymising and storage prior to image transfer to the trial office.

Retrospective reading study

Retrospective data collection

Statistics and analyses

Introduction

Digital breast tomosynthesis is a relatively new imaging modality, and few readers have extensive experience of it in a screening setting. This study aimed to establish whether or not increased experience of reading DBT images, as measured at the start and end of the TOMMY trial (a comparison of TOMosynthesis with digital MammographY in the UK NHS Breast Screening Programme), altered performance in terms of recall rate and cancer detection. This knowledge is important when designing training regimes for new readers.

Aims

The specific aims of the reader study were:

1. to investigate variability in performance of readers involved in the TOMMY trial

2. to determine if there was a change in individual performance over the course of the trial

3. to determine if performance is related to prior experience with DBT or digital mammography

4. to determine if change in performance over the course of the trial is related to the number of DBT images read during the trial

5. to determine whether or not change in performance during the course of the trial is related to previous experience.

Method

In total, 80 DBT cases were identified from those obtained prior to the TOMMY trial at King’s College Hospital in London. Of these, 39 cases contained either no abnormality or benign abnormalities, while 41 cases contained a cancer. Ground truth was provided, with cases classified as normal, benign, malignant in situ or malignant invasive (Table 6).

Twenty-eight readers from TOMMY trial sites assessed the set of 80 DBT cases, completing a proforma for every lesion identified. Of these, 22 of the readers assessed the image set on two occasions, before the start of the trial and at the end of it. Results are presented for those readers who completed both reads (Table 7). The order in which the cases were assessed was randomised for each read.

For these results we have focused on the suspicion score allocated to each case (Table 8).

Results

The results are shown in Table 7. A recall corresponded to a suspicion score of 3 or more. The proportion of cancer cases correctly recalled ranged from 0.80 to 1.00 at the first time point and from 0.83 to 1.00 at the second time point. The proportion of normal/benign cases not recalled ranged from 0.08 to 0.67 at the first time point and from 0.16 to 0.63 at the second time point. However, the number of cases that were actually normal was very low (7 out of 80 cases) so we would expect a significant recall rate for normal/benign cases. There was no significant difference between the recall rates before and after the trial (p = 0.501 cancer, p = 0.198 normal/benign).

Table 9 shows the results with a higher threshold on the suspicion score. The proportion of cancer cases correctly identified as suspicious or malignant ranged from 0.61 to 0.88 at the first time point and from 0.61 to 0.95 at second time point. The proportion of normal/benign cases scored normal, benign or probably benign ranged from 0.62 to 1.00 at the first time point and from 0.62 to 0.97 at the second time point. Once again there was no significant difference between time points (p = 0.470 and p = 0.339, respectively). Data from Tables 7 and 9 are plotted in Figures 3 and 4, respectively.

FIGURE 3.

Proportion of cancer cases recalled, plotted against the proportion of normal/benign cases not recalled at time point 1 (♦) and time point 2 (■).

FIGURE 4.

Proportion of cancer cases identified as suspicious or malignant, and the proportion of normal/benign cases scored normal, benign or probably benign at time point 1 (♦) and time point 2 (■).

Tables 10–12 show the results ranked by reader experience. Table 10 shows the change in performance between reads ranked according to the volume of DBT undertaken within the trial between the two time points. Table 11 shows the change in performance ranked according to full-field digital mammography (FFDM) experience (number of mammograms read) in the 12 months prior to the trial and Table 12 shows the baseline performance ranked by FFDM experience based on the number of mammograms read during the year. Table 13 shows the results ranked by number of years reading mammograms and Tables 14 and 15 show the analysis by site. Figure 4 shows performance at the initial read by FFDM load in the previous 12 months and Figure 5 by number of years reading mammograms prior to the study. Figures 6, 7 and 8 illustrate performance by site at the two time points.

FIGURE 5.

Performance at first read vs. FFDM load in previous 12 months.

FIGURE 6.

Performance at the first read ranked by number of years of experience reading mammograms.

FIGURE 7.

Recall rates by site at first (T1) and second (T2) reads. Site 1, Aberdeen; site 2, Glasgow; site 3, Jarvis; and site 4, Manchester. T1, time point 1; T2, time point 2.

FIGURE 8.

Cancer cases identified as suspicious/malignant by site. Site 1, Aberdeen; site 2, Glasgow; site 3, Jarvis; and site 4, Manchester. T1, time point 1; T2, time point 2.

Discussion

As the data set was rich in difficult cases and benign abnormalities as well as cancers, it was appropriate to look at the proportion of cancers scored 4 or 5 as well as the proportion of cancer cases recalled (scored 3 or more). In particular, one would expect many of the non-cancer cases to be scored as 3 (probably benign), since they contained benign abnormalities.

Even though there was little change with time (Tables 8 and 9; Figures 2 and 3), there are differences between readers, particularly when looking at cancer cases scored 4 and 5 (suspicious or malignant). In Figure 3, it is apparent that readers choose an operating point that balances the cancer detection rate against the proportion of normal cases scored as normal to probably benign.

Previous experience did not predict performance, in terms of either recall rates or detection of cancer. The greatest variability between readers was seen in the proportion of normal cases not recalled.

When assessing whether or not DBT should be introduced into the screening programme, it is important to consider extra reader time and the impact of reader fatigue. Timing reading with DBT in 2012, when readers were relatively inexperienced, the median time for interpretation of 2D images was 17.0 seconds with an interquartile range of 12.3–23.6 seconds, while for DBT it was 66.0 seconds, with an interquartile range of 51.1–80.5 seconds. The difference was statistically significant (p < 0.001). Reading times were significantly longer in FH clinics (p < 0.01). Although it took approximately four times longer to interpret DBT than 2D images, the cases were more complex than would be expected for routine screening and had higher mammographic density. 92

There is some difference when comparing reads at time point 1 and time point 2 for one of the sites (Jarvis). In this instance the proportion of normal/benign lesions that were not recalled increased by 20% over the course of the trial. In Manchester there was a 9% decrease in the same parameter. The proportion of cancers identified as suspicious or malignant (4 or 5) was similar to that of cases identified as normal/benign (1, 2 or 3) at most sites, except for Aberdeen, where the difference was 19% at the first read and 22% at the second read.

The lack of change over time indicates that a larger set of training images is probably unnecessary, although the intersite variations indicate that individuals operate according to local practice even under test conditions.

Introduction

Although the risk of developing breast cancer is dependent on the cumulative impact of a wide range of risk factors,93 increasing mammographic density has been shown to be one of the strongest independent and modifiable predictors of breast cancer risk. 4,94–100

Women with high breast density have been reported to have a four to six times increased risk of developing breast cancer compared with those with low breast density,95,101–105 and high breast density has also been linked to an increased risk of cancers not detected at screening,4,8,96,106,107 larger tumour size108–110 and positive lymph nodes. 109,111–113 The underlying cause of these links are thought to be numerous, and early studies hypothesised that a significant reason for an increase in breast cancer incidence with higher density breasts was as a result of a ‘masking bias’ that made mammographic screening less sensitive to cancer detection. 96,101 Later studies, however, have shown that there is increased risk for at least 7–10 years following a screening examination, indicating that ‘masking bias’ is only one of the mechanisms linking breast density to an increased cancer risk. 94,101,114 In addition, the increased radiation dose required in dense breasts and cumulative lifetime exposure from screening indicates that there may be a less favourable benefit to harm ratio associated with screening of women with dense breasts, particularly in younger women. 93

These issues are of particular concern for the NHSBSP, as it is now extending its programme14 to include younger, pre- or perimenopausal women, who are known to have a higher proportion of dense breast tissue,5,115 and it is also potentially problematic for women aged 40–49 years at moderate or high risk of developing familial breast cancer attending for annual mammography. 81

It has been suggested that population mammographic screening might be more effective if screening strategies were tailored according to mammographic breast density79 with more frequent screening or use of magnetic resonance imaging (MRI) or adjunctive ultrasound to improve detection in women with dense breasts. 99,116–119 In this context, a campaign to include breast density in mammography reporting is currently being debated in the USA. 120 Legislation has been passed in several states mandating that breast density be reported using the Breast Imaging Reporting and Data System (BI-RADS) scale, with women with > 50% breast density offered supplemental screening. 121

Measurement of breast density

The radiographic appearance of the breast on a mammogram reflects variations in the relative amounts of fat, connective tissue and epithelial tissue and their different X-ray attenuation characteristics, with breast density expressed as a percentage of the mammogram occupied by radiologically dense fibroglandular tissue. 101 Both qualitative and quantitative methods currently used to assess breast density by mammography have limits, since they are based on the projected area rather than the volume of breast tissue, are time-consuming, and are subject to inter- and intrareader variability. 97,122 For breast density assessment to be incorporated into a population-based screening programme such as the NHSBSP, efficient automated methods validated against screening programme outcomes such as sensitivity, interval cancer rates and tumour size at diagnosis are required.

Traditionally, assessment of mammographic density is performed by the radiologist evaluating the mammogram. He/she makes a judgement on the information presented in all the mammographic views in order to present a single score for each examination. Consistency of this measure requires an experienced observer to be able to correctly assess the relative proportions of glandular and fatty tissue while accounting for variations in breast shape, radiographic texture and the presence of cancer (leading to a localised increase in density). The radiologist is also able to take account of the variation in the radiographic acquisition of the mammogram. These density scores can then be presented as a percentage on a continuous scale or within discrete ranges such as the composition categories used in BI-RADS. 94

Although moderate agreement has been shown between observers in assigning percentage scores of breast density, studies suggest that training and experience are essential in ensuring that those scores are accurate and reproducible. 123,124 In addition, each manufacturer applies its own image processing, which is often designed to minimise the effect of density in the mammogram, further increasing the difficulty of density assessment (Dr Ralph Highnam, Matakina International Limited, 2013, personal communication). Methods have therefore been investigated to standardise density estimates by use of automated methods. The introduction of FFDM technologies, initially developed for digitised analogue mammograms, has provided an opportunity to implement breast density measurement algorithms. 123–126

At their simplest, these algorithms work by applying thresholds to the pixel values within the digital image to identify the area of the image that contains the breast and then to determine the proportion of that breast that is dense. For example, the pixel values with the highest signal can be seen to be the areas of the image where no breast tissue has attenuated the primary X-ray beam. The areas of lowest signal, on the other hand, represent areas where the X-rays have passed through a section of tissue that is relatively most attenuating. 127

Later developments have led to software that estimates the volume of dense fibroglandular tissue rather than just the area projected onto the mammogram. By using the image pixel data in combination with information about its acquisition found the DICOM data file, sophisticated algorithms are able to provide measurements of the relevant tissue volumes. For example, data regarding the breast thickness and the X-ray exposure’s tube potential, current, time, target and filter – in combination with knowledge of the radiation attenuation properties of different tissues – can enable a derivation of the breast composition represented by each pixel. 97,128 Improvements in the measurement are then made through advances in this derivation, for example providing better calibration of the image data and breast thickness estimation. 129

Two software tools were used in this study. Hologic’s Quantra has been validated and found to have good agreement with measurements of breast density from MRI data and reader assessments. 121,128 Matakina’s Volpara™ software has similarly found to be in good agreement with MRI and observer measurements of breast density. 121,130

The aim of this substudy within the trial was to evaluate and utilise two of the commercially available software packages for the measurement of volumetric breast density and compare these with observer-based scores of area density. This data would then be correlated with the results of the retrospective reading study designed to determine if the addition of DBT improved the detection of cancers in women at higher risk of developing breast cancer due to increased breast density. A secondary aim of this study was to assess whether or not there is a relationship between breast density and breast cancer incidence.

Materials and methods

In the prospective data collection phase of the trial, readers were asked to assess breast density on a 10-cm VAS. 129 The markings on this scale were then converted into a score ranging from 0 to 100%.

The two software packages used to assess the volumetric density of each mammogram were Quantra and Volpara. Each program’s output consisted of a number of results, including values for the absolute volume of fibroglandular tissue and overall breast volume as well as the volumetric breast density on a per image basis. Scores from each image of a full examination were then combined to derive a score for each case. Each software vendor provides an ability to combine these automatically. However, we chose to adapt the underlying logic provided by the Quantra system (Dr Ashwini Kshirsagar, Hologic Inc., 2013, personal communication) to minimise the effect of lesion presence on the result. Their generally applied logic takes the maximum total breast and fibroglandular tissue volumes calculated from the CC and MLO images. The values for left and right breasts are then added and the overall density determined. We believe this to be in line with the behaviour of observers and with findings from prevalence studies that risk may be associated with the density derived on the contralateral mammogram in the absence of prior imaging. 101

In order to obtain the overall score for each examination, the absolute values of total breast volume and fibroglandular tissue for each of the four views (left and right, CC and MLO) were examined. For cases where no cancer was assessed as being present, the largest results for each breast (from either the CC or the MLO view) were determined and the average volumes of the two breasts were calculated. For cases where cancer was confirmed, results were used from the contralateral breast. If no contralateral data were available, results from the affected breast were used. Volumetric density was calculated from the ratio of the fibroglandular tissue volume to the overall breast volume. The same logic was applied to the scores given by the Quantra system for area breast density, which should nominally be comparable with the observer scores.

With these density scores the aim of this study was to compare the observer’s score with the automated techniques (Quantra and Volpara), to establish any age-related correlation with density and to establish if cancer incidence is associated with breast density. Pathology reports were used to confirm cancer cases.

Results

A total of 8867 women’s standard 2D mammograms acquired for the trial were available for analysis. The software was unable to produce scores for every image analysed. Reasons for algorithm failure were varied but included magnification views where there was no background detected; cases in which information given in the DICOM image header was inconsistent with that normally expected in mammography, for example non-female gender selection; and invalid filter types or improbable thicknesses (e.g. 0 cm or > 30 cm).

The summary results for the study cohort are shown in Table 16. In total, 8391 cases were given an overall density score by the observers on the VAS, 8512 cases were calculated from scores from Quantra analysis of 33,966 images and 8532 cases were calculated from scores from Volpara analysis of 34,755 images.

Total breast volume

Figures 9 and 10 illustrate the distribution and comparison of total breast volume as measured by the Quantra and Volpara software throughout the study population. The two systems show a very good linear correlation with a coefficient of determination of 0.95. The mean difference between the values calculated for each case is 5.04% (± 0.32%, two standard errors), suggesting good agreement between the two systems.

FIGURE 9.

Histograms showing the distribution of breast volume across the study population as measured by (a) Quantra and (b) Volpara.

FIGURE 10.

Scatterplot comparing Quantra and Volpara scores for total breast volume.

Fibroglandular tissue volume

Figures 11 and 12 show the results for the fibroglandular tissue volume within the breast. The coefficient of determination for a linear correlation is 0.74 and the mean difference between the values calculated for each case is 21.19% (± 0.72%), with Quantra giving the larger of the two values. Our understanding is that this is because of differences in the way that each system handles the density associated with the skin within the assessment of fibroglandular volume. The differences between the two systems appear to get larger as the amount of fibroglandular tissue increases. This is particularly noticeable above 400 cm³. In addition, the range of results for fibroglandular volume is greater for the Quantra system, with most values lying between 0 and 400 cm³ as opposed to Volpara’s results lying between 0 and 300 cm³.

FIGURE 11.

Histograms showing the distribution of fibroglandular tissue volume across the study population as measured by (a) Quantra and (b) Volpara. The x-axes of each graph are designed to match for clarity but seven cases in the Quantra data set had volumes greater than 800 cm³.

FIGURE 12.

Scatterplot comparing Quantra and Volpara scores for fibroglandular tissue volume.

Volumetric breast density

Figures 13 and 14 show the results for the volumetric breast density. The coefficient of determination for a linear correlation is 0.65 and the mean difference between the values calculated for each case is 16.32% (± 0.69%).

FIGURE 13.

Histograms showing the distribution of volumetric breast density across the study population as measured by (a) Quantra and (b) Volpara.

FIGURE 14.

Scatterplot comparing Quantra and Volpara scores for volumetric breast density.

Figures 15 and 16 show the results for breast density estimated from the projected mammogram, the area-based breast density scored by the observers, and the Quantra software. The Volpara software did not give an area-based density result. There is poor correlation between the two measurements, with a coefficient of determination for a linear correlation of 0.31.

FIGURE 15.

Histograms showing the distribution of area-based breast density across the study population from (a) the observers and (b) the Quantra software.

FIGURE 16.

Scatterplot comparing the observer and Quantra scores for the area-based breast density.

Figure 17 compares the area-based breast density scored by the observers, and the volumetric density measurement from each program. The coefficient of determination for an exponential correlation is 0.33 and 0.38 for the Quantra and Volpara systems, respectively. The large values given at each 5% mark on the observers’ histograms in Figures 14 and 16 are a consequence of the way that the VAS scale was processed in two centres, with one rounding results to within the nearest 5% and the other the nearest 10%.

FIGURE 17.

Scatterplots comparing the observer area-based breast density scores with the volumetric measurements from (a) Quantra and (b) Volpara.

For FH cases, density was assessed by two observers from the same centre, giving us 638 cases with two scores for comparison. In 70% of these cases the score agreed to within 10%; however, 8% of cases disagreed by more than 20%.

Figure 18 shows the variation with decade of age in the four breast densities measures examined in this work.

FIGURE 18.

Graphs showing the decrease in breast density with age for the study group for the four available density measures. Error bars are two standard errors of the mean. (a) Area-based density in the observers; (b) area-based density in the Quantra; (c) volume-based density in Quantra; and (d) volume-based density in Volpara.

Breast density and cancer risk

Volpara breast composition data were available for 7019 of the 7060 cases (1157 of the 1160 cancers and 5862 of the 5900 non-cancers). Table 17 shows mean and standard deviation of breast composition measures using Volpara, by age, cancer status and cohort (assessment or FH screened). The absolute dense volume was generally higher in cancers than in non-cancers and declined with age in all groups. The percentage density showed the same tendencies, although less markedly. Adjusting for age, there was a significant effect of Volpara absolute dense volume on risk of cancer, with a 3% increase in odds of cancer per additional 10 cm³ of dense tissue [odds ratio (OR) = 1.03, 95% CI 1.01 to 1.05, p < 0.001]. The effect of dense volume on risk did not vary significantly by age (p = 0.2). Figure 19 shows the ORs and 95% CIs by quartile of absolute dense volume. There was no significant effect of percentage density on risk (p = 0.7).

TABLE 17 - Breast composition measures using Volpara

SD, standard deviation.

Age (years)	Breast composition measure	Mean (SD) for population
		Cancers	Assessment non-cancers	FH non-cancers	All non-cancers
< 50	Breast volume (cm3)	1063 (663)	1027 (673)	1041 (668)	1037 (669)
	Dense volume (cm3)	111 (53)	101 (67)	101 (61)	101 (63)
	% density	13 (7)	12 (6)	12 (7)	12 (7)
	No. of subjects	29	313	942	1255
50–59	Breast volume (cm3)	1153 (670)	1034 (614)	983 (597)	1033 (613)
	Dense volume (cm3)	94 (54)	84 (50)	84 (51)	84 (50)
	% density	10 (6)	9 (5)	10 (5)	10 (5)
	No. of subjects	460	3092	43	3135
≥ 60	Breast volume (cm3)	1092 (562)	1010 (557)	582 (75)	1009 (556)
	Dense volume (cm3)	77 (44)	73 (43)	53 (20)	73 (43)
	% density	8 (4)	8 (4)	9 (2)	8 (4)
	No. of subjects	660	1402	3	1405
All ages (including age missing)	Breast volume (cm3)	1116 (613)	1028 (601)	1040 (662)	1030 (613)
	Dense volume (cm3)	85 (49)	82 (49)	100 (61)	85 (52)
	% density	9 (5)	9 (5)	12 (7)	10 (6)
	No. of subjects	1157	4830	1032	5862

FIGURE 19.

Odds ratios by quartile of Volpara absolute dense volume. Quartile 1, < 51 cm³; quartile 2, 51–71.99 cm³; quartile 3, 72–102.99 cm³; and quartile 4, > 103 cm³. Vertical lines indicate 95% CIs.

Corresponding information for Quantra breast composition data was available for 7005 cases (1156 cancers and 5849 non-cancers). Table 18 shows the means and standard deviations of the Quantra measurements by age, cancer status and cohort. Again, dense volume was generally higher in cancers and declined with age. Adjusting for age, there was a significant effect of Quantra absolute dense volume on cancer risk, with a 2% increase in risk per additional 10 cm³ of dense tissue (OR = 1.02, 95% CI 1.01 to 1.03, p < 0.001). The effect did not vary significantly by age (p = 0.1). Figure 20 shows the ORs by quartile of Quantra density. There was also a significant effect of Quantra percentage density on risk, although this was less significant than that of absolute dense volume, with a 9% increase in risk per 5% increase in per cent density (OR = 1.09, 95% CI 1.02 to 1.16, p = 0.01). Table 19 shows the ORs by quartile of Quantra percentage density.

FIGURE 20.

Odds ratios by quartile of Quantra absolute dense volume. Quartile 1, < 59 cm³; quartile 2, 59–92.99 cm³; quartile 3, 93–142.99 cm³; and quartile 4, > 143 cm³. Vertical lines indicate 95% CIs.

TABLE 18 - Breast composition measures using Quantra

SD, standard deviation.

Age (years)	Breast composition measure	Mean (SD) for population
		Cancers	Assessment non-cancers	FH non-cancers	All non-cancers
< 50	Breast volume (cm3)	1118 (723)	1075 (696)	1088 (681)	1085 (685)
	Dense volume (cm3)	143 (99)	142 (128)	137 (99)	138 (107)
	% density	14 (7)	14 (7)	14 (7)	14 (7)
	No. of subjects	29	313	938	1251
50–59	Breast volume (cm3)	1195 (670)	1079 (636)	1044 (602)	1078 (636)
	Dense volume (cm3)	131 (93)	114 (99)	111 (82)	114 (88)
	% density	12 (6)	11 (6)	13 (9)	11 (6)
	No. of subjects	459	3088	43	3131
≥ 60	Breast volume (cm3)	1126 (582)	1054 (570)	612 (121)	1052 (570)
	Dense volume (cm3)	104 (71)	97 (76)	57 (41)	97 (76)
	% density	9 (5)	9 (5)	9 (5)	9 (5)
	No. of subjects	660	1402	3	1400
All ages (including age missing)	Breast volume (cm3)	1156 (642)	1073 (622)	1088 (678)	1075 (632)
	Dense volume (cm3)	116 (83)	111 (88)	135 (97)	115 (90)
	% density	11 (5)	11 (6)	14 (7)	11 (6)
	No. of subjects	1156	4821	1028	5849

TABLE 19 - Odds ratios by quartile of Quantra percentage density

% density	OR	95% CI
< 7	1.00	–
7–9.99	1.20	1.00 to 1.43
10–13.99	1.08	0.88 to 1.33
≥ 14	1.40	1.14 to 1.72

Of the three measures which were significantly associated with breast cancer risk, the highest standardised OR (OR per standard deviation of the measure) was observed for Volpara absolute density, 1.1718, with Quantra absolute density being very close to this, 1.1661. Quantra per cent density had a considerably lower standardised OR, 1.1076.

Discussion

The striking result from this analysis is the lack of correlation between observer scores of breast density and an automated analysis. Observer measurement of breast density has been shown in other studies to suffer from interobserver variability123,124,131,132 and a recent paper has attempted to provide a correction for differences between observers. 133

From our results it is clear that observers are able to discriminate different densities in subjects with low automated breast densities. However, when examining the histograms, shown in Figure 14, of the area-based measurements from both human observers and software analysis, there is clearly a difference in the distribution of scores. This may in part be a result of the observers applying a semi-volumetric approach to the assessment rather than a purely area-based one (Dr Ralph Highnam, Matakina International Limited, 2013, personal communication); however, when we compare the distribution with those found in the literature for visual and threshold methods,131,134 it is the software that produces the more comparable distribution. Results for the FH cases where two observers have scored the same images show that in 70% of cases there was good agreement. In 8% of cases there was, however, a disagreement of more than 20%, which is a result that has been noted elsewhere. 132,135 It should be noted that, although the observers were experienced at reading mammograms, they do not routinely estimate breast density in the NHSBSP. However, in the UK in symptomatic practice, diagnostic mammograms are categorised to a three-point scale (fatty, mixed and dense).

One possible technical reason for the difference in observer-based density scores and automated scores could be the processing of the displayed image. The software analyses the raw digital data. The observers make their estimation of density on processed images optimised for display on the workstation and have the ability to alter the window width and level of the greyscale applied to the image’s pixels. This adjustment can significantly alter the image presentation and, therefore, potentially affect the density estimate. In this study, readers were advised not to alter the window levels. Comparing the two automated techniques, it can be seen that there is very good agreement when evaluating the overall breast volume. This is presumably a result of the relative ease with which each algorithm identifies the breast against the background, while each applies its own proprietary corrections to account for compression paddle height and tilt. 121,128,130

There was a less agreement in the assessment of the fibroglandular volumes. There was relatively good agreement at lower volumes; however, it became poorer as those volumes increased, particularly above 400 cm³. This then led to differences in the resulting volumetric density measurements. The reasons for this discrepancy are unclear but are most likely a result of the different methods used by each software tool. Each algorithm allocates differing ratios of fibroglandular, adipose and skin tissues to each pixel based on their relative X-ray attenuation with reference to pixels in the image defined at pure adipose or fatty tissue. 121 As density increases, it becomes harder to identify these reference areas and each manufacturer’s solution to this problem is likely to result in differences in the final volumetric density results. 121,128

Some images produced quite different results as indicated by the outliers present in the fibroglandular volume and volumetric density scores shown in Figures 11 and 13. Further review of the images contributing to these outliers suggest that, on rare occasions, ‘non-standard’ image presentation can result in an incorrect estimation of the relevant tissue volumes while not triggering an error message in the results.

For example, in one left MLO view, the presence of a pacemaker resulted in significantly different results for fibroglandular tissue volume, with Volpara software scoring 425 cm³ and Quantra software scoring 303 cm³, leading to volumetric densities of 19.4% and 13%, respectively, for that image. A mammogram of a visibly extremely dense breast with unusual texture resulted in each system producing very different scores for all of the volume measures, including overall breast volume. The cause of this is unknown but again may be a result of the software requiring a particular reference point where the tissue composition is assumed, for example pixels that are entirely composed of fat. Such reference points may have been difficult to locate in such a dense image. Finally, significant differences in scores were found in an image where tissue was missing from the edge of the mammogram.

Analysis of the relationship between cancer incidence and volumetric density has shown that there is a significant association of increased risk with density. This relationship was seen to be stronger with absolute measurements of the fibroglandular tissue volume than the percentage volume from both the Quantra and Volpara software. The ORs calculated here are lower than those found in the literature;136 however, this population may be biased and not reflective of the general screening population. In addition, only screen-detected cancers were included here. Interval cancers collected in the future may show this study to have underestimated the risk.

If volumetric density is to be used to estimate breast cancer risk, it is important that the measurements be reliable. It is also important to understand that there are technical differences in the way in which each software package determines the fibroglandular tissue volume, and therefore the density. In addition, any long-term monitoring of breast density may require careful consideration if switching measurement methodology. The effect of the individual machine acquisition parameters and image processing together with the use of different software algorithms is worthy of further investigation to determine the overall effect on density scores in order that longitudinal studies can be undertaken.

The literature indicates that breast density may be an independent risk factor for breast cancer but is also correlated with age and body mass index and may be modified by hormone replacement therapy, FH and ethnicity. 93,101 Figure 17 illustrates the variation with age, and any attempt to use density to assess risk must take such confounding factors into consideration.

Recruitment

A total of 8869 participants were recruited from the six participating centres over a 21-month period (Tables 20 and 21 and Figure 21). This data set comprised 7684 assessment cases and 1185 cases from women aged 40–49 years at moderate or high risk of developing breast cancer owing to their FH (subsequently referred to as FH cases). The latter group was included to provide a group of cases with higher breast density for subanalysis of the impact of breast density on the diagnostic accuracy of DBT. At the start of recruitment, some centres were still in the process of setting up annual screening (National Institute for Health and Care Excellence Familial Breast Cancer Guidelines) for FH women;81 therefore, recruitment in this subgroup was at a slower rate initially. However, an interim analysis of data indicated that the trial was approaching its proposed recruitment target of 7000 cases, but with only 12% cancer cases, as opposed to the 18% predicted. The trial management group decided in September 2012 to cease recruitment of FH cases and continue with recruitment of a further 2000 assessment cases to provide sufficient cancer cases to give 90% power.

FIGURE 21.

Cumulative recruitment.

Data sets

After exclusions had been applied (n = 207), we were left with a prospective data set of 8662 cases (Figure 22). Not all of these could be included in the retrospective reading study, since we were restricted to reading only around 7000 owing to budget and time constraints. In total, 1412 cases were not allocated to the reading study. These included those cases with which there were problems in image transmission resulting in an incomplete set of images. Issues around transmission of images are described in Appendix 1. These appear to have been random and not attributable to any particular type, for example those with larger files. Problems were more likely to be a result of timing of image transfer, speed of connection and various system firewalls. This resulted in a reading study data set of 7250 cases (Figure 22).

FIGURE 22.

Study flow chart. Green text represents the total number of women; blue text presents the number of women with cancer.

There were a few cases (n = 190) for which we collected data for only one arm of the study. This was either because the case was not allocated to be read in any other arm (e.g. logistics, not being converted to 2D synthetic, image management issues) or because the data were not received back from sites (lost in transit, logistics). Discounting these resulted in a final data set for analysis of 7060 cases (assessment cases: n = 6020, FH cases: n = 1040; Figure 22).

Analysis

After exclusions, there were a total of 7060 subjects for analysis. The data set comprised 6020 (1158 cancers) assessment cases and 1040 (two cancers) FH surveillance cases. Reading data were available for 6927 (98%) cases by 2D only, 6959 (99%) by 2D + DBT and 6653 (94%) by 2D synthetic + DBT. Table 22 shows the characteristics of the study population.

The analysis included cases which were read in only two arms of the study to avoid introducing bias. It was run again using only cases which were read in all three arms and produced identical results.

In addition to the matched data available for 1137 cancer cases which had both 2D and 2D + DBT imaging, unmatched data were available for 18 cases for 2D imaging only and for another five cases for 2D + DBT imaging only. Pooling the matched and the unmatched data and analysing with a modification of the Mantel–Haenszel method137 resulted in an OR for cancer detection (2D + DBT vs. 2D alone) of 1.363 (95% CI 1.002 to 1.852, p = 0.048), indicating significantly better sensitivity for 2D + DBT than 2D alone.

Table 23 shows sensitivity and specificity for specific subgroups and for all subjects combined. For all subjects combined, sensitivity was 87% (95% CI 85% to 89%) for 2D only, 89% (95% CI 87% to 91%) for 2D + DBT and 88% (95% CI 86% to 90%) for synthetic 2D + DBT. The difference in sensitivity between 2D and 2D + DBT was of borderline significance (p = 0.07). There was no significant difference in sensitivity between 2D and synthetic 2D + DBT (p = 0.6). Specificity was 58% (95% CI 56% to 60%) for 2D, 69% (95% CI 67% to 71%) for 2D + DBT and 71% (95% CI 69% to 73%) for synthetic 2D + DBT. Specificity was significantly higher for 2D + DBT and for synthetic 2D + DBT than for 2D (p < 0.001 in both cases).

The increased specificity with 2D + DBT and synthetic 2D + DBT was observed in all subgroups of density and dominant radiological feature and across all age groups (p < 0.001 in all cases). Specificity tended to be lower by all three modalities for microcalcifications and higher by all three modalities for distortion/ASD density, but the significant improvement in specificity for both 3D modalities was consistently observed in these categories.

Sensitivity was significantly higher (p = 0.01) for 2D + DBT than 2D alone for age range 50–59 years. Sensitivity was significantly higher (p < 0.001) for 2D + DBT than for 2D alone for invasive tumours of size 11–20 mm, with a sensitivity of 86% (95% CI 82% to 90%) for 2D and 93% (95% CI 90% to 96%) for 2D + DBT.

For the purposes of analysis stratified by density, we classified density as below 50% or 50% and above. We had planned to use 70% density as the cut-off, but there were only 105 cancer cases with density of 70% or more.

Significantly higher sensitivity (p = 0.03) was also observed for 2D + DBT for those with density 50% or more, with a sensitivity of 86% (95% CI 82% to 90%) for 2D compared with 93% (95% CI 90% to 96%) for 2D + DBT. A similarly increased sensitivity (p = 0.01) was seen for 2D + DBT in grade 2 invasive tumours (but not grade 1 or grade 3), with sensitivity of 87% (95% CI 84% to 90%) for 2D and 91% (95% CI 88% to 94%) for 2D + DBT. A significant increase in sensitivity was observed for 2D + DBT (p = 0.04) where the dominant radiological feature was a mass, with 89% (95% CI 86% to 92%) sensitivity for 2D and 92% (95% CI 89% to 95%) for 2D + DBT.

For synthetic 2D + DBT, there was significantly (p = 0.006) higher sensitivity than for 2D alone in invasive cancers of size 11–20 mm, with a sensitivity of 92% (95% CI 89% to 95%). No other significant differences in sensitivity were noted.

Table 24 shows the distribution of detection of cancers for all combinations of modalities in those 1112 cancers for which we have reading data for all three reading arms. In total, 1079 (97%) cancers were detected by at least one reading arm; 840 (75%) cancers were detected by all three reading arms. A total of 142 (13%) cancers were missed by 2D, 118 (11%) by 2D + DBT and 136 (12%) by synthetic 2D + DBT. Of these, 33 (3%) cancers were missed in all three reading arms. DCIS cases showed similar distribution to all cancers combined.

Table 25 shows characteristics of these missed cancers for each imaging modality, plus those found by all three modalities. Two major differences among the modalities were that cancers missed by 2D alone tended to be of size 11–20 mm or to have a mass as the major radiological sign, compared with the other two modalities; and cancers missed by 2D + DBT were less likely to be of grade 2 or to have density less than 50% than the other two modalities.

Receiver operating characteristic analysis

Receiver operating characteristic analysis was performed to compare the diagnostic accuracy of the three reading arms (2D alone vs. 2D + DBT vs. synthetic 2D + DBT) by calculating the AUC and p-value. Areas under the curve were compared using the method of DeLong et al. 91

Figure 23 shows the ROC curve for each reading arm on the same graph.

FIGURE 23.

Receiver operating characteristic curves for the three reading arms.

For 2D alone, the AUC was 0.84 (95% CI 0.83 to 0.86), for 2D + DBT the AUC was 0.89 (95% CI 0.87 to 0.90) and for synthetic 2D + DBT the AUC was 0.87 (95% CI 0.86 to 0.89). Both 2D + DBT and synthetic 2D + DBT had significantly greater AUCs than 2D alone (p < 0.001 in both cases).

Although, in general, specificity was considerably higher for the two DBT arms at very low levels of sensitivity, 2D alone would have a slightly higher specificity than 2D + DBT or synthetic 2D + DBT. However, thereafter, the ROC curves for the two DBT arms would always be higher than for 2D alone.

ROC analysis was also carried out on specific subgroups:

1. cases with visually assessed density of less than 50%

2. cases with visually assessed density of 50% or more

3. cases where the dominant radiological sign was a soft-tissue mass

4. cases where the dominant radiological feature was microcalcification

5. cases where the dominant radiological feature was asymmetry or distortion.

Figure 24 shows the ROC curves for the three reading arms in those cases with visually assessed breast density of less than 50%. The AUCs were 0.85 (95% CI 0.83 to 0.87) for 2D alone, 0.89 (95% CI 0.87 to 0.90) for 2D + DBT and 0.88 (95% CI 0.86 to 0.90) for synthetic 2D + DBT. The AUCs for 2D + DBT and synthetic 2D + DBT were significantly greater than that for 2D alone (p < 0.001 in both cases). Figure 25 shows the corresponding curves in those with visually assessed density of 50% or more. The AUCs were 0.83 (95% CI 0.80 to 0.86) for 2D alone, 0.89 (95% CI 0.87 to 0.92) for 2D + DBT and 0.87 (95% CI 0.84 to 0.89) for synthetic 2D + DBT. Both 2D + DBT (p < 0.001) and synthetic 2D+ DBT (p = 0.005) had significantly greater AUCs than 2D alone.

FIGURE 24.

Receiver operating characteristic curves for the three reading arms in those cases with visually assessed density less than 50%.

FIGURE 25.

Receiver operating characteristic curves for all three reading arms in those with visually assessed density of 50% or more.

Figure 26 shows the ROC curves in those subjects in whom the dominant radiological feature was a mass. The AUCs were 0.86 (95% CI 0.84 to 0.88) for 2D alone, 0.92 (95% CI 0.90 to 0.94) for 2D + DBT and 0.91 (95% CI 0.89 to 0.93) for synthetic 2D + DBT. The AUCs for 2D + DBT and for synthetic 2D + DBT were significantly greater than for 2D alone (p < 0.001 in both cases).

FIGURE 26.

Receiver operating characteristic curves for the three reading arms in those cases where the dominant radiological sign was a mass.

The corresponding curves for those with microcalcifications as the dominant sign are shown in Figure 27. The AUCs were 0.73 (95% CI 0.69 to 0.77) for 2D alone, 0.73 (95% CI 0.69 to 0.77) for 2D + DBT and 0.72 (95% CI 0.68 to 0.76) for synthetic 2D + DBT. The AUCs did not differ significantly from 2D alone for either 2D + DBT (p = 0.9) or synthetic 2D + DBT (p = 0.8).

FIGURE 27.

Receiver operating characteristic curves for the three reading arms in those cases with microcalcifications as the dominant radiological feature.

Figure 28 shows the ROC curves for those with distortion or asymmetry as the dominant radiological feature. The AUCs were 0.82 (95% CI 0.78 to 0.86) for 2D alone, 0.86 (95% CI 0.82 to 0.89) for 2D + DBT and 0.87 (95% CI 0.83 to 0.90) for synthetic 2D + DBT. Significantly greater AUCs than for 2D alone were noted for both 2D + DBT (p = 0.03) and 2D synthetic + DBT (p = 0.002).

FIGURE 28.

Receiver operating characteristic curves for all three reading arms in those with distortion or ASD as the dominant radiological feature.

These differences in AUC are mostly driven by differences in specificity, including the low AUCs for all three reading arms in those with calcifications as the dominant radiological feature. However, there is clearly a shift to greater confidence in malignancy among cancers with the addition of DBT. With 2D alone, 39% of cancers were scored as 5 (malignant), whereas in both 2D + DBT and synthetic 2D + DBT 53% of cancers were scored as 5.

Future research

• Since no formal cost-effectiveness/cost–benefit evaluations have been published, we suggest that the feasibility and practicalities of implementing DBT into the workflow of a UK screening setting should be evaluated. This should include a cost-effectiveness evaluation or modelling. This should also include evaluation of the additional costs (e.g. upgrading equipment), providing information technology (IT) support for image archiving, connectivity to PACS and IT systems, increased reading time and potential benefits (e.g. increased cancer detection, particularly small invasive cancers, reductions in false-positive recalls, additional diagnostic testing, and reduction in interval cancers).

• For combined imaging with 2D + DBT to be implemented in screening, the use of synthetic 2D to minimise radiation exposure would be advantageous. The overall non-inferiority of synthetic 2D + DBT to 2D alone, shown in our study and in the publications of Skaane et al. 78 and Zuley et al. ,143 would justify use of this imaging combination in a RCT in a screening setting. However, before synthetic 2D + DBT could be recommended for screening, further comparative work with synthetic 2D and 2D alone should be undertaken; for example quantifying the effect on sensitivity and specificity for lesions with different radiological appearances, of different pathological types (e.g. DCIS, invasive, different grades of tumour) and in different types of breast (e.g. > 50% density).

• It is important to note that the majority of published studies have utilised the Hologic DBT system. Apart from some reading studies, relatively little literature exists for other commercially available DBT systems. Therefore, more information is required from screening studies with other DBT systems to verify the improvements in sensitivity and specificity shown by the Hologic system. This could be partly achieved by modelling using simulated lesions and observer studies such as OPTIMAM (Optimisation of breast cancer detection using digital radiograph technology). 146

• Comparison should be made with 2D on the size, grade, and type of cancers detected by DBT, for interval cancers and for cancers detected in the subsequent screening round, where we would not expect the same degree of improvement in the detection of invasive cancers.

• We would recommend prognostic modelling on existing data sets of screen-detected and interval cancers to predict the outcome of cases and projected impact on breast cancer mortality. This could be achieved using, for example, the Nottingham Prognostic Index. 147

• Mammographic density is one of the major risk factors for breast cancer and has the potential to be incorporated into strategies for personalised risk prediction and screening guidelines. For example, younger women with dense breasts, for whom standard 2D mammography is less effective, could benefit more from screening with DBT, and recommendations to customise breast screening frequency for women with different breast cancer risk profiles could also be made. These require development of a validated method of assessing breast density.

• Further research is required into the ability of DBT to discriminate between early aggressive cancers and slow-growing DCIS to address any concerns about DBT contributing to overdiagnosis.

Contribution of authors

Fiona J Gilbert (Professor of Radiology, University of Cambridge): chief investigator for the TOMMY trial, principal in conception and design of study, interpretation of data and writing final report.

Lorraine Tucker (Research Radiographer, University of Cambridge): co-ordinator of the TOMMY trial, responsible for day-to-day management of the trial, conducted QC image review and drafted and made revisions to final report.

Maureen GC Gillan (Research Fellow, University of Aberdeen): co-ordinator of the TOMMY trial when it was based in Aberdeen. Instrumental in drafting of trial proposal and trial protocol and assisted in drafting and review of final report.

Paula Willsher (Research Radiographer, University of Cambridge): conducted image review and QC checks of prospective data, conducted logic checks on final data and assisted in revision and review of final report.

Julie Cooke (Consultant Radiologist, Clinical Director, Jarvis Breast Centre, Guildford): PI at trial site, reader in retrospective reading study and reviewed and contributed to final report.

Karen A Duncan (Consultant Radiologist, North East Scotland Breast Screening Centre, Aberdeen): PI at trial site, reader in retrospective reading study and reviewed and contributed to final report.

Michael J Michell (Consultant Radiologist, Clinical Director Breast Radiology Department, King’s College Hospital, London): PI at trial site. Provided expertise from previous trial for study design, conducted tomosynthesis training for readers and provided images for test set. Participated in retrospective reading study and reviewed and contributed to final report.

Hilary M Dobson (Consultant Radiologist, Clinical Director West of Scotland Breast Screening Service, Glasgow): PI at trial site, reader in retrospective reading study and reviewed final report.

Yit Yoong Lim (Consultant Radiologist, The Nightingale Centre, University Hospital South Manchester): PI at trial site, reader in retrospective reading study and reviewed final report.

Hema Purushothaman (Consultant Radiologist, St Bartholomew’s Hospital, London): PI at trial site, reader in retrospective reading study and reviewed final report.

Celia Strudley (Principal Physicist, NCCPM, Royal Surrey County Hospital, Guildford): developed and conducted QC tests on tomosynthesis equipment used in the TOMMY trial, produced 6-monthly and final imaging equipment QC reports and reviewed final report.

Susan M Astley (Reader in Imaging Science, University of Manchester): involved in trial design. Supervised administration of reader study and analysis of test set results, produced test set report and reviewed final report.

Oliver Morrish (Lead Clinical Scientist, East Anglian Regional Radiation Protection Service, Cambridge University Hospitals NHS Foundation Trust): analysed breast density data and produced breast density report.

Kenneth C Young (Professor, Head of NCCPM and Director of Research, Royal Surrey County Hospital, Guildford): developed QC tests on tomosynthesis equipment used in TOMMY trial. Reviewed and contributed to final report.

Stephen W Duffy (Professor of Cancer Screening, Wolfson Institute of Preventive Medicine, Queen Mary University of London): advised on study design and data collection, conducted statistical analysis for the trial and cancer risk analysis for breast density report. Contributed to interpretation of results and review of final report.

On behalf of the TOMMY trial teams

Other acknowledgements

Ralph Highnam (chief executive officer, Matakina International Limited): for technical advice and support regarding operation of Volpara breast density assessment tool.

Ashwini Kshirsagar (Senior Principal Scientist and Manager, Hologic Ltd): for technical support and advice regarding operation of Quantra breast density assessment software.

Miriam Harris (Queen Mary University of London Trials Advisory Group): lay representative who participated in collaborators’ meetings.

Professor Janet Dunn (Clinical Trials Unit, University of Warwick), Professor Andrew Evans (Chairperson of Breast Imaging, University of Dundee), Professor Julietta Patnick (Director NHS Cancer Screening Programmes): independent members of the Trial Steering Committee.

A Comparison of TOMosynthesis with Digital MammographY in the UK Breast Screening Programme (the TOMMY trial)

You are being invited to take part in our research study. It is important for you to understand why the research is being done and what it will involve. Please take time to read this information sheet carefully and decide whether or not you wish to take part. One of our research team will answer any questions you may have about the study.

A Comparison of TOMosynthesis with Digital MammographY in the UK Breast Screening Programme (TOMMY trial)

You are being invited to take part in our research study. It is important for you to understand why the research is being done and what it will involve. Please take time to read this information sheet carefully and decide whether or not you wish to take part. One of our research team will answer any questions you may have about the study.

Design of the data transfer process

Early in the design of the trial it was decided to create a CDR to act as a hub for redistribution of the images for the retrospective study and to provide a batch processing facility for the density analysis. The CDR would also provide an archive for the total data set.

The CDR was designed and created at the University of Aberdeen on a HP DL385G7 running Red Hat Enterprise Linux, Red Hat Inc, Raleigh, NC, USA and attached to network storage.

Hologic provided an existing anonymisation system (the Windows-based ‘HICS’ workstation) which accompanied the commercial imaging equipment, and was installed at each site. Network transmission was chosen to be the method of transfer for the anonymised images from each HICS to a central repository. This satisfied the criteria of security, speed, flexibility, ease of use, reliability, data integrity and robustness. Customised user software was written for the project to invoke the data transfer applications.

Operation of the data transfer process

Methods

The imaging equipment used for the trial consisted of seven Hologic Dimensions Tomosynthesis mammography X-ray systems and the associated Hologic SecurView image display workstations, of which there were eight at the start of the trial and a further two were added during the course of the trial.

For 2D imaging performance, the existing UK NHSBSP QC protocols84,148 were applied. In the absence of any established protocols for tomosynthesis QC, it was necessary to devise suitable tests and phantoms for the purpose, some of which were an extension of the 2D tests, and others novel tests to check aspects of the quality of the reconstructed tomosynthesis images. The NCCPM provided test phantoms which were specially manufactured for use by radiographers and physicists at each centre to ensure the comparability of QC results between centres. The NCCPM prepared and provided QC protocols (see Appendices 15 and 16) to be used by radiographers and physicists at all centres during the trial. These protocols referred to existing NHSBSP test protocols for 2D testing and included additional tests of tomosynthesis performance.

The NCCPM carried out rigorous initial testing at all six centres during the period July to December 2011, before each of the seven X-ray systems commenced use in the trial. At this stage, efforts were made, with the assistance of the installing X-ray engineers, to minimise differences between systems to ensure that radiation dose and image quality was closely matched. Subsequent 6-monthly testing was carried out by physicists contracted by NCCPM, who, in addition to providing their usual reports on the standard 2D tests, returned the required 2D and tomosynthesis data and images to NCCPM for further analysis and reporting. Radiographers at all centres carried out routine QC on a daily, weekly and monthly basis, and returned data and images, on a weekly basis, to NCCPM for collation and analysis.

The QC testing carried out included tests of the image display equipment used in the retrospective reading phase of the trial, as well as the X-ray equipment used to acquire the patient images.

In accordance with NHSBSP 0604, 84 clinical breast doses were calculated for a sample of 50 or more patients imaged on each system.

Data from key measurements comparing the performance of systems at the start of the trial, and subsequent 6-monthly testing, are presented, as well as an overview of the routine QC carried out by radiographers and the results of the patient dose survey.

Results

Dose

The MGD to the standard breast model, simulated with polymethyl methacrylate (PMMA) phantoms, was assessed for conventional 2D imaging and tomosynthesis for a range of equivalent breast thicknesses. The initial results for the seven systems are shown in Figure 30 and Table 26. The NHSBSP dose limits for conventional digital mammography are also shown in Figure 30.

FIGURE 30.

Mean glandular dose to the standard breast, for (a) conventional exposures and (b) tomosynthesis exposures. BSC, Breast Screening Centre; FHC, Family History Clinic.

TABLE 26 - Mean glandular dose to the standard breast and contrast-to-noise ratio averaged over the seven systems and coefficient of variation, for conventional and tomosynthesis images

CNR, contrast-to-noise ratio; CoV, coefficient of variation.

Equivalent breast thickness (mm)	MGD for 2D images		MGD for tomosynthesis		CNR for 2D images		CNR for tomosynthesis images
	Mean (mGy)	CoV (%)	Mean (mGy)	CoV (%)	Mean	CoV (%)	Mean	CoV (%)
21	0.60	3.4	0.89	3.1	10.9	2.9	29.1	2.5
32	0.84	2.4	1.04	3.5	9.9	2.3	22.2	2.8
45	1.17	2.4	1.43	3.9	9.0	3.5	18.9	2.8
53	1.41	3.2	1.87	4.2	8.4	3.4	18.5	3.5
60	1.98	1.6	2.29	3.8	8.6	3.3	17.2	3.7
75	2.69	2.1	3.39	4.3	8.3	3.0	14.8	4.7
90	3.03	2.5	4.25	3.6	6.7	4.7	11.3	4.3

At 6-month intervals the MGD measurements were repeated. Table 27 summarises the overall averages and ranges of values for all systems throughout the trial.

TABLE 27 - Mean glandular dose measurements for all seven systems during the trial

Equivalent breast thickness (mm)	Average 2D MGD (mGy)	Minimum 2D MGD (mGy)	Maximum 2D MGD (mGy)	Average tomosynthesis MGD (mGy)	Minimum tomosynthesis MGD (mGy)	Maximum tomosynthesis MGD (mGy)
21	0.62	0.58	0.69	0.88	0.82	0.94
32	0.86	0.81	0.94	1.02	0.94	1.08
45	1.18	1.11	1.24	1.42	1.34	1.52
53	1.43	1.36	1.56	1.85	1.73	2.00
60	1.98	1.86	2.15	2.27	2.11	2.42
75	2.66	2.50	2.86	3.36	3.11	3.36
90	2.95	2.76	3.16	4.23	3.91	4.48

The data in Table 27 show that the maximum deviation from the mean for 2D or tomosynthesis dose measurements is 11%. To put these deviations into context, the remedial level for dose measurements (NHSBSP 060484) is a change of 25% relative to baseline dose levels.

Contrast-to-noise ratio

Average contrast-to-noise ratio (CNR) measurements in conventional and tomosynthesis modes for the seven systems involved in the trial are shown in Figure 31 and Table 26. Average CNR measurements in 2D and tomosynthesis modes for the seven systems throughout the trial are shown in Table 28.

FIGURE 31.

Contrast-to-noise ratio for (a) conventional exposures and (b) DBT exposures. BSC, Breast Screening Centre; FHC, Family History Clinic

The data in Table 28 show a maximum deviation from the mean of 13% for 2D and 18% for tomosynthesis, compared with a maximum deviation from the mean for the seven systems of 8% at the initial testing. This is an acceptable spread in CNR measurements, given that the remedial level for a single system is a 10% change from baseline measurements.

Tomosynthesis geometric distortion and interplane resolution

Tomosynthesis z-resolution

The z-resolution for 1-mm aluminium balls was found to range between 9.8 mm and 12.0 mm, with some dependence on position within the image. The mean value was 10.8 mm. No significant variation between the systems or over time during the course of the trial was seen.

Detector modulation transfer function

Measurements of modulation transfer function (MTF) were made at the start of the trial. Only small differences were seen between the seven systems. The MTF measurements are shown in Figure 32 and Table 29.

FIGURE 32.

Modulation transfer function measurements for the seven systems. BSC, Breast Screening Centre; FHC, Family History Clinic.

TABLE 29 - Mean modulation transfer function measurements for the seven systems

Spatial frequency (mm– 1)	Mean MTF	Coefficient of variation of MTF (%)
2	0.779	0.7
4	0.573	1.7
6	0.402	2.6
8	0.308	3.3
10	0.259	4.8
12	0.074	9.7

During the course of the trial, detector resolution for the seven systems was shown to have remained unchanged by measurement of the threshold contrast detail detection, which included assessment of small details.

Threshold contrast detail detectability

Figure 33 and Table 30 show the threshold gold thickness results for a range of detail diameters for conventional and tomosynthesis images of a CDMAM test object (version 3.4, UMC St. Radboud, Nijmegen University, the Netherlands). The results from initial testing on all systems are shown and compared against the minimum acceptable and achievable values for conventional mammography as defined in the NHSBSP protocol. 84

FIGURE 33.

Contrast detail curves from each of the systems for conventional (a) and tomosynthesis (b) images. BSC, Breast Screening Centre; FHC, Family History Clinic.

Repeat measurements of image quality using a CDMAM test object on all systems every 6 months demonstrated that there has been no significant change in image quality in either 2D or tomosynthesis modes and that all systems continue to exceed the achievable image quality standard for 2D mammography.

Quality control by radiographers

Daily, weekly and monthly tests of equipment performance were carried out by radiographers on the seven systems used in the trial, in accordance with the trial protocol for radiographer QC (see Appendix 17). All QC results were recorded electronically using a spreadsheet or database, and copies were sent to NCCPM for checking. By carrying out daily QC, radiographers were able to check that X-ray systems were functioning properly prior to use on patients and that no significant changes in dose or image quality had occurred. As well as checking that no significant abnormal artefacts were evident in images of plain PMMA, radiographers also examined reconstructed tomosynthesis images of a 1-mm-diameter aluminium ball, to ensure that there had been no change in its appearance, which might suggest a failure in the tomosynthesis image reconstruction. Weekly QC images sent to NCCPM for detailed analysis showed that dose and signal-to-noise ratio (SNR) in both conventional and tomosynthesis images have remained stable and no clinically significant artefacts have been seen. The variation in dose and SNR measurements for each system over time are shown in Figure 34. [The tomosynthesis SNR results for one centre (Jarvis) doubled half way through the trial owing to a software upgrade necessitated by an equipment breakdown. This led to a change in the calibration of the pixel values and hence SNR, which has since remained stable. Subsequent checks have confirmed the manufacturer’s assertion that the change is seen in QC images only, and that clinical image quality will not have been affected. This change also affected the physicist’s measurements of CNR as described in pages 99–100.]

FIGURE 34.

Variation in 2D mammography and tomosynthesis dose and SNR from analysis of weekly QC images of 45-mm PMMA. (a) Weekly 2D mammography dose, (b) weekly Tomo dose, (c) weekly 2D mammography SNR and (d) weekly Tomo SNR. BSC, Breast Screening Centre; FHC, Family History Clinic.

Radiographers were also responsible for regular checks on the SecurView image display workstations and reported to NCCPM that these checks were continued through to the end of the retrospective reading phase of the trial.

Patient dose survey

Exposure data from each centre were analysed in order to calculate the estimated patient doses. A summary of the data and calculated patient doses is presented. These results demonstrate that the seven systems employed for the trial were set up to give doses that varied by less than 10% from the mean in 2D and tomosynthesis modes. Although measurements of MGD to the standard breast model are one-third higher for tomosynthesis than for 2D, estimates of clinical doses show that tomosynthesis doses are on average only about 13% higher than 2D doses.

A summary of the patient data gathered at each centre is shown in Table 31.

TABLE 31 - Summary of patient dose data gathered from each centre

Centre	Number of patients for whom dose data was gathered	Period over which patient dose data was collected		Mean age of patient (years)	Mean compression force (N)
		From	To
Aberdeen (FH)	56	15 December 2011	19 March 2012	45	94
Aberdeen (screening)	106	19 December 2011	21 March 2012	57	109
Barts	74	12 July 2011	20 January 2012	55	80
Glasgow	99	30 August 2011	06 October 2011	57	119
Guildford	89	11 July 2011	25 August 2011	57	102
King’s College	105	6 September 2011	9 February 2012	57	91
Manchester	178	15 March 2012	30 April 2012	55	119

Average MGD to 50- to 60-mm breasts for each unit are shown in Table 32, together with the MGD to a 53-mm standard breast, simulated with 45-mm PMMA.

TABLE 32 - Mean glandular dose to 53-mm standard breast and average MGD for oblique views of 50- to 60-mm breasts

Centre	2D		Tomosynthesis
	MGD to 53-mm standard breast (mGy)	Average MGD to 50- to 60-mm breasts (mGy)	MGD to 53-mm standard breast (mGy)	Average MGD to 50- to 60-mm breasts (mGy)
Aberdeen (FH)	1.41	2.02	1.88	2.13
Aberdeen (screening)	1.36	1.87	1.80	2.04
Barts	1.39	1.62	1.92	2.11
Glasgow	1.41	2.16	1.79	2.10
Guildford	1.38	1.93	2.00	2.23
King’s College	1.47	2.15	1.80	2.02
Manchester	1.48	1.60	1.89	2.14
Average for all sites	1.41	1.87	1.87	2.11

The patient dose measurements from all sites are shown in Figure 35. Also shown are the average measurements from all sites of MGD to the standard breast model, as listed in Table 29.

FIGURE 35.

Patient dose data from all sites (all views). Also shown are lines representing the average MGD to the standard breast measured using PMMA for 2D and tomosynthesis exposures.

A summary giving the range of clinical patient doses (minimum, maximum and mean MGD measurements) for 2D and tomosynthesis views from patient data for all breast thicknesses at all sites is shown in Table 33.

TABLE 33 - Summary of patient dose measurements at all sites, all views, all breast thicknesses

Mammographic views	Minimum MGD (mGy)	Maximum MGD (mGy)	Mean MGD (mGy)
All 2D views	0.59	7.48	2.12
All tomosynthesis views	0.79	5.20	2.57

Discussion

Performance testing of the imaging equipment employed for the trial ensured that the seven X-ray systems were set up to give exposures that were closely matched, thus ensuring that there was no significant variation in image quality between the systems. Thorough testing by physicists on behalf of NCCPM at 6-month intervals ensured that there were no significant variations in imaging performance during the course of the trial. More frequent tests carried out by radiographers at daily, weekly and monthly intervals ensured that all systems were closely monitored during the trial so that any problems that could have arisen were dealt with promptly. Tests carried out on image display equipment used to view clinical images were tested regularly in accordance with NHSBSP protocols. The QC testing demonstrated that all of the imaging equipment met, or exceeded, existing UK standards for 2D mammography, and that the 2D and tomosynthesis imaging performance of all systems were well matched.

The survey of patient radiation doses from each centre confirmed the findings of the physical QC measurements, indicating that radiation exposures employed by the seven systems were well matched.

Conclusions

The QC program employed for the trial ensured that the imaging performance of all equipment employed for the trial was well matched and met current applicable UK standards for 2D digital mammography, both at the start and during the course of the trial. The experience gained has contributed to the development of QC protocols for tomosynthesis in the UK and Europe.

Compressed breast thickness indication

Use blocks of PMMA of known thickness. Position on breast support table overhanging the chest wall edge such that the back edge of the top block is 8 cm from the chest wall edge (thus allowing the paddle to tilt slightly as it would during a clinical mammogram). Compress to 100 N and record the actual thickness and the indicated thickness. Repeat for thicknesses of approximately 2 cm, 5 cm and 7 cm or 9 cm.

Reproducibility and homogeneity

Make 5 flatfield tomo exposures of large (24 × 30) 45-mm PMMA block with automatic exposure control (AEC) in Autofilter mode at the beginning of testing, with the paddle and using the same compressed breast thickness for all exposures to ensure selection of the same kV. Make a further exposure a few hours later and/or at the end of testing. These are used to check stability of mAs and SNR (though a valid SNR cannot be calculated for the Hologic Dimensions system due to the scaled pixel values), and also to check for homogeneity and artefacts.

Also make flatfield tomo exposures as above in AutokV and Autotime modes (with kV manually selected to match that for the Autofilter exposures) to ensure that the mAs at a given kV does not vary between AEC modes. Also try a combo exposure in each AEC mode and check that the exposure factors for the 2D and tomo exposures are not different from those obtained when making individual exposures.

During these scans measure tomo and scan times with a stopwatch. Tomo time begins when the expose button is pressed and ends at decompression. Scan time begins at start of first X-ray exposure to end of last X-ray exposure (measure this in Autofilter/Autotime AEC mode and in manual mode).

Polymethyl methacrylate dose and thickness compensation

The Hologic system chooses beam quality based purely on indicated compressed breast thickness which is unfortunately in error when large PMMA blocks are compressed due to the paddle being prevented from tilting as it normally would in clinical situation. PMMA doses and CNR measurements should therefore be measured without spacers and the paddle positioned at such a height as to give the desired compressed breast thickness (i.e. equal to the equivalent breast thickness for the thickness of PMMA on the table). For 2 to 4-cm PMMA the top block may need to be moved forwards in order to allow the paddle to come down far enough. PMMA dose measurements are combined with the thickness compensation measurements.

Tube output

Half-value layers (HVLs) and tube outputs are measured in conventional and tomosynthesis modes with the paddle in the beam for the purpose of MGD calculations. The Monte Carlo simulations on which MGD calculations are based were actually carried out with the paddle in contact with the ion chamber, which increases the dose measured by a few per cent. For the TOMMY trial we have decided to use doses measured with the paddle in contact with the ion chamber for both tomosynthesis and conventional MGD so that the measurements are comparable. The dose with the paddle not in contact with the ion chamber will also be measured so that the MGDs for conventional mammograms can be compared to those from other systems measured in the usual way with the paddle not in contact according to the UK protocol.

Place steel or lead plate on breast support table to protect detector (should be a piece of ‘heavy wood’ labelled MIS in the room).

Position ion chamber at standard position on midline 4 cm from chest wall edge 10 cm above the table. Position the paddle immediately above and in contact with the ion chamber. (It may be easier to put the paddle in position first.) Position a collimator above the paddle close to the tube port such that the light beam is collimated to the chamber. The HVL filters will be placed on top of this collimator.

Select 50 mAs and measure the output and HVL for the clinical range of beam qualities in conventional and tomosynthesis modes. The HVL should be measured using thicknesses of aluminium filters just below and above the actual HVL.

For the tomosynthesis exposures use the ‘zero degree tomo’ setting (a series of short exposures without rotation, dosemeter will need to be set to accumulation mode).

Record actual mAs (likely to differ slightly from setting) and dose reading, focus-to-chamber distance, focus-to-table distance.

Detector response, modulation transfer function, noise power spectrum

These tests are to be carried out in conventional 2D mode only.

Alignment

Geometric distortion

Place the geometric distortion grid on the breast support table with large slabs of PMMA on top to make a total thickness of around 60 mm. Bring paddle down to rest on top of the PMMA and image using flatfield tomo Autofilter AEC setting. Repeat with the geometric distortion phantom in the middle of and on top of the stack of PMMA. (If there is evidence of distortion in the images then further investigation may be required at additional heights within the PMMA stack.)

Z-resolution

Place large 5-mm sheet of PMMA on breast support table with Z-resolution phantom (5-mm PMMA containing 6 × 1-mm aluminium balls) on top. Add a further 40-mm PMMA to make up the total thickness to 50 mm. Bring paddle down to rest on top of the PMMA and image using flatfield tomo Autofilter AEC setting. Repeat with the Z-resolution phantom positioned approximately in the middle of the stack, and again 5 mm from the top of the stack.

Image quality

Position CDMAM aligned with the chest wall edge of the breast support table sandwiched between 2- × 2-cm PMMA and bring paddle down to rest on top of the PMMA. Check the image to ensure that the edges of the CDMAM are within the image. Repeat to give a total of 16 images, moving the CDMAM very slightly between exposures.

List of required tests

Instructions for quality control tests

For the conventional 2D digital mammography tests, the instructions given in NHSBSP 0702 should be followed. Further notes on some of these tests are included later in this document.

There are two additional tests required by Hologic which should be carried out according to their instructions given in the Hologic Selenia Dimensions QC Manual. One of these is the weekly flat field calibration for conventional 2D mammography, the other is the geometry calibration for tomosynthesis systems.

Instructions for the other three additional tomosynthesis tests for the TOMMY trial are included later in this document.

Quality control equipment

It is expected that mammography centres taking part in the trial will already have the required equipment for testing conventional digital mammography systems according to NHSBSP 0702.

Equipment for the manufacturer’s required QC (flat field and geometry calibrations) are supplied with the system.

National Co-ordinating Centre for the Physics of Mammography supply a test phantom with which to carry out the additional weekly remote QC tests, the images from which are sent to NCCPM.

Recording of quality control results

Conventional 2D QC should be recorded in the normal way as per local procedure. A weekly summary of 2D local QC is also to be entered onto the spreadsheet provided and sent to NCCPM each week. The spreadsheet also contains pages for the recording of the additional tomosynthesis tests. These pages can be printed out and used as paper forms to collect the data on a daily basis, to then be copied into the spreadsheet and sent to NCCPM each week. Where there is an existing digital QC spreadsheet in use, the TOMMY spreadsheet may be grafted onto it to avoid unnecessary duplication in data entry.

Physics support

National Co-ordinating Centre for the Physics of Mammography intend to appoint a local mammography physicist for each site to provide physics cover for the equipment used for the TOMMY trial. If it is not possible to appoint a local TOMMY physicist then NCCPM will provide the required physics cover. Where QC results fail remedial levels, and the problem is not resolved, then the TOMMY physicist should be asked for advice before continuing to use the equipment for the trial. If the local TOMMY physicist is not available then NCCPM should be consulted instead. In any event, NCCPM should be kept informed of any problems that occur.

Quality control baselines and remedial levels

Baselines for conventional 2D tests will be set in the usual manner according to local protocol and remedial levels will be set following recommendations in NHSBSP 0702. Baselines and remedial levels for the tomosynthesis tests should be set in a similar manner using the provisional remedial levels quoted in the table summarising remedial levels for all tests appended to this document. The TOMMY physicist should be consulted if assistance is required in setting baselines and remedial levels. Once the baselines and remedial levels have been set they should be sent to the TOMMY physicist and to NCCPM, so that they can be checked and compared with those from other sites in the trial. If there is ever any need to alter any baselines or remedial levels, the TOMMY physicist and NCCPM must be consulted. As experience is gained in the testing of tomosynthesis equipment, NCCPM may decide to make changes to test methods or remedial levels, and will inform all concerned.

Reporting of quality control to the National Co-ordinating Centre for the Physics of Mammography

Each week the following should be sent to NCCPM:

• The completed QC spreadsheet containing a summary of the 2D QC results and the tomo test records.

• The weekly remote QC images (not from SecurView or PACS):

  • The tomosynthesis image of aluminium ball in PMMA.

  • The tomosythesis image of plain PMMA.

  • The 2D image of plain PMMA.

The weekly QC summary should be sent by e-mail to the e-mail address given below.

Quality control images need to be written to a CD/DVD directly from the acquisition workstation and sent to NCCPM. We require raw tomosynthesis images which are not transferred to the PACS network or SecurView workstation. These can only be written to CD or DVD using the acquisition workstation in the X-ray room.

Instructions for downloading QC images to CD/DVD:

Open the weekly QC study and check that it contains the required QC images and not patient images. (Obviously it is very important that no patient image is included as this would contravene data protection laws). Put a blank CD or DVD in the drive and click on the ‘Export’ icon. Select destination ‘E:’ and start. Only one study can be written onto each disc.

When the facility becomes available it is planned that QC images will be transferred from the acquisition workstation to NCCPM via secure internet link.

Contact details for the National Co-ordinating Centre for the Physics of Mammography:

Celia Strudley is the main contact for TOMMY QC, but if not available other staff should be able to help.

Address: NCCPM (Medical Physics)

Level B, St Luke’s Wing Royal Surrey County Hospital Guildford

GU2 7XX

Telephone: NCCPM office: xxxxx xxxxxx

Celia direct: xxxxx xxxxxx ext xxxx E-mail: xxxxxxxxxxxxxxxxxxxxxxxxxx

(This e-mail address will be monitored by NCCPM staff dealing with QC issues during the trial)

Tomosynthesis quality control instructions

Notes on quality control tests

Acquisition of patient images for the trial may only take 12 to 18 months, after which time, the TOMMY trial requirements for QC on the X-ray units may cease, but QC will need to be continued for the reporting monitors until reading of the images for the trial has finished at the centre. When patient image acquisition has finished, weekly reporting of QC results to NCCPM will not be required and monthly reports of monitor QC to NCCPM will suffice.