Insights from UKCTOCS for design, conduct and analyses of large randomised controlled trials

Estimation of accrual of events (primary outcome)

A key factor in the power calculation that underpins any trial design is the number of primary outcome events in the control arm. An important consideration in the design of any trial with time-to-event data is the prediction of when these events will occur, and specifically how long it will take to accumulate the necessary number of control arm events implicit in calculations used to power the study. This involves the specification of a target sample size to be randomised plus a follow-up period. Accordingly, it is important that event rate prediction is as accurate as feasible, incorporating relevant and knowable factors.

In the initial design of UKCTOCS, the sample size calculations required a control event (ovarian cancer death) count of 222. To achieve this figure in terms of trial design, a simple calculation was employed: a prediction of 37 ovarian cancer deaths per 100,000 women per year based on national ovarian cancer mortality statistics in women aged ≥ 50 years. Six years of annual screening plus 1 year of additional follow-up, with an annual 4% ‘attrition rate’, was deemed adequate total analysis time to accumulate the required 222 events. 6

At a median follow-up of 5 years from randomisation, it was apparent that there were significantly fewer events in the control arm than originally estimated. 5 Our assumption, widely used, that mortality would be similar to the Office for National Statistics (ONS) data was one source of potential error. We, like many others, did not consider the impact of the eligibility criteria effect (ECE), which relates to the fact that all cancer screening and prevention trials will exclude those previously diagnosed with the cancer under study as well as those actively undergoing treatment for other cancers. In screening trials, this results in cancer-specific deaths occurring only in those diagnosed after randomisation. In UKCTOCS, in the control group, we estimated 37 ovarian cancer deaths per year. However, the total number of deaths in the first 2 years was only 18, as events were limited to deaths within 2 years of clinical diagnosis of ovarian cancer. When we revised our event rate calculation, we made an adjustment for the ECE, using external data on age- and period-relevant survival rates from time of clinical diagnosis.

Event rate calculations in the control arm were again needed for planning extended follow-up at the end of 2014 as our mortality analysis did not provide definitive results. We targeted an additional 233 control group ovarian cancer deaths, but on this occasion did not use predictions based on external data. 7 Rather, we used ovarian cancer mortality data from within our trial over the initial 13.6-year period for extrapolation using flexible parametric survival models, with adjustment for all-cause mortality. Although the models themselves can fit data to an arbitrarily close degree, as events are predicted beyond the range of the data, the extrapolation is close to a simple linear extension. It is not guaranteed that this method will prove more accurate than those based on external data as one cannot be sure that past trends will continue into the future. As it turned out, during regular event rate monitoring during extended follow-up, our initial calculations extrapolating past data proved a moderate overestimate, and the date of trial end was delayed by over a year from 31 December 2018 to 30 June 2020.

In addition to the above, there was an healthy volunteer effect (HVE). Those who tend to volunteer for health intervention studies are likely to be more health orientated and consequently less likely to succumb to the disease of interest compared to the national average rate. 8,9 As a result, the incidence and mortality rates of many cancers and major diseases were considerably less than those reported in the national statistics. 5

To maintain power and have the requisite number of events, like most, we decided it was more productive (and logistically simpler) to increase the follow-up time of randomised trial participants rather than recruit additional volunteers. Instead of each participant having an individual end date and a total of 7 years’ follow-up from randomisation, we extended follow-up with a common end date (31 December 2014) for all participants. As recruitment occurred between 2001 and 2005, this resulted in each volunteer having a different total follow-up time, giving the added complication of individual event rate predictions. To future-proof the revised calculation, we included not just individual age-adjusted risks but also the risk increase associated with ageing and the predicted losses to follow-up from other-cause mortality (i.e. excluding ovarian cancer). We therefore extended annual screening to the end of 2011 to ensure that there was minimal dilution of the screening effect. This resulted in screen arm participants having a maximum of 7–11 annual screens, depending on when they were recruited (2001–5), instead of the originally planned six annual screens.

Our experience highlights a common error of assuming national event rates when calculating target sample size in a clinical trial. Such an approach often leads to a significant shortfall of primary outcome events. This is likely magnified in early detection or prevention trials, where the eligibility criteria and the nature of the intervention being evaluated skew participation in favour of those who adopt healthy lifestyle behaviours such as exercise and balanced diet. The cause-specific standardised mortality/incident ratios of cancers and other diseases in our publication5 provide useful guidance when estimating the likely refactoring of national statistics-based calculations that is required in order to provide a more realistic event rate that accounts for a HVE.

An interesting final note was that at the end of the trial (median follow-up 16.3 years), it was apparent from the trend of the standardised mortality ratios for ovarian cancer over time that there was no HVE with regard to ovarian cancer. Ovarian cancer incidence in the control arm was similar to, if not higher than, national figures available at the start of the trial, and the mortality was as expected after years 6–8 post randomisation. It may be surmised that the initial shortfall, discounting the simplistic early calculations, was solely due to the ECE. This may not be surprising in the particular case of ovarian cancer, because UKCTOCS participants were found to be less socially deprived (lower Index of Multiple Deprivation scores) compared with those invited. 5 It is well established that ovarian cancer, unlike most others, is associated with more affluent populations. 7,10 By contrast, a general population screening trial in lung cancer, for example, should certainly factor for a likely HVE in event rate calculation.

It is important to note that even with the best possible assumptions, the predictions may differ greatly from the eventual outcome, and regular monitoring of control events and recalibrations of those predictions is highly advisable.

Primary analysis approach

The method of the primary analysis chosen for a cancer screening trial is a crucial part of the trial planning process. It is important to note firstly that over the last two decades the mortality results from cancer screening trials have repeatedly shown that when there is a positive screening effect, it appears after a period of delay,11–15 even up to 7 years post trial randomisation. 16 This is consistent with how screening aims to prevent deaths by detecting cancers early, before they reach an incurable state. The delay reflects the time required for participants in the control group to develop symptomatic disease and subsequently die of it, in substantial numbers. Additionally, in the early years post randomisation, opportunity to detect these cancers sufficiently early in their natural history is limited and, as a result, screening has less impact in preventing cancer deaths. 17,18

There are several aspects one may wish to consider when developing the (primary) analysis plan, and those elements will not all necessarily be in concordance. An obvious goal is to maximise statistical power. This can be done by adopting a statistical test that accommodates the late effects typical of cancer screening, either through giving higher weighting to later events or by specifically modelling the non-proportional hazard functions of the screen and control arms. However, screening trials have historically used simple methods, usually a single Poisson-based rate ratio11,14,16,19–21 or a Cox model with a single hazard ratio (HR)12,13 to estimate the mortality benefit. These methods will give a clear, easy to interpret result – a key analytical consideration, which is presumably why they are popular. However, the estimated screening effect indicated by a HR could be a gross simplification of the true nature of the screening trial. Reduction to a single summary estimate will provide a misleading impression of the impact of the screening regimen being evaluated, both on the thousands of participants screened in the trial and on what may be expected in the future.

Our experiences from UKCTOCS show the difficulties in reconciling the opposing forces of simplicity and ease of interpretability versus reflecting reality and a more relevant interpretation, with the added complication of statistical power. Our trial, like many others, initially used a proportional hazards Cox model for our primary analysis, and the first reporting of mortality data gave a non-significant result that suggested a late separation of the mortality curves at possibly 7 years post randomisation. 22 With further follow-up, and all subsequent events occurring in that post-separation period, it was hoped that the extended follow-up analysis might provide an unambiguous positive result. A lengthy period of introspection and debate within the trial management group and trial steering committee regarding the appropriate choice of primary analysis for long-term follow-up ensued. We ultimately dealt with this dilemma by formally seeking the counsel of 12 statistical and screening experts, the majority of whom gave backing to methods that reflected the non-proportionality expected in successful screening trials. 23 More generally, they also supported the concept of the pragmatic evidential approach, which challenges the fixed pre-specification viewpoint that would have prevented this alteration to the primary analysis.

That the long-term results for UKCTOCS unfortunately showed no discernible screening benefit should not affect the relevance and importance of these considerations. That said, there is naturally a reputational risk inherent in changing the statistical analysis plan after a portion of the data has been unblinded. Hence we would strongly advise that in the planning phase of a cancer screening trial, the extent of the projected screening effect delay be carefully deliberated. One should use current understanding of the natural history of the cancer of interest and other relevant factors to try and predict, as much as feasible, the likely time interval to mortality curve separation, as this will influence the most appropriate modelling choice. This applies in terms of both interpretation and power. The longer the delay in screening benefit, the greater statistical benefit to abandoning the proportional hazards approach. However, when the delay is modest, such as 3 years or less, it should also be cautioned that a late-weighted model will not necessarily give greater power, especially when total events will be low in the early years. In fact, if the final data suggest a result not dissimilar to proportional hazards, then the penalty paid in terms of power may even be greater than that for choosing proportional hazards in a scenario with more obvious late effects. 24,25 For that reason, one may consider a method for the primary analysis such as the Versatile test26 which will provide high, if not optimal, statistical power in all potential scenarios, obviating the need for accurate prediction of relative hazard functions. More complicated modelling techniques may also be utilised for estimation purposes.

Appropriate secondary analyses of primary outcome

As has been described already, it is crucial that the primary analysis is carefully considered and should reflect prior beliefs about the anticipated screening effect. It is also important that a similar level of prior thought is given to the secondary analyses of the primary outcome detailed in the statistical analysis plan. There is a tendency to list a number of additional analyses of the primary outcome, in an attempt to cover many bases. It is also tempting to deviate from the secondary analyses of the primary outcome detailed in the statistical analysis plan as new information and ideas come to light. Given we changed the primary analysis after extended follow-up, we do not suggest that aspects of secondary analyses should not be changed. However, it needs to be noted that excessive secondary analyses have the potential both to distract from the primary analysis and to impact on the message produced at the time of the final mortality analysis. Our first mortality analyses22 probably suffered from describing too many results, which proved hard for many readers to absorb. The prior-defined primary analysis was perhaps also compromised by the inclusion of a complicated secondary analysis as well as a post hoc analysis. We would therefore strongly urge that it is important to reconsider the inclusion of secondary analyses of the primary outcome that differ only slightly from the primary. Certainly, additional analyses that require significant description of methodology are probably best not included in the publication reporting the primary outcome of the trial. Indeed, important other analyses are best considered for a separate publication instead, to give them the breadth the data deserve.

Set-up

The trial was set up to be delivered through 13 sites [regional centres (RCs)]. The number was chosen to ensure that there was a balance between the distance women would have to travel to participate in the trial, the number of centres which the trial coordinating centre (CC) could support and monitor to a high standard and the efficient use of the pelvic ultrasound scanning machine which was purchased for each RC. Similar set-up has been described where the same prototype device has been used in triage of patients with cervical abnormalities on colposcopy or cytology in a smaller multicentre trial. 27 Each UKCTOCS RC had a dedicated trial team and ring-fenced trial space. Significant efforts were made to ensure that CC oversight of site teams released the site clinical principal investigators (PIs) from much of the workload associated with day-to-day running of the trial. The latter has since been found to be a key driver of successful trial recruitment in the Strategies for Trial Enrollment and Participation study. 28

Recruitment

As in all trials, at the start, the key challenge was timely recruitment. 33–35 This is magnified in general population screening and prevention trials which require large numbers of participants. UKCTOCS exceeded its target recruitment of 200,000 participants. During the recruitment phase (April 2001 to September 2005), the weekly average recruitment rate per centre was 96% (range 87–114%) of the weekly target of 100 women. Of note, this was accomplished with only 0.008% (98) of invited women complaining about recruitment-related problems: invitation to trial (32), trial information (28), recruitment appointment (17) and randomisation to control group (21). 36

The key factors responsible for our success are detailed below.

1. Use of age–sex PCT registers to invite women from the general population – The use of the electronic health records to identify women aged 50–74 years living in the vicinity of the 13 trial centres was crucial to ensuring that each centre met its monthly recruitment targets and that the allocated resources were maximally utilised. We obtained approval from 25 of the 27 individual PCTs to access contact details of women aged 50–74 years and their GPs. Specialised software that we commissioned from the NHS flagged the women whose details were shared on the PCT registers so that they were not invited again. Over the course of the trial, we invited over 1.2 million (one in six eligible UK women aged 50–74 years), of whom only 32 complained about being contacted without specific consent.

In 2000, when the trial was planned, such processes were novel and bespoke. To address the significant recruitment challenges faced by researchers, this process has since been significantly streamlined through the set-up of patient registers such as the Scottish Health Research Register of those who have agreed for their clinical records to be used for research. Similarly, NHS Digital in England via its DigiTrials initiative is facilitating identification of potential trial participants. This includes the recent NHS Galleri Trial that has recruited 140,000 individuals to evaluate a new blood test for early detection of cancer. 37 In keeping with the current legal and research governance framework which prevents researchers from making direct contact with patients, invitations and information are sent via the clinician responsible for the care of the patient. Automated downloads from the NHS DigiTrials platform in the future for approved projects should further streamline this process. Routinely collected health-care data (RCHD) are also being accessed to determine the feasibility of RCTs. 38 An alternative is to use media advertisement alone, but this is more challenging. Two projects which have successfully recruited participants using the latter mode are UK Biobank39 and the Scottish Diabetes Research Network. 40

The PCT lists enabled automated mass mailing of invitations from the CC using the TMS. An average of 2159 invitation letters were sent out every week during the recruitment phase. Key advantages of this approach included:

1. Personalised invitations – Most women were sent an invitation letter addressed to them personally. This was not possible for those resident in two PCTs which did not agree to share contact details and sent out standard letters instead. While there was no significant difference in acceptance rates between these two PCTs and the 25 which enabled sharing of data in UKCTOCS,36 on a somewhat related issue, a recent Cochrane Review of 24 studies on the uptake of cervical screening found that personalised invitations (targeted letters) rather than mass mailing appeared to be more effective. 41 On acceptance, we sent the women a fixed appointment to attend for recruitment. The latter review also found that women sent fixed appointments were more likely to take up the offer of screening. 41

2. Efficient flagging of recruited women for follow-up using NHS registers – inclusion of the NHS numbers in the shared contact details meant that we did not have to request these from either the women or their GPs. As a result, the data were complete and accurate, and we were able to efficiently link to electronic health records (cancer and death registers and routine hospital administrative data) for follow-up. Only 6 out of the 202,546 randomised eligible women could not be linked. 22

3. Informing GPs of all women invited for the trial – On receiving PCT lists of potential participants, automated processes in the TMS built a trial GP contact table and sent out a personal letter and an information pack to all GPs, who were only contacted once. Perhaps in the future, this could be done by e-mail in keeping with the need for ‘greener’ trials. Women were then sent invitations a minimum of a week later after their GP had been apprised of the trial. This ensured that no GP was unaware of the trial if their patient mentioned that she had been invited.

4. Ensuring recruitment resources were efficiently used – We arranged for 3-monthly electronic transfer of lists of 5000–10,000 women to the CC. The women were then invited to the trial. This approach minimised the number of invitations being sent to women who had died or moved away. Regular monitoring of acceptance rates helped establish the frequency and volume of mailing needed to ensure recruitment clinic appointments were full for the following 3 months. It also meant that women who accepted an invitation did not have to wait over 3 months for an appointment. Such a streamlined, partially automated approach safeguarded against delays and gaps in recruitment which would have had significant cost implications.

1. Providing high-quality standardised patient information throughout recruitment – This is a key requirement for all trials and is usually delivered through use of a standard PIS that has been approved by the overseeing ethics committee. We were aware that many patients did not read this detailed document. For us, additional challenges were the large number of participants, a recruitment period spanning 4.5 years, the multicentre set-up, individual research nurses recruiting an average of 100 women a week, and potential staff turnover. To address this, we developed further processes that included (1) an information video42–44 using a professional team and participants from our pilot screening trial. Such transmedia tools45 helped ensure a low-cost, high-quality, easily implementable solution. 36 The women watched the video at the recruitment appointment in groups of six to eight. (2) A group discussion led by the trial nurse that would facilitate the more reticent to ask questions and hear others broach issues that viewing the video might have raised. (3) A one-to-one discussion with the research nurse that provided an opportunity to discuss any private/personal concerns. The use of an information video and group discussion during recruitment was a novel feature in 2000. The Sussex Health Outcomes Research and Education in Cancer team who helped develop the UKCTOCS video has a long track record in producing excellent patient information videos. Some of the examples can be found here: https://shore-c.sussex.ac.uk/trial-dvds.html. Such an approach is being increasingly streamlined nowadays by incorporating podcasts that potential recruits can view prior to their recruitment appointment on trial websites or YouTube. 46 It is important to work with a professional team and involve trial participants and the lay public in producing such videos.

2. Proactive management of recruitment – The start of recruitment was phased across the 13 RCs. It was only started once adequate numbers of women had accepted the invitation, to ensure that recruitment appointments for 3 months were full at RC recruitment start. There was close monitoring of attendance at each of the recruitment clinics. Soon patterns emerged that were seasonal and sometimes RC-specific. For example, there was poor attendance during the summer holidays and during winter afternoons as it became dark earlier. To address this, we altered the clinic timings and overbooked scheduled appointment slots to ensure that the target of 100 per week was met as far as possible. When centres fell behind, additional large ad hoc ‘blitz’ clinics were set up, often on the weekend. These were a collaborative team effort, with CC staff travelling to the RC to assist local teams and senior staff including PIs joining in. They were supported using contingency funding, ring-fenced for such efforts.

The functionality we built into the TMS and the provision for participants to contact either the RC or CC administrative staff to reschedule the appointments led to more efficient use of resources. Overall, the TMS scheduled 1,109,351 trial appointments, of which 114,548 (10%) were rescheduled, in the main at the request of the participant. Overall, 8% (9149/114,548) of these rescheduled appointments were not attended, compared to 19% (186,098/994,803) of appointments that were not rescheduled.

Throughout the trial, there were monthly league tables of RC recruitment and vouchers for the winning team. There was a strong sense of camaraderie between all teams and a determination to make all aspects of the trial a success.

1. Engaging with patient advocacy charities and media – The uptake of invitations is impacted by individuals’ awareness of the trial and understanding of the relevance of the intervention being evaluated. From the start, we built a trial website which was regularly updated. To ensure that ovarian cancer screening and the trial were in the public domain, we worked closely with cancer (and particularly ovarian cancer) patient advocacy groups/charities and engaged with national and local media (TV, radio and newspapers). Trial investigators spoke regularly at the charity meetings and wrote articles for charity newsletters to raise awareness of the need for screening and to provide updates on trial recruitment and overall progress. We noted surges in uptake of invitations in response to some of the national media efforts. However, we did not collect data to measure impact. In the future, it would be worthwhile to consider a study within a trial (SWAT)47 to prospectively assess the impact of such interactions on recruitment rates. Some of the published SWATs to date exploring other recruitment strategies,48 such as use of optimised information sheets,49 advertising patient and public involvement (PPI)50 to potential trial participants, pre-notification of trial detail,51 use of Post-It^® Notes (3M, Saint Paul, MN, USA) in older patients,52 and impact of envelope colour53 did not find significant impact on recruitment rates.

In addition to television and radio, since 2005 (end of UKCTOCS recruitment), a number of other digital tools – e-mail, social media54 platforms such as Facebook (2006), Instagram, Twitter, data mining, and internet sites – have become increasingly available to create awareness and increase recruitment. 44 Most individuals, including older women such as those invited to UKCTOCS, have a mobile phone and an e-mail address. Social media engagement is much more prominent as a result of technological development and increasing acceptance. While no single platform will suit every study or every patient, the use of a combination of traditional (PIS, study brochures, radio and television advertisements) and social media tools is likely to increase successful recruitment. 54

There was one area where we did not do well. The trial did not have any ring-fenced resources for efforts to engage with minority or ‘hard-to-reach’ populations. All the invitations were in English. Given the age group we were targeting (50–74 years), the cohort is likely to have included women from minority populations who were unable to understand English. We did set up a facility for relatives of potential participants to ring and request a translator who would then explain the invitation content. However, this was seldom used, most likely as it required someone in the household to read the invitation we sent in English and contact us on behalf of the potential participant. There was not much use of the translating service for recruitment appointments either. We had hoped to set up a series of discussions on local minority-language radio stations but were unable to prioritise this. Overall, 96% of our participants were white, with rates varying from 97.9% to 99.4% in 10 of our 13 RCs. It was only in the two London RCs that we had significant minority participation, with over one in five from non-white communities (see Appendix, Table 1). When it came to the disabled community, too, our efforts were limited. We had a ticker tape (but not sign language) included in our video, which helped those with hearing impairment to some extent. We provided special appointments where individuals could request to view the video at their own pace with printed transcripts. Increasing minority participation has become a key aspect of trials over time. There are now recruitment strategies tailor-made to the participants’ background. 54

Delivery of screening

We evaluated two screening tests: one involving a blood test for CA125 and the other using imaging (transvaginal ultrasound).

The blood test was operationally easier to deliver. We set up dedicated venepuncture clinics, which we were able to staff relatively easily. The biomarker CA125 is very stable and we were therefore able to set up daily transfers of the blood tubes at room temperature from the RC to the central laboratory where all the samples were processed. CA125 is a well-established cancer biomarker, routinely assayed in the NHS laboratories using automated platforms. Centralisation of CA125 analysis was required to ensure close monitoring of assay imprecision for the duration of the trial. In routine clinical use, imprecision is only monitored near the upper limit of the reference range (35 kU/l) and at an elevated level of about 120 kU/l. However, in UKCTOCS, CA125 was not interpreted using clinical cut-offs but a longitudinal algorithm [Risk of Ovarian Cancer Algorithm (ROCA)] which incorporates serial changes in CA125 and therefore required precision across the whole range of CA125 levels. 55 Most importantly, in the majority of the participants (older postmenopausal women), the normal CA125 levels were < 10 kU/l. Small changes in these levels could impact on the ROCA risk estimation. As there is no commercially available quality control (QC) sample available at such levels, we developed additional QC verified samples with concentrations in the range of 8–12 kU/l. We used serum samples and QCs to assess the effect of changes in the manufacturer’s calibrator and reagent lots ahead of implementation and reserved at least 1 year’s supply of the same lot number to minimise the frequency of change, a practice not available to routine clinical laboratories. Changes in calibrator, reagent and QC lot numbers were staggered to be at least 1 month apart, and the impact on ROCA intermediate and elevated risk rates was monitored daily for unexpected increases or decreases. After one reagent lot change was associated with an increase in these rates, we determined this was due to a positive constant error in the assay and thereafter included serum samples with very low (< 2 kU/l) CA125 values in our performance monitoring. With increasing focus on longitudinal biomarker algorithms for early detection, it is critical that trialists pay close attention to the inter- and intra-assay variability of the novel biomarkers being evaluated.

The transvaginal ultrasound was logistically much more challenging to deliver. Like other imaging modalities, it could not be centralised. Participants attended dedicated trial ultrasound clinics at the 13 RCs. Staffing these clinics for 11 years was very difficult given the shortage of trained staff and the need within the NHS to deliver the clinical service. We addressed this by organising clinics out of hours and on weekends so that NHS staff could perform scans. We also set up our own staff agency to enable ultrasonographers to work on a sessional basis in our scan clinics. We had to pay premium rates which were outside the NHS pay scale.

While pelvic scanning was widely available, most sonographers were used to scanning symptomatic patients with a high prevalence of clinical pelvic masses. We had to develop specific training for scanning of postmenopausal adnexa, an accreditation programme56 with revalidation and quality assurance monitoring using ovarian visualisation rates. 56 These efforts required significant resources. In the future, the use of artificial intelligence algorithms for both ensuring quality and also classifying results is likely to make a significant impact on screening using imaging.

Retention and compliance with screening

From the start, we regularly emphasised to all trial staff the importance of retention if our successful recruitment efforts were to translate into an impactful trial. We were conscious that our resources to contact all participants individually were limited. We therefore made it a priority to ensure they could easily contact us. There was a dedicated trial phone line and a receptionist who manned it during working hours, at both the CC and each of the RCs. Participants were also able to contact the CC through a dedicated e-mail address that continues to be active even now after the end of the trial (ukctocs@ucl.ac.uk). During the trial, these means of communication were used mainly for rescheduling appointments and querying screening results. A key responsibility of trial teams was to ensure that participant messages on the answer phone or queries were answered within the next working day and that any discussions were noted on the electronic participant records. The latter ensured that conversations were seamless independent of who answered the phone, and participants did not have to repeat any relevant issues. Each opportunity was used to ensure that they understood that they were central to our efforts and their concerns/queries were being taken seriously. Additionally, the co-chief investigator reviewed any complaints raised by participants, and responded within 48 hours. No issue was too trivial, and all suggestions that might improve the participant journey were evaluated and implemented if feasible.

We also maintained engagement through two planned postal questionnaires, one sent 3–5 years post randomisation and the second in April 2014. Additionally, we were able to obtain separate funding to send a newsletter about 7 years post trial start and a ‘thank you’ card after the end of screening (details below) to all women. Regular trial update posters were displayed in the UKCTOCS clinics (phlebotomy or ultrasound scan clinics). This was, however, only available to women from the screening groups, who attended annually; we were less able to engage with those in the no screening (control) group. The challenge of carrying out a trial of the scale of UKCTOCS in 2001–5 was that many women did not have mobile phones or e-mail addresses. All the newsletters had to be printed/posted, at a significant cost. The mailing costs (over £50,000) were too high to sustain the annual newsletters which would have been our preference. Over the last 20 years, the ever-increasing use of the internet and widespread uptake of e-mail, including by older populations, now makes this a feasible, cheap option. Trial teams can even create study apps to engage participants throughout the trial. 54 We included trial updates in ovarian cancer charity newsletters as we were aware that a proportion of our participants accessed these. Where possible, we reinforced the message that we wished to communicate more regularly but were restricted by lack of funds. A recent systematic map has found that of all the methods currently used to maintain engagement with trial participants (social media, internet sites, e-mail and TV/radio), the most effective are e-mail and text messaging. 44

All of our efforts resulted in high rates of overall retention and compliance22 with screening. Among women eligible to undergo screening, the overall compliance with attending for the entire annual screening episode, including repeat screens if required, was 81% in the MMS and 78% in the USS arms. 22,38 This also reflects the commitment of the participants to contribute to research despite the length of the trial, the demands of annual attendance at screens or completion of questionnaires, and any possible psychological burden. A systematic review of 45 qualitative studies involving 1732 RCT participants reported that the effects included anxiety, the notion of ‘being a guinea pig’, and finding being randomised to the control arm difficult, which resulted in disappointment, anger and depression. Furthermore, they reported a range of psychological, physical, and financial burdens related to follow-up and trial closure. Poorly organised trials and follow-up that was very demanding were cited as some of the factors that affected participation. On the other hand, the trial participants reported taking part with satisfaction and the feeling of being ‘useful’, gaining ‘a sense of control’, and receiving special attention. 57

Follow-up

Access to routinely collected health data (RHCD) was an essential aspect of follow-up. As the primary end point was ovarian cancer mortality, women were linked using their NHS number to national cancer and death registration and hospital episode statistics (HES)/Patient Episode Database for Wales (PEDW) data. Written consent was obtained at recruitment. We had to revise the PIS to include specific mention of the ONS so that participants were aware that identifiable data would be shared with the organisation for the purposes of flagging. During the course of the trial, the ONS transferred this function to NHS Digital (formerly the Health and Social Care Information Centre, and prior to that the NHS Information Centre), which subsequently supplied HES data. Until 2014, the legal basis of data-sharing was consent. This included the 2010 Data-Sharing Agreement (DSA) with NHS Digital. However, in 2014, they declined to renew the DSA, stating that the consent was outdated. Reconsenting participants is usually only necessary where the original consent was flawed or there are significant changes in the research procedures, risks, potential benefits, or alternatives. 58 Given the large number of participants, it was not possible to reconsent or reaffirm consent. We therefore had to apply to the Health Research Authority Confidentiality Advisory Group for approval under Regulation 5 of the Health Service (Control of Patient Information) Regulations 2002 to process patient-identifiable information without consent (Section 251). Approval was granted in April 2015. The NHS Digital processes seemed to be at odds with the participants’ consent and wishes and the ethically approved scientific objectives of the project. This was supported by the fact that for the DSA renewal with NHS Wales Information Service, the chair of the Privacy Advisory Committee of Northern Ireland determined even in 2020 that the original express consent arrangements were a sufficient legal and ethical justification for disclosure of the information in question (patient-identifiable data) for the stated purpose of the trial. There have since been significant efforts by the Research Advisory Group within NHS Digital to advance the use and effectiveness of NHS Digital’s data and services for the research community. Future trialists working in partnership with NHS Digital and NHS DigiTrials may be able to avoid the challenges we faced.

The recruitment and screening phases of UKCTOCS pre-dated the introduction of the EU General Data Protection Regulation (GDPR), which came into effect in the UK on 25 May 2018. In order to comply with GDPR requirements, we published a privacy notice on our trial website. 59 In line with GDPR, NHS Digital provided English and Welsh individuals the ability to opt out of allowing their personal information to be shared outside of NHS Digital for purposes beyond the individual’s direct care (type 2 opt-out). Although we had explicit written consent which overrides type 2 opt-out, NHS Digital did not accept our participants’ written consent as they felt it was not sufficiently current. Additionally, they stated that our PIS did not contain sufficient information detailing the use of personal identifiable data in matching to NHS Digital records. This meant that we had to obtain Section 251 approval for the legal basis for accessing NHS Digital data on UKCTOCS participants. NHS Digital did not share medical data of individuals who had opted out as per the terms of Section 251.

There is an increasing use of RCHD to inform or supplement outcome data. 60,61 However, there is still no consensus on whether RCHD can be relied upon as the sole data source to determine outcome. One issue is that some participants will not appear on RCHD if they exit the registry due to immigration, imprisonment or military service. We therefore used additional data sources (three follow-up postal questionnaires, direct information from trial participants and their relatives and from RC staff as well as attendance at screening clinics) both to determine status (alive or dead) at censorship and to identify women who may have developed ovarian cancer. With regard to the latter, to ensure we cast as wide a net as feasible, we searched the RCHD data not only for International Statistical Classification of Diseases and Related Health Problems, Tenth Revision (ICD-10) codes for ovarian, tubal and primary peritoneal cancer but also for 16 other allied ICD-10 codes, including C80, disseminated malignancy of unknown origin. Postal follow-up at the end of the trial in 2020 helped us ensure we had as complete data as was possible. To be cost-effective, we limited mailing of this last questionnaire to 6586 participants on whom we had incomplete follow-up for a variety of reasons, such as difficulty ascertaining from HES data whether both ovaries had been removed, exiting from national registries etc. Even 20 years after trial start, we had a reasonable response rate of 38% (2493/6586) – in some cases, family members responded as the women had died.

As a result primarily of RCHD linkage, supplemented by the other sources listed above, we had complete follow-up for 98.9% of participants at our initial censorship (31 December 2014) and 95% at our final censorship (30 June 2020). 6 We have previously reported on the detailed contribution of these data sources towards ascertaining our primary outcome. 22 If we had only used RCHD data, we would have still been able to ascertain the majority of outcomes. Most important, sensitivity analyses determined that the results of the trial would not have changed if primary outcome events were restricted to those with either death or cancer registration. 22 Given the latter finding and the costs of mailing follow-up questionnaires to over 200,000 participants, at scientific review of our proposal for extended follow-up, we were asked to omit the latter and limit follow-up to RCHD linkage. Nowadays there is less need to make such a choice based on costs due to the current ease of texting or e-mailing a survey link to participants. A more critical element in decision-making should be to weigh the pros and cons of the additional details that such questionnaires can provide for secondary and subgroup analyses. However, this must be balanced against the significant improvements in data quality of RCHD.