Robotic-assisted surgery compared with laparoscopic resection surgery for rectal cancer: the ROLARR RCT

Article history

The research reported in this issue of the journal was funded by the EME programme as project number 08/52/01. The contractual start date was in March 2010. The final report began editorial review in March 2018 and was accepted for publication in March 2019. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The EME editors and production house have tried to ensure the accuracy of the authors’ report and would like to thank the reviewers for their constructive comments on the final report document. However, they do not accept liability for damages or losses arising from material published in this report.

Declared competing interests of authors

Alessio Pigazzi is a consultant and proctor for Intuitive Surgical Inc. (Sunnyvale, CA, USA) and receives personal fees from Covidien plc (Medtronic plc; Dublin, Ireland) and Ethicon, Inc. (Somerville, NJ, USA) outside the submitted work. David Jayne is a proctor for Intuitive Surgical Inc. and was formerly a member of the Efficacy and Mechanism Evaluation (EME) Strategy Group and the EME Prioritisation Group, and was previously involved in an EME Intraoperative Imaging Review. Claire Hulme was formerly a member of the National Institute for Health Research (NIHR) Health Technology Assessment (HTA) Commissioning Board. Julia Brown is a member of the HTA Clinical Evaluation and Trials Board, HTA Funding Board Policy Group, HTA Mental, Psychological and Occupational Health Methods Group, HTA Post-Board Funding Teleconference Group and NIHR Standing Advisory Committee.

Copyright statement

© Queen’s Printer and Controller of HMSO 2019. This work was produced by Jayne et al. under the terms of a commissioning contract issued by the Secretary of State for Health and Social Care. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science Park, Southampton SO16 7NS, UK.

2019 Queen’s Printer and Controller of HMSO

Objectives

The purpose of the trial was to perform a rigorous evaluation of robotic-assisted rectal cancer surgery by means of a randomised controlled trial. The chosen comparator was standard laparoscopic rectal cancer resection, which is essentially the same procedure but without the use of the robotic device. The two operative interventions were evaluated for short- and longer-term outcomes. The key short-term outcomes included assessment of technical ease of the operation, as determined by the clinical indicator of low conversion rate to open operation, and clear pathological resection margins as an indicator of surgical accuracy and improved oncological outcome. In addition, quality-of-life (QoL) assessment and analysis of cost-effectiveness were performed to aid evidence-based knowledge to inform the NHS and other service providers and decision-makers. The short-term outcomes were analysed after the last randomised patient had had 6 months of follow-up, to provide a timely assessment of the new technology, and were made available to the public, clinicians and health-care providers to inform health-care decision-making. Longer-term outcomes concentrated on oncological aspects of the disease and its surgical treatment with analysis of DFS, OS and local recurrence rates at 3 years’ follow-up.

Trial design

The ROLARR trial was an international, multicentre, prospective, unblinded, parallel-group randomised controlled trial22 comparing robotic-assisted with laparoscopic surgery for the curative treatment of rectal cancer (defined as an adenocarcinoma whose distal extent was situated at or within 15 cm of the anal margin) by low anterior resection (LAR), high anterior resection (HAR) or abdominoperineal resection (APR). The trial design required that each participating surgeon had performed a minimum of 30 minimally invasive (laparoscopic or robotic) rectal cancer resections (at least 10 laparoscopic and at least 10 robotic). The trial received national ethics approval in the UK and either ethics committee or institutional review board (IRB) approval as was required at the location of each of the international centres; all participants gave written informed consent. The trial conduct was overseen by an independent Trial Steering Committee (TSC) and Data Monitoring and Ethics Committee (DMEC). The trial was registered on the International Standard Randomised Controlled Trial Number (ISRCTN) register (ISRCTN80500123).

Participants

The inclusion criteria were:

1. Aged ≥ 18 years.

2. Able to provide written informed consent.

3. Diagnosis of rectal cancer (defined as an adenocarcinoma for which distal extent is situated at or within 15 cm of the anal margin, as assessed by endoscopic examination or radiological contrast study) amenable to curative surgery by LAR, HAR or APR, for example, staged T1–3, N0–2, M0 by imaging as per local practice. Although not mandated, computed tomography (CT) imaging with either additional magnetic resonance imaging (MRI) or transrectal ultrasound is recommended to assess distant and local disease.

4. Rectal cancer suitable for resection by either standard laparoscopic procedure or robotic-assisted laparoscopic procedure.

5. Fit for robotic-assisted or standard laparoscopic rectal resection.

6. An American Society of Anesthesiologists (ASA) physical status of ≤ 3.

7. Capable of completing required questionnaires at time of consent (provided questionnaires were available in a language spoken fluently by the participant).

The exclusion criteria were:

1. benign lesions of the rectum

2. benign or malignant diseases of the anal canal

3. locally advanced cancers not amenable to curative surgery

4. locally advanced cancers requiring en bloc multivisceral resection

5. synchronous colorectal tumours requiring multisegment surgical resection (a benign lesion within the resection field in addition to the main cancer would not exclude a patient)

6. coexistent inflammatory bowel disease

7. clinical or radiological evidence of metastatic spread

8. concurrent or previous diagnosis of invasive cancer within 5 years that could confuse diagnosis (non-melanomatous skin cancer or superficial bladder cancer treated with curative intent were acceptable; other cases were individually discussed with the chief investigator)

9. history of psychiatric or addictive disorder or other medical condition that in the opinion of the investigator would preclude the patient from meeting the trial requirements

10. pregnancy

11. participation in another rectal cancer clinical trial relating to surgical technique.

Preoperative investigation and preparation was as per institutional protocol. Laparoscopic mesorectal resection was performed in accordance with each surgeon’s usual practice. Robotic surgery involved either a totally robotic approach or a hybrid approach; the only absolute requirement was that the robot had to be used for mesorectal resection. For the purposes of the trial, a totally robotic operation was defined as a resection of the entire surgical specimen with the use of robotic assistance. A hybrid operation was defined as use of laparoscopic techniques to mobilise the proximal colon, with robotic assistance employed to perform the rectal mesorectal dissection. It was permissible to perform a partial mesorectal excision with a suitable distal margin, rather than a TME.

The specifics of each operation were at the discretion of the operating surgeon (e.g. port-site placement, mobilisation of the splenic flexure, inferior mesenteric artery/vein division, high vs. low vascular division, etc.), as was the decision to convert to an open operation. Detailed guidance was provided to ensure consistent histopathological analysis and reporting of the rectal dissection specimens in accordance with internationally agreed criteria. 23 Digital photographs of the anterior and posterior of the specimen and sequential cross-sectional views of the surgical specimen, as well as close-ups of the front and back of the levator/anal sphincter (if appropriate), were collected (prior to dissection) to allow blinded assessment of the quality of the plane of surgery. To enable a central pathology review, the tissue slides (or high-quality digital slide images) were submitted.

Postoperative care was as per institutional protocol; however, the protocol required that patients underwent a clinical assessment at 30 days and at 6 months post operation. Any further visits were in accordance with local standard clinical practice. Follow-up data were collected on an annual basis until the last participant reached 3 years post randomisation.

Participants completed questionnaires prior to randomisation (baseline), and at 30 days and 6 months postoperatively. General QoL [Short Form questionnaire-36 items version 2 (SF-36v2)] and fatigue [Multidimensional Fatigue Inventory-20 (MFI-20)] data were collected at baseline and at the 30-day and 6-month postoperative visits. In addition, bladder and sexual function questionnaires [International Prostatic Symptom Score (I-PSS) and International Index of Erectile Function/Female Sexual Function Index (IIEF/FSFI)] were completed by patients at baseline and at 6 months post operation. Participants in the UK and USA also completed the EuroQol-5 Dimensions (EQ-5D) at baseline and at 30 days and at 6 months post operation, and a resource utilisation questionnaire at 30 days and at 6 months post operation for the health economic component of the trial.

The SF-36v2,24 a well-validated, multipurpose standard health-related QoL evaluation questionnaire, was used to assess generic QoL. It generates an eight-scale profile of functional health and well-being scores, as well as summary measures of physical and mental health. This information related to the previous 4-week time period.

The MFI-20 was used to assess fatigue;25 it is a 20-item self-report validated instrument designed to measure current fatigue. It creates a global score as well as individual scale scores that cover the following dimensions: general fatigue, physical fatigue, reduced activity, reduced motivation and mental fatigue.

The I-PSS26 was used to assess bladder function. This questionnaire includes seven questions relating to lower urinary tract function, which form an overall symptom score that can be used to classify bladder dysfunction as mild, moderate or severe. To assess sexual function, the IIEF27 and FSFI28 were used. Both are brief male-/female-specific questionnaires developed to assess various domains of sexual function. The IIEF, FSFI and I-PSS questionnaires obtained information relating to the patient’s functioning over the previous 4 weeks.

For the health economic analysis, the EQ-5D questionnaire was used to assess self-reported utility. This is a standardised non-disease-specific instrument that describes and values health-related QoL and provides a single index value for a number of different health states. In addition, the resource utilisation questionnaire collected information on community-based medical resource usage [e.g. general practitioners (GPs), nurses, physiotherapists/occupational therapists, outpatients and medications]. Please refer to Appendix 12 for a summary of protocol changes.

End points

Pathology central review

Local pathology data were used to carry out the analyses. A central blinded review of the local pathology data for all assessable patients was carried out. The agreement of local pathology and central pathology with respect to factors feeding into the analyses (e.g. T-staging) was assessed via summaries.

Sample size

Randomisation

Randomisation took place as soon as possible after consent was obtained and after patients had completed their baseline patient-reported questionnaires (I-PSS, IIEF/FSFI, SF-36v2, MFI-20, EQ-5D). Randomisation took place as close to the date of surgery as possible. Surgeons were strongly encouraged to consent and randomise patients within 14 days of the planned surgery date whenever possible.

Following confirmation of written informed consent and eligibility, patients were randomised into the trial by authorised members of staff at the trial sites. Randomisation was performed centrally using the Clinical Trials Research Unit (CTRU) automated 24-hour telephone randomisation system. Authorisation codes and personal identification numbers (PINs), provided by the CTRU, were required to access the randomisation system.

Patients were randomised on a 1 : 1 basis to receive either robotic-assisted or standard laparoscopic rectal cancer surgery and were allocated a unique trial number. A computer-generated minimisation programme that incorporated a random element was used, with the following minimisation factors:

• treating surgeon

• patient sex (male or female)

• neo-adjuvant therapy (yes or no)

• nature of intended procedure (HAR, LAR or APR)

• body mass index (BMI) [calculated automatically from height (cm) and weight (kg) provided at randomisation and classified according to World Health Organization (WHO) criteria29]:

  • underweight/normal

  • overweight

  • obese class I

  • obese class II

  • obese class III.

Participating research sites were required to complete a log of all patients screened for eligibility who were not randomised either because they were ineligible or because they declined participation. Anonymised information was collected including:

• age

• sex

• date screened

• reason not eligible for trial participation

• eligible but declined and reason for this

• other reason for non-randomisation.

Blinding

As the two surgical procedures create incisions that can allow the patient to be blinded to the operative procedure performed, it arguably would have been scientifically preferable to blind patients to their surgical procedure, particularly in respect of patient-reported outcomes. However, it was anticipated that in practice maintaining the blind would have been extremely problematic (e.g. in countries such as the USA where private health-care insurance companies require disclosure of surgery details). Furthermore, it was anticipated that patients would also be seen by many health-care professionals throughout their time in the trial, increasing the risk that the blind may be broken. As a consequence, the trial design did not involve blinding patients to the operative procedure.

It should be noted that the trial end points are mainly objective measures and a central blinded assessment of these measures was included when possible (e.g. blinded central assessment of the quality of the plane of surgery).

Statistical methods

Unless otherwise stated, all analyses were prespecified and conducted on the intention-to-treat population (i.e. all randomised patients were categorised into treatment groups based on their randomisation, regardless of what treatment they subsequently received). All hypothesis tests were two-sided and conducted at the 5% level of significance. Estimates and their corresponding 95% confidence intervals (CIs) and p-values are presented for fixed effects. For the (random) surgeon effect, the intracluster correlation coefficient (ICC), estimated via the analysis-of-variance method, and bias-corrected bootstrapped 95% CIs are reported.

For most end points there was only a small number of missing data, such that a complete-case analysis was appropriate. For end points with non-negligible numbers of missing data, exploratory analyses were performed to consider the potential impact of the missing data. All models were fitted using SAS^® version 9.4 (SAS Institute Inc., Cary, NC, USA).

All analyses, unless otherwise stated, adjusted for the minimisation factors only (see Randomisation). For each end point, sensitivity analyses to include adjustment for treating centre and country were considered; Subgroup analyses gives further details of this.

Health economic evaluation

An economic evaluation was performed using a UK NHS perspective to aid the development of an evidence base to support NHS service providers and budget holders in their decision-making processes. Costs associated with robotic surgery excluded acquisition and maintenance costs. The evaluation estimated the expected incremental cost-effectiveness of robotic resection compared with laparoscopic resection at 6 months. It was planned that this would be extrapolated using a decision-analytic model to estimate lifetime cost-effectiveness.

The ROLARR trial collected information on the nature of all initial resection operations using trial CRFs. This included information on the type of operation and resources used within this operation, including instrumentation and times for operation theatres and staff. CRF data also captured information on postoperative and distal complications on all trial patients.

However, many types of resource utilisation were not collected for all patients in the ROLARR trial. In particular, given the challenges of conducting research within global trials, data on resource utilisation after the initial operation were collected only on patients from the UK and the USA. As the adjuvant chemotherapy is likely to both vary widely and be expensive, there is a danger that any small differences at this stage will be both unrelated to the surgery received and, given the cost of chemotherapy, outweigh any cost differences that are related to the intended type of surgery. For this reason, the cost data do not attempt to consider the chemotherapies received or antinausea drugs attached to these chemotherapies.

For patients in the UK and USA, information was collected alongside the trial-related questionnaires at approximately 30 days and at 6 months. It was expected that data collection might be poor and as a result it would be necessary to impute data for a substantial number of these patients.

Data were collected on both primary care and secondary care, including contact with GPs and primary care physicians (including the location of any contacts), nurse contacts (including at primary care, district/at home nursing, and stoma nursing) and outpatient visits.

The analysis considered costs in GBP with 2015 as the base year, from an NHS-payer perspective. Given the focus of this perspective, when clinical practice appeared to differ in the USA, particularly around pain medication, the approach taken was the one that appeared to apply in the UK. This perspective also means that unit costs are those costs that apply within the NHS.

Recruitment

Between 7 January 2011 and 30 September 2014, 1276 patients were assessed for eligibility by 40 surgeons from 26 sites across 10 countries (i.e. UK, Italy, Denmark, USA, Finland, South Korea, Germany, France, Australia and Singapore). In total, 471 (36.9%) of these patients were randomised: 234 to laparoscopic and 237 to robotic surgery (Figure 1). Four patients withdrew from data collection before their operation and one patient did not undergo surgery because of a complete clinical response to neo-adjuvant therapy. The remaining 466 patients underwent an operation, with 456 (97.9%) undergoing the allocated treatment.

FIGURE 1.

The CONSORT flow diagram. ITT, intention to treat.

Baseline data

All minimisation factors except treating surgeon are summarised by treatment group across all randomised patients in Table 6. The minimisation factor intended operating surgeon is summarised by treatment group for all randomised patients in Table 7. Summaries of selected additional baseline fields are given in Table 8.

Operative and pathology summaries

Crude summaries of operative and local pathology data fields are given in Table 9. The primary end point, conversion to open surgery, is summarised here but is considered in more detail in Primary end point: conversion to open surgery. The key secondary end point, CRM+, is also summarised here but is considered in more detail in Key secondary end point: circumferential resection margin positivity (CRM+).

Table 10 presents the different pathways of intraoperative conversions between robotic, laparoscopic and open surgery. For example, the pathway ‘Laparoscopic → Robotic’ indicates that the patient’s operation began as a standard laparoscopic operation, but was converted to a robotic operation intraoperatively. A pathway with only one type of operation indicates no conversions, for example ‘Laparoscopic’ indicates that the patient’s operation began as a standard laparoscopic operation and was completed without conversion to robotic or open surgery.

Primary end point: conversion to open surgery

The rate of conversion to open surgery was 47 out of 466 (10.1%) patients overall: 28 out of 230 (12.2%) in the laparoscopic group and 19 out of 236 (8.1%) in the robotic group (unadjusted difference in proportions 4.12%, 95% CI –1.35% to 9.59%). There was no statistically significant difference between robotic surgery and conventional laparoscopic surgery with respect to odds of conversion (adjusted OR 0.614, 95% CI 0.311 to 1.211; p = 0.16).

The random intercept model was preferred to the random slope model, because the random slope model did not offer sufficient improvement in model fit, which is clear from both the non-significant likelihood ratio test result and the increase in AIC (see Appendix 1, Table 51).

Table 11 presents adjusted estimates of ORs and 95% CIs from the random intercept model, as well as crude summaries and unadjusted risk difference estimates and 95% CIs for conversion to open surgery by treatment group and also by each of the minimisation factors. The model shows significantly increased odds of conversion in obese patients versus underweight/normal patients (adjusted OR 4.691, 95% CI 2.080 to 10.581; p < 0.01) and in males versus females (adjusted OR 2.444, 95% CI 1.047 to 5.708; p = 0.04). Patients whose intended procedure was a LAR had a significantly higher rate of conversion than patients whose intended procedure was APR (adjusted OR 5.435, 95% CI 1.595 to 18.519; p = 0.007). Operating surgeon had a mild to moderate effect on odds of conversion, as reflected by the ICC estimate of 0.056 (95% CI 0.007 to 0.056).

Key secondary end point: circumferential resection margin positivity (CRM+)

A total of 459 (98.5%) patients of the 466 who had an operation had complete pathology data available (Table 21). In that group, 26 out of 459 (5.7%) patients were CRM+, 14 out of 224 (6.25%) patients in the laparoscopic group and 12 out of 235 (5.11%) patients in the robotic group (unadjusted difference in proportions 1.14%, 95% CI –3.10% to 5.38%). There was no statistically significant difference in the odds of CRM+ between the groups (adjusted OR 0.785, 95% CI 0.350 to 1.762; p = 0.56). It should be noted that the variance component estimate for operating surgeon is 0, and consequently there is not a valid standard error estimate for this. This indicates that the variation of odds of CRM+ between surgeons was negligible (Table 22).

Key secondary end point: 3-year local recurrence

Follow-up times are summarised in Table 23. Median follow-up time from randomisation was 3.1 years. The seven patients with missing data in Table 23 had non-standard circumstances; three patients had benign disease (two laparoscopic, one robotic), one patient did not undergo surgery (laparoscopic) and three patients had palliative surgery only (one laparoscopic, two robotic). These seven patients were censored at time 0.

A local recurrence was observed in 30 out of 471 (6.4%) patients, 14 out of 234 (6.0%) in the laparoscopic group and 16 out of 237 (6.8%) in the robotic group. The date of local recurrence was defined as the date of the relevant assessment (i.e. clinical, radiological and pathological) that first detected the local recurrence.

The estimated cumulative incidence of local recurrence in each treatment group is presented in Figure 2 (note that the y-axis is truncated to 0–0.1). At 3 years, the estimated difference (robotic minus laparoscopic) in cumulative incidence of local recurrence is 0.002 (95% CI –0.041 to 0.046).

FIGURE 2.

Estimated cumulative incidence of local recurrence, by treatment group.

Table 24 presents the estimated adjusted HRs and corresponding 95% CIs and Wald test p-values from the shared frailty model. There is not a statistically significant difference between the treatment groups. The estimated adjusted HR suggests that a patient undergoing robotic surgery is 1.137 (95% CI 0.554, 2.335; p = 0.756) times more likely to experience local recurrence than a patient undergoing laparoscopic surgery, all else being equal.

There appears to be a substantial difference in probability of local recurrence between males and females, with males much more likely to experience the event (adjusted HR 3.184, 95% CI 1.109 to 9.174; p = 0.031). This is reflected in the plot of cumulative incidence by sex in Figure 3 (note that the y-axis is truncated to 0–0.1).

FIGURE 3.

Estimated cumulative incidence of local recurrence by sex.

Intraoperative complications

A total of 70 out of 466 (15.0%) patients had an intraoperative complication, 34 out of 230 (14.8%) in the laparoscopic group and 36 out of 236 (15.3%) in the robotic group (unadjusted risk difference 0.5%, 95% CI –6.0% to 7.0%). Table 25 presents the numbers of patients experiencing different types of intraoperative complications. The most common intraoperative complications were damage to an organ/structure, significant haemorrhage and surgical equipment failure. Table 26 presents the multilevel logistic regression model. There was no significant difference between the groups (adjusted OR 1.020, 95% CI 0.599 to 1.736; p = 0.94). There is significant evidence of a difference in the odds of having an intraoperative complication between males and females (adjusted OR 3.083, 95% CI 1.543 to 6.158; p = 0.0015). Note that the variance component estimate for operating surgeon is 0 and consequently there is not a valid standard error estimate for this. This indicates that the variation of odds of CRM+ between surgeons was negligible (Table 27).

Thirty-day postoperative complications

A total of 151 out of 466 (32.4%) patients had a postoperative complication within 30 days of their operation, 73 out of 230 (31.7%) in the laparoscopic group and 78 out of 236 (33.1%) in the robotic group (unadjusted risk difference –1.3%, 95% CI –9.8% to 7.2%). Table 28 presents the numbers of patients who experienced different types of postoperative complications within 30 days of their operation. The most common were gastrointestinal complications (including anastomotic leak), surgical site infections and urinary complications. Tables 29 and 30 present the multilevel logistic regression model. There was no significant difference between the groups (adjusted OR 1.043, 95% CI 0.689 to 1.581; p = 0.84). There is significant evidence of a difference in the odds of having a postoperative complication within 30 days of an operation between males and females (adjusted OR 3.083, 95% CI 1.573 to 4.183; p = 0.0002).

Six-month postoperative complications (after 30 days)

A total of 72 out of 466 (15.5%) patients had a postoperative complication after 30 days and within 6 months of their operation, 38 out of 230 (16.5%) in the laparoscopic group and 34 out of 236 (14.4%) in the robotic group (unadjusted risk difference 2.1%, 95% CI –4.5% to 8.7%). Table 31 presents the numbers of patients to experience different types of postoperative complications after 30 days and within 6 months of their operation. The most common was gastrointestinal complication (including anastomotic leak). Tables 32 and 33 present the multilevel logistic regression model. There was no significant difference between the groups (adjusted OR 0.719, 95% CI 0.411 to 1.258; p = 0.25).

Thirty-day operative mortality

Death within 30 days of operation was a rare event, with 2 out of 230 (0.87%) and 2 out of 236 (0.85%) events in the standard laparoscopic and robotic groups, respectively. All deaths involved a septic complication and were related to the surgical intervention. Owing to the small number of events, sophisticated statistical models were not fitted.

Patient self-reported bladder function

Higher I-PSS indicates worse bladder function and is measured on a scale of 0–35.

Baseline characteristics of the population of patients with complete I-PSS data, and a comparison with those patients with missing I-PSS data, are given in Appendix 4.

Figure 4 visualises the distribution of I-PSS scores at 6 months post operation in the two treatment groups. The distribution of scores is very similar between the groups.

FIGURE 4.

Box plot of observed I-PSS values at baseline and at 6 months post randomisation, by treatment group.

Tables 34 and 35 present the multilevel generalised linear model. Normal errors were assumed, so the estimates represent differences in the mean I-PSS. The estimated difference in mean I-PSS (robotic minus standard) is –0.7426 (95% CI –2.0722 to 0.5870; p = 0.2726). On the 35-point scale, this is a very small effect size that is also not statistically significant. The baseline score is highly prognostic of the 6-month score. The estimated difference in mean I-PSS between patients with a difference in baseline score of 10 points, all else being equal, is 4.20 (95% CI 3.23 to 5.17; p < 0.0001).

Patient self-reported sexual function: males

A higher IIEF score indicates better sexual function; the score is measured on a scale of 5–75.

Baseline characteristics of the population of patients with complete IIEF data, and a comparison with those patients with missing IIEF data, are given in Appendix 5.

Figure 5 visualises the distribution of IIEF scores at 6 months post operation in the two treatment groups. The distribution of scores is very similar between the treatment groups. Median IIEF scores at 6 months were notably lower than at baseline in both groups.

FIGURE 5.

Box plot of observed IIEF values at baseline and at 6 months post randomisation, by treatment group.

Tables 36 and 37 present the multilevel generalised linear model. Normal errors were assumed, so the estimates represent differences in the mean IIEF score. The estimated difference in mean IIEF (robotic minus standard) is –0.8020 (95% CI –5.7039 to 4.1000; p = 0.7468). On the 70-point scale, this is a very small effect size that is also not statistically significant.

Patient self-reported sexual function: females

A higher FSFI score indicates better sexual function; the score is measured on a scale of 2–36.

Baseline characteristics of the population of patients with complete FSFI data, and a comparison with those patients with missing FSFI data, are given in Appendix 6.

Figure 6 visualises the distribution of FSFI scores at 6 months post operation in the two treatment groups. The distribution of scores is very similar between the treatment groups.

FIGURE 6.

Box plot of observed FSFI values at baseline and at 6 months post randomisation, by treatment group.

Tables 38 and 39 present the multilevel generalised linear model. Normal errors were assumed, so the estimates represent differences in the mean FSFI score. The estimated difference in the mean FSFI score (robotic minus standard) is –1.2309 (95% CI –6.0030 to 3.5413; p = 0.6010). On the 34-point scale, this is a small effect size that is also not statistically significant.

Patient self-reported generic health

The SF-36 is a multipurpose, short-form health survey with 36 questions. It has eight scales of functional health: physical functioning, social functioning, role limitation physical, role limitation emotional, mental health, vitality, pain and general health that are scored on a 0–100 scale. It also provides a physical component score (PCS) and a mental component score (MCS), which are combinations of the eight scales.

The SF-36 was collected at baseline, and at 30 days and at 6 months post operation.

A higher score indicates a better QoL.

Baseline characteristics of the population of patients with complete generic QoL data, and a comparison with those patients with missing generic QoL data, are given in Appendix 7.

Tables 40 and 41 show the multilevel linear model for the PCS and MCS, respectively. Figures 7 and 8 illustrate the model estimates and 95% CIs at baseline and at 1 month and 6 months post randomisation of the average PCS and MCS respectively, split by treatment group. The estimated average difference in PCS between the groups at baseline is negligible: –0.1220 (95% CI –1.6281 to 1.3840; p = 0.8737). This is also the case at 1 month and 6 months post randomisation, as shown by the small magnitude and large p-values for the estimates of interaction between treatment effect and time (see Table 40). The estimated average difference in MCS between the groups at baseline is also negligible, –0.4875 (95% CI –2.6008 to 1.6258; p = 0.6508). Again this does not change notably over time, as seen in Figure 8 and by the small magnitude and large p-values of the estimates of interaction between time and treatment effect in Table 32.

FIGURE 7.

Adjusted estimates and 95% CIs of mean SF-36v2 PCS values at baseline, at 1 month and at 6 months post randomisation, by treatment group.

FIGURE 8.

Adjusted estimates and 95% CIs of mean SF-36v2 MCS values at baseline, at 1 month and at 6 months post randomisation, by treatment group.

Patient self-reported fatigue

The MFI-20 is a self-report instrument. It contains 20 statements that cover different aspects of fatigue.

These 20 items are organised in five scales: general fatigue, physical fatigue, reduced activity, reduced motivation and mental fatigue.

The scores per item run from 1 to 5. For each scale, consisting of four items, a total score is calculated by summation of the scores of the individual items. Scores can range from the minimum of 4 to the maximum of 20. The use of a total score over all 20 items is not recommended.

The MFI-20 was collected at baseline, at 30 days post operation and at 6 months post operation. A higher score indicates more fatigue.

Baseline characteristics of the population of patients with complete MFI-20 data, and a comparison with those patients with missing MFI-20 data, are given in Appendix 8.

Figure 9 illustrates the model estimates and 95% CIs at baseline, at 1 month and at 6 months post randomisation for each of the five scales, split by treatment group. The estimated differences between the treatment groups in scores at baseline were as follows: general fatigue –0.2517 (95% CI –0.5965 to 1.0999; p = 0.5603), physical fatigue 0.3964 (95% CI –0.4404 to 1.2332; p = 0.3527), reduced activity –0.1634 (95% CI –0.9777 to 0.6510; p = 0.6938), reduced motivation –0.03917 (95% CI –0.7324 to 0.6540; p = 0.9117) and mental fatigue 0.1374 (95% CI –0.6626 to 0.9374; p = 0.7360). All of these differences are small and none are statistically significant. Furthermore, this lack of a notable difference between the groups persists over time, as seen in Figure 9 and in the non–significant interaction terms in the model; further details of the models are given in Appendix 8.

FIGURE 9.

Adjusted estimates and 95% CIs of mean values of each of the five scales of the MFI-20 at baseline, at 1 month and at 6 months post randomisation, by treatment group. (a) General fatigue; (b) physical fatigue; (c) reduced activity; (d) reduced motivation; and (e) mental fatigue.

Plane of surgery

A total of 456 out of 471 (96.8%) patients had a returned pathology report with data for the mesorectal plane assessment. There were 351 out of 456 (77.0%) patients’ specimens graded as mesorectal plane in the pathology report, 178 out of 233 (76.4%) in the laparoscopic group and 173 out of 223 (77.6%) in the robotic group (unadjusted risk difference 1.2%, 95% CI –6.5% to 8.9%). Table 42 presents the crude summary of plane of resection (mesorectum) between the treatment groups. Tables 43 and 44 present the multilevel logistic regression model. There was no significant difference of the odds of ‘mesorectal plane’ between the groups (adjusted OR 0.943, 95% CI 0.565 to 1.572; p = 0.821). Patients undergoing APR have notably lower odds of a mesorectal plane grading [adjusted OR (vs. LAR) 0.358, 95% CI 0.185 to 0.694; p = 0.0024].

Disease-free survival

A recurrence was observed in 73 out of 471 (15.5%) patients, 38 out of 234 (16.2%) in the laparoscopic group and 35 out of 237 (14.8%) in the robotic group.

The date of recurrence was defined as the date of the relevant assessment (i.e. clinical, radiological and pathological) that first detected the recurrence.

Kaplan–Meier estimates of DFS in each treatment group are presented in Figure 10.

FIGURE 10.

Kaplan–Meier plot of disease-free survival, by treatment group. Product-limit survival estimates with number of patients at risk. Note that the y-axis is truncated to 0.4–1.0.

Tables 45 and 46 present the estimated HRs and corresponding 95% CIs and Wald test p-values from the shared frailty model. There is no statistically significant difference between the treatment groups. The estimated adjusted HR suggests that a patient undergoing robotic surgery is 1.030 (95% CI 0.713 to 1.489; p = 0.874) times more likely to experience a recurrence or a new primary cancer or to die than a patient undergoing laparoscopic surgery, all else being equal.

There appears to be a substantial difference in DFS between patients undergoing different types of operation. All else being equal, those patients undergoing an APR are most likely to have a recurrence, a new primary cancer or to die (adjusted HR vs. LAR: 1.602, 95% CI 1.035 to 2.479; p = 0.034), and those patients undergoing HAR are least likely to have a recurrence, a new primary cancer or to die (adjusted HR vs. LAR 0.421, 95% CI 0.191 to 0.925; p = 0.031).

Overall survival

Death was observed for 46 out of 471 (9.8%) patients, 23 out of 234 (9.8%) in the laparoscopic group and 23 out of 237 (9.7%) in the robotic group.

Kaplan–Meier estimates of OS in each treatment group are presented in Figure 11.

FIGURE 11.

Kaplan–Meier plot of OS, by treatment group. Product-limit survival estimates with number of patients at risk. Note that the y-axis is truncated to 0.7–1.0.

Tables 47 and 48 present the estimated HRs and corresponding 95% CIs and Wald test p-values from the shared frailty model. There is not a statistically significant difference between the treatment groups. The estimated adjusted HR suggests that a patient undergoing robotic surgery is 0.945 (95% CI 0.530 to 1.686; p = 0.848) times more likely to die than a patient undergoing laparoscopic surgery, all else being equal.

TABLE 47 - Overall survival: adjusted estimates of HRs and 95% CIs from random shared frailty model

Parameter	HR	95% CI	p-value
Treatment allocation: robotic (vs. laparoscopic)	0.945	0.530 to 1.686	0.8483
Sex: male (vs. female)	2.187	1.017 to 4.700	0.0450
Neo-adjuvant therapy: yes (vs. no)	1.380	0.756 to 2.522	0.2945
BMI classification: obese (vs. underweight/normal)	0.577	0.258 to 1.290	0.1804
BMI classification: overweight (vs. underweight/normal)	0.652	0.339 to 1.255	0.2007
Intended procedure: APR (vs. LAR)	1.442	0.741 to 2.804	0.2815
Intended procedure: HAR (vs. LAR)	0.520	0.155 to 1.739	0.2881

TABLE 48 - Overall survival: estimate of the variance component from random intercept model

Effect	Variance component
	Estimate	Standard error
Operating surgeon (random effect)	< 0.001	0.1649

There appears to be a notable difference in the risk of death between males and females. All else being equal, the probability of death in males is 2.187 (95% CI 1.017 to 4.700; p = 0.045) times greater than in females.

Health economics

The results of the primary analysis are reported in Table 49. There are very similar QoL figures across the two groups, with a difference of 0.014 QALYs across the first 6 months of treatment. Across both groups, the pattern is of baseline EQ-5D utilities being highest pre-surgery (0.810 vs. 0.828) and noticeably lower at 30 days (0.680 vs. 0.700). The utilities are much closer to their pre-surgery values by 6 months (0.774 vs. 0.798).

There is an overall cost difference of £980 across the two groups of the trial over the first 6 months of treatment. Across the different categories of costs in the model, robotic surgery is around £1085 more expensive than laparoscopic surgery. The main drivers of the higher operative costs for robotic surgery are (1) the duration of surgery (357 minutes and 408 minutes, respectively), which has a knock-on effect on the cost of theatre time and staff, and (2) the use of surgical instruments. There is very little difference in the number of staff in attendance (mean number of assistants 1.7 and 1.63). As more stomas are formed with laparoscopic surgery, the mean costs of both reversal surgeries (£585 vs. £481; £104) and stoma supplies (£547 vs. £486; £61) are slightly higher for this form of surgery. Mean costs are marginally higher for the robotic group in terms of medications and other health professional contacts (e.g. outpatients, GPs, nurses, etc.), although these differences are small (£590 vs. £656; –£66).

Across this scenario, the expected costs for robotic surgery are higher (£980) and robotic surgery provides marginally more health (QALY gain 0.014). These figures combine for an estimated incremental cost-effectiveness ratio (ICER) of £69,837 per QALY. This figure is well in excess of a standard threshold of £20,000–30,000 per QALY and suggests that robotic surgery, even when not including the sizeable cost of the robot, is not cost-effective. The implication for this is that, even if this means that the robot is lying idle, it is less cost-effective to use the robot for rectal resections.

The cost-effectiveness acceptability curve (Figure 12) for this scenario provides an estimate of the likelihood that this is cost-effective at a cost-effectiveness threshold of £20,000 or £30,000 per QALY. Across the cases calculated, robotic surgery is cost-effective only 10% of the time when the threshold is taken to be £20,000 per QALY and only 20% of the time when the threshold is £30,000 per QALY.

FIGURE 12.

Cost-effectiveness acceptability curve for the primary analysis.

Across the other scenarios (Table 50), there was no strong suggestion of cost-effectiveness. For those patients who intended to receive low anterior surgery at randomisation, the QoL benefit for robotic surgery is very close to zero; there is a benefit of 0.002 QALYs and the ICER for this scenario is nearly £700,000 per QALY. Although there is still some small uncertainty about cost-effectiveness (there is a 10% chance that robotic surgery is cost-effective at £30,000 per QALY), there is no clear case for cost-effectiveness in this subgroup. Although the corresponding cost-effectiveness would be higher among the other intended surgical groups at baseline, there is no clinical reason to justify the consideration of these subgroups.

Across the two remaining sensitivity analyses, the ICER varied greatly according to the group of patients considered. Only 51% of UK and US patients in the subgroup had complete data on all costing categories and all QoL values. Data on QoL were more complete, with 133 patients providing enough information to allow QALYs to be computed. In contrast, only 99 patients provided full cost data. (Two patients provided cost data but did not provide full data for QALYs.)

In the subgroup with complete data, the ICER was lower than the baseline value, although still above the standard values for thresholds, at around £44,000 per QALY. It is worth noting that, here, the chance of cost-effectiveness at £20,000 and £30,000 per QALY is still only 16% and 30%, respectively.

However, the complete-case data appear slightly misleading. Across the 133 patients observed to have sufficient data to calculate QALYs, there is a 0.017 benefit (vs. 0.014 in the base case and 0.028 in the complete case). Across the 99 patients with full cost data, the cost difference observed (£1169) is much more similar to those in both the complete case (£1241) and the base case (£980). This suggests that the base-case results are broadly consistent with the overall data set and more representative than the complete-case example.

In contrast, the imputation across the pan-world trial (as opposed to the USA and the UK) produces quite different numbers. In this case, the benefits estimated are much lower than in the baseline case, leading to an ICER of > £170,000 per QALY.

Limitations

The ROLARR study had several limitations. The much lower than anticipated rate of conversion to laparotomy limits the ability to provide conclusive evidence about our primary question of how robotic surgery compares with conventional laparoscopic surgery in terms of the odds of conversion. However, the fact that no statistically significant differences between the treatment groups were seen in any of the end points does suggest that robotic surgery, when performed by surgeons with varying robotic experience, does not confer a clinically important benefit over laparoscopic surgery in the short term.

No blinding to treatment allocation was incorporated into this trial. Our primary end point and the measure of mortality will certainly be unaffected by this, as an objective end point. However, there is the potential for end points that are not completely objective to have been affected. In our pathology end points, including CRM+, we have guarded against this by carrying out a blinded central review of pathology assessments.

Despite enforcing a mandatory minimum experience level for surgeon participation, operations in this trial were performed, on average, by a surgeon considered to be an expert in conventional laparoscopic surgery and who may still have been in their learning phase for robotic surgery. The prespecified sensitivity analysis of learning effects addresses this, by extending the primary analysis model to analyse the interaction between operating surgeon experience and the treatment effect.

The primary analysis adjusted only for stratification factors (including operating surgeon) and thus it did not include an adjustment for treating site in particular. A (prespecified) adjustment for treating site was considered in a sensitivity analysis, but model estimation issues were caused by the small sizes of the resulting strata, resulting in no meaningful output.

Patient and public involvement

Patients were involved in all stages of the design and delivery of the ROLARR trial. Christopher Garbett provided valuable input into all aspects of the study and served on the Trial Management Committee, and Lynn Faulds Wood served on the Trail Steering Committee.

The following institutions and surgeons participated in the trial.

Aarhus University Hospital (Denmark): Soren Laurberg, Niels Thomassen and Lene H Iversen; Aria Health (USA): Luca Giordano; Augusta Kranken Anstalt (Germany): Benno Mann; Azienda Ospedaliera SS Antonio e Biagio (Italy): Giuseppe Spinoglio; Barnet & Chase Farm Hospitals NHS Trust (UK): Daren Francis; Centre Oscar Lambret (France): Mehrdad Jafari; Christie Hospital (UK): Chelliah Selvasekar; Duke University (USA): Linda Farkas and Michael Hopkins; European Institute of Oncology: Unit of Integrated Abdominal Surgery (Italy) – Fabrizio Luca and Roberto Biffi; European Institute of Oncology: Unit of Minimally Invasive Surgery (Italy) – Paolo Pietro Bianchi and Roberto Biffi; Frimley Park (UK): Henry Tilney and Mark Gudgeon; Gangnam Severance Hospital (South Korea): Kang Young Lee; Hospital Herlev, Copenhagen University (Denmark): Jacob Rosenburg, Henrik Loft Jakobsen, Mads Bundgaard, Jens Ravn Eriksen, Jesper Olsen and Thomas Bent Harvald; Jackson South Community Hospital (USA): Gustavo Plasencia and Henry J Lujan; John Muir Medical Center (USA): Samuel Oommen; Leeds Teaching Hospital Trust (UK): David Jayne, Richard Baker and Julian Hance; National University Hospital (Singapore): Charles Tsang; Oulu University Hospital (Finland): Tero Rautio; Peter MacCallum Cancer Centre (Australia): Andrew Craig Lynch; Roskilde Hospital (Denmark): Per Jess, Michael Seiersen, Ole Roikjaer and Steffen Brisling; Royal Surrey County Hospital (UK): Tim Rockall; San Pio X Hospital (Italy): Jacques Megevand; Torbay Hospital (UK): Stephen Mitchell; University of California, Irvine Medical Center (USA): Alessio Pigazzi, Joseph C. Carmichael and Steven Mills; University of Pisa (Italy): Luca Morelli; Washington University in St. Louis School of Medicine (USA): Elisa Birnbaum and Paul Wise.

The following people were local pathologists for the trial:

Aarhus University Hospital (Denmark): Rikke Hagemann-Madsen and Katrine Stribolt; Aria Health (USA): Peter Farano, Thomas Rizzo Jr and Behnaz Toorkey; Augusta Kranken Anstalt (Germany): Stathis Philippou and Konrad Morgenroth; Azienda Ospedaliera SS Antonio e Biagio (Italy): Narciso Mariani, Paola Barbieri and Paola Re; Barnet & Chase Farm Hospitals NHS Trust (UK): Khurram Chaudhary and Anupam Joshi; Centre Oscar Lambret (France): Charles André; Christie Hospital (UK): Bipasha Chakrabarty, Guy Betts, Igor Racu-Amoasii and Khalifa Sawalem; Duke University (USA): Carol Filomena; European Institute of Oncology: Unit of Integrated Abdominal Pelvic Surgery (Italy) – Angelica Sonzogni and Luca Bottiglieri; European Institute of Oncology: Unit of Minimally Invasive Surgery (Italy) – Angelica Sonzogni and Luca Bottiglieri; Frimley Park (UK): Salome Beeslaar and George Kousparos; Gangnam Severance Hospital (South Korea): Soon Won Hong; Hospital Herlev, Copenhagen University (Denmark): Jill Levin Langhoff and Peter Ingeholm; Jackson South Community Hospital (USA): Rebeca Porto; John Muir Medical Center (USA): Barry Latner; Leeds Teaching Hospital Trust (UK): Padmini Prasad and Heike Grabsch; National University Hospital (Singapore): Teh Ming, Fredrik Petersson and Wang Shi; Oulu University Hospital (Finland): Markus Mäkinen and Tuomo Karttunen; Peter MacCallum Cancer Centre (Australia): Phillip Moss and Catherine Mitchell; Roskilde Hospital (Denmark): Peter Engel, Susanne Eiholm, Matteo Biagini and Anne Mellon Mogensen; Royal Surrey County Hospital (UK): Izhar Bagwan; San Pio X Hospital (Italy): Claudio Clemente, Anna Maria Ferrari, Stefania Rao and Cristina Campidelli; Torbay Hospital (UK): Nick Ryley, Ian Buley and Maria Consuelo Garrido; University of California, Irvine Medical Center (USA): Mark Li-Cheng Wu, Roberta Edwards and Philip Carpenter; University of Pisa (Italy): Daniela Campani and Luca Emanuele Pollina; Washington University in St. Louis School of Medicine (USA): Ilke Nalbantoglu.

Contributions of authors

David Jayne (Professor of Surgery) contributed to the design of the study, patient recruitment, data interpretation and manuscript writing.

Alessio Pigazzi (Consultant Surgeon) contributed to the design of the study, patient recruitment, data interpretation and manuscript writing.

Helen Marshall (Senior Statistician) contributed to the study design and oversight, data analysis and interpretation, and manuscript writing.

Julie Croft (Senior Clinical Triallist) contributed to the study design, study coordination and manuscript writing.

Neil Corrigan (Senior Statistician) contributed to the study design and oversight, data analysis and interpretation, and manuscript writing.

Joanne Copeland (Clinical Triallist) contributed to the study coordination and manuscript writing.

Philip Quirke (Professor of Pathology) designed the pathology protocol, provided training to other centres, reviewed the pathology and interpreted the pathology data.

Nicholas West (Academic Consultant Histopathologist) contributed to the design of the pathology protocol, reviewed the pathology and interpreted the pathology data.

Richard Edlin (Health Economist) contributed to the design of the cost-effectiveness study, data analysis and manuscript writing.

Claire Hulme (Professor of Health Economics) contributed to the design of the cost-effectiveness study, data analysis and manuscript writing.

Julia Brown (Professor of Statistics and Director of the Leeds Clinical Trial and Research Unit) contributed to the study design and oversight, data analysis and interpretation, and manuscript writing. All authors reviewed the final manuscript.

Publications

Collinson FJ, Jayne DG, Pigazzi A, Tsang C, Barrie JM, Edlin R, et al. An international, multicentre, prospective, randomised, controlled, unblinded, parallel-group trial of robotic-assisted versus standard laparoscopic surgery for the curative treatment of rectal cancer. Int J Colorectal Dis 2012;27:233–41.

Jayne D, Pigazzi A, Marshall H, Croft J, Corrigan N, Copeland J, et al. Effect of robotic-assisted vs. conventional laparoscopic surgery on risk of conversion to open laparotomy among patients undergoing resection for rectal cancer: the ROLARR randomized clinical trial. JAMA 2017;318:1569–80.

Corrigan N, Marshall H, Croft J, Copeland J, Jayne D, Brown J. Exploring and adjusting for potential learning curve effects in ROLARR: a randomised controlled trial comparing robotic-assisted vs. standard laparoscopic surgery for rectal cancer resection. Trials 2018;19:339–49.

Data-sharing statement

All available data can be obtained by contacting the corresponding author.

Patient data

This work uses data provided by patients and collected by the NHS as part of their care and support. Using patient data is vital to improve health and care for everyone. There is huge potential to make better use of information from people’s patient records, to understand more about disease, develop new treatments, monitor safety, and plan NHS services. Patient data should be kept safe and secure, to protect everyone’s privacy, and it’s important that there are safeguards to make sure that it is stored and used responsibly. Everyone should be able to find out about how patient data are used. #datasaveslives You can find out more about the background to this citation here: https://understandingpatientdata.org.uk/data-citation.

Disclaimers

This report presents independent research. The views and opinions expressed by authors in this publication are those of the authors and do not necessarily reflect those of the NHS, the NIHR, the MRC, NETSCC, the EME programme or the Department of Health and Social Care. If there are verbatim quotations included in this publication the views and opinions expressed by the interviewees are those of the interviewees and do not necessarily reflect those of the authors, those of the NHS, the NIHR, NETSCC, the EME programme or the Department of Health and Social Care.

Model diagnostics

Residual index plot

Figure 13 presents the index plot of raw residuals (including EBEs of random effects) on the probability scales (y-axis) versus patient ID (x-axis). Residual r_i for patient i is calculated as:

⁽⁵⁾ r i = Y i − p ^ i ,

where

⁽⁶⁾ Y i = { 0 , Otherwise 1 , Patient converted to open surgery ,

and p^i is the predicted probability of conversion to open surgery for patient i (including EBE of the random effect). Residuals with larger absolute values indicate poorer model fit. Patient 374 (labelled in Figure 13) stands out as an instance of poor model fit, with the model yielding a predicted probability of conversion of 0.676, but they were not converted. Many of the residuals for patients who did convert to open surgery (shown in green in Figure 13) are large, indicating that the model fitted low probabilities of conversion for those patients, but this is reasonable and perhaps expected because conversion to open surgery was an infrequent event. The empirical probability plot helps us to objectively determine what magnitude of residual is ‘expected’.

FIGURE 13.

Conversion to open surgery: index plot of raw residuals of the random intercept model. Scatterplot: raw residuals on the probability scale (including EBEs of random effects) by observed outcome.

Figure 14 presents the empirical probability plot for the primary analysis model, which can be used to compare actual Pearson residuals with expected Pearson residuals. The y-axis is the actual Pearson residual value, the x-axis is the empirical median Pearson residual expected under our fitted model assumptions. The dots represent patients. If the model was a perfect fit, then we would expect all of the dots to lie on the reference line. The band in Figure 14 represents the interval between the empirical 2.5th percentile and 97.5th percentile empirical Pearson residual. No values lie outside this region, indicating that we do not have any substantial outliers.

FIGURE 14.

Conversion to open surgery: empirical probability plot (including simulated envelope of 2.5th–97.5th percentile empirical Pearson residual) of raw residuals of the random intercept model.

Delta-betas

Figure 15 presents the plot of exponentiated delta-betas (y-axis) versus patient ID. Exponentiated delta-betas further from 1 indicate greater influence of the observation on the estimated treatment effect. Conversions to open surgery demonstrably have greater influence on the estimated treatment effect, which is perhaps expected given that conversion to open surgery was an infrequent event. Patient 374 stands out as having high influence for a non-conversion to open surgery. Patient 374 was in the robotic treatment group and their exponentiated delta-beta is 1.051, indicating that the estimated OR for conversion to open surgery (robotic vs. laparoscopic) increases by a factor of 1.051 when they are removed from the model, from the original 0.614 (95% CI 0.311 to 1.211; p = 0.16) to 0.645 (95% CI 0.325 to 1.280; p = 0.21).

FIGURE 15.

Conversion to open surgery: plot of exponentiated delta-betas from the random intercept model. Scatterplot: random intercept model delta-betas by observed outcome.

Sensitivity analysis: actual operating surgeon – further details

Sensitivity analysis: actual procedure – further details

The most notable effect that adjusting for actual procedure had on the model estimates was on the effect of APR (vs. LAR), which is attenuated compared with the primary analysis model. Specifically, in this model (Table 57) the OR is 0.433 (95% CI 0.165 to 1.134; p = 0.088) compared with 0.184 (95% CI 0.054 to 0.627; p = 0.007) in the primary analysis model.

Model diagnostics

Residual index plot

Figure 16 presents the index plot of raw residuals (including EBEs of random effects) on the probability scales (y-axis) versus patient ID (x-axis). Residual r_i for patient i is calculated as:

⁽⁷⁾ r i = Y i − p ^ i ,

in which:

⁽⁸⁾ Y i = { 0 , Otherwise 1 , CRM + ,

and p^i is the predicted probability of CRM+ for patient i (including EBE of the random effect). Residuals with larger absolute values indicate poorer model fit. Many of the residuals for patients who did have CRM+ (shown in green in Figure 16) are large, indicating that the model fitted low probabilities of CRM+ for those patients, but this is reasonable and perhaps expected because CRM+ was a rare event. There are no clear outliers in Figure 16. The empirical probability plot helps us to objectively determine what magnitude of residual is ‘expected’.

FIGURE 16.

Circumferential resection margin positivity: index plot of raw residuals of the random intercept model. Scatterplot: raw residuals on the probability scale (including EBEs of random effects) by observed outcome.

Figure 17 presents the empirical probability plot for the primary analysis model, which can be used to compare actual Pearson residuals with expected Pearson residuals. The y-axis is the actual Pearson residual value and the x-axis is the empirical median Pearson residual expected under our fitted model assumptions. The dots represent patients. If the model was a perfect fit, then we would expect all of the dots to lie on the reference line. The band in Figure 17b represents the interval between the empirical 2.5th percentile and 97.5th percentile empirical Pearson residual. No values lie outside this region, indicating that we do not have any substantial outliers.

FIGURE 17.

Circumferential resection margin positivity: empirical probability plot (including simulated envelope of 2.5th–97.5th percentile empirical Pearson residual) of raw residuals of the random intercept model.

There are two plots:

1. empirical probability plot with confidence region

2. same as plot 1 except restricted to only negative residuals (this has been added because the negative residuals are difficult to read on the other plot).

Delta-betas

Figure 18 presents the plot of exponentiated delta-betas (y-axis) versus patient ID. Exponentiated delta-betas further from 1 indicate greater influence of the observation on the estimated treatment effect. The CRM+ observations demonstrably have greater influence on the estimated treatment effect, which is expected given that CRM+ was a rare event. There are no clear overly influential observations.

FIGURE 18.

Circumferential resection margin positivity: plot of exponentiated delta-betas from the random intercept model. Scatterplot: random intercept model delta-betas by observed outcome.

Subgroup analyses

Model diagnostics

Deviance residuals

As a result of heavy censoring, the deviance residuals form a bimodal distribution, as seen in Figure 19. There are no clear outliers.

FIGURE 19.

Three-year local recurrence. (a) Histogram of deviance residuals; and (b) scatterplot of deviance residuals.

Figure 20 shows the standardised Schoenfeld residuals versus time. If the proportional hazards assumption was being violated, then we would expect the relationship between these residuals and time to deviate from a flat line in at least one of the treatment groups, but it does not. This is also reflected in Figure 21 (standardised score process), as the observed path lies within the simulated paths. Finally, the supremum test of the PH assumption returned a p-value of 0.265 relating to the treatment effect. All of this suggests that the proportional hazards assumption is viable for the treatment groups.

FIGURE 20.

Three-year local recurrence: Loess plot of standardised Schoenfeld residuals (by treatment group) for the shared frailty model.

FIGURE 21.

Three-year local recurrence: random sample of standardised score process simulated paths.

Delta-betas

Figure 22 presents the exponentiated delta-betas. As one might expect, given the low incidence of local recurrence, the influence of all observed events of local recurrence is notably greater than the influence of censored observations. There is, however, no clear overly influential observations.

FIGURE 22.

Three-year local recurrence: plot of exponentiated delta-betas from the random intercept model.

Subgroup analyses

Odds ratios presented in Tables 65 and 66 are derived from the linear combination of the estimated treatment (main effect) and treatment-by-subgroup interaction terms on the logit scale. The p-values are presented for the test of the treatment effect within each subgroup; this is the first column of p-values. For example, in Table 68 the test that the treatment effect is null (OR = 1) within the female subgroup is 1.000. The p-values are also presented for the test of heterogeneity of treatment effect across subgroups, the details of which are given in the footnotes of the tables. Note that a full model was not fitted to test T-stage subgroup analyses, because the small sample sizes and event rates within T-stage groups caused model convergence issues and so crude summaries are given.

Figures 23–26 display the Kaplan–Meier graphs for the effect of neo-adjuvant therapy, operation type, T-stage and sex, on 3-year local recurrence.

FIGURE 23.

Three-year local recurrence by neo-adjuvant therapy. (a) No neo-adjuvant therapy; and (b) neo-adjuvant therapy.

FIGURE 24.

Three-year local recurrence by operation type. (a) Cumulative incidence of local recurrence (operation = HAR); (b) cumulative incidence of local recurrence (operation = LAR); and (c) cumulative incidence of local recurrence (operation = APR).

FIGURE 25.

Three-year local recurrence by T-stage. (a) T1; (b) T2; and (c) T3.

FIGURE 26.

Three-year local recurrence by sex. (a) Male; and (b) female.