Conservative treatment for uncomplicated appendicitis in children: the CONTRACT feasibility study, including feasibility RCT

Inclusion criteria

• Children aged 4–15 years.

• Clinical diagnosis, either with or without radiological assessment, of acute appendicitis that, prior to study commencement, would be treated with appendicectomy.

• Written informed parental consent, with child assent if appropriate.

Exclusion criteria

• Clinical signs or radiological findings to suggest perforated appendicitis.

• Presentation with appendix mass.

• Previous episode of appendicitis or appendix mass treated non-operatively.

• Major anaesthetic risk precluding allocation to the appendicectomy arm.

• Known antibiotic allergy preventing allocation to the non-operative treatment arm.

• Antibiotic treatment started at referring institution (defined as two or more doses administered).

• Cystic fibrosis.

• A positive pregnancy test.

• Current treatment for malignancy.

Non-operative treatment arm

Children randomised to non-operative treatment were treated according to a clinical pathway designed specifically for this trial. This treatment pathway comprised fluid resuscitation, a minimum of 24 hours of broad-spectrum intravenous (i.v.) antibiotics (per local antimicrobial policy), a minimum of 12 hours of nil by mouth (NBM) and regular clinical review to detect signs and symptoms of significant clinical deterioration, including, but not limited to, increasing fever, increasing tachycardia and increasing tenderness. After the initial 12-hour period of NBM, oral intake was advanced as tolerated. Children successfully treated without an operation were converted to oral antibiotics (per local policy) once they were afebrile for 24 hours and tolerating oral intake.

Clinical reviews were completed at approximately 24 and 48 hours post randomisation. Any children who showed signs of significant clinical deterioration by 24 hours, or at any point during the trial, were treated with appendicectomy. Children who were considered stable or improving continued with non-operative treatment. At 48 hours, any child who had not shown clinical improvement underwent an appendicectomy. The decision to continue non-operative treatment at these time points, or to recommend discontinuation of non-operative treatment and to recommend appendicectomy instead, was made by the treating consultant and based on clinical judgement rather than any specific features that are not evidence based. All reasons for a change in treatment were recorded in detail to guide a clinical pathway in a future trial.

Any child who received an appendicectomy for an incomplete response to non-operative treatment followed a standardised postoperative treatment regime already in use at each institution and identical to that used in the appendicectomy arm. The reason for having an appendicectomy was recorded.

Children treated non-operatively received a total of 10 days of antibiotics following randomisation, unless decided otherwise by the treating clinician. Children who received non-operative treatment were not routinely offered interval appendicectomy, but were counselled about the risk of recurrence.

Appendicectomy treatment arm

Children randomised to the appendicectomy arm underwent either open or laparoscopic appendicectomy at the surgeon’s discretion, performed by a suitably experienced trainee (as per routine current practice) or a consultant.

Participants received i.v. antibiotics from the time of diagnosis and were treated postoperatively with i.v. antibiotics according to existing institutional protocols; however, the following recommended regime was used to guide practice: children with acute uncomplicated appendicitis or a macroscopically normal appendix received no further antibiotics. Children with a perforated appendix (defined as a faecolith or faecal matter in the peritoneal cavity, or visualisation of a hole in the appendix) continued to receive i.v. antibiotics for a minimum of 3 days, and received a minimum total course of antibiotics of 5 days (i.v. and oral). The duration of antibiotic therapy was not standardised beyond this owing to anticipated variation in intraoperative findings and in response to treatment. The type of antibiotics used was identical to those used in the non-operative treatment arm in each centre. Any child failing to respond to first-line antibiotics was treated as was clinically appropriate with a longer course of antibiotics or a change in antibiotic therapy, with the choice of antibiotic determined by intraoperative swab or fluid culture.

Postoperatively, children with uncomplicated acute appendicitis or a normal appendix did not routinely have a nasogastric tube or a urinary catheter. They received oral intake as tolerated after surgery.

The participant flow through the two treatment arms is shown in Figure 1.

FIGURE 1.

Overview of the trial pathway. a, Non-operative treatment: NBM/sips for the initial 12 hours, minimum, then advance diet as tolerated; i.v. antibiotics for 24 hours, minimum, change to oral once afebrile for 24 hours, total course 10 days; and analgesia; b, appendicectomy group: no routine use of nasogastric tube or urinary catheter, advance diet as tolerates; c, defined as either seeing a hole in the appendix or seeing faecal matter/faecolith in the peritoneal cavity; d, continue i.v. antibiotics until afebrile for 24 hours, then change to oral; minimum 5 days total on antibiotics; e, criteria for discharge include vital signs within normal limits, tolerating light diet, adequate oral analgesia and mobile.

Discharge assessment

Criteria for discharge home were identical in both treatment arms and were as follows: vital signs within normal limits for age, afebrile for ≥ 24 hours, tolerating light diet orally, adequate oral pain relief and able to mobilise. We aimed to determine the feasibility of a blinded discharge assessment in a future RCT by attempting to complete a blinded discharge assessment for each participant as follows. Once a decision to discharge the child had been made, a member of the clinical team who had not been involved directly in the child’s treatment was asked to complete a discharge assessment. This assessor did not have prior knowledge of the randomisation or treatment received by the child. On completion of the discharge assessment, the assessor ‘guessed’ which treatment the child received. If the assessor became unblinded during the assessment, this was recorded.

Follow-up

All participants were given a diary card [see the project web page: www.journalslibrary.nihr.ac.uk/programmes/hta/1419290/#/ (accessed February 2020)] to complete for the 14 days immediately following discharge from hospital and provided with a stamped addressed envelope to return this to the clinical trials unit on completion. This assessed whether or not the child had taken antibiotics and analgesia medication on each day following discharge, as well as an assessment of their recovery based on their ability to complete normal or full daily activities and attend school (if applicable). Finally, it also assessed whether or not parents had had to miss work as a result of their child’s illness. Follow-up appointments for all participants took place at 6 weeks and at 3 and 6 months following discharge, either in the outpatient clinic or in the clinical research facility at each centre. If a face-to-face appointment was not possible, the 3- and 6-month follow-ups were completed by telephone. Following an analysis of follow-up rates during the trial, we introduced an incentive in an attempt to improve follow-up rates. This was introduced in March 2018. All participants who attended all remaining follow-up visits from that point onwards were offered a £10 voucher.

Participant identification and recruitment

Participants were identified by the clinical team at the time of diagnosis and their eligibility was confirmed by the research team as soon as possible. Eligible patients were approached by the treating clinical teams with support from dedicated research nurses. Potential participants were provided with written information about the trial [see the project web page: www.journalslibrary.nihr.ac.uk/programmes/hta/1419290/#/ (accessed February 2020) and shown a short video describing the study (see http://tinyurl.com/contract-f]. From the time of first discussing the trial with potential participants and their families, a maximum of 4 hours was permitted before a decision could be made regarding participation. This was to ensure that there was no delay in providing treatment as a result of considering trial participation. After receiving written informed consent (and assent from children aged ≥ 12 years who wished to give it), a member of the trial team randomised the participant to one of two treatment groups in a 1 : 1 ratio via an independent web-based system [TENALEA (Alea Clinical, Abcoube, the Netherlands)]. This online system allowed complete pre-randomisation concealment of treatment allocation and provided instant assignment to either the appendicectomy group or the non-operative treatment group. Minimisation was used to account for recruiting centre and to ensure balance between the groups in factors that may affect diagnostic accuracy and outcome of treatment. The factors taken into account were (1) sex, (2) age (4–8 years or 9–15 years), (3) duration of symptoms (onset of pain to recruitment to trial: < 48 hours or ≥ 48 hours) and (4) recruiting centre. In addition to the data required to complete randomisation, limited additional data were collected at baseline, including the use of any diagnostic imaging, and an Alvarado score21 (a scoring system used to help predict the severity of appendicitis) was calculated for each participant. This was used to provide an overview of the severity of illness of each child as a descriptive term; it was not used as a minimisation variable nor was a minimum Alvarado score used in the eligibility criteria.

Data collection and analysis

Data were recorded by dedicated research nurses at each site directly into an electronic, secure, web-based case report form (Medidata Rave^® database; Medidata Solutions, Inc., New York, NY, USA). Data analysis was performed by the study statistician, who was blinded to treatment allocation by the use of coded data. As this is a feasibility study, all analyses were treated as preliminary and exploratory and data are reported descriptively. Feasibility outcomes (number of eligible patients, recruitment/retention rates, reasons for non-participation, success of blinding of the discharge assessor), treatment outcomes and complications are presented as simple summary statistics with 95% confidence intervals (CIs). Clinical outcomes were compared between treatment groups in an exploratory analysis.

Trial oversight

A Study Management Group (SMG) was responsible for overseeing the day-to-day management of the trial. An independent Trial Steering Committee and Data Monitoring and Safety Committee were convened to provide oversight of the trial. Their roles and responsibilities, which included adverse event (AE) monitoring, were agreed at the beginning of the trial and documented in specific charters. Specific processes to report AEs in a timely manner to the relevant committee were agreed.

Protocol, registration and ethics approval

The trial was carried out in accordance with a published protocol22 that was developed in accordance with the Standard Protocol Items: Recommendations for Interventional Trials – Children (SPIRIT-C) guidance,23 and was registered prior to recruitment of the first participant (International Standard Randomised Controlled Trial Number: ISRCTN15830435). The overall study was given ethics approval by the Hampshire A Research Ethics Committee (reference number 16/SC/0596). The study is reported in accordance with the Consolidated Standards of Reporting Trials 2010 statement. 24

Primary outcome

The primary outcome was to assess the feasibility of conducting a multicentre RCT testing the clinical effectiveness and cost-effectiveness of a non-operative treatment pathway for the treatment of acute uncomplicated appendicitis in children. This was evaluated as the proportion of eligible patients who were approached and recruited to the study over 12 months.

Secondary outcomes

The secondary outcomes were predominantly centred on the qualitative and COS substudies, contributing to the development of a future RCT. Those specifically relating to the RCT are marked with an ‘*’:

• *Determine the willingness of parents, children and surgeons to take part in a randomised trial comparing operative treatment with non-operative treatment, and identify the anticipated recruitment rate. This was assessed from audio-recorded family–surgeon recruitment consultations; interviews with patients, parents, surgeons and nurses; surgeon surveys; and focus groups.

• Identify strategies to optimise surgeon–family communication using the consultation and interview data.

• Design a future RCT from the perspectives of stakeholders at participating sites (children, parents, surgeons, nurses, etc.), informed by the consultation and interview data, surgeon surveys and focus groups.

• Assess the equipoise and willingness of UK paediatric surgeons to participate in a future RCT through surgeon surveys and focus groups.

• *Assess the clinical outcomes of trial treatment pathways, including (1) overall success of initial non-operative treatment (measured as the number of participants who were randomised to non-operative treatment and discharged from hospital without appendicectomy), (2) complications of disease and treatment (measured during hospital stay and during the 6-month follow-up period) and (3) the rate of recurrent appendicitis during the 6-month follow-up period.

• *Assess the performance of the study procedures, including retention of participants for the duration of the trial and the feasibility of outcome-recording and data collection systems.

Version 2, 10 April 2017

• Minor clarification of serious adverse event (SAE) exceptions in section 6.2.1.

• Addition of ISRCTN reference on front page.

Version 3, 4 July 2017

• Change to co-investigator at St George’s Hospital.

• Reference to online patient video access.

• Consent process oversight by Southampton Clinical Trials Unit (SCTU).

• Telephone consent process for qualitative substudy.

• Specification of office hours for randomisation backup.

• Timeline for questionnaire completion.

• Time frame for AE reporting.

• Update to COS protocol.

Version 4, 8 March 2018

• Addition of an incentive during the follow-up stage of the trial.

Compliance with outpatient antibiotic treatment

We had initially planned to assess compliance with outpatient antibiotic treatment in the non-operative treatment arm through the use of diary cards. However, owing to the low diary card completion rate and higher than anticipated crossover to appendicectomy during initial hospital admission, these data should be interpreted with some caution. They are included here for completeness (Figure 6).

FIGURE 6.

Compliance with oral antibiotics after discharge in the non-operative treatment arm (n = 11 until day 3, n = 10 thereafter). Proportion of participants in the non-operative treatment arm who reported taking antibiotics at or following discharge.

Baseline characteristics

Baseline characteristics for the participants overall and described by treatment allocation are shown in Table 3.

The distribution of Alvarado scores between groups is shown in Figure 7.

FIGURE 7.

Distribution of Alvarado scores across treatment arms.

Treatment received and outcome

In the appendicectomy arm, all 27 children received the allocated intervention and were treated according to the clinical pathway. Histological diagnosis of the resected appendix revealed simple acute appendicitis in 17 participants (63%), perforated appendicitis in eight participants (30%) and a normal appendix in two participants (7%).

In the non-operative treatment arm, 19 of the 27 children (70%) responded to non-operative treatment successfully, followed the clinical pathway and were successfully discharged from hospital. The remaining eight children underwent appendicectomy during their initial hospital admission. Reasons for appendicectomy were parental choice (withdrawal from treatment arm, as discussed in Adherence to treatment allocation and protocol, trial retention and other feasibility outcomes) at 8 hours following randomisation (n = 1), deterioration in clinical condition (according to protocol) at range 12.3–44.4 hours following randomisation (n = 6) and no improvement after 48 hours of non-operative treatment (in accordance with protocol) (n = 1). For the children whose clinical condition deteriorated (n = 6), reasons for deterioration included persistent fever and pain, worsening peritonism and worsening pain. For the one child who did not have any symptomatic improvement after 48 hours, the primary persisting complaint was that of ongoing severe abdominal pain. All eight patients underwent successful appendicectomy during the initial hospital admission. Histological findings were simple acute appendicitis in four and perforated appendicitis in four.

Clinical outcomes related to hospital stay are reported on an intention-to-treat basis and are shown in Table 4. Both decision to discharge and actual time of discharge were recorded as there may be non-medical factors (e.g. availability of transport or other social factors) that delay actual discharge.

For participants in the non-operative treatment group, the time from randomisation to discharge from hospital for children who responded to non-operative treatment and for those who underwent appendicectomy during the initial hospital admission are shown in Table 5.

Of the 35 appendicectomies performed on participants in the trial during the initial hospital phase, 33 were performed laparoscopically, one was open surgery and one was laparoscopic converted to open surgery.

Early post-discharge outcomes

Data relating to early post-discharge recovery obtained from diary cards are summarised in Figures 8–10. A total of 26 diary cards were returned: 15 from the appendicectomy arm and 11 from the non-operative treatment arm. For one of the 11 participants in the non-operative treatment arm, data were reported up to and including day 4 only. For the remaining participants who returned diary cards, data were complete.

Pain medication after discharge home

The proportion of participants in each treatment arm who reported taking pain medication at and on the days following discharge home is shown in Figure 8. These data clearly suggest that analgesia use following discharge was lower following non-operative treatment than following appendicectomy. In accordance with the protocol, we have not performed a formal comparative analysis of these data.

FIGURE 8.

Analgesia use following discharge from hospital as reported in diary cards.

Return to normal activities

The proportion of participants in each treatment arm who reported being able to return to normal daily activities at and on the days following discharge home is shown in Figure 9. These data suggest that trial participants were able to return to normal activities faster following non-operative treatment than following appendicectomy.

FIGURE 9.

Return to normal activities as reported in diary cards.

Return to full activities

The proportion of participants in each treatment arm who reported being able to return to full activities at and on the days following discharge home is shown in Figure 10. Like the data on return to normal activities, these data suggest that trial participants were able to return to full activities faster following non-operative treatment than following appendicectomy.

FIGURE 10.

Return to full activities as reported in diary cards.

Parental absence from work

The proportion of participants in each treatment arm whose parents reported missing work as a result of their child’s illness or recovery is shown in Figure 11. These data suggest that parents were able to return to work more quickly if their child was allocated to non-operative treatment than if their child was allocated to appendicectomy. All parents of children allocated to non-operative treatment and who completed diary cards had returned to work by 9 days following discharge, whereas the parents of some children allocated to appendicectomy had not returned to work 2 weeks after discharge.

FIGURE 11.

Parental absence from work as reported in diary cards.

The 6-month follow-up

During the 6-month follow-up period, a total of seven children, all of whom were randomised to non-operative treatment, developed recurrent appendicitis. This equates to 24% of the total number randomised to non-operative treatment and 37% of the 19 participants who initially responded to non-operative treatment and were discharged from hospital without having had an appendicectomy. Of these seven children, six underwent appendicectomy (laparoscopic or open); histological findings were simple acute appendicitis in four of these children and perforated appendicitis in the remaining two. The remaining participant presented with an appendix mass at the time of recurrence, which was successfully treated non-operatively, and subsequently underwent interval laparoscopic appendicectomy. At the final follow-up of the trial, 11 children initially randomised to receive non-operative treatment (41%) had not undergone appendicectomy.

In the appendicectomy arm, three children (11%) were re-admitted to hospital following initial discharge for investigation and/or treatment of potential complications related to appendicectomy.

Adverse events

Adverse events during the 6-month follow-up period were captured and are summarised in Appendix 1, Tables 15 and 16. In the appendicectomy arm, 22 AEs were reported in eight participants; in the non-operative treatment arm, 24 AEs were reported in 15 participants. A small number of these AEs were assigned as being ‘possibly’ or ‘definitely’ attributable to the trial intervention.

In the non-operative treatment arm, the AEs included a rash at the time of receiving antibiotics in two participants. In the appendicectomy arm, three children developed wound complications (one dehiscence, one infection and one suture complications); one child developed a postoperative intra-abdominal abscess requiring percutaneous drainage and a peripherally inserted central catheter (PICC) for prolonged antibiotics; and one child had intra-abdominal fluid collection, which was treated with intravenous antibiotics. Other AEs, including sickness, pain, visits to the general practitioner (GP), visits to the emergency department and investigations through blood tests and ultrasonography, occurred in both treatment groups. Overall, three children in the appendicectomy arm and eight children in the non-operative treatment arm were re-admitted to hospital. All hospital re-admissions in the non-operative treatment arm were for treatment of recurrent appendicitis, which resulted in appendicectomy for seven participants and a PICC for one participant.

Overview

We drew on an integrative qualitative communication approach, involving analysis of audio-recordings of CONTRACT consultations and in-depth interviews with parents, patients (i.e. children and young people) and health professionals. 47 This integrative qualitative communication approach is particularly suited to explore clinical communication in trials. The consultation recordings allowed us to explore how health professionals explained CONTRACT during consultations with families, and the interviews allowed us to explore children’s, parents’ and health professionals’ perspectives on communication about CONTRACT. 42 The communication study was included in the ethics approval given for CONTRACT. Patients and parents were provided with a summary of the results on completion of the study and we received no subsequent feedback from participants regarding the findings.

Recruitment training and phases

CONTRACT recruited for 12 months from March 2017 and the communication study ran concurrently. Before recruitment began, we delivered generic recruitment training in December 2016. We analysed the qualitative data iteratively throughout the CONTRACT recruitment period (and afterwards), and these ongoing analyses informed the development of bespoke recruitment training. While the trial was ongoing, we delivered two bespoke recruitment training sessions at all three CONTRACT sites in July 2017 and November 2017. We therefore classified ‘phase 1’ of recruitment as March–June 2017, ‘phase 2’ as July–October 2017, and ‘phase 3’ as November 2017–February 2018. We refer to these phases in Results. All training sessions involved Microsoft PowerPoint^® (Microsoft Corporation, Redmond, WA, USA) presentations and discussion between the communication study team and health professionals at each individual site. The recruitment training that was delivered at the end of phases 1 and 2 was specifically informed by the ongoing analyses of the qualitative data. Health professionals were asked to complete training evaluation forms (available at www.journalslibrary.nihr.ac.uk/programmes/hta/1419290/#/) pre training, and after training sessions 1 (generic recruitment training), 2 and 3 (bespoke recruitment training sessions). Finally, we provided health professionals with written ‘hints and tips’ on ways to optimise CONTRACT consultation communication. We updated this information periodically in the light of the ongoing qualitative analysis.

Participants

Health professionals approached families of children who were eligible for CONTRACT at one of three UK hospitals. The families of all children who were eligible for CONTRACT were eligible for the communication study. During this period, we also approached and interviewed health professionals who had been involved in CONTRACT at one of the three sites.

Procedure

Consultations

We obtained consultation recordings for half of families that were approached about CONTRACT (58/115, 50%). Most of the families that we obtained consultation recordings from participated in CONTRACT (38/58, 66%). Of these 38 families that participated in CONTRACT, 19 were randomised to surgery and 19 to non-operative treatment (antibiotics). In recruitment phases 1, 2 and 3 (see Recruitment training and phases for definition of phases), consultations were recorded for 17, 23 and 18 families, respectively. Of the 58 families with consultation recordings, 23 were from site 1, 19 were from site 2 and 16 were from site 3. The majority of children were male (39/58, 67%) and children ranged from 4 to 15 years of age [median 10 years, interquartile range (IQR) 8–12 years]. We also obtained families’ postcodes when possible (52/58, 90%) and used the 2015 English Indices of Multiple Deprivation (IMDs)52 to indicate socioeconomic status: 15 out of 52 (29%) families lived in areas of high deprivation (IMD deciles 1–3), 18 out of 52 (35%) families lived in areas of moderate deprivation (IMD deciles 4–7) and 19 out of 52 (37%) families lived in the least deprived (IMD deciles 8–10) areas of England. 52

Often, health professionals had multiple brief consultations with families; we asked them to audio-record all of these if possible. Thirty-eight families had one consultation audio-recorded, 14 families had two audio-recorded consultations, five had three audio-recordings and one family had four audio-recordings. The duration of initial CONTRACT consultations ranged from 38 seconds to 24 minutes [median 10 minutes (IQR 5–12 minutes)], although initial consultations usually lasted longer than subsequent ones.

Family interviews

In total, the research team attempted to contact 57 families to arrange an interview. Common reasons for non-participation in the family interview included the family not responding to telephone calls and the family declining to participate because they was too busy. Twenty-eight families completed an interview (see Figure 12). Most families that completed an interview were randomised to CONTRACT (19/28, 68%) and, of those, just over half were randomised to the non-operative treatment arm (10/19, 53%).

Of the 10 families interviewed that were randomised to the non-operative treatment arm, four eventually had an appendicectomy owing to non-operative treatment failure. Of these four, one child was later re-admitted because of surgical complications, and three suffered recurrent appendicitis and were re-admitted. Of the three who were re-admitted with recurrent appendicitis, two were treated again with non-operative treatment (as opposed to appendicectomy), and one family withdrew from CONTRACT owing to concerns that the child was deteriorating; the child was subsequently treated with an appendicectomy. Of the nine families interviewed that were randomised to appendicectomy, one child was re-admitted to hospital because of surgical complications.

Eight, 12 and eight families from sites 1, 2 and 3, respectively, completed an interview. Of the 28 families interviewed, eight (29%) lived in areas of high deprivation (IMD deciles 1–3), seven (25%) lived in areas of moderate deprivation (IMD deciles 4–7) and 13 (46%) lived in the least deprived (IMD deciles 8–10) areas of England. 52 Twelve families were interviewed face to face in their homes, whereas the remaining interviews were completed by telephone. Overall, interviews lasted from 22 minutes to 89 minutes [median 59 minutes (IQR 46–66 minutes)]. In recruitment phases 1, 2 and, 3, seven, 14 and seven families were interviewed, respectively.

Most parents (19/28, 68%) completed an interview without their child being present. In total, 15 interviews were completed with mothers only, seven were completed with fathers only and six were completed with both parents present. Of the children eligible for interview (n = 25), 14 completed an interview. Children did not participate in interviews for several reasons, including parents feeling that their child was too young to be interviewed, parents stating that their child did not want to be interviewed and parents not talking to their child about doing an interview. Most children completed an interview with a parent present (11/14). Children who completed an interview were aged 8–14 years [median 11 years (IQR 9–13 years)], and most were male (11/14).

Health professional interviews

Fifty-one health professionals were invited to participate in an interview and 35 were interviewed on one or two occasions (constituting 40 interviews in total). Of the 35 professionals, 25 were surgeons, seven were research nurses and three were ward nurses. Fifteen, 11 and nine health professionals from sites 1, 2 and 3, respectively, completed an interview. Interviews were face to face in the place of work (n = 23) or by telephone (n = 17). First interviews lasted from 20 to 79 minutes [median 48 minutes (IQR 38–56 minutes)] and repeat interviews (n = 5) lasted from 39 to 69 minutes [median 51 minutes (IQR 42–67 minutes)].

Qualitative analysis

In the following sections, we describe how health professionals communicated about CONTRACT during consultations and family and health professional experiences of CONTRACT throughout the three recruitment phases (i.e. after each recruitment training session). We outline how the interim qualitative findings informed the recruitment training sessions, and examine the impact of training on recruitment practice and communication about CONTRACT.

Recruitment: phase 1

Recruitment: phase 2

Recruitment: phase 3

Families’ experiences of CONTRACT

In the family interviews, we also explored families’ broader experiences of CONTRACT. In this section, we describe key topics of importance to families that should be considered in designing and implementing future trials in this context. Although some of the points are specific to CONTRACT and a future definitive trial of the treatment of simple appendicitis in children and young people, many of the issues we raise have implications for future trials in paediatric urgent care more broadly. In particular, families discussed their views on optimising children’s and young people’s involvement in research discussions in an acute setting, the optimal length of time for families to decide whether or not to participate, fear of perforation and post-surgical discussions with surgeons.

Parents’ reasons for surgery preference

As discussed, parents often preferred surgery to non-operative treatment, and therefore declined participation in CONTRACT. The most common underlying reasons for such preferences included the perception that surgery would avoid perforation and the belief that surgery would result in immediate pain relief. Furthermore, concerns regarding perforation and pain were compounded by treatment delays, which further drove families to opt for surgery.

Health professionals’ experiences of CONTRACT

In the interviews with health professionals, we explored their broad experiences of CONTRACT and the topics they raised regarding the design and conduct of a future trial. Some of the points are specific to CONTRACT and a future definitive trial of the treatment of acute uncomplicated appendicitis in children, but many have implications for future trials in paediatric urgent care more broadly.

Health professionals described influences on their clinical equipoise, their concerns regarding patient eligibility for CONTRACT, their experiences and reservations about exploring families’ treatment preferences, their experiences of dealing with conflicting treatment preferences within families, issues with conducting blinded discharge assessments and the challenges of explaining the concept of feasibility in the urgent care setting.

Quantitative analysis of the impact of recruitment training on recruitment and surgeon confidence in approaching families about CONTRACT

In addition to assessing the impact of training qualitatively (by how CONTRACT was discussed with families in recruitment conversations), we have examined the effect of retraining quantitatively. Across the three phases of CONTRACT, recruitment rates rose from 38% at the end of phase 1 of recruitment to 50% in phase 2 and to 62% in phase 3, a modest rise throughout the duration of the study. Table 6 provides a breakdown of recruitment rates by phase and site; recruitment rate by month is shown in Figure 13.

TABLE 6 - Number of families approached and randomised to CONTRACT, across sites and by recruitment phase

Phase	Site 1		Site 2		Site 3		Overall, by phase
	Approached (n)	Randomised, n (%)	Approached (n)	Randomised, n (%)	Approached (n)	Randomised, n (%)	Approached (n)	Randomised, n (%)
1	13	4 (31)	14	8 (57)	10	2 (20)	37	14 (38)
2	9	6 (67)	24	9 (38)	11	7 (64)	44	22 (50)
3	15	11 (73)	14	8 (57)	5	2 (40)	34	21 (62)

FIGURE 13.

Recruitment to CONTRACT, by communication substudy phases.

We used a chi-squared test for trend to investigate the relationship between recruitment rate and phase of training, with the assumption that each training episode (between phases 1 and 2, and then between phases 2 and 3) was identical. The increase in recruitment rate from phase 1 to phase 3 was statistically significant [χ²(1) = 4.06; p = 0.04].

Although we cannot solely attribute this rise in recruitment rates to a causal effect of the training, the change in how CONTRACT was discussed with families across the phases supports a beneficial effect of retraining. Furthermore, through evaluations of the training episodes provided by attendees, a trend of increased confidence in discussing various aspects of CONTRACT with increasing retraining is shown (see Appendix 2, Table 17).

Impact of communication training on CONTRACT recruitment

Informed by previous qualitative embedded studies, we identified key areas of non-optimal communication that can impede recruitment success, including use of imbalanced terminology42,54 and a lack of treatment preference exploration. 43,44 Following bespoke recruitment training, the use of imbalanced or potentially misinterpreted terminology decreased, and some health professionals began to elicit and balance families’ treatment preferences. However, health professionals did not alter all aspects of non-optimal communication: many continued to omit surgical risks because of concerns about unduly worrying families, and they rarely challenged families’ preferences for non-operative treatment owing to worries about dissuading families from participating in CONTRACT.

Similar to previous research,46,60 the current findings suggest that, although most health professionals saw the value in the research question CONTRACT was aiming to address, many had a preference for surgery. Health professionals spoke about their personal biases about treatments when interviewed, but these biases were also apparent in their communication with families when they used terms that were loaded in favour of one of the treatments, usually surgery. We identified frequently asked questions from parents and children; in the training, we highlighted and discussed these, which seemed to help health professionals feel more confident in addressing families’ questions. Treatment preference exploration has previously been found to optimise informed consent and recruitment,43,44 but some health professionals were reluctant to gently challenge or balance families’ treatment preferences owing to concerns that it might be coercive. It is currently unclear how parents and children experience treatment preference exploration and whether or not they ‘feel’ coerced by such approaches. Previous research has found that some health professionals can find approaching families about trials to be aversive, yet families were, at worst, neutral about being approached, and some were highly positive about it. 61 Further research with parents and children would help to establish whether their views about treatment preference exploration are similar to, or diverge from, those of health professionals.

We explored the effect of recruitment training on trial communication and recruitment both qualitatively and quantitatively, throughout the course of CONTRACT. The recruitment rate gradually increased throughout the recruitment phases, but it increased more markedly after the second session of tailored training. Future work is needed to robustly evaluate the impact of such recruitment training on trial recruitment and to examine sources of variation in response to training, such as site, staffing and the effect of being audio-recorded, as this was beyond the scope of the current study. Further evaluation might include a RCT of the training intervention. Our qualitative analysis indicated changes in health professionals’ communication that may explain the increases in recruitment rates that we observed. For example, many stopped using terms that could be misinterpreted. We know from health communication literature that tailored messages can be more persuasive and have a greater impact on behavioural outcomes;62 perhaps this is why we saw a cumulative, desirable effect on communication following the tailored recruitment training (sessions 2 and 3). Frequent, tailored recruitment training could have a more powerful effect on health professional communication and trial recruitment than the initial, generic training alone.

The statistical analyses of the relationship between recruitment training and recruitment rates from phase 1 to phase 3 using bivariate analyses indicated that the training was associated with an increase in the recruitment rate. However, a nested RCT of recruitment training would be needed to infer causality. Nevertheless, qualitatively, we identified changes in communication behaviour across all three sites and all three recruitment phases. We could not determine the impact of training on the communication of individual health professionals as we did not have a formal record of training attendees, nor were we able to identify recruiters from audio-recordings.

Background

A lack of knowledge and understanding regarding which outcomes are important to patients and clinicians may result in important outcomes being omitted from clinical trials. Differences in outcome selection and reporting between studies and how outcomes are defined and measured also make it difficult, sometimes impossible, to synthesise the results of studies (e.g. in meta-analysis) and apply them in a meaningful way. To address these problems, COSs have been proposed as a means of standardising outcome selection, measurement and reporting in health-care research, and in clinical trials in particular. 63,64 The development of a COS and its adoption by researchers is intended to help avoid inconsistencies in outcome selection, measurement and reporting that may otherwise exist. If an established COS is not adopted for a trial, there is a risk that suboptimal outcomes will be selected, and it is unlikely that such a trial will produce usable information. 65 Reporting outcomes in a consistent way between studies also facilitates data synthesis when combining resulting from more than one trial.

Prior to performing further efficacy studies of the treatment of appendicitis in children, it is imperative to identify the most relevant outcomes for inclusion in the design of comparative studies. This is of particular importance when evaluating a novel treatment approach, as the outcomes of importance may differ from those commonly reported with standard-of-care therapies. In particular, the pathway, complications and outcomes experienced by children and young people with acute uncomplicated appendicitis are potentially very different between those receiving non-operative treatment and those undergoing appendicectomy; therefore, an understanding of the outcomes that are important to different stakeholder groups is important.

A review of the relevant literature and electronic resources failed to identify a COS for children with appendicitis. 66 Furthermore, a wide range of outcomes were reported and a range of different primary outcomes were used across studies. 66 To advance our understanding of which outcomes are important and to fulfil an unmet need in our future research programme, we aimed to develop a COS for the measurement of effectiveness of treatment interventions in children and young people (aged < 18 years) with acute uncomplicated appendicitis.

Objectives

• To determine which outcomes have previously been reported in studies comparing treatments for acute uncomplicated appendicitis in children.

• To qualitatively explore outcomes of value to patients and parents of children who have had acute uncomplicated appendicitis, to inform the initial list of core outcomes.

• To prioritise the treatment outcomes of children with acute uncomplicated appendicitis from key stakeholder groups’ perspectives [including paediatric surgeons, general surgeons, patients (aged 12–18 years) and parents of children who have had acute uncomplicated appendicitis].

• To compare and contrast which outcomes of paediatric acute uncomplicated appendicitis treatment are prioritised by key stakeholder groups (detailed above).

• To achieve consensus between key stakeholder groups on a COS to evaluate the overall success of treatment for acute uncomplicated appendicitis in children.

Scope of the core outcome set

The COS is intended to be used to evaluate the overall success of any treatment intervention in children who are assigned a clinical and/or radiological diagnosis of acute uncomplicated appendicitis. The finalised COS includes outcome measures identified as important within 12 months of treatment initiation and longer-term outcomes, if applicable. The COS focuses specifically on treatment of acute uncomplicated appendicitis (i.e. thought to be uncomplicated at the time of treatment initiation); the treatment of appendicitis thought to be perforated (with or without abscess) and appendix mass is outside the scope of this COS.

Design overview

The development of the COS entailed three key stages:

1. developing an initial list of outcomes

2. a three-phase online Delphi consensus process

3. a consensus meeting.

Systematic review

The COMET Initiative recommends the use of systematic reviews in informing the first phase of the Delphi process. 69 Two recent systematic reviews,16,66 both led by a member of the study team, were used to inform the initial list of potential outcomes to be considered for the COS. The first review identified 115 outcomes reported in RCTs and meta-analyses of appendicitis treatments administered to children. 66 Of these 115 outcomes, 106 were combined on the basis of their similarity, to give 37 outcomes, which were mapped to Outcome Measures in Rheumatology (OMERACT) domains (Figure 14).

FIGURE 14.

List of outcomes identified from a previous review of the literature, assigned to core areas. PROM, patient-reported outcome measure; WCC, white cell count. Reproduced from Hall et al. 66 © The Authors, 2015. This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (https://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reproduced from Hall et al. 66 © The Authors, 2015. This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (https://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

The second review aimed to determine the safety and efficacy of non-operative treatment for acute appendicitis, and identified 10 articles reporting on 413 children who had received non-operative treatment. 16 We updated this review, using the same search strategy, to include further articles published between 2016 and April 2017. This additional search revealed 28 articles, but only six of these that met the inclusion/exclusion criteria were not already in the review. 16 Appendix 3, Table 18, shows the characteristics of the 10 articles identified from the initial review of non-operative treatment16 and of the six further articles identified from the updated review.

We extracted outcomes from these papers as described in the previous review of outcomes. 66 In brief, two researchers (SE and FCS) extracted outcomes from the articles; disagreements were resolved by a third reviewer (NJH). Outcomes were mapped onto the condensed list of 37 outcomes identified in the previous review. 66 We also sought to list potentially novel outcomes that could not be mapped to the previously defined list; however, no completely new outcomes were identified. The outcomes that were identified from these 16 additional papers, and how they were mapped to those previously described, are shown in Appendix 3, Table 19.

Qualitative interviews

Full methods and results of the CONTRACT communication study are detailed in Chapter 4. Although the main aims of the communication study were to optimise recruitment and to inform a future trial, we also asked patients and parents about their views and experiences of acute appendicitis to ascertain the outcomes that were of particular value to them. We drew on the preliminary qualitative results to inform the initial list of outcomes for the Delphi process. Key outcomes identified from these interviews included pain, loss of appetite, re-admission to hospital/GP visits following treatment, wound healing, child’s psychological well-being, time away from school or physical activities, and fever. Raw data underlying the way in which these outcomes were identified from interview transcripts are shown in Appendix 3.

These outcomes identified in interviews were mapped to the list generated from the systematic reviews (see Appendix 3, Table 20). All outcomes identified could be mapped to outcomes already listed, meaning that no new outcomes were defined.

Finalising the initial list of outcomes

After generating an initial list of outcomes from the systematic reviews and interviews, the outcomes list was refined with the SSAG. The descriptors for each outcome were discussed with the SSAG to improve their clarity and comprehensibility to young people and parents. Other outcomes suggested by the SSAG were also considered. From all these sources, an initial list of 40 outcomes was generated, along with descriptions (see Appendix 3, Table 21).

Methods

Delphi consensus process

An online three-phase Delphi process was conducted across the three stakeholder panels in parallel. Participants were presented with the initial list of 40 outcomes, grouped into themes. We also provided participants with an opportunity to propose other outcomes at the end of Delphi process phase 1.

Outcome scoring and consensus definition

In all three phases of the Delphi consensus process, participants were asked to score each outcome using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) scale. 71 The scale was presented in the format 1–9, with 1–3 labelled ‘not important’, 4–6 labelled ‘important but not critical’ and 7–9 labelled ‘critical’.

When scoring each outcome, surgeon participants were asked the key question ‘How important do you consider the following outcomes to be when considering which treatment to offer children with uncomplicated acute appendicitis?’. A similar question was posed to parents and patients, with the wording altered as informed by our SSAG. Figure 15 displays a screenshot of the Delphi process phase 1 survey and shows how the outcomes were presented to participants.

FIGURE 15.

A screenshot of the Delphi process phase 1 survey.

‘Consensus in’ was defined as ≥ 70% of participants rating the outcome as 7–9, and < 15% rating it as 1–3. Outcomes were defined as ‘consensus out’ if ≥ 70% of participants rated it 1–3 and < 15% rated it 7–9. Outcomes not meeting these definitions were classified as ‘no consensus’.

Delphi process: data collection

Following registration, participants were sent a personalised link to access and complete the first phase of the Delphi process. Software for a COS Delphi process hosted on a secure server, developed by the National Perinatal Epidemiology Unit (University of Oxford, UK) and used successfully to develop two previous paediatric surgical COSs,72,73 was used to administer all three phases. Procedures were implemented throughout the Delphi process to attempt to limit attrition,70,74 such as sending reminder e-mails and newsletter summaries of the study progress. We aimed to send the link to the first phase to all participants on the same day that they registered,75 but this was not feasible owing to an administrative problem. The link to the first phase was sent to all participants concurrently.

Participants were asked to complete each phase of the Delphi process within 3 weeks, and two reminder e-mails were sent to non-responders during that time. Participants who did not complete the questionnaire within 4 weeks of being requested to do so were deemed not to have completed that phase.

Delphi process: phase 1 data analysis

When possible, we recorded the number of participants who were invited to register and, of those who registered, the number from each stakeholder panel who completed phase 1. The scores for each outcome were analysed for each stakeholder panel, and descriptive statistics were generated. All outcomes were carried forward to phase 2. Two members of the COS study team reviewed additional outcomes that participants had proposed at the end of phase 1 to consider if they represented new outcomes. Those that were deemed to be new were included in phase 2 if they were proposed by at least two participants. Participants were sent a study ‘newsletter’ prior to phase 2, which provided feedback from the previous phase and instructions on how to complete the forthcoming phase. Separate newsletters were tailored to the different stakeholder panels and e-mailed to participants prior to phase 2 [see the project web page: www.journalslibrary.nihr.ac.uk/programmes/hta/1419290/#/ (accessed February 2020)].

Delphi process: phase 2

All participants who completed phase 1 were invited to participate in phase 2. Participants were individually presented with their own score for each outcome from phase 1, alongside the distribution of scores for each outcome from their stakeholder panel in phase 1. They were asked to rescore each outcome, taking into account the views of the other participants in their stakeholder panel. Participants were asked to score any new outcomes identified in phase 1. Figure 16 displays a screenshot of the phase 2 survey and shows how the scores from phase 1 were presented to participants.

FIGURE 16.

A screenshot of the Delphi process phase 2 survey. The gold circle around the radio button indicates the score given by the participant in phase 1. The blue bars display the distribution of scores for the given stakeholder panel, and the blue dot represents the panel’s mean score.

Delphi process: phase 2 data analysis

The data analysis process described for phase 1 (see Delphi process: phase 1 data analysis) was repeated. Any outcomes that met the criteria of ‘consensus out’ were removed from the outcomes list prior to phase 3. All other outcomes were carried forward to phase 3. Again, participants were sent a study ‘newsletter’ prior to phase 3, which provided feedback from the previous phase and instructions on how to complete the forthcoming phase. As with phase 1, newsletters were tailored to the different stakeholder panels and e-mailed to participants.

Delphi process: phase 3

Participants who completed phases 1 and 2 were invited to participate in phase 3. Owing to attrition among patients and a desire to ensure that the final COS represented the views of children and young people as much as possible, all patients who completed phase 1 were invited to complete phase 3 regardless of whether they had completed phase 2. The data collection process described for phase 2 (see Delphi process: phase 2) was repeated; however, as well as being shown scores for their own stakeholder panel, participants were shown scores for every other stakeholder panel separately. This allowed participants to consider other stakeholder panels’ views before rescoring the outcomes. 76 Figure 17 displays a screenshot of the phase 3 survey, showing how outcomes were presented to participants. We provided participants with instructions on how to complete the survey, highlighting that, on each outcome chart, the bars represented the distribution of scores among surgeons (blue bars), patients (green bars) and parents (orange bars). We also explained that mean scores were shown for surgeons (blue dot), patients (green diamond) and parents (orange square).

FIGURE 17.

A screenshot of the Delphi process phase 3 survey.

Delphi process: phase 3 data analysis

The data analysis process described for phase 2 (see Delphi process: phase 2 data analysis) was repeated. All outcomes from phase 3 were carried forward to the consensus meeting.

Attrition analysis

To address the potential for bias due to between-phase attrition, we compared (1) phase 1 outcome scores of participants who completed phases 1 and 2 with the scores of those who completed phase 1 only and (2) phase 1 outcome scores of participants who completed all three phases with the scores of those who completed phase 1 only. This was done using Mann–Whitney U-tests for each individual outcome. 72 The effect of attrition was also analysed across all outcomes using a multilevel modelling approach (level 1: outcome, level 2: participant and level 3: stakeholder panel) using MLwiN version 3.01 (Centre for Multilevel Modelling, University of Bristol, Bristol, UK).

Face-to-face consensus meetings

The aim of the consensus meeting was to ratify outcomes for which consensus (‘in’ or ‘out’) had been achieved, to discuss outcomes for which consensus could not be achieved and to finalise the COS. We invited all participants who completed all three rounds of the Delphi process to the consensus meeting and, owing to attrition during the Delphi phases, all children and young people who registered, to ensure that their views were represented. We aimed to have a minimum of 40 stakeholders confirm their attendance, with equally weighted participation across the three panels. Representatives from each stakeholder panel were required for the consensus meeting to be quorate.

Delphi phases’ results

Consensus meetings’ results

Results of voting and review of outcomes

Finalising the core outcome set

Following both consensus meetings, the COS study team verified the results and finalised the COS. Figure 18 shows the study flow from the list of initial outcomes to the final COS. Overall, we aimed to achieve a manageable COS with a maximum of approximately 10 outcomes. For the final COS, surgeons and families mutually agreed on 10 outcomes to include in the COS. Additional outcomes voted ‘consensus in’ by only surgeons included ‘hospital length of stay’ and ‘time away from full activity’, whereas additional outcomes voted ‘consensus in’ by only families included ‘wound complication’ and ‘patient stress’ (which parents proposed should include a measure that examines psychological impact also). Overall, the COS includes 14 outcomes.

FIGURE 18.

Study flow and finalised COS. a, Antibiotic failure only reported in studies of non-operative treatment; b, outcomes that achieved consensus among parents and patients only; c, outcomes that achieved consensus among surgeons only.

The OMERACT initiative provides a COS development framework that is useful across various health-care domains. 81 In developing the current COS, we also drew on this framework, which comprises three core domains (death, life impact and pathophysiological manifestations) and one strongly recommended domain (resource use). The framework recommends inclusion of at least one applicable measurement instrument for each core domain. It also recommends inclusion of AEs. Two researchers (FCS and SE) categorised the core outcomes by domain, and disagreements were resolved by a third researcher (NJH) (see Appendix 3).

Background

Central to the design of any RCT is the identification of an appropriate primary outcome. Historically in research, investigators have used a primary outcome that is most relevant to them in determining a specific aspect of their investigation in which they are most interested. More recently, greater importance has been placed on selecting a primary outcome that has relevance to a wide range of stakeholder groups that have an interest in the condition and treatments being investigated. Design of an appropriately powered RCT of appendicectomy versus non-operative management is especially problematic as the two arms may have markedly different outcomes. The outcomes from appendicectomy are likely to be superior to those from non-operative management if any of the usual measures are chosen as a single primary outcome. In the context of the current study, therefore, we propose that the primary outcome for a future RCT should be relevant not just to surgeons who care for children with acute uncomplicated appendicitis, but to a wider range of interested parties including, most importantly, the patients themselves and their families, and those delivering and funding health care. Consistent with our intention to deliver a future RCT that is pragmatic, we aim to ensure that the primary outcome selected is meaningful to patients and parents in particular, but that it also provides relevant information to surgeons.

Methods

We took the opportunity of having contact with a range of stakeholders in the COS development process to survey preferences for a candidate primary outcome in a future trial. We then held a discussion related to the survey responses at both COS consensus meetings to try to reach agreement among those present on a primary outcome that met the criteria of being important, relevant and acceptable. At the surgeon consensus meeting, we also discussed what an acceptable effect size for this putative primary outcome might be, on the assumption that a future trial would be designed on a non-inferiority basis.

Consensus meeting discussion: primary outcome selection

Initial discussion in the surgeons’ meeting was that they would want all data relating to important clinical outcomes specific to each treatment approach to be reported as a composite primary outcome. In the appendicectomy arm, this would include complications related to surgery, and in the non-operative treatment arm, this would include antibiotic failure and recurrence rate. Of note, negative appendicectomy was not considered to be important by the surgeons in this context. They also felt that it would be beneficial if any such composite primary outcome was defined in terms of defining treatment ‘success’ as opposed to treatment ‘failure’. Overall, however, the surgeons were attracted to the idea of trying to identify a single outcome that could be measured meaningfully in both treatment groups, and preferred this to a composite primary outcome as long as treatment-specific outcomes were also reported (as secondary outcomes). Some suggested that QoL would make a good secondary outcome if non-operative treatment was found to be non-inferior to surgery in terms of treatment success. Hospital length of stay and re-admission to hospital were also proposed as potential primary outcomes that could effectively assess the impact of both non-operative and operative treatment. The most popular single outcome was time to return to normal (premorbid) activity. Of note, this is not an outcome that was considered in the COS development process. It was noted that this would be difficult to measure as the premorbid state would not have been measured. It was also highlighted that this may be analysed on a superiority basis rather than on a non-inferiority basis.

In the young people and parents’ meeting, the initial preference was for a single primary outcome: QoL was identified as the forerunner. The group clearly understood the challenge in selecting a primary outcome that was meaningful to both groups but that was prevalent enough in this population to be distinguishable between patients treated with either non-operative treatment or appendicectomy. However, the group identified that QoL is difficult to measure and had concerns about the appropriate time to measure QoL, and hence, on balance, considered it less appropriate. The investigators mentioned that, in the CONTRACT study, QoL measures had been taken at various time points, meaning that data on the appropriate time to measure QoL might be available. The group discussed length of stay in hospital as a suitable primary outcome, as it reflects treatment success for both non-operative and operative treatment. However, on balance, this was rejected as the group was less concerned about the time it took to recover (so long as it was within a reasonable timeframe) than about recovery being complete. They were also concerned about not using the treatment-specific outcomes as the main determinant of treatment success and discussed this in some detail. On balance, they agreed that a composite primary outcome that included treatment-specific outcomes, including complications and negative appendicectomy (appendicectomy arm) and antibiotic failure rate and recurrence rate (non-operative treatment arm), was preferable.

Consensus meeting discussion: non-inferiority margin

The surgeon group considered the non-inferiority margin of a trial that was hypothetically designed as a non-inferiority trial and used a treatment-specific composite primary outcome. Although the surgeons felt that a 10% non-inferiority margin was too narrow as any such trial would almost certainly report in favour of appendicectomy for design reasons, they felt that a non-inferiority margin should not exceed 20%.

Discussion on primary outcome selection

In this work, we aimed to identify a primary outcome for use in a future effectiveness trial comparing non-operative treatment with appendicectomy in children with uncomplicated acute appendicitis. The identification of a primary outcome is an important step for any RCT. Along with the anticipated effect size, it is a key determinant of the sample size required. Perhaps more importantly, however, the primary outcome is the outcome that will be used to arrive at a decision on the overall result of the trial by those interested in its findings. In the past, this ‘headline’ outcome was typically the outcome that the investigators were most interested in. This approach risks using an outcome that is not interpretable by anyone other than the investigators and their stakeholder group; therefore, guidance90 suggests that greater emphasis should be given to selecting an outcome of greater interest to a broader range of stakeholders, including patients and, in the case of children, their families. The PRagmatic Explanatory Continuum Indicator Summary-2 (PRECIS-2)91 framework, which provides guidance on the design of pragmatic trials, recommends the selection of an outcome that is of obvious importance from the perspectives of all stakeholders. In designing a future effectiveness trial, therefore, we seek a primary outcome that is relevant and meaningful to patients, parents and surgeons, that is all the groups that we anticipate will be interested in the results of the trial.

To our knowledge, no precise guidance exists on how to identify a primary outcome with these characteristics. We therefore sought the opinions of multiple stakeholder groups that we were already in contact with through our COS development process. Through a survey and subsequent discussion, we have understood the views of important stakeholder groups. In the quantitative survey, it was evident that the majority of respondents identified a condition-specific measure as being the most important. Antibiotic failure was the outcome included by the most people as at least part of a primary outcome. A combination of at least one of antibiotic failure, recurrent appendicitis or re-admission appears to satisfy 84% of respondents.

Interestingly, in discussions, other less disease-specific outcomes were considered. The appeal appeared to come at least partly from the fact that a single outcome could be satisfactorily applied to both treatment arms. Eventually, one of these non-specific outcomes was preferred by surgeons over the disease-specific parameters. There was agreement that time to return to normal (premorbid) activities was a good marker of recovery from illness. Surgeons identified this as being important in discussion with one another, whereas previously, in a survey setting, disease-specific outcomes had been preferred. Parents and young people, on the other hand, having considered generic outcomes in discussion, felt that the disease-specific outcomes were more important to them in this context.

How to identify the target effect size in a clinical trial has recently been the topic of the Difference ELicitation in TriAls (DELTA²) guidance on how to determine this. 92 The authors acknowledge that deciding on a non-inferiority margin in a non-inferiority trial is a controversial topic. One proposal is that the non-inferiority margin be set as the largest difference that is clinically acceptable, so that a difference bigger than this would matter in practice. 93 Implicit in this is the concept that different stakeholders may have very different views on the biggest difference that would matter in practice.

This is perhaps particularly true when the treatments being compared are very different, as in the trial we propose. This is because the potential adverse effects (or benefits) of the treatments may be very different, giving rise to a potential ‘trade-off’ of one treatment against the other. For instance, those who are keen to avoid appendicectomy owing to the risk of perioperative complications may be willing to accept a reduction in efficacy of treatment (with non-operative treatment) in the interest of realising the potential benefit they seek. What margin of reduction in treatment efficacy they would be willing to accept may vary from person to person and will almost certainly vary between patients, parents and surgeons. Seeking a non-inferiority margin that is truly acceptable to all stakeholders may be challenging.

Resource use, valuation and costs

In this feasibility study, health-care service use was measured for each participant using clinical records, case report forms completed by research nurses and the Client Service Receipt Inventory (CSRI)99 completed by parents/carers. In our empirical investigation, we sought to estimate and assess (1) the level of agreement between data sources, (2) the quality of the data and (3) the level of precision and impact in terms of the future cost-effectiveness analysis. Figure 19 presents the different costing methods and data sources used.

FIGURE 19.

Data sources and methods used.

We adopted a comprehensive approach of collecting data during the inpatient phase of treatment from hospital, and from the wider health-care system following hospital discharge. The following data were recorded at the following time points.

• During hospitalisation, discharge assessment, follow-up appointments and re-admissions:

  • patient clinical inventories (PCIs) that were designed to capture the full duration of resource use during hospitalisation and were informed from hospital records including in-hospital and outpatient clinical records, laboratory and pharmacy records, diaries, radiology department records and relevant correspondence. These allowed the capture of all resource use data relating to hospitalisation, discharge assessment, follow-up appointments and re-admissions (see Appendix 4, Figure 25, for an example of a PCI). In-hospital medication use was recorded from electronic case report forms (eCRFs) completed during hospitalisation by research nurses. These data were used not only to conduct microcosting, but also to assess the integration of routinely collected clinical data into research.

• Post-discharge primary care, outpatients and emergency services:

  • eCRFs completed by research nurses interviewing parents/carers following discharge at 6 weeks and at 3 and 6 months (see Appendix 4, Figure 26, for a screenshot of an eCRF). These data were used as the ‘gold standard’ against which the patient-completed questionnaires (CSRI) were evaluated.

  • Patient diary cards were used to record resource use during the 14 days immediately after discharge from hospital [see the project web page: www.journalslibrary.nihr.ac.uk/programmes/hta/1419290/#/ (accessed February 2020)]. These provided data on oral antibiotics, pain medications and anti-inflammatory medications (outpatient use). A modified version of the CSRI completed by parents/carers of participants was used to collect data on other health-care appointments and additional family-borne costs at 6 weeks and 6 months post discharge. The data collection also included reporting of days lost from work for parents and absence from school for children participating in our study [for the CSRI, see the project web page: www.journalslibrary.nihr.ac.uk/programmes/hta/1419290/#/ (accessed February 2020)].

Data

The most commonly used approach incorporating the cost of hospitalisation in an economic evaluation is to use NHS reference costs94 for the relevant Healthcare Resource Group (HRG) code. These are national average unit costs for predefined services. Microcosting is the direct enumeration and costing of every input consumed in the treatment of a particular patient. 100–102 Our intention was not to identify every possible individual cost and its values, but rather to quantify resource use by treatment pathway and define domains within the treatment pathways by recording resource use and main cost drivers, hence providing valuable information and guidance on data collection requirements for a definitive RCT, that is identifying what costs need to be collected and how these resource use and cost data can be collected. 103

Although, primarily, our interest was to define the average cost for the ‘average patient’ (a macrocosting approach), there was also interest in following a microcosting approach and extending costing by including and defining an individual patient’s resource use. This allowed us to compare the total per-case cost against the widely used NHS reference costs. 94 We also compared our microcosting-derived cost with the actual cost of treatment provided by participating hospitals. This was especially important because non-operative treatment of appendicitis is a relatively new proposition and we needed to define the treatment pathway in terms of overall resource use and costs.

Data from PCIs were used to conduct microcosting of both treatment pathways and to explore what the determinants of variation in costs between the two treatment arms are and the potential economic implications for the NHS. For the valuation of resource use, we used the unit cost for each individual resource included in the microcosting exercise; these were obtained from participating hospitals. The method adopted includes identification of services, how the service works and which components of cost are incurred during the delivery of each service. In collaboration with the clinical team, we designed and mapped processes involved in service delivery and identified relevant resource use. During the data collection forms (PCIs) design stage, we created patients’ treatment pathways following the first 10 patients randomised to the study. This allowed us to identify resource use items in each domain within different stages of the treatment pathway. The PCIs were checked and discussed with the clinical team and, following modifications, the final PCIs were piloted for all patients recruited and randomised during the second 6 months of the study across all three participating sites. This process provided us with not only the design of a detailed individual PCI, but also the reporting of a comprehensive health resource use profile. Unit cost data used in the valuation of resource use were obtained from the finance department of participating hospitals.

Following discharge, the items selected for comparison were primary care resource use, outpatient and emergency appointments from secondary care and resource use due to hospital re-admission. Data were collected using two methods: first, by interviewing parents/carers (eCRFs) and, second, parents/carers completed questionnaires on their own time (CSRI). Unit cost data used in the valuation process of the eCRFs and CSRI data were obtained from NHS reference costs94 and Personal Social Services Research Unit data. 104

The process we followed intended not only to define the need for data collection in a future RCT, but also to assess the quality of data in terms of missing values and accuracy of data. During the feasibility stage, it is not appropriate to directly compare the trial treatment arms; therefore, our work is mainly reporting in the form of descriptive statistics. This allowed us to assess data quality, identifying the most appropriate method and tools to use in a future definitive RCT. All costs are presented in 2017/18 prices; when necessary, the unit costs were adjusted for inflation using the Hospital and Community Health Service (HCHS) index. 104

Effectiveness outcomes: health-related quality of life and estimating using utility values and quality-adjusted life-years

There is a strong debate among methodologists regarding the measurement and valuation of QoL in paediatric research studies. Rich literature105–109 highlights the problems associated with quantifying QoL for children; these problems range from who conducts these assessments (self vs. parent/proxy assessment) to what are the most appropriate value sets (tariffs) to use. In the UK, we are motivated to use the EuroQoL-5 Dimensions (EQ-5D)110,111 instrument in adult populations, following recommendations by the National Institute for Health and Care Excellence. 112 This creates a bias towards using the EuroQoL-5 Dimensions-Youth version (EQ-5D-Y)113 for children as most appropriate option for this population. However, the suitability of this approach has been questioned because the EQ-5D-Y applies the same tariff (at the time of the design of this study) as the adult version of the questionnaire; the only difference is the wording of the questions to make it more accessible for self-assessment by children. In this feasibility economics substudy, we decided to collect and assess two different instruments that have been used in paediatric research. The order of the two instruments was random when completed by parents/carers, making sure that the quality of data and missing values are not influenced or biased by the order of completion. The instruments used were the EuroQoL-5 Dimensions, five-level version (EQ-5D-5L),114 which comprises five levels of response assessing five dimensions (mobility, self-care, usual activities, pain/discomfort and anxiety/depression), and the Child Health Utility-9 Dimensions (CHU-9D)115–120 paediatric questionnaire developed at the University of Sheffield, which comprises five levels of response assessing nine dimensions (worried, sad, pain, tired, annoyed, school work/homework, sleep, daily routine and able to join activities). The CHU-9D is the only health utility questionnaire that has been developed with children and the value set was obtained by a UK-based general population sample.

Our aim was to detect the effects of the interventions on HRQoL by collecting and assessing the performance of the most commonly used (in the UK) HRQoL instruments. We collected both instruments at baseline, at discharge and at 2 weeks to determine any short-term difference in HRQoL that may not be apparent in later follow-up, and then at the 6-week and at the 3- and 6-month follow-ups. Responses from the two questionnaires were used to estimate utility values and to assess the short- and long-term implications of the two treatments for HRQoL. As we stated in our health economics protocol,95 our aim was not only to assess which instrument shows superior performance in terms of sensitivity to change and quality of data in this patient group, but also to assess how the different data collection points (timing) could affect the results of a cost-effectiveness analysis.

Resource use and costs

Microcosting, source: patient clinical inventory from hospital records

Presentation of results in this section and the data collection method refers to event pathways for activity costing so that context and information are not lost in the final outcomes reported. We have therefore calculated, and report here, the estimates for each treatment profile and pathway, but we also define variation within each domain by reporting the mean and standard deviation (SD). The two treatment pathways presented are defined by the composition of events relevant to each arm. Figure 20 shows the treatment pathways as defined for this study and the classification of domains that describe the different stages of the treatment pathway. The process presented in this figure is not a clinical representation but, instead, follows the classification of resource use and costs following the same format as the tables with results. Crossover from the non-operative arm to the appendicectomy arm was possible at any point of this process if clinically necessary; in these cases, the process is identical to the one in the appendicectomy arm. Classification of service systems involved allows an itemised reporting and implies focusing on variation at individual and aggregate level by treatment arm.

FIGURE 20.

Treatment pathways for each treatment arm separated into domains. Please note that these are classification of cost domains, not clinical stages. Crossover from the non-operative arm to the appendicectomy arm could happen at any point of the process.

Health-related quality of life, health profiles, utility scores and quality-adjusted life-years

The EQ-5D-5L and the CHU-9D questionnaires were used to collect data at baseline, at discharge, at 2 and 6 weeks and at 3 and 6 months. Responses obtained using the EQ-5D-5L and the CHU-9D questionnaires were used to estimate utility values and QALYs using the health profiles reported for each participant. Figure 21 shows the health profiles from the EQ-5D-5L questionnaire for each dimension across time points and Table 12 states the estimated utility values from the EQ-5D-5L and the CHU-9D questionnaires, also showing that the completion rate for EQ-5D-5L was higher. Overall, a noticeable aspect from Figure 21, but mainly from Table 12 following transformation of health profiles into utility values, is that normalisation in terms of HRQoL and especially utility values was already achieved by 3 months for both groups, raising questions regarding the necessity to collect data beyond that timeframe. Regarding the EQ-5D-5L health profiles presented in Figure 21, it is clear that, although at baseline (dark blue bars), both groups reported low health values (higher values indicate more problems), at 3 months, both groups reported very similar health profiles (orange bars). The figure also shows that, at discharge (light blue bars), the appendicectomy arm participants reported lower health profiles than participants in the non-operative arm. Of note, this difference between treatment arms at hospital discharge was apparent for all dimensions assessed (mobility, self-care, usual activity, pain/discomfort and anxiety/depression). This is not surprising (following operation), but highlights the need to collect short-term data and to carefully consider the duration for which QALYs (area under the curve) will be estimated to assess cost-effectiveness.

FIGURE 21.

Quality-of-life scores (from the EQ-5D-5L) across time points.

The two instruments have produced relatively similar utility values. However, Appendix 4, Tables 43 and 44, shows the utility values at each time point for both instruments alongside the incremental (Δ) utility values between the two arms. It is worth noticing that even small differences in utility values can have a detrimental effect on estimating QALYs. This is due to the method used to calculate QALYs, taking the area under the curve approach, incorporating both duration and quality of life. As a result, even a small change in utility values over a long period of time may result in a significant effect in QALY terms. This is apparent in Table 13, in which the differences between the two arms is reported in QALY terms for both instruments; reporting 95% CIs shows that differences detected between the two arms are not statistically significant (including zero), and this is the same for both instruments. However, it is worth noticing that these are QALYs for the full duration of the study (6 months) where more than half of the time both groups are normalised and the true difference detected at discharge accounts for only a limited fraction of the time. An interesting aspect of these results is the need to consider very carefully the short- versus long-term implications in terms of QALYs.

Assessment of resource use and costs data collection tools and methods

The first aim of this health economic substudy was to identify what type of costs need to be collected to assess the economic implications of the proposed intervention, to assess data collection tools and to identify sources for estimating these costs. We used treatment pathways during hospitalisation and the subsequent follow-up period for each trial treatment arm to ensure that all relevant costs were included and to define the most important cost drivers to consider in future research.

Assessing the tools used for data collection, our study shows that data of superior quality were collected by research nurses during interviews with parents/carers than those collected by CSRIs completed by parents in their own time. Our study also shows that hospital records provide a valid and extremely reliable data source when used in clinical research. Although hospital records are extremely reliable, the time requirement to obtain accurate data from them remains a restrictive factor to a great extent. However, it is thought that this approach will soon be easier as the adoption of electronic health records progresses in the majority of hospitals.

Reassuringly, the cost estimates produced using microcosting were in relative agreement with the actual costs incurred as reported at hospital level. However, they were less in agreement with the NHS reference cost data. 94 One important observation from this work is that the NHS reference cost for appendicectomy varies depending on the level of complications observed – an important factor to consider if macrocosting (NHS reference costs) is to be used. Similarly, the cost of non-operative treatment is difficult to identify using NHS reference costs (as there is no specific HRG tariff for this) and this is complicated even further when non-operative treatment is unsuccessful and is followed by appendicectomy during the same hospital admission.

Our results indicate that two costs are the main drivers of overall cost: the ward stay cost and the cost of the operation. The difference in overall cost between treatment arms confirms that a health economic analysis is an important component of any future effectiveness trial. The magnitude of difference in the total cost between the treatment arms suggests that non-operative treatment may have significant positive economic implications for the NHS. However, this is first dependent on ascertaining the clinical effectiveness of non-operative treatment compared with appendicectomy.

More generally, we have demonstrated that it is possible to use clinical data as valuable resources for health economic research. We have developed and tested the functionality of using clinical data to support research outcomes. We reviewed and assessed the quality of clinical data for this purpose. A further step forwards would be to amalgamate data derived from these sources with other patient-reported outcomes to enable economic evaluations alongside health technology assessments.

Assessment of health-related quality-of-life data collection tools and methods

Our assessment indicates that both HRQoL instruments provided similar results overall, yet the instruments produce different utility values and, potentially, the QALYs produced are not comparable. In terms of the quality of data, the EQ-5D-5L presented fewer missing values, indicating that it may be more acceptable to those completing it. The key implication from our assessment relates to the most appropriate timing of the collection of QoL data in a future trial. Although differences in QoL were observed between treatment arms in the short term (at discharge and 2 weeks), these differences had largely resolved by 6 weeks, and beyond, following randomisation. A failure to collect short-term data could result in missing differences in QoL between treatment arms that could have a significant effect in assessing utility. Similarly, there may be no added benefit of collecting QoL data beyond 6 weeks, as we would not (based on our data) anticipate any change over time, and reducing the number of time points when QoL is recorded would reduce burden to participants and cost. Therefore, our assessment highlights the need to carefully consider the timing of data collection, taking into account the short- and long-term implications of the intervention under scrutiny.

Strength and weaknesses

We consider a specific strength of this study to be the detailed identification and quantification of costs incurred in each arm in our microcosting process. Our ability to measure these costs is even more important owing to the lack of unit cost data (HRG tariff) for the non-operative treatment of acute appendicitis. Another strength of this study is the use of multiple time points for the collection of QoL data. This highlighted the crucial role that the timing of QoL data collection could play in estimating QALYs when assessing the cost-effectiveness of two treatment alternatives.

A potential weakness of the study is that the data informing our microcosting analysis used unit costs from a single participating hospital. It is known that there are differences in the costs of providing similar services between different NHS trusts because of, among other reasons, differences in contractual arrangements and estates costs. Therefore, the microcosting results may not be representative nationally for all UK hospitals. Unfortunately, we were unable to obtain financial information from all participating hospitals because of non-disclosure policies relating to commercially sensitive data (e.g. contractual cost of medications). However, our results strongly suggest that the main cost drivers identified through this study are likely to be the same for other hospitals, thereby supporting our conclusions.

Respondent characteristics

A total of 137 responses were received, of which 109 were complete (i.e. the participant answered every question). A total of 106 respondents (77%) were male; 121 respondents were UK consultant paediatric surgeons (88%) and the remaining 16 were UK-based trainees.

Responses to individual questions

Data for responses to all questions are shown graphically in Appendix 5, Figures 27–34.

Involved people: our study-specific advisory group

We involved 10 children and young people who, at the beginning of the study, were aged 9–18 years. Some had a history of appendicitis, and some did not. Five parents were involved as well, which represented three parent–child dyads. All parents had children who had had appendicitis. This group was the SSAG.

Children and young people were identified through the clinical connections of the chief investigator, the London Generation R Young Persons’ Advisory Group (YPAG), and a YPAG associated with the Southampton Children’s Hospital and Wellcome Trust Clinical Research Facility. For all but one child (who was aged 18 years, and with whom the PPI co-investigator had had a prior involvement relationship via the London Generation R YPAG), the parent was approached first, usually by the chief investigator. Then, following approval to share their contact details, the PPI co-investigator contacted them to explain the study, the PPI activity that was planned and to offer the parent and their child(ren) the opportunity to ask questions about the proposed involvement. All except two children subsequently joined the SSAG. The parents of those two children expressed a strong interest in their child’s involvement, but, as discussions developed, it transpired that the parent and/or child were not sufficiently available to join, or to facilitate their child joining, the SSAG.

Meetings

We planned to meet with our SSAG nine times throughout the course of the study, to correspond with times when the project most required the perspective of children, young people and parents. This meant that we met with the SSAG more frequently at the beginning and end of the study, when we most wanted the SSAG to contribute to the recruitment materials for the trial, and the dissemination materials. Actual meeting timings, topics covered and outputs are presented in Report Supplementary Material 1. We held nine meetings with our SSAG, as we had proposed at the very beginning, with our application for funding. All SSAG meetings were attended by the PPI co-investigator and the chief investigator. Other members of the study team attended meetings as required by the needs of the overall study.

Further involvement and engagement activities

One parent joined the SMG, as a representative of the SSAG. Then, following the 19 April 2017 meeting, two young people became more actively involved in the COS development work, or substudy, taking part in monthly meetings with the COS study team. The aim of this targeted PPI was to ensure that any decisions made by the SMG or COS team about the running of the study or the design of the COS work would be as acceptable as possible to children, young people and their families, in an attempt to optimise their engagement and participation. Finally, a parent–child dyad attended the study’s final results meeting on 5 October 2018, which involved presentations from all workstreams of the project, as a small engagement exercise.

Remuneration

We recognise the value that SSAG members added to the CONTRACT study, and remunerated them following guidelines (Mental Health Research Network and INVOLVE124) and experience of the PPI co-investigator (EW). Every member – regardless of whether they were a parent or child – was given a £25 Love2shop voucher (Love2shop Business Services, Liverpool, UK) following each SSAG meeting. These are vouchers that can be used in main high-street retailers. We provided catering at all meetings of the SSAG, reimbursed travel costs and offered to pay for the cost of childcare to enable parents to attend.

On the study

As demonstrated, the PPI work in this project was embedded throughout the research cycle and, we believe, made a great impact in numerous ways. The most substantial was that the SSAG helped keep the study grounded in the interests and priorities of children and young people, and parents of children and young people who have had appendicitis or whose child(ren) may develop appendicitis in the future. Although there were never any instances when the views of the study team differed wildly from those of the SSAG members, it was especially validating when views were congruent. It was revealing that the children and young people struggled to understand the concept of a COS, giving the study team an added appreciation for the complexity in communicating such a concept, and a warning that the Delphi process might not be as straightforward as anticipated. The videos used in the COS registration process, and for the COS substudy, were significantly improved by involving the SSAG in their production.

Although we did not assess this, it is possible that participants better understood the nature of the RCT and communication substudy as a result of the recruitment video and information sheets (which were improved as a result of the SSAG), and that their consent was truly and properly well informed. It is also possible that the same is true of the surgeons, parents and children and young people who participated in COS substudy.

It is the hope of the study team that the dissemination products we co-produce with members of the SSAG will be more accessible, interesting and relevant to children, young people and parents, as a direct result of their involvement.

On the researchers

The children and young people, in particular, made a profound impression on the study team, although the parent members also had an impact. The SSAG was highly enjoyable to work with and always embraced every involvement task. It was motivating and heart-warming to see the young people develop in the group – the two youngest girls (aged 11 years when they started) spoke little in the beginning and required more drawing-out to get their opinions, yet, over time, they became the first to offer their views confidently. The group worked well together, despite the members all being from different schools and of different ages. As young people often only mix with peers their own age, and as it can be intimidating for younger children to work with older children, or frustrating for older children to work with younger children, observing the children and young people of the SSAG treating each other with respect and compromising made all the hard work of organising and planning involvement activities rewarding and validating. Despite all the children, young people and parents being busy in their lives outside the study, the fact that they made time and committed to being involved was particularly rewarding to the study team, driving further motivation to enable the PPI activity to flourish. This commitment was particularly true of one parent member who returned following maternity leave, and one of the young people who returned to the group during her break from university – possibly because they found the work important and meaningful. Both the PPI co-investigator and the chief investigator further developed their skills in facilitating PPI and engagement activities; in facilitating, in particular, a mixed group of children, young people and parents, as this was a novel experience; and in how to involve children and young people in COS development.

The study team also found the experience of working with the SSAG an immensely enjoyable one from a personal perspective. Although the SSAG activity contributed in a material way to improving the study itself and optimising its acceptability to its target participants, the process of working with the SSAG was incredibly rewarding to the research team. We believe that this additional emotional motivation cannot be underestimated when considering the value that it brings to the research team.

On the children, young people and parents

In our final meeting in December 2019, we asked SSAG members to reflect on their experiences of being involved in the study, from July 2016. We asked, ‘In a couple of years from now, what I will remember is . . .’, ‘What I would have said to a younger me, back when I was joining the group . . .’ and ‘Final comments I want to share in a publication about this project . . .’. Although not all SSAG members attended, those six who did had been the most active children, young people and parents in the PPI work of the project. Anecdotal evidence, however, from e-mails and conversations in SSAG meetings, suggests that children and young people have gained confidence in themselves and in speaking in front of a group, the ability to balance and hold several viewpoints, a honed interest in medical research, improved abilities to work in a group of dissimilar individuals and knowledge about involvement in research that has been of great interest to them.

Lessons learned

Although we had originally planned to involve children and young people from Southampton, Liverpool and London, the SSAG members were mostly based in or around Southampton. When forming the SSAG, principal investigators at all three CONTRACT clinical sites sought volunteers. However, the only volunteers who came forward were from the Southampton area. Although the representativeness and diversity of the group could be questioned as a result, the SSAG members were enthusiastic and engaged throughout. It could be argued that, in fact, we were able to develop a more intimate group dynamic, possibly due to group members’ regional proximity to each other and to the lead site, Southampton Children’s Hospital, where we held most of the SSAG meetings. It was also probably much easier to manage and co-ordinate activities this way. For our future work in this field, we will continue to attempt to achieve greater geographic diversity among PPI group members.

Keeping a group of children and young people together over 2.5 years, in a highly unique and cerebral extracurricular activity that happens at irregular time points, as young people continue to develop physically and psychosocially, can be very difficult. It requires semiregular contact, and a great deal of work on developing and maintaining interpersonal relationships. We believe that a named person who will lead on PPI activities is essential for any study such as this, to be able to develop and maintain these relationships over time. Another specific challenge of communicating with our SSAG was the need to engage in a meaningful way at an appropriate level with children, young people and adults, all of whom have different levels of understanding. During SSAG meetings, we endeavoured to use age-appropriate language for the attendees present, but were conscious of not ‘speaking down’ to adults because of a need to use language that was understandable to children and young people.

Many of the involvement tasks, for example reviewing PISs, helping to put together a recruitment video and advising on the COS Delphi process video, were fairly straightforward and easily comprehended by the children and young people. However, other aspects of the study were more difficult. For example, discussing the COS development substudy was particularly challenging. It took repeated efforts, by a few members of the study team (NJH, EW and SE), and variation in explanation of what a COS constitutes, and it is possible that a couple of the children and young people in the SSAG still did not acquire the concept.

Things we learned that worked well with the PPI in CONTRACT were maintaining regular contact with SSAG members, putting together a Doodle (Zurich, Switzerland) poll to pick future meeting dates, meeting at weekends (and once during the week, when it was the school summer holiday) and providing feedback to the SSAG members about what effect the involvement had on the study. Patients and the public are frequently dissatisfied with how little feedback and acknowledgement they are provided with, although feedback is an important component in learning and development, and in enjoyment of involvement. 125 As a result, at each meeting, we went over what had happened at the previous meeting and what suggestions the study team were able to action, and explained why suggestions were not actioned. We also repeatedly acknowledged that the group was making important contributions to the study and that members were viewed as experts on the project, and we expressed heartfelt gratitude for their involvement. We also contacted young members and parents at the same time via e-mail, so there was open and transparent communication.

Why do children and young people get involved in patient and public involvement? Is there a ‘right’ reason for involvement?

A matter of continuous reflection during the CONTRACT study was why the children, young people and parents became, and continued to be, involved. Although we cannot know for certain as we have yet to undertake our reflective exercise with the group, Johannesen126 can provide some compelling clues. These are gratitude, flattery, access, quest for answers, community, altruism/charity and agents for change. It is possible that reasons for involvement can change over the course of a project.

With so much concern over safeguarding and protecting children and young people, it can become easy to worry about children and young people having the ‘right’ reason(s) for involvement. In the past, with facilitating one child and young person involvement group for 2.5 years, it started to feel to the PPI co-investigator that some of the members attended meetings only for the vouchers received as remuneration. Even though it was not a large amount of money, £25 can be a significant amount to a child or young person. A couple of members of that group were also the children of members of staff whom the group was associated with, which called into question whether the children and young people themselves had a genuine interest in the group or whether it was an activity that the parents wanted their children to do. Although we do not have the sense that these reasons are the reasons members of our SSAG became involved or continued their involvement, we plan to explore the reflective experiences of the SSAG at a final meeting, with the aim of enhancing our future PPI activity.

It is possible that some members of the SSAG joined the group out of a sense of gratitude or duty to the chief investigator – a surgeon who treated some of the children and young people – and the PPI co-investigator – who had a previous PPI relationship with one member – as these were the individuals who recruited members to the group. However, this is an approach advocated for by the NIHR Research Design Service London. 127 The fact that all members continued their involvement over a period of 2.5 years may indicate that they joined for appropriate reasons of genuinely wanting to contribute in an involvement capacity. Moreover, the chief investigator and PPI co-investigator consciously worked to reduce any perceived power imbalances during meetings and to ensure that all members of the SSAG were not only valued, but each valued equally.

Finally, it could also be argued that, no matter what the reason for joining a study-specific involvement group of this nature, as long as people do join, continue to contribute and are meaningfully involved, then the reason does not matter. The involvement is what matters, as this very involvement enhances the research.

Optimising recruitment in trials comparing surgical treatment with non-surgical treatment

CONTRACT compared a surgical with a non-surgical treatment. Trials that include such markedly different treatments are known to be particularly difficult to conduct as a result of treatment preference issues. 34,35 In CONTRACT, this was particularly pertinent because both parents and surgeons perceived surgery as a ‘trusted’ treatment, relative to non-operative treatment. 36 Parents’ preferences for surgery were often born out of concerns about the risk of perforation. Parents viewed perforation as a dangerous event that could severely compromise their child’s condition and worried that it could occur at any time. Such concerns influenced families’ willingness to participate, as some viewed surgery as a more immediate treatment and, in their view, as reducing the risk of perforation. Longer duration of symptoms has been reported as a risk factor for perforated appendicitis,142 but no association has been found between perforation and in-hospital time prior to surgery for children with acute appendicitis. 143 This suggests that parents’ concerns about the impact of in-hospital delays on perforation were not well founded. Moreover, parents often expressed these concerns in the qualitative interviews, but not in the recorded consultations, which further underscores the importance of exploring treatment preferences with families. Exploring parents’ treatment preferences, in a future definitive trial and for other trials comparing surgical treatment with non-surgical treatment, would be beneficial in alleviating families’ anxieties and could help address families’ concerns about participation. 44

Following surgery, the details surgeons provided to parents who participated in CONTRACT of the removed appendix were often vivid. For parents of children who had complicated or perforated appendicitis, these details seemed to add to their sense of guilt or regret for allowing their child to participate in the trial. This finding has implications for other trials comparing surgical treatment with non-surgical treatment. For a future definitive trial, health professionals, including those who are involved in the child’s treatment and care but not directly involved in the trial, should be aware of the possibility of inducing or adding to such feelings of regret when speaking with parents post surgery.

• Recommendation 1: explore families’ treatment preferences in pre-consent trial discussions. In CONTRACT, this includes exploring families’ understanding of perforation and allaying their concerns.

• Recommendation 2: be aware of family sensitivities when explaining post-surgery findings in the context of a trial.

Optimising recruitment in paediatric trials in acute settings

In the communication study interviews, many families reported making a shared decision as to whether or not to participate in CONTRACT. In consultation recordings, we found that few of the younger children engaged in CONTRACT discussions and decision-making. Regardless of age, children were often unwell and so they relied on their parents to make the decision for them. These findings align with current guidance on how best to involve children and young people in research. 38

Parents described how the pain their child experienced impeded their child’s engagement in CONTRACT discussions and decision-making. Parents who declined to participate in CONTRACT also often explained that they viewed surgery as a more immediate way of relieving their child’s pain than non-operative treatment. Health professionals have been found to adopt behavioural and cognitive strategies to emotionally cope with children’s pain, which can compromise clinical communication about pain. 144 Providing families with advance information about how a child’s pain is managed in both treatment arms could help to reduce their anxiety about this.

In CONTRACT, children and parents often described conflicting treatment preferences. Some surgeons suggested that randomisation in CONTRACT offered a means of resolving this conflict. In interviews, parents and children who had conflicting preferences and who were randomised to the parent’s preferred treatment spoke of this as upsetting for the child. Families suggested that allocation discussions were generally brief and parents were often left to comfort their child. Again, exploring treatment preferences pre allocation could help to avoid such difficulties. 42 Exploring treatment preferences pre allocation could also help to avoid families withdrawing from the trial because they do not want to continue with the allocated treatment, which is one of the most common reasons for attrition in paediatric trials. 145 If a child is upset at being allocated to a non-preferred treatment, it could help to further explore the reasons underlying their preferences. If the child remains upset about the prospect of continuing in the trial, the option of withdrawal should be discussed with them. 38

As discussed, previous research suggests that exploring treatment preferences can improve informed consent and trial recruitment. 43,44 In children mature enough to understand, this exploration should probably be done for both the child and the parents. This involves recruiters communicating in ways that help to balance views about treatments and so ensure that families have the necessary information to make an informed decision. 43 It can also include providing information about treatment risks. However, some parents of younger children expressed concerns about health professionals providing such information to their children and felt that such discussions compromised their child’s trust in health professionals. When exploring treatment preferences to improve informed consent in paediatric trials, health professionals therefore have the additional complexity of needing to do so while being sensitive to parents’ anxieties about what their children hear.

Informed consent should allow sufficient time for health professionals to explain the study thoroughly and for potential participants to decide whether or not to participate. 146 However, in time-critical settings, such as urgent care, it is rarely feasible to offer the traditionally advocated 24-hour timeframe to decide. 147 Some families in CONTRACT described making an instantaneous decision to participate, whereas others felt that the time they had was not sufficient. Families typically had 1 or 2 hours to decide, but a few families were left for up to 7 hours before a health professional returned to find out their decision. Parents appreciated that such delays were often unavoidable in the busy clinical setting, but extensive delays left them anxious about the appendicitis progressing. It also compromised their sense of being cared for, which ultimately influenced willingness to participate in CONTRACT. A process that allows families to signal when they have made a decision could help to allay parents’ concerns and optimise recruitment. Further work should explore how best to implement such a process in an urgent care setting.

Finally, CONTRACT recruited children in an acute hospital setting, often outside normal hours, when research nurse support was not available. A common reason for trial recruitment failure is a lack of recruiting staff or insufficient funds to reimburse recruiting staff. 148 Although families often presented at hospital outside normal working hours and surgeons described how having research nurse support was advantageous, few health professionals suggested that recruiting outside normal working hours was detrimental to recruitment. Nevertheless, staffing strategies to support health professionals to recruit families to a future definitive trial should be considered.

• Recommendation 3: involve children and young people in research discussions and decision-making when possible, as per current guidance,38 while being sensitive to parents’ anxieties about what their children hear.

• Recommendation 4: provide families with advance information about how a child’s pain will be managed in both treatment arms.

• Recommendation 5: when child and parent treatment preferences conflict, randomisation may offer a means to resolve this conflict. Sensitivity is needed in communicating to families which treatment arm they have been allocated to; if a child is upset with treatment allocation, further exploring their treatment preference following randomisation may help to allay their concerns.

• Recommendation 6: time to decide – develop a strategy to allow families to indicate when they have made a decision regarding participation, thereby minimising delays from the perspective of families.

• Recommendation 7: consider staffing strategies to support health professionals in recruiting families outside normal working hours.

Optimising discussions about the purpose of feasibility trials

Informed consent is central to the ethical conducting of research, and conveying the ‘purpose’ or aim of a study is a necessary element of informed consent. 149 As found in a previous qualitative study embedded in a feasibility trial,150 health professionals in CONTRACT rarely described its feasibility aims. Rather, they described CONTRACT as aiming to find out how best to treat appendicitis. When interviewed, health professionals typically suggested that it was unnecessary to describe the feasibility aim of CONTRACT, adding that families might find the concept of feasibility confusing or overwhelming (especially in the urgent care context). Some health professionals thought that describing it could dissuade families from participating. In interviews, we did not ask families about their understanding of the purpose of feasibility trials and how best to discuss it because the topic was rarely mentioned in CONTRACT consultations. However, there is a need for work to explore children’s and parents’ understanding of feasibility, the acceptability of feasibility trials and optimal ways for health professionals to explain the aims of a feasibility trial, to optimise informed consent.

• Recommendation 8: further research is needed to explore how best to discuss the purpose of feasibility trials with families in acute settings.

Optimising recruitment to a future definitive trial

This section discusses findings of the communication study and final recommendations, specifically in the context of developing a future definitive trial comparing non-operative treatment with appendicectomy for the treatment of acute uncomplicated appendicitis in children and young people.

In the UK, patient history and physical examination are routinely used to determine acute uncomplicated appendicitis, but 28–57% of children aged ≤ 12 years are misdiagnosed. 151 In CONTRACT, health professionals used the same diagnostic techniques to establish whether or not children had acute uncomplicated appendicitis, as opposed to perforated appendicitis. In their interviews, they explained the difficulties of accurately establishing patient eligibility. Some proposed introducing additional diagnostic measures to more confidently diagnose children with acute uncomplicated appendicitis, for instance level of CRP, which can indicate perforation, but not as a sole indicator. 152 Diagnostic uncertainty was a core concern for health professionals. Additional strategies to improve the accuracy of diagnosis should be considered to improve families’ and health professionals’ confidence in the trial.

As described earlier, some families who experienced non-operative treatment failure had expected that surgery would be performed almost immediately after such failure had been established clinically. These findings from the communication study informed changes to the PIS. Following discussion of this issue in the recruitment training, health professionals started providing families with information on the timing of surgery if non-operative treatment were to fail and the estimated timeframe in which families could expect to see an improvement in their child’s condition after starting non-operative treatment. The study investigators may wish to emphasise this finding in training for a future trial.

Some parents suggested that there had been delays in their child receiving the initial course of antibiotics. They questioned whether or not the delay was because of the time taken to voice their decision as to whether or not they wanted to participate in CONTRACT, and indicated that the experience discouraged them from participating in CONTRACT. Health professionals should avoid delay in commencing treatment prior to CONTRACT participation whenever possible.

Finally, one of the nurses who completed blinded discharge assessments highlighted that assessors working on the same wards as CONTRACT participants may become unblinded to their treatment allocation. Ideally, assessors should be from different wards to those on which CONTRACT participants are treated. Furthermore, assessors may appreciate brief training about the study and the opportunity to participate in study events.

• Recommendation 9: consider additional strategies to improve the effectiveness of distinguishing children with acute uncomplicated appendicitis from those with perforated appendicitis in assessing patient eligibility.

• Recommendation 10: provide families with balanced information regarding both treatment arms. This should include information on the timing of surgery if non-operative treatment fails and the estimated timeframe in which families allocated to the non-operative treatment arm should expect to see an improvement in their child’s condition. This could be included in training for a future trial.

• Recommendation 11: some parents reported delays in their child starting the initial course of antibiotics, which they linked to the additional procedures required for CONTRACT, and so this discouraged them from participation. Whenever possible, health professionals should avoid delays in delivering the initial antibiotics.

• Recommendation 12: ensure that blinded discharge assessments are completed by a health professional who does not work on the same wards as those where CONTRACT participants are treated, and offer the assessor brief training on the study.