Tonsillectomy compared with conservative management in patients over 16 years with recurrent sore throat: the NATTINA RCT and economic evaluation

Economic burden of adult sore throat disease

There have been several attempts to quantify the economic impacts of sore throat. One recent study initially estimated the annual cost of consultations and lost productivity to be £40M, but concluded that the actual all-encompassing bill might be as high as £2.35B. 5 A more current estimate of primary care costs can be drawn from the pilot data of an NHS study: Sore Throat Test and Treat. 6 Extrapolation of the Sore Throat Test and Treat pilot figures to the estimated 42,000 full-time equivalent general practitioners (GPs) in England and Scotland would translate to 5.9 million consultations per annum. Taking the cost of a GP appointment from the unit costs of health and social care produced by the Personal Social Services Research Unit7 to be £34, these consultations equate to a nationwide bill of over £201M per annum. In addition, if there are antibiotics prescribed in approximately three-fifths of those attending primary care with sore throats, the total cost even of simple penicillin V at £1.25 for 28 500-mg tablets four times per day for 1 week is £19.2M. 6 The cumulative total of consultations and antibiotic prescriptions is, therefore, in the order of £219M per annum in England and Scotland. The latest available NHS Healthcare Resource Group data give an expenditure of over £70M for tonsil surgery per year in England, in all ages. Between 2013 and 2017, there were 4600 tonsillectomies per year on average, in Scotland,8 costing an additional £7.5M, although the emergence of the Scottish Intercollegiate Guidelines Network (SIGN) guidance has been reported to have impacted differently in England and Wales than in Scotland. 9

Numerous additional interventions take place in the independent sector that are not recorded in NHS digital data. The cost of pharyngeal inflammation emergency admissions, as outlined below, exceeds £20M. Adding these secondary care episodes to those in the community gives an overall, all-age cost of £309M.

Association over time between the rate of surgical tonsil intervention and admission to hospital with pharyngotonsillar inflammation

Several reports12 indicate that as the UK rate of tonsillectomy has fallen, the number of acute admissions has risen, although no causality has been proven in this association. For example, looking at English Hospital Episode Statistics data,13 in 2009–10 there were 21,540 tonsillectomy operations in adults aged ≥15 years, and around 15,000 emergency adult pharyngotonsillitis admissions. In 2019–20, there were only 16,000 adult tonsillectomy operations, whereas emergency adult admissions in England with a main diagnosis of pharyngitis and tonsillitis had risen to over 30,000. 11 A similar reduction in surgical intervention was seen in recent years in Germany, where the tonsillectomy rate fell by 46% in males and 40% in females from 2005 to 2017. 14

Sore throats and recurrent tonsillitis

‘Sore throat’ refers to pain in the pharyngotonsillar area and can be caused by viral or bacterial infection, or by non-infective irritants, such as smoke and hay fever. ‘Tonsillitis’ is a subset of sore throats in which there is inflammation of the tonsils, and this can range from relatively mild symptoms associated with viral infections to bacterial tonsillitis and quinsy (peritonsillar abscess). The pathway through which patients are offered tonsillectomy is inevitably predicated on patient complaints, not only sore throat but also, as our own NATTINA patient and public involvement (PPI) group advised, the concomitant experience of general illness and malaise. It is the systemic nature of the illness and the impact on the patients’ quality of life (QoL) that leads patients to seek treatment15 and, eventually, to be referred to secondary care. Our PPI group reported varied access to antibiotic therapy for tonsillitis in primary care.

In the USA, the term ‘strep throat’ is widely used to distinguish between bacterial tonsillitis caused by group A Streptococcus and viral (usually milder) tonsillitis, but this distinction is less commonly made in the UK, particularly in primary care. Although anyone (with tonsils) can experience tonsillitis, a subset of people experience multiple episodes of acute tonsillitis over an extended period of time: recurrent tonsillitis. Recurrent tonsillitis is more common in females, and studies in twins demonstrate a strong hereditary element. 16 A failure to differentiate between different forms of sore throat may have contributed to an inadequate understanding of the features and impacts of recurrent tonsillitis and limited the development of effective alternatives to antibiotics and tonsillectomy, in spite of the high prevalence and morbidity associated with recurrent tonsillitis. Recent work has identified specific genetic variations in the immune system that are associated with a risk of recurrent tonsillitis, suggesting potential future treatment targets. 17 In NATTINA, consistent with SIGN18 and National Institute for Health and Care Excellence (NICE) guidance,19 we refer throughout to ‘recurrent sore throats’.

Role of tonsillectomy in the management of recurrent sore throats

The optimum role of tonsillectomy in the management of adults with recurrent sore throat remains uncertain. Audits consistently report over 95% of patients gaining symptom relief from surgery,20,21 but a lack of level I evidence22,23 contributes to ongoing UK regional variation in tonsillectomy rates,24 even where compliance to SIGN guidance18 approaches 90%. 25 The 2014 Cochrane review22 identified only one evaluable, small, adult trial comprising 70 participants26 with only 90 days’ follow-up. It concluded that reasonable levels of evidence were available for tonsillectomy in children only. Since then, there has been a further small (≈ 40 per arm) Finnish tonsillectomy trial,27 which demonstrated no impact on the primary outcome, that is the proportion of patients with severe inflammation of the pharynx within 5 months. However, there were 10 times as many consultations for pharyngitis in the conservative management arm.

Except in rare situations (when tonsillar material remains or regrows), tonsillectomy should remove the possibility of tonsillitis, but its impact on non-tonsillitis sore throats, if any, is unknown. The questions that patients, doctors and healthcare providers wish to answer relate to the relative costs, risks and benefits of tonsillectomy, which must be weighed against those of conservative alternative treatments for recurrent sore throat. Research may also usefully establish whether or not more refined surgical indications could maximise the benefit–risk ratio.

The risks of tonsillectomy

The most common complications of tonsillectomy are pain and bleeding. From 1995 to 2010, there were 40 claims of clinical negligence related to NHS tonsillectomy, most commonly bleeding related and followed by nasopharyngeal regurgitation. 28 Less common but intrusive complications include changes in taste29 or tongue sensation. 30 In the Swedish 2012 audit,21 almost 14% of patients had an unplanned postoperative clinic visit. A Danish study of 614 outpatient tonsillectomy patients of all age groups identified unscheduled postoperative contacts in 23% of patients, about half because of pain. Contacts were more frequent when discharge occurred in under 4 hours. 31 Comprehensive ascertainment of complications may be difficult precisely because patients are discharged promptly, increasingly on the day of operation; thus, not every complication will involve a hospital contact. Supplementing postal surveys by telephone calls can increase response rates on postoperative events following tonsillectomy by about 50%. 32

Conservative therapy for recurrent sore throats

Assessment of baseline sore throat severity

Our previous North of England and Scotland Study of Tonsillectomy and Adenotonsillectomy in Children (NESSTAC)3,64 showed that the procedure reduced the prevalence of sore throats during the 24 months following tonsillectomy, but did not allow accurate mapping of outcomes to baseline severity. Baseline severity in tonsillitis, as in many other conditions, is an important predictor of outcome and may be the main determinant of patient satisfaction with tonsillectomy. 65 NESSTAC’s inability to stratify outcomes according to disease severity or QoL impact significantly curtailed the trial’s impact on clinical decision-making.

The assessment of baseline severity in tonsil disease has been simplified by the advent of the Tonsillectomy Outcome Inventory-14 (TOI-14) patient-reported outcome (PRO). The TOI-14 depicts the health-related QoL of patients with chronic tonsillitis and is validated for the German population. 66 The English TOI-14 paediatric version has also been shown to have good psychometric characteristics. 67 The TOI-14 adult version invites patients to reflect on the prior 6 months when responding. It is a straightforward PRO with appropriate face validity. We have shown the TOI-14 to have discriminant validity in a comparison of recurrent tonsillitis patients with healthy volunteers68 and in peritonsillar abscess. 69,70 Importantly, tonsillitis is a disease primarily of younger adults, usually with education or workplace activity profiles. Small descriptive series report substantial benefits in reduction in days lost from work or education following tonsillectomy. 71

The NATTINA RCT was a mixed-methods study, incorporating a feasibility study, an internal pilot and a Phase III multicentre trial, randomising patients to immediate tonsillectomy or conservative management. Access to tonsillectomy in the UK is currently governed by application of SIGN/NICE guidance. This mandates that qualifying sore throats are well documented, clinically significant and adequately treated. The ‘required’ number of episodes for consideration of NHS tonsillectomy is seven in the previous year, five in each of the two consecutive previous years, or three in each of the three preceding years. 18 Our PPI NATTINA preparatory work reported varied access to antibiotic therapy for tonsillitis in primary care, possibly as some clinicians’ therapeutic research goal has been ‘sore throat days saved’. In contrast to professionals, patients reported that they are often looking to minimise days of feeling ill rather than days of sore throat. Thus, any tonsillitis/pharyngitis treatment trial needs to reflect a wider range of outcomes than a simple count of sore throat days. 54

Research is required in the form of a modelling approach to identify what optimal treatment strategies might be and to inform future research. Patients in NATTINA were asked to identify at the time of a reported sore throat episode whether they regarded it as being (1) mild, (2) moderate or (3) severe. This distinction was supported by work in primary care showing that patients’ own perception of sore throat severity was the single best predictor of streptococcal tonsillitis. 72

Context

Feasibility study aim and objectives

The aim of the feasibility study was to address the key methodological issues raised via our PPI group and clinicians.

The objectives of the feasibility study were as follows:

• Evaluate otolaryngologists’ and recruiting specialist nurse practitioners’ willingness to randomise and randomly allocate patients to the randomised arms, and patients’ willingness to be randomised, taking account of predicted variation in severity of sore throat.

• Identify barriers to and facilitators of, and propose strategies to address, the above items.

• Define clear-cut eligibility criteria for a trial acceptable to all stakeholders.

• Assess the acceptability of the usual-care randomised arm to patients whose primary care clinicians have referred them for specialist intervention. In addition, to review primary care clinicians’ willingness to refer.

• Provide further information to maximise patients’ equipoise when they present in the tonsil study secondary care clinic, including meetings/seminars with research participants from other sites.

• Investigate the feasibility of our proposed data collection methods and outcome measures.

• Explore illness features of concern to our PPI group to ensure that these were captured in trial outcome measures.

• Develop and test sore throat weekly data collection methods and storage, and Sore Throat Alert Return (STAR).

• Explore the processes of patient identification and recruitment.

• Develop patient recruitment materials, including the production of the standardised NATTINA randomisation online video-recording/digital versatile disc (DVD). 77

Patients who met the proposed inclusion criteria were identified and invited by study otolaryngologists to participate in the feasibility study (involving qualitative interviews, role play and cognitive interviews).

Internal pilot objectives

The NATTINA internal pilot target was 72 patients randomised in 6 months among six of our nine sites. In addition to assessing the ability to recruit, the internal pilot aimed to:

• ascertain whether or not all of the trial processes, including patient identification, eligibility criteria, randomisation and data collection processes, work as intended and cohesively

• gauge more precisely the number of potentially eligible patients identified in NATTINA screening clinics

• investigate referral, recruitment and acceptability across the baseline severity spectrum

• identify barriers to patient recruitment, suggest improvements to have an impact on recruitment rates and measure patient compliance in weekly submission of days of sore throat, plus STARs, during sore throat episodes

• identify any major emerging systematic differences between recruited patients and patients who declined to participate

• collate and report reasons for ineligibility/non-consent

• quantify missing data and measure attrition in sore throat data

• review activity against the go criterion – six screening clinics established, with target throughput of 396 eligible patients screened in 6 months and a minimum target of 72 randomised patients.

The structure of the internal pilot was, however, eventually abandoned, as described in Chapter 2, owing to unexpectedly severe barriers to recruitment, particularly in certain regions of England where Clinical Commissioning Groups (CCGs) appeared to be following an audit commission health briefing, which labelled tonsillectomy among a number of ‘relatively ineffective procedures’. 4

Main trial objectives

Inclusion criteria

• Aged ≥16 years.

• Recurrent sore throats that fulfil current SIGN guidance18 for elective tonsillectomy. This guidance stipulates that to be eligible for tonsillectomy on the NHS sore throats should be a result of acute tonsillitis and that:

  • the episodes of sore throat should be disabling and prevent normal functioning

  • the patient has experienced seven or more well-documented, clinically significant, adequately treated sore throats in the preceding year, five or more such episodes in each of the preceding 2 years, or three or more such episodes in each of the preceding 3 years.

• The patient has provided written informed consent for participation in the trial prior to any trial-specific procedures.

Exclusion criteria

• Previous tonsillectomy.

• Listed directly (i.e. added to waiting list without prior elective ENT outpatient appointment) during emergency admission (e.g. owing to peritonsillar abscess/quinsy).

• Primary sleep breathing disorder.

• Suspected malignancy.

• Tonsilloliths (as primary referral).

• Pregnant or breastfeeding.

• Bleeding diathesis (including haemophilia, sickle cell disease and platelet dysfunction).

• Therapeutic anticoagulation.

• Inability to complete self-reported questionnaires and STARs.

Tonsillectomy

Participants randomised to the tonsillectomy arm received elective surgery to dissect the palatine tonsils, preferably within 6 weeks and no longer than 8 weeks following randomisation. The method of the surgical procedure was at the discretion of staff at the participating sites and was identical to that in standard care. Except for randomisation allocation, participants in both arms received standard non-surgical care, as per local policies. Participants who were randomised to undergo a tonsillectomy, but whose surgery was delayed owing to severe tonsillitis or other reasons, remained in the trial and continued to follow the surgery pathway.

Conservative management

Participants randomised to the conservative management arm had the option of deferred surgery for up to 24 months in addition to standard non-surgical care. A 12-month face-to-face review allowed confirmation of willingness to remain in the conservative management arm. This pathway was informed by the NATTINA PPI group and the qualitative feasibility study for NATTINA. 80–82 A participant contemplating surgery was required to fulfil the SIGN guidance18 at the time point of review to be considered for tonsillectomy.

Participants in the conservative management arm received standard non-surgical care, which typically comprised self-administered analgesia plus or minus ad hoc primary care prescription of antibiotics, and attendance at walk-in centres or the accident and emergency (A&E) department for more severe episodes of sore throat. Participants were permitted to cross over to the tonsillectomy arm at any point in the trial on the condition that they continued to be eligible for surgery in accordance with the SIGN guidance. 18

Funding of the trial intervention

Funding for the surgery was not required, as the procedure was a part of the usual standard-care pathway for this cohort of patients.

Primary outcome

The primary outcome was the number of sore throat days collected through weekly ‘returns’ from the participants over the 24-month period following randomisation. The data allowed comparison of tonsillectomy with conservative management to determine their effectiveness in treating recurrent sore throat.

Secondary outcomes

The secondary outcomes were:

• Clinical effectiveness. Intergroup comparison over the 24-month follow-up period of:

  • STAR severity scores of any sore throats experienced.

  • TOI-1466,68 scores collected 6-monthly. The TOI-14 is a 14-item questionnaire with a supplementary final open response item for additional symptoms that are related to tonsillitis over the preceding 6 months. 84 We have shown the TOI-14 to have discriminant validity in a comparison of recurrent tonsillitis patients with healthy volunteers85 and in peritonsillar abscess. 69,70

  • An adjusted estimate of intervention effectiveness in the light of baseline covariates, including severity of tonsillitis, estimated from TOI-14 baseline scores and categories.

  • Intergroup comparison of QoL as recorded over 24 months by the SF-12. 83 This 12-item Short Form Survey is an abbreviated Short Form questionnaire-36 items generic QoL assessment tool, covering the same eight health domains of physical and mental health over the previous 4-week period.

  • The impact of alternative NHS sore throat patient pathways by observation and statistical modelling of outcomes.

  • To assess to what extent trial participants are representative of the total population of sore throat patients referred to ENT clinics through interrogation of site screening logs.

• Economic evaluation. To compare the following between the two arms over 24 months:

  • Costs. The average total cost per participant to the NHS and PSS to manage recurrent sore throats. Participant costs were considered in a sensitivity analysis.

  • QALYs using the area-under-the-curve method based on SF-6D scores derived from the SF-12 responses. 86

  • Willingness to pay to avoid a sore throat day. A contingent valuation questionnaire was administered at baseline to quantify the effect that a sore throat day has on individuals’ QoL in monetary terms.

  • The number of AEs, visits to the GP/walk-in clinic/A&E, prescriptions issued and additional interventions required for sore throats and related events through STAR/STARLET data, and supported by data linkage to primary care patient records.

  • Cost-effectiveness analysis (CEA) – incremental cost per sore throat day avoided.

  • Cost–utility analysis (CUA) – incremental cost per QALY gained.

  • Cost–benefit analysis (CBA) – incremental net benefit.

• Qualitative process evaluation. To document the views and experiences of patients and clinicians regarding tonsillectomy and conservative management, and how patient experience may shape future research (see Chapter 5).

• Future research. To propose further research questions using newfound cost–benefit information to develop algorithms that guide and assess management of health services.

Secondary outcome data collection

The questionnaires used for secondary outcome data collection can be found on the project webpage (https://fundingawards.nihr.ac.uk/award/12/146/06; accessed 29 September 2021). Participant-reported outcomes were collated into questionnaire packages for each 6-month contact (Figure 1 and Table 1).

FIGURE 1.

Study pathway. PIS, participant information sheet; Q&A, question and answer.

TABLE 1 - Details of questionnaires included in the packages at each follow-up contact

At the 18-month visit, the Participant Time and Travel questionnaire was also completed, but not by all participants as it was removed in an attempt to increase the response rates of the postal 18-month questionnaire in substantial amendment 9. Schedule amendments are given in Appendix 1, Table 28. Those undergoing surgery were telephoned at 1 and 2 weeks to record any related AEs or serious adverse events experienced.

Visit	Questionnaires collected at each time point
	TOI-14	SF-12	Health Service Utilisation	About You	Value of Avoiding a Sore Throat Day	Participant Time and Travel
Baseline	✗	✗	✗	✗	✗
6 months	✗	✗	✗
12 months	✗	✗	✗
18 months	✗	✗	✗			✗a
24 months	✗	✗	✗

The ‘About You’ set of questions collected data about participants’ ethnic origin, education and employment. Primary healthcare service data were accessed from the participants’ general practices following the completion of the 24-month follow-up visit, including the 12 months prior to randomisation. NCTU initially sent a GP linkage case report form for completion by the practice for each participant, along with a prepaid return envelope. A move to an electronic process achieved a higher response rate. The participant record report form defined the time window of enquiry – 12 months prior to 24 months post randomisation – on the following episodes for a sore throat or related event:

• attendance at a general practice (with a GP/nurse)

• telephone call with a GP/nurse from the practice over the telephone

• attended a walk-in clinic

• attended A&E

• attended an outpatient appointment

• admitted to hospital

• had any other intervention or contact with a healthcare service

• any medications prescribed.

The purpose of collecting these data was to capture participants’ consultation rates, outcomes, prescribing information and additional interventions required for sore throat or related events throughout the period stipulated above. GP practices were paid £50 for each participant return.

Feasibility study

The aim of the feasibility study was to assess the willingness of clinicians to refer and randomly allocate patients to the study arms, patients’ willingness to be randomised and the acceptability of the deferred surgery randomised arm.

Qualitative and cognitive interviews were carried out by a researcher from Newcastle University. The study was planned to last for approximately 5 months, or until data saturation, and to take place over nine sites.

Fifteen ENT patient interviews were planned: these were with adult patients with acute tonsillitis who had been referred to ENT outpatient clinics for recurrent sore throat and who met the eligibility criteria (inclusion and exclusion criteria for the feasibility study were the same as for the main trial listed in Participants).

Twenty ENT staff interviews were planned for ENT staff members at one of the nine participating sites that were likely to be involved in the proposed NATTINA trial. Ten GP interviews were planned, with no eligibility criteria for this group of participants.

The outcome measures were:

• Patient identification and recruitment, eligibility criteria and ENT clinic staffs’ willingness to randomise and randomly allocate patients to the randomised arms.

• Primary care clinicians’ willingness to refer.

• Patients’ willingness to be randomised and their acceptability of the conservative management arm for patients whose primary care clinicians have referred them for specialist intervention. Feedback on proposed data collection methods, including weekly sore throat alerts, STARs and questionnaires, was also collected.

In total, 15 ENT patients, 11 GPs and 22 ENT staff were recruited.

The success criteria for progression to the internal pilot study were met, with a plan to address potential barriers to recruitment and patient equipoise. Patient participants reported that recurrent sore throats severely affected their family, work and social life; ENT staff reported that patients faced increasing barriers to secondary care service access; and GPs were guided to reduce referrals for tonsillectomy owing to it being seen as having limited clinical value. This highlighted the need for further research regarding the effectiveness of tonsillectomy for this cohort of patients.

Internal pilot

NATTINA was originally planned to include a 6-month internal pilot with six of its nine sites that were planned to be opened to recruitment. The process was overseen by the Data Monitoring Committee (DMC) and the Trial Steering Committee (TSC) and was reviewed by the NIHR HTA programme to decide whether or not to continue with the main trial phase.

The proposed ‘stop/go criteria’ to be achieved in the 6-month pilot79 were:

• six sites to be opened with screening clinics established

• 396 eligible patients screened over the six sites

• a minimum of 72 participants randomised.

Other aims were to:

• ascertain if all trial processes, including patient identification, eligibility criteria, randomisation and data collection, worked as intended and if the eligibility criteria were cohesively operational

• gauge more precisely the number of potentially eligible patients identified in NATTINA screening clinics

• investigate referral, recruitment and acceptability across the baseline severity spectrum

• develop participant recruitment materials, including the production of the standardised NATTINA randomisation online video-recording/DVD

• identify barriers to participant recruitment and suggest improvements to have an impact on recruitment rates

• measure participant compliance with the proposed weekly submission of number of sore throat days per week, plus STARs during sore throat episodes

• develop and test sore throat weekly data collection methods and storage, and STARs (participant interviews, cognitive interviews) with input from Tay Dynamic software

• explore illness features reported to us as of concern by our PPI group to ensure that they were captured in trial returns, for example chronic sore throat, night-time sore throat and disruption of ability to undertake not only usual activities but also leisure pursuits

• identify any major emerging systematic differences between recruited participants and individuals who declined to participate

• collate and report reasons for participation/ineligibility/decline

• quantify missing data and measure attrition in sore throat data.

The recruitment target of 72 participants was, however, not met in the pilot timescale. The Trial Management Group (TMG) and the NIHR HTA programme monitoring team met on 10 September 2015, 4 months from the first patient’s first visit, and agreed that the proposed nine-site model would not succeed. The conversion rate from screening to recruitment had been lower than expected, and the embedded qualitative work81,82 showed that this was because patients’ willingness to participate in the trial, particularly in certain regions, was much less than that anticipated. Some patients were having to wait so long to gain a referral to ENT clinics that they were unwilling to contemplate the prospect of further delay. The NIHR HTA programme team suggested a recovery plan with an increase in the number of sites from 9 to at least 20, as rapidly as was practicable. It also came to light that a draft sample size of 600 had been carried forward into the trial, but in fact applied to an earlier design with a third arm; therefore, the total number of participants required was only 510, not 600. The recovery plan was discussed at a second meeting on 23 May 2016 and was agreed in summer 2016.

Identification, screening and recruitment of participants

The clinical team identified potential participants who had been referred by a GP to be considered for tonsillectomy. A participant information sheet (PIS) along with an invitation letter and appointment letter, if appropriate, were posted out to the patients. The NATTINA PIS outlined details of the trial and signposted potential participants to the NATTINA information DVD/online video-recording on the trial website. 77 The recruitment DVD/online video-recording was developed for the study because it was found to be useful in the NESSTAC trial. 3,64 Video information is more accessible to those who do not read extensive documents regularly and also ensured a degree of consistency of verbal information given across all of the trial sites.

As many patients as possible who were referred to ENT clinics with recurrent sore throat were screened for NATTINA eligibility. All site staff who were involved in screening and recruitment activities were fully trained on the trial protocol, had the necessary experience and qualifications to perform these tasks, and were delegated to do so on the site delegation log. Potential participants, who were previously posted a PIS, were shown the information DVD/online video-recording at their referral visit (unless already viewed online)77 and were given the opportunity to discuss the trial with the designated member of the research team. Patients interested in NATTINA participation underwent an entry eligibility criteria check. Eligible patients were invited to join the trial. Potential participants at the clinic who did not previously receive a PIS were given verbal information. Those with initial interest took away a PIS to read and consider, and if they were interested a further appointment was made for the consent visit, allowing a minimum intervening interval of 24 hours for their decision-making.

Screening and recruitment logs were completed at all sites to document all subjects who attended a referral visit, including their outcome status, reasons for ineligibility, if applicable and, where offered, reason for declining participation.

Declining patients

Patients who were eligible but decided not to take part were invited to provide anonymised baseline comparison data for the NATTINA database. These included age, sex, an estimate of number of sore throat days over the prior 6 months and the completion of a TOI-14 questionnaire. This allowed an analysis of the comparability of trial participants with the total pool of those referred at each of the sites. Patients who declined were also invited to take part in a qualitative interview with a researcher (see Chapter 5). Those who declined the trial will be the subject of a separate publication, incorporating factor analysis of the TOI-14 (in preparation). For the number of declining patients with data, see Table 5. Information on reasons for declining to be randomised and the missing TOI-14 items for these patients are presented separately in Appendix 3, Tables 31 and 32.

Consent procedure

Eligible patients wishing to take part were given the opportunity to discuss the trial and ask any questions before providing written informed consent on the informed consent form (ICF), which was witnessed, signed and dated by a member of the research team who had documented, delegated responsibility to do so. All consent forms were monitored by the NCTU for completeness and accuracy.

Randomisation

Randomisation was carried out via the NCTU secure web-based computer allocation system by delegated site research team members.

Participants were randomised to receive tonsillectomy or conservative management in a 1 : 1 ratio stratified by centre and severity using a blocked allocation (permuted random blocks of variable length) system.

The participant’s severity category was determined by the total TOI-14 score: mild = 0–35, moderate = 36–48 or severe = 49–70.

Treatment, crossover and withdrawals

Participants randomised to receive conservative management were asked to defer surgery for up to 24 months and were informed that they would be assessed on their willingness to continue to delay surgery at the 12-month follow-up visit. However, these participants could decide to cross over to have the tonsillectomy at any point of the study, if they continued to meet the SIGN guidelines for tonsillectomy at that time point. 18 Some participants who were randomised to receive tonsillectomy (see Figure 2) changed their minds prior to receiving the surgery and crossed over to the conservative management pathway, and some crossed over otherwise, for example some failed to receive surgery owing to reasons at the site level. The investigator could also withdraw a participant from the trial intervention if this was believed to be in the participant’s best interests. Any reason for treatment withdrawal offered was recorded.

The number of withdrawals was recorded along with the time from randomisation (see Figures 2 and 3, and corresponding summary statistics can be found in Appendix 8, Table 36). The number of crossovers was also recorded (see Table 6). The number of participants crossing over and reasons for crossover were tabulated (see Appendix 2, Tables 29 and 30). Those crossing over randomised arms could either continue with follow-up visits and data collection or withdraw completely and undertake no further follow-up visits or data submission if this was their wish.

Participants had the right to withdraw from the trial at any time without giving a reason. All data collected up until the point of withdrawal were included within the analysis.

Schedule of events

Figure 1 details the participant flow through the trial, which was designed to maximise data accrual from the greatest available number of sources.

The baseline questionnaire package was completed by the participants at the screening/baseline visit after consent and before randomisation so that the total TOI-14 score was available to inform baseline stratification. Baseline, 12-month and 24-month contacts were in the clinic. Six-month and 18-month questionnaires were postal. Table 1 lists the questionnaires completed at each time point.

At both clinic visits, conservative management participants were asked if they preferred to undergo surgery. The operation date for those in the surgery arm was cross-checked in clinic to facilitate identification of sore throat days directly attributable to surgery in the subsequent statistical modelling. After the final visit, participants’ GPs were asked to provide information on participants’ healthcare resource use for the previous 3 years via the GP linkage case report form (see Chapter 4).

Safety

Only AEs related to the trial intervention (tonsillectomy) were reported for this trial; these were captured and documented at the 1-week and 2-week post-surgery telephone calls. Participants were asked if they had experienced any AEs immediately after, or during recovery from, the tonsillectomy. Only participants who received the surgery were contacted.

All serious adverse events (SAEs) were recorded throughout the duration of the trial for all participants, regardless of whether or not participants underwent tonsillectomy; these did not include pre-planned hospitalisations, routine follow-up of the studied condition or treatment for pre-existing health conditions, not resulting in clinical deterioration. SAEs that were related to the trial intervention have been listed for the trial (see Appendix 10, Table 42).

All AEs that occurred in this trial, whether or not they were serious, were expected following the tonsillectomy procedure, with the exception of two SAEs that were related to the anaesthetic. The following AEs may be expected after receiving tonsillectomy:

• common AEs

  • post-operative pain

  • post-operative bleeding

  • temporary changes in taste/tongue sensation

  • difficulty swallowing

  • nausea

  • vomiting

  • infection

• uncommon AEs

  • long-term changes in taste/tongue sensation

  • chip/knock out of tooth

• very rare AEs

  • death.

Frequencies were estimated from the available data. Full requirements for the reporting of AEs and SAEs were described in the protocol, including guidance for the assessment of the relatedness and causality for SAEs in relation to the intervention. 79

Definition of the end of the trial

The end of the trial was the date when the final participant’s 24-month follow-up data were obtained and when all SAEs were resolved.

Sample size calculation

The planned target recruitment number was 510 participants, which included 72 participants in the internal pilot. This allowed for a total loss to follow-up of 25% over 24 months. In total, 382 participants (two groups of 191 participants providing complete data at 24 months) gave 90% power to detect an effect size of 0.33 [corresponding mean intergroup difference of 3.6 days of sore throat, based on a pooled estimated standard deviation (SD) of 10.8 days], assuming a type 1 error rate of 5%. The choice of this effect size was based on a number of considerations, including our prior experience with the NESSTAC study. 64,89

Statistical analysis

Primary analyses

The primary hypothesis to be tested was H₀: the number of sore throat days in 24 months is same for both randomised arms. A two-sided significance level of p < 0.05 was used throughout.

The primary outcome measure of the total number of sore throat days experienced over the 24 months of follow-up was analysed using NBR to compare the change between the NATTINA arms while adjusting for potential confounders, including the stratification variables: recruiting centre (as a random effect) and baseline severity (as a fixed effect). This analysis was undertaken on an ITT basis; however, patients could switch over from conservative management to tonsillectomy. In the NATTINA design, patients were asked to commit to ‘deferred surgery’ (model 1).

Count data, by convention, often have an exposure variable indicating the duration of the observation period. Exposure variables for the NATTINA models were applied to the NBR model developed to account for differences in the proportion of the weekly returns completed by each participant (range from 0 to 1, with 1 representing a situation whereby all follow-up data were present).

The response variable is the total number of sore throat days over 24 months from randomisation to last follow-up. All participants were included in the analysis on an ITT basis with participants kept in their randomised arm, with the exception of the two participants randomised in error, as described in Chapter 2.

Sensitivity analysis

The true effect of tonsillectomy is likely to lie between the estimate from the ITT analysis, which is the most parsimonious account owing to anticipated crossover into surgery, and the as-treated analyses, which tend to maximise the effect size of any surgical intervention. Outcome data analysis was at the end of the study and for DMC review, and followed the full SAP developed prior to the start of the analysis. Secondary analysis included estimation of the effects of tonsillectomy adjusted for potentially important clinical and demographical variables.

Sensitivity analyses on the ITT population were carried out to examine continuous baseline severity measure TOI-14 (model 2) and any other important baseline factors found to be potentially significant (model 3).

Further multivariable analyses considered other important baseline factors in the NBR model, including sex, age, ethnicity, education level, employment status, site and baseline levels. Non-linear continuous covariates were transformed when appropriate using simple first-degree polynomial transformations.

Additional analyses of the primary outcome

Secondary outcome measures

The analysis of other secondary outcomes followed a similar strategy, with summary statistics tabulated and comparative box plots provided.

The analysis of secondary outcomes followed a broadly similar strategy: repeated measures were analysed using a random-effects model with an appropriate error structure. For example, QoL scores (SF-12) were calculated at baseline and 6, 12, 18 and 24 months post randomisation.

Scores were statistically analysed using longitudinal repeated-measure maximum-likelihood models developed for longitudinal data. The dependent variable was the overall QoL score [SF-12 mental component score (MCS) and SF-12 physical component score (PCS)] and the analysis repeated with the dependent variable TOI-14 score for an individual participant at a particular time point. Both variation between participants and variation between responses nested within participants were modelled as random effects with a normal distribution. The results were adjusted for baseline scores and site, as previously. Differences between groups and changes over time were modelled as fixed effects. The analysis was adjusted for the treatment groups and stratification factor. The model assessed differences at follow-up time points beyond the initial 6-month time point, using the 6-month visit data as the reference value to compare the 12-, 18- and 24-month follow-up scores to.

Trial Management Group

The TMG ran the day-to-day NATTINA management and constituted the chief investigator, co-investigators, statisticians, health economists, qualitative researchers and the NCTU trial management team [senior trial manager, trial manager, data(base) manager, clinical trial administrator and trial secretary]. The TMG met monthly during recruitment and 6 weekly thereafter.

Oversight committees

A TSC was convened to oversee NATTINA, under an independent chairperson. Members were the chief investigator, independent clinicians, an independent statistician and PPI representatives. The committee met six times: prior to the start of the internal pilot and annually thereafter.

An independent DMC was established to undertake an independent review and to monitor efficacy and safety end points. The committee consisted of an independent chairperson, an independent clinician and an independent statistician. The DMC met seven times: first to discuss and advise on the inclusion of an interim analysis and possible adoption of a formal stopping rule for efficacy or safety, at the end of the internal pilot, three times in 2017 at the recovery plan and initial 12-month extension stage, and annually thereafter. Independent members of both the TSC and the DMC were approved by the funder prior to confirmation of their appointment.

Randomisation and stratification factors

Participants were randomised to receive immediate tonsillectomy or conservative management. The allocation ratio was 1 : 1, stratified by recruiting site and baseline severity. The distribution of randomised participants by baseline severity and randomised arm is shown in Table 2. The distribution by site is shown in Appendix 4, Table 33.

Note that some participants were mis-stratified based on their baseline TOI-14 scores. In total, there were 36 such mis-stratifications, detailed in Appendix 4, Table 34.

To mitigate for mis-stratification, we carried out a sensitivity analysis of the primary analysis using continuous baseline TOI-14 scores instead of the stratified categories (see Sensitivity analyses). The principal reason for this sensitivity analysis was, however, to utilise the full information from the continuous measure. The mitigation of mis-stratification was a secondary consideration.

Consolidated Standards of Reporting Trials flow diagram

In the CONSORT flow diagram (Figure 2), the numbers of returned TOI-14 and SF-12 questionnaires at the 6-monthly time points were different. If only one was returned/collected, the higher of the two numbers being returned is cited in the CONSORT flow diagram follow-up questionnaire boxes. Therefore, in the later secondary analyses sections, some numbers may differ from those presented in Figure 2.

FIGURE 2.

The CONSORT flow diagram.

Characteristics of those declining to participate

A total of 1403 patients declined to participate and 1002 (71%) of these patients gave the following reason: ‘wanted tonsillectomy off study’. The full list of reasons given for declining to join the trial is summarised in Appendix 3, Table 31. Patients who declined to participate were invited to provide anonymised baseline comparison data for the NATTINA database (age, sex, an estimate of number of sore throat days over the prior 6 months and a TOI-14 questionnaire) and 609 patients provided these data. Declining patients were also invited to participate in a qualitative interview with a researcher.

It can be seen from Table 3 that the proportions of males (21%) and females (78%) who accepted or declined to be randomised into the trial were the same. There was a tendency for those who accepted to be randomised to be, on average, 2.5 years older (95% CI 1.7 to 3.3 years) and to have lower TOI-14 scores (mean difference –5.19, 95% CI –6.5 to –3.9) than those who declined to be randomised who provided data.

Differences between the distributions of TOI-14 scores between the two arms can be seen in the box plots in Appendix 3, Figure 19.

Baseline participant characteristics

The population was ITT. Demographic and baseline characteristics were compared across randomised arms descriptively. Descriptive statistics were tabulated by randomised arm and overall (see Table 4):

• No baseline data were collected for the two participants who were randomised but were ineligible for the trial because the participants did not consent.

• No baseline data were collected for participants at the unresponsive site.

• The recoding of ‘other’ categories for employment (n = 11) and education (n = 9) was completed manually from the text provided by the participant, with the approval of the TMG. For a list of the free text provided for ‘other’ description, and how it was recoded, see Appendix 5.

Table 4 shows the baseline demographics for 453 eligible participants. It can be seen that the population recruited into the NATTINA trial was predominantly female (78%); white (90%); relatively young, with a median age of 23 years [interquartile range (IQR) 19–30 years]; and in paid employment (62%).

Data quality and completeness

Table 5 shows the expected number of responses, actual number of responses and response rate for the trial case report forms. Table 5 also includes the number of participants who returned at least one weekly STAR and counts of (1) the primary outcome measure ‘sore throat days’, collected via the weekly STAR response texts, and (2) the resultant STAR questionnaires when sore throat days were >0. Compliance for the two telephone calls scheduled for 1 and 2 weeks post tonsillectomy is also presented. AEs reported during these telephone calls are reported in Safety.

Summary of primary outcome data: Sore Throat Alert Return

A total of 32,978 weekly STAR responses was received. Of these responses, 809 were for the weeks after the end of the follow-up period. Participants who withdrew from the trial were included in the analysis up to the point of withdrawal. This resulted in a further 117 STAR responses being omitted from the primary analysis data set. Therefore, 32,052 STARs were retained for the primary analysis.

Assessment of compliance with face-to-face and postal completion of questionnaires

Table 6 shows participant engagement with the trial follow-up questionnaire at the follow-up visits and postal questionnaire returns points:

• In Table 6, ‘complied (± 6 weeks)’ indicates participants who returned forms or attended visits within ± 6 weeks of each scheduled data collection point (6, 12, 18 and 24 months after the randomisation date).

• ‘Early’ represents forms returned/visits completed before the 6-week lower compliance window.

• ‘Late’ represents forms returned/visits completed after the 6-week upper compliance window.

• Further details, including a table of summary statistics of the time from randomisation to visits/postal returns of secondary outcome questionnaires and histograms showing compliance windows for the four follow-up data collection points, are shown in Appendix 6, Table 35, and Figures 20–23.

Note that the relatively low levels of engagement with follow-up visits and postal questionnaire completion did not affect the primary analysis of the trial. The primary outcome measure was the number of sore throat days reported over the 24-month follow-up period, which was collected via the weekly STAR responses. These were independent of scheduled follow-up 6-monthly visits and postal returns, which primarily concerned the collection of secondary outcome data.

Treatment received

There were 43 withdrawals from the trial (see Figure 2). Appendix 8, Table 36, reports the summary statistics for time from randomisation to withdrawal in weeks, by arm and overall. The box plots in Figure 3 also illustrates the withdrawal information. Most withdrawals occurred in the first 12 months of the trial, with those randomised to receive conservative management tending to withdraw earlier than those randomised to receive tonsillectomy.

FIGURE 3.

Box plots showing time from randomisation to withdrawal by arm (n = 43). (a) Conservative management (n = 27); and (b) immediate tonsillectomy (n = 16). The outlier in immediate tonsillectomy requested to withdraw before the end of follow-up.

The preparatory PPI group work had advised that deferred tonsillectomy as the conservative management arm would facilitate recruitment. Therefore, the protocol permitted participants to opt out of their randomised treatment allocation and effectively cross over to the other randomised arm.

Table 7 presents how the participants were randomised, how many received the allocated treatment, how many requested and received the treatment from the other arm, and how many withdrew (see Figure 2).

For participants randomised to the tonsillectomy arm (i.e. to receive immediate tonsillectomy), 172 out of 233 (74%) went on to receive the procedure. Of these participants, 95 (41%) received their tonsillectomy within 8 weeks of randomisation, in line with the protocol, and the remaining 77 (33%) received their tonsillectomy late. A total of 234 out of 453 (52%) eligible participants fully complied with the trial protocol (not crossing over or withdrawing and, in the case of the tonsillectomy arm, receive tonsillectomy within 8 weeks of randomisation). However, 10 of these participants did not make any STARs and, therefore, did not have primary outcome data and were not included in the per-protocol analysis, leaving 224 (49%) participants in this sensitivity analysis. A total of 139 out of 220 (63%) participants randomised to the conservative management arm went on to receive that treatment; however, of these participants, 15 had requested to receive tonsillectomy but had not received it by the end of follow-up, so were assumed to comply with conservative management (see Table 7).

A total of 94 out of 453 (21%) participants requested to stop the treatment that they were randomised to (shaded in Table 7): 21 out of 233 (9%) participants randomised to the immediate tonsillectomy arm requested to switch to conservative management, whereas 73 out of 220 (33%) participants randomised to the conservative management arm requested to receive tonsillectomy. There was no protocol provision for these participants to receive the tonsillectomy within a specified time compliance window, but 59 out of 73 (81%) of those requesting to cross over had received their tonsillectomy by the end of their 24-month follow-up period. This includes one participant who had tonsillectomy off trial, so did not complete a crossover form. Reasons given for requesting a crossover between randomised arms are tabulated in Appendix 2, Tables 29 and 30.

A total of 43 out of 453 (9%) participants officially withdrew their consent to further participate in the trial. All data collected up until withdrawal were retained for NATTINA research purposes, as no one requested that their prior data be withdrawn. More details of these 43 withdrawals are shown in Appendix 8, Table 37. The summary statistics of time from randomisation to withdrawal are shown in Appendix 8, Table 36.

Time from randomisation to tonsillectomy

Figure 4 shows the distribution of time from randomisation to tonsillectomy for participants randomised to the immediate tonsillectomy arm (with a protocol compliance window of within 8 weeks of randomisation shown for participants randomised to receive immediate tonsillectomy). The width of each bar is 8 weeks and, therefore, it can be seen that 95 participants complied with the protocol of receiving tonsillectomy within 8 weeks of randomisation. A further 55 participants received tonsillectomy in the 8-week period following protocol compliance. Summary statistics for the time from randomisation to tonsillectomy are shown in Appendix 9, Table 38.

FIGURE 4.

Histograms showing distribution of time from randomisation to tonsillectomy by arm. (a) Conservative management (n = 59); and (b) immediate tonsillectomy (n = 172). Red dashed line is 8 weeks after randomisation (protocol compliance for immediate tonsillectomy).

Type of tonsillectomy received

Of the 231 surgeries carried out, 122 (53%) were cold dissection, 91 (39%) were bipolar diathermy, 1 (<1%) was coblation, 1 (<1%) was laser, 8 (3%) were described as mixed (combination of cold and bipolar dissection) and 1 (<1%) was the tonsillectomy carried out off trial. For the remaining seven surgeries (3%), sites did not provide details of the tonsillectomy type received. Appendix 9, Table 39, shows the breakdown of tonsillectomy type by the arm originally randomised to. Ten of these (4%) reported complications in the procedure (see Safety).

Serious adverse events

Fifty-one SAEs, reported in 45 participants, were attributed to the intervention by the surgeon. These are reported in Appendix 10, Table 43, where they are categorised as mild, moderate and severe and are split by randomised arm.

No fatal SAEs were reported in NATTINA, nor any life-threatening SAEs or SAEs that caused persistent or significant disability or incapacity.

Table 9 shows the 51 reported SAEs recategorised based on the free text provided in the original MACRO entries.

The median time from tonsillectomy to the first SAE in these 45 participants was 5 days (IQR 2–7 days).

The SAE resulted in an extended stay in hospital in 50 out of the 51 (98%) SAEs judged to be causally related to surgery. All participants recovering from a SAE had fully recovered by the end of the trial follow-up. Further details are provided in Appendix 10, Table 42.

Medication was the single most common remedial (32/51, 63%). Medication was combined with a blood/fluid transfusion in six cases (12%), surgery and medication were used in two cases (4%), surgery alone was used in one case (2%), blood/fluid transfusion was used in two cases (4%) and surgery/medication/transfusion was used in one case (2%). No action was required in four (8%) of the cases and information was not entered onto MACRO in three cases (6%).

Retention flow diagram

The flow chart in Figure 5 shows the number of randomised participants and how they engaged with the weekly STAR response procedure.

FIGURE 5.

Flow chart showing primary outcome data.

Primary outcome measure

The primary outcome measure was the total number of sore throat days reported over the 24-month follow-up period. As Table 5 shows, 72% of STARs were returned.

Table 10 summarises the raw data of total sore throat days over the 24-month period. Participants in the tonsillectomy arm tended to report fewer sore throat days than those in the conservative management arm. The total reports of sore throats were difficult to compare formally owing to variability in the total number of returns per participant. To make a fairer comparison of the sore throat rates in the two arms, we have also presented the summary of the average weekly STAR for each participant, calculated as the total number of sore throat days reported in the 24-month (105-week) follow-up period divided by the number of returns made, by randomised arm and overall, for the ITT population. The data again show a positive skew: immediate tonsillectomy participants had median of 0.33 sore throats per week compared with 0.57 sore throats per week in the conservative management arm.

Primary analysis

This primary analysis was undertaken on an ITT basis. The response variable is the total number of sore throat days over 105 weeks from randomisation to last follow-up. As the primary outcome is a count variable, all analyses use the NBR model. This model can allow for the variable number of STARs by including the proportion of STARs (out of a total of 105) as an exposure variable. All NBR analyses include the exposure variable.

Negative binomial regression of the primary outcome

Table 11 shows the results of the unadjusted analysis (i.e. no covariates or stratification factors included in the model) of the primary outcome on the ITT population. The difference between arms indicates that participants randomised to receive tonsillectomy are estimated to have a total number of sore throat days 0.575 times smaller than participants randomised to receive conservative management (95% CI 0.47 to 0.71; p < 0.001).

Count data can be modelled using the Poisson model; however, NBR regression was judged to be more appropriate in this case because the coefficient of dispersion was high. For the raw data (both arms combined), the mean sore throat weekly rate reported during follow-up was 0.90, with variance 1.6384. Therefore, the coefficient of dispersion, variance/mean, is 1.82, implying that there is overdispersion of the weekly sore throat rate.

In the NBR model presented in Table 11, the dispersion parameter of 1.113 is significantly greater than zero; therefore, these results also confirm that data are over-dispersed and are better modelled using a negative binomial model than a Poisson model.

Several adjusted NBR analyses were conducted. The first of these was carried out to compare the NATTINA arms while adjusting for the stratification variables used at the point of randomisation in the trial: recruiting centre (as a random effect) and baseline severity (as a fixed effect). This is the primary analysis of the NATTINA trial and is referred to as model 1, and is presented in Table 12. The estimated difference between randomised arms, when adjusting for site and baseline severity, indicates that immediate tonsillectomy participants are estimated to have 0.528 times fewer total sore throat days in the 24-month follow-up than conservative management participants (95% CI 0.43 to 0.65; p < 0.0001). There is a significant reduction in sore throat days in those participants randomised to immediate tonsillectomy. In other words, for every 10 sore throat days reported by participants in the conservative management arm over the 24-month (105-week) follow-up, there are only 5.28 sore throat days reported in participants randomised to tonsillectomy. The median number of sore throat days in the conservative management arm is 30 days. Hence, the model would predict a reduction of sore throats in this case by 14 days.

Baseline severity was found to not be statistically significantly related to the total number of sore throat days reported in the 24-month follow-up, despite the fact that it was a stratification factor in the trial design. Participants with moderate or severe baseline severity in terms of TOI-14 score are estimated to have 1.213 and 1.031 times more sore throat days, respectively, than those with mild category scores while holding the other variables constant in the model. Including site as a random effect demonstrates the limited impact of variation between sites on primary outcome measure, but is retained as a stratification variable in the trial analysis. Table 13 shows the summary statistics of the raw total number of sore throats reported by arm and baseline severity. This confirms that the number of sore throat days reported does not seem to be dependent on baseline severity. The analysis that allowed interaction with baseline severity (see Appendix 13) showed that there was a weak tendency for those classed as mild at baseline to have more of an improvement with tonsillectomy than those classed as moderate or severe at baseline; however, the interaction terms were not statistically significant. Hence, overall, we see a strong benefit in those randomised to the tonsillectomy arm and have no evidence that there is major variation in the impact of the tonsillectomy pathway by baseline participant characteristics.

Given that alpha is, again, significantly greater than zero (1.025), the data are over dispersed and are estimated better using a NBR than using a Poisson model.

Exploration of Sore Throat Alert Return profile by randomised arm over the 105 weeks

The primary analysis (model 1) shows a statistically significant reduction in sore throats in those randomised to the immediate tonsillectomy arm. It is important to understand where this strength of difference comes from because there is crossover between arms and tonsillectomy causes sore throat days post operatively.

The weekly average number of sore throats reported by the primary ITT analysis population by arm is shown in Figure 6. The graphs also include embedded histograms showing the timing of the tonsillectomy intervention. The effect of the tonsillectomy on the number of sore throat days is seen clearly in the immediate tonsillectomy arm.

FIGURE 6.

Weekly average sore throat days with histograms showing point of tonsillectomy by arm. (a) Conservative management (n = 205); and (b) immediate tonsillectomy (n = 224). Histogram of tonsillectomies: conservative management, n = 59; immediate tonsillectomy, n = 172.

Participants who were originally randomised to the conservative management arm underwent tonsillectomy much later than the 8-week protocol target for the participants in the immediate tonsillectomy arm. The weekly average of number of sore throats also falls over time in this arm, but the decline is less obviously linked to the intervention (as in the immediate tonsillectomy arm) owing to there having been fewer tonsillectomies, which were more dispersed over follow-up. It can also be seen that at the end of the follow-up period, the average weekly rate in the conservative management arm was around 0.5 sore throat days per week, whereas in the tonsillectomy arm it was substantially lower (around 0.2 days per week).

The pattern of reporting of sore throat days in relation to the timing of any tonsil surgery is detailed in Appendix 12, Table 44, and is summarised in Figure 7. The patterns for sore throat days looked similar in both arms for the three periods for those who received surgery and for the one period for those who did not receive surgery. In both arms, there tended to be an increase in sore throats immediately following surgery, then thereafter post surgery the rates dropped below those seen for the no surgery group.

FIGURE 7.

Average weekly sore throat rate for participants not receiving tonsillectomy and those who received tonsillectomy. (a) Conservative management; and (b) immediate tonsillectomy.

The box plots shown in Figure 7 show the distribution of sore throats by time period. The box plots show that the pre-tonsillectomy rates look similar, that there is a sharp increase in sore throats reported immediately following tonsillectomy, and that rates are very low after the first 2 post-operative weeks. More than 2 weeks post tonsillectomy, the sore throat weekly rate is very much lower in those who received tonsillectomy than in those who did not receive tonsillectomy.

In Figure 7, the average weekly sore throat rate in three time periods is shown: pre-tonsillectomy (mid-blue), 2 weeks following tonsillectomy (orange) and the remainder of the 24-month follow-up period post operation (light purple). The non-tonsillectomy participants are also displayed (light blue).

Addressing missing data

Although 72% of STARs were returned, we were interested to understand if there was differential missingness in the STAR rate by arm or subgroup of interest. Table 14 shows the cumulative percentage of missing primary outcome data over the 24 months following randomisation. This is the length of time measuring the primary outcome. The numbers and percentages are presented by category, by arm and overall.

Eighty-one out of 455 (18%) randomised participants had complete data. A total of 227 (50%) participants provided >90% of their STAR responses and 263 (58%) returned >80%. A slightly higher proportion of the tonsillectomy arm completed at least 90% of STARs than the conservative management arm.

We explored this further by dividing the 24-month follow-up period into 6-monthly intervals. Weekly STAR responses can be categorised during each 6-month interval from the point of randomisation as:

• complete, in which participants completed all of the expected weekly STAR responses within that time frame

• partial, in which some but not all expected weekly STAR responses were received

• no returns, in which participants did not return any of the expected weekly STAR responses within that time frame.

Figure 8 shows the STAR responses categorised as complete, partial or no returns in a bar chart at each of the 6-month follow-up intervals.

FIGURE 8.

Bar chart showing observed number of STARs by categories (complete, partial and none) by arm at time points (6, 12, 18 and 24 months).

Table 15 shows the expected number of responses, observed complete responses, observed partial responses and missing responses (no weekly STAR response returns) for 6, 12, 18 and 24 months when other trial questionnaires were due to be completed. Tables 14 and 15 show that there are slight differential STAR rates by arm. For example, 146 (62%) participants in the immediate tonsillectomy arm completed more than 80% of their returns compared with just 117 (53%) participants in the conservative management arm.

We looked at the time of the last STAR for each participant. We define retention as the time in the trial, which is calculated as the time from randomisation to the last weekly STAR response. The retention times in the trial for the participants who sent at least one weekly STAR response are presented in box plots in Figure 9. This allows an assessment of whether or not retention appeared to be balanced by randomised arm. Although 50% of participants in both arms completed almost all STARs, the box plots in Figure 9 show that participants randomised to the immediate tonsillectomy arm were more likely than those in the conservative management arm to continue sending STAR responses further into the follow-up period. Table 14 breaks down the distribution by completeness rather than by last return.

FIGURE 9.

Box plots showing distribution of time from randomisation to last STAR response in the 24-month follow-up period by arm. Conservative management (n = 205); and (b) immediate tonsillectomy (n = 224).

Per-protocol analysis

The per-protocol analysis was restricted to the 224 participants who complied with the protocol (see Table 7): 95 participants in the tonsillectomy arm and 139 participants in conservative management arm, excluding the 10 participants with no STARs. See Appendix 7 for details of the analysis. Participants who withdrew were omitted from these analyses.

As might be expected, the per-protocol analyses showed a stronger effect of the intervention, with a reduction of 0.415 times the sore throat days. Therefore, the results of the primary analysis were found to be robust compared with these sensitivity analyses, as shown in the forest plot (see Figure 11 and Appendix 7).

Addressing crossover (per-treatment analysis)

Participants were able to switch over from conservative management to tonsillectomy, and also from tonsillectomy to conservative management. A new group variable was created indicating whether or not tonsillectomy was received by the participant. This replaced ‘arm randomised’ to capture the effects of tonsillectomy for per-treatment analysis.

For this ‘per-tonsillectomy’ analysis, we included 224 participants who received tonsillectomy and 205 who did not receive tonsillectomy, making a total of 429 participants who had also completed at least one STAR (primary outcome measure for the primary analyses).

A second per-treatment analysis was carried out according to four categories:

1. randomised to receive immediate tonsillectomy and tonsillectomy completed (at any time during follow-up) (n = 167)

2. randomised to receive immediate tonsillectomy but tonsillectomy not completed (n = 57)

3. randomised to receive conservative management but tonsillectomy received (n = 57)

4. randomised to receive conservative management with no tonsillectomy completed (n = 148).

This reflects whether or not participants actively chose to swap from their randomised arm. This was carried out to assess if there were differences in severity and/or outcome among those who deliberately chose to swap from their randomised arm compared with those who were content to remain as randomised.

The instrumental variable sensitivity analyses (CACE and interactions of baseline severity with randomised arm allocation) were not included in the forest plot, and are presented in Appendix 13, Table 45.

We repeated the four category per-treatment analysis, limiting the population to those who returned at least 80% of STARs (n = 263). These sensitivity analyses were adjusted by stratification variables with reference categories as in the original analyses. The pattern is similar to the full per-treatment analysis (and some of the groups now have very small numbers), but those randomised to receive tonsillectomy who then chose not to have tonsillectomy had significantly fewer sore throats than those who were randomised to and received conservative management. This would be consistent with this group tending to switch away from surgery because of their lesser sore throat burden.

The forest plots in Figures 10 and 11 demonstrate that the primary and sensitivity ITT analyses show a strong signal of differences between the two randomised arms in terms of IRRs and 95% CIs.

The per-protocol and two-group per-treatment analyses were consistent with the primary analysis NBR model 1 in that they too showed a statistically significant reduction in total sore throats when comparing the immediate tonsillectomy arm with the conservative management arm. The only slight signals at odds with this were those seen in the crossover groups in the four-group analysis. We therefore felt that it was important to explore the patterns of STARs and sore throat profiles over the 24 months by the various analysis populations. See Appendix 11 for further details.

Those who received tonsillectomy having crossed over from conservative management [(n = 57) in Figure 11 per-treatment four categories group] saw not a fall but an (non-statistically significantly) increase in sore throat days (IRR 1.277, 95% CI 0.921 to 1.768; p = 0.141) (see Appendix 12, Table 44). The anomaly here is that those having tonsillectomy after crossing over from conservative management reported more sore throats than those remaining in the conservative management arm. This could be because of two factors. First, these participants had higher rates of sore throat and, therefore, were more motivated to try tonsillectomy, whereas those who remained in the conservative management arm, as randomised, may have tended to be those with lower rates of sore throat. Second, the effect of surgery itself induced additional sore throats. The later that the surgery is received, the less time there is for benefit from the surgery to compensate for the (late) cost of additional, postoperative sore throat days. The four graphs in Figure 12 of average weekly sore throats over time appear to support this interpretation. Those in the conservative management arm who switch to tonsillectomy seem to have higher initial rates of weekly sore throats than those in the tonsillectomy arm who switch to conservative management. Conversely, participants who were randomised to receive tonsillectomy, but who chose not to receive it, started off with fewer sore throat days than the other participants in that arm. This may explain why they opted not to have tonsillectomy (see Figure 12).

FIGURE 12.

Weekly average sore throat days by randomised arm and occurrence of tonsillectomy. (a) No surgery as randomised (n = 148); (b) no surgery but randomised to surgery (n = 57); (c) surgery as randomised (n = 167); and (d) surgery crossed (n = 57). Graphs by tonsillectomy and randomised arm.

Figure 13 shows the rate of weekly STARs (exposure) by the four per-treatment categories. These box plots show that those randomised to receive immediate tonsillectomy and who received surgery had the highest return rates. Those randomised to receive immediate tonsillectomy but who did not receive tonsillectomy had the lowest return rates.

FIGURE 13.

Box plots showing percentage of total STARs by four category per-treatment groups (n = 429). (a) No surgery as randomised (n = 148); (b) no surgery but randomised to surgery (n = 57); (c) surgery as randomised (n = 167); and (d) surgery crossed (n = 57). Graphs by surgery received and arm randomised to.

Missing secondary outcome data

Appendix 15 includes details of the distribution of missing data among the baseline severity categories (mild, moderate and severe) according to STAR responses and TOI-14 questionnaires.

Tonsillectomy Outcome Inventory-14

The responses at the five time points to the TOI-14 are shown in Figure 14 and are summarised using appropriate descriptive statistics along with 95% CIs in Table 17. These were similar at baseline as expected, but there was a marked difference at follow-up visits (after surgery) in the overall TOI-14 scores (non-overlapping CIs). Appendix 14 details the four TOI-14 subscales in a similar fashion – throat discomfort, general health, resource impact, and social and psychological:

• Questionnaires missing one or two items were completed by imputation of the item score calculated as the average of the completed items.

• Analysis and charts in the rest of this section include the recovered questionnaires with imputed missing items.

FIGURE 14.

The TOI-14 at 6, 12, 18 and 24 months: ITT population. (a) Conservative management; and (b) immediate tonsillectomy.

Details of imputation are included in Appendix 14, Table 51.

Reductions in the TOI-14 scores (symptom improvements) were seen in both randomised arms, but were greater in the immediate tonsillectomy participants, similar to the benefits demonstrated in Figure 14. Summary statistics are presented in Table 17, in which the non-overlapping CIs show the differences at follow-up visits between the randomised arms.

Box plots for the per-protocol and four category per-treatment population groups are presented in Appendix 14, Figures 28 and 29.

The difference between the means, with CIs and the corresponding p-value at each time point, is presented in Table 18. The difference is calculated as conservative management minus immediate tonsillectomy and higher values equate to worse symptoms.

There is no difference at baseline, which was expected owing to the randomised nature of the trial. Baseline TOI-14 scores were used as stratification factors for the trial. Significant differences can be seen at the follow-up time points, however, with conservative management having higher average TOI-14 scores than tonsillectomy. This shows how the participants who were randomised to immediate tonsillectomy report significantly better outcome at every follow-up point, with a larger difference apparent at the 12-month visit at which more participants provided TOI-14 questionnaires (n = 239). This supports the conclusions of the box plots in Figure 14.

In addition, scores were statistically analysed using longitudinal repeated-measure maximum-likelihood models developed for longitudinal data. The dependent variable was the overall TOI-14 score for an individual participant at a particular time point. Both variation between participants and variation between responses nested within participants were modelled as random effects with a normal distribution. Differences between groups and changes over time were modelled as fixed effects. The analysis was adjusted for the treatment groups and stratification factor. There are statistically significant differences between randomised arms, with both randomised arms reporting improvements from baseline. The differences reported at follow-up time points beyond the initial 6-month time point are not significantly different, which shows that the majority of the improvements occur within the first 6 months after randomisation for participants randomised to either arm.

The mixed-model repeated-measure analysis carried out on the TOI-14 scores at the five time points showed that there is also a statistically significant difference in improvement between the randomised arms, with those randomised to receive immediate tonsillectomy reporting greater improvement than those randomised to receive conservative management. In summary, the mean scores tended to be lower in the tonsillectomy arm than in the conservative management arm by 13.17 (95% CI –17.41 to –8.92), which indicates a reduction in symptoms. See Appendix 14, Table 47, for the full results table. The longitudinal repeat-measure maximum-likelihood model confirmed the conclusions drawn from the graphical interpretation for TOI-14 in Figure 14. Summary statistics by arm and trial time points are shown in Table 17.

Similar conclusions can be drawn for the TOI-14 subscales, as outlined in Appendix 14, Tables 46 and 48–50, and Figures 24–27.

Quality-of-life analysis: Short Form questionnaire-12 items

The analysis set is the ITT set. Quality-of-life scores based on the SF-12 were calculated using the Optum scoring software (Quality Metric Incorporated) at baseline and 6, 12, 18 and 24 months post randomisation. Scores are described with summary statistics and are graphically presented over time. The software takes baseline levels into account and adjusts for follow-up time points accordingly. The algorithm contained within the software generates physical (SF-12 PCS) and mental (SF-12 MCS) health composite scores for comparison with the normative data for the UK population. In the normative data, the mean score is transformed to be 50 (SD 10), adjusted for sex and age, with scores greater than 50 indicating better physical or mental health than the mean and scores less than 50 indicating worse physical or mental health than the mean.

At baseline, the mean scores are just below 50, indicating worse mental and physical health scores than the UK population average. Mental health scores are further below 50 than physical health scores, which is an indication that the population suffers more with their mental health than their physical health when entering the trial.

For participants randomised to receive immediate tonsillectomy, improvements can be seen in both health measures, with physical health improving more, as reported at the follow-up collection time points. Smaller improvements are seen in the physical health score for participants randomised to receive conservative management than for those randomised to receive tonsillectomy. In addition, less improvement is seen in the mental health score for those randomised to the conservative management arm than for those randomised to the tonsillectomy arm, with average scores in the conservative management arm still falling below the population average and average scores in the tonsillectomy arm falling above the population average.

In addition, scores were statistically analysed using longitudinal repeated-measure maximum-likelihood models developed for longitudinal data. The dependent variable was the overall QoL score (SF-12 MCS and PCS) for an individual participant at a particular time point. Both variation between participants and variation between responses nested within participants were modelled as random effects with a normal distribution. Differences between arms and changes over time were modelled as fixed effects. The analysis was adjusted for the randomised arms and stratification factor.

Both randomised arms reported improvements from baseline. The differences reported at follow-up time points beyond the initial 6-month time point were not significantly different, showing that the majority of the improvements in both arms occurred within the first 6 months after randomisation.

There is also a statistically significant difference in improvement between the randomised arms. Both scores tended to be higher in the tonsillectomy arm than in the conservative management arm (SF-12 MCS 3.71 units higher, 95% CI 2.10 to 5.47; SF-12 PCS 2.77 units higher, 95% CI 0.30 to 5.23). See Appendix 16, Tables 55 and 56, for the full results tables. The longitudinal repeat-measure maximum-likelihood models confirm the conclusions drawn from the graphical interpretation for SF-12 MCS and PCS in Figures 15 and 16. Summary statistics by arm and trial time points are shown in Table 19.

FIGURE 15.

Box plots of normalised mental health scores (SF-12 MCS) at five time points by randomised arm. (a) Conservative management; and (b) immediate tonsillectomy. Red line is average normalised score for UK population. (continued)

FIGURE 16.

Box plots of normalised physical health scores (SF-12 PCS) at five time points by randomised arm. (a) Conservative management; and (b) immediate tonsillectomy. Red line is average normalised score for UK population.

The box plots in Figures 15 and 16 show the summary statistics graphically.

For the SF-12 MCSs in Figure 15, the medians are higher than the benchmarked average for the UK population (red reference line) for the tonsillectomy arm in follow-up. This is better than for the conservative management arm, for which the medians are closer to the population mean.

For the SF-12 PCSs in Figure 16, greater improvement compared with the UK population average can be seen in the tonsillectomy arm, for which the lower quartiles are well above the average. This implies that >75% of participants in the tonsillectomy arm experience sufficient improvement in physical symptoms to achieve physical QoL above the national average. Improvement is also seen in the conservative management arm, but not as widespread throughout all of the participants in that arm.

Cost data collection

The costs collected as part of the economic evaluation were based on the cost of the interventions themselves and the use of health care and PSS over the 24-month follow-up period. All costs are reported in 2018/19 Great British pounds.

Derivation of costs

Participant-level data on resource use were combined with unit costs and were presented in the following categories:

• tonsillectomy costs

• subsequent treatment costs

• participant costs.

General practitioner linkage

Healthcare resource use was also captured in the GP linkage case report form. Initial consent included access to participants’ GP health records to assess healthcare use during each participant’s 24-month follow-up and also 12 months prior to randomisation. The following data were collected on the GP linkage case report form:

• AEs

• attendance at general practice/walk-in clinic/A&E for sore throat or related event

• hospitalisations and emergency referrals

• prescriptions issued

• any additional interventions required.

The GP linkage data were used to estimate the total healthcare resource use for each participant for the duration of the follow-up as a secondary analysis. Unit costs were collected from routine sources, as described above. Prescription costs were not considered in this analysis because the data from the STAR/STARLET were used to inform prescription costs. The frequency of medications reported in the GP linkage case report form is provided in Report Supplementary Material 1, Table S5.

Effectiveness measures

Three effectiveness measures were used in this economic evaluation, one for each of the analyses: CEA – number of sore throat days; CUA – QALYs; and CBA – WTP to avoid a sore throat day.

Comparative incremental analyses of costs and effects

Both unadjusted and adjusted analyses were performed to estimate the cost-effectiveness of tonsillectomy compared with conservative management. All results were presented as unadjusted point estimates of mean total costs and effects, and adjusted point estimates of the mean incremental costs, effects and cost-effectiveness. Both costs and effects incurred during the second year of follow-up were discounted at the UK recommended rate of 3.5%. 113 If an arm was found to be more costly and more effective, then it was considered to be the dominant strategy and, hence, cost-effective. If tonsillectomy was found to be, on average, more effective but more costly, then consideration had to be made as to whether or not the extra costs are worth the extra effectiveness. In this situation, decisions were based on the following:

• The incremental cost-effectiveness ratio (ICER). This ratio equates to the difference in average total costs divided by the difference in average total effects [e.g. £20 per sore throat day avoided could equate to £20 (difference in average total costs)/1 day (difference in average total number of sore throat days) = £20 (ICER)]

• Society’s WTP threshold for an additional unit of effect.

The same cost estimates were used in all three analyses, but the analyses differed in terms of outcome measures.

Adjusted analysis: seemingly unrelated regression

An adjusted analysis was used for all analyses to estimate the point estimates of the mean incremental costs, effects and cost-effectiveness using seemingly unrelated regression (SUR). 115 Costs and effects were estimated simultaneously using SUR, which accounts for possible correlations between costs and effects when using individual participant-level data. 116 Additional covariates were also controlled for using SUR (e.g. age, randomisation arm, baseline severity/utility and income).

Sensitivity analysis

In the base-case analysis, missing cost and utility data were imputed using chained multiple imputation methods,117 which assumed data were missing at random. Differences in baseline utility and costs between those missing data and those with complete data were undertaken to test this assumption. Sensitivity analyses were conducted to assess the robustness of the results to realistic variations in the levels of underlying data.

The following sensitivity analyses were undertaken to determine what effect, if any, changing the assumptions in the base-case analysis had on the overall conclusions:

• Surgery costs were estimated using microcosting.

• Participant costs were considered.

• Healthcare costs were estimated using GP linkage data.

• Utility data associated with sore throat episodes collected in the STAR/STARLET data were incorporated into the QALY equation.

• Healthcare costs were estimated for participants with cost data at least at one time point post randomisation and estimated effects were based on the data available.

Deterministic sensitivity analyses were used to address any uncertainty in the assumptions used in our base-case analysis, including using the microcosting tonsillectomy cost as the intervention cost and the difference in healthcare costs when the GP linkage data were used.

Stochastic sensitivity analyses, using the bootstrapping technique,118 explored the impact of the statistical imprecision surrounding estimates of costs, effects and cost-effectiveness. In essence, the process entails extracting and estimating the difference in costs and effects from a paired sample (one participant from each arm) from the trial data set. Those are then returned to the data set and the process is repeated 1000 times. The bootstrapped results from all three analyses were presented on a cost-effectiveness plane. The cost-effectiveness plane was used to illustrate the distribution of incremental costs and incremental effects from which we can clearly identify the spread of uncertainty in our results. 119

The bootstrapped results from the CEA and CUA were presented as cost-effectiveness acceptability curves (CEACs). CEACs determine the treatment option that maximises net benefits at each WTP value for an additional unit of health effect (i.e. a reduction in sore throat days and QALYs). 120 For the CEA, the threshold from which to interpret these results can be informed by the values obtained from the WTP exercise. A similar approach to the CEAC is not required for the CBA. For the CBA, the intervention with the greatest net monetary benefit is considered efficient. Therefore, for the stochastic analysis, the results were presented in terms of the probability that the intervention was associated with the greatest net monetary benefit.

Data validity and completeness

Table 20 presents the response rates for the participant-completed data collection tools used to inform the economic analysis. Predictably, the baseline questionnaires, which included the contingent valuation survey, were comprehensive, yielding primary outcome data for nearly 95% of participants. Attrition over time peaked for postal questionnaires. Only 20% of participants had complete data on health utilisation and the SF-12. The pattern of non-response was similar across both arms of the trial and at all time points. There was no difference in baseline costs (p = 0.67) or baseline utilities (p = 0.50) between those with data and those with no data available over the 24-month follow-up.

Resource use and costs

Over the 24-month follow-up period, participants reported a number of contacts with different healthcare professionals (see Appendix 17, Table 59). The most common visits were to GPs and pharmacists. On average, participants in the conservative management arm reported using more healthcare services than participants in the tonsillectomy arm, except at 6 months. The difference at 6 months appears to be a result of more secondary care visits (outpatient and inpatient), which could be participants reporting their surgery. The majority of tonsillectomy surgeries were performed as day procedures by either a consultant or a specialist registrar (see Appendix 17, Table 60).

The unit costs used in the analysis are provided in Report Supplementary Material 1, Table S2. Table 21 presents the average total costs per cost category and per randomised arm. Participants in both arms reported similar average baseline healthcare resource use. The average total cost of treatment (tonsillectomy and post-operative medications) was higher in the tonsillectomy arm than in the conservative management arm, which was expected given that 74% of participants in this arm received a tonsillectomy compared with 27% of participants in the conservative management arm. Total healthcare costs could be estimated for 308 participants as they responded to at least one health utilisation questionnaire post randomisation. On average, and as would be expected given the data reported in Appendix 17, Table 59, those in the conservative management arm reported higher primary and secondary care costs than those in the tonsillectomy arm over the total 24-month follow-up period. After the total costs were combined to estimate the average total cost to the NHS, on average, tonsillectomy was more costly than conservative management. This result was consistent when missing cost data were imputed using the assumption that data were missing at random.

Data from the STAR/STARLET used to inform direct and indirect costs incurred by participants are summarised in Appendix 17, Table 58, and Report Supplementary Material 1, Tables S1–S4 and S6. These daily tariffs were combined with each reported sore throat episode based on the severity of the episode and multiplied by the duration of the sore throat episode. When costs to participants were considered, conservative management was, on average, more costly than tonsillectomy {mean difference £889 [standard error (SE) £432], 95% CI £40 to £1738}. The difference in the average total costs was mainly driven by the indirect costs associated with sore throat episodes (see Appendix 17, Table 58).

Effectiveness outcomes

Economic evaluation

Incremental cost-effectiveness analysis

Table 23 presents the unadjusted average total cost and average total sore throat days per randomised arm and the adjusted incremental costs and incremental QALYs that were used to estimate the ICER. Tonsillectomy was, on average, more costly and more effective (in terms of number of sore throat days avoided) than conservative management (see Table 23). For the adjusted analysis, the ICER (incremental cost per sore throat day avoided) was £24 (see Table 23). This value is difficult for decision-makers to interpret given that there is no current threshold available with which it can be compared. It was for this reason that a formal valuation exercise using contingent valuation was conducted. The results of this are shown in this section.

TABLE 23 - Cost-effectiveness analysis results: outcome – sore throat days, results (multiple imputation)

Point estimates are based on the unadjusted analysis (costs n = 448, sore throat days n = 429).

Incremental estimates are based on the adjusted analysis (n = 425).

Investigation strategy	Cost (£) (95% CI)a	Incremental cost (£) (95% CI)b	Effect (95% CI)a	Incremental effect (95% CI)b	ICER (£)	Probability that tonsillectomy is cost-effective for different threshold values for society’s willingness to pay to avoid a sore throat day
						£0	£20	£30	£50	£100
Conservative management (costs, n = 217; outcomes, n = 205)	879 (774 to 984)		55.42 (44 to 67)			1.00	0.72	0.29	0.06	0.01
Tonsillectomy (costs, n = 231; outcomes, n = 224)	1365 (1273 to 1458)	503 (362 to 664)	34.15 (29 to 39)	–21.01 (–33 to –9)	24	0.00	0.28	0.71	0.94	0.99

The deterministic results presented below also do not reflect the imprecision in estimates of costs and sore throat days. For this we need the stochastic sensitivity results, which consider the impression surrounding costs, effects and cost-effectiveness (Figures 17 and 18). Figure 17 illustrates the 1000 iterations of the CEA bootstrapped results for the adjusted analysis. In 100% of these iterations, tonsillectomy is more costly and more effective than conservative management. As Figure 18 shows, at a WTP threshold of £25 to avoid 1 sore throat day, tonsillectomy begins to have the higher probability of being considered cost-effective. This probability increases as the WTP threshold to avoid a sore throat day increases.

FIGURE 17.

Cost-effectiveness plane for tonsillectomy vs. conservative management using the adjusted bootstrapped CEA multiple imputation results.

FIGURE 18.

Cost-effectiveness acceptability curves for tonsillectomy vs. conservative management using the adjusted bootstrapped CEA multiple imputation results.

The results (see Table 23, and Figures 17 and 18) also show that if decision-makers used the WTP value of £40 to avoid 1 sore throat day, which was derived from the contingent valuation question, tonsillectomy has 87% probability of being considered cost-effective.

Sensitivity analysis

Design

An embedded, qualitative study using semistructured interviews.

Setting and sample

The sample for the qualitative process evaluation consisted of recruiting otolaryngology staff, GPs who had patients taking part in the NATTINA trial and ENT patients, including both those recruited to the NATTINA trial and those who declined to participate. Sampling was purposive, seeking maximum variation in terms of age, sex, phase of trial (pilot/main) and randomised arm (including trial participants who crossed over). Sample size was determined by reaching data saturation, whereby no new themes emerged in three consecutive interviews. 123

Recruitment

Recruiting staff (otolaryngologists, research nurses, nurse practitioners and clinic managers) and GPs who had patients taking part in the NATTINA trial were identified through trial records. NATTINA trial participants were asked to consent to be contacted for a qualitative process evaluation interview when consenting to either the pilot or the main trial. Patients who declined participation in either the pilot or the main trial were informed about the qualitative process evaluation interviews by the clinical team and invited to complete an expression of interest form. Otolaryngology staff, GPs and a sample of NATTINA trial participants and declining patients were contacted by telephone, e-mail or letter inviting them to participate in a telephone or face-to-face interview with either Dr Lorraine McSweeney or Dr Lyndsay Lindley (both trained and experienced qualitative researchers) at a time and location convenient for the participants. Invitations were accompanied by a PIS. Verbal consent was given at the time of the telephone interviews with signed written consent returned post interview. Signed written consent was given at the time of face-to-face interviews.

Interviews

Semistructured interviews were based on flexible topic guides (see Report Supplementary Material 3) derived from the literature and issues raised by the NATTINA PPI group, as well as in conjunction with the trial otolaryngologists and GPs. Interviews with trial participants covered expectations and motivations for participating, views and experiences of the randomised arms, and views about sore throat. Interviews with declining participants explored expectations, reasons for not participating and views about sore throat. Recruiting staff and GP interviews explored the practicality and suitability of the treatments, research tasks and randomisation, and any barriers to or enablers of treatment delivery. Interviews for the qualitative process evaluation were carried out during the internal pilot and towards the end of the main trial. Trial participants and declining participants were reimbursed for their involvement in the qualitative process evaluation via a £15 shopping voucher.

Data management and analyses

Interviews, lasting up to 1 hour, were digitally recorded. We then transcribed them accurately word for word. The Framework Analysis, which we used to generate a matrix output [rows (cases), columns (codes) and ‘cells’ of data],124 is regarded as the optimum approach by which the aims of qualitative health research are linked to quantitative investigation. 125 This method allowed coding to be transparent and shared. 125 NVivo software (QSR International, Warrington, UK) facilitated coding. Data were repeatedly read and coded by Dr Lorraine McSweeney/Dr Lyndsay Lindley, with reference to anticipated, participant-reported or data-generated issues. Themes were discussed with the qualitative lead (Professor Catherine Haighton) and the trial team to reduce bias.

Findings

Crossover

In a surgical trial, it is particularly important to assess the impact of crossover (see Table 6) owing to the long-acknowledged risk of accepting a null hypothesis if the non-surgical arm ultimately contains too many individuals who have undergone surgery, resulting in an underestimation of the treatment effect size. 136 In other words, although random allocation eliminates one source of bias it needs to be taken into account as an explanatory factor in the ‘real-world’ results that we observed.

Around 25% of participants did not receive the treatment that they were randomised to, which meant that some opted to not receive a tonsillectomy and some opted to crossover to tonsillectomy. Despite these crossovers, the ITT, per-protocol and per-treatment analysis all confirm that there was a significant reduction in total sore throats for those randomised to tonsillectomy.

Of the NATTINA participants randomised to the immediate tonsillectomy arm, 26% did not receive the intervention, although only 9% actively requested to crossover to conservative management. Of the 172 participants who underwent tonsillectomy, 95 did so within the specified 8 weeks following randomisation. Of the participants allocated to receive conservative management, 33% requested to crossover, with almost 27% undergoing surgery (see Table 7). Interestingly, although the numbers requesting crossover were very different between the two arms, the proportions of those who received ‘other arm’ management were roughly similar.

The impact of the crossover between treatments is to reduce the difference that would be expected to occur between groups. However, our ITT analysis provided evidence that the tonsillectomy arm provided additional benefits compared with the conservative management arm. Further analysis by treatment received explored the extent of the benefit further.

There is evidence to suggest that those with higher rates of sore throats following randomisation were more likely to either crossover to, or remain in, the tonsillectomy arm. Conversely, those with slightly lower sore throat rates were more likely to remain in the conservative management arm or opt out of the tonsillectomy arm. This is intuitively sensible given that those with less severe symptoms are less likely to be willing to undergo an invasive procedure.

Clinical effectiveness

The primary analysis was ITT based on a sample of 429 participants comparing the total number of sore throat days over 24 months from randomisation to the date of last follow-up, while adjusting for the stratification variables (recruiting centre, as a random effect, and baseline severity, as a fixed effect). The data were transformed by dividing each participant’s total number of sore throat days by the proportion of the 105 weeks for which they returned data.

Secondary outcome measures

Safety

There were 244 AEs reported by 119 participants, and 51 SAEs, which were defined as potentially life-threatening, requiring prolonged in-hospital stay or requiring re-admission for treatment. There were 47 re-admissions to hospital in the 231 participants undergoing tonsillectomy (20%). This is at the upper end of the re-admission rate in a Scottish national survey. 8 Four participants had a prolonged in-hospital stay following surgery. There were 54 episodes of post-operative haemorrhage reported in 44 participants. This equates to 44 out of 231 participants undergoing tonsillectomy (19%). Of these participants, 37 (15% of those receiving tonsillectomy) required re-admission. This compares, for example, with 0.5% in a 2009 Norwegian survey. 142 Thus, overall, the rates appear high, but the use of telephone call check-ups at 1 and 2 postoperative weeks served to maximise ascertainment of the true rate of operative complications. Supplementing mailed questionnaires with telephone interviews may increase the validity of surgical outcome studies. 32 The haemorrhage rate observed in NATTINA is important supporting information to inform the content of doctor–patient shared decision-making going forward, given that most published rates, which are reliant on patient self-report, show lower haemorrhage rates.

In a regional study in New Zealand,143 there was a time trend of increasing haemorrhage rates, which the authors postulate may relate to a transition to greater use of diathermy technique and a routine use of post-operative non-steroidal anti-inflammatory drugs and steroids. 143 However, a meta-analysis failed to link peri-operative steroids with bleeding risk. 144 The Cochrane review22 of the use of steroids in controlling post-operative symptoms, such as pain, nausea and vomiting, included over 6000 participants in 64 studies. Intravenous steroids produced a clinically significant reduction in post-operative nausea and vomiting (adult odds ratio 0.32, 95% CI 0.16 to 0.67). Steroids administered intravenously or locally reduced immediate post-operative pain, but there are very few local steroid studies in adults. Haemorrhage rates were unaffected. 39 A systematic review and meta-analysis on treatments for post-tonsillectomy pain in adults and adolescents identified 29 RCTs with over 1800 subjects. The authors of this review concluded that use of several analgesics produced greater pain relief than single agents. 40

Strengths

Trial processes were generally deemed to be acceptable and few sites experienced barriers to treatment. The popularity of our use of technology to collect data was reflected in our respective response rates to texts compared with paper questionnaires. Drop-out rates and missing primary outcome data were higher in the conservative management arm than in the tonsillectomy arm, and the NATTINA statistical analysis accounted for this discrepancy. The additional analyses undertaken supported the primary ITT analysis conclusion. Both the per-protocol and per-treatment analyses are in concordance with the ITT results, demonstrating the reduction in sore throat days in the tonsillectomy arm. The counting of sore throat days, as specified by the funder and as recommended by Cochrane,22 proved a valuable pragmatic metric for a large clinical trial such as NATTINA.

To our knowledge, this is the first economic evaluation investigating the cost-effectiveness of tonsillectomy compared with conservative management in the management of recurring sore throats in adults, a major NATTINA strength. Although the response rates to the data collection tools used in the economic evaluation were lower than anticipated, this was not surprising given the average age of the participants. A similar trend was seen by Bhattacharyya and Kepnes. 71

Our embedded economic evaluation considered costs incurred by the NHS, Personal Social Services and participants, and a wide range of benefits (clinical effectiveness, QoL and patient benefits measured in monetary terms). All three strands were concordant and were further supported by the results of our robust sensitivity analyses that actively sought to explore the impact of changes that could change conclusions. Another strength of the economic evaluation was how the impact of sore throat days on participants’ well-being was incorporated into the analyses. First, participants completed the SF-12 questionnaire when they reported a sore throat episode, and a utility decrement associated with each sore throat episode was integrated into the QALY equation. In this analysis, we identified the negative effect that sore throat days have on participants’ QoL, as the average total QALY per randomised arm was lower than the average total QALY per randomised arm reported in the base-case analysis [base-case analysis mean QALYs: conservative management 1.44 (SD 0.17), tonsillectomy 1.55 (SD 0.15); sensitivity analysis incorporating sore throat days mean QALYs: conservative management 1.38 (SD 0.18), tonsillectomy 1.50 (SD 0.18)]. Second, participants completed a contingent valuation questionnaire at baseline, which assigned a value on the effect that a sore throat day has on their overall well-being. The results of this questionnaire helped us to quantify the burden of sore throat days not only on participants’ health-related QoL but also on all aspects of their life. It also puts the results of the CEA into context for decision-making and was used in the CBA to quantify the reduction in patient benefit associated with sore throat days.

One of the strengths of NATTINA is the number of TOI-14 data that we generated, with well over 900 questionnaires available, roughly equally distributed between trial participants and those declining participation. Those declining the trial had a higher proportion of severe tonsillitis scores, although severity categories were represented within NATTINA. Appendix 3, Figure 19, shows box plots of TOI-14 scores for patients declining and accepting to enter NATTINA.

Limitations

There appeared some evidence that those with the most severe extent of disease were reluctant to enter the trial (see Appendix 3, Table 31, for reasons for declining). Screening log data showed that 1002 out of 1403 (71%) of those giving a reason for declining to participate did so because they wanted to receive a tonsillectomy. We must accept that some symptoms may not be due to tonsillitis, although in a paediatric study, parents appeared clearly to be able to distinguish between the two. 146

Despite the TMG strongly advising to have dedicated clinics for recruitment to the trial, some centres chose not to run these and tended to have not as good recruitment as a result. Staff who lacked equipoise struggled to recruit to the trial, even when patients themselves were in a position of equipoise. All non-recruiting sites were typically research active and displayed concern at their lack of recruitment for NATTINA; therefore, regional variation in referral processes and access to tonsil surgery should also be acknowledged. Clinical research network funding supports nurses for portfolio studies; however, there is a high turnover of staff and prioritisation is a constant battle, particularly for a relatively low headcount recruiting study, such as a surgical trial.

Another challenge for surgical trials is the need for the operative cohort participants to pause their normal life processes for at least 1 week, if not more, which is not the case for medical treatment trials. ENT staff alluded to having to negotiate surgery slots and compromising with patients over preferred surgery dates, which at times meant that they had to deviate from trial protocols.

Not all patients were offered, or chose, to watch the trial recruitment video, which may have contributed to patients’ lack of understanding.

The ITT analysis is likely to offer a conservative underestimate of the true impact of tonsillectomy in reducing sore throat days, as a result of patients crossing over to receive tonsillectomy.

Based on early feedback on non-compliance with the STAR form questionnaires, the STARLET questionnaires were introduced. Unfortunately, a number of participants had changed address or other contact details but did not know how to pass this on to the trial teams, so were not able to receive questionnaires in the post or appointments for follow-up 12-/24-month visits. Simple measures to capture changes in contact details might have further improved response rates in such a young, transient study population. Overall, the design required several amendments to the methodology as lessons were learnt, notably from the response patterns of young adults to information-gathering. Most were incorporated with minimal impact on data robustness, but each was time-consuming for the NCTU staff.

The qualitative study uncovered problems of communication at the primary–secondary care interface, including issues in informing the correct GP that their patient was involved in the trial.

One of the main challenges of the economic evaluation was the progressive loss of data over the 2-year follow-up period. Utilisation data responses were near complete at baseline: 98% for conservative management and 99% for tonsillectomy. These figures reduced to 53% and 52%, respectively, at 12 months, and to 45% and 42%, respectively, at 24 months. SF-12 data responses were similar. The poor response rate to the STAR observed during the trial required its refinement into the STARLET and assumptions were made to maximise the data available, which included creating cost and utility tariffs that were assigned to each sore throat day depending on the self-reported severity of the sore throat episode. In doing so, we estimated the variability in costs and utilities based on clinical severity.

Existing research into medical management of sore throat has primarily focused on one-off episodes of sore throat and, therefore, is of less value in understanding how best to support a recurrent tonsillitis population. The strong evidence of benefit from tonsillectomy in patients with recurrent tonsillitis and the compelling accounts of patients about the impact of their tonsillitis raises questions as to whether or not this group of patients would benefit from earlier identification and about their optimal management in primary care. Optimal strategies for non-surgical management of recurrent tonsillitis should be explored in future research.

Intention-to-treat population restricted to those with 80% STAR data

Although sample sizes were now much smaller (n = 263), for the primary analysis the IRR was similar to when all 429 participants were included with any amount of STAR data (IRR 0.533, 95% CI 0.427 to 0.666; p < 0.001). Similarly for the four category per-treatment, no tonsillectomy randomised to tonsillectomy IRR (CI) was now slightly lower (IRR 0.468, 95% CI 0.300 to 0.734; p = 0.001), tonsillectomy as randomised was slightly higher (IRR 0.584, 95% CI 0.451 to 0.757; p < 0.001), whereas the tonsillectomy crossed over from conservative management was similar (IRR 1.205, 95% CI 0.830 to 1.750; p = 0.326).