A double-blind randomised placebo controlled trial of topical intranasal corticosteroids in 4- to11-year-old children with persistent bilateral otitis media with effusion in primary care

Antibiotics

Re-evaluation of the benefits of antibiotics in OME has shown smaller effect sizes than previously reported by systematic reviews that included poor-quality non-placebo controlled trials (unpublished BMJ Clinical Evidence: last search date, and critical appraisal, March 2007). 2 One systematic review based on eight randomised controlled trials (RCTs) of antibiotics versus placebo included 1292 children. No significant difference in cure rate was found: 179/813 (22%) receiving antibiotics and 85/479 (18%) receiving placebo; absolute risk increase (ARI) of cure 4.3% [95% confidence interval (CI) –0.1% to 8.6%], NNT 23. 12 Prescribing antibiotics encourages belief in them, re-attendance and increasing antibiotic resistance in strains of Streptococcus pneumoniae. 37–42 Antibiotic resistance, medicalising effects, side effects, costs and substantial compliance issues for longer than three or four times a day courses over at least 10 days (and likely to be repeated) renders them now untenable as a treatment for OME. Furthermore, growing concerns about inappropriate use of antibiotics in the community over the past decade have further sharpened the issues for primary care management of children with OME, for which, because of ongoing demand, there is a perceived need to respond positively with some form of treatment. It appears plausible that antibiotics are increasingly misused in this way for OME. 1 (However, it may also be that in general practice recurrent acute otitis media and OME are only loosely labelled in records and hence confused, thus requiring better differentiation.) Antibiotics are not recommended in the recent NICE guideline, so with persistent demand this is likely to lead to displacement prescribing to other ineffective treatments. 4

Decongestants and antihistamines

A systematic review found no difference between antihistamines plus decongestants versus placebo at 4 weeks. 43 However, a considerable number of harms were noted including hyperactivity, insomnia, drowsiness, behavioural change, blood pressure variability and seizures; NNT to harm = 9.

Auto-inflation

Two systematic reviews found limited evidence that auto-inflation improved clearance of effusions compared with no treatment from 2 weeks to 3 months. The earlier review found that children using a purpose-manufactured balloon were more likely than untreated control subjects to gain clearance of effusions: absolute risk (AR) 63/195 (32%) with auto-inflation versus 27/191 (14%) with control [odds ratio (OR) 3.53, 95% CI 2.03 to 6.14]. 44 A second more rigorous review (although the devices were classified differently between reviews) found no benefit before 1 month in 423 patients from four RCTs, relative risk (RR) 2.47, 95% CI 0.93 to 6.58. 45

Difficulties arise for younger children attempting to inflate their Eustachian tubes through the required manoeuvre which also needs to be performed fairly regularly throughout the day to achieve optimal results. This severely limits its use in preschool children in particular, which is the main cohort of children suffering with the condition. However, no serious harms are associated with this approach. Older school-age children may gain benefit from this treatment, particularly when a purpose (mass) manufactured device is chosen. One such device (ear-popper), however, is particularly expensive for a condition with high natural resolution rates.

Oral steroids

The use of systemic steroids has been recommended in combination with antibiotics as being cost-effective in OME, but this is based on a low-quality meta-analysis, which included trials rejected by the Cochrane review. 9,46 Oral steroids to be taken repeatedly for a common but non-life-threatening condition would raise legitimate concerns over the side effects, particularly on children’s growth or severe idiosyncratic reactions. 47 These concerns, in the absence of better evidence of sustained and worthwhile effect from the small and heterogeneous trials included in Cochrane, effectively preclude the use of these steroids for a mild condition with an episodic natural history such as OME. 48–55 There are several theoretical bases for corticosteroid treatment, and these include (1) a direct anti-inflammatory action on the middle ear and Eustachian tube by reducing arachidonic acid concentration, thereby inhibiting the cyclo-oxygenase and lipo-oxygenase synthetic pathways for pro-inflammatory mediators; (2) an increase in Eustachian tube surfactant, improving tubal function; (3) shrinkage of peritubal lymphoid tissue or encroaching adenoidal tissue, thus improving tubal function; and (4) reducing middle ear viscosity through an effect on mucoproteins. 9,56–59

Topical intranasal corticosteroids

Of the theoretical reasons given above, only the third would be anticipated to be a direct benefit from topical steroids, although anti-inflammatory anti-atopic decongestant effects on the nasal mucosa may cause secondary benefits to middle ear drainage and function, for example in a manner analogous to the beneficial effects of topical nasal steroids improving resolution in acute and chronic rhino sinusitis. Thus, on a priori grounds, topical intranasal steroids are a logical treatment for evaluation in OME in children, and are more acceptable with fewer harms than oral corticosteroids that might need to be taken over several months.

Indeed, for these and other reasons, topical intranasal corticosteroids (INCS) are already widely prescribed off licence in ENT departments and to some degree in primary care.

Therapeutic use of topical intranasal steroids in OME has now been identified to be of potential value by the Cochrane review (date of last search January 2002). The review, however, does not actually recommend use of topical nasal steroids, because of insufficient high-quality evidence, although the favourable trial by Tracy and Demain60 was highly rated on methodological criteria. This trial included only 61 children, and was set in a military airbase in the USA, limiting generalisability to a UK general population. Although the paper evaluated short- and intermediate-term efficacy, it did not address the appropriate longer term cost-effectiveness via the broader outcomes necessary for a comprehensive evaluation of this frequently and very variably managed childhood condition. However, this preliminary evidence, if shown to be repeatable in UK general practice, might prove to be highly efficient in reducing referrals by effectively buying many children in the system a disease/disability-free year. This could be maximised by synchronising the critical management decisions and timing of treatment with the major natural seasonal phase of resolution (from winter to summer).Thus any treatment should be aimed at the winter months (the time of maximal incidence) and, due to the relatively slow resolution of OME, should preferably be given for several months. There are some unpublished data in a small cohort of children followed up to age over 4 years (G Scadding, Royal National Throat, Nose and Ear Hospital, 2002, personal communication) with a related publication: a double-blind RCT of Flixonase^® in children aged under 4 years from a tertiary care setting in the UK. 61 This very small trial has good adherence over 2 years’ follow-up and appears observationally effective in preventing recurrences of OME in a severe case-mix group. 61 An older, low-quality study showed no benefits of topical steroids,62 but a more recent small RCT from Turkey has shown benefit from topical steroids in clearing effusions. 63 Serious side effects for inhaled topical steroids are rare, but there are concerns that, as with asthma treatments, growth may be affected. 64,65 This makes it imperative that a topical steroid is used with minimal systemic effects. 65–68

No RCTs from a UK primary care population have been previously performed or published, hence treatment effects are unknown in the main setting in which children present, and thus there is no evidence base to guide the optimal management of the bulk of significant but generally milder cases than seen in hospitals (differences of case-mix limits generalisability to primary care, from published secondary care trials). Any trial on cost-effectiveness needs to consider which groups are most likely to benefit. Thus the current study aims to define what might be feasible and adequate cost-effective temporising management in primary care, by focusing on children with bilateral disease in whom disability is worse and where natural resolution has not occurred quickly, i.e. after tympanometric confirmation, and in the group most likely to be referred, i.e. 4 years and older. Medical treatment in these groups is most likely to impact on NHS resource use. To increase the robustness and stringency of the trial, microtympanometry was used. There is a need to evaluate improved systems of AM and treatment for affected children and their families at a time when demand for surgery could rise again when the TARGET findings are published with some policy expectations for the NHS (changing patterns of NHS use, and an overall increase in referral?). 4 Thus an NHS trial not only should document referral rates in long-term follow-up, but also needs to assess the potential impact of different referral rates and thresholds on secondary management. A well-delivered and well-timed course of nasal steroids has the potential to reduce ineffective prescribing and referral of children for consideration of surgery and so be cost-effective for the NHS. But because it is unclear about the efficacy of nasal steroids as a treatment for OME, the case for their efficacy has to be established first.

In summary, a review of the evidence made it clear that there was a need for a trial of topical nasal steroids in OME with the following features:

• children with previous or recurrent otitis media confirmed as bilateral effusions (OME) on tympanometry

• includes follow-up in the medium to longer term (9 months)

• addresses validated child-centred outcomes (e.g. QoL issues) in addition to audiometry and tympanometry

• uses a topical steroid treatment with low systemic absorption for at least 3 months (during the winter months)

• assesses benefit in those children who are most likely to be referred (i.e. 4 years and older)

• assesses health service resource use and models the impact of potential changes in referral pattern.

Aims

• To test the clinical effectiveness and cost-effectiveness of topical intranasal steroids over 1 year in a pragmatic clinical trial based in primary care.

• To build a health economic model of total health-care utilisation costs for an affected cohort, were such an intervention to be appliedto identifiable children at feasible stages in the health-care system.

Design

The study was designed as a double-blind randomised placebo-controlled trial conducted and reported in accordance with the CONsolidated Standards On Reporting Trials (CONSORT) guidelines. In order to obtain level 1 evidence relating to both efficacy and effectiveness of topical INCS (see Evidence for topical intranasal corticosteroids).

Training

All RNs attended a training day held centrally in London and conducted by the chief investigator, the study manager, MRC senior nursing staff and regional training nurses (RTNs). In-depth training was given on all aspects of the study, including finding potential participants, providing information, taking consent, data protection, the different assessments and procedures. A study handbook provided detailed instructions on all aspects of the study protocol for each RN. The RNs received detailed training on how to use the study equipment (MTP–10 tympanometer) from a representative of the supplying company (Starkey Laboratories). The central co-ordinating team also learnt the techniques in order to troubleshoot any queries the RNs had once they got started. The Starkey representatives also offered their services throughout the entire study for more technical and mechanical queries. Information regarding the nasal spray was supplied by the company (Schering-Plough) and training was also given for the appropriate method of using the spray with the chin up so that the maximal dose to the posterior nasal space was achieved. Quality control visits were performed by the RTNs. They visited each RN three times: (1) to observe consent, (2) to observe the RN completing the baseline measures and (3) on a follow-up visit. The RNs were monitored to check adherence to the study protocol and ICH-GCP. The RTNs also provided support to the RNs whenever required.

Withdrawals

Children were withdrawn according to ethical practice and where any serious adverse event occurred or serious reaction was suspected.

OM8-30

The OM8-30 is a short form assessment for OME that is divided into nine domains: (A) Global Health; (B) Respiratory Symptoms; (C) Ear Problems; (D) Reported Hearing Difficulties; (E) Behaviour; (F) Speech and Language; (G) Sleep Patterns; (H) School Prospects; and (I) Parent Quality of Life (Appendix 6). It produces valid and reliable measures of outcome (scores) for each of these domains (Table 3 gives details). The OM8-30 is a standardised efficient assessment tool, defining cases as to health and development status not just pathology, and can also provide data acting as clinical indicators for treatment decisions. The OM8-30 refers to the 3 months prior to its completion and was completed by the parent (guardian) at baseline and the 3- and 9-month follow-up assessments.

Patient symptoms (diary)

Each parent (guardian) was supplied with a diary at baseline that covered 4 weeks before his or her next assessment in 1 month’s time and then a second diary for 8 weeks up until the 3-month assessment. The diaries were weekly and asked the parent (guardian) to rate how problematic seven symptoms were each week (0 = not present at all, 6 = as bad as it could be) and also the duration of three other symptoms over the week (see Appendix 10 for a sample week page from the diaries).

Impact on child’s life

This was mainly measured using the Costs to Parents form and was to be completed at baseline (Appendix 6), and at the 9-month follow-up by the parent (guardian, Appendix 9). This form also contained additional questions relating to the occupation of the parents to determine their socioeconomic grouping and whether their child suffered from asthma, eczema or hay fever. The diaries that were kept from baseline to the 3-month follow-up also measured impact on the child’s life, e.g. disturbed sleep and days off school/playgroup (see Appendix 10 for a sample week page from the diaries).

Adherence and compliance

Seven days after receiving the nasal sprays, both at baseline and at 1 month post baseline, the RN telephoned the parents (guardians) to ask questions about their children’s adherence and compliance with the spray using a semi-structured interview by going through the adherence forms (Appendices 6 and 7). The parents (guardians) were also asked what spray they believed their child was taking, i.e.active, placebo or unsure. Adherence was also recorded at the 1- and 3-month assessments by the RN asking the parents (guardians) how often their child had taken the medication (Appendices 7 and 8). Spray compliance was also measured more objectively by weighing the used spray bottles at the co-ordinating centre and recording how much spray was used from baseline to the 1-month assessment (spray 1) and between the 1- and 3-month assessments (spray 2).

Referral

At the 1- and 3-month follow-up assessments, the parents (guardians) were asked if their child had been referred to an ENT surgeon and whether or not surgery had been recommended (Appendices 7 and 8).

Adverse events

The parent (guardian) was asked at the 1- and 3-month assessments (Appendices 7 and 8) whether or not any of the following side effects/adverse events had occurred while his or her child was taking the spray: stinging in the nose, nosebleed, dryness and irritation at back of the throat, diarrhoea or cough. The first three of these were also available in the diary each week, for the parent (guardian) to say how much of a problem they had been if present (Appendix 10).

Health-care resources and other economic data

The health-care costs of importance to the economic evaluation were the direct costs associated with usual primary/GP care, costs associated with the interventions and other NHS costs incurred over the 9-month follow-up period.

The use of health-care resources for ear-related problems only, in both primary (e.g.number of GP surgery consultations, number of health visitor consultations, medications prescribed and their dosage) and secondary care (e.g.number of referrals, where to and why) were recorded by the RNs using the children’s general practice medical records and a purposely designed form (Health Economics Evaluation form). This was carried out at baseline retrospectively for 15 months to take into account 12 months prior to the 3-month period of AM (Appendix 6) and at the 9-month assessment for the previous 9 months (Appendix 9), thus giving 2 years of data.

Health-related quality of life

A disease-specific impact on child and family score was to be derived from the OM8-30 questionnaire and used in the health economic analysis. In addition to this disease-specific measure, two generic utility instruments were introduced partway through the trial in order to enable a cost–utility analysis (CUA) to be conducted. These measures were completed at baseline and at both 3 and 9 months. The first instrument comprised the health utilities index (HUI) Mark 2 and Mark 3, 15-item questionnaire for proxy-assessed/administered completion, which includes the questions required to calculate utilities for both the Mark 2 and Mark 3 versions of the HUI instrument.

The second instrument comprised a version of the EuroQoL 5-dimension (EQ-5D) questionnaire. As the health-related QoL experienced by patients with OME will generally be good, the standard EQ-5D questionnaire (which comprises five dimensions, each with three levels) may not be sufficiently sensitive to detect differences between the treatment groups. Subsequently, the trial used a modified version of the child-friendly72 EQ-5D questionnaire (referred to as EQ-5D₅ within this report) that incorporates five levels within each dimension through the insertion of additional tick boxes between the three levels included within the standard EQ-5D questionnaire (Appendix 13). This modification of the questionnaire was proposed by one of the main groups responsible for developing the EQ-5D instrument, which also suggested a number of possible methods for establishing a valuation tariff for this questionnaire. 73 Subsequent research has shown that five-level EQ-5D questionnaires have been found to be more sensitive for mild conditions,74,75 have less of a ceiling effect76 and have higher discriminative ability76 – in terms of sensitivity both to changes over time and to differences between patient groups. 77

Note: The EQ-5D and HUI were not used from the beginning of the study but after a protocol modification (see Changes to the original protocol).

Exit interview

At the 9-month assessment, the RNs used an open question exit interview (Appendix 9) to collect the parents’ (guardians’) and children’s comments about being part of a trial, to ask them what treatment preferences they had and what they will do about the condition now the study had finished. Parents (guardians) and children were able to answer freely and the RNs recorded their responses word for word. These data were therefore qualitative and were analysed accordingly.

Primary outcome

The primary analysis was carried out on an intention to treat (ITT) basis with children as the unit of analysis rather than ears. The proportion of children cleared of bilateral effusions at 1 month in the two groups was compared using a logistic regression model with adjustment for four covariates:

1. season (January/February/March versus the rest of the year)

2. age at randomisation (continuous in months)

3. atopy (defined as the combination of asthma/eczema/hay fever that best predicts outcome in a blind analysis of children ignoring randomisation)

4. clinical severity {defined as the first principal component of the baseline variables: frequency of surgery attendance in last 12 months for ear problems; tympanogram readings; age at first episode of hearing infection/problem; total reported episodes of ear problems over the last 12 months; adenoidal symptom score [respiratory symptoms (RESP) score from the OM8-30] – identified in an analysis of these variables ignoring randomisation group}.

Effect modification

Interaction tests were carried out between randomisation group and each of (a) age, (b) atopy and (c) clinical severity score – defined as above. Interaction tests were carried out using the Likelihood Ratio Test on logistic regression models with and without each interaction (a–c, defined above). In the event that these were statistically significant (p < 0.05), separate results would be presented in subgroups.

Secondary outcomes

The proportion of children cleared of bilateral effusions at 3 and 9 months in the two groups was compared using a logistic regression model with adjustment for four covariates as for the primary outcome.

Results were expressed as ORs with 95% CIs. Subgroup results were not undertaken, as the interaction tests in (b) above were not statistically significant. Differences between active and placebo groups in reported hearing difficulties (RHD), respiratory symptoms (RESP), hearing loss (ACET), physical health and sleep score (PHYS), developmental (DEV) and total OM8-30 scores were investigated using non-parametric tests.

For the main analyses, missing data were assumed to be missing at random and therefore subjects with missing data were not included in analysis. The effect of AM or not was investigated using chi-squared tests for the main tympanometric outcomes at all the time points.

spss versions 12.0 and 16.0 were used for the statistical analyses of all clinical outcome measures.

All statistical analyses on cost and resource use (Chapter 5, Analysis of resource use and costs) were performed using Microsoft excel 2003, and the difference in cost and resource use between the study arms was tested using independent-sample t-tests, assuming unequal variances. All tests were two-tailed and an alpha value of 0.05 was used. Mean total health-care costs, including values imputed using multiple imputation, were calculated using the same methods for utilities imputed using multiple imputation (see point 2 below).

Statistical tests on health utilities (Chapter 5, Analysis of utility measures) were conducted using three different methods:

1. Analysis of utilities of quality-adjusted life-years (QALYs) based on a complete case analysis or using mapped utilities Treatment groups were compared with respect to health utilities using independent-sample t-tests assuming equal variance, which were conducted in stata Version 10.0. Comparison of the treatment groups with respect to categorical end points (e.g. the proportion of patients with no problems on any given scale) was tested using chi-squared tests, including Yates’ correction in cases of 1 degree of freedom (df), which were conducted in Microsoft excel 2003.

2. Analysis of utilities, QALYs or total health-care costs using data sets in which missing data were estimated using multiple imputation Standard errors (SEs) around the means for each treatment group were calculated using Equation (1), later in this chapter. The SEs around the mean difference between the two study arms were calculated for each imputed dataset based on SE²_difference = SE²_treatment + SE²_placebo; these SEs for the five imputed data sets were used to calculate the overall SE around the difference in means using Equation (1). In both cases, p-values were based on t-tests, whereby t equalled mean divided by SE and p was calculated based on the t-distribution in Microsoft excel 2003.

3. Analysis of utilities using a regression-based adjustment for utilities Linear regression analyses to adjust for baseline utilities were conducted using the ‘regress’ command in stata Version 10.0, which conducts ordinary least squares (OLS) regression. In cases in which data imputed using multiple imputation were analysed in this way, the ‘micombine’ option in stata was used to generate estimates of coefficients and p-values that combined all five datasets and allowed for the uncertainty around imputed values.

All tests were two-tailed and used an alpha value of 0.05. The statistical methods used in the analysis of cost-effectiveness are described below.

Primary objective of economic research

The economic evaluation aimed to assess the cost-effectiveness of topical intranasal steroids in the management of OME compared with standard care (without use of steroids) based on the data collected within the trial.

Steroid treatment itself is likely to have at least two economic research aspects, which both relate to clinical effectiveness. The first is the short-term side effects and relief from primary symptoms and direct consequences of the condition on costs and health-related QoL. The second is the long-term effects in terms of reduced disability and any long-term adverse reactions from treatment. This study assessed only short- to medium-term outcomes, although the protocol allowed for extrapolation of the short-term effects and costs over a longer time horizon if the results had demonstrated a difference in short-term outcomes. This longer term modelling would have been based on the natural history of the disease and additional evidence from the literature in the event that the trial revealed significant benefits for intranasal steroids.

The analysis took the perspective of the NHS. Costs incurred by children’s families or education services were excluded from the analysis. Data on the quantity and cost of resources for personal and social services use were not collected due to the practical difficulties of such an analysis.

Two main analyses of incremental cost-effectiveness were conducted. The first analysis comprised a cost-effectiveness analysis (CEA) calculating the incremental cost per additional child with resolution of OME at either 1 or 3 months, while the second comprised a CUA calculating the incremental cost per QALY gained through treatment.

Note A protocol modification was made that involved changes to the collection of the health economic data. The data collection, calculation and analyses are described in Health economic evaluation – data collection, calculation and analysis.

Analysis plan (version 2, dated 16 June 2004)

Following discussion at a TSC meeting in February 2004, a revised analysis plan was written into the protocol – the previous multiple subgroup analyses were removed to reduce chances of false positive findings and need for Bonferroni corrections. The primary and secondary outcome analyses were clearly restated together with potential effect modifications (interactions with age, gender, atopy and clinical severity only). Clinical severity as a first principal component could be clinically useful and was retained in the plan.

Removal of active monitoring (version 3, dated 5 May 2005)

By 18 months into the study it was evident that the rate of recruitment had been slower than expected. This was largely due to an initial 3-month delay (June–August 2003) pending a successful appeal against an initial rejection decision by COREC [Central Office for Research Ethics Committees; now NRES (National Research Ethics Service)], resulting in a 3-month delay in recruiting general practices and RNs (RNs did not start inviting children until January 2004). At the end of 2004 there was a further 3-month delay as a result of the MRC financial restructuring of payments to practices.

The decision was made to relax the rigorous entry criteria and, in line with parent (guardian) and child feedback, GP treatment behaviour and TSC support, the 3-month period of AM was removed from the study design. Thus the inclusion criteria changed:

• Children aged 4–11 years.

• Attendance at the GP surgery with at least one episode of a related ear problem in the previous 12 months.

• Failing tympanometry, i.e. confirmed bilateral OME (B/B or B/C2) tympanograms.

The removal of AM impacted on the case report forms. The Beginning of watchful waiting (AM) form (Appendix 4) was no longer required and the End of watchful waiting (AM) form (Appendix 5) was changed to the First Screening form (Appendix 12).

Collection of more detailed health economic related data (version 3, dated 5 May 2005)

The removal of AM coincided with the departure of the study’s original health economist. Dr Stavros Petrou from the University of Oxford was employed as the replacement and, on his suggestion, changes were made to the health economic capture forms.

The Costs to Parents form was changed to collect data on health-care use as reported by the parent (guardian). This revised form was used at baseline (Appendix 13) and the 9-month assessment (as previously, but it was also included at the 3-month assessment (Appendix 13). At baseline this form covered the previous 12 months, at the 3-month assessment it covered the previous 3 months and at the 9-month assessment it covered the previous 6 months, therefore it provided data over 21 months. The baseline Costs to Parents form also included sociodemographic questions; parents’ (guardians’) educational attainment, their marital status, their child’s ethnicity, whether English is their first language and their gross family income. These forms at baseline, 3-month and 9-month assessments no longer directly considered the impact of OME on the children’s lives, this was now solely obtained through the diaries.

The Health Economics Evaluation form at baseline and the 9-month assessment was also changed. It was disaggregated to cover health-care resource usage for non-ear-related problems as well as ear-related ones. The baseline form looked back over the 12 months before randomisation (Appendix 13), and the 9-month assessment form covered the 9 months the child had been in the study (Appendix 13).

Two new measures were incorporated into the study from this point, the EQ-5D instrument (Appendix 13) and the HUI (Appendix 13). Cost evidence was synthesised with utility data from these two multi-attribute utility measures, in order to estimate the incremental cost per QALY gained attributable to topical intranasal steroids.

The EQ-5D and HUI were completed by the parent (guardian) at baseline, 3-month and 9-month assessments, when possible, with their child’s help.

Table 4 summarises the initial and revised schedules of assessments carried out at each time point. When AM was removed and the revised assessment forms brought in, some children had already been randomised, therefore when they came to their 3- and 9-month assessments the RN used the revised forms. This meant that for some randomised children who had been in the ‘with AM’ part of the study there were some data that were not collected on the other randomised children who had been in AM. These data, where possible, were used for the health economic analyses.

Collection of resource use data

Data were collected about all significant health service resource inputs over the 9-month time horizon of the study. The study data forms provided a record of all appointments with community health-care providers; medications prescribed for the treatment of OME; medication prescribed for other reasons; investigative tests carried out; and hospital inpatient and outpatient service use, which included length of stay, reasons for admission or appointment and any operations carried out, as well as the name of the hospital provider, its location, the duration of contact, and the ward or clinic attended. These data were obtained through two principal means. First, the RN in the GP practice of each child retrospectively completed forms relating to the child’s attendances and prescription of medicine, as well as referrals to hospitals and other community health service providers over the 9-month follow-up period (Appendices 6, 9 and 12). Second, forms were completed by parents of each child relating to their child’s use of medications and hospital and community health services (Appendices 6, 9 and 12). These parent-completed data were used to validate the information collected directly by the RNs, as previous research had indicated that parents are relatively accurately in their recollection of their children’s use of health services. 82 When the parental reports of hospital and community services were compared directly with the data collected by the RNs from the GP practice data collection systems, parental reports tended to underestimate the numbers of admissions, referrals and contacts. It was therefore decided to use the resource use data collected by the RNs in the base-case analyses.

Unit costs

Unit cost for resources used by children who participated in the study were obtained from a variety of primary and secondary sources, with the majority obtained from secondary sources (Appendix 14). All unit costs employed followed recent guidelines on costing health and social care services as part of economic evaluation. 83 Secondary information was obtained from ad hoc studies reported in the literature.

Unit costs of community health and hospital costs were largely derived from national sources84 and took account of the cost of the health professionals’ qualifications. Some costs were valued using the NHS reference costs, a catalogue of costs compiled by the Department of Health in England. 85 Drug costs were obtained from the British National Formulary (BNF). 86 Costs for individual preparations were used as well as costs for chemical entities, i.e. drugs were grouped by chemical entity and unit costs for these chemical entities were calculated. All of the costs are expressed in pound sterling and valued at 2006–7 prices. Unit costs were combined with resource volumes to obtain a net cost per child covering all categories of hospital and community health service costs.

Calculation of utilities and quality-adjusted life-years

The responses to the utility measures collected during the trial (HUI2/3 and EQ-5D₅) were converted into utilities using standard tariffs/weightings. The standard multiplicative multi-attribute utility functions were used for HUI2 and HUI3. 87,88

As described in Health-related quality of life, the study used a five-level child-friendly version of the EQ-5D (the EQ-5D₅) in order to reduce the ceiling effect commonly to be observed when the three-level questionnaire is used in mild conditions. Although there is currently no formal ‘tariff’ for the EQ-5D₅ questionnaire used in the trial, Kind and Macran73 have suggested a number of different possible methods for calculating utilities for this questionnaire: (1) assuming that the coefficients for the intermediate levels lie half-way between those for the three levels for which data exist; (2) rounding responses up or down to the nearest of the main three levels; and (3) using a new set of coefficients generated from a data set for which both measures were used based on the assumption that all levels are equally spaced. The first method (applying the standard N3 tariff,89 while assuming that intermediate levels have coefficients mid-way between those of the standard three levels) was used for the analysis of EQ-5D₅ in order to make use of the potential increase in sensitivity that is conferred by the five-level questionnaire, while minimising the number of assumptions required.

For the base-case analysis, utilities were based on the HUI3 as this instrument has been widely used and validated in children,90–92 and is likely to have greater sensitivity and a less pronounced ceiling effect than the EQ-5D₅. Sensitivity analyses were conducted using utilities based on the HUI2 and EQ-5D₅ questionnaires.

No utility data were collected prior to the protocol changes that removed the AM period and enabled the collection of further health economic data. Furthermore, utility data were also missing for a large number of children recruited after the protocol changes, such that overall around 45% of all potential utility measurements were missing. Subsequently, analyses were conducted to ‘map’ or ‘cross-walk’ responses on the OM8-30 generic measure onto the utility measures used in the trial. These analyses are described in more detail in Appendix 15. Briefly, a range of regression models were investigated to identify the model that best predicted children’s utility based on their responses/scores in the OM8-30 questionnaire and key demographic characteristics. The choice of model was based predominantly on the mean absolute error (MAE) between predicted and observed values, but was also informed by the proportion of predictions that were more than 25% from the actual values, statistics on goodness of fit [R2, root mean squared error (MSE) and information criteria statistics] and the degree of consistency and logical plausibility of coefficients. The best model was a linear regression with suppressed constant that predicted children’s disutility based on their scores for the nine OM8-30 facets, plus predicted hearing level based on tympanometry [ACET (air conduction estimated from tympanometry) from OM8-30]. However, an OLS model with suppressed constant that predicted children’s disutility based on their scores for the DEV and PHYS domains of the OM8-30, RHD, age, sex and predicted hearing level also fitted the data well and was used in a sensitivity analysis. The model was fitted to a randomly selected subset comprising 75% of the observations for which data from both OM8-30 and utility measures were available; the remaining 25% of observations were used to test the model generated. The primary analysis was conducted using the HUI3 as the dependent variable, although analyses were repeated for the HUI2 and EQ-5D₅.

The utilities predicted from the OM8-30 mapping algorithm were used as additional predictors in the multiple imputation (see below). Mapped values were also used directly in sensitivity analyses that used mapped values in place of data directly collected from completion of utility measures in cases in which no utility data were available.

The number of QALYs accrued over the 9-month follow-up period was calculated using linear interpolation (Figure 2). In sensitivity analyses in which multiple imputation was not conducted, missing data at 3 months were overcome by assuming that children’s utility changed in a linear fashion between baseline and 9 months, while missing data at 9 months were overcome by assuming that the child’s utility at 9 months was the same as that at 3 months. Children lacking utility data at baseline and those lacking both 3- and 9-month utilities were excluded from such analyses.

FIGURE 2.

Methods used to calculate QALYs: the area shaded grey represents the QALYs accrued after adjustment for baseline utility. The methods shown in panel (a) were used to calculate QALYs for all children in analyses using data derived from multiple imputation and for those children who had utility data at all three time points in analyses that did not use multiple imputation.

Preliminary statistical analyses highlighted an imbalance in baseline utility values between the two treatment groups. It was therefore necessary to adjust utility values to allow for this imbalance in order to generate an unbiased estimate of treatment effect. For the purposes of the CUA, this adjustment was conducted by simply subtracting each child’s baseline utility value from their on-treatment utilities before calculating QALYs as described above. 93 This method effectively indexes the utilities relative to baseline and calculates the QALY gain or loss that each child has experienced during the study period compared with the QALYs that would have been accrued if the child had remained at their baseline level for the entire study period. For example, if a patient had a baseline utility of 0.7, which increased to 0.8 at 3 months and 0.9 at 3 months, their baseline-adjusted QALYs would be 0.0875: {[(0 + 0.1)/2] × 0.25 years} + {[(0.1 + 0.2)/2] × 0.5 years} relative to baseline, compared with 0.6125 unadjusted QALYs: {[(0.7 + 0.8)/2] × 0.25 years} + {[(0.8 + 0.9)/2] × 0.5 years}. This method of adjusting for baseline utilities was used in the base-case economic evaluation in order to facilitate use of boot-strapping within the CUA.

In order to more accurately assess the statistical significance of any differences in the QALYs accrued between the two groups, while avoiding the problems of regression to the mean, an alternative method of baseline-adjustment of utilities was used alongside the simple subtraction method within the QALY analysis. The regression/ANOVA (analysis of variance) approach was conducted by running a simple linear regression to assess the impact of treatment and baseline utility on the total (unadjusted) QALYs accrued by each child. 93 The coefficient around the treatment dummy variable was used as an estimate of the incremental QALY gain from treatment. However, this approach was not used within the CUA.

Methods for dealing with missing data

Multiple imputation was used in the base-case analysis in order to overcome any biases associated with missing data and fill in all gaps within the data on costs, clinical end points and utilities. Multiple imputation was conducted using the ‘ice’ command within stata 10.0, which employs widely used statistical techniques which have been described in previous work. 94,95 In order to allow for the highly skewed distribution of utilities and (to a lesser extent) costs, disutilities and costs were transformed onto a log scale using the transformation A_Trans = [ln(A + 0.00001)], in which A is the untransformed value and ln is the natural logarithm. The constant 0.00001 is used to enable values that are equal to 0 to be transformed onto a logarithmic scale.

In order to make use of the fact that OM8-30 scores correlate with utility (Appendix 15), an algorithm was developed to estimate utilities based on responses to the OM8-30 (Appendix 15). The predicted disutility that the child would be expected to have at each time point based on his or her OM8-30 facet scores was included in the imputation analysis in addition to demographic characteristics, costs and utilities. This predicted disutility was included in the analysis in preference to the OM8-30 facet scores, as it was anticipated that including 30 additional variables (HL and the nine OM8-30 facet scores that were observed at each of the three time points) would prevent estimation of any reliable imputation model. No transformation was applied to the predicted disutilities, as the predictions followed a symmetrical and approximately normal distribution; furthermore, predicted negative disutilities were left as negative values in order to preserve the distribution and reflect the OM8-30 responses more accurately.

The following variables were included in the imputation analysis:

• age: no missing data

• sex: no missing data

• study protocol (dummy for whether the child was recruited before or after the protocol change that removed the AM period): no missing data

• treatment allocation (dummy for whether the child received active treatment or placebo): no missing data

• total cost based on retrospective review of children’s medical records (transformed on a log scale; match option used)

• total cost based on parents’ costing questionnaire (transformed on a log scale; match option used)

• HUI3 utility at baseline (transformed on a log scale; match option used)

• HUI3 utility at 3 months (transformed on a log scale; match option used)

• HUI3 utility at 9 months (transformed on a log scale; match option used)

• HUI2 utility at baseline (transformed on a log scale; match option used)

• HUI2 utility at 3 months (transformed on a log scale; match option used)

• HUI2 utility at 9 months (transformed on a log scale; match option used)

• EQ-5D utility at baseline (transformed on a log scale; match option used)

• EQ-5D utility at 3 months (transformed on a log scale; match option used)

• EQ-5D utility at 9 months (transformed on a log scale; match option used)

• predicted HUI3 disutility at baseline that was calculated using the HUI3 facet model described in Appendix 15 (match option used). A mapping model was used to predict HUI3 utility based on patients’ facet scores on the OM8-30 questionnaire, based on a randomly selected subset of 75% of patients in the GNOME study who completed the HUI3 and OM8-30 questionnaire fully

• predicted HUI3 disutility at 3 months that was calculated using the HUI3 facet model described in (match option used)

• predicted HUI3 disutility at 9 months that was calculated using the HUI3 facet model described in (match option used)

• composite clinical outcome (dummy variable indicating whether the child had been cured at 1 or 3 months).

The match option was used for multiple imputation, as even the log-transformed utility and cost variables had significant skew and differed significantly from a normal distribution. The match option works by generating predicted values for each child for each variable (including those children with complete data) based on linear or logistic regression functions; the predicted value for each observation with missing data is then compared with that for children who have a value recorded for the variable in question, and assumes that the value in question is equal to the closest match. This constrains the imputed values to be within the range of values that were observed and is less dependent on assumptions of normality. Standard imputation (without using the match option) produced implausible values for costs and utilities, even after log transformation of the data; in particular, utility values well below zero were imputed for many children, despite the fact that no children in the sample had HUI3 utilities below zero. By contrast, when the match function was used, all values generated were plausible, and the distributions, means and standard deviations (SDs) of the imputed data were similar to the observed values.

Other than use of the match option, the default assumptions for the ‘ice’ command were used for the imputation analysis; this involved use of logistic regression for the composite clinical end point, and linear regression for all other variables. Five imputed data sets were generated. The values generated within each imputation were transformed back to a natural scale where necessary using the reverse of the transformation formula shown above. Children’s utility was assumed to be one if a value of one was imputed for the perfect health variable and was otherwise based on the value imputed for the HUI3 utility. Bootstrapping was conducted on all five data sets generated in the multiple imputation in order to allow for uncertainty between imputed datasets when calculating 95% CI and the probability that treatment is cost-effective94 as described in Calculation of cost-effectiveness ratios.

However, several different approaches to dealing with missing data were investigated within sensitivity analyses. Firstly, a complete case analysis was conducted, whereby the results of multiple imputation were not used and only children with complete data on costs and outcomes were included in the analysis. In the CUA, the complete case analysis included only patients with complete cost data who completed the HUI3 questionnaire at baseline and both 3 and 9 months after start of treatment; the CEA complete case analysis included only patients with clinical outcome data at either 1 or 3 months who also had complete cost data. Secondly, two analyses were conducted in which mapped estimates for HUI3 utility were included alongside observed utility values; these analyses included only those children with complete resource use data and utility data at baseline and at either 3 or 9 months from HUI3 or the OM8-30 mapping exercise. Thirdly, the impact of alternative imputation methods was investigated for the CEA in the form of a mean imputation analysis (whereby all missing data on costs or clinical outcomes were assumed to equal the mean value for the relevant treatment group); a best case analysis (whereby all children missing clinical outcome data were assumed to have been cured); and a worst case (or ITT) analysis (whereby all children with missing clinical outcome data were assumed to not have been cured).

Calculation of cost-effectiveness ratios

As described above, the primary clinical outcome measure for the study was the presence or absence of OME (i.e. cured or not cured) 1 month after starting the course of intranasal steroids, with adjustment for covariates. Clinical outcome data for the study were collected at 1, 3 and 9 months. However, the 9-month outcome data were not used for the CEA. The rationale for this came from previous research,96 which has shown that children who are susceptible to OME tend to have more separate episodes of effusion rather than an increased overall duration of episodes. Such children are primarily distinguished by the likelihood with which they acquire the disease than by their ability to recover from it. Thus, any outcomes observed beyond 3 months might not be due to the active treatment, but could be attributed to the fluctuating nature of the condition. Hence, the 9-month data were not used because of this highly recurrent nature of OME.

Instead of using 9-month data, the base-case CEA used a composite outcome measure that was created using the 1- and 3-month data. This composite measure was created by assuming that if a child was cured of OME at either 1 or 3 months, they were considered cured and if a child still had OME at both time points, they were considered not cured. For children missing data at one of these time points, outcomes were based on outcomes at the time point at which data were available.

This end point was chosen in preference to the primary clinical end point (cure at 1 month adjusted for covariates) as it has a number of advantages. Firstly, the composite end point combines two different trial end points by allowing for children who are cured by the end of treatment as well as cures occurring by 1 month. Furthermore, this end point requires less imputation of data than cures by a single time point. It is common for trial-based economic evaluations to use a different (generally longer) time horizon than clinical end points and in this case, it would not be practical to use exactly the same primary end point as that used in the clinical analysis (the proportion of children cured by 1 month, after adjustment for covariates) without adopting a net benefit framework and greatly complicating the analysis. However, the choice of clinical outcome used in the CEA was varied in sensitivity analyses, which calculated cost-effectiveness based on outcomes at 1, 3 or 9 months.

For both the CEA and the CUA, differences in mean costs and effects between the groups were calculated. The incremental cost-effectiveness ratio (ICER) was calculated as the difference in costs (ΔC) divided by the difference in effects (ΔE). For the CEA, cost-effectiveness was expressed as the incremental cost per case of OME cured. The CUA calculated the cost per QALY gained, with QALYs being calculated based on the methodology described above.

Both the CEA and the CUA took a 9-month time horizon for costs within the base-case analysis, as this comprised the maximum duration of the trial and ensures that any difference in costs or health-care resource use that results from the intervention was captured. No discounting of future costs or benefits was applied as the time horizon was less than 12 months.

To account for the skewed nature of the cost and utility data, non-parametric bootstrap estimation was used to calculate the probability that treatment is cost-effective and derive the 95% CIs for mean incremental costs and benefits between the placebo and treatment groups.

For the base-case analysis and all subgroup or sensitivity analyses that included data that were imputed using multiple imputation, the five imputed data sets generated using multiple imputation were bootstrapped separately in order to allow for the uncertainty between (as well as within) imputed datasets. 94 For the base-case analyses of both the CUA and the CEA, 5000 bootstrap replicates were conducted for each of the five imputed data sets; 1000 bootstrap replicates were conducted for each data set within all sensitivity and subgroup analyses except for those that involved a complete case analysis or imputation techniques other than multiple imputation (which used 1000 bootstrap replicates for the single data set used in those analyses).

The mean costs and mean benefits (the mean number of QALYs or mean number of children cured) were based on the average of the raw data for all imputed datasets, which was equal to the mean of the means from each of the five imputed data sets. In order to allow for both sampling uncertainty and uncertainty around imputed values, the SEs around the mean costs, mean benefits and the mean difference in costs/benefits were calculated using the equation described by Briggs et al. 94

where M represents the number of imputed data sets generated (in this case five), θ^i represents the parameter of interest for data set i, var^(θ^) represents the variance (SE2) around the parameter of interest across all data sets (including both variability within and variability between data sets) and var^(θ^i) represents the within-data set variability for data set i.

The SEs calculated using this equation were used to calculate 95% CI based on a t-distribution with df equal to (M–1)(1+r^1-)². 94

The proportion of variability that was due to uncertainty around imputed values was calculated by dividing the term

by the total variance var^(θ^).

Estimates of the probability of treatment being less costly, more effective, dominant or dominated relative to placebo at different ceiling ratios were calculated across all bootstrap replicates for all five imputed datasets.

Uncertainty around the conclusions about whether or not treatment was cost-effective was represented in the form of a cost-effectiveness acceptability curve (CEAC). 97 This shows the probability of intranasal steroids being cost-effective at a range of maximum values (termed ceiling ratios, R_c) that decision-makers may be willing to pay for an additional case of OME cured or an additional QALY. The CEACs and the probability of treatment being cost-effective were calculated based on the proportion of simulations (across all five imputed data sets) with positive net benefit at a range of ceiling ratios. CIs and SEs around the mean costs and benefits were calculated by assuming normality. The bootstrap simulations of the ICER were plotted on the cost-effectiveness plane to give a non-parametric illustration of the joint density of costs and effect differences.

In cases in which the results of multiple imputation were not used (e.g. for the complete case analyses and analyses using mapped utilities), bootstrapping was conducted on a single data set, using 1000 bootstrap replicates. In these analyses, SEs and CIs were calculated using a standard parametric approach and CEACs and the probability of treatment being cost-effective was calculated across all bootstrap replicates run.

Extrapolation and additional analyses

The trial protocol allowed for the option of constructing decision-analytical models to extrapolate the results of the trial beyond the 9-month time horizon using additional data taken from the literature to calculate the long-term impact of treatment on costs and benefits, including allowing for the incidence of disability and surgery. However, given that the analysis found no evidence that treatment conferred significant clinical benefits, the trial results were not extrapolated beyond the end of the trial.

Sensitivity and subgroup analyses

In addition to the base-case analysis, a number of sensitivity and subgroup analyses were conducted for both the CEA and the CUA.

Two sensitivity analyses were common to both CEA and CUA:

1. basing costs on parents’ (or guardians’) responses to resource use questionnaires completed at 3 and 9 months

2. adding in the cost of tympanometry at baseline for all children.

Seven sensitivity analyses specific to the CEA included:

1. mean imputation of costs and clinical outcomes for children for whom data were missing

2. estimating cost-effectiveness in terms of incremental cost per case of OME cured at 1 month

3. estimating cost-effectiveness in terms of incremental cost per case of OME cured at 3 months

4. estimating cost-effectiveness in terms of incremental cost per case of OME cured at 9 months

5. worst case/ITT analysis: assuming all children with missing outcome data were not cured at either 1 or 3 months

6. best case analysis: assuming all children with missing outcome data were cured at either 1 or 3 months

7. complete case analysis: including only those patients with no missing data on the composite end point or on costs.

Sensitivity analyses 1 and 5–7 did not use the results of multiple imputation and were therefore based on 1000 bootstrap replicates of a single data set. Analyses 2–4 used a different run of multiple imputation in which the composite end point was not imputed, but the clinical outcomes at 1, 3 and 9 months were included as three separate variables. This was conducted to minimise the number of variables used in the imputation run used for the base-case analysis (which was necessary to ensure that stable and realistic estimates of missing data on utilities, costs and composite clinical outcomes were generated. Other than this change to the clinical outcome data, this run of multiple imputation was conducted using the same methods as the base-case imputation analysis, although outcomes differ slightly due to the change in the variables used.

A further six sensitivity analyses relating to the calculation of QALYs were conducted for the CUA:

1. use of utilities based on responses to the EQ-5D₅ questionnaire

2. use of utilities based on responses to the HUI2 questionnaire

3. making no adjustment for baseline utilities

4. complete case analysis: including only those patients for whom the HUI3 questionnaire was fully completed at all three time points and who had complete cost data

5. using HUI3 utilities predicted using the mapping model of OM8-30 facet scores that is described in Appendix 15 instead of values estimated using multiple imputation

6. using HUI3 utilities predicted using the mapping model of OM8-30 domain scores (plus age and sex) that is described in Appendix 15 instead of values estimated using multiple imputation.

The following six sets of subgroup analyses were conducted for both the CEA and the CUA, in which the incremental costs per additional unit of health outcome were calculated for the following subgroups of children:

1. children younger/older than 6.5 years at baseline

2. boys/girls

3. children with/without atopy symptoms at baseline

4. children with severe/non-severe disease at baseline, defined by whether the child’s clinical severity score was in the worst 25% of the cohort; this equated to cases with clinical severity scores of 0.62 or higher being classed as severe

5. children recruited to the trial in January, February or March, compared with those recruited between April and December

6. with/without AM (i.e. before/after the protocol changes described in Changes to the original protocol).

All of the subgroup analyses should be considered post hoc as they were not pre-specified. However, this list comprises an exhaustive list of the subgroups investigated and no subgroup analyses that were conducted are omitted from this report.

Screened children

Table 8 details the characteristics of the 2093 children who were screened. The minimum age shown in Table 8 is 44 months, this was 4 months less than the age at which children could be randomised (i.e. 48 months, 4 years). This lower age was acceptable at screening in children who were in the first part of the study with AM as this was a period of 3 months, and with the difficulties in scheduling assessments, 44 months was the lower limit for inviting children for screening. Eligible children had to be 48 months old (4 years old) at the point of randomisation as per the inclusion criteria.

Table 9 details the baseline characteristics of the 217 children randomised and of their parents (guardians) by the different treatment groups for 13 variables of potential significance as confounders. There were no obvious differences between the two treatment arms of the study other than the ratio of males to females, 1:1 in the active groups and 1:0.8 in the placebo group.

Timing of follow-up assessments

The follow-up assessments were at 1, 3 and 9 months post baseline; the RNs were instructed to conduct these assessments as close as possible to the time intervals required, subject to the child’s availability. All three assessments required some flexibility due to the limited times at which the RNs could see the children (the study age group meant most of the children were at school). Unless an appointment coincided with a school holiday, the only times the RNs could see the study children was from 4 [sc]pm onwards. This restricted time frame meant that if an appointment was missed the next convenient or available one could be a week later, thus delaying the assessment. Table 10 provides the timings of each follow-up assessment.

Table 11 shows the data that were available at each assessment from baseline onwards. Ninety-three per cent of children returned for their 1-month follow-up, 83% for their 3-month visit and 75% for their 9-month follow-up assessment. These figures are not shown in the CONSORT diagram (see Figure 3) as this uses follow-up figures related to the main outcome measure of the study, i.e. the tympanometry measuring presence of ear effusions at 1, 3 and 9 months post baseline, therefore the CONSORT percentages for follow-up, 89%, 79% and 66% respectively, are based upon numbers of children having tympanometry performed at each visit.

Exclusions and losses to follow-up

Children who were not screened having attended their first appointment [56 (4%) with AM and 36 (4%) without AM] mainly possessed an exclusion criterion or did not have all the inclusion criteria. Table 12 lists the reasons for exclusion at this stage.

Of the 281 children who entered AM, 84 (30%) were lost to follow-up, i.e. they did not return at the end of the 3-month period. Figure 3 details the reasons for the loss at this stage. The majority of these (n = 49, 82% in the ‘Other’ category of Figure 3 for this stage) were due to the break in trial spray supplies by Schering-Plough. This break in supply meant that if children did return, and were screened and failed (thus eligible) they could not be randomised as there was no trial medication. It was decided not to bring these children back at their allotted time, and the parents (guardians) were notified of the situation and offered a screening that would not be part of the trial. When the trial medication supply was recommenced, these children were rescreened and entered into AM if they failed the tympanometry. However, a review of the protocol (version 2, dated 16 June 2004) took place and AM was removed from the methodology (version 3, dated 5 May 2005) during the break in medication supply. Many of these children who were effectively ‘stuck in AM’ were reinvited in the autumn of 2005 for rescreening and, if eligible, they were randomised into the trial, now without AM. Therefore, most of these children were not lost from the study, they were assigned a new study number (NB: a previously randomised child could not be reinvited into the study).

Two hundred and sixty-one children in total were eligible for randomisation, 15 of the 87 (17%) with AM and 29 of the 174 (17%) without AM were lost at this stage. Figure 3 details the reasons for these 44 children not being randomised. The ‘Other’ category in the AM group, n = 10, consisted of scheduled appointments that coincided with notification of the break in trial medication supply (children were screened as they entered the surgery for their appointments), as did the without AM group, n = 3.

The losses to follow-up post randomisation are given in Figure 3 for the active and placebo groups. Persistent non-attendance (all RNs followed the same procedure from their handbook for non-attendees, the child was considered lost to follow-up only after two telephone calls and one recorded delivery reminder to the parents) did not differ between the two groups. New losses to follow-up did not increase over time with each successive follow-up assessment as may be expected but peaked at the 3-month assessment (at 1 month 22% of total losses were non-attendees, 48% at 3 months and 32% at 9 months). Parent/child choice withdrawals in total decreased over time (1 month 43%, 3 months 33% and 9 months 16%); however, there were over twice as many withdrawals of this kind in the placebo group overall compared with the active treatment group (n = 17, 39% versus n = 7, 19%). Non-interpretable tympanometry was consistent across the assessments with n = 4; however, tympanometry not done, which was due to the presence of grommets, was variable with three children at 1 month, one child at 3 months and seven children at 9 months. As expected, more children had had grommets by the 9-month assessment as, by taking part in the study, a potential hearing problem had been highlighted to the parents/GPs, and therefore referral was more likely.

Clinical outcomes

Of the pre-specified potential effect modifiers in the protocol analysis plan [age continuous variable p = 0.93; season (school spring term January–March versus other months dichotomous) p = 0.69; atopy (any history of asthma, eczema, hay fever/allergic rhinitis versus none) p = 0.61; and clinical severity score (defined as the first principal component – accounting for 24.6% of the variance with an Eigen value of 1.38 – of the following baseline variables: frequency of surgery attendance in last 12 months for ear-related problems; total parent-reported episodes of ear problems over last 12 months; age of first episode of hearing infection/problem; tympanogram readings; OM8-30 RESP score (adenoidal factor) p = 0.006], only clinical severity had an effect on outcome (Table 13). These variables were used in the logistic regression to derive AORs for the main outcomes.

Interaction tests were then carried out between randomisation group and each of age, season, atopy and clinical severity score – defined as above (Table 14).

Three models containing the interactions listed in Table 14 (Models 2, 3 and 4) were compared with Model 1 (containing no interaction) using the likelihood ratio test. The likelihood ratio statistic was compared with a chi-squared distribution with 1 df, whose critical value to reach significance at the 5% level is 3.841. It can be seen from Table 14 that none of the interactions included in Models 2, 3 and 4 reached significance at the 5% level.

The main outcome was based at 1 month and, using objective tympanometric criteria for children cured, the AOR of 0.93 (95% CI 0.50 to 1.75) favoured placebo treatment (Table 15 and Figure 4). The 95% CIs include an OR of 1 and so were not statistically significant. The risk reduction of the treated group calculated at 1 month was –4.3% (95% CI –18.95% to 9.26%). The effect size using the upper 95% confidence limit therefore is not likely to contain a clinically useful effect, i.e.be less than an NNT of 11 for a 1-month course of treatment.

FIGURE 4.

AORs with 95% CI for active versus placebo comparative efficacy at 1, 3 and 9 months post baseline.

Thus nasal steroids are very likely to be clinically ineffective because they are no better than placebo in producing resolution of middle ear effusions at 1 month, and also have non-significant secondary outcome efficacy in clearing effusions at 3 months (see Table 15 and Figure 4) and demonstrate no longer term efficacy at 9 months (see Table 15 and Figure 4).

A sensitivity analysis was performed because some children in this study received AM for 3 additional months. When the inclusion criteria of previous history and bilateral fail criteria on two occasions 3 months apart were changed later in the trial to previous history and a bilateral fail on one occasion (i.e.without the addition of AM), a sensitivity analysis was required, which found that there was no significant difference in cure rates whether children were in the AM group or not. It can be seen that there was no significant difference between the odds of cure for the two groups (with and without AM), as the CIs overlap (Table 16).

Testing for an association between AM and tympanometry pass rate at 1 month using a chi-squared test did not give a significant association in the placebo group (p = 0.726) or in the active group (p = 0.378). Similarly, testing for an association between AM and tympanometry pass rate at 3 months using a chi-squared test did not give a significant association in the placebo group (p = 0.143) or in the active group (p = 0.186). These findings support the view that topical steroids are inefficacious even in the more persistent cases as defined by tympanometry fails on two occasions 3 months apart (one definition of severity).

The method of study entry, either by computer audit (c) or in-house referral at presentation (i) did not effect tympanometric outcomes. At 1 month 58% (c) versus 55% (i) were not cured; at 3 months 45% (c) versus 43% (i); and at 9 months 40% (c) versus 33% (i).

Effectiveness outcomes

The OM8-30 measure, developed by the MRC, showed equally null results for this important secondary outcome at 3 and 9 months. The main outcomes presented are median scores with interquartile ranges (IQRs) based on scales developed by the MRC for use with this questionnaire: a total score (p-values at baseline, 3 months, 9 months; p = 0.33, p = 0.55, p = 0.77 respectively) (Figure 5); DEV (p = 0.94, p = 0.83, p = 0.24) (Figure 6); RESP (adenoidal factor) (p = 0.83, p = 0.22, p = 0.17) (Figure 7); PHYS (p = 0.41, p = 0.91, p = 0.69) (Figure 8); and RHD (p = 0.32, p = 0.08, p = 0.47) (Figure 9).

FIGURE 5.

Box plots of total OM8-30 score at baseline, 3 months and 9 months showing median and IQR by treatment group (n = 39 in active group and n = 43 in placebo group at baseline, 3 and 9 months).

FIGURE 6.

Box plots of DEV at baseline, 3 months and 9 months showing median and IQR by treatment group (n = 48 in active group and n = 55 in placebo group at baseline, 3 and 9 months).

FIGURE 7.

Box plots of RESP at baseline, 3 months and 9 months showing median and IQR by treatment group (n = 68 in active group and n = 66 in placebo group at baseline, 3 and 9 months). + indicates outliers.

FIGURE 8.

Box plots of PHYS at baseline, 3 months and 9 months showing median and IQR by treatment group (n = 58 in active group and n = 53 in placebo group at baseline, 3 and 9 months).

FIGURE 9.

Box plots of RHD at baseline, 3 months and 9 months showing median and IQR by treatment group (n = 69 in active and placebo groups at baseline, 3 and 9 months).

In the following figures, missing data resulted in low analysis rates for the specific questionnaire-derived scores because of validity issues. Those presented required complete data for every question on the 30-item questionnaire. The box plots present the median and IQR and the vertical lines show the range unless the point is considered to be an outlier as determined by spss version 12.0. (The definition of an outlier in this statistical package is 1.5 times the IQR.)

All these scales, despite the probability of a false positive outcome, showed non-significant differences between groups. What was also clearly apparent from the figures was a consistent recovery process by 3 months with little further gain by 9 months. While the MRC measure is currently widely validated as part of the Eurotitis study, its validity against a QoL measure is yet to be fully determined. Nonetheless, as a disease-specific functional health status measure it is likely to be the most sensitive and responsive psychometric measure currently available to evaluate impact of OME on a child’s symptoms and life. The sample size analysed would be expected to detect 0.5 SD effect on the scales, which would be clinically important. The null findings of effectiveness found here reinforce further the null tympanometric efficacy findings between active and placebo arms.

Four measures of hearing were used in this study: two subjective, the reported HL scale on the OM8-30 (see Figure 9) and days with reported hearing loss on the prospective child’s 3-month diaries (Mann–Whitney test for prospectively recorded data in 3-month diaries showed no significant differences between groups, p = 0.45 for days with suspected hearing loss); and two objective, pass fail on sweep hand-held audiometers at 25 dB HL (fail on more than two frequencies both ears at 0.5, 1, 2, 3, 4 kHz) (Figure 10) and the audiometrically validated scale ACET, a continuous severity scale for middle ear function (Figure 11). None of these outcomes, although improving over time, showed any significant differences between groups (Table 17).

FIGURE 10.

Percentage of children failing audiometry at baseline, 1, 3 and 9 months (± 95% CI).

FIGURE 11.

Box plots of ACET at baseline, 3 months and 9 months showing median and IQR by treatment group (n = 53 in active group and n = 57 in placebo group at baseline, 3 and 9 months).

A Spearman correlation showed a moderate correlation between the two subjective measures (the reported HL scale on the OM8-30 and days with reported hearing loss from the 3-month diaries) (r = 0.567, p < 0.001).

Days with otalgia or earache were considered an important secondary outcome for which 3-month prospective diary information was collected and also retrospectively measured on the OM8-30. Days with otalgia were not significantly different between treatment groups at 1 month [p = 0.43; median (IQR): placebo = 0 (0–2.25), active = 0 (0–3)] nor at 3 months [p = 0.46; median, IQR: placebo = 1 (0–4), active = 1.5 (0–5)].

Adverse events/side effects

Adverse events/side effects are presented in Table 18. In total, 45 children (45/96, 46.9%) in the treatment group and 35 (35/98, 35.7%) in the placebo group reported side effects at 1 month, and 29 (29/86, 33.7%) and 23 (23/86, 26.7%) respectively at 3 months. No serious adverse events, suspected serious adverse reactions or related hospitalisations occurred during the study. At 3 months more side effects were reported from use of the active sprays particularly in relation to nosebleeds, dry throat and cough. Side effects were relatively minor but may have affected a child’s QoL.

Adherence

Reported adherence data are presented in Table 19. At 1 month the reported adherence was 96% for the active treatment and 90% for the placebo, and 88% active and 88% placebo at 3 months (good compliance was considered to be when the parent reported the child as having taken the spray most or all of the time, see Table 19). A structured support adherence questionnaire was used to improve reporting (Appendices 6 and 7).

The weights of the returned spray bottles were compared with the reported adherence data. Figures 12 and 13 show the relationships between the amounts of spray used and the reported usage. The percentage compliance was in excess of 100% for some children, most likely because the nasal applicator on the sprays had a tendency to become blocked and required cleaning. Following such a procedure the RNs were instructed to tell the parents (guardians) that they must reprime the spray (seven actuations). The predicted used weight denominator was determined as an initial priming and one spray per nostril per day for either 28 days (baseline to 1-month assessment) or 56 days (1-month to 3-month assessment), no prediction was made for numbers of times the device may have required cleaning and therefore repriming. The other reason for more than 100% use could be a non-compliance issue with the spray being wasted to mimic adherence. However, the scatter plots (see Figures 12 and 13) show a positive relationship between the compliance determined by spray weight used and the reported usage, suggesting that on the whole the sprays were used correctly. No statistics were performed on these data as this was not in the protocol (version 3, dated 5 May 2005).

FIGURE 12.

Percentage compliance from the spray bottle weights vs the reported adherence from parents (guardians) at 1 month.

FIGURE 13.

Percentage compliance from the spray bottle weights vs the reported adherence from parents (guardians) at 3 months.

Concealment

Concealment was evaluated by asking the parents (guardians) to report whether they thought their child was taking active or placebo spray at 7 days post baseline and 7 days post the 1-month assessment (i.e. approximately 35 days post baseline) (see Appendices 6 and 7). The percentage correct guesses were no better than chance alone (Table 20). However, most parents (guardians) thought their children were receiving active treatment (83.5% at 7 days post baseline and 65.2% 7 days post 1-month assessment); this demonstrates that blinding for the study was satisfactory with good concealment but confirms the placebo effect (bias) in the study.

Referrals

Overall, few referrals were made, 32 (taken from health-care usage audits carried out at 9-month assessment, see Appendices 6, 9 and 12) out of 217 or 14.7% over 9-month follow-up, 15 in the active group and 17 in the placebo group. This number was lower than anticipated and may reflect the introduction of AM, a new treatment most parents (guardians) thought was active, or a Hawthorn effect, or all three.

Clinical severity score

This score appeared to be normally distributed and therefore t-tests were used to test between groups. A higher baseline clinical severity score meant that the child had a more severe condition. There was a significant difference in baseline clinical severity score between children who passed and failed tympanometry at 1 month (p = 0.004). This result was expected from the outcome of the logistic regression, which was a better method of analysis as it corrected for other factors such as age (as a continuous variable), treatment group and atopy in the model (see Clinical outcomes). There was no significant difference in baseline clinical severity score between children in the different treatment groups (p = 0.128).

Clinical severity score and age group

There was a significant difference in baseline clinical severity score between children in the different age groups (4–6.49 years and 6.5+ years) (p = 0.023). Table 21 describes the two age groups in terms of the clinical severity score. The children in the 6.5+ years age group are significantly less severe at baseline.

Age group and tympanometric cure outcomes

Risk estimates are presented as ORs with the 95% CIs for tympanometric cure at 1, 3 and 9 months by age group (Table 22). Although there appeared to be a significant OR for treatment of older children at 3 months, the 95% CIs for the two populations overlapped and so this was not significant. An association between dichotomised age and pass rate at 3 months by treatment allocated was tested using the chi-squared test and showed no significant difference for either treatment (placebo group p = 0.146 and active group p = 0.07).

RESP score (OM8-30) and age group

There were no significant differences in this score at baseline in the active treatment group, but the younger group had a significantly worse score in the placebo group (U-test p = 0.048). Dichotomised age had no effect on the RESP score at 3 months – placebo group p = 0.14, active group p = 0.57 – to suggest an effect of the treatment on the adenoids in relation to age effects.

Adherence and age group

The analysis combined placebo and active groups as parents and children were blinded to treatment group. There was no difference in adherence at 1 month between the two age groups (Fischer’s exact test, p = 0.61). At 3 months p = 0.04, which, although significant, probably did not represent a true difference as no linear-by-linear trend associations were found (p = 0.40).

Natural history and risk factors

Tympanometric criteria for resolution are the main efficacy outcomes for the entire follow-up period and, with a null trial for efficacy found, any contingent clinical effectiveness outcomes therefore require robust explanation. Very few trials actually report 9-month outcomes which are important in assessing clinical effectiveness, because with 41.9% non-resolution at 3 months post baseline treated and 34.7% non-resolution untreated and significant relapse rates for OME in both groups after 3 months of the intervention, the condition is likely to continue to cause and further NHS treatment in relation to re-attendance and referral. Unless there are over-riding concerns, the high proportions showing a natural resolution (similar treated or untreated) should be sufficient to support more widespread AM of children in a primary care setting for a 3-month period, over which time natural cure is probable with low-cost structured support potentially avoiding treatment and referral costs. The probability of cure is presented in Figure 14 and assumes a null treatment effect and is based on all available data. These data are, however, based on individual children’s time lines through the trial; so, for example, a child has a probability of only 0.21 of not being cured at any stage from 0 to 9 months, with a slightly higher probability of 0.28 of being cured at 1 month and remaining cured throughout the 9-month period. These data could be shared with patients in primary care during AM to show the likelihood of cure in more detail.

FIGURE 14.

Probabilities of cure, failure and relapse based on a null effect.

The study found only two significant potential risk factors at the p < 0.02 level (appropriate for number of variables ∼50; see Table 23). The first was the same variable noted in the pre-specified logistic regression analysis, i.e. clinical severity score. The second was practice recruit type (high versus low recruiters) for cure or not at 3 months, but because there is no clear reason why this should be the case it may be a chance result. Age as a dichotomous variable is a subgroup of further interest found in Chapter 5 (Figure 20a) and so adjustments in probabilities could be made in relation to baseline severity and age for Figure 14, to help structured advice in primary care management.

(1) Comments on taking part in the study

Overall, parents (guardians) and children expressed that being in the study had been a very positive experience: 86.6% of responses (136/157) responded with positive comments, e.g. ‘good’, ‘enjoyed’, ‘easy’, ‘happy’, ‘worthwhile’, ‘brilliant’. Of responses, 5.1% (8/157) were negative or bad, e.g. ‘difficult’, ‘worse’, ‘did not like’. The remaining 8.3% (13/157) were indifferent, giving overall responses such as ‘no comment’ and ‘nothing comes to mind’. Most of the parents (guardians) and children gave quite detailed responses to this question so these were broken down further by uncovering themes and counting how many times these themes were mentioned; examples of the responses are given below.

(2) Treatment preferences

Parents (guardians) responded to the question ‘Did you have any treatment preference throughout the trial?’ in nine different ways:

1. no preference (n = 49, 34%)

2. trial spray (n = 43, 30%)

3. only used spray so cannot state a preference (n = 26, 18%)

4. antibiotics (n = 10, 7%)

5. nasal drops (n = 3, 2%)

6. other sprays (e.g. beconase) (n = 2, 1%)

7. decongestant syrup (n = 1, 1%)

8. ear spray (n = 1, 1%)

9. no response (n = 8, 6%)

(3) Action to be taken by parent (guardian) after the end of the study

Parents (guardians) responded to the question ‘What will you do now with regard to your child’s condition?’ in ten different ways:

1. wait/monitor (n = 54, 38%)

2. get referred or already referred (n = 37, 26%)

3. see GP (n = 24, 17%)

4. nothing/not concerned/happy at the moment (n = 13, 9%)

5. see GP to get study spray if successful (n = 7, 5%)

6. want spray not antibiotics (n = 4, 3%)

7. child’s other problems more pressing (asthma) (n = 2, 1%)

8. self medicate/change behaviour, environment (n = 1, 1%)

9. try alternative (complementary) medicine (n = 1, 1%)

Base case

The CEA evaluated the cost-effectiveness of intranasal steroids in terms of natural units, calculating the cost per additional child cured of OME. In the base-case analysis, a composite end point of outcomes at 1 and 3 months was used, whereby children were considered to be cured if they were found to be free of OME (based on tympanometry) at either 1 or 3 months. This differs from the primary outcome used in the clinical analysis (cure at 1 month, adjusted for covariates), although the definition used was varied in sensitivity analyses. The incremental cost-effectiveness of intranasal steroids is shown in Table 30.

As described in Chapter 2, Methods for dealing with missing data, the base-case analysis used the techniques of multiple imputation94 to impute cost and clinical outcome data for any children who were missing such information. In total, 17 children were missing clinical outcome data, while 10 were missing cost data. Due to the relatively small degree of missing data for these outcomes, taking account of the variability between multiple imputation datasets had relatively little effect on estimates of the variance for either costs or the clinical outcome measure. Overall, 3.49% of the variance around the incremental efficacy and 2.58% of the variance around the incremental cost was due to uncertainty about the true value of missing data.

Within the base-case analysis, which included all 217 children randomised, the average cost was £454 per child in the active treatment group, compared with £442 per child in the placebo group (see Table 32). The costs presented in Tables 30–32 differ from those presented in the penultimate entry in Table 25 as the latter represents a complete case analysis including only the 207 patients with complete cost data, whereas the base-case analyses presented in Tables 30–32 include all 217 patients randomised, including imputed cost values for the 10 patients with missing cost data. However, there was no statistically significant difference in costs between the two groups, with 53.72% of bootstrap replicates finding steroids to be more costly than placebo.

Intranasal steroids were therefore dominant over placebo, being more effective and less costly. However, there was substantial uncertainty around this finding. The variability around the base-case estimates of cost-effectiveness, which is shown in the cost-effectiveness plane displayed in Figure 15, indicates that there is a 19.42% chance that steroids are dominated by placebo, in addition to a 34.89% chance that steroids dominate placebo. The dots in Figure 15 occur in discrete stripes as within each bootstrap replicate conducted in the trial an integer number of children will have been cured in each treatment arm.

FIGURE 15.

Cost-effectiveness plane for base-case analysis. The scatter graph shows the mean difference in costs and in the proportion of children cured based on the bootstrapping analysis. For clarity, only the first 200 bootstrap replicates drawn from each of the five multiple imputation data sets are shown.

The CEAC shown in Figure 16 indicates that the higher the value that decision-makers place on an additional case of OME cured, the higher the probability that intranasal steroids will be more cost-effective. At the notional willingness to pay threshold (or ceiling ratio) of £1000 per additional case of OME cured, the probability that use of intranasal steroids is cost-effective was estimated to be 56.4%. Although no previous research has empirically shown how much society or the NHS may or should be willing to pay to cure a case of OME, this figure may be in the region of £1000 based on surgery costing around £1000 per child treated. 4,100,101 At the notional willingness to pay threshold of £3000 per additional case of OME cured, the probability that intranasal steroids are cost-effective increased to 63.2%.

FIGURE 16.

CEAC for the base-case CEA. The dotted and dashed lines indicate the ceiling ratios or willingness to pay thresholds described in the text (£1000 and £3000 per case of OME cured respectively).

Sensitivity analyses

Sensitivity analyses were conducted to determine the impact of changing particular parameter values or assumptions on the ICER (Figures 17 and 18 and Table 30).

In total, 17 children (8.5% of the trial population) were missing data for the composite clinical outcome (cure of OME at 1 or 3 months), while 10 were missing cost data. Subsequently, sensitivity analyses employed a number of different imputation techniques to test the impact of different assumptions for filling in such gaps in the clinical outcome data.

Within the base-case analysis, the cost of tympanometry was excluded from total costs because it is not routinely used in UK general practice at present. A sensitivity analysis investigated the impact of including the cost of baseline tympanometric assessment (£18.81 per child; see Appendix 14) to the costs incurred in both arms of the model. Given that the total cost was increased by the same amount in both arms of the model, this change had no impact on incremental cost-effectiveness (see Table 30 and Figure 17a,b).

FIGURE 17a.

CEACs for sensitivity analyses using different assumptions or methods. Each curve shows how the probability that intranasal steroids are cost-effective varies with changes in the amount that society is willing to pay to cure one case of OME.

FIGURE 17b.

Including incremental cost of tympanometry: cost-effectiveness plane showing the mean differences in costs and the proportion of children cured based on 1000 bootstrap replicates. In this analysis, the cost of tympanometric assessment at baseline (Appendix 14) was added to the total costs for all children in both arms of the trial. For clarity, only the results of the first 200 bootstrap replicates for each of the five imputed datasets are shown; all values are per child.

Making the most optimistic possible assumptions about missing clinical outcome data (assuming that all children who were missing data on clinical outcomes had been cured by 1 or 3 months) increased the proportion of children in the treatment group who were assumed to have responded by 0.0176 and in the placebo group by 0.0355 (see Table 30 and Figure 17a,c). This analysis also affected the incremental cost, which fell to £1.52 per child, as this analysis excluded those children who had missing cost data. Although the point estimate of the ICER for active treatment relative to placebo fell to £105 per additional case cured due to the reduction in incremental costs, the probability that treatment was cost-effective at a £1000 per cure threshold increased to 36% as the probability that treatment was cost-effective fell to 59%.

FIGURE 17c.

Best case analysis: cost-effectiveness plane showing the mean differences in costs and the proportion of children cured based on 1000 bootstrap replicates. In this analysis, all children with missing data for the primary outcome measure were assumed to be cured.

Making the most pessimistic assumptions and running a form of ITT analysis, whereby all children with missing clinical outcome data were assumed not to have been cured had the reverse effect, increased the probability that active treatment is more effective than placebo to 76% and increased the probability that treatment is cost-effective if the NHS is willing to pay £1000 per case of OME cured to 64% (see Table 30 and Figure 17a,d).

FIGURE 17d.

Worst case/ITT analysis: cost-effectiveness plane showing the mean differences in costs and the proportion of children cured based on 1000 bootstrap replicates. In this analysis, all children with missing data for the primary outcome measure were assumed to have not been cured.

The impact of adjusting for missing cost data was also investigated using mean imputation: within this simple imputation technique, the total NHS costs for any child with missing cost data were simply assumed to be equal to the mean cost for that study arm, and clinical outcomes for those children with missing clinical outcome data were assumed to be equal to the probability of response in that study arm (see Table 30 and Figure 17a,Figure 17e). Similarly, the mean clinical outcome data for children with missing outcomes at both 1 and 3 months was based on the mean for that study arm. This analysis had minimal impact on the results (see Figure 17a,f) as only 10 children had missing cost data and 17 had missing outcome data.

FIGURE 17e.

Mean imputation of missing cost and clinical outcome data: cost-effectiveness plane showing the mean differences in costs and the proportion of children cured based on 1000 bootstrap replicates. In this analysis, all children with missing cost data were assumed to have total treatment costs equal to the mean total cost across children in the relevant study arm and similarly with clinical outcomes.

In the complete case analysis, children with missing data on either costs or the composite clinical outcome were excluded from the analysis (see Table 30 and Figure 18a,b). Based on this analysis, the active treatment group had higher costs than the placebo group; however, the difference was not statistically significant. In this analysis, the point estimate of the ICER showed active treatment as costing £178 per additional case of OME cured, with a 55% probability of being cost-effective at a ceiling ratio of £1000 per case cured.

FIGURE 18a.

CEACs for sensitivity analyses using different assumptions or methods. Each curve shows how the probability that intranasal steroids are cost-effective varies with changes in the amount that society is willing to pay to cure one case of OME.

FIGURE 18b.

Complete case analysis: cost-effectiveness plane showing the mean differences in costs and the proportion of children cured based on 1000 bootstrap replicates. In this analysis, all children missing data on either costs or clinical outcomes were omitted from the analysis.

Three sensitivity analyses evaluated the impact of evaluating clinical outcomes at 1, 3 and 9 months after start of treatment, instead of using a composite end point of cure at either 1 or 3 months (see Figure 18). Because outcomes at 1 and 3 months are closely correlated with the composite end point, these three analyses were based on a separate run of multiple imputation in which the composite clinical end point was replaced with outcomes at the three individual time points. This means that the imputed costs used in these analyses differ slightly from those used in the base-case analysis.

The analysis using outcomes at 1 month matches the time point of the primary outcome measure in the trial. The proportion of children whose OME was cured within 1 month of starting treatment was lower in the active treatment group than in the placebo group (see Table 30 and Figure 18a,c). Subsequently, basing clinical outcomes on the proportion of cases cured by the 1-month follow-up rather than a composite end point including cures by either 1 or 3 months suggested that treatment with intranasal steroids would be dominated by placebo, being more costly and less effective than no treatment. If the NHS were willing to pay £1000 to gain one less cure of OME in order to accrue savings of at least £1000, active treatment would have a 37% probability of being cost-effective.

FIGURE 18c.

Outcomes measured at 1 month: cost-effectiveness plane showing the mean differences in costs and the proportion of children cured. In this analysis, data on the primary outcome measure were based on outcomes assessed 1 month after start of treatment. For clarity, only the results of the first 200 bootstrap replicates for each of the five imputed data sets are shown; all values are per child.

However, as in the base-case analysis, assessing outcome at 3 months suggested that treatment is slightly (but not significantly) more effective than placebo in terms of the cost per case of OME cured. When clinical outcomes were based on assessments at 3 months, active treatment cost £159 per case of OME cured and had a 71% chance being cost-effective at a £1000 ceiling ratio (see Table 30 and Figure 18a,d).

FIGURE 18d.

Outcomes measured at 3 months: cost-effectiveness plane showing the mean differences in costs and the proportion of children cured. In this analysis, data on the primary outcome measure were based on outcomes assessed 3 months after start of treatment. For clarity, only the results of the first 200 bootstrap replicates for each of the five imputed data sets are shown; all values are per child.

The proportion of children whose OME was cured within 9 months of starting treatment was also lower in the active treatment group than in the placebo group (see Table 30 and Figure 18a,e). The probability of treatment being cost-effective at a ceiling ratio of £1000 when outcomes were assessed at this time point was 29.56%, and treatment was dominated by placebo.

FIGURE 18e.

Outcomes measured at 9 months: cost-effectiveness plane showing the mean differences in costs and the proportion of children cured. In this analysis, data on the primary outcome measure were based on outcomes assessed 9 months after start of treatment. For clarity, only the results of the first 200 bootstrap replicates for each of the five imputed data sets are shown; all values are per child.

Basing costs on data collected from the resource use questionnaire completed by parents (or guardians) 3 and 9 months after start of treatment (rather than using cost data collected in a retrospective review of patients’ medical records) also had a substantial effect on cost-effectiveness (Figure 18a,f). Based on the questionnaires completed by parents (or guardians), the active treatment group within the trial was associated with substantially higher costs than the placebo group (£458 and £274 per child, respectively), although the difference in costs did not reach statistical significance on a two-tailed test, as there was a 4.06% chance that treatment would be less costly than placebo. Within this analysis, active treatment cost (on average) £5704 per case of OME cured and had a 64% chance of being cost-effective if the NHS were willing to pay £1000 per case cured.

FIGURE 18f.

Parental costs/resource use: cost-effectiveness plane showing the mean differences in costs and the proportion of children cured. In this analysis, data on costs were based on the resource use questionnaire completed by parents at 3 and 9 months after start of treatment. For clarity, only the results of the first 200 bootstrap replicates for each of the five imputed data sets are shown; all values are per child.

Subgroup analyses

In order to explore how incremental cost-effectiveness varies by characteristics of the children eligible for treatment, subgroup analysis was performed (Table 31). Subgroups investigated in such analyses comprised age, clinical severity, atopy, season, gender and with or without AM. Figures 19 and 20 show the results of the subgroup analysis displayed graphically on cost-effectiveness planes and CEACs, while Table 31 shows the cost and effect differences as well as the ICERs and the probabilities of the active treatment group being cost-effective in each subgroup.

FIGURE 19a.

Younger children (< 6.5 years at baseline): scattergraph showing the distribution of incremental costs and effects for intranasal steroids relative to placebo.

FIGURE 19b.

Older children (≥ 6.5 years at baseline): scattergraph showing the distribution of incremental costs and effects for intranasal steroids relative to placebo.

FIGURE 19c.

Children without atopy: scattergraph showing the distribution of incremental costs and effects for intranasal steroids relative to placebo.

FIGURE 19d.

Children with atopy: scattergraph showing the distribution of incremental costs and effects for intranasal steroids relative to placebo.

FIGURE 19e.

Boys: scattergraph showing the distribution of incremental costs and effects for intranasal steroids relative to placebo.

FIGURE 19f.

Girls: scattergraph showing the distribution of incremental costs and effects for intranasal steroids relative to placebo.

FIGURE 19g.

Children recruited with AM: scattergraph showing the distribution of incremental costs and effects for intranasal steroids relative to placebo.

FIGURE 19h.

Children recruited without AM: scattergraph showing the distribution of incremental costs and effects for intranasal steroids relative to placebo.

FIGURE 19i.

Children with non-severe disease: scattergraph showing the distribution of incremental costs and effects for intranasal steroids relative to placebo. Severe disease was defined as a clinical severity score above 0.62 (the upper quartile limit of the sample). Clinical severity was defined as the first principal component of the baseline variables – frequency of surgery attendance in the last 12 months for ear problems; tympanogram readings; age at first episode of hearing infection/problem; total reported episodes of ear problems over last 12 months; RESP score – identified in an analysis of these variables ignoring randomisation group.

FIGURE 19j.

Children with severe disease: scattergraph showing the distribution of incremental costs and effects for intranasal steroids relative to placebo.

FIGURE 19k.

Children recruited during January, February and March: scattergraph showing the distribution of incremental costs and effects for intranasal steroids relative to placebo.

FIGURE 19l.

Children recruited between April and December: scattergraph showing the distribution of incremental costs and effects for intranasal steroids relative to placebo.

FIGURE 20a.

CEACs for different age subgroups. Each curve shows how the probability that intranasal steroids are cost-effective varies with changes in the amount that society is willing to pay to cure one case of OME. All analyses are based on 1000 bootstrap replicates for each of the five imputed data sets generated in multiple imputation.

FIGURE 20b.

CEACs for different atopy subgroups. Each curve shows how the probability that intranasal steroids are cost-effective varies with changes in the amount that society is willing to pay to cure one case of OME. All analyses are based on 1000 bootstrap replicates.

FIGURE 20c.

CEACs for different gender subgroups. Each curve shows how the probability that intranasal steroids are cost-effective varies with changes in the amount that society is willing to pay to cure one case of OME. All analyses are based on 1000 bootstrap replicates.

FIGURE 20d.

CEACs for different trial subgroups (with and without AM). Each curve shows how the probability that intranasal steroids are cost-effective varies with changes in the amount that society is willing to pay to cure one case of OME. All analyses are based on 1000 bootstrap replicates.

FIGURE 20e.

CEACs for different severity subgroups. Each curve shows how the probability that intranasal steroids are cost-effective varies with changes in the amount that society is willing to pay to cure one case of OME. All analyses are based on 1000 bootstrap replicates.

FIGURE 20f.

CEACs for different season subgroups. Each curve shows how the probability that intranasal steroids are cost-effective varies with changes in the amount that society is willing to pay to cure one case of OME. All analyses are based on 1000 bootstrap replicates.

The subgroup analysis found the results to be particularly sensitive to age, both in terms of costs and outcomes. In children aged 6.5 years and over, older children, the active treatment group accrued lower NHS costs than the placebo group, suggesting that treatment would save an average of £152 per child. By contrast, in younger children aged between 4 and 6.5 years, the treatment group had higher costs than placebo by an average of £101, although the difference between treatment groups did not reach statistical significance in either subgroup. However, younger children accrued higher total health-care costs than older children regardless of which treatment they received.

Furthermore, the proportion of patients cured of OME by 1 or 3 months was significantly higher in the treatment group than for placebo in the subgroup of older children, in whom there was a 99% probability that treatment was more effective. Conversely, fewer patients were cured in the active treatment arm than in the placebo group in the subgroup in younger children, although the difference in this group did not reach statistical significance.

The marked differences in costs and benefits translated into substantial differences in cost-effectiveness. Treatment dominated placebo and had a 99.2% probability of being cost-effective at a £1000 per cure threshold in children aged 6.5 years and over, but had a 13.6% probability of being cost-effective in children aged between 4 and 6.49 years in whom the active treatment group was dominated by placebo (i.e.more costly and less effective). The difference in outcomes may reflect differences in compliance but more likely the probability of spontaneous recovery, while the difference in cost may reflect changes in the probability of undergoing surgery or be the result of the difference in clinical outcomes. However, further research is required to confirm or refute these hypotheses.

Marked differences were also observed between the early and later phases of the trial, which differed in a number of aspects described in Chapter 2, Changes to the original protocol – particularly in the use of AM. The active treatment group was dominant over placebo in the subgroup of children who were recruited with AM, being more effective and less costly, and there was a 74% probability that steroids were cost-effective at a £1000 per case cured ceiling ratio. By contrast, active treatment was more costly in the subgroup of children recruited under the amended protocol who did not have a period of AM and cost £1964 per QALY compared with placebo, with a 40% probability of being cost-effective among those children recruited without AM.

Base case

The incremental costs calculated for the CUA are the same as those for the CEA, as the same cost data were used. However, whereas the CEA suggests that there was a non-significant trend towards better outcomes in the intranasal steroid arm of the trial than in the placebo group, the CUA observes a non-significant trend towards worse outcomes with active treatment. As described above (see Table 29), the active treatment group accrued an average of 0.0166 fewer QALYs per child than the placebo group. There was found to be a 24% probability that intranasal steroids are more effective than placebo in terms of the QALYs accrued over the 9-month trial period, which means that there was no significant difference between study arms in terms of the number of QALYs accrued (p = 0.24 on a one-tailed test). The base-case analysis therefore suggests that steroid treatment is dominated by placebo, being more costly and less effective. This finding differs from the CEA, which found active treatment to cost £347 per QALY gained, as a higher proportion of children in the active treatment group achieved tympanometric cure at 1 or 3 months (than those in the placebo group) despite having lower QoL.

However, there is a very large degree of uncertainty around the findings of both analyses (see Figures 15, 16 and 21 and Tables 30 and 32). The scattergraph showing the distribution of incremental costs and benefits for the CUA shows that the true incremental cost per QALY gained could fall into any of the four quadrants on the cost-effectiveness plane (see Figure 21b). There is a 12% chance that steroid treatment is more costly and more effective (falling into the top-right quadrant of the cost-effectiveness plane), a 12% chance that it is dominant over placebo (i.e.less costly and more effective), a 42% chance that steroids are dominated by placebo (i.e.more costly and less effective) and a 34% chance steroids are less costly and less effective (see Figure 21b).

FIGURE 21a.

Base-case analysis: scattergraph showing the distribution of incremental costs and effects for intranasal steroids relative to placebo on the cost-effectiveness plane. For clarity, only the results of the first 200 bootstrap replicates for each of the five imputed data sets are shown; all values are per child.

FIGURE 21b.

Base-case analysis: CEAC showing the probability that intranasal steroids are cost-effective at a range of ceiling ratios representing possible values for the maximum amount that society may be willing to pay to gain an additional QALY.

If society is willing to pay £20,000 to gain an additional QALY, there is a 24% chance that steroids are a cost-effective use of NHS resources (see Figure 21a); given that most of the bootstrap replicates found active treatment to be less effective than placebo, the probability that treatment was cost-effective fell as the ceiling ratio increased.

As for the CEA, the base-case CUA included imputation of missing data for both costs and utilities using multiple imputation techniques (see Chapter 2, Methods for dealing with missing data). Imputation was particularly influential for utility data, as 62% (134/217) of children were missing HUI3 utility data for at least one time point. Due to the substantial amount of missing utility data, uncertainty about the correct value for data points that had been imputed accounted for 28.0% of the total variance around the mean incremental QALY gain from treatment, but accounted for just 2.3% of the variance around the mean incremental cost.

Sensitivity analyses

A number of sensitivity analyses were conducted in order to assess the impact of the assumptions and decisions about analytical methods used in the base-case analysis. The first set of analyses assessed the results that would be obtained if the two alternative utility instruments used in the study were used in place of HUI3. This analysis showed that the choice of utility instrument has a large effect on the probability that intranasal steroids are more effective than placebo, the probability that treatment is cost-effective (Figure 22) and the point estimates for the ICER (see Table 32).

When the HUI2 questionnaire was used, the active treatment group accrued more QALYs than placebo (not significant) and there was a 66% chance that active treatment would produce more QALYs than placebo (compared with 24% in the base-case analysis). Furthermore, active treatment cost £2161 per QALY gained and had a 63% chance of being cost-effective relative to placebo at a £20,000 per QALY threshold (see Figures 22a,b).

FIGURE 22a.

CEACs for analyses using HUI2 and EQ-5D: utilities were based on either the HUI2 or EQ-5D₅ instruments and include values imputed in the main multiple imputation analysis (in which predicted HUI3 utilities from the mapping models were included as explanatory variables, but predictions of HUI2 and EQ-5D utilities were not). Other than the instruments used to generate utilities, the data, assumptions and methods used in this analysis were the same as for the base-case analysis.

FIGURE 22b.

HUI2: scattergraph showing the distribution of incremental costs and effects for intranasal steroids relative to placebo. Utilities were based on the HUI2 instrument and include values imputed in the main multiple imputation analysis (in which predicted HUI3 utilities from the facet mapping model were included as explanatory variables, but predictions of HUI2 and EQ-5D utilities were not). All other parameters were conducted using the same data and methods as the base-case analysis. For clarity, only the results of the first 200 bootstrap replicates for each of the five imputed data sets are shown; all values are per child.

Using the EQ-5D₅ questionnaire increased estimates of the incremental benefits of treatment still further, such that intranasal steroids had an 89% chance of being more effective than placebo and cost £418 per QALY gained with an 89% chance of being cost-effective at a £20,000 per QALY ceiling ratio (see Figures 22a,b).

FIGURE 22c.

EQ-5D₅: scattergraph showing the distribution of incremental costs and effects for intranasal steroids relative to placebo. Utilities were based on the EQ-5D instrument and include values imputed in the main multiple imputation analysis (in which predicted HUI3 utilities from the mapping models were included as explanatory variables, but predictions of HUI2 and EQ-5D utilities were not). All other parameters were conducted using the same data and methods as the base-case analysis. For clarity, only the results of the first 200 bootstrap replicates for each of the five imputed data sets are shown; all values are per child.

The second set of sensitivity analyses conducted explored the impact of adjusting for baseline utility. For simplicity, such analyses were conducted using only the HUI3 instrument. When no adjustment was made for baseline utility, the absolute difference in QALYs between the two groups increased from 0.0166 to 0.0225 and the probability of active treatment being more effective than placebo fell to 11%. The probability of treatment being cost-effective at a £20,000 per QALY ceiling ratio fell to 12% (Figure 23a,b).

FIGURE 23a.

CEAC for analysis with no adjustment for baseline utility. All other parameters were conducted using the same data and methods as the base-case analysis.

FIGURE 23b.

No adjustment for baseline utility: scattergraph showing the distribution of incremental costs and effects for intranasal steroids relative to placebo. The total number of QALYs calculated for each child (including imputed values) was used directly in bootstrapping analyses, with no adjustment for baseline utility; all other parameters were conducted using the same data and methods as the base-case analysis. For clarity, only the results of the first 200 bootstrap replicates for each of the five imputed data sets are shown; all values are per child.

Thirdly, alternative methods for dealing with missing data were investigated. The complete case analysis assessed outcomes for the 104 out of 217 children who had full data on costs and for whom the HUI3 questionnaire was fully completed at all three time points (Figures 24a,b). Among this subgroup of children, costs were substantially higher in the active treatment group (£550 per child) and lower in the placebo group (£352 per child), although the absolute difference in QALYs between the two groups was reduced to 0.0059. However, the probability that treatment was cost-effective at a £20,000 per QALY ceiling ratio was similar to the base case at 25%.

FIGURE 24a.

Complete case analysis and analyses including mapped utilities: CEAC. Within the complete case analysis, only children who fully completed the HUI3 questionnaire at baseline and at both the 3- and 9-month follow-ups and had full resource use data were included and no values were imputed. Within the analyses using mapped utilities, missing utility data were imputed (where possible) using two of the mapping algorithms described in Appendix 15 that estimate HUI3 utilities based on OM8-30 scores: the OLS model using OM8-30 facet scores and the OLS model using OM8-30 domain scores (plus age and sex). Multiple imputation analysis was not used for the complete case analysis or for either of the analyses using mapped utilities; these analyses were therefore based on 1000 bootstrap replicates with a single set of raw data.

FIGURE 24b.

Complete case analysis: scattergraph showing the distribution of incremental costs and effects for intranasal steroids relative to placebo, including only children who fully completed the HUI3 questionnaire at baseline and at both the 3- and 9-month follow-ups; all other parameters were conducted using the same data and methods as the base-case analysis.

We also investigated the impact of imputing missing utility data using the mapping equations produced in the analysis mapping between OM8-30 scores and HUI3 utilities (Appendix 15) instead of imputing both costs and utilities using multiple imputation. In these analyses, children who were missing HUI3 utility data at any given time point but for whom full information on predicted HL and all OM–30 facets or domains were available were assumed to have the utility that would be predicted based on the ordinary least-squares domain or facet-level model shown in Appendix 15. Linear interpolation/extrapolation was used to estimate QALYs for children who were missing mapped or observed utility data at either 3 or 9 months (see Figure 2), although no other imputation was used to estimate utility data that were missing after mapped values had been included. Similarly, the costs included in this analysis included only those costs observed directly in the trial, rather than values estimated using multiple imputation. As this comprised a sensitivity analysis, uncertainty around children’s predicted utility was not taken into account within bootstrapping.

When the analysis was restricted to those children (154 out of 217 children) who had complete cost data and utility data at baseline and at one or more follow-up time points (obtained from either direct completion of HUI3 or OM8-30 mapping), the average cost was £475 per child in the active treatment group, compared with £376 per child in the placebo group (Table 32). However, as in the base-case analysis, there was no statistically significant difference in costs between the two groups. The difference in costs between this analysis and the base-case analysis is due to the different populations of children included in each analysis: children for whom the HUI3 and/or OM8-30 questionnaires were not completed tended to have higher costs if they received placebo and lower costs if they received active treatment than children for whom utility data were available.

In addition to having higher average costs, the active treatment group accrued fewer QALYs per child than the placebo group, regardless of whether the domain or facet model was used to estimate utilities, although this difference did not reach statistical significance. Regardless of which mapping model was used to estimate missing utility data, active treatment was dominated by placebo, and the probability of treatment being cost-effective at a £20,000 per QALY threshold was just 39% (Figure 24c–d).

FIGURE 24c.

Mapped utilities using the OM8-30 facet model: scattergraph showing the distribution of incremental costs and effects for intranasal steroids relative to placebo. Missing utility data were imputed (where possible) using the OLS mapping model using OM8-30 facet scores (Appendix 15), with no multiple imputation of either costs or benefits.

FIGURE 24d.

Mapped utilities using the OM8-30 domain model: scattergraph showing the distribution of incremental costs and effects for intranasal steroids relative to placebo. Missing utility data were imputed (where possible) using the OLS mapping model using OM8-30 domain scores plus age and sex (Appendix 15), with no multiple imputation of either costs or benefits.

Basing costs on the report forms that parents (or guardians) completed at 3 and 9 months, rather than using the resource use data extracted from medical records, substantially increased the probability that use of intranasal steroids would increase NHS costs from 54% to 95% (Figure 25) and increased the incremental cost of treatment from £11 to £185, although the difference did not reach statistical significance on a two-tailed test. When resource use estimated by parents was used in the analysis, the probability of steroids being cost-effective relative to placebo at a £20,000 per QALY threshold fell to 14%.

FIGURE 25a.

CEAC for analysis using costs derived from parents’ (or guardians’) resource use questionnaires; all other parameters were conducted using the same data and methods as the base-case analysis.

FIGURE 25b.

Costs based on parents’ (or guardians’) resource use questionnaires: scattergraph showing the distribution of incremental costs and effects for intranasal steroids relative to placebo. All other parameters were conducted using the same data and methods as the base-case analysis. For clarity, only the results of the first 200 bootstrap replicates for each of the five imputed data sets are shown; all values are per child.

By contrast, adding in the cost of one baseline tympanometric assessment for all children in both arms had no impact on incremental costs or benefits, as costs increased by the same amount for all children (data not shown).

Subgroup analyses

The CUA was repeated for the six sets of subgroups that were investigated in the CEA, using the same assumptions and methods used for the base-case analysis.

As for the CEA, subgrouping children by age had a dramatic effect on the results, although the trends observed differ based on how outcomes are measured. Within the subgroup of children aged over 6.5 years, the group receiving intranasal steroids accrued lower costs than the group assigned to placebo (Figure 26b), while active treatment was found to be more costly within the subgroup of younger children (Figure 26c). However, for both subgroups, the active treatment arm accrued fewer QALYs than the placebo arm, although the difference between the groups was larger in older children. This contrasts with the finding in the CEA, in which treatment appeared to slightly increase the probability of cure in older children but reduce it in younger children. The difference between these analyses appears to result (at least in part) from differences in the time horizon used for outcomes: older children randomised to active treatment tended to have a higher chance of tympanometric cure at 1 and 3 months than the placebo group, but had a slightly lower chance of being cured at 9 months; by contrast, younger children randomised to active treatment had a lower probability of being cured than those in the placebo group at all three time points. However, these variable findings are probably due to chance as there is no simple explanation.

FIGURE 26a.

CEACs for subgroups stratified by age. Both analyses were conducted using the same methods as the base-case analysis.

FIGURE 26b.

Older children (≥ 6.5 years at baseline): scattergraph showing the distribution of incremental costs and effects for intranasal steroids relative to placebo. In this scattergraph and all of those that follow, the results of only the first 200 bootstrap replicates for each of the five imputed data sets are shown and all values are per child.

FIGURE 26c.

Younger children (< 6.5 years at baseline): scattergraph showing the distribution of incremental costs and effects for intranasal steroids relative to placebo.

Given that both the incremental costs and incremental QALYs are negative in older children, the point estimate for this ICER falls in the south-west quadrant and suggests that using active treatment would save £6611 per QALY lost compared with placebo. In the south-west quadrant, ICERs have the opposite interpretation to those in the north-east quadrant, whereby treatment would be considered cost-effective if its ICER were above the ceiling ratio (rather than below); this means that treatment would not be considered cost-effective in older children at an ICER of £6611 per QALY lost. By contrast, treatment is dominated by placebo for younger children and for the study population as a whole. Despite the substantial difference in incremental costs and ICERs, the probability of active treatment being cost-effective at a £20,000 per QALY ceiling ratio was relatively similar across the two groups, being 35% in older children and 26% in younger children (Figure 26a).

The difference in incremental costs between the sexes was also pronounced, although less extreme than the finding for age. In girls, use of steroids was associated with slightly lower costs than the placebo group by an average of £32 per child, whereas in boys, active treatment increased costs by an average of £62 per child, although neither difference reached statistical significance (see Table 32). Although boys had greater improvements in QoL relative to baseline than girls, the active treatment group accrued 0.014–0.017 fewer QALYs than the placebo group regardless of sex. Based on the point estimates for ICERs, active treatment was dominated by placebo for boys (with a 28% probability of being cost-effective at a £20,000 per QALY threshold) and saved £2322 per QALY lost for girls (with a 35% probability of being cost-effective at this threshold; Figure 27).

FIGURE 27a.

CEACs for subgroups stratified by sex. Both analyses were conducted using the same methods as the base-case analysis.

FIGURE 27b.

Boys: scattergraph showing the distribution of incremental costs and effects for intranasal steroids relative to placebo.

FIGURE 27c.

Girls: scattergraph showing the distribution of incremental costs and effects for intranasal steroids relative to placebo.

Subgrouping by the presence/absence of atopy had little impact on QoL or the probability of treatment being cost-effective within the CUA, although active treatment was associated with higher costs (£51 per child) than placebo in children with atopy, compared with cost savings of £5 per child relative to placebo in the group with no atopy at baseline (Figure 28 and Table 32).

FIGURE 28a.

CEACs for subgroups stratified by the presence/absence of atopy. Both analyses were conducted using the same methods as the base-case analysis.

FIGURE 28b.

Children with atopy: scattergraph showing the distribution of incremental costs and effects for intranasal steroids relative to placebo.

FIGURE 28c.

Children without atopy: scattergraph showing the distribution of incremental costs and effects for intranasal steroids relative to placebo.

Subgrouping by disease severity (severe disease was defined as a clinical severity score above 0.62, the upper quartile limit of the sample) also had a substantial effect on cost-effectiveness (Figure 29). In children with severe disease, use of intranasal steroids increased NHS costs by £145 per child compared with placebo (with a 75% probability that treatment is more costly). By contrast, in the less severe subgroup, active treatment was associated with lower costs than placebo (difference: £93 per child). Although the absolute difference in QALYs and the probability of treatment being less effective than placebo were lower in children with non-severe disease, there was relatively little difference in the probability of active treatment being cost-effective at a £20,000–30,000 per QALY threshold (Figure 29).

FIGURE 29a.

CEACs for subgroups stratified by disease severity [categorised based on clinical severity score (see Chapter 4, Clinical outcomes) using a cut-off of 0.62 (the 75th percentile)]. Both analyses were conducted using the same methods as the base-case analysis.

FIGURE 29b.

Children with severe disease (highest 25% of children by severity score): scattergraph showing the distribution of incremental costs and effects for intranasal steroids relative to placebo.

FIGURE 29c.

Children with non-severe disease (lowest 75% of children by severity score): scattergraph showing the distribution of incremental costs and effects for intranasal steroids relative to placebo.

The incremental cost of treatment also varied with season: among children recruited to the study between January and March (inclusive), the active treatment arm tended to accrue higher NHS costs than the placebo group and active treatment was dominated by placebo and had a 31% chance of being cost-effective at a £20,000 per QALY ceiling ratio (Figure 30). By contrast, among children recruited between April and December, there was a non-significant trend suggesting that active treatment is less costly than placebo, would save £2963 per QALY lost and would have a 30% probability of being cost-effective.

FIGURE 30a.

CEACs for subgroups stratified by the season at the time of enrolment. Both analyses were conducted using the same methods as the base-case analysis.

FIGURE 30b.

Children randomised in January, February or March: scattergraph showing the distribution of incremental costs and effects for intranasal steroids relative to placebo.

FIGURE 30c.

Children randomised between April and December: scattergraph showing the distribution of incremental costs and effects for intranasal steroids relative to placebo.

Given that substantial changes were made to the protocol part-way through the study that resulted in the removal of the AM period, changes to child recruitment, the addition of utility instruments and changes to the collection of resource use data, the costs and benefits of treatment in children recruited before and after the protocol change were evaluated in a subgroup analysis. During the initial trial period (with AM), for which most utility data came from the OM8-30 mapping analysis, costs were lower in the active treatment group than in the placebo group (probability = 79%) and there was a 44% chance that treatment was cost-effective at a £20,000 per QALY threshold (Figure 31a–c). Among children recruited after the protocol change, total costs in the active treatment arm were an average of £91 per child higher than those for placebo, and active treatment had a 17% chance of being cost-effective. The difference in costs is unlikely to be due to the changes in the way that resource use data were collected as such amendments simply resulted in the collection of additional data, but this explanation is hard to completely exclude. It is also unlikely to reflect changes in the management of OME over a short study time frame, with no major changes in practice, but earlier study entry (i.e. in the natural history) may have pushed up initial antibiotic and medication costs (non-significantly) in the group that took slightly longer to resolve. However, as the clinical trial showed no efficacy differences (using a sensitivity analysis), these cost differences might reasonably be considered as chance subgroup findings. The difference in the incremental QALY loss from treatment may be (at least in part) due to the much greater degree of missing data in the early trial period, which were imputed using multiple imputation.

FIGURE 31a.

CEACs for subgroups stratified into before and after the protocol amendment that led to the introduction of utility measures and the removal of AM. Both analyses were conducted using the same methods as the base-case analysis.

FIGURE 31b.

Initial trial period with AM: scattergraph showing the distribution of incremental costs and effects for intranasal steroids relative to placebo.

FIGURE 31c.

Later trial period without AM: scattergraph showing the distribution of incremental costs and effects for intranasal steroids relative to placebo.

Clinical outcomes

The study is the largest double-blind randomised placebo-controlled trial to date of topical INCS in children with OME, and one of the very few of any type of intervention for OME in primary care where the majority of children continue to be seen. The main findings show that 3 months’ use of topical INCS in 4- to 11-year-old children in this setting is no better than placebo in improving clearance of effusions and in improving other important outcomes.

The main study outcome was for efficacy because, while several small studies have suggested efficacy for topical nasal steroids and they are often used off licence for this condition,61,62,64 efficacy has not yet been demonstrated in the literature. Using objective tympanometric criteria for children cured, the AOR of 0.93 (95% CI 0.50 to 1.75) was not significant. The risk reduction of the treated group calculated for 1-month resolution of bilateral effusions in children rather than ears was –4.3% (95% CI –18.05% to 9.26%). Thus using the upper 95% confidence limit for an effect, we can be confident that a useful effect even as low as 9% increased tympanometric resolution of bilateral glue ear in children is very unlikely in our sample, setting a lower limit for an NNT of 11.

Non-significant secondary outcome efficacy was also found in clearing effusions at 3 and at 9 months. Thus the null hypotheses relating to efficacy that are implicit in the first study aim cannot be rejected for a 3-month course of topical steroid versus placebo for short, medium or longer term efficacy outcomes using clearance of effusions in children.

The most robust and responsive clinical effectiveness outcome currently available, the OM8-30 questionnaire (a condition-specific functional health status measure), also showed negative results at 3 and 9 months. All the substituent eight scales measuring various impacts of the condition on the child’s physical health and development showed non-significant differences between treatment arms despite the probability of a false positive outcome. The devised continuous scales would be anticipated to show useful treatment effects of about 0.5 SD, making clinically important false negative effects unlikely.

OM8-30 scales showed recovery profiles, most marked at 3 months (which probably mirrors the tympanometric clearance rates). The fact that the respiratory symptom subscale score, a measure of adenoidal symptoms, was also no different between the allocated arms at 3 months indicates that no other potential treatment benefits than on the ears103–106 were found. Three-month prospective diary information was also collected for other important continuous variables (collected retrospectively on the OM8-30), such as days with reported otalgia, which was not significantly different between groups at 1 month (p = 0.43) nor at 3 months (p = 0.46).

Hearing level as an objective outcome is problematic for a primary care study because the gold standard of pure tone audiometry is difficult to perform reliably – particularly in the younger children study and in a community setting where high levels of background noise tend to invalidate the results further. HL as assessed by audiometry is not a known effect modifier and so fail was not set as an inclusion criterion for this study. Four measures of hearing were used but none of these outcomes, although improving over time, showed any significant differences between groups.

The only variable found to affect outcome was clinical severity, based on clinical attendance records and reported frequency of relevant ear problem episodes over the preceding year, age of first episode, bilateral B tympanograms rather than B/C2, and RESP (adenoidal symptom) score. It was noted that by using an age cut-off of 6.5 years, the older children had significantly less severe disease at baseline and so constituted an important subgroup for analysis. Predicted factors such as atopy and season were not significant.

This study has demonstrated the feasibility and acceptability of deploying a 3-month AM scheme (sometimes called watchful waiting) in primary care over which time, and based on a null result, 55% of children will spontaneously clear the fluid from at least one of their ears and thus considerably reduce their disability risk. This rate of clearance is similar to that found in a Dutch-based primary care and epidemiological study. 30,79 In this study, 72 children were randomised to AM during the first winter periods of the study. Slow recruitment primarily, but also taking into account feedback from children, parents (guardians) and RNs resulted in a DMEC-approved protocol change allowing study entry to children with relevant recent ear problem histories and a single time point fail on tympanometry in both ears (i.e. theydid not have to fail tympanometry on two occasions 3 months apart before being randomised to treatment). A sensitivity analysis was performed on the study sample including and excluding the AM group before randomisation, and no significant differences were found for the main tympanometric outcomes at 1 and 3 months, so these populations were subsequently combined in the main analyses. A feature of this study was the rather low referral rate to ENT of just under 15% with 15 cases from the active group and 17 from the placebo group by 9 months. Using MRC-developed referral accuracy criteria, about 60% of the referrals were appropriate, which is relatively high for a condition shown to have a five-fold variability in referral rates. It was uncertain as to what this may be attributed to: the structured AM process, the high reported patient satisfaction with the study, the placebo bias, a Hawthorn effect or various combinations of these. However, it does tend to refute the notion that introducing microtympanometry into primary care would lead to over-referral because it is oversensitive as a diagnostic tool and requires further research of these factors, especially treatment/placebo effects in this setting.

Health economic outcomes

While treatment effects are likely to dissipate rapidly after 3 months, the incomplete natural resolution and relapse rate after 3 months suggest continued NHS resource use with ongoing monitoring in primary care, and referrals should be measured for up to 9 months post randomisation. No studies to date were found that examine these types of longer term outcomes.

The economic evaluation found no evidence that intranasal steroids are cost-effective. However, a non-significant trend towards lower costs in the treatment group than for placebo was observed in older children (over 6.5 years) and children with less severe disease (clinical severity score greater than 0.62). However, these analyses should be interpreted with caution as they were not pre-specified and include small numbers of children. Conversely, analysis of utility instruments found that use of intranasal steroids may reduce children’s health-related QoL. The risk of treatment causing harm was particularly pronounced in a number of subgroups: there was found to be a 72% chance that steroids reduce the number of QALYs accrued in older children and a 83% chance in children with severe disease.

Randomisation and concealment

The study was randomised using computer-generated randomisation sequences in which the generator and executor were kept entirely separate. The company supplying the medication mailed the randomisation packs directly to the practices at the start as requested by participating practices. The company had the only copy of the complete randomisation code until all data had been entered on the database and the DMEC authorised unblinding. There were no serious adverse events, and no individual trial code envelopes were opened throughout the trial.

Concealment was evaluated in children and parents (guardians) for which prediction of the correct group was no better than chance. The placebo appeared particularly close to the active treatment as over 80% of parents (guardians) thought their children were receiving the active treatment.

Intention to treat analysis and losses to follow-up

The study data were analysed by group allocated irrespective of treatment received. There was a high level of retention: 93% at 1 month, 83% at 3 months and 75% at 9 months. However, the CONSORT diagram (see Figure 3) shows slightly lower levels of retention – 89%, 79% and 66% respectively – as these figures were based upon numbers of children having tympanometry performed at each visit.

The missing data were censored assuming them to be missing at random rather than attempting to impute all missing data for a full ITT. From the data it was not possible to make any informed assumptions about loss to follow-up, e.g. dropouts were better or worse, etc. Thus an ITT analysis is reported with ∼90% follow-up for the main outcome at 1 month.

Clinical severity and persistence analyses

Clinical severity analyses found baseline severity to be an outcome but not an effect modifier. The main trial found that any potential treatment effect was in the milder cases which are more likely to resolve naturally, which does not contradict the null result for use of the intervention. A sensitivity analysis of the cohort plus or minus AM over 3 months, a measure of persistence, did not significantly affect outcomes.

Adherence

Considering the children in the trial were as young as 4 years, the reported adherence at 1 and 3 months was excellent. The original study was intended for children aged 3–11 years but, because of pilot work on nasal sprays in an unreported study, the lower age cut-off was increased to 4 years before the main study was started. Staff training and parent/child interest in the study probably contributed to the high levels of adherence reported in the study, with the possibility of avoiding referral and surgery. However, it is possible that adherence was not as high as reported.

Multiple outcome measures and frequency of follow-up

The study used a wide range of outcome measures and scales other than the main outcome without any statistically significant findings at 1, 3 and 9 months. The study was unusual in following a non-surgical intervention over a longer time frame necessary to establish cost-effectiveness.

Power calculation

Two hundred and forty children were required, assuming a 15% dropout rate and 3% non-interpretable rate for an alpha of 0.05 and a beta of 0.2 and assuming 28% tympanometric resolution in the topical steroid group and 12% in the placebo group. 19 Differences of 15% or less for tympanometric outcomes are not likely to be clinically significant as tympanometry is a disease measure with only a moderate PPV of 0.4980,81 for a relevant clinical outcome – the pure tone hearing level. Although only 217 children were in fact recruited, an end-of-study analysis by the DMEC found that the study had achieved a meaningful negative trial finding in relation to the primary outcome. With an NNT of 11 or more, clinically important effects are not likely to be present in the population from which the study sample came.

A larger sample size is always desirable. The number of children randomised could have been much larger if less stringent inclusion criteria had been applieds and more outcome data would have been available if a by-ear rather than by-child analysis had been applied. This study sample used a more generous but equally rigorous and widely accepted definition of cure than the study used for the original power calculation (type A only) which excluded proportions of about 30% higher resolution to type CI tympanograms, generally considered normal and used in other primary care studies. 79

Much higher proportions of treated and control children resolving were observed in this study sample than in the American sample60 because of our revised tympanometric criteria (A/C1 = cure) and may be attributed also to spectrum bias (not able to be considered in the original calculation assumptions).

The University of Southampton

Appendix A

The University of Southampton

Trial flow list of procedures