Saline in Acute Bronchiolitis RCT and Economic evaluation: hypertonic saline in acute bronchiolitis - randomised controlled trial and systematic review

Primary objective

The primary objective of the study was to assess whether or not the addition of nebulised 3% HS to usual supportive care resulted in a reduction in time to being declared fit for discharge.

Secondary objectives

1. Assessment of the impact of the intervention on other clinical outcomes and the quality of life of infants and carers at 28 days post randomisation.

2. Investigation of the impact on outcomes between those infants with human respiratory syncytial virus (RSV) infection and those with acute bronchiolitis due to other causes, including other viruses (non-RSV).

3. Assessment of the economic impact of the intervention on both the NHS and parents at 28 days post randomisation.

Trial design

The SABRE study was a multicentre, parallel-group randomised controlled trial (RCT), with a one-to-one allocation ratio and economic evaluation. It aimed to determine whether or not the addition of nebulised 3% HS to usual care resulted in significant (25%) reduction in the duration of hospitalisation of infants with acute bronchiolitis. Subjects were recruited from the paediatric wards and assessment units of the participating centres in England and Wales between October 2011 and December 2013. A table of changes made to the protocol over the course of the project is presented in Appendix 1.

Important changes to methods after trial commencement

Participants and eligibility criteria

The target population was infants less than 12 months of age admitted to hospital with a clinical diagnosis of acute bronchiolitis and requiring supplemental oxygen as part of routine supportive care. Subjects were recruited from paediatric wards and assessment units from the 10 participating centres between October 2011 and December 2013.

Settings and locations where the data were collected

The data used in this study came from the following sources:

• for patients who were eligible for randomisation into the study (including patients who were randomised):

  • patient recruitment form, which contained the age, sex and details regarding entry criteria

• for patients who were randomised into the study:

  • randomisation schedule, which contained randomisation codes and allocated intervention group

  • case report form, which contained patient demographics, characteristics at presentation, investigations and events during treatment, assessment of ‘fit for discharge’, resource use and adverse events (AEs) over subsequent 28 days

  • post-discharge data collection forms, comprised of the symptom and health service utilisation diary.

Interventions

Infants were randomly allocated to either standard supportive care (control) or standard supportive care plus nebulised 3% HS solution (intervention) using a remote access, computer-generated allocation algorithm.

Three per cent HS is licensed as a medical device in the UK under the brand name MucoClear^® (PARI GmbH, Starnberg, Germany) and is indicated for the mobilisation of secretions in the lower respiratory tract in patients with persistent mucus accumulation such as those with acute bronchiolitis or cystic fibrosis. It was presented as a 4-ml plastic ampoule for nebulisation and came in packs of 20 or 60. The product contained no preservatives and had an expiry date of 3 years from manufacture. The product was administered via the PARI Sprint nebuliser (PARI Medical Ltd, Surrey, UK). The dose was 4 ml every 6 hours, in accordance with previously published studies. It was administered by a nurse, with infants inhaling the aerosolised saline. The saline was discontinued once the fit for discharge criteria had been met – in air for 6 hours with oxygen saturations of at least 92% and feeding satisfactorily.

For the purpose of this trial, MucoClear^® 3% was sourced by and dispensed locally from each site’s pharmacy department. Although this was not a clinical trial of a medicinal product and therefore not subject to the Medicines for Human Use (Clinical Trials) Regulations 2004,45 the principal investigator in conjunction with his or her local pharmacy department ensured that the principles of good clinical practice (GCP) were applied to ensure robust accountability of the supply and administration of 3% HS.

Outcomes

Table 1 shows the timing of outcome assessments.

Changes to trial outcomes after the trial commenced, with reasons

Sample size

Based on the current mean time to discharge of around 3 days, we felt that a 25% reduction would be the minimum clinically significant effect, and this was the magnitude of the effect observed in previous studies. 47 As LoS varies considerably across different settings, we used UK Hospital Episode Statistics data as the basis of our sample size justification. Assuming a log-normal distribution, the standard deviation (SD) was estimated at 32 hours. While a similar, or smaller, SD was expected for our primary outcome measure, a slightly inflated SD of 46 hours was used because of uncertainties over its derivation here. In order to have 90% power to detect a 25% difference in time to meeting discharge criteria, the study needed 139 patients per group at a two-sided α-level of 5%. The dropout rate was thought to be negligible for the analysis of the primary outcome measure and therefore a conservative estimate of sample size was 150 patients per group. Overall, 10 centres recruited patients to the study. Targets of 25 participants for district general hospitals and 40 participants for teaching hospitals were set.

Explanation of any interim analyses and stopping guidelines

The Data Monitoring and Ethics Committee (DMEC) charter allowed the study to be stopped prematurely on the grounds of safety or futility based on recommendations from the DMEC or the funder. No formal interim analyses were planned for efficacy and consequently no adjustment was initially made for multiplicity. At the request of the DMEC, one interim analysis was subsequently undertaken in July 2012 to assess efficacy, in order to inform a funding extension decision necessitated by recruitment being slower than expected. An O’Brien–Fleming stopping rule was retrospectively employed in which superiority would be declared if statistical significance was demonstrated at the 1% level in the interim analysis or at the 4.5% level at the final analysis. 48 The DMEC had the authority to recommend that recruitment to the study should be terminated if the primary outcome (time to fitness for discharge) was significantly shorter in the saline group at the α = 1% level of statistical significance.

Method used to generate the random allocation sequence

A central web-based randomisation service delivered by the CTRU was used after patient eligibility and written consent were confirmed. Patients were randomly allocated via the online system to receive either (1) nebulised HS with usual care (n = 150) or (2) usual care (n = 150). The randomisation schedule was computer generated prior to the study by the CTRU Randomisation Service.

Type of randomisation; details of any restriction (such as blocking and block size)

Participants were individually randomised using centralised, web-based randomisation system. Randomisation was conducted in randomly ordered blocks of size 2, 4 or 6, stratified by hospital.

Allocation concealment mechanism

The allocation schedule was concealed through the use of the centralised web-based randomisation service, which also allowed unblinding in the case of emergency. The randomisation sequence was not revealed to any person involved in patient recruitment. The data analysts were blind to treatment allocation until after the statistical analysis plan was finalised, the database locked and the data review completed. All unblinding (emergency and end of trial) would have been automatically logged by the CTRU randomisation system, which would have included the date, time and user responsible.

Sequence generation, enrolment and assignation

The randomisation sequence was computer generated and was not revealed to any person involved in patient recruitment. Recruitment was undertaken by GCP-trained trainee paediatricians, consultants or staff with equivalent training (e.g. GCP-trained paediatric nursing staff responsible for acute admissions). They approached parents/guardians to discuss the study at the point of first contact as soon as the infant had been identified as eligible for the study. Written information was then provided to parents/guardians willing to consider their infant taking part in the study. If agreed, written informed consent was obtained from the parents/guardians.

A flow chart detailing the consent and randomisation procedure can be found in Appendix 3. Owing to the acute nature of the condition, it was not possible to follow a usual time frame of at least 24 hours between receiving information and giving written informed consent. However, every opportunity was taken to ensure that parents/guardians fully understood the implications of taking part in our study. They were invited to ask questions and reflect prior to signing the informed consent document. No payment was offered to participants. The recruiter entered the participant’s details onto an online system and was asked the following:

• to confirm written consent was received

• to confirm the participant was in oxygen

• to enter the time a paediatrician or paediatric advanced nursing practitioner made the decision to admit.

Participants were then randomly allocated via the online system to receive either (1) nebulised HS with usual care (n = 150) or (2) usual care (n = 150). The allocation was explained to participants by the hospital staff member obtaining consent.

Blinding

The study compared the intervention plus usual care with usual care alone, with no placebo. The use of placebo in this setting is ethically problematic as it would result in infants in the placebo arm receiving an intervention that may have a significant effect on outcome. The extra handling involved in nebulised therapy may have a deleterious effect, as has been shown to be associated with the increased handling associated with physiotherapy. 49 Similarly, the placebo agent may cause harm, as has been suggested in previous studies using nebulised distilled water, or may have an unexpected positive impact for an agent, such as NS. 32

The randomisation codes were stored electronically on the CTRU randomisation system. All other electronic data were held separately on the CTRU database system. Access to any data that would unblind the study was limited to members of the CTRU who were independent of the trial. All summaries presented to the DMEC were by treatment group. In order to maintain the blinding of the trial statistician, it was the responsibility of the CTRU to provide this by-treatment group information to the DMEC. No member of the study team had access to unblinded data sets or the unblinded reports until the final analyses.

Statistical methods

The detailed statistical analysis plan can be found in Appendix 4. As time to being declared ready for discharge could be regarded as a survival time, initial differences between groups were assessed using the log-rank test. In order to adjust for centre, a Cox proportional hazards regression model was used with centre fitted as a fixed effect (FE). One centre, Rotherham NHS Foundation Trust, randomised one patient: for the purposes of adjusting for centre, this patient was combined with those recruited from Doncaster and Bassetlaw Hospitals NHS Foundation Trust, given the similarities that exist between these two geographically proximate populations in south Yorkshire. The proportionality of the hazards was assessed by examining the scatter of scaled Schoenfeld residuals against time. 50 This model was further extended by inclusion of viral status (RSV vs. non-RSV) to allow an examination of whether or not viral status had an impact on the effectiveness of treatment. In order to answer this question, the coefficient for the interaction between infection group (RSV vs. non-RSV) and treatment group was calculated together with its 95% confidence interval (CI) and p-value. The collection of RSV test data was not a protocol requirement and such data were not collected at any particular centre except as part of routine practice.

The analysis of actual time to discharge was similar to the primary outcome measure, time to ‘fit to discharge’. Rates of admission to HDU/ICU and readmission rate within 28 days from randomisation were compared using Fisher’s exact test; although it is possible that children could be readmitted multiple times as a result of their single episode of bronchiolitis, it is much more likely that they will be readmitted just once. Thus, the percentage of children readmitted at least once was used for these analyses. In order to test the assumption that the results between groups differed according to RSV status, a logistic regression model was fitted and a coefficient for the interaction between infection group (RSV vs. non-RSV) and treatment group was calculated together with its 95% CIs and p-value. Each of the nine dimensions of the ITQoL used in this study was examined for differences between treatment groups, initially using a t-test. Where the assumptions underlying the t-test do not hold, a Mann–Whitney U-test was used.

It was originally envisaged that symptom duration would be analysed as a survival outcome. However, diary data were returned for only 108 individuals and, of these, data to 28 days were unavailable for almost one-quarter (25 out of 108: 1 at 15 days; 1 at 17 days; 1 at 18 days; 22 at 27 days) and almost one-quarter (24 out of 108) reported that they were still experiencing symptoms at 28 days. Although the majority of infants whose carers stopped sending in data before 28 days were no longer experiencing symptoms, because of the large amount of censoring at 28 days, we defined a new outcome variable that evaluated whether or not the patient was still experiencing symptoms at 28 days. This was analysed using the methods outlined above for binary outcomes.

The trial was originally designed to have a two-sided significance level of α = 5%. As a result of the aforementioned interim analysis, the significance threshold for the final analysis was changed to 4.5% in order to preserve the overall trial-wise significance level at 5%. All CIs remain two-sided 95% intervals. Analyses were conducted using IBM SPSS version 20 (IBM Corporation, Armonk, NY, USA) and SAS version 9.3 (SAS Institute Inc., Cary, NC, USA).

Analysis populations

Four populations were used in the analyses:

1. all screened patients: patients who were screened for eligibility to the study, including those randomised

2. full analysis set (FAS): all randomised patients, with the following exclusions:

   1. patients who had previously been randomised, in which case only data relating to the first admission were analysed

   2. patients for whom no recorded informed consent was obtained from carers (oral or written)

   3. patients whose carers withdrew consent before any study medications were given

   4. patients whose carers withdrew consent retrospectively (i.e. requested that all the patient’s data were removed)

3. per protocol (PP): the subset of patients in the FAS who did not deviate from the protocol

4. safety: all randomised patients, with the exception of those for whom there was no recorded informed consent.

Summaries based on the FAS and PP populations were on an intention-to-treat basis, with patients assigned to the treatment group as originally randomised. Summaries based on the safety population analysed patients by the actual treatment received. Aside from RSV status (see Statistical methods) no other subgroup analyses were planned.

All outputs presented to the DMEC were based on the FAS population unless otherwise requested.

Patient and public involvement

Background

Nebulised HS is a relatively cheap intervention: approximately £50 for the nebuliser plus 12 doses of saline. Consequently, a large reduction in LoS is not required to suggest that the introduction of nebulised HS may be cost neutral or even cost saving. However, a broader economic evaluation that captures a wider set of costs and also includes patient outcomes is preferred. For example, if the intervention is not cost neutral, it may still be cost-effective once the value of patient health effects are also considered. Alternatively, the intervention may be cost-saving from the perspective of the hospital, but worse symptoms and more associated care following discharge may show the intervention not to be cost-effective.

Overview

A cost–utility analysis (CUA) was undertaken from the NHS perspective, with a time frame of 36 days post randomisation. The economic evaluation was originally designed to have a 28-day follow-up, but one patient in the study had a LoS of 35.7 days, so the time frame was extended to avoid censoring. A longer time frame was relevant, as there are no long-term sequelae associated with acute bronchiolitis. Given the difficulties of measuring utilities in the very young, the CUA was supplemented with a cost–consequences analysis (CCA) which considers the secondary clinical outcome measures alongside costs to allow a more qualitative assessment of cost-effectiveness.

Resource use

The main items of resource use collected were LoS by type of ward and number of readmissions. Nursing time was not included because, although nurse time is required to set up the equipment in the intervention group, increased monitoring time may be needed in patients who are not receiving active treatment. Consequently, a detailed observational study would be required to measure these costs in both arms. It was considered that any cost difference would be small, and inconsequential compared with any difference in LoS, and that calculating any difference would incur huge costs . Furthermore, if there were no difference in LoS, then clinical practice would not change based on small differences in nursing time.

Although other costs were also considered to be negligible in comparison with the inpatient costs, ease of data collection led us to collect data on the use of nebulised saline, concomitant medications and use of primary care services in the 28 days following discharge (e.g. GP or emergency department visits). Inpatient resource use was collected by staff using the same case report forms as those used for the clinical evaluation. Primary care resource use was collected using a patient diary, which was to be completed by the parent or guardian on a daily basis; tick boxes were available for GP contacts, NHS Direct, NHS walk-in centres, minor injury units and the emergency department.

Unit costs

All unit costs used were at 2012/13 price levels. Costs for a day in hospital were based on 2012–13 NHS Reference Costs, nebuliser costs were supplied by one of the participating hospitals, saline and medication costs were taken from the British National Formulary (BNF)51 and primary care costs were taken from NHS reference costs52 and the Unit Costs of Health and Social Care publication produced by the Personal Social Services Research Unit. 53 These are summarised in Table 2.

The cost per day on a ward was derived using the NHS reference costs52 for an admission for paediatric acute bronchiolitis without complications (PA15B) and its associated number of bed-days. Separate costs per day for paediatric intensive care and high-dependency care are available directly from NHS reference costs. 52 The cost for paediatric acute bronchiolitis with complications or comorbidities (PA15A) was not considered relevant as our separate costing of paediatric intensive care and high-dependency care captures the costs of complications.

NHS Direct costs were taken from a report by the Medical Care Research Unit, which used a Department of Health contract value to derive a cost per call. 54 Emergency care unit costs (defined here as walk-in centres, minor injury units and emergency departments) relate to category 1 investigation (e.g. biochemistry) and category 3–4 treatment (e.g. oxygen).

Outcomes

For the CUA, it is usual for utilities to be calculated based on utility instruments, such as the EQ-5D, administered to patients. However, this is not possible for the participants in this trial, and even proxy completion by a parent would not be valid, as the descriptive systems of available utility instruments do not fit well with this patient group. Our original approach was to undertake a separate valuation study to estimate quality-adjusted life-year (QALY) losses related to hospitalisation and recovery and then apply these to the trial data. However, the National Institute for Health Research was not willing to fund this work, so we identified a published estimate relating to hospitalisation in paediatric populations that could be used. 55

Carroll et al. 55 estimated the utility decrement from full health during hospitalisation as 0.05 (with a standard error of 0.01) using the time trade-off technique. This then needs to be combined with the general population value for infants of the same age. The best method for combining comorbidities is considered to be through multiplication of the relevant values. 56 Following a search of the literature, the most relevant general population value using a paediatric utility instrument for the children recruited to the SABRE study was found to be a Health Utilities Index-II survey of 8-year-old Canadian children. The general population value in that study was 0.95 (compared with 0.82 for children of extremely low birthweight). 57 For the SABRE study, therefore, the non-hospitalised utility is 0.95 (taken from Saigal et al. 57), while the hospitalised utility is 0.9025 (calculated as the product of the Saigal et al. 57 and Carroll and Downs55 utility estimates).

Analysis

Differences in costs and QALYs were assessed using non-parametric permutation t-tests that allow robust comparisons of the means in the presence of non-normal data. A permutation test samples from the observed data to calculate the correct distribution of the test statistic under the null hypothesis. Paired cost and QALY data were then bootstrapped with 5000 replications and plotted on the cost-effectiveness plane. The focus of the economic analysis was the resultant cost-effectiveness acceptability curve and the probability that the intervention will be cost-effective at £20,000 per QALY.

The main cost driver was expected to be hospital costs. Deterministic sensitivity analyses were planned for the cost per day on a ward, in intensive care and in high-dependency care. Alternative plausible unit costs to our preferred ward estimate of £547 are £558 per day (with the latter also including complicated admissions) and £425 (derived using excess bed-days costs for uncomplicated admissions). Alternative plausible unit costs to our preferred intensive and high-dependency care estimates of £1946 and £1061 are £1743 and £886, respectively (derived using only the basic care categories within each type of unit).

Probabilistic sensitivity analysis is undertaken through the bootstrapping procedure described previously, thereby incorporating the sampling uncertainty related to all clinical effects and resource use items. Probabilistic sensitivity analysis using distributions around unit costs and utility values is not routinely undertaken in economic evaluations alongside trials.

Where appropriate, missing data were to be imputed using multiple imputation. This assessment would be based on the degree of missingness and the mechanism by which missing data were generated (e.g. random or non-random).

Rationale

To produce an overview of the clinical effectiveness of HS for the treatment of inpatients with acute bronchiolitis.

Objective

The objective was to systematically review the evidence relating to the use of nebulised HS in young children, hospitalised for the treatment of acute bronchiolitis. The primary outcome of consideration was whether or not the use of this intervention resulted in a reduced hospital LoS.

Protocol and registration

The PROSPERO registration (registered 3 March 2014 and revised 18 December 2014) outlining the study review protocol can be found in Appendix 5.

Eligibility criteria

Information sources

The electronic databases searched included MEDLINE (via Ovid) (1946 to January 2015), EMBASE (1974 to January 2015), the Cochrane Central Register of Controlled Trials, Google Scholar (Google Inc., Mountain View, CA, USA) (2010 to January 2015) and Web of Science (2010 to January 2015). The full search strategy used in each database is listed in Appendix 6. No restrictions or limits (e.g. age, language and publication date) were applied to any of the databases other than Google Scholar and Web of Science, which were searched for articles from 2010 onwards, when the Cochrane Systematic review by Zhang and colleagues71 was last updated.

The following trial registries were searched, using the terms ‘bronchiolitis’ and ‘hypertonic saline’, to identify any unpublished data: ClinicalTrials.gov; UK Clinical Trials Gateway; Centre for Reviews and Dissemination databases (Database of Abstracts of Reviews of Effects, NHS Economic Evaluation Database, HTA Database); controlled-trials.com; centrewatch.com. The National Research Register was searched using the terms ‘bronchiolitis and ‘hypertonic saline’ on 5 March 2014, as the website could not be accessed when the searches were updated. We also hand-searched the journals Chest, Paediatrics and Journal of Paediatrics in January 2015 using the terms HS and bronchiolitis. The reference lists of all eligible trial publications were checked to identify any further published trials.

Search

A range of search strategies were employed, including pearl growing and checking reference lists. 72,73 Pearl growing involved identifying medical subject headings from articles identified from the previous systematic review,25 which were then combined with initial search terms and the searches rerun again. The final search strategy is outlined in Appendix 6. All searches were performed between January 2013 and January 2015. Any duplicated articles were removed from the final list prior to applying the eligibility criteria to the trials to select the studies of interest.

Study selection

After removal of duplicates, two researchers (CM and HC) independently screened titles and abstracts for eligibility; differences were resolved by discussion with DH and ME. If the abstract suggested eligibility, if no abstract was available for a particular citation or if the title was vague and unclear, the full paper was retrieved. Full papers were screened for eligibility in the same way.

Unsuccessful attempts were made to contact trial investigators of two unpublished studies74,75 to request additional unreported data. Although studies would not have been excluded based on the primary outcome, all but three of the studies were appropriate to be included in the meta-analysis of the primary outcome of interest, LoS.

Data collection process

The Cochrane risk of bias assessment tool76 was modified after piloting on one study of known eligibility. Details of the modified data extraction tool can be found in Appendix 6 and Table 3. CM and HC then used the modified tool independently to extract data. Attempts were made to contact study teams for missing data, particularly for five studies74,75,84,88,89 for which data were available only on ClinicalTrials.gov or in abstract form. Owing to the constraints in time, it was not possible to contact all the study authors for each of the items of missing data, for example risk of bias assessment.

Data items

1. Study overview (country, year).

2. Participant characteristics (age, number randomised, baseline imbalances assessed by the authors in trials, withdrawals, % allocated completing follow-up, illness severity, eligibility).

3. Intervention and control group details (number randomised in each group, intervention details: duration, delivery, other drugs and compliance).

4. Outcomes data:

   1. Continuous outcomes: LoS (mean LoS, SD and number of patients in each group, measured by who); CSS score (mean final CSS score, SD, number of patients for both groups).

   2. AE data as available.

Risk of bias in individual studies

The Cochrane risk of bias assessment tool allows reported trial results to be differentiated from the methods used, with lower risk studies associated with more conservative effect estimates of risk. 91 CM and HC independently graded the quality of trials as ‘low’ where there is low risk, ‘high’ where there is high risk of bias or ‘unclear’ where the risk is unclear. Discrepancies were resolved by discussion. Risk of bias assessment was conducted at both study and outcome level. The Cochrane Handbook for Systematic Reviews of Interventions,76 chapter 8.5.2, was used as a guide to make judgements on the methodological quality of the trials for the following domains of bias: sequence generation; allocation concealment; blinding, completeness of outcome data; and selective reporting. A supporting statement for each bias domain was extracted where a grade is given. Where an unclear grade is given, effort was made to obtain further information to categorise the trial by contacting the trial authors or searching for the study protocol to identify sources of reporting bias. Risk of bias data were entered into the Cochrane Review Manager (RevMan version 5.2, Cochrane, The Nordic Cochrane Centre, Copenhagen, Denmark) tool for analysis. Both a risk of bias graph and risk of bias summary were derived from the data and are reported in the results section.

Summary measures

All outcomes were continuous; the summary measure was the median difference and its associated 95% CI.

Synthesis of results

Analyses were undertaken using both RevMan version 5.2 and Stata version 12.1 (StataCorp LP, College Station, TX, USA). A FE model was used to combine (unstandardised) median differences between HS and the control. The FE model assumes that the LoS outcome would estimate the same effect size in each of the studies and weights studies according to their precision (standard error), which in turn relates to the sample size: the greater the sample size, the greater the weight assigned to the trial. 92 In addition, a second analysis was undertaken using the random-effects (RE) approach of DerSimonian and Laird,79 which incorporates the variability among trial estimates into the weighting of the trials.

Three-armed trials, which compared more than one concentration of HS with a control, were included in the review. The Cochrane Handbook for Systematic Reviews of Interventions recommends combining the active arms, but notes that heterogeneity can be better assessed by keeping dose arms separate. 76 As high levels of heterogeneity were identified in our review (see Chapter 5, Primary outcome: results and synthesis), this was the approach we adopted. To avoid double-counting of control patients, in three-arm trials control group numbers were divided by 2.

Mean final (post-treatment) CSS scores for participants in each arm of the trial were also extracted. A scoring system devised by Wang et al. 78 was used to assign CSS scores at baseline. Some of the studies reported a final CSS score for both the intervention and the control groups.

Data on AEs were synthesised narratively.

We used the I²-statistic to measure statistical heterogeneity with the following guidance on interpretation from the Cochrane Handbook:76

1. I²-statistic of 0–40% = heterogeneity may not be important

2. I²-statistic of 30–60% = may mean moderate heterogeneity

3. I²-statistic of 50–90% = may mean substantial heterogeneity

4. I²-statistic of 75–100% = considerable heterogeneity

5. Overlapping CIs are an indication of lower heterogeneity within the trials.

Forest plots also display the chi-squared test to test for statistical heterogeneity, with corresponding p-values at or below the threshold of 0.1 conventionally indicating a statistically significant heterogeneity. 70

Risk of bias across studies

A funnel plot was generated to explore the possibility of publication bias in the primary outcome, LoS, for all trials. The main group was then split into subgroups to investigate reasons for heterogeneity.

Heterogeneity was explored by separating the group by comparator based on the following:

Intervention comparator:

1. nebulised HS alone versus NS alone (n = 4 trials)

2. nebulised HS plus a bronchodilator (e.g. salbutamol) versus NS alone (n = 0 trials)

3. nebulised HS plus a bronchodilator (e.g. salbutamol) versus NS plus same bronchodilator (n = 12 trials)

4. nebulised HS alone or plus a bronchodilator versus no treatment (n = 1 trial).

Further analyses were performed to separate the type of bronchodilator administered, for example beta-2 agonist compared with adrenaline. The subgroups for these are listed below:

1. nebulised HS alone versus NS alone (n = 4 trials)

2. nebulised HS plus a bronchodilator (e.g. salbutamol) versus NS alone (n = 0 trials)

3. NEBULISED HS plus a beta agonist versus NS plus same beta agonist (n = 7 trials)

4. nebulised HS plus adrenaline versus NS plus same adrenaline (n = 5 trials)

5. nebulised HS alone or plus a bronchodilator versus no treatment (n = 1 trial).

Outcome reporting bias arises from selectively picking outcome measures to report for the study based on the results achieved. 93 Smaller, lower-quality trials may in themselves introduce bias whereas larger, more expensive studies which may be more methodologically sound are more likely to be published even with negative results. 94 Using methods suggested by Dwan and colleagues,95 we also assessed the risk of outcome reporting bias.

Participants who were randomly assigned, received intended treatment and were analysed for the primary outcome

Between 26 October 2011 and 23 December 2013, the trial recruited and randomised 317 participants, with 158 patients allocated to the nebulised 3% HS group and 159 allocated to usual care (Figures 1 and 2). There were five patients from the nebulised 3% HS group who did not receive the intended treatment. Of the 317 patients randomised, 26 of these patients were excluded because they were randomised when ineligible and for one patient primary outcome data were unavailable because their medical notes were lost; therefore, 290 patients were included in the primary outcome analysis.

FIGURE 1.

Participant recruitment curve.

FIGURE 2.

The CONSORT flow diagram. ITT, intention to treat.

Losses and exclusions after randomisation

The number of post-randomisation exclusions together with the reason for exclusion is displayed in Table 4. Overall, 26 patients were excluded from the study: 16 from the intervention group and 10 from the control group.

Of the 317 randomised patients, 26 were excluded, as described previously. Of the remaining 291 patients, five were not included in the PP analysis for the following reasons: four withdrew before receiving any HS/intervention and one patient’s medical notes were lost (therefore no treatment data were available). Of the remaining patients in the PP analysis, 32 withdrew from treatment having received at least one dose of HS and three withdrew from the study, two from the standard care group and one from the standard care plus intervention group.

Dates defining the periods of recruitment and follow-up

The trial consisted of three recruitment seasons. The study recruited patients during winter 2011/12, winter 2012/13 and from October 2013 to December 2013. Patients were followed up for a period of 28 days after randomisation to collect data on readmissions, duration of respiratory symptoms post discharge, health-care utilisation post discharge and infant and parental quality of life.

Why the trial ended or was stopped

The trial closed to recruitment after reaching the accrual target on 23 December 2013.

Primary outcome: results and synthesis

The data extracted for the primary outcome are displayed in Table 23. Although all trials stated that LoS was being measured, a closer review of the methods revealed that most in fact had measured the time to being fit for discharge (variably defined). To avoid confusion among those familiar with the included trials, we will continue to use the term LoS for the remainder of this section, but will revisit this issue of nomenclature in Chapter 6, Discussion.

The mean LoS, SD and sample size in each trial for each group. Data are provided for all of the trials; however, Nemsadze et al. did not provide the number of patients in each group and so cannot be included in the meta-analysis. 84 In addition, the mean, SD and sample size for LoS were not available for two trials, as information was available only in abstract form or on ClinicalTrials.gov. 75,89

To avoid double-counting of control patients, in the case of three-arm trials we divided control group numbers by two (see Chapter 2, Synthesis of results). Teunissen et al. 86 (3% HS group, n = 84; 6% HS group, n = 83; and control, n = 80) was included as two effective studies in the analysis [‘Teunissen (3% HS)’, n = 84 vs. 40; ‘Teunissen (6% HS)’, n = 83 vs. 40],86 and the same approach was taken with Al-Ansari83 [3% HS group, n = 58; 5% HS group, n = 57; and control, n = 56; ‘Al-Ansari (3% HS)’ n = 58 vs. 28; ‘Al-Ansari (5% HS)’ n = 57 vs. 28].

Figures 13 and 14 present the LoS for each trial, together with the pooled difference between HS and control, using both FE and RE methods. The weighted average of the effect sizes observed across 15 RCTs (n = 1922 participants) would indicate that HS, alone or in combination with other therapies, reduces the LoS in children with acute bronchiolitis by just over one-third of a day [mean difference (MD) –0.36, 95% CI –0.50 to –0.22] compared with no HS (see Figure 13). However, the level of statistical heterogeneity associated with this estimate is too high for it to be a safe basis for clinical decision-making, with over three-quarters of the between-study variability not attributable to the play of chance alone (I² = 78%). Of particular note is that our own trial was one of the largest of the 15 trials included, yet it received a mere 4.2% weight in the FE analysis. The most probable reason for this apparent incongruity is the excessive heterogeneity among the studies, with differential case mix (e.g. patient severity or discharge policies) being typified by differences among the SDs and hence study weights. RE analyses have been advocated to address this; this method yields a larger MD (–0.48 days; 95% CI –0.81 to –0.15 days). Nevertheless, this does not overcome the problem of excessive heterogeneity; the between-trial variation dominates the within-trials variation and effectively allocates similar weights to each study irrespective of their size. 119,120 The same pattern was observed when combining standardised MDs (i.e. dividing the MD by the underlying SD78).

FIGURE 13.

Difference in length of hospital stay by intervention subgroup, 3% HS. D&L, DerSimonian and Laird; IV, inverse-variance; N/A, not applicable.

FIGURE 14.

Difference in LoS by intervention subgroup, all concentrations of HS. D&L, DerSimonian and Laird; IV, inverse-variance; N/A, not applicable.

Within the pre-planned, protocol-specified intervention subgroups (see Figure 13) there are lower levels of statistical heterogeneity, suggesting that clinical heterogeneity deriving from differences in care regimens could underlie some of the observed between-trial differences in treatment effect. However, the risk of bias from other sources (see Risk of biases across studies and Chapter 6, Strengths and weaknesses of the systematic review) means that the summary estimates even from the subgroup analyses should be treated with great caution.

The main analysis as presented in Figure 13 includes only comparisons of 3% concentrations of HS with control interventions, as in the SABRE study. Two trials also included third arms delivering higher doses of HS. When these arms were included, the pooled effect size and associated heterogeneity are reduced marginally without altering the results appreciably (MD –0.34, 95% CI –0.48 to –0.21, using a FE analysis; MD –0.44, 95% CI –0.74 to –0.14, using RE; see Figure 14).

What is clear is that a large amount of the heterogeneity is driven by two trials from the same team, led by Luo,79,80 with outlying results, relatively small sample sizes but narrow CIs (around 1 day, compared with a 1.5 days in the SABRE study87 and the other large northern European study, Teunissen et al. 86). Together, the Luo et al. trials contribute 14.5% of the weight in the main analysis (see Figure 13). Their central estimates of clinical effect, –1.4 days79 and –1.6 days,80 are included in the CIs of only four of the remaining trials. 22–24,81 The removal of these two studies from the main analysis considerably reduces the effect sizes and statistical significance in the analyses of HS versus NS alone (MD –0.07 days, 95% CI –0.41 to 0.27 days; p = 0.67) and HS + beta-2 agonist versus NS + beta-2 agonist (MD –0.03 days, 95% CI –0.24 to 0.17 days; p = 0.74). The elimination of Luo79 from the beta-2 agonist subgroup reduces the heterogeneity from I² = 82% to I² = 0%. The elimination of these two studies reduces the overall FE pooled effect size from 0.36 days (95% CI –0.50 to –0.22 days; p < 0.00001; test for heterogeneity p < 0.00001, I² = 78%) down to –0.16 days (–0.31 to –0.01 days; p = 0.03; test for heterogeneity p = 0.02, I² = 32%). On the other hand, the elimination of even the three most negative studies from the main analysis (see Figure 13) makes little difference. 74,86,87 In short, the Luo studies are major drivers of heterogeneity.

Secondary outcomes: results and synthesis

There was no mention of AEs in seven studies;74,75,77,81,84,89,90 however, Maheshkumar et al. 81 stated ‘HS is safe’, implying that no AEs were observed, and Nemsadze et al. 84 was published only as an abstract. Tal et al. 23 stated that no AEs were observed in either of the groups. 23 Kuzik et al. reported that there were no withdrawals because of AEs by medical staff; however, two patients in the intervention group were withdrawn on parental request (one because the infant cried vigorously during two inhalations and the other because of agitation on one inhalation). 24 Luo et al. 79 stated in the first season that all patients completed treatment and that wheezing and coughing did not worsen over course of treatment. In the second season, Luo et al. 80 highlight that no infants were withdrawn as a result of AEs and that coughing and wheezing never worsened during treatment. Five infants (two in the intervention group) experienced hoarse voices but this improved after 3 or 4 days. Mandelberg et al. 22 report that no AEs were observed. Pulse rate and room air saturation did not differ between the two groups on any day. One patient was excluded from analysis because of a need for mechanical ventilation. Sharma et al. 82 also concur that no AEs related to any of the patients were reported by parents, caregivers or treating medical attendants in either group. Al-Ansari et al. 83 also state that no safety concerns or adverse reactions were identified. Pandit et al. 85 reported that vomiting and diarrhoea occurred in four infants, all in the control group; in addition, they noted no tremors or paleness during treatment. Teunissen et al. 86 reported 95 AEs in the 3% HS group, 119 in the 6% HS group and 76 in the NS group. Silver88 stated that only nine participants in the 3% HS group and four in the NS group did not complete the study because of AEs. According to additional information provided by the author, 10 participants in the HS arm and nine in the control arm were transferred to ICU/clinical worsening. Finally, for the SABRE study,87 AEs are described in detail in Table 10; overall, there were 51 AEs recorded in the standard care plus intervention group and 43 in the standard care group.

Readmission rate was not reported in 15 trials. Al-Ansari et al. 83 state that short-stay readmission was required for 10 infants in the 5% HS group, eight infants in the 3% HS group and seven infants in the 0.9% saline group. 83 Silver88 reported that four participants in the 3% HS group and three in the 0.9% NS group were readmitted within 7 days. In addition, Everard et al. 87 showed a lack of evidence to suggest that there was a difference between treatment groups in terms of the numbers admitted to HDU/ICU and readmitted within 28 days of randomisation. The final mean CSS scores were also analysed for the five studies for which data were available23,77,79,80,86 (Table 24). The average change of –1.36 points (95% CI –1.52 to –1.20 points) on the CSS observed in participants given HS (Figure 15) should be treated with caution because of the number of studies contributing data to this analysis and the risk of bias discussed in Risk of biases across studies and Chapter 6, Strengths and weaknesses of the systematic review.

FIGURE 15.

Mean final CSS scores. IV, inverse-variables.

Publication bias

A funnel plot (plotting standard error against the MD) was generated to further assess the risk of bias across studies in relation to publication bias (Figure 16).

FIGURE 16.

Funnel plot, difference in length of hospital stay (whole group). SE, standard error.

This is not a classically symmetrical funnel plot, in which the study with the smallest standard error (in this case Sharma et al. 82) sits at the top of the inverted funnel indicated by the dashed lines, which show the expected 95% CIs around the FE summary effect. But neither does it show the typical absence of small studies in the bottom right-hand corner, characteristic of meta-analyses affected by publication bias,121 although there are fewer studies to the right-hand side of the Sharma study82 than one would expect. Instead, we see four studies with comparatively small standard errors outside the dashed lines. These include the second and third largest studies (Sharma82 and Teunissen,86 respectively) as well as the two studies by Luo and colleagues. 79,80 A plausible explanation for this shape may be the high levels of heterogeneity discussed in Primary outcome: results and synthesis. 121

Outcome reporting bias

The results of the steps suggested by Dwan et al. 95 can be seen in Table 25.

The primary outcome was scored as ‘high risk’ for outcome reporting bias for one of the studies, owing to hospital LoS being stated as a secondary outcome, but no results were reported. 95 The majority returned a score of no risk (rate of readmission) or low risk/no risk where outcomes had been measured but partially reported, not necessarily analysed or not measured or analysed. Outcome reporting bias is concluded to be low for the review outcomes.

Strengths and weaknesses of the trial

The sample size was calculated based on a difference in mean times but analysed using Cox regression. Although not ideal, doing so enabled us to calculate a minimum clinically important effect size in absolute units while analysing the data by methods which allow both for the likely skewed data and for any loss to follow-up. We presented both absolute and relative differences between time to fit for discharge and time to discharge. In the event, the use of Cox regression was unnecessary; only one patient had partial notes available.

Some readers may consider the absence of blinding a potential limitation of the study. A number of previous studies involving HS and other nebulised interventions such as the antiviral agent ribavirin have used a randomised blinded approach using either distilled water or NS as the nebulised placebo agent. These approaches have been criticised because hypo-osmolar distilled water can induce bronchoconstriction, thereby hindering the infant’s recovery, while proponents of hypertonic therapy have argued that failure to show a difference when compared with nebulised NS is because of the beneficial effects of the depositing NS in the lower airways. 26 The SABRE study team consider the use of an open pragmatic study design a strength, in that it avoids potential confounders and determines whether or not the use of nebulised HS may have a place in routine clinical practice. We believe that the primary outcome, assessed through routine records of hourly oxygen saturations, is robust against the risk of bias, regardless of the absence of blinding. To introduce systematic bias would have required multiple nurses across shifts and across 10 sites deliberately falsifying routine observations, which we do not believe is credible.

Among the strengths of this study was the multicentre design using teaching hospitals and district general hospitals, which ensures that the results are generalisable to all paediatric secondary care units admitting such infants while the numbers of subjects recruited are such that there can be considerable confidence in the conclusions drawn from the data. Moreover, the study was designed to assess the potential role of this intervention in the most severely affected infants, namely infants admitted to hospital requiring supplemental oxygen therapy. Such patients place a significant burden on the NHS and other health-care systems around the world. This is because of both the large numbers of infants admitted to hospital every winter and the duration of their hospitalisation which is significantly prolonged when compared with a median acute paediatric stay of around 24 hours. Hence they place a significant burden on paediatric units in terms of bed-days occupied. Nebulised 3% HS was used because this was the concentration used in the majority of previous studies and because there is a commercial product on the market being sold for this indication. For comparison, the concentration of salt in blood is 0.9% and that in sea water is 3.4% (range 3.2–3.8%).

Attrition rates were low for this study, although follow-up for some secondary outcomes was less satisfactory. Only 49 out of 149 participants returned the diary and ITQoL in the control group and only 54 out of 141 did in the intervention group. Of 142 participants from the control group, 40 were not RSV tested; in the intervention group 39 out of 149 were not tested.

Strengths and weaknesses of the economic analysis

There are some uncertainties in the health economic analysis. LoS data were missing for one patient, and after excluding those with only limited resource use we have 64% missing data relating to primary care use. We also elected not to include the costs of concomitant medications into estimates. However, it is not considered plausible that any of these will materially affect the results of the CUA. Uncertainties around unit costs for the major cost drivers were explored in sensitivity analyses and were found to have no impact on the results, owing to the lack of any differences in LoS.

There are also some uncertainties with the methods employed. The two most important relate to hospital cost calculations and the utility values. In terms of the hospital costs, the cost per day figures shown in Table 2 are derived from NHS Reference Costs 201352 by dividing the relevant total cost by the number of hospital days. It is well recognised that, in some circumstances, such average costs will not accurately represent the costs attributable to a marginal change in the LoS. In other words, the cost of a patient’s final day in hospital is different from the average because typically they are less unwell just prior to discharge. Although this argument is relevant to surgical and long-stay patients, it is not thought to be particularly pertinent to the patient population within this trial. It should also be recognised that these costs are economic, not financial; they represent the value of the resource rather than an identifiable expenditure.

The utility values used in the analysis are probably best described as notional. The generation of utility values for such a young population is extremely challenging and it would take a separate study to produce values that would be considered robust. Given the context of the study – a cheap intervention and large cost reductions associated with any reduction in LoS – the methods employed were considered appropriate. It could be argued that the use of a generic hospitalisation decrement might underestimate the impact of a severe, acute condition. However, the utility value for ICU hospitalisation was only slightly lower (0.87 vs. 0.94)57 and, so, even using this much more severe health state would have no significant effect on the results or conclusions. There is one further methodological uncertainty associated with the utility values from Saigal et al. ,57 which is that, while the authors describe their results as population values, the description of the elicitation task used does not make it clear whether the responses relate to those of a parent, those of a patient or a mixture of both. Consequently, in the light of the clinical results of the study, and the lack of robust evidence of a difference in LoS, it could be argued that the estimated QALY differences should be ignored.

The limitations of the utility estimates were highlighted at the design stage of the study and so, to guard against an over-reliance on the CUA and resultant ICER, we proposed that a CCA also be undertaken. In contrast to a CUA, a CCA does not focus on a single measure of outcome (i.e. the QALY) or produce a single definitive measure of cost-effectiveness (i.e. the ICER). A CCA is merely the comparison of costs and all relevant outcomes between the two arms of the study. As such, it requires readers to make their own assessment of whether or not the differences in outcomes are valued more highly than the costs. In the context of the SABRE study, the relevant outcomes are defined by the protocol as primary and secondary outcome measures: time to fit for discharge, actual time to discharge, admission to ICU/HDU, readmissions, duration of respiratory symptoms, health-care use, infant/parental quality of life and AEs. Across all these measures there is only one statistically significant difference between the study arms, relating to the family cohesion domain of the ITQoL. Given that no allowance has been made for multiple significance testing and the ITQoL result relates to a small self-selected subsample of the trial population, the robustness of this difference is open to question. Consequently, the CCA simplifies down to the cost difference reported in Chapter 4 (£132; p = 0.694) and no evidence of patient benefit.

Less importantly, another source of uncertainty is that the additional staff time required to manage a patient while on a nebuliser was not included in our analysis. Therefore, the costs in the intervention groups would be considered to be underestimated, albeit by a small amount.

Finally, the analysis employed a time frame of 36 days post randomisation, rather than the planned 28 days. This was because one patient had a time to discharge of 857 hours (35.7 days) and, thus, there were important costs beyond the time frame of the planned analysis. In order to conform to NICE economic evaluation guidelines, the time horizon of the analysis was adjusted to the first whole day following discharge of all patients. A sensitivity analysis was conducted that truncated time to discharge at 28 days and this did not have any noticeable impact on the results or conclusions.

Strengths and weaknesses of the systematic review

The results of the systematic review are problematic to interpret because of high levels of statistical heterogeneity. While it was not within the scope and resources of our review to investigate this between-study variation with a formal meta-regression, some notable sources of clinical and methodological heterogeneity may provide an explanation (Table 26). Across intervention subgroups, some of this heterogeneity may be explained by the presence of concomitant medications, with the effect of HS being largest in trials in which concomitant medications were not given. This notwithstanding, two trials conducted in China with small sample sizes (n = 9379 and n = 11280) but small standard errors appear to be driving the I² value (see Chapter 5, Results and meta-analyses). As the quality of the studies seems adequate, it may be unfair to assume that there is some kind of fault with the individual trials. On the other hand, with the funnel plot (see Figure 16) suggestive of publication bias, it may be that there are a large number of trials missing as a result both of our restrictive eligibility criteria (the inclusion of only English-language publications) and of the documented publication bias associated with certain countries. 122

The systematic review was restricted to English-language articles because of the time and resource available for this review. The effect of this eligibility criterion has unknown consequences for the effect size, precision, heterogeneity and overall risk of bias associated with our meta-analysis. Although such restrictions can be seen as acceptable,123 we acknowledge the resulting exclusion of two trials of definite relevance at the full-paper stage68,69 and eight studies of possible relevance at the abstract stage59–64 is a limitation. 124 The use of language restrictions in systematic review-based meta-analyses is not always thought to cause systematic error, although it will potentially reduce the precision of pooled estimates. 125

The availability of adequate primary outcome data for all but one of the studies was a strength of the review, allowing for meta-analysis, subgroup analyses and investigation of publication bias. However, complete data were not available for all secondary outcomes or on features of trial design. The availability of more complete trial data may have enabled investigation of the underlying causes of statistical heterogeneity through metaregression,126 although this is not recommended when adequate data are only available for fewer than 10 studies. 76 It seems likely that disease severity and concomitant medications as well as local service configuration may be affecting between-study variation in effect size, but there are several other potential sources of heterogeneity which severely limit the use of aggregated level meta-regression analyses. In the absence of extensive individual patient data, the impact of these must remain speculative.

Unsuccessful attempts were made to contact trial investigators of one unpublished study74 to request additional unreported data. Although studies would not have been excluded based on the primary outcome, all but one of the studies were appropriate to be included in the meta-analysis of the primary outcome of interest, LoS. A number of the domains in the risk of bias assessment could not be assessed because of inadequate reporting. The biggest issue was with the ‘selective outcome reporting’ domain for which the absence of published protocols or responses from study teams made it difficult to assess pre-specified outcome measures. Attempts to contact the authors of one trial were unsuccessful,74 and the data retrieved from another author were of limited utility, being only in abstract form. 84 Similarly, incomplete and heterogeneous reporting the analysis of the final CSS score involved only five studies. Finally, there were two trials with three arms, in each case an arm with no HS, a 3% HS arm and an arm with a higher concentration of HS, either 5% (Al-Ansari) or 6% (Teunissen). We chose to compare both doses against control rather than combine them into one arm, since this might have helped in understanding and explaining some of the heterogeneity – although it seems unlikely that concentration of HS alone would explain all of the heterogeneity.

The synthesis of the systematic review’s primary outcome (LoS) is not in itself straightforward. The included studies were conducted in different countries and health-care settings with local guidelines on the assessment of infants as fit for discharge. No study reported on whether or not investigators adhered to these guidelines. We have no way of telling whether or not the difference between fitness for discharge and actual discharge (caused by the time and day of, consultant availability, etc.) was even across study arms, with only one study acknowledging this as a limitation. 23 Certainly, the definition of discharge fitness criteria differed across the trials – as indeed did the starting point, being sometimes from admission, sometimes from randomisation and sometimes from treatment start.

Of the 14 studies in the review, two of the studies reported hospital readmission rate. As previously discussed, this is an outcome of clinical relevance, certainly in the UK, as suggested by SIGN4 and NICE, but only two of the trials (both of which were non-UK based studies) reported on this outcome, which has potential health and economic implications. The Outcome Reporting Bias In Trials (ORBIT) classification (see Table 25) showed that the majority of readmission rates were classified as I (no risk of outcome bias), as this was potentially not considered important by trials included. However, lack of readmission data may mean that although the treatment is effective in the short term, patients are being readmitted and, therefore, the intervention may not be as economically beneficial as the summary result suggests. AEs may also be omitted by this scenario.

Overall, the defining feature of this systematic review is its heterogeneity, which limits any attempt to synthesise the evidence. The dissimilarity of the SDs of LoS indicates that the studies have taken place in very different settings. Two trials (n = 9679 and n = 11280) reported SDs of less than 1 day, indicating that their patients were typically discharged within a reasonably short time of each other. At the other extreme, the SABRE study and the other large northern European study86 reported SDs of over 3 days. This has consequences for the statistical analysis, as larger SDs lead to less study weight. The SABRE study87 contributed less than 4% of the total weight, despite being the largest of the 15 studies in the meta-analysis, and the Teunissen et al. study86 (n = 247) contributed 7.1% of the weight. The fact that large trials are contributing so little weight is counterintuitive and makes any overall summary of the 15 trials highly contentious. Of the possible sources of heterogeneity, the background care (in particular, concomitant medications) seems a likely explanation for at least some of these differences. In view of the number of ways by which the studies differed, however, we have not undertaken any formal meta-regression to attempt to explain this.

The question of whether to favour a FE or RE approach for the analysis – and for meta-analyses in general – is a controversial and unresolved issue. 92,127,128 The FE assumes a common underlying effect of treatment, which is highly questionable in the presence of heterogeneity. 92 The RE approach reweights studies to incorporate both the precision of the individual trial and the variability among trials into the weighting, but as a consequence all studies receive similar weights when between-trial variability is excessive and unexplained. 129 Neither approach seems satisfactory for situations such as this, and indeed we recommend against using an overall average however calculated. Rather, we suggest that the reader consider the specific setting within which each study was undertaken and apply findings only of those which appear consistent with their own setting.

Doing so, our recommendation for the UK NHS setting is unequivocal: hospital stay is not affected by adding HS to the care package. Although findings varied across the trials identified by the systematic review, the trials carried out in systems most comparable to the UK demonstrated almost zero difference between HS and the control treatments. The findings within our own trial were consistent across the 10 centres that recruited into the trial. Patients testing positive for RSV tended to remain in hospital longer than those testing negative, but HS did not appear to act differentially between the two subgroups.

The SABRE study

This study was designed to understand the effects of regular nebulised 3% HS on reducing the time until an infant is ready for discharge and does not address another disputed and key question. We were precluded from assessing whether or not 3% HS provides short-term temporary symptomatic relief due to the lack of blinding and the subjective nature of current scoring systems; previous studies have shown a correlation between scores produced with, for example, the Wang score and objective physiological assessments such as those obtained with pulse oximetry. 78

The results presented in the SABRE study appear to conflict with the conclusions of a Cochrane review and the update to it presented herein. 25 The review reported a significant reduction in the LoS of patients admitted with acute bronchiolitis. 25 Of the five studies included, two were conducted by the same group (but with different patients), with Tal et al. 23 (n = 41) being an extension of the original study from Mandelberg et al. 22 (n = 52). The same centre has also published a positive study in a group of ambulatory subjects. Two studies followed from China,79,80 using 0.9% or 3% saline to deliver salbutamol in ‘mild to moderate subjects’ and 0.9% or 3% saline apparently without salbutamol in more severe patients. The severity of entry was assessed using the Wang score but there is no report of saturations, need for oxygen therapy or other assessments of severity. 78–80 Although the LoS was reduced in the group treated with 3% HS (twice hourly for three doses, four times hourly for five doses, then six times hourly) the mild to moderate group had a significantly longer stay than the moderate to severe group (7.4 days vs. 6.4 days in the 0.9% HS treated groups). 79,80 It is of interest to see how many studies use the Wang scoring system despite the conclusion in the abstract from the Wang et al. publication that ‘the limited agreement for clinical signs makes comparison of patient illness severity between studies difficult’. 78 This is one of the reasons that we did not use scoring systems, as they are not validated and almost certainly generate fewer robust data than, for example, the saturation recorded in room air and duration of oxygen therapy. One study from Italy reported a decrease in LoS when 0.9% or 3% HS was used to nebulise adrenaline but, interestingly, not a single patient was discharged prior to day 4. 77 The final study included 64 patients from the United Arab Emirates and 32 from Canada, with LoS being defined by a physician as being ready for discharge. 24

One study83 reporting a non-significant difference in LoS in those treated with 0.9%, 3% and 5% HS involving 165 patients was left out of the inpatient analysis in the Cochrane review58 for reasons that are unclear. These relatively mild patients were admitted to a short stay unit, with a short stay being defined as fewer than 8 days, so in effect they were hospitalised. 83 Two studies, which had been presented in 2011 as abstracts, that failed to identify a significant difference in LoS were also not included. One study from The Netherlands involving 160 subjects found no difference in LoS when 0.9%, 3% and 6% saline nebulised four times hourly were compared,130 while an Italian study involving 30 patients comparing 2.5 ml of 0.9% saline with 7% saline plus 0.1% hyaluronic acid at a dose of 2.5 ml twice a day for 3 days was also negative for the primary end point of LoS. 112

Since the Cochrane review25 was published, results from a number of studies have consistently found no reduction in hospitalisation rates from the use of HS. These include 250 participants from a comparison of 0.9% and 3% HS reported by Sharma from India, 82 participants from an Argentinian study comparing the same interventions and 40 participants from a second Indian study also comparing the two interventions. 74,81,82 Another study from Georgia was reported in abstract form in 2013, in which HS was associated with a reduction in LoS of half a day (albeit not statistically significant) in comparison with NS. 84 The abstract states that 42 patients were randomised, but it does not state the number in each group, or how many were included in the analysis. Since this study would account for at most 2% weight in a FE analysis, we have excluded it from this analysis. Thus, in total, 455 patients have participated in studies which have failed to show an impact on LoS, in addition to the 291 participants included in this study.

Sources of heterogeneity

In undertaking this systematic review it has become apparent that there are a number of semantic, methodological and cultural differences across the studies, all of which impact on the results obtained and the generalisability of an individual trial’s findings. We propose some of these factors and offer an explanation for how these may impact on the interpretation of the review’s findings.

Strengths and weaknesses compared with earlier systematic reviews

As the results of both the SABRE study and our systematic review and meta-analysis contradict the findings of the Cochrane review team, it is worth outlining the timeline over which these pieces of work have been conducted. The first, the 2008 version of Zhang et al. ’s Cochrane review25 included three inpatient trials that were comparable to the SABRE study: Mandelberg et al. in 2003,22 Tal et al. in 200623 and Kuzik et al. in 2007. 24 These trials suggested that HS reduced the MD in length of inpatient stay by –0.94 days (95% CI –1.48 to –0.40 days; p = 0.0006), with no heterogeneity. The SABRE study grant application was made on 8 January 2010. The Zhang et al. Cochrane review25 was assessed as up to date on 6 June 2010. The SABRE study commenced on 6 October 2011 and recruited its first participant on 26 October 2011. Although the Cochrane website does not register this, the Zhang review was updated in 201171 to contain one additional inpatient trial (a total of four trials) comparable to the SABRE study: Luo et al. 79 At this time, HS appeared to reduce the MD in length of inpatient stay by –1.16 days (95% CI –1.55 to –0.77 days; p < 0.00001), with no heterogeneity. Finally, the review was updated by Zhang et al. in 2013,141 including two additional inpatient trials, Luo et al. 201180 and Giudice et al. 201277 At this time they also included, in their review of studies recruiting in the emergency department, a study which we have classified as an inpatient trial (Al-Ansari et al. 201083). We found the following studies that could have been identified by Zhang et al. ’s searches in April, week 4, 2013:141 Espelt in 201274 and Maheshkumar et al. in January 2013. 81 In addition, the following studies were captured by our review but were published after the Zhang et al. 2013141 update: Pandit et al. in May 2013,85 Sharma et al. in 2013,82 Nemsadze et al. in 2013,84 Teunissen et al. in 2014,86 Silver in 2014,88 Ojha et al. in 2014,90 Ozdogan et al. in 2014,89 Sosa-Bustamante et al. in 201475 and Everard et al. in 2014. 87 Overall, 11 additional studies (n = 1367 participants, excluding Sosa-Bustamante et al. in 201475 as the number of patients randomised is unknown) were included in our review, eight of which were included in the meta-analysis in addition to those in the Cochrane review 2013 update. Our systematic review protocol was registered on PROSPERO on 3 March 2014 and the protocol was revised on 18 December 2014 following peer-review comments. The searches for this systematic review were run up to 14 January 2015.

The meta-analysis in the 2013 update of the Cochrane review by Zhang and colleagues contained six RCTs (n = 480) evaluating the effect of HS on length of inpatient stay, the largest of which randomised 112 participants. 58 Our meta-analysis contains 15 RCTs (n = 1922), including three much larger studies, all of which show null results. The eligibility criteria and search strategies of the two reviews differed, as a result of which our review picked up a number of publications which, we believe, are relevant to clinical decision-making and which might have been picked up the Cochrane review team’s searches in April–May 2013. With more than 10 studies, we were able to investigate the potential for publication bias. 76 Importantly, our meta-analysis also identified much higher levels of statistical heterogeneity than those reported by the Cochrane team, beyond what would be expected from the addition of this number of studies. 142 The Cochrane review provided what might have seemed a secure basis for clinical decision-making, suggesting that HS reduced inpatient stay by over 1 day (six studies, n = 480, MD –1.15 days, 95% CI –1.25 to –0.15 days; p< 0.0001, I² = 30%). Our review suggests that the evidence base is far more equivocal, heterogeneous and, with publication and other biases at work, unsafe as a platform for decision-making (15 studies, n = 1922, MD –0.34 days, 95% CI –0.48 to –0.21 days; p< 0.00001, I² = 75). In the long-run, the resources and standards of Cochrane may allow non-English-language and unpublished studies not identified by, or excluded from, our study to be incorporated into a research synthesis. Nonetheless, these salient differences, observed between closely contemporary reviews, confirm that duplicate systematic reviews can sometimes be valuable143,144 rather than wasteful. 145

Writing group

Mark L Everard, Daniel Hind, Kelechi Ugonna, Jennifer Freeman, Mike Bradburn, Simon Dixon, Chin Maguire, Hannah Cantrill, John Alexander, Warren Lenney, Paul McNamara, Heather Elphick, Philip AJ Chetcuti, Eduardo F Moya, Colin Powell, Jonathan P Garside, Lavleen Kumar Chadha, Matthew Kurian, Ravinderjit S Lehal, Peter I MacFarlane, Cindy L Cooper and Elizabeth Cross.

Trial management group

Daniel Hind, Elizabeth Cross, Colin O’Keeffe (trial manager, University of Sheffield), Kirsty Sprange (trial manager, University of Sheffield), Chin Maguire, Jennifer Freeman, Mike Bradburn, Simon Dixon, Mark L Everard, Kelechi Ugonna, Heather Elphick, Sarah Shortland (Sheffield Children’s NHS Foundation Trust, Doncaster and Bassetlaw Hospitals NHS Foundation Trust, Rotherham NHS Foundation Trust), Wendy Swann (Sponsor Representative, Sheffield Children’s NHS Foundation Trust), John Alexander, Kate Leech (University Hospital of North Staffordshire NHS Trust), Joanne Tomlinson (University Hospital of North Staffordshire NHS Trust), Marie Phipps (University Hospital of North Staffordshire NHS Trust), Ruth Jones (University Hospital of North Staffordshire NHS Trust), Eric Roe (University Hospital of North Staffordshire NHS Trust), Helen Parker (University Hospital of North Staffordshire NHS Trust); Philip AJ Chetcuti, Louise Blackburn (Leeds Teaching Hospitals NHS Trust), Rebecca Mottram(Leeds Teaching Hospitals NHS Trust), Marjorie Allen (Leeds Teaching Hospitals NHS Trust), Nicola Balatoni (Leeds Teaching Hospitals NHS Trust), Eduardo F Moya, Kelly Young (Bradford Teaching Hospitals NHS Foundation Trust), Ruth Skelton (Bradford Teaching Hospitals NHS Foundation Trust), Louise Akeroyd (Bradford Teaching Hospitals NHS Foundation Trust), Heather Rostron (Bradford Teaching Hospitals NHS Foundation Trust), Lavleen Kumar Chadha, Mathew Kurian, Janet Queen (Doncaster and Bassetlaw Hospitals NHS Foundation Trust), Jonathan P Garside, Rachel Swingler (Calderdale and Huddersfield NHS Foundation Trust), Colin Powell, Caroline Amphlett (University Hospital of Wales), Pauline Jones (University Hospital of Wales); Ravinderjit S Lehal, Jennifer McKenna, Louise Willis (Oxford University Hospitals NHS Trust), Rebecca Beckley (Oxford University Hospitals NHS Trust); and Paul McNamara, Ian Sinha (Alder Hey Children’s NHS Foundation Trust), Gail Wallace (Alder Hey Children’s NHS Foundation Trust), Hannah Leyland (Alder Hey Children’s NHS Foundation Trust), Timothy Henderson (Alder Hey Children’s NHS Foundation Trust), Karen Phelan (Alder Hey Children’s NHS Foundation Trust) and Peter I MacFarlane.

Trial Steering Committee

Jonathan Grigg (independent chairperson, Barts and The London School of Medicine and Dentistry), Eirini Koutoumanou (independent statistician, University College London), Jeremy Hull (independent clinician, Oxford University Hospitals NHS Trust), Penny Broadley (independent parent representative, Sheffield Children’s NHS Foundation Trust), Mark L Everard (chief investigator), Kelechi Ugonna (chief investigator), Elizabeth Cross, Colin O’Keeffe, Kirsty Sprange, Wendy Swann, Alexa Cross (HTA representative) and Jennifer Freeman (trial statistician).

Data Monitoring and Ethics Committee

Tim Cole (chairperson, University College London), Neil Gibson (Royal Hospital for Sick Children, Glasgow) and Jayesh Bhatt (Nottingham University Hospitals NHS Trust).

Co-applicants

Mark L Everard, Cindy L Cooper, Simon Dixon, Jennifer Freeman, Warren Lenney, Philip AJ Chetcuti, Eduardo F Moya, Matthew Kurian, Jonathan P Garside, Peter I MacFarlane, Heather Elphick and Colin Powell.

Local investigators

Mark L Everard, Kelechi Ugonna, Heather Elphick, John Alexander, Philip AJ Chetcuti, Eduardo F Moya, Lavleen Kumar Chadha, Matthew Kurian, Jonathan P Garside, Colin Powell, Ravinderjit S Lehal, Paul McNamara and Peter I MacFarlane.

Study background

The SABRE study is a parallel-group, RCT comparing whether or not the addition of 3% HS to usual supportive care results in an improvement in outcomes for infants admitted to hospital with acute bronchiolitis. It is funded by National Institute for Health Research HTA programme and sponsored by the Sheffield Children’s Hospital Trust.

The trial has one main objective and three secondary objectives.

Outcome measures

Sample size

Based on a current time to discharge criteria of around 3 days, a 25% reduction was considered to be the minimum clinically significant effect. This was thought to be a realistic effect, as it is the magnitude of the effect observed in previous studies. 47 Using Hospital Episode Statistics data relating to LoS, and assuming a log-normal distribution, the SD is estimated at 32 hours. While a similar, or smaller, SD is expected for our primary outcome measure, a slightly inflated SD of 46 hours is used because of uncertainties over its derivation here. In order to have 90% power to detect a 25% difference in time to meeting discharge criteria, the study will need 139 patients per group at a two-sided alpha level of 5%. The dropout rate is thought to be negligible for the analysis of the primary outcome measure and therefore a conservative estimate of sample size is 150 patients per group. It is expected that up to eight centres will recruit patients into the study. Assuming that all eight recruit a similar number of patients, each site would be expected to recruit approximately 38 patients.

Randomisation

A central web-based randomisation service delivered by the CTRU will be used after patient eligibility and written consent have been confirmed. Patients will be randomly allocated via the online system to receive either (1) HS with usual care (n = 150) or (2) usual care (n = 150). The randomisation schedule will be computer generated prior to the study by the CTRU randomisation service. The allocation schedule will be concealed through the use of the centralised web-based randomisation service, which also allows unblinding in the case of emergency. The randomisation sequence will not be revealed to any person involved in patient recruitment. The data analysts will be blind to treatment allocation until after the statistical analysis plan is finalised, the database locked and the data review completed. All unblinding (emergency and end of trial) will be automatically logged by the CTRU randomisation system, which will include the date, time and user responsible.

Blinding

The study will compare the addition of the intervention to usual care without placebo. The use of placebo in this setting is problematic in that this would involve an intervention that may have a significant effect on the outcome for subjects in the ‘placebo’ arm. The extra handling involved in nebulised therapy may have a deleterious effect, as has been shown to be associated with the increased handling associated with physiotherapy. 49 Similarly, the ‘placebo’ agent may cause harm as has been suggested in previous studies using nebulised distilled water or may have an unexpected positive impact for an agent such as a NS. 32

Interim analyses

As the study was designed to be of short duration with the data collection period anticipated to last for only a single bronchiolitis season, no interim analyses were planned. However, poor recruitment during the first season resulted in a single interim analysis at the end of this first year, resulting in a decision to continue with the trial and a small reduction in the significance threshold as detailed in Level of significance.

Study monitoring

The trial will be monitored and audited in accordance with the monitoring standard operating procedures of the clinical research facility and the CTRU. Three committees will be established to govern the conduct of the trial:

1. TSC. The TSC will consist of an independent chair, the chief investigator, two independent members (statistician and clinician) and a parent representative.

2. DMEC. The DMEC will consist of an independent statistician and two independent experts. Its responsibilities will include reviewing the trial data at regular intervals and implementing stopping rules in accordance with MRC guidance. 151

3. TMG. The TMG will include the chief investigator, co-applicants, principal investigators, CTRU assistant director, trial manager, research nurses, trial statistician and health economist.

Data sources

The data used in this study will come from the following sources:

Protocol deviations

Patients who are deemed not to have adhered to the study procedures will be considered as deviations from the protocol. Deviations will be classified as either major or minor depending on their importance, and secondary analyses may be performed excluding patients who are considered to be major protocol deviations. These analyses are termed ‘per-protocol’ analyses, and the set of patients who are included in these analyses will be termed the PP analysis set. The definition of protocol deviations will be agreed and finalised prior to final database lock. In deciding upon protocol deviation, particular weight will be given to the following:

• randomised despite not fitting the entry criteria

• not receiving randomised treatment

• receiving concomitant medications during treatment stage.

Analysis populations

Four analysis populations will be used in the analyses:

1. All screened patients: patients who are screened for eligibility to the study, including those randomised.

2. FAS: all randomised patients, with the following exclusions:

   1. patients who have previously been randomised. Where this occurs, only data from the first admission will be analysed.

   2. patients from whom there is no recorded informed consent obtained from carers (oral or written).

   3. patients whose carers withdraw consent before any study medications are given.

   4. patients whose carers withdraw consent retrospectively (i.e. request that their entire data are removed).

   5. patients who were randomised despite not fulfilling the inclusion criteria.

3. Per-protocol (PP): the subset of the FAS who do not deviate from the protocol, as defined in Protocol deviations.

4. Safety: all randomised patients, with the exception of those who have no recorded informed consent.

Summaries based on the FAS and PP populations will be on an intention-to-treat basis with patients assigned to the treatment group as originally randomised. Summaries based on the safety population will analyse patients by the actual treatment received.

For the outputs presented to the DMEC, all outputs will be based on the FAS population unless otherwise requested.

General considerations

Data completeness

The following summary will be presented for all screened patients by centre and overall:

• Enrolment: the number of days recruiting, the number of patients screened, the number of patients recruited, the number of patients recruited per day, the number of screened patients not recruited, and the reason for non-recruitment.

The following summary will be presented for the FAS:

• Data completeness

  • by treatment group: the number of patients with complete data for key parameters

  • by centre: the number of patients with complete data for key parameters

• Protocol deviations

  • By treatment group: number of patients who deviate from the protocol

  • By centre: the number of patients who deviate from the protocol

The inclusion of key parameters may be allowed to vary at the request of the DMEC during the trial. In order to allow time for data to be entered onto the system, the follow-up and questionnaire data will be considered complete if they have been entered within 30 days of the scheduled data collection time point.

Missing, spurious and unused data

Questionable data will be queried with the study manager and referred back to the site investigators where appropriate.

Demographics, baseline characteristics and concomitant medications

The following summaries will be presented:

• demographics: centre, age, sex, ethnicity, age at admission, gestation, birth weight, current weight

• background: smoker in household, child-care method, feeding method, number of siblings, family history of asthma, eczema and hay fever, previous illnesses, medical problems, length of current illness prior to admission

• concomitant medications given by time period:

  • in the 2 weeks prior to admission

  • during admission.

Efficacy

The following summaries will be presented:

• time to being declared fit for discharge

• actual time to discharge

• percentage transferred to HDU/ICU

• percentage readmitted within 28 days from randomisation

• time from randomisation to resolution of respiratory symptoms

• health-care utilisation, post discharge and up to 28 days from randomisation.

Safety outcomes

The following summaries will be based on the FAS population:

• study medication usage

• concomitant medications

• AEs, overall and by type

• SAEs, overall and by type

• side effects, overall and by type

• observations during trial treatment (supplemental oxygen, feeding method).

In summaries of AEs and medications, the number of patients experiencing any event will be presented by actual treatment received; patients will be counted only once in each row, even if more than one event is noted. Where no event is recorded the patient will be defined as having not experienced an event.

The following listings will also be provided. Note that each will contain the treatment, patient ID, event details, date/time and time to onset. For AEs the seriousness will also be reported. For SAEs the duration, expectedness, relationship to study drug, frequency, intensity, action taken and outcome will all be reported:

• all AEs

• all SAEs

• all side effects.

Subgroup analysis

The study will investigate outcomes between those infants with human RSV infection and those with acute bronchiolitis due to one of the many other respiratory viruses that can cause the same clinical phenotype (non-RSV). There is currently no evidence about whether the response to treatment would differ between these two patient groups and if so what the probable magnitude might be. While the study has not been powered to investigate this, it is of interest to investigate and thus the coefficient for the interaction between infection group (RSV vs. non-RSV) and treatment group will be presented together with its 95% CIs and p-value. In addition, results will also be reported separately for the RSV and non-RSV groups in order to provide information for a power calculation for a future study, investigating the relative efficacy of nebulised HS for RSV compared with non-RSV patients.

Disposition and data completeness

Efficacy analyses

Trial documents

1. Trial protocol (V0.10, 28 June 2011).

2. Data collection forms:

   1. Eligibility form (V1.0, 3 August 2011)

   2. Baseline information form (V1.0, 3 August 2011)

   3. Treatment data (V1.0 3 August 2011)

   4. Discharge form (V1.0, 10 August 2011)

   5. Follow-up forms including symptom diary (V1.0, 10 August 2011)

   6. Study completion (V1.0, 3 August 2011)

   7. AE recording form (V1.0, 3 August 2011)

   8. 1-month questionnaire (V1.0, 3 August 2011).

Standard operating procedures

1. Statistician input and the statistical analysis plan (Shef/CTRU/ST001, version 1.1, 29 October 2007).

2. Data evaluability (Shef/CTRU/ST003, version 1.0, 15 October 2007).

Chin Maguire, Hannah Cantrill

Search strategy

1. Bronchiolitis/ (2310 results)

2. Bronchiolitis.mp. [mp = title, abstract, original title, name of substance word, subject heading word, keyword heading word, protocol supplementary concept, rare disease supplementary concept, unique identifier] (9842 results)

3. 1 OR 2 (9842 results)

4. Humans/ (13,622,085 results)

5. Humans.mp. [mp = title, abstract, original title, name of substance word, subject heading word, keyword heading word, protocol supplementary concept, rare disease supplementary concept, unique identifier] (13,701,574 results)

6. 4 OR 5 (13,701,574 results)

7. 3 AND 6 (8487 results)

8. Aerosols/ or “Nebulizers and Vaporizers”/ (30,740 results)

9. nebulize$.mp. [mp = title, abstract, original title, name of substance word, subject heading word, keyword heading word, protocol supplementary concept, rare disease supplementary concept, unique identifier] (11,059 results)

10. 8 OR 9 (33,444 results)

11. 7 AND 10 (238 results)

12. Sodium Chloride/ or Saline Solution, Hypertonic/ (54,351 results)

13. Hypertonic saline.mp. [mp = title, abstract, original title, name of substance word, subject heading word, keyword heading word, protocol supplementary concept, rare disease supplementary concept, unique identifier] (4987 results)

14. 12 OR 13 (56,498 results)

15. 11 AND 14 (43 results)

Search strategy

1. Bronchiolitis/ (9588 results)

2. Bronchiolitis.tw. (11,033 results)

3. 1 OR 2 (14,695 results)

4. Human/ (15,229,989 results)

5. Human.mp. [mp = title, abstract, subject headings, heading word, drug trade name, original title, device manufacturer, drug manufacturer, device trade name, keyword] (15981086 results)

6. 4 OR 5 (15,981,086 results)

7. 3 AND 6 (12,665 results)

8. Aerosol/ or nebulizer/ or nebulization/ (50,225 results)

9. Nebulize$.mp. [mp = title, abstract, subject headings, heading word, drug trade name, original title, device manufacturer, drug manufacturer, device trade name, keyword] (11,285 results)

10. 8 OR 9 (53,556 results)

11. 7 AND 10 (413 results)

12. Sodium chloride/ (134,876 results)

13. Hypertonic saline.mp [mp = title, abstract, subject headings, heading word, drug trade name, original title, device manufacturer, drug manufacturer, device trade name, keyword] (6139 results)

14. 12 OR 13 (136,836 results)

15. 11 AND 14 (102 results)

Search strategy

1. Medical subject heading (MeSH) descriptor: [Bronchiolitis] (279 results)

2. Bronchiolitis (749 results)

3. 1 OR 2 (750 results)

4. MeSH descriptor: [Humans] (1100 results)

5. Human (504536 results)

6. 4 OR 5 (504536 results)

7. 3 AND 6 (444 results)

8. MeSH descriptor: [Nebulizers and Vaporizers] (1889 results)

9. Nebulize$ (13 results)

10. MeSH descriptor: [Aerosols] (2107 results)

11. 8 OR 9 OR 10 (3713 results)

12. 7 AND 11 (78 results)

13. MeSH descriptor: [Saline Solution, Hypertonic] (404 results)

14. Hypertonic saline (949 results)

15. 13 OR 14 (949 results)

16. 12 AND 15 (18 results)