PLEASANT: Preventing and Lessening Exacerbations of Asthma in School-age children Associated with a New Term a cluster randomised controlled trial and economic evaluation

Article history

The research reported in this issue of the journal was funded by the HTA programme as project number 11/01/10. The contractual start date was in January 2013. The draft report began editorial review in February 2016 and was accepted for publication in June 2016. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The HTA editors and publisher have tried to ensure the accuracy of the authors’ report and would like to thank the reviewers for their constructive comments on the draft document. However, they do not accept liability for damages or losses arising from material published in this report.

Declared competing interests of authors

Jennifer Campbell, Rachael Williams and Robin May are employees of Clinical Practice Research Datalink who received payment from the University of Sheffield during the conduct of the study and funding from multiple organisations outside the submitted work.

Copyright statement

© Queen’s Printer and Controller of HMSO 2016. This work was produced by Julious et al. under the terms of a commissioning contract issued by the Secretary of State for Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science Park, Southampton SO16 7NS, UK.

Research aims and objectives

The aim of the study was to assess if a NHS-delivered public health intervention (a letter sent from the GP to parents/carers of school-aged children with asthma) can reduce the number of unscheduled medical contacts after the school return.

The primary objective of the study was to assess whether or not the intervention reduces the September peak in unscheduled medical contacts.

Ethics approval and research governance

Ethics approval for the study was given by South Yorkshire Research Ethics Committee on 25 October 2012 (reference number 12/YH/04). NHS permissions to conduct the study were obtained for all the primary care trusts in England and health boards in Wales.

The trial was registered with the International Standard Randomised Controlled Trial Number ISRCTN03000938.

Trial design

The study was a cluster randomised trial17 to assess if a letter sent by a GP to the parents/carers of school-aged children with asthma, reminding them to take their medication, reduces the number of unscheduled medical contacts after return to school following summer holiday in September. The unit of cluster was general practices; site recruitment commenced in January 2013, with the intervention being delivered during the week commencing 29 July 2013. Data for the trial were collected via the CPRD.

Clinical Practice Research Datalink

The CPRD GOLD is the world’s largest validated computerised database of anonymised longitudinal medical records for primary care. 18 Records are derived from GP software systems and contain complete prescribing and coded diagnostic and clinical information, as well as information on tests requested, laboratory results and referrals made at or following on from each consultation. 19

The CPRD is thus able to capture all medical contacts, from prescription request through to out-of-hours contacts, along with the reason for the contact. This therefore negated the need to request this information from the individual GP practices.

Settings and locations where the data were collected

The setting was primary care, with the unit of cluster being general practices. Site eligibility required practices to be using the Vision IT software [INPS (In Practice Systems), London, UK] and be part of CPRD. Site recruitment was conducted by CPRD and the National Institute for Health Research (NIHR) Primary Care Research Network.

Clinical Practice Research Datalink recruitment

Practice recruitment was carried out predominantly by CPRD. A practice recruitment pack, consisting of a detailed study information sheet and an expression of interest (EoI) form, was sent to all 433 practices contributing to CPRD in England and Wales at the time of recruitment. This was sent by post to the preferred contact at the practice as specified in CPRD’s records for the practice. Non-responding practices were sent a reminder e-mail, followed by a second reminder e-mail and then final reminders by e-mail and post. In addition to this, some practices were contacted by telephone, either by CPRD or by members of the study team at the Sheffield Clinical Trials Research Unit (CTRU).

Practices that wanted to take part in the study, or to decline participation, returned the completed EoI form, confirming or updating as necessary the information about the practice held by CPRD. Responses were tracked by CPRD to ensure practices that had expressed interest or declined to participate were not contacted again. The EoIs were then forwarded to the study team to contact practices and complete site set-up.

The details of the recruitment processes and contacts required to enrol GP practices have been published. 20

National Institute for Health Research Primary Care Research Network

The Primary Care Research Network also advertised and invited recruitment to the trial. Eligibility criteria for general practices were that they were using Vision IT software and agree to be signed up to CPRD if they were not already. Completed EoIs were returned to the study team for follow-up and site set-up.

Site set-up

The study team contacted interested practices to complete site set-up; this was done via telephone or Skype™ (Microsoft Corporation, Redmond, WA, USA). Once set-up was complete and verbal consent obtained from the practice to participate, practice details were forwarded to the study statistician for randomisation. See Randomisation and blinding for details of randomisation.

Practices randomised to intervention group were sent GP packs that included the intervention letter template, which was added to practice headed paper, with procedures on confirming patient eligibility and instructions on the process and timing for delivery of the intervention via DocMail (www.cfhdocmail.com/; accessed 18 January 2016; CFH Docmail Ltd, Radstock, UK).

Practices randomised to the control group were to continue with care as usual; no other activity was required.

Participants and eligibility criteria

Participants were school-aged children with asthma, aged between 4 and 16 years, who were registered with a GP.

Trial intervention

Sites were randomly allocated to:

1. intervention group: sending out the letter

2. control group: standard care (no letter).

For the intervention, a letter sent from a GP to the parents/carers of children with asthma reminding them to maintain their children’s medication and collect a prescription if they are running low. It also advised that, should their child have stopped their medication, it should be resumed as soon as possible (see Appendix 1).

The letter template was developed based on standard letters already used in general practice. The wording of the letter had input from the study team, which includes a GP, health psychologist and consultant respiratory paediatrician, and was also discussed in detail at two patient and public events that included school-aged children with asthma and their parents. 21

The intervention letters were sent out during the week commencing 29 July 2013 to obviate the distraction of planning for family holidays and yet left enough time for parents and children to renew prescriptions and gain benefit from the medication. The timing of the letter was decided following discussion with the patient and public involvement (PPI) group. 21

Outcomes

The effectiveness of the intervention was assessed on the basis of prescription uptake prior to the school term and medical contacts thereafter. Analyses of medical contacts were defined in four overlapping time intervals:

1. September 2013 (the primary study period)

2. September to December 2013 (the extended study period)

3. September 2013 to August 2014 (the 12-month study period)

4. September 2014 (the echo substudy).

The primary study period spans from 1 to 30 September 2013, as this was the period when, prior to the start of the study, the intervention was considered most likely be able to demonstrate an impact. The extended study period was from 1 September to 31 December 2013, since asthma-related appointments are more frequent in these months. The full follow-up period was 12 calendar months from 1 September 2013 to 31 August 2014. There was also an echo (or follow-on) study period in September 2014 to see if the effect from September 2013 was maintained when there was no actual study intervention.

Prescription uptake and scheduled medical contacts were evaluated during three periods:

1. August 2013

2. August 2013 to July 2014

3. August 2014 (the echo substudy).

The health economic analyses were based on a 12-month period from 1 August 2013 to 31 July 2014. The period starts 1 month earlier than the evaluation of medical contacts in order to incorporate the cost associated with delivering the intervention including any increase in prescriptions or medical contacts in response to the intervention that occurred during August 2013.

Changes to trial outcomes after the trial commenced, with reasons

An additional secondary outcome has been added to include data up to September 2014 to evaluate whether or not there was a carry-over effect into the following year. If the intervention were to increase prescription uptake and reduce unscheduled medical contacts in the original study period, it would be of interest to know whether or not the effect was repeated in the subsequent year without the need for a repeat intervention. As routinely collected data were used throughout the study, there was no cost associated with this extension. Therefore, the follow-up period was extended by 1 month. This was agreed by the Trial Steering Committee (TSC) and NIHR Health Technology Assessment (HTA) programme as a non-cost extension to the trial (see Appendix 2).

The non-cost extension also facilitated a survey of the GP practices that were in the intervention arm of the study to inform the health economic evaluation of the study by asking questions on resource use in the sending of the intervention letter. 22

A second change concerned the analysis of adherence to medication. The planned statistical analysis defined the medical possession ratio as the total number of days of prescriptions in the last year. During the study it became clear that the CPRD data did not hold sufficient data to enable this analysis. The amount of medication dispensed (e.g. the number of inhalers) is captured usually, but the amount of medication required (e.g. the number of puffs an individual is required to take) is not. This information is required to calculate the medical possession ratio. For this reason, our planned analyses of adherence were not possible.

Data collection, data extraction and methods for allocation of data

Data were extracted from the December 2013 CPRD GOLD database build. Patients who were between 4 and 16 years of age on 1 September 2013 and currently registered at a participating practice were considered for inclusion. A Read Code list, supplied by the study team at the Sheffield CTRU, was applied to identify all patients who had a diagnosis code for asthma. The population was then limited to patients who had been prescribed an asthma medication in the 12-month period of March 2012 to March 2013 using a Multilex code list. Patients who had a Read Code indicating neoplastic disease were excluded. All data for the included patients were extracted from the database. In order to protect the identity of included patients, the patient and practice identification numbers (IDs) were pseudonymised by the research team at CPRD before the data were delivered to the CTRU. Read Code lists used for the data extraction are available on request and are also on the website of the PLEASANT study at the following address: www.sheffield.ac.uk/scharr/sections/dts/ctru/pleasant/index (accessed 31 May 2016).

Subsequent data extractions were done for the same patients on the April 2014, June 2014 and January 2015 builds in order to increase the length of follow-up.

Data handling

The primary data source was anonymised, and GP and NHS contacts extracted by the CPRD and forwarded to the CTRU via a password-protected zip file.

Every NHS service contact is coded by the general practice and captured within the practice database. These codes, which include diagnostic, consultation, prescription and test result codes, were used to enable allocation to either scheduled or unscheduled contact. This allocation was carried out by presenting a summary of the consultation codes, medcode descriptions and other tables used (as described in Methods for allocation of data to scheduled/unscheduled contacts) to an independent GP adjudication panel comprising three GPs. The GP adjudication panel reviewed the data and confirmed assumptions to use in order to code contacts as scheduled, unscheduled or not applicable (not relevant). Detail and definitions used for this coding are provided in the following section (see Methods for allocation of data to scheduled/unscheduled contacts).

The age for inclusion in the study is age as of 1 September 2013. Day of birth is missing for all children in the data set so there is no patient-identifiable information. For day of birth, the value ‘15’ was used. If month of birth was also missing the value ‘September’ was used. The latter assumption was used because data provided by CPRD have been checked for meeting the eligibility criteria, and hence subjects with a missing month will be eligible with this month. All children with a missing month were born in the years 1997, 1998 and 1999. Thus assuming September, October, November or December for the missing months ensures that all subjects born in 1997 are included in the analysis.

Note that the data were provided by CPRD as anonymised to prevent the identification of the children and the practices in the study.

Given CPRD anonymisation of the practices, it was not possible to reconstruct the full disposition of some children. For example, in the present report, 5917 children are allocated to the intervention arm and 6262 to the control arm in 141 practices, which is based on the data extracted from the database for the analysis. In a different source (the original practice size listing used for the randomisation), 5907 subjects are allocated to the intervention and 6431 to the control in 142 practices (including one practice, with a size of 99, that subsequently withdrew its consent after randomisation – see Recruitment and participant flow). Owing to the blinding, it was not possible to fully describe the discrepancy in numbers of children.

A similar issue is found with subject allocation to the per-protocol (PP) population. Although it is possible to ascertain whether or not a subject is included in the PP population, because of blinding it was not possible to fully link to the reason (e.g. ‘letter not sent’ or ‘letter sent late’) without a little interpolation.

Detailed data management processes are set out in a data management plan. Data will be retained in accordance with the Data Protection Act 199823 and CTRU data management standard operating procedures (SOPs).

Methods for allocation of data to scheduled/unscheduled contacts

A scheduled contact was defined as any contact that is part of the planned care for the patient, for example an asthma review, a medical review, repeat prescription or immunisation. An unscheduled contact was defined as any contact not part of the patient’s care plan that is either patient initiated or a result of illness.

To ensure that the allocation of scheduled and unscheduled contacts was robust, a GP Adjudication Panel comprising three independent GPs attended meetings to review the data blind to treatment. The GP Adjudication Panel reviewed the unique terms (17% of the unique terms were reviewed, which accounted for 90% of the data). During these meetings the GP Adjudication Panel devised the assumptions (i.e. rules) used to allocate to scheduled, unscheduled or not applicable (irrelevant) contacts. These assumptions were documented (see Appendix 3) and approved by the GP Adjudication Panel.

All types of ‘consultation’ are recorded within the data that CPRD provides; each consultation was considered a medical contact. Not all consultations are considered relevant to the study. One ‘consultation’ in the consultation table was considered one contact. All consultation data supplied are taken into account for the study, not just those that are asthma related. Only consultations that happened on or after 1 August were included.

Assumptions used to code records as scheduled, unscheduled or not applicable were based on a GP Adjudication Panel review of the clinical, immunisation, therapy, referral, and test and consultation data.

The ‘medcode description’ from the clinical data was used first, as it was felt that this table gave most description about the reason for the consultation. If contact type could not be determined by the ‘medcode description’, then clinical consultation was referenced.

Following the GP Adjudication Panel review of the medcode descriptions and clinical consultation types, in which over 90% of the data were reviewed (17% of the unique terms), clinical records were identified to be marked as scheduled (these include asthma annual review terms and other obvious types of planned appointments), unscheduled (e.g. examinations, emergency appointments), not applicable or unknown.

The clinical data contain more than one record per consultation and the same consultation ID can have more than one clinical contact type. For these clinical records, we assumed that unscheduled takes precedence (i.e. they are likely to have come in for an unscheduled visit but had a scheduled ‘type’ of procedure at the same time). Of all the contacts coded, a small proportion (2.27%, 10,011 of 440,429) could have been defined as both, based on the assumptions made about the clinical data.

Consultation data marked as unknown, based on the clinical data, as well as consultation data that did not link to clinical data, are coded based on immunisation, therapy, referral, and test and consultation data. If at least one match was found in the immunisation record, then the data were coded as scheduled. If at least one match was found with therapy (medication) data, then the data were coded as unscheduled. If the data matched with the test data as part of the routine asthma review, then they were coded as scheduled. If they matched with test data of peak expiratory flow rate, then they were coded as unknown. Otherwise, a match with test data was coded as unscheduled.

Finally, contacts were coded on the consultation type in the consultation table, where follow-up/routine visits, repeat issue and medicine management were coded as scheduled; consultation types that indicated an emergency visit were coded as unscheduled; administrative-type consultation types were coded as not applicable; and those that were unclear, for example clinic, surgery consultation and other, were coded as unscheduled because of the likelihood that most scheduled consultation types would be clearly recorded. The process of allocation is detailed in Appendix 3.

Changes to the data collection, data extraction and methods for allocation of data after the trial commenced, with reasons

When reviewing the code lists used by CPRD to identify patients, after the study was completed and it was being reported, it became apparent that there was a discrepancy between the codes provided by the study team and the codes used by CPRD. The missing codes included some asthma, neoplasm and medication codes.

The clinicians on the Trial Management Group (TMG) reviewed the discrepancies between the codes used by CPRD and those provided by the study team. With reference to the omitted asthma codes, the clinical view was the codes were secondary codes and so there was no concern that children with asthma would have been missed. For the omitted neoplasm codes, the clinical view was that there was no concern as the codes were usually for adults and not children. However, there was a concern with the medication codes, which had been on the original code list but which were omitted from the CPRD extraction. This is of importance, as one of the inclusion criteria for the study was asthma medication prescribed in the previous 12 months. The clinical view was this may have resulted in fewer children having been identified as eligible for the trial than should have been if the correct, full list of product codes had been used. The views of the TMG, which were subsequently endorsed by the TSC, was that a further data extraction should be undertaken by the CPRD to quantify the effect of using the correct medication code list compared with the one used in error.

The CPRD undertook a review to identify the source of the problem and identified human error that was not picked up in their quality assurance. The CPRD has amended its documentation and quality assurance procedures to ensure that this type of human error is more readily identified and corrected at the time of the mistake.

In response to the request by the TSC, the set of product codes supplied by the study team was used by CPRD to identify those children with asthma who had received medication for asthma in the previous 12 months. This was done to estimate the magnitude of the impact upon the eligible patients. The CPRD conducted a post hoc analysis identifying children aged between 4 and 16 years, and still registered with a study practice, on 12 March 2013 using the April 2013 build of the CPRD GOLD database, utilising both lists of product codes. This date and database were chosen as they were the most recent versions documented to have been used for the identification of eligible patients for the trial. Owing to the human error, it is not possible to exactly extract the same data set as used in the PLEASANT study. However, it would give an estimate of the relative effect on the same size.

Using the CPRD list of medical codes, 10,753 children were identified, compared with 11,273 children identified using the full list of medical codes. This equates to an estimated 5% of children with asthma who potentially could have been in the trial but who were not. Thus, although this error has been identified, it was unlikely to have a major impact upon the trial.

The error is unfortunate, but CPRD has introduced procedures to prevent such an error from happening again.

Sample size

From previous research in the CPRD practice population, 30% of school-aged children with asthma had at least one unscheduled medical contact during the month of September. 13 We postulated that the intervention may reduce the number of children who have unscheduled medical contacts from 30% to 25% (i.e. an absolute reduction of 5%). The average practice size in the CPRD is 8294. Thus, we expected c. 100 school-aged children with asthma per practice (based on 12% of a practice being school-aged children and 11% of school-aged children having asthma). Therefore, to detect a difference of 5% with 90% power and two-sided significance level of 5%, and with an intraclass correlation (ICC) of 0.03 to account for clustering, we required 70 practices per arm. The sample size of 140 practices would equate to approximately 14,000 school-aged children with asthma.

Ukoumunne et al. 24 give estimates of ICCs for patients with respiratory symptoms in general practice. Based on the work of Ukoumunne et al., an ICC of 0.03 is a conservative estimate. The power of the study for ICCs of 0.01, 0.02, 0.03, 0.04 and 0.05 was 99.4%, 96.0%, 90.0%, 83.1% and 76.2%, respectively.

As a further sensitivity analysis, we investigated the effect of practices not sending out the letter as planned. Suppose 10 practices failed to send out the letter, these would still be included in the primary analysis under the ITT principle. However, the effect that could be observed would be reduced to 4.3%. Under the sample assumptions (ICC = 0.03, etc.), the power for the same sample size is reduced to 79.3%. This is a little under 80%, but it does demonstrate reasonable robustness to at least one deviation in the planned design.

Randomisation and blinding

Randomisation was at cluster (general practice) level, and was stratified by size of general practice (i.e. the ‘list size’), to ensure that there was an equal sample size, in terms of number of school-aged children with asthma, in each arm of the trial. The randomisation sequence was generated by the main trial statistician based within the CTRU, and allocation concealment was ensured by restricting access to the two CTRU statisticians. The randomisation was undertaken by a statistician within the CTRU, in line with a study-specific randomisation plan. Once practices had agreed to participate, their identifier and list size were forwarded to the trial statisticians for randomisation to one of the two groups (intervention or usual care). The allocation was subsequently revealed to the study manager and research assistant.

The study team were unblinded throughout the study, but had no access to data until after a statistical analysis plan was developed, and had no influence on data capture. The GP Adjudication Panels did not have access to the randomisation group when reviewing the data.

Statistical methods

Patient and public involvement

Trial oversight

Two committees were established to govern the conduct of the trial: the TMG and TSC.

All committees are governed by Sheffield CTRU SOPs. The TMG comprised the principal investigator, co-investigators and key staff within the CTRU. The role of the TMG was to implement all parts of the trial.

The TSC comprised an independent chairperson (GP), two independent members (academic GP and statistician), two lay members (parents of children with asthma), the principal investigator and key staff within the CTRU (as non-voting members). The role of the TSC was to provide supervision of the protocol; a statistical analysis plan; and to provide advice on, and monitor, the progress of the trial.

Safety assessments

The trial intervention aimed to optimise usual asthma care and improve adherence to medications already prescribed by the GP, thus reducing the potential exacerbation of asthma following return to school in September. Therefore, involvement in the trial was not expected to result in any adverse or serious adverse events arising from participation.

Any asthma complications relating to the health of the child were expected to be picked up by their GP or out-of-hours service and managed as per usual care. On advice from the TSC, no formal reporting procedures for adverse events or serious adverse events were put in place.

Practices randomised to the intervention were provided with a short template to report any incidents that they felt were related to the conduct of the trial.

Recruitment and participant flow

In total, 142 GP surgeries agreed to take part in the study (Figure 3). 20 Of these, one (a control group practice with 99 children with asthma) withdrew consent after the start of the study for the data to be extracted and stored by the CPRD (independent of the study); this practice was excluded from all analyses. In total, 70 practices (comprising 5917 individuals) were randomised to the intervention (letter) group and 71 practices (6262 individuals) to the control group.

FIGURE 3.

Participant recruitment curve.

Baseline characteristics

The descriptive statistics (age, sex and surgery size) of the 12,179 subjects included are given in Tables 1 and 2. Summaries reported are stratified by intervention type and overall.

An analysis has been undertaken on practice recruitment into the trial. For the practices recruited through CPRD, it was found that there was little difference in terms of the size of the practice. 20 It was also found that practices that have been involved in more research were more likely to be in the PLEASANT study, and that the more studies the practice had previously participated in, the greater the likelihood of entering the trial.

Number of participants and analysis subsets

Analyses were conducted using outcome data from four overlapping time periods and one baseline period. For each period, analyses were based only on practices that contributed data to the entirety of that period. In other words, if practices stopped submitting data to the CPRD before the end of a given follow-up period, they were excluded from all analyses for that time period.

Figure 4 shows the flow of subjects from the overall population (aged 4–16 years) to the main cohort (aged 5–16 years). Of the 456 practices invited, 433 were through the CPRD and 23 were through the primary care research network and joined the CPRD. 20

FIGURE 4.

The CONSORT diagram of the number of GP surgeries and individuals in the PLEASANT study. It is not a mistake that there are zero GP exclusions in the arm that did not send letters, as it is impossible for the GPs to exclude individuals from receiving letters when no one in that arm is receiving letters. a, In comparison to those in the experimental arm (n = 5917); b, in comparison to those in the control arm (n = 6262); and c, these figures include withdrawals and patients aged 4 years.

Table 3 provides the number of practices and the number of individuals aged 5–16 years (the primary analysis population) included for each time period.

Adherence to protocol

Of the 70 intervention practices, two did not send letters to any of the patients identified and four sent the intervention out late, on 6, 8, 12 and 23 August. In addition, GPs were given discretion to withhold the letter from any children they believed were unsuitable candidates; among the remaining 64 practices (5222 individuals), letters were not sent to 786 children. These individuals were included in the primary ITT analyses but were excluded from the PP analyses.

Outcomes and estimation

Primary outcome

Unscheduled medical contacts in September 2013

The proportion of individuals with at least one unscheduled contact is summarised for each of the four populations (including the aforementioned primary analysis population) in Figure 5. Overall, 2399 individuals (45.2%) in the intervention arm had at least one unscheduled medical contact, compared with 2441 (43.7%) in the control arm [adjusted odds ratio (OR) 1.09, 95% CI 0.96 to 1.25]. The actual number of contacts was similar in the two groups, but there were 81 unscheduled contacts per 100 children in each arm [adjusted incidence rate ratio (IRR) 1.02, 95% CI 0.94 to 1.12]. Restricting the analyses to the PP population gave a similar (but slightly greater) increase in the effect sizes.

FIGURE 5.

Unscheduled medical contacts in September 2013. (a) Number of children with one or more unscheduled contact; and (b) mean number of unscheduled contacts per child.

The ICC for the primary analysis was 2.6%, which was consistent with the ICC used for the sample size calculation. 29

Similar results were observed for 5- to 16-year-old children who had been prescribed preventative steroids. Among children aged under 5 years, the differences were larger, and of borderline statistical significance, with the intervention being associated with more unscheduled visits for all subgroups. In all cases the effect among the PP population was greater than that observed in the ITT population.

The percentage of children aged 5–16 years who required one or more unscheduled contact between August and December 2013 is given in Table 4. The most immediate feature is the excess unscheduled contacts in August 2013, which is out of keeping both with the following months and with the equivalent figures in the previous year. Overall, the proportion of children making an unscheduled contact was higher in 2012 than in 2013, but only August (and to a lesser extent, September) 2013 showed a pronounced difference between the groups.

TABLE 4 - Percentage of children aged 5–16 years who had at least one unscheduled contact

Month	2012 (preceding year)		2013 (intervention year)
	Intervention (%)	Control (%)	Intervention (%)	Control (%)
August	41.8	41.0	41.1	34.4
September	47.4	48.2	45.2	43.7
October	51.4	50.0	44.5	45.8
November	51.5	50.1	43.7	44.3
December	49.0	49.1	42.2	41.8

To further investigate the effect observed in Table 4, the analysis of unscheduled contacts by month was repeated (Table 5), but only for children who received preventative medication. The effects are similar to those observed in Table 4.

TABLE 5 - Percentage of children aged 5–16 years and using preventative medication who had at least one unscheduled contact

Month	2012 (preceding year)		2013 (intervention year)
	Intervention (%)	Control (%)	Intervention (%)	Control (%)
August	41.9	41.3	39.0	34.1
September	47.8	48.9	43.2	41.5
October	52.0	52.0	42.7	43.5
November	52.1	50.9	41.5	40.5
December	49.9	50.0	39.9	38.5

In both Tables 4 and 5, after the initial increase in unscheduled contacts associated with the intervention in August and September there is a fall. The fact that there seems to be a reduction in contacts after September will be discussed further in Chapter 3, Contacts in the extended post-intervention phase (September to December 2013) and in Health economic methods.

In Scheduled visits and steroid prescriptions in August 2013, in the analysis of prescription data it will be highlighted how the intervention caused an increase in the proportion of prescriptions being collected. A likely explanation for the possible increase in August and September in the letter group is patients who have not collected a prescription in a while needing to see their GP before a new prescription could be given. This could be caused by patients wishing to see their GP or by the GP requiring an appointment with the patient before a prescription is given. The excess observed in August/September would therefore be, for some patients, a level of planned care that would then be reflected in the subsequent reductions in following months. To investigate this we have the results in Table 6.

TABLE 6 - Percentage of children aged 5–16 years and using preventative medication who had at least one unscheduled contact broken down by when they had their last prescription in the 12 months prior to the start of the study

Month	Time period (%)
	August–October 2012		November 2012–January 2013		February–April 2012		May–July 2013
	Intervention (n = 385)	Control (n = 385)	Intervention (n = 645)	Control (n = 670)	Intervention (n = 738)	Control (n = 813)	Intervention (n = 1927)	Control (n = 2004)
August	29.2	24.9	30.4	28.9	38.5	34.5	51.6	45.5
September	35.5	30.8	35.9	37.3	42.1	39.7	55.2	54.3
October	31.3	33.6	33.5	41.2	37.4	41.8	58.6	56.4
November	33.4	24.2	33.0	36.7	40.0	37.7	54.1	53.0
December	30.8	25.6	34.3	34.1	36.0	36.5	53.2	50.3

The results in Table 6 are the same data as in Table 5 but broken down by when a patient last collected a prescription. There is little evidence of a difference in terms of the excess in unscheduled contacts in the letter arm in August. For children who last collected a prescription within the previous 3 months, 51.6% in the letter arm had an unscheduled contact in August, compared with 45.5% in the control arm, therefore 6.1% more children in the letter arm had a scheduled contact. For children who had collected a prescription within the previous 3–6 months, 38.5% in the letter arm had an unscheduled contact, compared with 34.5% in the control arm, which represents a 4.0% excess.

In September there does seem to be a difference in the proportion of children having an unscheduled contact according to when they last collected a prescription. If a prescription had been collected within 3 months, then in the letter arm 55.2% of children had an unscheduled contact in September, which is comparable to the 54.3% in the control arm. Conversely, if it was 3–6 months since the last prescription was collected, 42.1% of children had an unscheduled contact in the intervention arm, compared with just 39.7% in the control arm, which is an increase of 2.4%. Therefore, the effect in September seems to be greatest in children who had not collected a prescription recently.

One important thing to note is that when making the assessment of unscheduled contacts, we cannot determine whether the unscheduled contact is at the request of the patient or of the GP. This factor is important in the interpretation of the results in Tables 4–6.

Secondary outcomes

Scheduled visits and steroid prescriptions in August 2013

An objective of the PLEASANT study was that the intervention would increase the proportion of children who had a prescription in August 2013, which would follow through to an increase in medication usage and, thereby, a reduction in unscheduled medical contacts. Although the latter was not evident from the data in Primary outcome, and adherence could not be assessed, the intervention (letter) was associated with an increased uptake of prescriptions in the month of August 2013. Among children aged 5–16 years, 876 (16.5%) requested at least one prescription, compared with 703 (12.6%) in the control group (adjusted OR 1.43, 95% CI 1.24 to 1.64); the total number of prescriptions was also higher (adjusted IRR 1.31, 95% CI 1.17 to 1.48). These findings are displayed graphically in Figure 6.

FIGURE 6.

Uptake of steroid inhaler prescriptions, August 2013. (a) Number of children with one or more prescription; and (b) mean number of prescriptions.

Scheduled contacts made in August 2013 are displayed in Figure 7. The percentage of children with a scheduled medical contact was higher in the intervention group compared with the control group (see Figure 7a), but this was not statistically significant. The actual number of scheduled contacts were significantly increased in the intervention group (see Figure 7b).

FIGURE 7.

Scheduled medical contacts in August 2013. (a) Number of children with one or more scheduled contact; and (b) mean number of scheduled contacts per child.

Unscheduled medical contacts associated with respiratory diagnosis in September 2013

Unscheduled respiratory-related medical contacts are important, as these are the most sensitive outcome for determining whether or not the intervention is preventing episodes of asthma exacerbation. In absolute terms, however, these contribute only a small fraction of the total attendances (Table 7). Among the primary ITT analysis population of children aged between 5–16 years, a total of 513 subjects experienced at least one unscheduled respiratory-related contact across all practices, with slightly higher uptake in the intervention group. In the intervention group, 279 (5.3%) required at least one unscheduled respiratory related contact compared with 234 (4.2%) in the control group (adjusted OR 1.30, 95% CI 1.03 to 1.66). The percentages for other subgroups are presented in Figure 8a, and the total number in Figure 8b.

TABLE 7 - Breakdown of contact types for children aged 5–16 years (ITT population)

Allocation	Total, n	Contact
		All				Respiratory-related
		Relevant (n)	Scheduled, n (%)	Unscheduled, n (%)	Unclassified (n)	Relevant (n)	Scheduled, n (%)	Unscheduled, n (%)	Unclassified (n)
September 2013
Letter (n = 5305)	7480	5585	1306 (23.38)	4279 (76.62)	1895	682	374 (54.84)	308 (45.16)	34
No letter (n = 5586)	8400	6126	1591 (25.97)	4535 (74.03)	2274	748	492 (65.78)	256 (34.22)	39
September–December 2013
Letter (n = 5097)	30,084	21,981	5745 (26.14)	16,236 (73.86)	8103	2833	1674 (59.09)	1159 (40.91)	181
No letter (n = 5384)	32,138	24,368	6504 (26.69)	17,864 (73.31)	7770	3027	1901 (62.80)	1126 (37.20)	151
September 2013–August 2014
Letter (n = 4541)	71,126	52,330	11,089 (21.19)	41,241 (78.81)	18,796	6492	3909 (60.21)	2583 (39.79)	642
No letter (n = 4549)	72,175	54,962	12,344 (22.46)	42,618 (77.54)	17,213	6546	4134 (63.15)	2412 (36.85)	414
September 2014
Letter (n = 4411)	6191	4590	1079 (23.51)	3511 (76.49)	1601	447	255 (57.05)	192 (42.95)	32
No letter (n = 4438)	6267	4629	1144 (24.71)	3485 (75.29)	1638	486	300 (61.73)	186 (38.27)	20

FIGURE 8.

Unscheduled respiratory-related medical contacts in September 2013. (a) Number of children with one or more unscheduled respiratory-related contact; and (b) mean number of unscheduled respiratory-related contacts per child.

All medical contacts (scheduled and unscheduled) in September 2013

The total number of medical contacts (excluding those deemed irrelevant) is presented in Figure 9. In contrast to previous analyses, children in the intervention arm had fewer contacts, although, again, none of the comparisons was statistically significant. Among children aged 5–16 years, 57.8% of the intervention arm participants made one or more contact, compared with 58.4% of those in the control arm (adjusted OR 0.99, 95% CI 0.80 to 1.22). The total number of appointments per child was 1.05 in the intervention arm, compared with 1.10 in the control group (adjusted IRR 0.97, 95% CI 0.87 to 1.07). Similar findings were observed for children on preventative steroids. By contrast, an increase in contacts was observed among children under 5 years.

FIGURE 9.

Total medical contacts in September 2013. (a) Number of children with one or more contacts; and (b) mean number of contacts per child.

Contacts in the extended post-intervention phase (September–December 2013)

Data on medical contacts in the extended study period (September–December 2013) were available for 65 of the original 70 intervention practices and 67 of the 71 control practices. The results are presented in Figures 10–12. In some subgroups, a statistically significant excess of contacts was observed in the intervention group, although it should be noted that this could be because of the multiple outcomes and hypotheses tested.

FIGURE 10.

Unscheduled medical contacts in the period September–December 2013. (a) Number of children with one or more unscheduled contact; and (b) mean number of unscheduled contacts per child.

FIGURE 11.

Unscheduled medical contacts associated with a respiratory diagnosis in the period September–December 2013. (a) Number of children with one or more unscheduled respiratory-related contact; and (b) mean number of unscheduled respiratory-related contacts per child.

FIGURE 12.

All medical contacts in September–December 2013. (a) Number of children with one or more contact; and (b) mean number of contacts per child.

The total number of unscheduled contacts is of particular relevance in the period September to December 2013. This includes the months (October to December) when the intervention arm seemed to reduce the number of contacts. It is this time interval that forms part of the health economic analysis, which analyses the total number of contacts rather than the percentage of children who required one.

The total number of contacts declined over the period from September to December. Although unscheduled respiratory-related contacts demonstrated a slight increase, the proportion of children aged 5–16 years requiring any medical contact remained higher in the intervention arm (although not statistically significant) for unscheduled contacts (see Figure 10a), unscheduled respiratory contacts (see Figure 11a) and all contacts (see Figure 12a). The overall number of contacts and the number of unscheduled was slightly reduced in the intervention arm (see Figures 10b and 12b) for children aged 5–16 years, but not those aged under 5 years, for whom the number of unscheduled respiratory contacts was also higher (see Figure 11b). However, these differences were, generally, not statistically significant.

Contacts over 12 months (September 2013–August 2014)

Data on medical contacts in September 2013–August 2014 were available for 58 intervention practices and 54 control practices. The results are presented in Figures 13–15. The differences in percentages between the intervention and control groups were generally modest and not statistically significant on the ITT population, and differed according to the subgroups. For the primary population (the ITT among 5- to 16-year-olds), the number of unscheduled contacts was similar (see Figure 13) and respiratory contacts remained higher (see Figure 14), but overall, contacts were reduced (see Figure 15). The total number of contacts over the 12-month period was 11.5 per child in intervention group, compared with 12.1 in the control group, which equated to a 5% reduction overall (adjusted IRR 0.95, 95% CI 0.91 to 0.99). This analysis is particularly relevant to the economic analyses in the following section, which primarily considered the overall difference in resource costs between the groups and was largely based on this time period.

FIGURE 13.

Unscheduled medical contacts from September 2013 to August 2014. (a) Number of children with one or more unscheduled contact; and (b) mean number of unscheduled contacts per child.

FIGURE 14.

Unscheduled medical contacts associated with a respiratory diagnosis from September 2013 to August 2014. (a) Number of children with one or more unscheduled respiratory contact; and (b) mean number of unscheduled respiratory contacts per child.

FIGURE 15.

All medical contacts from September 2013 to August 2014. (a) Number of children with one or more contact; and (b) mean number of contacts per child.

Echo substudy

The protocol was amended to include additional outcomes for the subsequent year. We refer to this as the ‘echo substudy’, the rationale of which was to assess whether or not any immediate intervention effect in 2013 was echoed the following year. A total of 110 practices (57 intervention and 53 control) contributed data to this time period.

In a survey of the practices in the intervention it was found that, of those that responded, 54% (13 out of 24 responding practices) had sent out the intervention again in 2014. 22 This would also contribute to an echo effect

Steroid prescriptions and scheduled contacts in the echo substudy (August 2014)

Although the increase in prescriptions found in 2013 was not as marked in 2014, the under-fives subgroup (i.e. children who were now aged under 6 years) did demonstrate an increase overall in terms of prescription uptake. These findings are displayed graphically in Figure 16. Unexpectedly, the proportion of children making at least one scheduled contact was lower in the intervention arm (Figure 17a). However, this association disappeared when evaluating the total number of scheduled medical contacts (see Figure 17b).

FIGURE 16.

Uptake of steroid inhaler prescriptions in August 2014. (a) Number of children with one or more prescriptions; and (b) mean number of prescriptions.

FIGURE 17.

Scheduled contacts in August 2014. (a) Number of children with one or more contact; and (b) number of contacts per child.

Contacts in the echo substudy (September 2014)

The findings were similar to those of September 2013. Unscheduled contacts, unscheduled respiratory-related contacts and scheduled contacts were all marginally higher in the intervention group (Figures 18–20), although the size of the difference was more modest than that observed in the previous year.

FIGURE 18.

Unscheduled medical contacts in September 2014. (a) Number of children with one or more unscheduled contact; and (b) mean number of unscheduled contacts per child.

FIGURE 19.

Unscheduled respiratory-related medical contacts in September 2014. (a) Number of children with one or more unscheduled contact; and (b) mean number of unscheduled contacts per child.

FIGURE 20.

All medical contacts in September 2014. (a) Number of children with one or more contact; and (b) mean number of contacts per child.

Time to first unscheduled contact

The time to first unscheduled and respiratory-related unscheduled contacts are presented in Figure 21 (September 2013) and Figure 22 (September–December 2013). Consistent with the number of contacts as previously demonstrated, the intervention group tended to make their first contact earlier than the control group. As the majority of contacts were unscheduled, the time to first contact of any type was similar to the time to first unscheduled contact.

FIGURE 21.

Time to first contact in September 2013. (a) First unscheduled contact; and (b) first respiratory-related unscheduled contact.

FIGURE 22.

Time to first contact in September–December 2013. (a) First unscheduled contact; and (b) first respiratory-related unscheduled contact.

Health economic methods

Health economic results

Summary of key results from the cost-effectiveness, sensitivity and subgroup analyses

The cost-effectiveness analysis found that there is considerable uncertainty regarding the impact of the letter intervention on both benefits to patients and costs falling on the NHS. Although the adjusted base-case analysis showed a mean cost saving of £36.07 per patient and a mean QALY loss of 0.00017, the CIs around each were wide and did not demonstrate significantly significant differences. The differences in costs and QALYs from the bootstrapped analysis can also be visually interpreted from the cost-effectiveness planes for the unadjusted and adjusted main analyses as presented in Figures 23 and 24, respectively. While the intervention was cost-saving in 96.3% of samples, it also resulted in a QALY loss in 82.9% of samples (Table 23 and Figure 24). Overall, the intervention was found to be cost-effective in 93.8% of samples when valuing a QALY at £20,000 (see Table 23 and the cost-effectiveness acceptability curve for the unadjusted and adjusted main analysis in Figure 25).

FIGURE 23.

Cost-effectiveness plane for the letter intervention vs. no letter from the main analysis.

FIGURE 24.

Cost-effectiveness plane for the letter intervention vs. no letter from the BA main analysis.

TABLE 23 - Mean (SE) ICER, percentage of ICERs in each quadrant of the cost-effectiveness plane, and probability of cost-effectiveness for main analysis, adjusted analysis and sensitivity analysis

Cost-effectiveness plane quadrants are south-east (less costly, more effective), south-west (less costly, less effective), north-east (more costly, more effective) and north-west (more costly, less effective).

Incremental results are the letter group (1) minus the no-letter group (0).

Negative ICERs can indicate an intervention is either dominant (mean ICER in south-east quadrant) or dominated (mean ICER in north-west quadrant); negative ICERs are not reported, but are instead replaced with a definition of the intervention being dominant or dominated by the control.

Analysis (1–0)a	Mean (SE) ICERb [£/QALY]	ICERs by cost-effectiveness plane quadrant (%)				Percentages of cost-effectiveness at λ < willingness to pay (%)
		South-east	South-west	North-east	North-west	λ < £0	λ < £20,000
Main	88,733 (114,126)	14.6	52.7	2.5	30.2	67.3	63.1
BA main	597,103 (187,787)	17.0	79.3	0.1	3.6	96.3	93.8
Sensitivity analysis: ‘other’ unit cost
Cost	Dominated	11.6	26.6	5.5	56.3	38.2	34.6
BA cost	171,716 (167,195)	16.7	73.5	0.4	9.4	90.2	86.5
Sensitivity analysis: duration of exacerbation
3 days	279,489 (288,070)	23.5	43.8	4.3	28.4	67.3	66.0
BA 3 days	683,777 (314,408)	27.6	68.7	0.2	3.5	96.3	95.5
2 weeks	43,121 (112,808)	11.6	55.7	1.8	30.9	67.3	59.9
BA 2 weeks	105,496 (156,183)	13.3	83.0	0.1	3.6	96.3	90.3
Sensitivity analysis: utility of exacerbation
Utility	41,607 (53,125)	14.9	52.4	2.6	30.1	67.3	59.9
BA utility	101,793 (91,714)	17.4	78.9	0.1	3.6	96.3	89.8
Sensitivity analysis: type of contacts
Respiratory	Dominated	2.0	35.5	1.9	60.6	37.5	32.5
BA respiratory	65,020 (22,731)	3.4	76.2	0.5	19.9	79.6	70.7
Sensitivity analysis: cost estimation period
5 months	Dominated	11.7	29.3	5.4	53.6	41.0	34.2
BA 5 months	37,358 (45,190)	16.1	58.9	1.0	24.0	75.0	62.4
Subgroup analysis: under-fives
Under-fives	Dominated	1.4	4.2	2.6	91.8	5.6	4.5
BA under-fives	Dominated	2.8	30.6	1.2	65.4	33.4	26.3

FIGURE 25.

Cost-effectiveness acceptability curve for the letter intervention vs. no letter. Note that this graph demonstrates the probability of cost-effectiveness at a range of decision-maker ceiling willingness-to-pay values for the letter intervention from the main analysis (unadjusted) and the baseline cost-adjusted main analysis.

The very small QALY loss means that the incremental cost-effectiveness ratio (ICER) is very large for all the analyses. For example, for the adjusted main analysis, the ICER based on the mean point estimates was £597,103 (SE from the bootstrapped estimations: £187,787) per QALY, which is the ICER for the cost savings per QALY forgone, rather than the slightly more common cost per QALY gained associated with reported ICERs. The intervention was dominated (the letter intervention was more costly and less effective than no letter at the mean point estimate) when the ‘other’ unit cost was assigned a pooled weighted cost, when accounting for only ‘respiratory’-related contacts, when restricting the analysis to 5 months after intervention, and in the under-fives subgroup analysis; however, this was for the unadjusted analysis and the intervention was not dominated in the adjusted analysis except in the subgroup analysis. No ICER could be defined as dominant (a scenario in which the letter intervention would be cost-saving and more effective at the mean point estimate) as a result of this analysis. These ICERs are suggestive of an intervention that would be generally deemed not cost-effective if effectiveness (QALY gain) was important for decision-makers.

The sensitivity analyses showed that the cost-effectiveness results were sensitive to the assumptions regarding the costing of ‘other’ contacts, the duration and utility decrement assigned to a period of exacerbation, the types of contact included in the analysis, as well as the period of cost-estimation and whether or not the focus was on 5- to 16-year-olds or under-5-year-olds. The probability of cost-effectiveness in the adjusted analysis for the 5- to 16-year-olds, however, generally remained above 62% at a willingness-to-pay (WTP) threshold (λ) of £20,000 and above 75% when focused on the cost savings (rather than effectiveness; WTP threshold of £0) of the intervention. The probability of cost-effectiveness in the adjusted analysis for the under 5 years of age was 26.3% (λ < £20,000) or 33.4% (λ < £0). It should also be noted, however, that none of the sensitivity analyses demonstrated a significant difference in QALYs or costs, so the overall conclusion is that there is considerable uncertainty around the cost-effectiveness of the letter intervention.

Although there appears to be some discordance between the cost-effectiveness results and the trial’s primary clinical outcome of proportion of children having an unscheduled contact in September, the cost-effectiveness results are consistent with the mean contacts per child endpoints, particularly over the wider time interval.

Main findings

Previous work has shown an increase in the number of unscheduled medical contacts by children in autumn months (September to December), which may be a result of the start of the new school term. 13 By sending a letter in July to remind children (and their parents) of the importance of using their inhaler, it was hypothesised that the increase may be averted. More specifically, the hope was that a reminder letter would lead to a greater uptake of inhaler prescriptions in August, that this, in turn, would also lead to an increased adherence and, finally, that fewer unscheduled medical appointments would be required.

There is evidence of an impact on the first part of this pathway, as the intervention group demonstrated a higher uptake of prescriptions in August 2013. There was also an increase in scheduled contacts in the same month in this group. The data are not available to confirm actual medicine use (as quantified by the medicine possession ratio), and so it is unclear whether or not the increased uptake also translated into increased use.

The primary end point was unscheduled medical contacts in September 2013, which coincided with the start of the new school term. There was no evidence of a reduction in the intervention group, but the finding of a greater number of unscheduled medical contacts (albeit not statistically significant) is unexpected. We can offer three potential explanations for this.

First, a repeat prescription request may not be dispensed without a review in situations in which the child has not received a prescription for several months, or in which the parent wishes to discuss the advantages and/or disadvantages of recommencing treatment that the child has stopped some months before. This in turn may be classified as an unscheduled contact in the coding algorithm that we used to define contacts as unscheduled. The evidence to support this is the large increase in unscheduled contacts in the intervention group in August (relative to the control group). In addition, it seems the longer the time since a patient last collected a prescription, the higher the likelihood of an unscheduled contact in September in the intervention group. This implies that patients with more troublesome asthma may be more likely to have collected a prescription recently; conversely, patients who have not collected a prescription recently may have stopped their preventative medication and may be seeing their GP to check whether or not it is still necessary.

Second, the letter may have acted inadvertently as a trigger to contact the practice in relation to an unrelated medical issue they had been meaning to discuss, which may increase the number of contacts in the short term.

Third, and finally, the data collected at the time may be equivocal in the coding of the contact, leading us to incorrectly adjudicate certain contacts as unscheduled when the contact was scheduled, a limitation of routine data that we will return to in the next section; this is a factor that is important for this intervention, which did increase scheduled contacts in the first instance.

Despite the increase in unscheduled contacts in September, both the total number of contacts per child (i.e. scheduled plus unscheduled) and unscheduled contacts were lower in the intervention group than in the control over the extended study period (September–December 2013) and the full year (September 2013–August 2014). Although the effects were not statistically significantly, the minimal cost associated with the intervention meant that the intervention was found to have a high probability of being cost-saving overall. The economic analysis (which used data over a 12-month period from August 2013 to July 2014) estimated a mean cost saving across the base case of £36.07 per child and 96.3% probability that the intervention is cost-saving. By contrast, the cost-effectiveness results are suggestive of an intervention that would be generally deemed not cost-effective if effectiveness (QALY gain) was important for decision-makers, as it also resulted in a QALY loss in 82.9% of samples and a mean loss of 0.00017 QALYs.

The small effects observed could be because of the strength of the link between prescription uptake and unscheduled medical contacts. Around 5–6% more children received a prescription for asthma medication in August 2013, a difference that, while substantial, may not be sufficient to achieve the postulated 5% reduction in unscheduled medical contacts that the study was planned to detect. On the other hand, it could be that the increased prescription uptake could have reduced the severity (and days off school), which we could not detect from routine health-care data.

Strengths and weaknesses

Patient and public involvement input on the trial results

The children and parents fed back that they felt that the saving per individual as a result of the intervention was an important finding from the study. 27 Their feedback was that this should be emphasised more in the reporting of the result. We had the children and parents involved throughout, but this input was from October 2015.

A key point that came from the feedback was the question of whether the results would have been different if the study had been conducted over a 3-year period rather than 1 year. The parents felt that if the letter was something they expected each summer it could help in their planning for the start of the school year.

The trial in context: other studies and differences in results

We have not identified any other studies that have examined the economic benefits of a simple postal intervention in asthma patients and, therefore, it is difficult to compare our results with those of existing published studies. Yong and Shafie43 have published a systematic review that looked more broadly at non-pharmacological interventions aiming to improve asthma management. The interventions included by Yong and Shafie varied from educational and self-management interventions to environmental interventions. While the PLEASANT study intervention letter could be considered a simple form of patient education, the educational interventions included by Yong and Shafie were all more intensive, and the population was not restricted to school-aged children, making comparisons difficult. However, the broader evidence reviewed by Yong and Shafie suggests that non-pharmacological interventions that aim to improve individuals’ management of their asthma have the potential to be cost-effective.

Meaning of the study and implications for clinicians or policy-makers

The intervention in the PLEASANT study caused an increase in prescription collection in August, as well as an increase in scheduled medical contacts. It also had the effect of increasing medical contacts in August and September. After September, there was evidence of a fall in medical contacts, which, although not statistically significant, did follow through in the economic analysis to give a high probability of the intervention being cost-saving.

The increase in prescriptions and scheduled contacts in August could lead to individual GP practices wishing to implement the intervention. Evidence from the trial suggests that this would not increase overall costs associated with the asthma management, and may improve scheduled care. However, the evidence from the PLEASANT study is not sufficient to support a general recommendation for all GP practices.

Recommendations for future research

An objective of the study was to increase the take-up of prescriptions in August, as well as the adherence of children with asthma to taking their medications. The study was not able to assess the latter aspect.

Using routine data has many advantages in terms of trial efficiency in assessing public health or population-level interventions or in the assessment of proven interventions in real-world settings. Our analysis of the PLEASANT study data set suggests that further work is required to determine how to assess adherence using such data.

A suggested refinement for future trials using routine data could be the inclusion of a prompt for clinicians to answer study-related questions for patients in the study. For example, if patients enrolled in the study were to be given a specific study code, then clinicians, when having a consultation with such patients, could be automatically presented with a template for reporting key data during the study period. For the PLEASANT study, this would involve asking the clinician one simple question: whether the appointment was scheduled or not (and perhaps, second, if this was a respiratory-related consultation).

An additional point would be to emphasise to clinicians the importance of ensuring that the routinely collected patient data needed for the trial are complete, for example, in the PLEASANT study, prescription data to assess adherence. This could be done through a system prompt.

An investigation of the intervention on emergency contacts (such as out of hours, walk-in centres and emergency departments) would be of interest.

Future research in assessing interventions to improve adherence in school-aged children with asthma could include additional qualitative interviews with key stakeholders such as practice nurses, GPs and a wider group of children with asthma.

A study estimating the impact of asthma exacerbations on school-aged children, using a preference-based measure of HRQoL that has been validated for use in children, would be useful to inform future cost-effectiveness analyses.

The PLEASANT Research Team

Writing Group: Professor Steven A Julious, Dr Michelle J Horspool, Neil Shephard, Mike Bradburn, Dr Amanda Loban, Sarah Davis, Professor Cindy L Cooper, Dr Matthew Franklin (University of Sheffield), Robin May, Rachael Williams and Jennifer Campbell (CPRD).

TMG: Professor Steven A Julious, Dr Michelle J Horspool, Neil Shephard, Dr Amanda Loban, Cara Mooney, David White, Professor Paul Norman, Sarah Davis (University of Sheffield), Dr Jonathan Boote (University of Hertfordshire), Professor W Henry Smithson (University of Cork), Dr Heather Elphick (Sheffield Children’s Hospital), Mr Robin May, Rachael Williams and Jennifer Campbell (CPRD).

TSC: Dr Steve Holmes (Independent Chair, GP), Professor Andrew Wilson (Independent Academic GP, University of Leicester), Dr Martyn Lewis (Independent Statistician, Keele University), Camilla Mills (Independent Parent Representative), Professor Steven A Julious (Chief Investigator, University of Sheffield), Dr Michelle Horspool (Trial Manager, University of Sheffield), Mike Bradburn (Senior Statistician, University of Sheffield) and Dr Amanda Loban (Data Manager, University of Sheffield).

Co-applicants: Professor Steven A Julious, Dr Michelle J Horspool, Professor Paul Norman, Sarah Davis, Professor Cindy L Cooper (University of Sheffield), Dr Jonathan Boote (University of Hertfordshire), Professor W Henry Smithson (University of Cork) and Dr Heather Elphick (Sheffield Children’s Hospital).

GP Adjudication Panel: Dr Mark Boon (Conisbrough Group Practice), Dr Julie Hackney (Avenue Medical Centre) and Dr Karen Forshaw (Bentley Surgery).

Contributions of authors

Steven A Julious (Professor of Medical Statistics), Michelle J Horspool (Trial Manager), Sarah Davis (Health Economist), Mike Bradburn (Senior Statistician), Amanda Loban (Data Manager), Matthew Franklin (Health Economist), Wei Sun Kua (Health Economist), Robin May (CPRD), Jennifer Campbell (CPRD) and Rachael Williams (CPRD) together produced the first draft of the report.

The following conceived of or designed the work: Steven A Julious (Professor of Medical Statistics), Michelle J Horspool (Trial Manager), Sarah Davis (Health Economist), Paul Norman (Professor of Health Psychology), Cindy L Cooper (Sheffield CTRU Director), W Henry Smithson (Professor of Primary Care), Jonathan Boote (PPI Lead) and Heather Elphick (Consultant Respiratory Paediatrician).

The following were involved in the acquisition of data for the work: Jennifer Campbell (CPRD) and Rachael Williams (CPRD).

The following were involved in the interpretation of data for the work: Steven A Julious (Professor of Medical Statistics), Michelle J Horspool (Trial Manager), Neil Shephard (Medical Statistician), W Henry Smithson (Professor of Primary Care), Amanda Loban (Data Manager) and Saleema Rex (Data Analyst).

The following were involved in the analysis of data: Neil Shephard (Medical Statistician), Sarah Davis (Health Economist), Mike Bradburn (Senior Statistician), Matthew Franklin (Health Economist) and Oscar Bortolami (Medical Statistician).

The following drafted the monograph: Steven A Julious (Professor of Medical Statistics), Michelle J Horspool, (Trial Manager), Sarah Davis (Health Economist), Mike Bradburn (Senior Statistician), Amanda Loban (Data Manager), Matthew Franklin (Health Economist), Wei Sun Kua (Health Economist), Robin May (CPRD) and Jennifer Campbell (CPRD).

The following reviewed the work critically for important intellectual content: Steven A Julious (Professor of Medical Statistics), Michelle J Horspool, (Trial Manager), Sarah Davis (Health Economist), Mike Bradburn (Senior Statistician), Paul Norman (Professor of Health Psychology), Neil Shephard (Medical Statistician), Cindy L Cooper (CTRU Director), W Henry Smithson (Professor of Primary Care), Heather Elphick (Consultant Respiratory Paediatrician), Amanda Loban (Data Manager), Matthew Franklin (Health Economist) Robin May (CPRD) and Jennifer Campbell (CPRD).

All authors agree to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Journal articles and reports

Boote J, Julious S, Horspool M, Elphick H, Smithson WH, Norman P. PPI in the PLEASANT trial: involving children with asthma and their parents in designing an intervention for a randomised controlled trial based within primary care. Prim Health Care Res Dev 2016;17:536–48.

Hatfield I, Julious SA, Davis S, Horspool M, Norman P and Mooney C. An Assessment of the Resources Used by General Practices in the Intervention Arm of the PLEASANT Study in Sending out the Intervention. ScHARR Report Series No: 30. Sheffield: School of Health and Related Research (ScHARR), University of Sheffield; 2015.

Horspool MJ, Julious SA, Boote J, Bradburn MJ, Cooper CL, Davis S, et al. Preventing and Lessening Exacerbations of Asthma in School-age children Associated with a New Term (PLEASANT): study protocol for a cluster randomised control trial. BMC Trials 2013;14:297.

Horspool MJ, Julious SA, Mooney C, May R, Sully B, Smithson WH. Preventing and Lessening Exacerbations of Asthma in School-aged children Associated with a New Term (PLEASANT): recruiting primary care research sites – the PLEASANT experience. NPJ Prim Care Respir Med 2015;25:15066.

Data sharing statement

Access to patient-level data is provided by the CPRD for health research purposes and is dependent on approval of a study protocol by the Medicines and Healthcare products Regulatory Agency Independent Scientific Advisory Committee. More information on the Independent Scientific Advisory Committee and the protocol submission process can be found at www.cprd.com/isac (date accessed 30 May 2016).

Disclaimers

This report presents independent research funded by the National Institute for Health Research (NIHR). The views and opinions expressed by authors in this publication are those of the authors and do not necessarily reflect those of the NHS, the NIHR, NETSCC, the HTA programme or the Department of Health. If there are verbatim quotations included in this publication the views and opinions expressed by the interviewees are those of the interviewees and do not necessarily reflect those of the authors, those of the NHS, the NIHR, NETSCC, the HTA programme or the Department of Health.

Data received from the Clinical Practice Research Datalink

Initial test data set received 10 May 2013.

First baseline data set received 19 December 2013.

Second baseline data set received 3 February 2014.

Third baseline data set received 13 July 2014.

Baseline and follow-up data set received 19 January 2015.

Clinical Practice Research Datalink data

Data from the consultation, clinical, immunisation, test, referral and therapy tables from the CPRD Gold Data dictionary were used.

Overview

Figure 26 shows a very broad overview of how the data have been processed and the number of records. Full details of assumptions are now described.

FIGURE 26.

Decision tree showing an overview of how medical contacts have been allocated.

General assumptions

One ‘consultation’ (based on the combination of patient ID, practice ID and consultation ID) in the consultation table is considered one contact.

All consultation data supplied, not just those that are asthma related, are taken into account for the study.

Only consultations that happened on or after 1 August 2012 are included.

Assumptions used to code records as scheduled or unscheduled (contact type) are based on clinical, immunisation, therapy, referral, test and consultation data.

Records with unmatched event dates

Each consultation record is supplied with an event date; this event date does not always match the event date recorded for that consultation within the other tables. This is most likely a result of information entered into the database historically. Those contacts within the clinical table were included if they were relevant and unlikely to be duplicated (see Inclusion of ‘unmatched/historical’ data section for more details). All immunisation, therapy, referral and test records that did not match the event date supplied in the consultation data were excluded.

Clinical data

Clinical data are linked to consultation data, and all matched records are included.

Inclusion of ‘unmatched/historical’ data

If the event date does not match but the clinical event date is within the dates of interest (i.e. from 1 August 2012 to 30 September 2014), then those records that are both relevant and unlikely to be duplicated are included. The decision on which records to include was made by the GP Adjudication Panel after reviewing the most common unique terms (10% of the terms, which covered 88% of the data). The most common 10 terms and the decision of whether or not to include them is shown in Table 25 for information; rules based on this review were used to decide whether or not to include the 12% of data not reviewed, for example ‘if it contains seen it is relevant and unlikely to be duplicated’.

Summary of coding using clinical data

The records included are used to determine scheduled or unscheduled contacts based on ‘medcode description’; the clinical data references Pegasus medical data using the field ‘medcode’ to get ‘medcode description’. If contact types cannot be determined by ‘medcode description’, then clinical consultation type ‘constype’ (consultation type) is referenced.

Following GP Adjudication Panel review of the medcode descriptions, in which over 90% of the data were reviewed (17% of the unique terms), clinical records to be marked as scheduled, unscheduled, not applicable or unknown were identified based on terms (see Boxes 1–4 and Tables 26 and 27 for examples; full details are available on request). Based on this review, rules to apply to the data were also determined (see Table 26). Finally, decisions on how to code the remaining records based on the clinical consultation type were made (see Table 27).

Conflicting clinical contact types

Clinical data contain more than one record per consultation. In some cases, the same consultation ID can have more than one clinical contact type. For these clinical records, we assume that unscheduled takes precedence (i.e. they are likely to have come in for an unscheduled visit but had a scheduled ‘type’ of procedure at the same time) over all other contact types; that scheduled takes precedence over not applicable and unknown; and that unknown takes precedence over not applicable.

Clinical to consultation

The code assigned in accordance with clinical contact type as described above is linked to the consultation data.

Consultation data marked as ‘unknown’ based on the clinical data as well as consultation data that did not link to clinical data are coded based on immunisation, therapy, referral, test and consultation data as described in the following sections.

Immunisation data

Uncoded consultation data are matched against immunisation data; if at least one match is found in the immunisation record, then these are marked as ‘scheduled’.

Therapy

Those that do not match with immunisation data are linked to medication data. If at least one match is found, they are marked as ‘unscheduled’.

Referral

Those that do not match with immunisation or therapy data are linked to referral data. If at least one match is found, they are marked as ‘unscheduled’.

Test

Those that are still uncoded are linked to test. If linked to test it is coded as either ‘scheduled’, ‘unknown’ or ‘unscheduled’ based on a review of the data. In general, if the test is part of the routine asthma review, then it is coded as ‘scheduled’; if it is testing peak expiratory flow rate then it is coded as ‘unknown’. Otherwise, it is coded as ‘unscheduled’ (full details are available on request).

When a consultation links to more than one record, the same rules of precedence apply as outlined in the Conflicting clinical contact types section.

Consultation data

For consultation data that are still ‘unknown’, ‘unlinked’ and, therefore, uncoded, consultation type is used to determine whether it is scheduled, unscheduled or not applicable (see Table 28 for a summary; full details are available on request). In this way, all contacts will now be coded as either ‘scheduled’, ‘unscheduled’ or ‘not applicable’.

Emergency contacts

In addition to coding as scheduled, unscheduled and not applicable, some consultation types from the consultation table were coded as emergency (Table 29).

Scoping

A scoping search was conducted to establish the likely quantity and relevance of published literature. This was done by searching MEDLINE and The Cochrane Library (Cochrane Database of Systematic Review, HTA database and NHS Economic Evaluation Database) using a limited number of population terms in addition to a search filter for quality of life. It was found that there was a lack of utility data derived from EQ-5D in children with asthma. Although EQ-5D is the preferred outcome measure, the standard version of EQ-5D is not designed to be used with children. EQ-5D-Youth is available for children and adolescents, but there is not yet a validated UK tariff. In view of this, the NICE reference case states to use other validated preference-based measures developed for children, but does not specify the preferred quality-of-life instrument. 35 Therefore, a broad approach was taken in the search to identify utility values derived from EQ-5D, as well as other preference-based measures. EQ-5D values estimated from mapping studies were also considered.

Search strategy

Inclusion and exclusion criteria

The inclusion and exclusion criteria for the review are summarised in Table 30. Systematic reviews and protocols were not included, but were used to identify relevant papers. Modelling studies were examined to determine the source of utility values used. Modelling studies that described utility data not reported elsewhere were included in the review.

Selection of studies

In the first stage of study selection, titles and abstracts of the searched results were screened against the inclusion/exclusion criteria. Full articles were assessed if titles and abstracts were unclear. All studies identified at titles and abstracts were further screened at full text. The studies were screened by a single reviewer.

Quality assessment

Quality assessment of articles in this review followed the criteria (sample size, number loss at follow-up and handling of missing data) recommended by Papaioannou et al. 45 in the Decision Support Unit Technical Support Document on the identification, review and synthesis of health state utility values from the literature.

Data extraction

Data extracted comprised characteristics of study population, study design and details of outcome measurements (descriptive system, tariff used, method of valuation, time of measurement, mean utility data and other relevant measures).

Selection of utility data for use in the economic analysis

Selection of utility data to use in the economic analysis was based on (1) quality of the study; (2) the relevance of utility data to the population and health states in the PLEASANT study; and (3) the extent to which the measurement method was in accordance with the NICE reference case.

Results of systematic review of health-related quality-of-life data

A total of 927 studies were retrieved from the database search and reference tracking. After removal of duplicates, 683 studies were screened at titles and abstract. A total of 659 studies were excluded at this stage (see Appendix 8). Subsequently, 24 papers were screened at full text and 10 papers were excluded, with reasons given in Appendix 9. Finally, 14 papers were included in this review. Figure 27 shows the search process of this review.

FIGURE 27.

Flow diagram of search process. EQ-VAS, EuroQol Visual Analogue Scale.

Study characteristics for the included studies are summarised in Table 31. The study populations are summarised in Table 32 and methods used to measure HRQoL are summarised in Table 33. Details regarding study quality are provided in Table 34. Details regarding the suitability of the studies for use in the economic model, based on the criteria described above, are provided in Table 35.

Six studies included UK patients,38,46,47,52,53,56 three of which were multinational studies. 38,53,56 Three papers were from the USA,49,55,57 two were based in Canada50,51 and one each was from the Netherlands,36 Belgium48 and Spain. 54 Only the studies by Juniper et al. ,51 Chiou et al. 49 and Powell et al. 46 directly measured HRQoL in populations confined to children. Chiou et al. 49 recruited children aged between 7 and 12 years with diagnosed asthma of at least mild persistent severity, while Juniper et al. 51 studied children with symptomatic asthma with a mean age of 12 years (range 7–17 years) and Powell et al. 46 included children aged between 2 and 16 years with acute severe asthma. Two studies, by Rodríguez-Martínez et al. 54 and Carroll and Downs,55 elicited preferences from parents regarding health states in children. Other studies comprised populations with mixed age groups. Of these, the studies by Mittmann et al. 50 and Willems et al. 36 presented HRQoL data stratified by age.

The populations in the included studies differed in asthma severity and characteristics. Five studies measured HRQoL using EQ-5D. 36,46–48,52 Other studies used outcome measurements, such as the Pediatric Asthma Health Outcome Measure (PAHOM) (n = 2) and Health Utilities Index Mark 2 (HUI-2) (n = 1) and Mark 3 (HUI-3) (n = 1). Direct valuation using vignettes was used in two studies. This review also included three modelling studies,38,53,56 which estimated EQ-5D data from mapping exercises.

The EQ-5D is a generic preference-based measure in which the descriptive systems consist of five dimensions: mobility, depression/anxiety, self-care, usual activities, pain and discomfort. Each dimension has three levels of severity, and this gives rise to 243 possible health states described by the EQ-5D. In the UK, scoring of EQ-5D was based on time trade-off (TTO) in a representative sample of 2997 adults administered using York Measurement and Valuation of Health TTO protocol. Public preferences were obtained for 43 health states and regression was used to model data for the remaining health states. Utility score from the algorithm was anchored at ‘1’ for perfect health and ‘0’ for a state equivalent to death. 58

Willems et al. ,36 Price et al. 47 and Powell et al. 46 were randomised controlled trials (RCTs) that elicited an EQ-5D index score using UK preferences, whereas the EQ-5D score in a cohort study by Brusselle et al. 48 was based on the Belgian tariff. Norman et al. 52 was a modelling study that used EQ-5D data collected from the Evaluate Xolair for Asthma as Leading Treatment (EXALT) trial. The tariff used in the EXALT study is not described by Norman et al. ,52 but the data are described as being consistent with the NICE reference case, suggesting that the UK TTO valuation set was used.

In the MAGNEsium Trial In Children (MAGNETIC), Powell et al. 46 included a population of children (n = 508) with severe acute exacerbations, as defined by British Thoracic Society (BTS)/Scottish Intercollegiate Guidelines Network (SIGN). The MAGNETIC was a prospective, double-blind, multicentre RCT in the UK, designed to compare efficacy of nebulised magnesium sulphate with usual care. EQ-5D and Paediatric Quality of Life Inventory (PedsQL™) postal questionnaires were collected at 1 month post exacerbation. EQ-5D data were obtained for children aged ≥ 5 years and were filled out by parents as proxy, while PedsQL™ were obtained for all children and were self-completed if children were over 5 years. Respondents were asked to recall events in the previous 4 weeks while filling out the outcome measures. The adult UK tariff was applied to EQ-5D to obtain utility value for each child. Utility values for patients under 5 years were estimated through mapping between EQ-5D and PedsQL™. In this study, baseline EQ-5D data during exacerbation were not collected for ethical reasons. Therefore, asthma symptom scores (ASSs) at exacerbation were mapped to EQ-5D based on experts’ opinions. The expert team comprised a paediatric consultant and two respiratory nurses who routinely treated asthmatic paediatric patients. An EQ-5D health state of 11111 was assigned to ASSs of 1–3 in the base case, while ASSs of 4–6 and 7–9 were mapped to EQ-5D health states of 22222 and 33333, respectively. In our opinion, the subjective nature of this mapping between ASS and EQ-5D was considered to make the EQ-5D scores estimated at the time of exacerbation very uncertain. Furthermore, these data would be relevant only to the subgroup of patients who have severe acute exacerbations requiring treatment in secondary care, as this was the population recruited into the MAGNETIC. This study was blinded to patients, health-care providers and outcome analysts. Therefore, it had low risk of performance and detection bias. However, the study was subjected to risk of attrition bias due to the low response rate of EQ-5D questionnaires. The authors addressed this limitation by using a mapping function to estimate EQ-5D data for those with PedsQL™ data. This was based on the subset of patients for whom both PedQL and EQ-5D data were available. Following mapping estimations, a total of 218 EQ-5D data were available for analysis for the outcome 1 month after exacerbation.

Price et al. 47 included patients in the UK aged between 12 and 80 years with poorly controlled asthma at BTS/SIGN treatment step 2 or 3. The mean age of patients was 44.74 years (SD 16.49 years) at step 2 and 50.02 years (SD 15.93 years) at step 3. In step 2 patients, a leukotriene receptor antagonist was compared with inhaled corticosteroid. In step 3 patients who were already receiving inhaled corticosteroid, a leukotriene receptor antagonist was compared with a long-acting β₂-agonist. EQ-5D data were directly measured from patients and were presented by treatment steps and interventions at baseline, 2 months and 2 years. Utility values were estimated using UK preferences. This RCT had a high retention rate, with 5–10% loss to follow-up. A large proportion (75%) of patients presented with less than four missing data, and missing data were handled using multiple imputation. This single-blind RCT (n = 687) was robust, with large sample size, low risk of attrition bias and measured outcomes with EQ-5D. However, utility data presented were not stratified by age nor related to asthma exacerbations. Therefore, these data lack applicability to the PLEASANT trial and the health states modelled.

Willems et al. 36 used UK preferences to estimate utility scores for asthmatic patients in the Netherlands. Populations comprised adults (n = 53) and children (n = 56) with mild to moderate asthma [Global Initiative for Asthma (GINA) states I–III]. EQ-5D questionnaires were filled by carers for children under 12 years and self-completed for those aged ≥ 12 years. There were only four children lost to follow-up, and various imputation techniques were applied. Missing baseline scores were imputed with mean scores. Quality of life scores at baseline (usual care 0.96, nurse monitoring 0.92) were consistent with the good lung function of the study’s population [mean forced expiratory volume in the first second (FEV₁) above 90% predicted]. However, these results were elicited from a non-UK population, but did use a UK valuation set. Willems et al. 36 did not examine the utility decrement in exacerbation.

Brusselle et al. 48 conducted a 1-year cohort study (n = 158) to determine the efficacy and safety of omalizumab by looking at changes from baseline in a single-arm study. The mean age of the population studied was 48.17 years (SD 17.18 years) and age ranged from 12 to 83 years. Patients included had poorly controlled severe allergic asthma (FEV₁ < 80% predicted) and past history of exacerbations. The Belgian tariff was applied to the collected EQ-5D data at baseline and 1 year. Only 126 of 158 patients had baseline EQ-5D values, and only 67 had EQ-5D data at 1 year. Handling of missing data, however, was not reported. This tariff was obtained from public preferences in Belgium using the visual analogue scale (VAS) valuation method. 59 However, valuation using VAS is not a choice-based method. In the UK, NICE expressed a preference of using TTO as the valuation method, and, in the absence of TTO, other choice-based methods such as subgroup analysis are preferred over VAS. 60 Therefore, utility data estimated from this study do not meet the NICE requirement of using a choice-based valuation method.

Two USA-based studies, by Chiou et al. 49 and Gerald et al. ,57 used PAHOM, an asthma-specific preference-based measure designed for children. It consists of a descriptive system with three dimensions: symptoms, emotions and activity. The symptoms dimension is classified to three levels of severity, while emotions and activity are dichotomous choices to indicate presence or absence of problems. Unlike EQ-5D with a recall period of 1 day, respondents are asked to describe health states for the past 7 days using PAHOM. The utility value of a health state is calculated as the average utility values over 7 days. Preference weights for PAHOM were elicited from 114 adults in Seattle, WA, USA, who responded for children. Subgroup analysis and VAS were used to value health states. As not all health states were valued using subgroup analysis, because of cognitive burden, VAS values were transformed into subgroup analysis values using relative risk attitude equation. 49

Chiou et al. 49 measured utility value in 72 children (aged 7–12 years) with diagnosed asthma of at least mild persistent severity as 0.83 (converted subgroup analysis value). Chiou et al. 49 also reported mean VAS and subgroup analysis values for patients according to asthma severity, with subgroup analysis values of 0.79 for mild or no symptoms, 0.70 for moderate and 0.28 for severe. A limitation of this study was the small sample size, which may have affected the accuracy and validity of results, particularly for the estimates stratified by severity. Values stratified by presence or absence of exacerbation were not reported.

Gerald et al. 57 performed a modelling study on different screening strategies for asthma. A decision tree and Markov models for a cohort of children were constructed. The Markov model consists of five health states: asthma symptom-free day, symptom days, exacerbation recovery days, emergency department visits and hospitalisation days. The utility value for each health state was derived using PAHOM. PAHOM states were allocated to the modelled health states. When several PAHOM states could describe a modelled health state, utility values of the relevant states were averaged to estimate a single utility value. For example, three or four PAHOM states were thought to characterise ‘symptom days’ in the model. The utility values of these states were averaged to derive utility value for ‘symptom days’. The authors highlighted that this approach may fail to capture valuation of ‘symptom days’ accurately. In our opinion, the subjective nature of this mapping from modelled health states to PAHOM states reduces the robustness of these utility estimates. In addition, a general concern regarding PAHOM was that this measure was not validated for its psychometric properties. Furthermore, validation of the relative risk attitude equation used to derive subgroup analysis values was not performed. 49

Two Canadian-based observational studies used the Health Utilities Index (HUI) as an outcome measure. Juniper et al. 51 studied the minimum skills required by children to complete outcome measurements unassisted. The Paediatric Asthma Quality of Life Questionnaire, Feeling Thermometer, HUI and direct valuation were administered to 52 children aged 7–17 years (mean 12 years) with symptomatic asthma (mean FEV₁ 85% predicted). The HUI-2 Canadian tariff was applied to obtain utility value. The mean HUI baseline value for asthma was reported as 0.89 (SD 0.09).

The six-dimensional version of HUI-2 is a common generic outcome measure in children. Each dimension has 3–5 levels, allowing 8000 unique health states to be defined. The HUI-2 tariff was estimated from a sample of 293 parents of school children in Ontario, Canada. Valuations were performed using VAS and three health states were valued with VAS and subgroup analysis. A power function was then derived to map VAS values to subgroup analysis values, and multiattribute utility theory was used to derive the valuation functions. 58

Mittmann et al. 50 conducted a cross-sectional study to measure HRQoL of 20 chronic diseases. The HUI-3 was administered through interview to 17,626 household residents (≥ 12 years) in Canada. HUI-3 is an adapted version of HUI-2 with additional dimensions and levels. HUI-3 weights were elicited from a random sample of adults (n = 504) in Ontario, Canada. In this study, however, the HUI-2 scoring algorithm was used for HUI-3 data. The mean HUI score reported for children (aged 12–19 years) with asthma was similar to that reported by Juniper et al. 51

In measuring and valuing children’s health, NICE is less clear on the preferred instrument, but advises use of a standardised and validated preference-based measure designed for children. Although HUI is an example of an instrument that meets the mentioned criteria, the HUI data from these studies may not be valid, as the study designs lack rigour. First, the small sample size (n = 52) recruited by Juniper et al. 51 may introduce inaccuracy to the results. Second, HUI-3 data were inappropriately scored in the study by Mittmann et al. 50 and utility scores estimated were deemed to be provisional by the authors. Furthermore, neither of these studies reported the utility decrement attributable to asthma exacerbation.

Four modelling studies performed mapping to estimate EQ-5D values. Brown et al. 56 and Norman et al. 52 constructed Markov models to evaluate the cost-effectiveness of omalizumab in addition to standard care. Norman et al. 52 used EQ-5D scores measured in the EXALT study for day-to-day asthma symptoms. The EXALT study was an open-label RCT, comprising 404 patients in the UK (age range from 12 to 75 years) with poorly controlled severe allergic asthma (FEV₁< 80% predicted). Utility for day-to-day symptoms (by treatment arm) was estimated from EQ-5D scores recorded in the EXALT study.

Norman et al. 52 also conducted a systematic review of HRQoL literature to identify HRQoL data of relevance to both adult and paediatric populations. In their base-case analysis they used data from Lloyd et al. ,37 a study conducted in an adult population that provides estimates of the health utility decrement (loss) associated with exacerbations requiring oral steroid treatment and exacerbations requiring hospitalisation. The decrement was measured by comparing baseline EQ-5D values with those reported at 4 weeks for patients who did and did not experience exacerbations during that 4-week period. They cited another study by Steuten et al. ,61 which also provided utility values for exacerbations in an adult population. However, this study collected data at 3- to 6-month intervals, which could make it harder to detect the relationship between short-term exacerbations and health utility than the 4-week interval used by Lloyd et al. 37

Brown et al. 56 used a published algorithm by Tsuchiya et al. 62 to map the mini-Asthma Quality of Life Questionnaire (AQLQ) scores from the ETOPA trial onto the EQ-5D. The ETOPA trial was a multinational open-label trial that recruited 312 patients aged between 12 and 73 years (mean > 35 years) with poorly controlled allergic asthma (mean FEV₁ < 73% predicted). 63 (Note that Brown et al. 56 used data from the subgroup of ETOPA patients with severe disease, but baseline characteristics are not described for this subgroup, so Table 32 provides characteristics for the ETOPA trial as a whole.) The AQLQ scores were mapped to EQ-5D for patients separated by disease state and responder status. The mapping algorithm used by Brown et al. 56 was derived from a RCT of 3000 adults in the UK with a wide range of asthma. 62 In the RCT used to generate the mapping algorithm, both EQ-5D and AQLQ were collected. 64 Domains in EQ-5D were found to overlap with those in AQLQ, with correlations between 0.56 and 0.65. Six main mapping models and two supplementary models were derived using the regression method and were validated using an external data set. However, these mapping functions were associated with large marginal errors, and should be considered only as second best to direct elicitation of EQ-5D data. 58

In the economic modelling study by Brown et al. ,56 literature-based estimates were used to model the decrement associated with exacerbations, as the authors stated that the ETOPA trial collected insufficient patient quality-of-life data during exacerbations. The literature-based estimates cited by Brown et al. 56 appear to be from an earlier publication65 of the study by Lloyd et al. 37

The modelling studies by Briggs et al. 38 and Doull et al. 53 mapped AQLQ scores from the 52-week Gaining Optimal Asthma ControL (GOAL) trial onto EQ-5D values. The GOAL study was a multinational double-blind RCT designed to evaluate efficacy of a combination of fluticasone/salmeterol compared with fluticasone in terms of asthma control. The GOAL study comprised 3416 patients (mean age > 35 years; range 12–80 years) with uncontrolled asthma (mean FEV₁ < 80% predicted) from 44 countries. 66 Asthma control in the GOAL trial was classified as totally controlled, well controlled, not well controlled or exacerbation requiring oral steroid or secondary care by Briggs et al. 38 using the GINA definition. As the GOAL trial collected only AQLQ data, a mapping function obtained through personal communication with Macran and Kind (no further details of this communication provided by Briggs et al. 38) was used to transform AQLQ scores to EQ-5D values. Subsequently, the utility value for each asthma control health state was derived using regression. In the regression model, a UK indicator was added as a dummy variable to adjust for a UK specific-population. The dependent variable was the utility value, whereas asthma control and the UK indicator were the independent variables. Both independent variables were found to be significant predictors of quality of life. The quality-of-life data from this study are of relevance to the PLEASANT trial. However, the mapping function used in the analysis by Briggs et al. 38 was inadequately described by the authors, and a published article providing more details could not be identified from searches. Therefore, an assessment of mapping performance was not possible.

Doull et al. 53 adapted the analysis by Briggs et al. ,38 and reclassified asthma control to ‘symptom free’ and ‘with symptoms’. Totally controlled asthma was classified as ‘symptom free’, while other states were classified as ‘with symptoms’. The weekly utility in the ‘with symptom’ state was equivalent to the weighted average of the weekly utility in well controlled, not well controlled and exacerbation health states from Briggs et al. 38 Regression was used to estimate the relationship between asthma control and quality of life, when quality of life was obtained by mapping AQLQ scores to EQ-5D. Asthma control and the UK indicator were entered into the model as the independent variables, while weekly utility was entered as the dependent variable. Subsequently, utility for the ‘with symptoms’ and the ‘symptoms-free’ health states were estimated from the regression coefficients. As utility data in this study were adapted from Briggs et al. ,38 which mapped AQLQ scores to EQ-5D using the mapping function by Macran and Kind, the validity of mapped data was likewise not assessable.

The method used in Carroll and Downs55 and Rodríguez-Martínez et al. 54 involved valuation of hypothetical health states by parents. Parents were asked to value health states described in vignettes by imagining their children affected by those states. Descriptions in vignettes, however, differed across studies. Rodríguez-Martínez et al. 54 developed asthma-specific vignettes based on PAHOM,49 and these were validated by expert opinions, whereas Carroll and Downs55 developed general descriptions of 29 health states with the inclusion of time as a factor. Rodriguez-Martinez et al. 54 requested parents (n = 76) to value vignettes using subgroup analysis, while Carroll and Downs55 used subgroup analysis and TTO methods in a sample of 4016 parents (Note that each parent valued only three of a potential 29 states providing around 415 values per state.) Neither study constructed vignettes based on rigorous methods such as a focus group. The lack of standardised descriptive systems of vignettes and different valuation methods also resulted in a lack of comparability of results between studies. In addition, vignettes are limited to specific descriptions of a condition, and may not fully reflect all experiences of a patient. Therefore, vignettes do not meet the NICE reference case and are considered of little value in economic evaluation. 60 In view of the various limitations associated with vignettes, utility values from Carroll and Downs55 and Rodríguez-Martínez et al. 54 were not considered suitable for use in the PLEASANT study economic analysis.

Health state utility values used in the analysis

The utility values used in the economic evaluation by Briggs et al. 38 appear to be particularly relevant to our proposed model structure, as they are reported for relevant health states, including an exacerbation state, and have been estimated from a trial population that included some children. However, the mapping algorithm used to convert from the condition-specific HRQoL measure (AQLQ) to the EQ-5D utility score is not from a published source, and is not described in detail, making it difficult to assess its validity. However, if the values reported by Briggs et al. 38 are taken at face value, they provide an estimate of the utility loss for exacerbation versus total asthma control of –0.216 (SE 0.007). It is possible that some patients do not have total asthma control in the absence of an exacerbation and the difference between the utility values for the exacerbation state and the not well controlled states is smaller, at –0.112. The data from Briggs et al. 38 suggest that the utility decrement for exacerbation in the average patient is likely to fall in the range –0.112 to – 0.216. The utility decrements provided by Lloyd et al. 37 from an adult population are –0.1 and –0.2 for exacerbations requiring oral steroids and exacerbations requiring hospitalisation, respectively. It therefore appears that there is reasonable agreement between the values reported by Briggs et al. 38 and Lloyd et al. 37

We accept that the estimates provided by Briggs et al. 38 and Lloyd et al. 37 probably underestimate the degree of utility loss in children with a severe or life-threatening acute exacerbation during the period of hospitalisation. This is because the utility values were not measured during the acute exacerbation period itself. In the MAGNETIC, which estimated utility scores in children attending EDs with severe acute asthma, the utility was estimated to be reduced from a baseline of 0.88 to 0.516 during the initial acute period, giving a utility decrement of 0.364. However, in the MAGNETIC, this more severe utility decrement was only applied until hospital discharge, with the average length of hospital stay being 1 day. If we apply a decrement of 0.364 for 1 day and assume a loss of 0.2 in the remaining 6 days, the average utility loss over the whole week of exacerbation (–0.22) would be similar to that reported by Briggs et al. 38

Given the uncertainty regarding the mapping algorithm used by Briggs et al. ,38 and the previous use of data from Lloyd et al. 37 in a number of published economic evaluations, we decided to use the data from Lloyd et al. 37 in the base-case analysis. The data from Briggs et al. 38 have been explored in a sensitivity analysis using the difference between the total control state and the exacerbation state (–0.216) to estimate the quality-of-life decrement from exacerbations. This sensitivity analysis is considered to provide an upper limit on the utility decrement attributable to exacerbation.

For patients without an exacerbation, we have taken the baseline utility score for the control arm of the study by Willems et al. ,36 as this provides an estimate based on the child version of the EQ-5D valued using the adult UK TTO valuation set. The population was Dutch children aged 7–18 years with a GINA severity stage I–III receiving standard outpatient care. The value applied to patients without an exacerbation will affect the calculation of absolute QALYs in each trial arm, but does not affect the estimation of incremental QALY gain that goes into the cost-effectiveness ratio. Therefore, the selection of this data source is less critical than that used to determine the decrement attributable to exacerbations. The data that have been applied in the model are summarised in Health outcomes.