Medications for chronic obstructive pulmonary disease: a historical non-interventional cohort study with validation against RCT results

FP-SAL exposed compared with unexposed analysis

Step 2: selection of pool of unexposed patients

Patients who had time periods in which they were unexposed to FP-SAL on or after the ‘eligible for the TORCH trial’3 inclusion date and who did not meet any of the TORCH trial3 drug exposure exclusion criteria (see Box 1) were selected (Figure 2). 3 The start of follow-up date (i.e. the index date) for the unexposed time period was selected as a random date between the start and end of the unexposed period (see Figure 2). Individuals in CPRD were able to contribute more than one such unexposed time period to the total pool of unexposed time periods (see Figure 2) to avoid placing a restriction on a study entry that would not have existed if the potential participants were going to be recruited to a trial (i.e. they could have been recruited to a trial during any one of the eligible periods and we did not want to restrict to only one of these periods at this stage just because we were performing a study using data that had already been collected). Unexposed time periods were then removed from the cohort if the patient met any of the remaining TORCH trial3 exclusion criteria prior to the index date. 3

FIGURE 2.

Management of FP-SAL-exposed and FP-SAL-unexposed time periods in selection of people from the CPRD. Step 1: selection of all potentially eligible patients. Six example patients in CPRD. Green arrow = date at which individual met TORCH trial3 inclusion criteria, grey time periods = FP-SAL exposed, white time periods = FP-SAL unexposed. Step 2: selection of pool of unexposed patients. Unexposed time periods selected and exclusion criteria relating to drug exposures applied. An unexposed record index date is then assigned as a random date within each unexposed period (indicated by diamond symbols) and further TORCH trial3 exclusion criteria applied based on this date. In this example, one unexposed record from each of persons 1, 4 and 6 were excluded prior to step 3. Step 3: selection of unexposed-to-FP-SAL time periods by 1 : 1 matching to TORCH trial3 participants. Dotted red lines indicate matching. Matching characteristics assessed on index date of specific unexposed time period and only one time period per person could be matched to the TORCH trial. 3 Step 4: selection of exposed-to-FP-SAL time periods and application of TORCH trial3 exclusion criteria. In this example, one exposed time period from persons 3 and 6 was excluded based on TORCH trial3 exclusion criteria. Step 5: selection of comparable FP-SAL-exposed participants. Pre matching there was one record per person for the FP-SA-unexposed cohort and one or more per person for the FP-SAL-exposed cohort. After matching there were an equal number of each. Exposed and unexposed records then followed up and analysed from index date onwards following an ‘intention-to-treat’ approach. Reproduced with permission from Wing et al. 15 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: https://creativecommons.org/licenses/by/4.0/. The figure includes minor additions and formatting changes to the original.

Reproduced with permission from Wing et al. 15 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: https://creativecommons.org/licenses/by/4.0/. The box includes minor additions and formatting changes to the original.

Selection of participants: FP-SAL-exposed participants compared with salmeterol-exposed participants

The participant selection approach was analogous to the FP-SAL-exposed compared with the FP-SAL-unexposed participant selection, except that the comparator group selected was those exposed to SAL (rather than those unexposed to FP-SAL). The resulting differences in participant selection were as follows. For step 1, the study period was from 1 January 2000 to 1 January 2017 (increased to ensure sufficient numbers of eligible SAL-exposed individuals). For step 2, instead of selecting unexposed-to-FP-SAL time periods occurring on or after the ‘eligible for the TORCH trial’3 inclusion date, we selected periods of SAL exposure. Individuals in the CPRD who had more than one SAL-exposed eligibility period within their record were able to contribute more than once to the pool of SAL-exposed participants (with the covariates and person-time contributed unique to the specific SAL-exposed eligibility period). The index date for each SAL-exposed record was the first date of the eligible SAL exposure period (i.e. the first day of the SAL prescription). All other aspects of step 2 and steps 3–6 were then as described for the FP-SAL-exposed compared with FP-SAL-unexposed participant selection (with SAL-exposed records in place of FP-SAL-unexposed records wherever mentioned).

Exposures

For all objectives, exposures were determined using CPRD prescribing records and codelists for COPD treatments [codelists are available from https://datacompass.lshtm.ac.uk/1655/ (accessed 28 May 2021)].

For all objectives, being prescribed FP-SAL was the primary exposure of interest and the comparison exposure groups were (1) people not being prescribed FP-SAL and (2) people being prescribed SAL only. In addition to FP-SAL and SAL, periods of exposure to oral corticosteroids (OCSs), ICS, FP, any LAMA or any LABA were identified to facilitate application of the inclusion and exclusion criteria described in Selection of participants.

For all drug exposures, duration of an exposure period was derived by multiplying the CPRD quantity variable by any relevant dose information stored in the packtype variable and then dividing by the value in the numeric daily dose CPRD variable. For example, for a prescription record with quantity = 1, packtype = ‘60 dose inhaler’ and numeric daily dose = 2, the duration of the exposure period was (1 × 60)/2 = 30 days. For prescription records where it was not possible to calculate this exposure period (e.g. because of a missing quantity variable), the median value for that specific drug substance and packtype combination was imputed as the exposure duration. To attempt to account for any uncertainty in the end date of an exposure period (e.g. because of people not taking the medicine as directed or relying on additional medication previously prescribed and kept at home), a grace period of half the median duration for the specific drug substance/pack type combination was added to the calculated exposure duration to estimate the end date of the exposure period.

Outcomes

Outcomes were COPD exacerbation, all-cause mortality, pneumonia and time to treatment discontinuation, and these were defined as follows:

• COPD exacerbation – defined using a CPRD-HES algorithm that was developed previously by one of the co-authors of this study. 11

• All-cause mortality – recorded in Office for National Statistics mortality statistics (i.e. data that are linked to CPRD data).

• Pneumonia – defined using a CPRD-HES algorithm that was published previously by authors of this study. 12

• Time to COPD treatment discontinuation – treatment discontinuation classified as a period of ≥ 90 days with no further prescription for the specific drug.

Covariates

Covariates available for inclusion in the propensity score models included lung function, age, sex, alcohol consumption, vascular disease, prescriptions for aspirin or statins, prior treatment with other COPD medication, type 2 diabetes, history of cancer, renal disease and health-care utilisation (i.e. rate of consultations, hospitalisations, hospital procedures and drug prescriptions).

Objective 1

Assuming a baseline conservative exacerbation rate of 0.5 per patient per year,11 we required a sample of 408 patients per treatment group to detect a reduction in annual exacerbation rate of 0.4 per year, with 80% power and 5% significance. Our estimated sample size based on feasibility work assessing the number of people meeting TORCH trial3 inclusion criteria was ≈ 12,000, providing ample power for the main outcomes of interest but also allowing stratification by patient characteristics to determine stratified results. For example, to detect a reduction from 0.5 to 0.4 exacerbations per year with 80% power and 1% significance we would have needed ≈ 600 people in each treatment group.

Objectives 2 and 3

We were also confident that we would have sufficient numbers to allow well-powered analyses for objectives 2 and 3. For example, a feasibility count looking at the number of people aged > 80 years eligible for inclusion in objective 2 estimated that there would be > 2000 people in each exposure group.

We were aware that further application of TORCH trial3 exclusion criteria would lessen sample sizes further, but it was not possible to estimate the extent that this would happen from the data that were available to us prior to undertaking the study.

Propensity score for addressing confounding

The propensity score for objective 1 was constructed using the principle that predictors of the exposure (i.e. FP-SAL) and outcome (i.e. exacerbations, mortality and pneumonia) or outcome only should be included. We considered a wide range of variables as the pool of initial variables for inclusion (as listed in Covariates) based on a priori knowledge of potential association with exposure or outcome, such as age, sex, BMI, alcohol consumption and a wide range of comorbidities (e.g. type 2 diabetes, coronary heart disease, cerebrovascular disease, peripheral vascular disease, heart failure, hypertension, renal disease and cancer). We also considered adjusting for health-care utilisation intensity (e.g. number of prior visits, hospitalisations, number of distinct medications used, number of procedures), as these are generic correlates of disease state and the likelihood of recording completeness. Our group has substantial prior experience of building propensity models. 16–19

For us to then select variables for inclusion in the propensity score, we removed those variables from the pool of initial variables not associated with outcome in crude analysis before applying multivariable logistic regression (on drug exposure status) to generate propensity scores. 1 Variables were selected for inclusion in the final propensity score multivariable logistic regression model using log-likelihood ratio tests (LRTs) for goodness of fit. Starting from an initial fully adjusted model that included all initial variables found to be associated with outcome, goodness of fit was tested after removing variables sequentially from the logistic regression model (starting with the variable most weakly associated with exposure in the fully adjusted model). Variables with a LRT p-value > 0.1 were removed from the model until all variables remaining in the model had a LRT p-value < 0.1. These remaining variables were the final variables that we used to calculate the propensity score. Separate propensity scores were developed in this way for each outcome. Standardised differences were used to assess any residual imbalances after matching (with a standardised difference > 0.1 indicating substantial/important imbalance). 18

The variable list used for the propensity score model obtained in objective 1 was the basis for propensity score modelling in objectives 2 and 3, but additional variables from the pool of initial variables were also considered, given the different nature of the patient populations being studied in these objectives. We also assessed the impact of adjusting for the propensity score (rather than matching) for these analyses.

Methods of analysis

For all objectives, comparisons were made according to FP-SAL (or other drugs being analysed, as specified in Exposures) status for rate of COPD exacerbation, pneumonia and mortality over 3 years. All analyses were performed according to the ‘intention-to-treat’ principle (as was carried out in the TORCH trial3), meaning that if a participant entered the study as either an exposed or an unexposed participant then they would remain assigned to that exposure category for the entire duration of their follow-up (irrespective as to whether or not their true exposure status changes). For exacerbations, a negative binomial model was used, accounting for variability between patients in the number and frequency of exacerbations, with the number of exacerbations as the outcome and the log of treated time as an offset variable. Time to mortality and treatment change was analysed using Cox proportional hazards regression. Risk of pneumonia was analysed using Poisson regression. This mirrors the TORCH trial3 end points of major benefit and harm. We anticipated that the propensity matching process would allow us to assemble treated and untreated groups that were very similar with respect to baseline characteristics, except FP-SAL treatment status. However, this was tested by assessing standardised differences for each baseline variable. If substantial differences were noted for important variables, we made further adjustments to the statistical models. This could also include examining the effect of using a greedy matching approach (i.e. where once a match is made it is fixed) compared with an optimal matching approach (i.e. where the algorithm reconsiders all previously made matches before making a new match) to obtain the closest propensity score match and/or matching at a ratio other than 1 : 1. 19

Validation of results against the TORCH trial3

Our findings were validated against the TORCH trial3 as part of objective 1 by determining whether or not results of the CPRD FP-SAL compared with no FP-SAL treatment analysis were compatible with the TORCH trial3 exacerbations rate ratio for FP-SAL compared with placebo [0.75, 95% confidence interval (CI) 0.69 to 0.81]. This outcome was selected because it is an outcome of key significance for people with COPD5 and the result in TORCH trial3 shows a clear benefit with 95% confidence limits of < 1. We set two criteria that needed to be met for us to conclude that results were consistent. First, the effect size needed to be clinically comparable with TORCH trial3 findings (i.e. the rate ratio for exacerbations in the CPRD had to be between 0.65 and 0.9). This range was deliberately not symmetrical around the TORCH trial3 estimate of 0.75, as we anticipated that the treatment effect in routine clinical care would be weaker than that seen in the optimised setting of a randomised trial. We recognised that this rule could be met with a poorly powered, inconclusive result, and so a second criterion was that the 95% CI for the rate ratio had to exclude 1. For the FP-SAL with SAL alone comparison (see Exposures), the 95% CI also needed to exclude 1 and the rate ratio had to be between 0.81 and 0.95 (compared with the TORCH trial3 FP-SAL vs. SAL result of 0.88, 95% CI 0.81 to 0.95).

Sensitivity analyses

Participants

FP-SAL exposed compared with FP-SAL unexposed

Between 1 January 2004 and 1 January 2017 there were 125,671 people in the CPRD with a diagnosis of COPD, 73,889 (59%) of whom were from HES-linked CPRD practices (Figure 3). Application of TORCH trial3 inclusion criteria reduced this to 18,715 people, contributing 35,746 unexposed-to-FP-SAL time periods and 26,390 exposed-to-FP-SAL time periods. After applying TORCH trial3 exclusion criteria, dropping records with missing covariate data and matching the unexposed patients to TORCH trial3 participants, there were 4196 unexposed patients available for propensity score matching to 10,463 FP-SAL-exposed time periods. The final propensity score-matched cohorts included 2652 patients in each exposure group for the exacerbations analysis, 2708 patients in each exposure group for mortality and 2779 patients in each exposure group for pneumonia.

FIGURE 3.

Flow of number of individuals included in the exposed to FP-SAL vs. unexposed to FP-SAL cohort analysis. Note that the current/previous use of COPD drugs relates to any of the drugs studied in the TORCH trial,3 long-acting bronchodilators and OCSs (see Box 1). SES, socioeconomic status.

FP-SAL exposed compared with SAL exposed

For the FP-SAL compared with SAL analysis, there were 154,785 people with a diagnosis of COPD in the CPRD between 1 January 2000 and 1 January 2017, 91,733 (59%) of whom were from HES-linked CPRD practices (Figure 4). A total of 1146 SAL-exposed patients were available for propensity score matching to 11,235 FP-SAL-exposed periods. The final propensity score-matched cohorts included 991 (exacerbations), 432 (mortality), 935 (pneumonia) and 996 (treatment discontinuation) patients per exposure group.

FIGURE 4.

Flow of number of individuals included in the exposed to FP-SAL vs. exposed to SAL cohort analysis. SES, socioeconomic status.

Application of TORCH trial3 inclusion/exclusion criteria and matching to the TORCH trial3

Applying the TORCH trial3 inclusion/exclusion criteria and matching to the TORCH trial3 resulted in cohorts that were much more similar to those recruited to the TORCH trial3 (e.g. FEV₁% of predicted for the FP-SAL vs. unexposed to FP-SAL analysis was 66.3 in the CPRD before applying any criteria or matching, compared to 47.2 after these steps, compared to a TORCH3 placebo group value of 44.2) (see Table 3). The largest residual difference to the TORCH trial3 placebo group was for prior cardiovascular disease for both comparisons (Tables 3 and 4).

Propensity score matching of Clinical Practice Research Datalink cohorts

Details of the variables included in the final propensity score models are provided in Table 5.

FP-SAL exposed compared with FP-SAL unexposed

Prior to propensity score matching, for the exacerbations, analysis differences by exposure status were noted for sex, FEV₁, BMI, prior exacerbations, coronary heart disease, peripheral vascular disease, cerebrovascular disease, prescriptions for aspirin, COPD medications, number of general practitioner (GP) consultations and number of distinct medications (Table 6). After propensity score matching, only the differences with respect to coronary heart disease, peripheral vascular disease and LABA persisted (see Table 6). Plots of propensity score distributions indicated close propensity score matching for exacerbations and all other outcomes under study (Figure 5).

TABLE 6 - Characteristics of the exposed to FP-SAL vs. unexposed to FP-SAL cohort before and after propensity score matching for the exacerbations analysis (CPRD non-interventional population)

IQR, interquartile range; SD, standard deviation.

See Table 5 for list of variables included in final exacerbations propensity score model. Variables in this table that were included in the propensity score are in bold.

TORCH trial3 inclusion/exclusion criteria applied and matched to TORCH trial3 individual patient data.

TORCH trial3 inclusion/exclusion criteria applied.

Closest record prior to the index date.

All counted within the year prior to the index date, includes exacerbations recorded in primary or secondary care.

Any diagnosis for condition prior to the index date.

Number of people who had at least one prescription within the previous year.

Single product only.

Combination product.

Variable	Before propensity score matching			After propensity scorea matching
	Unexposed to FP-SALb (N = 4196 people)	Exposed to FP-SALc (N = 10,463 exposed time periods from 4259 people)	Standardised difference	Unexposed to FP-SAL (N  = 2652 people)	Exposed to FP-SAL (N  = 2652 people)	Standardised difference
Age (years), median (IQR)	67 (61–73)	68 (62–74)	0.103	68 (61–73)	68 (62–74)	0.083
Sex (male), n (%)	3175 (76)	6515 (62)	0.293	1868 (70)	1850 (70)	0.015
Lung functiond
FEV1 per cent of predicted, median (IQR)	47 (38–56)	50 (40–60)	0.297	49 (39–57)	48 (38–56)	0.024
FEV1 : FVC per cent, median (IQR)	53 (44–61)	53 (44–63)	0.073	53 (44–62)	52 (43–61)	0.045
BMI (kg/m 2 ), d median (IQR)	26 (22–29)	26 (23–31)	0.191	26 (23–30)	26 (22–30)	0.024
Prior exacerbations,e mean (SD)	0.51 (0.92)	0.66 (1.13)	0.148	0.56 (0.96)	0.62 (1.04)	0.060
Cardiovascular disease, n (%)f
Coronary heart disease	1114 (27)	1783 (17)	0.232	720 (27)	441 (17)	0.257
Peripheral vascular disease	390 (9)	648 (6)	0.116	253 (10)	166 (6)	0.122
Cerebrovascular disease	434 (10)	714 (7)	0.126	212 (8)	222 (8)	0.014
Other atherosclerosis	11 (0)	20 (0)	0.015	7 (0)	7 (0)	0.008
Statin prescription, n (%) g	2066 (49)	4614 (44)	0.103	1227 (46)	1238 (47)	0.008
Aspirin prescription, n (%)g	1563 (37)	3129 (30)	0.156	954 (36)	828 (31)	0.101
Other COPD medication prescriptions, n (%)g
LABAh	295 (7)	333 (3)	0.175	197 (7)	106 (4)	0.148
ICSh	530 (13)	842 (8)	0.151	280 (11)	333 (13)	0.063
LAMAh	1450 (35)	6284 (60)	0.528	1166 (44)	1177 (44)	0.008
ICS plus LABAi	526 (13)	488 (5)	0.284	196 (7)	258 (10)	0.084
Type 2 diabetes, n (%)f	543 (13)	1496 (14)	0.040	373 (14)	337 (13)	0.04
History of cancer, n (%)f	696 (17)	2105 (20)	0.091	486 (18)	451 (17)	0.035
Chronic kidney disease, n (%)f	540 (13)	1477 (14)	0.037	389 (15)	333 (13)	0.062
Health-care utilisation, median (IQR)e
Number of GP consultations	21 (15–29)	16 (10–26)	0.409	18 (14–29)	16 (10–26)	0.143
Number of distinct medications	4 (2–7)	5 (3–8)	0.180	4 (2–7)	5 (3–8)	0.073
Number of hospitalisations	0 (0–1)	0 (0–1)	0.008	0 (0–1)	0 (0–1)	0.007
Number of hospital procedures	0 (0–0)	0 (0–1)	0.022	0 (0–0)	0 (0–0)	0.011

FIGURE 5.

Propensity score distributions before and after matching. (a) FP-SAL exposed (n = 10,926 before matching) vs. FP-SAL unexposed (n = 4391 before matching); and (b) FP-SAL exposed (n = 11,235 before matching) vs. SAL exposed (n = 1146 before matching). Treatment discontinuation not included for these analysis as only one of the exposure groups was receiving treatment.

Main results

Analysis of impact of (1) TORCH trial3 matching and (2) TORCH trial3 criteria (post hoc analysis)

Repeating the FP-SAL compared with SAL analysis and omitting the TORCH trial-matching3 step led to an exacerbations rate ratio of 0.87 (95% CI 0.81 to 0.94) (Table 10), which is very similar to both the main analysis and the TORCH trial3 result. By contrast, neither applying the TORCH trial3 criteria nor matching led to a completely different effect estimate (rate ratio 1.64, 95% CI 1.52 to 1.77).

Comparison with previous studies

Implications and further work

When studying COPD treatment effects, if (1) the analysis is of active comparators, (2) trial exclusion and inclusion criteria are applied and (3) the propensity score models that we developed for each outcome are applied to balance exposure groups, then the results of studies carried out in routinely collected non-interventional data can be considered robust in the sense that they will be highly comparable to trial results. This now provides a methodological framework for being able to analyse COPD drug treatment effects in real-world data, focusing on groups that were either not included or under-represented in trials. 1

Our inability to replicate placebo-controlled analyses suggests uncontrolled confounding by indication, a well-known bias in pharmacoepidemiology that is highly likely to be present when performing a comparison between people prescribed a drug and people not prescribed a drug. 45–48 An established design approach for addressing this bias is to perform an active comparator analysis (i.e. comparing the effects of one medicine with another, rather than one medicine with no treatment). 45–48 Based on the likelihood of confounding by indication having an impact on our results compared with the results of no treatment, we proceeded to perform the active comparator analysis and obtained results very similar to the trial, which indicates that confounding by indication is highly likely to be the reason for being unable to replicate placebo-controlled analyses in this setting.

One possibility for how this confounding by indication may be manifesting relates to an aspect of our study design that allowed people to be included in both the exposed and unexposed cohorts (i.e. the result we obtained could be strongly influenced by people initially in the unexposed group who are relatively healthy but then get more sick over time and require FP-SAL treatment and end up in the exposed group). However, in a post hoc analysis in which we dropped the 730 people (out of a total of 2652 per group) who appeared in both cohorts, our effect estimate was nearly identical (RR 1.33, 95% CI 1.20 to 1.47). We do consider, however, that because COPD treatment is based on a step-up approach, it is highly likely that patients not exposed to FP-SAL in routine primary care are generally likely to be those with milder COPD.

An additional point that further explains our inability to replicate the placebo-controlled analysis relates to the large difference in incidence rate between the TORCH trial3 placebo group (1.13 exacerbations/person/year) and our FP-SAL-unexposed group (0.53 exacerbations/person/year). To investigate underlying reasons for this discrepancy, we performed a post hoc analysis in which we compared the characteristics of the 1753 people from TORCH trial3 who were not able to be matched to our unexposed-to-FP-SAL population in step 3 with those who were successfully matched. We found that those not matched were younger (mean age 60.7 vs. 65.8 years), more sick (history of cardiovascular disease: 93% vs. 46%), had worse lung function (FEV₁ 34.9 vs. 45.9) and included a higher proportion of people recruited from Eastern European trial sites (27% vs. 17%). People with these characteristics may have been highly suitable for recruitment to the TORCH clinical trial,3 but are very difficult to find in UK primary care, providing another reason why we were not able to replicate placebo-controlled analyses. Furthermore, both TORCH trial3 placebo-assigned patients and patients in our own cohort were permitted to use other COPD treatments during follow-up, but given the time and setting differences between our FP-SAL-untreated group and the TORCH trial3 placebo group, people in our placebo group are much more likely to have been prescribed, for example, an ICS during follow-up. More generally, it is also likely to be challenging to obtain comparable absolute rates in emulated cohorts within a single country based on historical international trials for reasons such as this.

Previous authors have gone as far to specifically recommend that when trying to emulate trial results it is important to choose an active comparator trial. 34 There are, however, examples in which placebo-controlled analyses have been successfully replicated. 30,32 One possibility is that replication of placebo-controlled results works better when the drug studied is (1) preventative and (2) used in a generally healthy cohort (e.g. the cited studies were of statins and of postmenopausal hormone therapy both prescribed, in some instances, to people without a specific underlying chronic disease, in contrast to the patients with COPD who received therapy in our study). We consider that further avenues of research could be followed to understand if there remains a possibility of replicating placebo-controlled studies within a non-interventional setting for COPD therapies. These could include the application of high-dimensional propensity scores or the use of instrumental variables. Our work also suggests that treatment discontinuation in the setting of non-interventional data may be driven by very different factors to those seen in trials and, at least in the setting of COPD, may not be a useful outcome to study. Interpretation of treatment discontinuation in routine data is challenging. For example, it is difficult to establish from routinely collected data whether a patient has truly stopped taking their medicine or is just taking the medicine differently than prescribed (e.g. is taking less than has been prescribed over a longer period), a point that was emphasised by our patient and public involvement representative.

Finally, in our post hoc analysis, we found that the application of the trial-matching step did not confer any advantage over the application of trial criteria alone in this setting. This suggests that treatment–covariate interactions are not as critical as we initially thought in this therapeutic area.

Limitations

Some of the TORCH trial3 inclusion criteria were not fully assessable using CPRD data, meaning that the inclusion/exclusion criteria are analogous with TORCH trial3 criteria, but we acknowledge that they are not identical. For example, TORCH trial3 inclusion criteria included negative reversibility spirometry criteria, but it was not possible to replicate this in the CPRD data. In addition, at entry to the TORCH trial3 2-week run-in period all ICSs and inhaled long-acting bronchodilators were discontinued, but it was not possible to ensure that all patients selected from the CPRD had discontinued these at the start of the run-in period that we applied.

We originally planned to apply frequency of COPD therapy prescriptions in the previous year as a matching character/criterion. In practice, this was not feasible. However, it appears that matching at this level of detail was not required to be able to replicate trial results for active comparator analysis. Furthermore, it is clear from Figures 3 and 4 that we had to drop around one-fifth of patients as they did not have spirometry measurements recorded in the CPRD. This does mean that our initial study population may be missing key groups of people who tend not to have their spirometry measured (e.g. people with COPD who have the least contact with the health services). This is a problem for any COPD research performed using routinely collected primary care data, and although we acknowledge the issue we think that in this work it is likely to have minimal impact, as our aim was to create trial-analogous exposure groups that were highly comparable to each other, not to select a highly generalisable sample.

Finally, within the TORCH trial,3 the dose of the fixed combination product FP-SAL was specified as 500 µg of FP and 50 µg of SAL (500/50) and the dose of SAL alone as 50 µg, whereas in our study we did not limit to a specific dose. The reason for this is that dosage information is incompletely captured in the CPRD. These are the only approved doses of FP-SAL and of SAL for COPD in the UK and so we consider the doses that people were prescribed in our study would have been generally similar to that administered in the TORCH trial,3 although we do acknowledge that there are likely to be some differences in dosing between the TORCH trial3 and our cohort because of the long-term management of patients with varying disease severity and varying concomitant conditions.

Participants

After applying the TORCH trial3 exclusion criteria, as outlined in part 1, to the 91,733 people from HES-linked CPRD practices with a diagnosis for COPD between 1 January 2000 and 1 January 2017 but with exclusion criteria altered relating to (1) aged > 80 years, (2) presence of asthma or (3) selection of people with milder COPD based on spirometry, the final cohort sizes before and after propensity score matching were as shown in Table 11.

Propensity score matching of cohorts of people who would have been excluded from the TORCH trial3

Details of the variables included in the final propensity score models are provided in Table 12. As described in Chapter 2, Propensity score for addressing confounding, variables from the pool of initial variables that were not associated with the specific outcome under study were not included in the multivariable logistic regression model used to generate the propensity score. This, therefore, meant that the final propensity score models for each of the outcomes under study could contain a different set of variables, depending on the outcome being studied. For example, for people with milder COPD, the mortality propensity score included sex and age, but these variables did not end up in the final propensity score for exacerbations or pneumonia. Although sex and age were associated with each of these outcomes in crude analysis, exacerbations and pneumonia were dropped from the subsequent multivariable logistic regression models that had drug exposure as an outcome (specified in Chapter 2, Propensity score for addressing confounding) because of the lack of association with exposure after multivariable adjustments.

Main results

Table 16 shows the results for the association between FP-SAL and COPD exacerbations, mortality and pneumonia in the patient populations aged > 80 years, those with asthma and those with mild COPD. Crude associations and associations from propensity score-matched and propensity score-adjusted analyses are shown.

Comparison with previous studies

Implications and further work

Our propensity score-matched results for the > 80-year-old cohort suggest that the effect of FP-SAL (vs. SAL) on exacerbations is of a similar protective magnitude as it is in a cohort of people with similar characteristics to those in the TORCH trial3 (RR 0.59, 95% CI 0.36 to 0.95 in the > 80-year-old group compared with RR 0.85, 95% CI 0.74 to 0.97 in our TORCH trial-analogous3 cohort). This provides some reassurance of the effectiveness of this treatment in this population in a UK primary care setting. Further work could be to extend this analysis to other drug treatments (e.g. looking at inhaled triple combination therapy LAMA + LABA + ICS, as recently added to NICE guidance5) and also assessing whether or not the same effect is seen when using the same methods applied in a different setting (e.g. an EHR database from another country). For the mortality and pneumonia effects, although both our propensity score-matched and our propensity score-adjusted associations were consistent with those found in our TORCH trial-analogous3 analysis, there were wide CIs. One possibility could be to repeat these analyses and include the additional EHRs now available in CPRD Aurum. 58

For our asthma cohort, the increased mortality observed in the FP-SAL group (in contrast with the null effect found in our TORCH trial3 analysis) could be a spurious result, possibly due to imbalances in the cohort that were introduced by our propensity score method (although from Appendix 1, Table 19, the only observed imbalance introduced to matching is a small difference in FEV₁). Further work is, therefore, needed to investigate this finding. However, as mentioned previously, people with both asthma and COPD have a poorer prognosis than those with just COPD, and how this has a selective impact on those on FP-SAL compared with those on SAL while having a protective effect on exacerbations and a null impact on pneumonia needs further consideration. Future work (in both CPRD GOLD and other data sources) should focus on identifying any patterns in cause-specific mortality and whether or not this represents any possible causal association with treatment or unadjusted confounding due to underlying differences between treatment groups.

Finally, for the cohort of people with mild COPD, our results indicate that there is a stronger protective effect on exacerbations than those people in the TORCH trial-analogous3 cohort. This is a potentially reassuring result; however, as it has not been seen before, it would be advisable to try to replicate this finding in a completely separate setting using this methodology (e.g. an EHR database containing a completely distinct population to the CPRD).

Limitations

Some of our analyses had very small numbers. In particular, the propensity score-matched groups for those aged > 80 years were all < 100 (see Table 11), despite using one of the largest data sets available for these kinds of analyses. To help address this, we also presented results that were adjusted for the propensity score rather than matched. Although some of the point estimates differed when comparing matched with adjusted estimates, none of our conclusions changed for this group when considering the propensity score-matched cohort with the propensity score-adjusted cohort. Nonetheless, the more recent availability of even larger quantities of EHR data (i.e. CPRD Aurum) will allow for more precise estimates in future and would likely mean that any residual discrepancy between matched and adjusted estimates would be accounted for.

A key limitation of this part of the analysis (that also effects all observational studies) is that we cannot have the same level of certainty regarding causal associations as would be possible in a well-conducted randomised trial. Owing to the nature of the research questions in this part of the project, we cannot know if our results are valid and would replicate those of a reference trial, as for this stage we were not matching our cohorts to actual trial participants or comparing the results to actual clinical trial results. However, as discussed in Analysis of impact of (1) TORCH trial3 matching and (2) TORCH trial3 criteria (post hoc analysis), it was the application of the trial inclusion/exclusion criteria when selecting cohorts that led to trial-comparable results not matching to the individual trial participants, and we did apply the same rigorous trial selection-type approach (described in Chapter 2, Selection of participants) to the selection of people for inclusion in all of the cohorts in this section. Furthermore, our methods for preparing the propensity score models were identical to those used in the TORCH trial-analogous3 analysis (described in Chapter 2, Propensity score for addressing confounding).