Identifying exceptional cystic fibrosis care services: combining statistical process control with focus groups

Cystic fibrosis

Cystic fibrosis (CF) is a common genetic disease, affecting approximately 9000 people in the UK. 1 The genetic mutation leads to mucus obstruction and infection in the lungs. 2 Other organs, such as the liver, intestines and pancreas, are also affected, which, in turn, leads to poor nutrient absorption and malnutrition. 3 The prognosis for patients with CF has improved dramatically over the years, and it was recently shown that there has been a clear increase in the median age at death since the early 1970s. 4 In 2009, the median predicted survival for UK patients was 38 years. 1

People with CF are treated at specialist centres that offer dedicated paediatric or adult care; transition to adult care usually occurs after a person’s 16th birthday. Care involves daily treatments that can include drugs to clear mucus and prevent lung infections, pancreatic enzymes to aid digestion, and physiotherapy.

The CF Trust – the national UK charity for CF – maintains a national registry database (the UK CF Registry) of detailed clinical and demographic information on all patients attending these specialist centres. Patients are invited to attend annual reviews at their specialist centre and, based on these review data, the Trust produces annual reports describing the health of patients. The reports are available on the CF Trust website to be accessed by patients and clinicians alike [www.cysticfibrosis.org.uk/uk-cf-registry/reporting-and-resources (accessed 6 July 2018)].

Centre comparisons

From 2008 to 2014, the annual reports published by the CF Trust included comparisons between centres in terms of key clinical outcomes using simple rankings. Although these comparisons give a sense of the distribution of outcomes between centres, they encourage the reader to assume, for example, that centres with the highest values of lung function measure are ‘better’ than those with lower values. This is misleading on two grounds:

1. The rankings make no allowances for the differences in patient intakes. It could be argued, for example, that centres with younger patients would have higher age- and sex-adjusted lung function measures than centres with older patients, because lung function declines with age in people with CF. As such, between-centre comparisons that do not account for the age of patients risk not detecting good outcomes in centres where patients are older. Similarly, poor outcomes in centres with younger patients may not be detected.

2. The rankings include no formal tests comparing centres. Without formal comparisons, we cannot say that there is evidence that outcomes are better at one centre than another, let alone conclude that any observed differences are related to the processes of care.

Such shortcomings are important if the reader interprets the CF Trust graphs as league tables. Marshall and Spiegelhalter5 showed that league tables – even after adjustment for patient intake – are unreliable in identifying poor performance. Furthermore, Lilford et al. 6 suggest that league tables – in which a centre’s position could be caused by a host of reasons, such as differences in definitions and data quality between providers, chance or inadequate case-mix adjustment to differences in institutional management – can generate undue tension as blame is attributed without identification of the reason. After all, the very nature of a league table is that one provider is ranked bottom, whether or not care is poor, and ranking may simply be due to chance. 7 Without providing any explanation for observed differences, we are no further forward in knowing how to improve care, which should be the ultimate aim of any institutional comparisons.

Quality improvement in cystic fibrosis

Quality improvement programmes of varying sorts are not new in CF: there is a long history of such programmes in the USA8,9 and they have been described in Germany. 10 In the USA, the national CF Patient Registry performs centre comparisons in its routine reporting and this has also encouraged local units to undertake their own within-centre quality improvement work, making use of the national registry data. 11,12 However, in the UK there has been no work testing the hypothesis that there are key differences in the delivery of care that affect outcomes. We therefore adopted a ‘quality improvement’ approach to understanding whether or not differences between centres are related to the processes of care, which builds upon and improves on existing methodologies in CF centre comparisons. In this study, we considered all centres in our analyses and used statistical process control (SPC) charts to determine whether or not there are statistically meaningful differences. These charts were developed as a means of studying the variability of ‘processes’ over time13 and have been shown to be useful when comparing health-care providers (whether hospitals, clinics or primary care practices) as well as the outcomes of single providers (e.g. Harold Shipman) over time. 14

For the SPC charts, summary outcome measures for individual units were plotted against time or the size of the unit, with control limits [either at 2 or 3 standard deviations (SDs)]. It is anticipated that there will be some variability between units, which is intrinsic to the process itself and referred to as ‘common-cause variation’. 13 SPC charts are designed to highlight variability caused by factors outside the process, referred to as ‘special-cause variation’. Centres that have summary measures outside the control limits (either higher or lower) are said to exhibit special-cause variation.

An example of a SPC chart is presented in Figure 1. Each dot represents a different CF unit and these CF units are being compared in terms of the mean forced expiratory volume in 1 second (FEV₁) in children aged 15 years, with the size of the unit on the x-axis, given by the number of 15 year olds with FEV₁ data. The dashed lines represent the control limits. Units located between the limits show ‘common-cause variation’ and those outside the limits show ‘special-cause variation’. Limits of 95% are equivalent to 2 SDs and limits of 99.8% are equivalent to 3 SDs.

FIGURE 1.

Example of funnel plot. FEV₁, forced expiratory volume in 1 second.

Centre comparisons

We used data from the CF Registry to compare CF centres in the UK in terms of key clinical outcomes.

Consultations to identify factors that would help to understand variability

Analyses in young adolescents: 2007–12

Between 2007 and 2012, 9914 unique patients had at least one annual review recorded on the UK CF Registry. This includes 2112 patients aged 12–15 years who attended paediatric centres; 31 paediatric centres contributed data to the CF Registry during this period.

Our prespecified outcomes in children each relate to different age groups:

• 12 years – FEV₁

• 15 years – FEV₁ and BMI percentile

• 13–15 years – FEV₁ change from ages 13 to 15 years.

Given that these outcomes differ in nature (some focus on an outcome at a single age and others focus on changes in outcomes over time), different eligibility criteria are applied to each, as described in Chapter 2.

For each outcome, we described the identified patient population in terms of their demographic characteristics and the completeness of the outcome and case-mix data that they provided (Table 1). In the analysis of each outcome, there was an approximately equal split of boys and girls and most of the patients were taking pancreatic enzyme supplements for pancreatic insufficiency. Just over half of patients were homozygous DF508, whereas < 10% of genotyped patients had no DF508 mutations. Using home postcode data to determine area-level SES, we note that those in the most deprived quintile were under-represented for each outcome analysis.

As these variables were to be used in case-mix adjustment, and only those patients with non-missing data for these variables would be included in case-mix adjustment models, rates and patterns of ‘missingness’ had the potential to bias our results. We therefore sought to quantify and explore variations between centres in rates of ‘missingness’ for these variables. For the variables considered, there were few missing data (< 5%) in any of the analysis populations (inclusion in an analysis population depended in part on having non-missing outcome data).

Outcome: FEV₁ in children aged 12 years

Data were available for a total of 1193 12-year-olds across the 31 paediatric centres. Estimates of the predicted FEV₁ can be obtained using the Knudson reference equations,18 but, more recently, it has become common to use the GLI reference equations. 19 We considered both methods and analysed the data first using the Knudson reference equations. 18

Of the 1193 12-year-olds studied, 87 did not have FEV₁% predicted data available. A funnel plot illustrated the proportion of patients at each centre with non-missing data and showed that, although the vast majority of centres had good levels of non-missingness (≥ 85%), centres 39 and 43 exhibited low levels of non-missingness and were outside the lower 3-SD limits (see Appendix 2, Figure 18a).

The means of the observed (i.e. not case-mix adjusted) FEV₁% predicted of 12-year-olds at each of the 31 CF paediatric centres were plotted into a funnel plot (Figure 2a). This allowed us to look at the differentiation between the centres in terms of their observed FEV₁% predicted. The plot shows eight centres outside the 2-SD control limits, although only four were outside the 3-SD limits. The results suggest that some centres have higher mean FEV₁ values than the average, whereas others have lower mean FEV₁ values than the average. Centres with fewer patients tended to have slightly lower than average FEV₁ values and the two largest centres had slightly higher mean FEV₁ values. The same pattern was observed when we calculated the FEV₁% predicted using the GLI reference equations (Figure 2c).

FIGURE 2.

FEV₁ at 12 years of age. (a) Mean of the unadjusted FEV₁% predicted (Knudson18); (b) predicted random intercepts of FEV₁% predicted (Knudson18); (c) mean of the unadjusted FEV₁% predicted (GLI19); and (d) predicted random intercepts of FEV₁% predicted (GLI19).

We explored whether or not there were between-centre differences in the case-mix variables (see Appendix 2, Figure 19) using funnel plots. Although there was some variability between centres, this rarely exceeded the 3-SD control limits. The exception to this is centre 43, which had a larger proportion of patients in the most deprived quintile than the other centres (see Appendix 2, Figure 19c).

To assess whether or not the observed between-centre variation was attributable to differences in the distribution of case-mix variables, we generated multilevel models of our outcome adjusted for these factors (see Appendix 3, Table 17). The results of our model suggest, as expected, that males have higher FEV₁ values than females and that patients taking pancreatic enzyme supplements have lower FEV₁ than those not requiring this treatment. There was also some evidence that patients in more socioeconomically deprived areas had lower FEV₁ values than those in the most affluent quintiles. A funnel plot of the random effect variation (see Figure 2b) after case-mix adjustment reveals that, although many of the same centres remain outside the 2-SD control limits as in the unadjusted analysis, centre 39 remains outside the 3-SD limits. These results suggest that differences in case mix between centres may be responsible for some – although not all – of the differences in FEV₁. Similar results were obtained when FEV₁% predicted was calculated using the GLI reference equations (see Figure 2d).

Outcome: FEV₁ in children aged 15 years

Data were available for a total of 1151 15-year-olds across the 31 paediatric centres. Of these, only 85 did not have FEV₁% predicted values.

Funnel plots illustrated the proportion of patients at each centre with non-missing FEV₁% predicted and it was observed that, although the vast majority of centres had good levels of non-missingness (≥ 85%), as was observed with 12-year-olds, centres 43 and 39 exhibited low levels of non-missingness (see Appendix 2, Figure 20a).

The means of the observed FEV₁ of 15-year-olds at each of the 31 paediatric centres were plotted into a funnel plot (Figure 3a). Eight centres were outside the 2-SD limits and many (centres 12, 26, 31, 39 and 59) were also outside these limits in the analysis of 12-year-olds. In contrast to the plots of 12-year-olds, however, none of the centres was outside the 3-SD limits.

FIGURE 3.

FEV₁ at 15 years of age. (a) Mean of the unadjusted FEV₁% predicted (Knudson18); (b) predicted random intercepts of FEV₁% predicted (Knudson18); (c) mean of the unadjusted FEV₁% predicted (GLI19); and (d) predicted random intercepts of FEV₁% predicted (GLI19).

As was observed in 12-year-olds, case-mix factors differed between centres (see Appendix 2, Figure 21); none of the centres was outside the 3-SD limits.

Our case-mix adjustment model (see Appendix 3, Table 18) showed similar patterns of association for 15-year-olds as was observed for 12-year-olds, in that FEV₁ was lower among females, lower for those with a more deprived SES and lower for those taking pancreatic enzyme supplements. The SD of the between-centre variation of the random effect is comparable between the two models. Inspection of the funnel plots of the predicted random intercepts showed no centres outside the 3-SD control limits and showed that larger centres tended to have FEV₁ values that were slightly greater than the mean (see Figure 3b).

Analyses were repeated using the GLI reference equations19 and the results were very much the same for the unadjusted funnel plot (see Figure 3c) and the funnel plot adjusted for case mix (see Figure 3d).

Outcome: body mass index percentile in children aged 15 years

In total, BMI percentile data were available for 1117 15-year-olds across the 31 paediatric centres. Funnel plots illustrated the proportion of patients at each centre with non-missing BMI (percentile) and it was observed that, although the vast majority of centres had good levels of non-missingness (≥ 85%) (see Appendix 2, Figure 22), one centre (39) had a lower proportion of missingness than the average.

The means of the observed BMI of 15-year-olds at each of the 31 paediatric centres were plotted into a funnel plot (Figure 4a). No centres were outside the 3-SD control limits, but a few were outside the 2-SD limits: centres 31, 52 and 54 had a higher than average BMI percentile and centres 12, 53 and 59 had a lower than average BMI percentile.

FIGURE 4.

BMI percentile at 15 years of age. (a) Mean of the observed BMI percentile for each of the centres; and (b) predicted random intercepts for BMI percentile for each of the centres.

We considered the same case-mix variables as with our FEV₁ analyses and, in case-mix adjusted models, we showed that gender, pancreatic enzyme use and living in the most deprived quintile had a significant effect on lowering a patient’s BMI percentile (see Appendix 3, Table 19).

Inspection of the funnel plots of the predicted random intercepts from the case-mix adjusted model showed no centres outside the 3-SD control limits and only centre 52 near the upper 2-SD limit (Figure 4b).

Outcome: change in FEV₁ in children aged 13–15 years

As one of the aims of CF care is to maintain good lung function, we sought to assess whether or not there were statistically meaningful differences between centres in the rate of change of FEV₁ during the early teenage years (age 13–15). We identified 1180 patients who had had at least two annual reviews with FEV₁% predicted recorded between the ages of 13 and 15 years during the study period. Although most centres contributed data for the vast majority of their patients (see Appendix 2, Figure 23a), centres 39 and 43 had sufficient FEV₁ data to calculate decline for < 60% of their eligible patients. These same centres also had poor completeness of FEV₁ data for 12- and 15-year-olds, as described in earlier sections.

We performed linear regression analyses for each patient, calculating the annual decline in FEV₁ in those aged between 13 and 15 years. These individual changes in FEV₁ were then pooled by centre to generate funnel plots of mean change in FEV₁ by centre. The mean FEV₁ decline across centres was a decline of 1.2% predicted per year (SD = 11.1) and there was equal variability between centres above and below this mean. Although three centres were outside the 2-SD control limits, none was outside the 3-SD limits (Figure 5a).

FIGURE 5.

Change in FEV₁ between 13 and 15 years of age. (a) Mean of the unadjusted FEV₁% predicted (Knudson18); and (b) predicted random intercepts of FEV₁% predicted (Knudson18).

A multilevel model, using the estimated change in FEV₁ as the outcome, was generated to explore the impact of case-mix variables (see Appendix 3, Table 20). Unlike earlier models for FEV₁ at ages 12 and 15 years, none of the case-mix variables was statistically significant. Furthermore, the between-centre SD was 0.58, suggesting minimal difference between centres. This was confirmed in a funnel plot of the predicted random intercepts (Figure 5b), in which all centres were very close to the mean.

As described in Chapter 2, when using multilevel modelling for case-mix adjustment it is possible that the model draws centre estimates to the overall mean in a process known as ‘shrinkage’. This can give the false impression that it is the case-mix variables that are responsible for differences between centres.

To explore the potential role of shrinkage in our case-mix adjusted analysis, we re-ran the adjustment using fixed-effects GLMs. This analysis showed one centre to be outside the 3-SD limits (centre 54; see Appendix 2, Figure 24). The fixed-effects case-mix adjustment model explained very little of the variability in the outcome (R² = 0.0071), however, making it difficult to infer what the centre-level estimate means.

Analyses in young adolescents: 2007–15

Our analyses of children aged 12 and 15 years – described in the previous three sections – were repeated with the addition of annual review data from 2013 to 2015. When these data became available to researchers, we used them to see if our previous findings were altered by the inclusion of the new data. We did not explore lung function decline between 13 and 15 years of age as there was no evidence of special-cause variation in the case-mix adjusted results using multilevel modelling; we felt that it was highly unlikely that this would change with the addition of further data.

Outcome: FEV₁ in children aged 12 years (2007–15)

With the addition of the 2013–15 data, we had annual review data on 1853 12-year-olds attending paediatric CF centres. Of these, 1737 (93.7%) had non-missing FEV₁% predicted data. These patients came from the same centres as studied previously, but now each centre contributed the data of more patients to the analysis.

The means of the observed (i.e. not case-mix adjusted) FEV₁% predicted were plotted into a funnel plot (Figure 6a). As with the analysis of the 2007–12 data, the plots showed eight centres to be outside the 2-SD control limits, although these were not the same centres as before (see Figure 2a). Only two of the centres (12 and 39) were outside the 3-SD limits, and these remained outside after case-mix adjustment (Figure 6b). In our original analysis, centre 25 was on the upper 3-SD control limit after case-mix adjustment (see Figure 2b), but, when the later data were added, this centre was drawn closer to the overall mean.

FIGURE 6.

FEV₁ at 12 years of age using data from 2007–15. (a) Mean of the unadjusted FEV₁% predicted (Knudson18); and (b) predicted random intercepts of FEV₁% predicted (Knudson18).

As in the analysis of change in FEV₁, we explored the role of shrinkage by generating a funnel plot of the ‘percentage predicted’; the outcome is presented in Appendix 2, Figure 25. This model shows some consistency with Figure 6b, in which, after case-mix adjustment, centres 12 and 39 remained outside the 3-SD control limits. Interestingly, smaller centres such as 23 and 26 – which, when using multilevel models for case-mix adjustment, moved from between the lower 2- and 3-SD control limits to well within the 2-SD limits – remained outside the 2-SD limits when the role of shrinkage was explored, but they were never outside the 3-SD control limits. Our results suggest, however, that case mix was not necessarily the driver in moving these centres closer to the overall mean.

Outcome: FEV₁ in children aged 15 years (2007–15)

With the addition of the 2013–15 data, we had annual review data on 1844 15-year-olds attending paediatric CF centres. Of these, 1736 (94.1%) had non-missing FEV₁% predicted data.

The means of the observed FEV₁ of 15-year-olds at each of the 31 paediatric centres were plotted into a funnel plot (Figure 7a). Eleven centres were outside the 2-SD limits, including all of those identified in the earlier analysis using data from 2007–12 (see Figure 3a). Using the more recent data, however, moved three of the centres outside the 3-SD control limits (centres 12, 31 and 39), in addition to bringing three centres outside the 2-SD limits for the first time (centres 18, 25 and 61). After case-mix adjustment, many of these centres were brought within the 2-SD limits, but centres 12 and 39 remained outside the lower 3-SD limit.

FIGURE 7.

FEV₁ at 15 years of age using data from 2007–15. (a) Mean of the unadjusted FEV₁% predicted (Knudson18); and (b) predicted random intercepts of FEV₁% predicted (Knudson18).

We explored the effect of shrinkage on our results (see Appendix 2, Figure 26). As seen in the earlier case-mix adjusted analysis in Figure 7, centres 12 and 39 had FEV₁ values outside the lower 3-SD control limit. Centre 31, however, shifted slightly to within the 3-SD limits.

Outcome: body mass index percentile in children aged 15 years (2007–15)

Incorporating data from 2013–15 increased the number of 15-year-olds with non-missing BMI percentile data to 1804. As in the original analysis, there were no centres outside the 3-SD control limits (Figure 8a) and the same centres tended to be outside the 2-SD limits as in the original analysis (see Figure 4). The only differences were centre 52, which was no longer outside the upper 2-SD limit, and centre 44, which moved outside the upper 2-SD limit. Case-mix adjustment largely brought centres closer to the mean and in the updated analysis only centre 59 was outside the 2-SD control limits after case-mix adjustment.

FIGURE 8.

BMI percentile at 15 years of age. (a) Mean of the observed BMI percentile for each of the centres; and (b) predicted random intercepts for BMI percentile for each of the centres.

As seen in Figure 8, when we used fixed-effects modelling to explore the role of shrinkage, we found that none of the centres was outside the 3-SD control limits (see Appendix 2, Figure 27).

Analyses in all children aged 6–15 years, by year (2007–15)

To complement the analyses of outcomes in targeted age groups that were carried out by pooling data across multiple years, we conducted all-age analyses by year. By exploring outcomes in all ages year by year, we can see whether or not there are trends in clinical outcomes.

For the analyses of paediatric centres we focused on FEV₁% predicted in all patients aged 6–15 years, excluding those patients who had previously had a lung transplant as these patients are likely to have much better lung function. First, we used FEV₁, measured using the Knudson reference equations18 as reported in the UK CF Registry, and, second, sensitivity analyses were performed (see Global Lung Function Initiative reference equations) using the GLI reference equations. 19

We considered data from 2007 to 2015 inclusive; the number of patients having annual reviews, and those providing FEV₁ data at these reviews, had changed over that period, reflecting overall improvements in the completeness of the UK CF Registry, as highlighted in Table 2.

Unadjusted analyses

Centre-level analyses unadjusted for any case-mix factors illustrated that centres varied widely in size and that outcomes varied between centres (Figure 9a and b).

FIGURE 9.

Funnel plots of FEV₁ unadjusted for case mix, by year. (a) 2007; (b) 2008; (c) 2009; (d) 2010; (e) 2011; (f) 2012; (g) 2013; (h) 2014; and (i) 2015.

Each year’s plot was investigated to see whether or not centres appeared inside or outside the 2- or 3-SD control limits. We also took note of centres that ‘hovered’ near control limits without being outside those limits, so that we could monitor whether or not they had crossed the control limits previously or in subsequent years. These results are summarised in Table 3.

TABLE 3 - Centres showing evidence of special-cause variation and ‘borderline’ centres, by year

Centres approaching the 2-SD control limit without being outside it.

Year	Centres with lower than average FEV1 values		Centres with higher than average FEV1 values
	Outside lower 3-SD limit	Outside lower 2-SD limit	Outside upper 2-SD limit	Outside upper 3-SD limit
2007	12, 52 and 59	11, 16,a 23 and 53a	14, 31, 36, 37 and 54	25
2008	12 and 52	1, 13, 18 and 26	36 and 44	25 and 28
2009	11, 12 and 52	16, 23, 43 and 59	36, 46 and 60	25 and 31
2010	12 and 39	23, 52 and 59	36, 46a and 54a	31
2011	12 and 23	18 and 39	31a and 61	–
2012	39	23,a 26, 50a and 52a	25, 31,a 43 and 54	28
2013	39	23, 26, 50 and 53	25, 28, 31 and 61	60
2014	53	25, 39, 50 and 52	13, 28, 31, 44, 60 and 61	–
2015	53	25, 26, 39,a 50 and 52a	16, 28, 31, 43, 60a and 61	–

We note that a large number of centres appear between the 2- and 3-SD limits and few appear to be outside the upper 3-SD limit, which indicates higher than average FEV₁ values. The patterns in the data suggest that some centres appear outside (or near) the 3-SD control limits repeatedly, and we focused our attention on these.

• Centres with lower than average FEV₁ values: the centres appearing outside the lower 3-SD limit most frequently were centres 12, 39 and 52. Centres 12 and 52 were outside the limits in the early years (2007–11) but subsequently moved to within the limits and sometimes within the 2-SD limits. Centre 39, in contrast, was outside the 3-SD limits in the later years (2010, 2012 and 2013) and similarly moved within the 2-SD limits outside these years.

• Centres with higher than average FEV₁ values: the centres appearing outside the upper 2- or 3-SD limits most frequently are centres 25, 28 and 31. From 2013, however, they were generally between the 2- and 3-SD limits.

As was done with the earlier age-specific analyses, we conducted a pyramid of investigation approach to understanding the data.

Pyramid of investigation: case mix

Age

Earlier analyses of FEV₁ in select, narrow age groups did not account for age, but FEV₁ declines with age and – as the mean age of patients varies slightly between centres (see Appendix 3, Table 21) – it is essential to adjust FEV₁ accordingly.

Statistical adjustment should account for the documented non-linear association between FEV₁ and age. To do this, for each year of analysis we ran a GLM of FEV₁% predicted across age as a categorical variable (with 6 years as the referent group) and from this obtained model-predicted values. Our age-adjusted FEV₁% predicted was then calculated using the ratio of the original FEV₁% predicted to the values predicted by the regression model multiplied by 100. The results are described in Appendix 2, Figure 28, and Appendix 3, Table 22, and show very similar patterns to the unadjusted analysis, with some centres moving from outside to inside the 3-SD limits. Notably, centre 28, which had previously been outside the upper 3-SD limits, was now consistently within these limits and, thus, closer to the mean.

We also ran a separate analysis simply incorporating age as a linear term into multilevel models and we used the random intercept predictions, which estimate the difference between the overall mean and the centre-level mean, after adjustment for age. As a result, the y-axes for this graph (see Appendix 2, Figure 29) will be different from those in earlier funnel plots (see Appendix 2, Figure 28). In both cases, however, we are looking at differences from the mean. The results from this analysis are broadly similar to those of the other adjustment (for age), but in various years brought some centres outside the 3-SD limits to either within these or within the 2-SD limits (see Appendix 2, Figure 29, and Appendix 3, Table 23). The centres affected by this were centres 12, 23, 31, 50, 53, 54 and 60.

Additional case-mix variables

Beyond age, a number of other factors that may influence FEV₁, which are not influenced by care and which are collected on the UK CF Registry, as outlined below.

Sex (see Appendix 3, Table 24)

A number of studies have shown that females have poorer CF outcomes than males. We explored whether or not centres with better outcomes have a lower proportion of female patients. We observed that approximately 50% of patients were female in most centres.

Genotype: homozygous DF508 (see Appendix 3, Table 25)

As CF is a recessive genetic disease, every effort is made to have all patients genotyped, and this information is recorded on the UK CF Registry. The most common genotype in the UK is DF508, and it has been shown in many studies that those who have two of these mutations (i.e. are homozygous for DF508) have poorer outcomes. In those centres with lower than average FEV₁ values, the proportion of patients who have homozygous DF508 was generally ≥ 50%. There appeared to be less variability between the centres with higher than average FEV₁ values. If we look uniquely at 2015, half of the centres had just under 50% homozygous DF508 patients and half had slightly more.

Genotype: any G551D mutation (see Appendix 3, Table 26)

In 2012, a new CF therapy called Ivacaftor became available in the UK for patients with a G551D mutation. Clinical trials showed promising results and uptake of the drug was swift in those patients who were eligible to receive it. There is also some evidence26 suggesting that patients with at least one G551D mutation have less severe CF than those who are homozygous DF508. The mutation was not very common in the centres selected for analysis here: in 2015, the centres with the highest prevalence of the mutation were centre 31 (10.6%) and centre 39 (8.2%). The former had higher FEV₁ than the average and the latter had lower FEV₁ than the average.

Pancreatic sufficiency (see Appendix 3, Table 27)

A marker of CF severity is pancreatic sufficiency (i.e. pancreatic enzymes are not required to digest food). Here we considered the proportion of patients who were pancreatic sufficient. We noted that in centres with higher than average FEV₁ values, the proportions of patients who are pancreatic sufficient varied widely. For example, there was a near twofold difference in the proportion of patients who are pancreatic sufficient between centres 25 and 31, although both report higher than average FEV₁ values. In centres with lower than average FEV₁ values, there was variability between centres, but pancreatic sufficiency was rarely as common as in centre 31.

Socioeconomic status (see Appendix 3, Table 28)

As with many conditions, there is evidence that individuals from more disadvantaged backgrounds have poorer outcomes. Here, SES was summarised in quintiles: we explored what proportion of patients were in the fifth (most deprived) quintile. All other things being equal, we would expect that 20% of patients in a given centre would be in any given quintile. Although this was observed in some centres (such as centres 39 and 52, which had lower than average FEV₁ values, and centre 28, which had higher than average FEV₁ values), patients in the most deprived quartile were under-represented in other centres reporting lower than average FEV₁ values (centre 12) and higher than average FEV₁ values (centre 31). Contrary to expectation, centre 36, which reported higher than average FEV₁ values, had a consistently high proportion of patients (> 30%) from the most deprived quintile.

All of these results suggest that case-mix factors shown in the literature to be associated with FEV₁ differ between centres, although not in entirely consistent ways. Adjustment for these factors may be helpful in explaining some of the associations seen in our earlier funnel plot.

Case-mix adjusted analysis

All case-mix variables were included in multilevel models and adjustment for age was undertaken, adopting the approaches described earlier in this section. Both approaches gave largely similar results, as described in Tables 5 and 6 and Figures 10 and 11. Centre 31 moved from outside the upper 3-SD limits to within these limits in 2010 and centre 28 moved within the upper 3-SD limits in 2012. Conversely, centre 25 moved outside the 3-SD limits in 2013, although this was seen only in the analysis that adjusts for age using a GLM.

TABLE 5 - Centres showing evidence of special-cause variation and ‘borderline’ centres in analyses adjusted for age (using GLMs), sex, pancreatic sufficiency, genotype (homozygous DF508, heterozygous DF508 or other) and SES quintile, by year

Centres approaching the 2-SD control limit without being outside it.

Year	Centres with lower than average FEV1 values		Centres with higher than average FEV1 values
	Outside 3-SD limit	Outside 2-SD limit	Outside 2-SD limit	Outside 3-SD limit
2007	52 and 59	12 and 23	31 and 54a	25
2008	12 and 52	–	28, 44 and 50a	25
2009	12 and 52	1 and 59	31, 36, 46a and 50a	25
2010	12 and 39	23, 52a and 59	25, 31, 36a and 44a	–
2011	–	1,a 12, 23a and 39	25a	–
2012	39	12 and 52a	14,a 25, 28 and 43a	–
2013	39	50	31	25
2014	–	39,a 50 and 52	–	–
2015	–	25,a 50a and 52a	31,a 61 and 43	–

TABLE 6 - Centres showing evidence of special-cause variation and ‘borderline’ centres in analyses adjusted for age (using a linear term), sex, pancreatic sufficiency, genotype (homozygous DF508, heterozygous DF508 or other) and SES quintile, by year

Centres approaching the 2-SD control limit without being outside it.

Year	Centres with lower than average FEV1 values		Centres with higher than average FEV1 values
	Outside 3-SD limit	Outside 2-SD limit	Outside 2-SD limit	Outside 3-SD limit
2007	12, 52 and 59	23	31, 36a and 54a	25
2008	12 and 52	–	28, 44a and 50a	25
2009	12 and 52	1 and 59	31 and 36	25
2010	12 and 39	23, 52 and 59	25, 31, 36a and 44a	–
2011	–	12, 23a and 39	25a	–
2012	39	12 and 52a	14, 25, 28 and 43a	–
2013	–	39 and 50	31	25
2014	–	39,a 50 and 52	44a	–
2015	–	25,a 50a and 52a	31, 43 and 61	–

FIGURE 10.

Funnel plots of FEV₁ adjusted for age using GLMs, then further adjusted for case mix using multilevel models, by year. (a) 2007; (b) 2008; (c) 2009; (d) 2010; (e) 2011; (f) 2012; (g) 2013; (h) 2014; and (i) 2015.

FIGURE 11.

Funnel plots of FEV₁ adjusted for age (linear term) and other case mix using multilevel models, by year. (a) 2007; (b) 2008; (c) 2009; (d) 2010; (e) 2011; (f) 2012; (g) 2013; (h) 2014; and (i) 2015.

As with the other analyses, we ran a sensitivity analysis to assess the role of ‘shrinkage’. Our findings were broadly similar to those obtained using the multilevel model approach. A summary of contrasting approaches is presented in Appendix 3, Table 29, and the funnel plots are presented in Appendix 2, Figure 30. Occasionally, however, centres that had been brought inside the 3-SD limits using the multilevel modelling approach stayed outside those limits when adopting the GLM approach (centre 28 in 2008, centres 12 and 23 in 2011, centre 28 in 2012, centre 39 in 2013 and centre 53 in 2015). In these instances, concluding that case mix was solely responsible for special-cause variation in the unadjusted models would be inappropriate.

Pyramid of investigation: care

Given that we saw more consistent signals of special-cause variation in these year-by-year analyses, we explored measures of clinical care to see if these are associated with the trends observed in FEV₁. It has been shown in the USA that centres that use more i.v. antibiotics have better FEV₁ outcomes. 27 One way of exploring whether or not that trend exists in the UK is to look at the proportion of patients receiving i.v. antibiotics and at the total number of days on i.v. antibiotics for those patients receiving any antibiotics (see Appendix 3, Table 30). There was no consistent trend showing that centres with lower than average FEV₁ values have systematically different rates and durations of i.v. antibiotic treatment than those centres with higher FEV₁. The most consistent trend among these centres was that centre 31 tended to have a lower proportion of patients taking i.v. antibiotics (2008–11) and a shorter duration of treatment among those patients receiving them (2012). This trend of lower than average proportions of patients on i.v. antibiotics at centre 31 was also in evidence in the funnel plots presented in Figure 12, in which centre 31 frequently shows special-cause variation, being outside the lower 3-SD limit.

FIGURE 12.

Funnel plots of the proportion of patients receiving i.v. antibiotics, by year. (a) 2007; (b) 2008; (c) 2009; (d) 2010; (e) 2011; (f) 2012; (g) 2013; (h) 2014; and (i) 2015.

Analyses in young adults: 2007–12

There were 1439 unique patients aged 18–21 years attending adult CF centres during the study period who fulfilled the study inclusion criteria and had non-missing outcome data. There were 28 adult centres contributing data to the UK CF Registry during this period.

Our two prespecified outcomes in adults were changes in FEV₁ and BMI between 18 and 21 years of age. To be eligible for the analysis, patients required at least two annual review encounters at the same centre between 2007 and 2012. Table 9 shows the identified patient population in terms of their demographic characteristics and the completeness of the outcome and case-mix data that they provided. We considered using measures of patient health at the time of transition to adult care as case-mix variables, as this would allow us to account, in part, for the care delivered in paediatric centres. However, this information was infrequently available for older adults, who were less likely to have data from when they were 16 years old, as UK CF Registry records go back as far as 2007 only.

Outcome: change in FEV₁ in adults aged 18–21 years

We identified 1439 patients with non-missing FEV₁ data who fulfilled the inclusion criteria for this analysis. As was carried out in the paediatric analyses, we performed linear regression analyses for each patient, calculating the annual decline in FEV₁ between 18 and 21 years of age. These individual changes in FEV₁ were then pooled by centre to generate funnel plots of mean change in FEV₁ by centre. The mean decline across centres was 1.78%. Although two centres were outside the 2-SD control limits, none was outside the 3-SD limits (Figure 13a).

FIGURE 13.

Change in FEV₁ between ages 18 and 21 years. (a) Mean of the unadjusted % predicted (Knudson18); and (b) predicted random intercepts of FEV₁% predicted (Knudson18).

A multilevel model, using the estimated change in FEV₁ as the outcome, was generated to explore the impact of case-mix variables with the addition of age at the first visit in this analysis (Table 10). Only sex proved statistically significant in this analysis. A funnel plot adjusted for case mix shows that intercepts (Figure 13b) for all centres were very close to the mean.

TABLE 10 - Fixed and random components of the multivariable model analysis for factors associated with FEV1% predicted [n = 1392 (ages 18 to 21 years)]

CI, confidence interval.

Fixed effect	Estimate (95% CI)	p-value
Constant	0.09 (–2.28 to 2.47)	0.939
Sex (male)	–1.56 (–2.49 to –0.62)	0.001
Genotype
Homozygous DF508	–	0.643
Heterozygous DF508	0.23 (–0.79 to 1.25)
No DF508	0.87 (–0.98 to 2.72)
Pancreatic enzyme supplement use	–0.88 (–2.88 to 1.21)	0.389
SES
1 (least deprived)	–	0.764
2	–0.89 (–2.35 to 0.57)
3	–0.59 (–2.07 to 0.90)
4	–0.81 (–2.24 to 0.63)
5 (most deprived)	–0.41 (–1.91 to 1.09)
Age at first visit	0.29 (–0.86 to 0.28)	0.325
Random effect
Between-centre SD	0.78 (0.29 to 2.11)
Within-centre SD	8.81 (8.48 to 9.14)

As with other outcomes, we considered the possible role of shrinkage and used a fixed-effects model to adjust for case mix. The results are shown in Appendix 2, Figure 32, and no centres are outside the 3-SD limits.

Outcome: change in body mass index in adults aged 18–21 years

We identified 1432 patients with non-missing BMI data who fulfilled the inclusion criteria for this analysis.

As for the earlier adult analyses, we performed linear regression analyses for each patient, calculating the annual decline in BMI between the ages of 18 and 21 years. These individual changes in BMI were then pooled by centre to generate funnel plots of mean change in BMI by centre. The mean change across centres was a 0.17 unit increase. Only one centre was outside the 2-SD control limits; it was within the 3-SD limits, however (Figure 14a).

FIGURE 14.

Change in BMI between ages 18 and 21 years. (a) Mean of the unadjusted BMI; and (b) predicted random intercepts of BMI.

A multilevel model, using the estimated change in BMI as the outcome, was generated to explore the impact of case-mix variables (Table 11). None of the case-mix variables proved statistically significant in this analysis. A funnel plot adjusted for case mix shows that intercepts (Figure 14b) for all centres were very close to the mean.

TABLE 11 - Fixed and random components of the multivariable model analysis for factors associated with BMI (kg/m2) [n = 1387 (ages 18–21 years)]

CI, confidence interval.

Fixed effect	Estimate (95% CI)	p-value
Constant	–0.05 (–0.40 to 0.30)	0.778
Sex (male)	0.13 (–0.01 to 0.27)	0.068
Genotype
Homozygous DF508	–	0.7450
Heterozygous DF508	0.04 (–0.11 to 0.19)
No DF508	0.10 (–0.17 to 0.37)
Pancreatic enzyme supplement use	0.05 (–0.25 to 0.34)	0.755
SES
1 (least deprived)	–	0.295
2	0.10 (–0.11 to 0.32)
3	0.01 (–0.21 to 0.23)
4	0.10 (–0.11 to 0.31)
5 (most deprived)	0.22 (0.002 to 0.44)
Age at first visit	–0.03 (–0.11 to 0.05)	0.485
Random effect
Between-centre SD	0.13 (0.06 to 0.30)
Within-centre SD	1.29 (1.25 to 1.34)

To assess the impact of shrinkage we conducted sensitivity analyses, as we had done for other outcomes. As observed in the analysis of change in FEV₁ in children, we observed some outliers in this analysis that were not evident in the unadjusted or multilevel model-adjusted analysis (see Appendix 2, Figure 33). As before, however, the case-mix adjustment model explains very little of the outcome variability (R² = 0.007) and it is difficult to interpret what these results mean.

Analyses of all patients attending adult centres: 2007–15

To complement the analyses of outcomes in targeted age groups pooling data across multiple years, we conducted all-age analyses by year, as was done for the paediatric sample.

For the analyses of adult centres, we focused on FEV₁% predicted in all patients aged ≥ 16 years, excluding those patients who had previously had a lung transplant. We used FEV₁ measured using the Knudson reference equations,18 as reported in the UK CF Registry, and sensitivity analyses were performed using the GLI reference equations. 19

We considered data from 2007 to 2015 inclusive; the number of patients having annual reviews and those providing FEV₁ data at these reviews changed over that period, reflecting improvements in the completeness of the UK CF Registry (Table 12).

Unadjusted analyses

Centre-level analyses unadjusted for any case-mix factors illustrated that centres varied widely in size and that FEV₁ varied between centres (Figure 15).

FIGURE 15.

Funnel plots of FEV₁ unadjusted for case mix, by year. (a) 2007; (b) 2008; (c) 2009; (d) 2010; (e) 2011; (f) 2012; (g) 2013; (h) 2014; and (i) 2015.

Each year’s plot was investigated to see if centres appeared inside or outside the 2- or 3-SD control limits. We also took note of centres that ‘hovered’ near control limits without being outside them. These results are summarised in Table 13.

TABLE 13 - Centres showing evidence of special-cause variation and ‘borderline’ centres, by year

Centres approaching the 2-SD control limit without being outside it.

Year	Centres with lower than average FEV1 values		Centres with higher than average FEV1 values
	Outside 3-SD limit	Outside 2-SD limit	Outside 2-SD limit	Outside 3-SD limit
2007	9	2 and 35	47	7 and 40
2008	9	35a	5 and 48a	7
2009	9	2, 10, 17a and 21	5, 7a and 40	34
2010	9	2, 10 and 17	40 and 51	34
2011	9	2, 21, 35a and 41	5a and 40	34
2012	9	8a and 10	6, 40, 47 and 51a	34
2013	–	9, 10, 20,a 21 and 56	34, 40 and 47	–
2014	9 and 10	20, 21 and 29a	17, 34, 40, 45 and 47	–
2015	–	9, 10, 20, 29 and 56	34, 40, 45 and 48	47

There are some patterns in the data that suggest that some centres are outside the control limits consistently.

• Centres with lower than average FEV₁ values: centre 9 is outside the 3-SD limits for most years, although – as the largest centre by a large margin and located where the control limits are at their narrowest – it is generally close to those limits. Centre 10 was frequently outside the 2-SD limits from 2010 (and outside the 3-SD limits in 2014). In the early years of the study period (2007–11), centre 2 was frequently outside the limits, as was centre 21 from 2011 onwards.

• Centres with higher than average FEV₁ values: centres 34 and 40 were frequently outside the control limits and, between 2009 and 2012, centre 34 was outside the 3-SD limits. In the later years of the analysis, centre 47 was also frequently outside the 2-SD limits and was outside the 3-SD limits in 2015. In 2007 and 2008, centre 7 was outside the 3-SD control limits and from 2009 was either borderline on the 2-SD limits or within the 2-SD limits. Similarly, centre 5 was often outside the 2-SD limits in the early years, but moved to a borderline position within the control limits after this.

We will explore the patterns seen at those centres appearing outside the 3-SD limits (centres 7, 9 and 34) in more detail, adopting the pyramid of investigation approach, and will consider whether or not issues in the data have led to these trends.

Pyramid of investigation: case mix

Age

It is known that FEV₁ declines with age; therefore, as the mean age of patients varies between centres (see Appendix 3, Table 33), it is essential to adjust FEV₁ accordingly.

Statistical adjustment should account for the documented non-linear association between FEV₁ and age. To do this we adopted two approaches:

1. For each year of analysis we ran a GLM of FEV₁% predicted across age deciles and from this obtained model-predicted values. Our age-adjusted FEV₁% predicted was then calculated using the ratio of the original FEV₁% predicted to the values predicted by the regression model multiplied by 100. When considering those centres identified as exhibiting special-cause variation in the unadjusted analysis, we saw that adjustment for age resulted in some shifting across the control limits. Centre 9 moved to within the 3-SD limits in 2008, 2012–15. The results are described in Appendix 2, Figure 34, and Appendix 3, Table 34.

2. We used beta-splines for age with knots chosen at 18, 28 and 40 years based on visual inspection of the 2012 data. This was done to accommodate possible non-linear associations between age and FEV₁, as has been seen in a number of studies. 22 These splines were incorporated into multilevel models and we used the random intercept predictions in our funnel plots. With this adjustment, some of the centres that showed evidence of special-cause variation in unadjusted analyses were now within the 2-SD control limits (centres 2, 5, 7 and 21). Centre 9 continued to show evidence of special-cause variation, but in 2008, 2010 and 2012 moved to within the 3-SD limits. However, some ‘positions’ remained unchanged, such as that of centre 34, which remained outside the 3-SD limits in 2009–11 and outside the 2-SD limits in 2012–15. The results of these analyses are presented in Appendix 2, Figure 35, and Appendix 3, Table 35.

3. This analysis was repeated using a simple linear adjustment for age and similar results were obtained (results not shown).

These results suggest that age may have accounted for some of the differences between centres, but not in all cases. In addition, regardless of the method used to account for age, consistent signals are detected in centres exhibiting the greatest difference from the mean.

Additional case-mix variables

In addition to age, there are a number of other factors that may influence FEV₁, which are not influenced by care and which are collected on the UK CF Registry.

Sex (see Appendix 3, Table 36)

A number of studies have shown that women have poorer CF outcomes than men. For this reason, we explored whether or not centres with better outcomes have a lower proportion of female patients. We observed that, except for a few outliers, at most centres and at most times the proportion of females ranged from 40% to 55%.

Genotype: homozygous DF508 (see Appendix 3, Table 37)

As CF is a recessive genetic disease, every effort is made to have all patients genotyped, and this information is recorded on the UK CF Registry. The most common genotype in the UK is DF508, and it has been shown in many studies that those who have two of these mutations (i.e. are homozygous DF508) have poorer outcomes. We considered this and noted that in most centres approximately 50% of patients are homozygous DF508. It is also of note that centre 40, which consistently had higher than average FEV₁ results, reports < 35% of its patients being homozygous DF508. This suggests that patients at centre 40 have a different CF genotype profile from patients at other centres.

Genotype: any G551D (see Appendix 3, Table 38)

In 2012, a new CF therapy called Ivacaftor became available in the UK for patients with a G551D mutation. Clinical trials showed promising results and uptake of the drug was swift in those patients who were eligible to receive it. We explored what proportion of patients had at least one G551D mutation and found that the mutation was not very common, but was more common at centre 34, which showed higher than average FEV₁ values, in line with our hypothesis.

Pancreatic sufficiency (see Appendix 3, Table 39)

A marker of CF severity is pancreatic sufficiency (i.e. pancreatic enzymes are not required to digest food). Here we considered what proportion of patients were pancreatic sufficient. We noted that there was variability between centres and that the centre with the highest proportion of pancreatic-sufficient patients was centre 34, which had higher than average FEV₁ values. Centre 7, which had higher than average FEV₁ values, had the lowest proportion of pancreatic-sufficient patients.

Socioeconomic status (see Appendix 3, Table 40)

As with many conditions, there is evidence that individuals from more disadvantaged backgrounds have poorer outcomes. Here, SES was summarised in quintiles: we explored what proportion of patients were in the fifth (most deprived) quintile. All other things being equal, we would expect that 20% of patients in a given centre would be in any given quintile. This was the pattern observed in some of the centres, but we noted that centre 7 (which had better FEV₁ outcomes) had far fewer patients in the fifth quintile. In the later years of the analysis, centre 40 (better FEV₁ outcome) also had fewer patients in this quintile. Contrary to this trend, however, centre 34 – which consistently had better FEV₁ outcomes – had > 25% of its patients in the fifth quintile.

All of these results suggest that some case-mix characteristics may explain some of the FEV₁ patterns we observed and, therefore, adjusting for these in our analyses would be worthwhile. There are, of course, many other variables unrelated to care at an adult CF centre that could influence clinical outcomes but we are limited to those available on the UK CF Registry and those that have good data completeness.

Case-mix adjusted analysis

Case-mix variables were included in multilevel models and adjustment for age was carried out using the approaches described earlier. Both approaches gave very similar results, as described in Tables 14 and 15 and Figures 16 and 17. This additional adjustment generally reduced the number of centres between the 2-SD and 3-SD limits in the case of centres with lower than average FEV₁ values. In the case of those centres with higher than average FEV₁ values, in later years this adjustment seemed to bring centres that were once outside the 3-SD limits to between the 2- and 3-SD limits.

TABLE 14 - Centres showing evidence of special-cause variation and ‘borderline’ centres in analyses adjusted for age (using GLMs), sex, pancreatic sufficiency, genotype (homozygous DF508, heterozygous DF508 or other) and SES quintile, by year

Centres approaching the 2-SD control limit without being outside it.

Year	Centres with lower than average FEV1 values		Centres with higher than average FEV1 values
	Outside 3-SD limit	Outside 2-SD limit	Outside 2-SD limit	Outside 3-SD limit
2007	9	–	7 and 40	–
2008	9	–	–	7
2009	9	10	7,a 29a and 40a	34
2010	9	10	40	34
2011	9	41a	–	34
2012	–	9 and 10	34	–
2013	–	9	34 and 40	–
2014	–	9 and 10	34 and 40	–
2015	–	9 and 10a	34, 47a and 48	–

TABLE 15 - Centres showing evidence of special-cause variation and ‘borderline’ centres in analyses adjusted for age (using splines), sex, pancreatic sufficiency, genotype (homozygous DF508, heterozygous DF508 or other) and SES quintile, by year

Centres approaching the 2-SD control limit without being outside it.

Year	Centres with lower than average FEV1 values		Centres with higher than average FEV1 values
	Outside 3-SD limit	Outside 2-SD limit	Outside 2-SD limit	Outside 3-SD limit
2007	9	–	7 and 40	–
2008	9	–	–	7
2009	9	10	7a and 29	34
2010	–	9 and 10	40a	34
2011	9	41a	7a	34
2012	–	9 and 10	34	–
2013	–	9	34 and 40	–
2014	–	9 and 10	34 and 40	–
2015	–	–	34, 40,a 47a and 48	–

FIGURE 16.

Funnel plots of FEV₁ adjusted for age using GLM then further adjusted for case mix using multilevel models, by year. (a) 2007; (b) 2008; (c) 2009; (d) 2010; (e) 2011; (f) 2012; (g) 2013; (h) 2014; and (i) 2015.

FIGURE 17.

Funnel plots of FEV₁ adjusted for age (splines) and other case-mix factors using multilevel modelling, by year. (a) 2007; (b) 2008; (c) 2009; (d) 2010; (e) 2011; (f) 2012; (g) 2013; (h) 2014; and (i) 2015.

Recognising the possible role of shrinkage in moving centres from outside the 3-SD control limits to within these limits after case-mix adjustment, we conducted sensitivity analyses as described above; in this case, splines were used to adjust for age. The resulting funnel plots are presented in Appendix 2, Figure 36, and, to compare this method of case-mix adjustment with that using multilevel modelling and unadjusted results, a summary of centres outside the 2- and 3-SD control limits using all three methods is presented in Appendix 3, Table 41. In focusing on the centres outside the 3-SD limits, there were only occasional years in which the case-mix adjusted results using multilevel modelling drew centres within the control limits, whereas the method using GLM resulted in them being outside the limits. Often, centre 40 was outside the 3-SD limits in the fixed model-adjusted analysis but was between the 2- and 3-SD control limits in the multilevel modelling approach.

Stage 1: understanding structures, processes and policies associated with high-performing centres

Two 1-day workshops were held with clinicians (1 day for clinicians from adult centres and 1 day for clinicians from paediatric centres). The aim of these was to understand from clinicians the SPPs that were considered to be associated with high-quality care. Details of how the focus groups were run and how the data were analysed are described in Chapter 4.

Stage 2: generating a questionnaire to gather data about structure, process and policy in cystic fibrosis centres

The data collected in the focus groups, as described in Chapter 4, confirmed that there were commonalities in the way in which care was delivered, as identified in previous investigations of quality improvement in US centres. 27,28 An important issue in quality improvement is paying attention to important aspects of care. It was intended that the design of the questionnaires would elicit the extent to which clinicians in individual centres concentrated on the structures, processes and policies that were thought to be associated with high-quality care. Two questionnaires were developed – one for use in adult units and one for use in paediatric units.

Stage 3: collecting and analysing questionnaire data from UK cystic fibrosis centres

The questionnaires were sent out to adult and paediatric centres by e-mail using SurveyMonkey^® (Palo Alto, CA, USA). The initial aim was to analyse the data by centre to understand the SPPs by centre. However, the response rate was too low to allow this; instead, the responses were aggregated and analysed in two groups: (1) across all adult units and (2) across all paediatric units.

Stage 4: comparing structures, processes and policies in centres by outcome

The aim of stage 4 was to compare the SPPs in the centres with the best outcomes with those with the worst outcomes. The within-centre response rates did not allow for the definition of SPPs at centre level, so this part of the work could not be completed and, hence, this aspect of our work is unable to give a perspective on SPPs across the centres that might provide an insight into areas in which challenges arise.

Response rate and respondents

The questionnaire was sent by e-mail using SurveyMonkey to the centre directors at all 29 adult centres. A total of 136 responses from 16 centres were obtained, with a median of six responses per responding centre [interquartile range (IQR) 4–16 responses]. Appendix 3, Table 45, shows the distribution of responses by centre and clinical role; it is observed that the majority of responses came from only five adult centres. Respondents had worked in CF for a median of 7 years (IQR 4–10 years), with nine respondents having ≥ 20 years of CF experience. Thirty-three doctors (of whom 25 were consultants), 25 nurses, 24 physiotherapists and 17 dietitians made up the majority of respondents.

To explore how centres that participated in the survey compared with those that did not, we examined annual review data from all centres in 2015, as this was the year of the survey. We considered the size of the centre to be the number of annual reviews conducted that year and found that the centres responding (regardless of the number of responses within the centre) were slightly larger than the centres that did not respond (median number of annual reviews: 194 vs. 169). The mean FEV₁% predicted of all patients at annual review was also compared, and only minimal differences were observed between participating and non-participating centres (mean 65.9% predicted vs. 66.2% predicted, respectively). Participating centres included sites in England and Scotland; non-participating centres included the only adult centre in Northern Ireland and six of the eight adult centres in Scotland. It is noteworthy that the participating centres that returned the most questionnaires included the sites of collaborators (King’s College Hospital and Northern General Hospital, which also helped to pilot the questionnaire). It is impossible to know how widely the questionnaire was disseminated within teams, as centre directors were asked to cascade the weblink to the survey to their teams. As such, we are unable to comment on the composition of the MDTs of non-participating centres.

Facilities, staffing and resources

Respondents were asked about the facilities, staffing and resources in their centres by seeking their agreement or disagreement with statements describing attributes that were deemed to be desirable by the focus groups. The responses are shown in Appendix 2, Figure 39, with the proportion on the x-axis indicating the proportion of respondents agreeing with the statements. Less than half of the respondents felt that their centre had adequate administrative staffing, access to a gastroenterologist as required, a dedicated CF-related diabetes clinic or a dedicated diabetologist who has a high level of expertise in CF-related diabetes. More common than this were responses relating to access to CF outpatient facilities, allowing the team to meet the CF Trust standards of care and having a strong co-ordinator (> 80%; see Appendix 2, Figure 39).

Capacity and demand

Respondents were asked about capacity and demand; Appendix 2, Figure 40, shows that, although < 50% of respondents could provide appropriate home visit support, > 90% could review patients of concern within 48–72 hours.

Leadership

Respondents were asked about the leadership within the hospitals and centres where they worked; Appendix 2, Figure 41, shows the proportion of respondents agreeing with the statements. Although < 50% of respondents felt valued by hospital management, > 90% of respondents felt that their teams had skilled clinical leadership.

Measurement and monitoring

Respondents were asked about the monitoring systems to support CF care that are available within their centres; Appendix 2, Figure 42, shows the proportion of respondents agreeing with the statements. Less than one-third of respondents felt that their centre had an excellent system specifically to support CF, but > 80% felt that they were able to promptly identify patients with new Pseudomonas aeruginosa infections or declining lung function.

Multidisciplinary team processes

Respondents were asked about the organisational processes that support care delivery (see Appendix 2, Figure 43). Less than half of respondents reported that they had a ‘fool-proof’ system and never overlooked results or felt that their centre was known for running efficiently. Encouragingly, > 80% of respondents felt that all members of the MDT had their voices heard.

Quality improvement of care delivery processes

Respondents were asked about the quality improvement work occurring in their units (see Appendix 2, Figure 44). Less than one-third of respondents reported that their centre routinely uses quality improvement tools, whereas most reported that the team measures and shares outcome, satisfaction, clinical process and safety data with the MDT in order to make improvements.

Treatment policies

Respondents all showed high rates of agreement with policies regarding the use of i.v. antibiotics that might be considered to reflect prompt and proactive care (see Appendix 2, Figure 45).

Time lag to starting antibiotics for cough and cold

Appendix 2, Figure 46, shows that almost all respondents would start antibiotics in adults with cough and cold symptoms if symptoms persisted for 7 days.

Expectations

Respondents were asked about expectations of care to try to understand how determined teams were in their delivery of care and in not tolerating decline. Appendix 2, Figure 47, shows that > 40% of respondents have very high expectations of care, insofar as they expect no yearly decline in lung function.

Response rate and respondents

As with the adult questionnaire, the questionnaire to paediatric centres was sent via e-mail using SurveyMonkey to the lead clinicians in 33 CF centres. A total of 95 responses were received from 24 (73%) of the 33 centres, but in half of these there were ≤ 3 individual respondents. Forty (42%) respondents were doctors, 16 (17%) were physiotherapists, 11 (12%) were nurses and 10 (11%) were dietitians. Respondents had worked in CF for a median of 12 years (IQR 7–20).

As was the case with the adult centres, we used 2015 annual review data to compare centres that did and did not respond to the survey, based on the number of patients attending annual review and the FEV₁% predicted at these centres. As was observed for the adult centres, centres that participated tended to be slightly larger (median number of patients: 124 vs. 106). Mean FEV₁% predicted at the centres participating was slightly higher than at the centres not participating (87.9% predicted vs. 83.8% predicted), and their patients tended to be slightly older (mean age 8.7 years for participating centres compared with 8.0 years for non-participating centres). It is noteworthy that some centres contacted the research team to indicate that the questionnaire could be difficult to complete in the case of network clinics with different facilities and resources.

Facilities, staffing and resources

Appendix 2, Figure 48, describes the responses to questions investigating resource issues. Only 20% of respondents felt that they had adequate social worker resources; however, 80% of respondents felt that they had adequate consultant time for patients.

Capacity and demand

Appendix 2, Figure 49, shows that centres were able to meet capacity and demand requirements with varying degrees of success. Whereas only 41% of respondents felt able to provide popular food for inpatients, 97% of respondents felt able to review patients of concern within 48–72 hours.

Leadership

Appendix 2, Figure 50, shows that participants generally felt more valued by the clinical leaders in their team (86%) than by hospital management (43%). More than 80% of participants felt that their teams were well organised and had skilled and experienced CF clinical leadership.

Measurement and monitoring

Appendix 2, Figure 51, shows the range of responses in the paediatric teams’ views about measuring and monitoring. Whereas only one-third of respondents thought that they had an excellent computerised system for monitoring patients, > 90% of respondents thought that they identified new Pseudomonas aeruginosa infections promptly.

Multidisciplinary team processes

Appendix 2, Figure 52, shows that just over 50% of teams thought that they had ‘foolproof’ systems, which ensured that they never overlooked important results. This means that almost 50% of respondents did not have such systems. Most respondents (> 80%) felt that their annual review processes were effective in delivering treatment plans that were followed through, that in their MDT meetings all voices were heard and that the post-clinic meetings were held with the appropriate people.

Quality improvement of care delivery processes

Appendix 2, Figure 53, shows the respondents’ answers to questions about the processes used to improve systems within the CF centre. It can be seen that, although < 10% of respondents were using formal tools for improvement, almost 60% were using data to improve team performance. The lack of use of formal quality improvement tools in paediatric centres is consistent with what was observed in adult centres.

Treatment policies

Appendix 2, Figure 54, shows that 78% of respondents believed that they could avoid rescue i.v. antibiotics by using preventative treatment, and 96% of respondents agreed that nutrition is important and interventions are expected.

Paediatrics

We studied FEV₁ in two different ways in children: (1) by pooling data across multiple years for adolescents and (2) by studying all children aged 6–15 years in repeated single-year analyses. We observed that centre 39 was frequently an outlier, with FEV₁ levels outside the lower 3-SD limit in our unadjusted analyses of young adults. It also repeatedly showed special-cause variation in our year-by-year analyses of the wider paediatric patient population, along with centres 12 and 52 (outside the lower 3-SD limit) and centres 25, 28 and 31 (outside the upper 3-SD limit).

The first step in the pyramid of investigation model involves checking the data. Members of the study group had already contributed to annual processes of data validation of CF Registry data – through either conducting the validation themselves or providing a validation framework for others within the CF Trust to follow. As a result, the data analysed had already gone through lengthy validation checks involving querying outlier data with the centres, and so they were not verified again. We did, however, explore the completeness of the outcome data used in our analyses of young adults. We noted that data were relatively complete for the case-mix variables considered, but some centres (39 and 43) had lower than average rates of complete FEV₁ data among those patients having annual review encounters. As lung function measurements form a key part of the annual review process, such data should be available.

Continuing with the pyramid of investigation approach, we explored differences in case mix and found some differences in the patient populations at those sites showing special-cause variation in FEV₁. Adjustment for age, for example, brought centre 28 within the 3-SD control limits, despite patients at this centre being only slightly younger than the mean. Centres 12 and 31 were brought within the control limits after further adjustment for genotype, pancreatic sufficiency, sex and SES. Compared with the centres overall, centre 12 had more female patients and more patients who had the milder G551D mutation, in addition to having fewer patients living in the most deprived SES quintile. Centre 31 had fewer patients who were homozygous DF508 and more who were pancreatic sufficient than the overall paediatric patient population. For these two centres, we suggest that differences in FEV₁ are a result of the patient case mix.

After adjustment for our case-mix variables, three of the original six centres continued to show patterns of special-cause variation in FEV₁: centres 39 and 52 (lower than average FEV₁ values) and centre 25 (higher than average FEV₁ values). We next considered care; an objective and routinely collected measure of CF care is the use of i.v. antibiotics. In the USA, this has been shown to be associated with better clinical outcomes. 27 We reported the median number of days on i.v. antibiotics as well as the proportion of patients receiving these treatments. We noted that there was a tendency for centre 25 – which had better FEV₁ outcomes – to be near or outside the lower control limits for the proportion of patients on i.v. antibiotics. Conversely, centres 39 and 52 – which had poorer FEV₁ outcomes – tended to have higher than average proportions of patients on i.v. antibiotics. This, however, tells us about receiving treatment and not about needing treatment, as the CF Registry does not explicitly record pulmonary exacerbations for which i.v. antibiotics would be needed, nor does it record whether the course of antibiotics was planned or unplanned. A US study showed that there is variation in how pulmonary exacerbations are treated in paediatric CF care. 30

A summary of possible explanations for special-cause variation base on the pyramid of investigation is presented in Table 16.

Adults

As performed in the paediatric analysis, we studied FEV₁ in different manners, from decline in early adulthood to studying all adults in repeated single-year analyses. In analyses unadjusted for case mix, we observed that centres 7, 9 and 34 were frequently outside the 3-SD control limits, and centre 40 appeared outside the limits in 2007. All centres except centre 9 showed evidence in these analyses of having mean FEV₁ values that were higher than the overall average.

Following the pyramid of investigation we found that adjustment for case mix is essential to avoid misattributing differences in outcomes to care and to avoid missing important differences. Adjustment for age, for example, brought centre 40 from within the 3-SD control limits to outside these limits for 2009–11 and 2013–15.

We then considered other patient case-mix characteristics to see if these were responsible for our findings. In considering models adjusted for case mix, this adjustment substantially affected centre 40 only, which was brought within the 3-SD limits. This centre differed from the average in many ways: the centre had fewer patients who were DF508 homozygous, more patients with the milder G551D mutation and more patients who were pancreatic sufficient. Our findings at the other centres – 7, 9 and 34 – could not be attributed to case mix.

As discussed in our summary of results from paediatric centres, an objective measure of CF care is the use of i.v. antibiotics, which has been shown to be related to better clinical outcomes. 27 For this reason, we explored how the centres compared in terms of the proportion of patients receiving i.v. antibiotics. Centre 9 frequently exhibited special-cause variation in this variable and had a smaller proportion of patients on i.v. antibiotics. It is difficult to explore the number of days on i.v. antibiotics using funnel plots because of the skewed distribution of the data, but we observed that, among patients at this site receiving i.v. antibiotics, the median number of days on antibiotics in 1 year tended to be fewer than the median across all sites – a pattern observed both overall and when considering the subgroup of patients with lower lung function (FEV₁ < 40% predicted).

Summary

These findings confirm that the annual review data from the UK CF Registry can be used to identify differences in clinical outcomes between centres and that case-mix characteristics might explain some of these differences. This adjustment does not, however, account for all differences; therefore, further work is needed to explore the results we obtained.

Data

This study has many strengths and limitations. Its greatest strengths are the coverage and completeness of the data source. Virtually all CF patients consent to their annual review data being included in the CF Registry and data completeness has grown such that, by 2015, 89% of registered patients provided complete data. 31 Data from the CF Registry are used for clinical care commissioning and pharmacovigilance work in recent years, thus encouraging complete returns. When UK ethics approval for the new web-based system was granted in 2007, there was a big increase in centres being involved and there was a learning curve for data clerks.

However, despite the large number of data collected and satisfactory patient coverage, we were not always able to adjust for all relevant case-mix characteristics. For example, when studying adult care it would be helpful to adjust for clinical characteristics at the time of transition to adult care. Adults who transitioned to adult care before 2007, or in the early years of the CF Registry in which data completeness was not as good, will not necessarily have relevant data. We explored using characteristics data at transition in our analyses of 18- to 21-year-olds but did not pursue using these data as it would have meant losing a large proportion of our patients for centre comparisons because of missing case-mix variables.

The number of data collected is also related to the size of CF centres. Although we observed evidence of special-cause variation in large and medium-sized centres, we cannot exclude the possibility that in very small centres, for which the control limits are widest, we were unable to detect statistically meaningful departures from the mean. Absence of special-cause variation, therefore, does not exclude the possibility of true differences from the mean. Recent comparisons between adult centres indicate that for small absolute differences in FEV₁ predicted of 5% to be reliably detected at the 95% significance level, a minimum of 273 patients per centre would be required. 32

In terms of the appropriateness of the outcomes under study, we met with CF centre directors and patient advisors early in the design stage of the study to discuss our intentions and proposals. This was done to ensure that the outcomes directly related to the issues raised by the clinical community and patients. There are new variables being collected that were not available at the time we began this study, so these may form a framework for the future analyses.

Analysis

Conclusions based on Shewhart’s13 SPC approach are subject to two types of error. The first is to treat an outcome resulting from a common cause as if it were a special cause. The second is to treat an outcome resulting from a special cause as if it were a common cause. Although we cannot entirely eliminate the possibility of making these errors, we can minimise it. For this reason, Shewhart13 argued that variation from stable processes lies within limits that – combining mathematical theory, empirical evidence and pragmatism – can be usefully set at 3-SD limits from the mean. For this reason, and in keeping with Shewhart’s original work in this field,13 other centre comparison studies using funnel plots and the pyramid of investigation approach,33 we focused much of our attention on identifying special-cause variation at 3-SD limits.

Our results were largely robust and replicated in sensitivity analyses, and we obtained very similar results regardless of whether we used the GLI reference equations19 or Knudson17 reference equations. We cannot exclude the possibility, however, that there were unmeasured case-mix variables that might have explained some of the differences we observed.

Analyses of changes in FEV₁ over time failed to show much variability after case-mix adjustment analysis using multilevel models; this may be because we were not necessarily able to get a good measure of ‘change’, owing to the small number of data points available. The individual year analyses, however, afforded us the opportunity to explore whether or not centres showed special-cause variation multiple times over the study period, but this approach also meant that we risked observing regression to the mean. 34 As the CF Registry grows, there will be more opportunities for longitudinal analyses considering changes in lung function over time as well as patient characteristics at the time of transition.

An issue arising from the use of funnel plots for centre comparisons is one of overdispersion. 35 This can occur when there is insufficient risk adjustment and is exhibited by the presence of many outliers in the plots. Our research team benefited from the expertise of a group of clinicians with a great deal of knowledge of the outcomes being studied and of the quality of the data on the CF Registry; with them, we identified those clinically meaningful variables that were well recorded on the CF Registry and could affect outcomes. Invariably, however, there will be other factors, or better measures of the factors that we considered, that may be more appropriate. For example, we considered only current area-level measures of deprivation, but it may be more important to look at early-life exposures to second-hand tobacco smoke. However, these data were not available. Similarly, we are aware that our analyses of outcomes in adult centres could be improved with the addition of more information on the health of patients at the time at which they transitioned to adult care. We are, however, limited by the data available. One recommended method of dealing with overdispersion is the use of mixed-effects modelling, which is the approach we used. With this approach, we found few centres exhibiting special-cause variation.

A consequence of the use of multilevel modelling, however, is that ‘shrinkage’ can occur when the adjusted results are drawn to the overall mean, thus giving the false illusion that case mix explains variability. This can have the greatest effect for smaller centres. We attempted to address the impact of this in our analyses by running sensitivity analyses in which we used fixed-effects GLMs that did not account for centre to generate predicted outcomes. We compared the observed and predicted outcomes by generating the ‘% predicted’ outcomes and generated our funnel plots from these new outcomes. Comparing the results obtained using this method of analysis with the multilevel modelling approach generally yielded similar results. Occasionally, a centre that was within the 3-SD limits in the analysis using multilevel modelling stayed outside those limits when a fixed-effects modelling approach was used. This is of particular relevance to adult centre 40, the results of which were thought to be driven largely by case mix, but which remained an outlier in our fixed-effects modelling approach. Nonetheless, our results were generally consistent. When looking at change in FEV₁ in children and change in BMI in adults, however, we observed some large outliers in the fixed-effects modelling approach that were not present in the multilevel modelling approach. Those centres outside the control limits were within the control limits in the unadjusted analysis. Predictions from our case-mix adjusted model were very poor, making interpretation of our modified outcome of ‘% predicted’ difficult.

Consultations with clinicians

We ran two successful workshops with clinicians – from adult and paediatric centres, separately – to gain insight into the SPPs needed to deliver good CF care. Participants in these discussions came from a variety of disciplines, thereby ensuring that we had insight covering all aspects of care delivered by MDTs. This approach uncovered broad areas that clinicians felt were important and from these consultations we devised questionnaires that were disseminated to all centre directors for them and their teams to complete.

The clinician survey alone would be insufficient to gain insight into the SPPs associated with good care, but it was designed to help inform the conversation with sites and provide a starting point for discussions regarding the causes of variations in clinical outcomes. Owing to the low response rate for this questionnaire, we were unable to get a clear sense of the structures and processes of care at all centres. We can obtain supplemental information from the CF Registry (such as days on i.v. antibiotics and use of chronic preventative therapies, which have been suggested as a reason for better lung function outcomes in the USA than in the UK36) but this does not cover the full picture of how care is structured and delivered at a centre.

Alternative approaches to understanding how sites deliver care – such as through observation – may have yielded more information than a questionnaire. Questionnaires about how care is delivered can be prone to the selective bias of recalling more incidences of desirable behaviours and fewer incidences of undesirable behaviours. However, we felt that responses to our questions might signpost areas of further inquiry. The response rate to our questionnaire reflects the intense pressure that clinicians are under. Our choice of a relatively unsophisticated approach was an attempt to pragmatically sample a population that we knew to be very busy.

As indicated, it was anticipated that these questionnaires would provide hypotheses of the reasons for the differences in clinical outcomes observed, which are discussed in Chapter 3. These would then be explored in site visits at a sample of adult and paediatric centres. Centres were to be chosen in such a way that some would have average outcomes, some would have below average outcomes and others would have above average outcomes. Our plan for these site visits was described in the protocol; it involved an iterative process whereby the centre comparisons on clinical outcomes would be shared with the centre directors, who would be invited to comment and provide their own hypotheses, which we could then investigate with the data collected. Based on these analyses, we would have further discussions and discuss hypotheses generated from the centre director survey. However, it was felt that conducting the site visits without clear hypotheses driven by the survey would not be helpful or informative, so these were not conducted.

Consultation with patients

Our consultation with patients yielded insight into what patients feel is important in the delivery of CF care. It is only because of time restrictions that we were unable to disseminate, analyse and present our results within the context of this study.

The questionnaire on its own was not designed to enable us to perform statistical hypothesis testing, but it can be best conceptualised as a qualitative survey tool; as such, we would be looking for data saturation as an indicator that the important themes had been uncovered. However, understanding differences between centres would be difficult.

Questions raised by this project

There remain questions to be addressed. How does i.v. antibiotic treatment (and, potentially, other medications) affect outcomes in CF patients? We observed that centre 9 was frequently outside the 3-SD limits for FEV₁, despite adjustment for case mix, and there was some evidence that this centre has a smaller than average proportion of patients receiving i.v. antibiotics. It has been suggested that CF centres with better FEV₁ give more i.v. antibiotic treatments27,37 and analyses comparing FEV₁ in US and UK CF patients suggested that the determined use of treatments in the USA may be a driver of differences between US and UK CF centres. 36 Future work could explore the role of antibiotic treatments through more detailed analysis of CF Registry data and discussions with sites.

Are there other measures that could be used for centre comparisons? The outcomes studied in this project were those that were felt to be important to clinicians and that have been shown to be associated with survival. Mant38 suggests, however, that process measures would be more sensitive to change in quality improvement than clinical outcomes. Therefore, future studies may involve exploring which process measures would be most reliable and relevant to studying CF care. We, and another research group,37 studied the number of days on i.v. antibiotics as process measures and found interesting results that encourage further exploration. We studied the number of days on i.v. antibiotics as it was the most reliable process measure routinely collected by the CF Registry at the time, but the exploration of other data sets may suggest other measures that, in future, could be incorporated into national comparisons. There is, for example, current work funded by the CF Registry that is exploring the potential of linkages between the CF Registry and other routine data sets such as Hospital Episode Statistics. Such linked data could potentially yield useful data in this regard.

If we are to concentrate centre comparisons on measures of lung function, are there other measures of lung function that might prove insightful? In recent years, the UK CF Registry has collected information on ‘best’ FEV₁ as well as FEV₁ at annual review. Future research could involve comparisons between centres on this ‘best’ measure and – if data become available in the future – an exploration of the care received by patients in the period leading up to their ‘best’ FEV₁. There is evidence from those sites providing both measures of FEV₁ in 2013 that ‘best’ FEV₁ differs from FEV₁ values at annual review (when patients are ‘stable’) and that differences are greater for larger and smaller centres (see appendix C37).

Are there better ways of adjusting for patient case mix? The case-mix adjustment models used in the current analyses were based on clinical feedback on important patient characteristics and the data available. A more detailed exploration of patient histories, using evolving CF Registry data, may allow for refinement of the case-mix adjustment models, which could be validated against data from other well-established CF Registries, such as those in Canada and the USA.

What is the role of SES on observed differences between centres on FEV₁? The importance of SES in health outcomes in CF has been studied extensively. 39 There is evidence that disparities exist, at least from the earliest point (at age 6 years) at which lung function is routinely recorded. It is not obvious what the mediating factors are, but it has been proposed that exposure to tobacco smoke may play a role. Future work to understand how centres can address this might, for example, involve exploring more closely the SPPs of those centres that have good clinical outcomes as well as a mixed patient population in terms of SES.

Beyond the potential role of the use of i.v. antibiotics, we were unable to identify the SPPs related to good CF care. Our survey of clinicians did not yield sufficient data to inform discussions with the sites. Taking an ethnographic approach based on observation, however, may yield more useful information.

How do patients feel about the care that they receive? We developed a detailed questionnaire for patients, which we did not have time to disseminate and analyse within the context of this study. Relevant approvals for this project have been obtained and we hope to disseminate the questionnaire soon. Analyses here can consider different age groups, as the concerns of the parents of young children will potentially be different from the views of young adults and older adults who will have had a lifetime of CF care.

Aims from this project

Despite the unanswered questions and limitations encountered, we have developed part of the framework we intended to, which will support future centre comparison analysis of UK CF Registry data, thereby benefiting the CF community – both patients and clinicians. The framework, as currently developed, relates to having established models and processes for making centre-level comparisons on key clinical outcomes and routinely collected measures of care (such as i.v. antibiotic usage), although these are subject to a number of limitations, as described above. Our workshops and focus groups have elicited important information on what patients and clinicians feel is important in delivering good CF care and achieving good outcomes. These events have helped us to devise questionnaires to be distributed more widely to the clinical and patient community.

This forms the basis of future work linking care and clinical outcomes, which could complete our intended aims. To resolve the outstanding elements of our proposed work, the following is required:

1. Disseminate the patient questionnaire that was developed in this project.

2. Collect information on the SPPs at centres. We accept that our survey design was inadequate and are exploring other possibilities, including an ethnographic approach using observation.

3. Use the information gathered in the first two steps to inform conversations with sites identified in our centre comparison work to explore whether or not the differences observed relate to care.

Preamble

Hello everyone, and thank you for joining this telephone-based focus group. During the next couple of hours, we are going to discuss some issues that we think help a CF unit to perform well. There are no right or wrong answers to the questions I am going to ask, so please feel free to agree or disagree with each other as we go along. Because this focus group is being tape recorded for later analysis, can I please ask you not to talk over each other? Please try to wait for whoever is speaking to finish before starting to speak yourself. Whilst we are keen to hear about your own personal experiences, you are welcome to also mention the experiences of other CF patients who you know about.

Schedule

• Can we start by considering the level of information that CF units should provide to patients and carers?

  • Is it important in your view that CF units educate their patients about the condition? What benefits would this have?

  • What specific aspects of CF is it important for units to educate patients and carers about? And how is this achieved in your experience?

  • To your knowledge, what educational materials are routinely given to patients? Are there any gaps in your view?

  • Do you think units should have explicit policies and procedures in place around education of patients? Would this be valuable?

  • Would you find it helpful if members of the MDT asked questions to confirm your understanding of aspects of your treatment? What benefits would this have?

  • What helps or hinders CF units in educating their patients and carers in your view?

  • In an ideal world, what would a well-performing CF unit look like, with respect to patient education?

• Can we turn next to consider the extent to which units encourage CF patients to have influence and control over their treatment and care?

  • So first of all, do you personally feel that you are encouraged to take control over your care and treatment? How/in what ways?

  • In your experience, do patients and carers want greater control and influence over their treatment and care? If so, what aspects of their treatment and care do they want more control over?

  • Is it important that CF units encourage patients to take greater control of the management of the their condition? What benefits would this have?

  • What helps or hinders CF units in encouraging their patients to have more influence and control over their treatment and care in your view?

  • In an ideal world, what would a well-performing CF unit look like, with respect to the degree of control and influence a CF patient has over their treatment and care?

• The next issue for discussion is how CF units develop a constructive dialogue and relationship with patients. So, first of all, do you generally feel listened to when receiving care at your CF unit?

  • Does the MDT try to understand your lifestyle, and fit your treatment around your lifestyle? Could you provide an example of this?

  • How do you feel that CF professionals go about developing a constructive dialogue and relationship with yourself and other CF patients?

  • Do you think that CF patients are interested in developing a constructive dialogue with specific members of a unit?

  • What helps or hinders a CF unit from developing a constructive dialogue with patients in your view?

  • In an ideal world, what would a well-performing CF unit look like, with regards to how it develops a constructive dialogue with patients?

• For the next part of the focus group, I’d like us to consider how units measure and manage CF within the MDT. So first of all, what in your view should units be measuring to know that they are treating patients effectively?

  • Do you feel that you understand your target lung function and weight and what these mean? Are there other aspects of your health that you feel it is important that CF units monitor and manage?

  • Are you given information about your own condition to help you evaluate how your treatment and care is progressing?

  • Have you experienced variation in the standard and quality of your care in visits to your unit?

  • Do you feel that the MDT in your unit communicates well and delivers a consistent message?

  • Do you feel that the MDT has a shared understanding of your treatment and care?

  • Are you confident that the MDT shares information well, so that they all know about your treatment and care?

  • Do you think it is important for patients and carers to be involved in monitoring and managing the performance of CF units? What benefits would this have?

  • Should CF patients know how their unit performs in relation to other units in the country? What benefits would this have?

  • In your view, what factors help or hinder CF units measuring and managing CF effectively?

  • In an ideal world, what would a well-performing CF unit look like, with respect to the measurement and management of CF?

• The next part of the focus group relates to resources and multidisciplinary staffing in CF units. First of all, have you, or patients who you know, ever experienced difficulties in seeing certain members of the MDT as often as you would like?

  • Have you, or patients who you know, ever experienced difficulties in obtaining relevant treatments such as nebulisers?

  • Are you confident that all members of the MDT in your unit are equipped and trained to manage your CF effectively?

  • What, in your view, helps or hinders CF units to obtain the resources and staffing to ensure a good level of performance?

  • In an ideal world, what would a well-performing CF unit look like with regard to staffing and resources?

• I’d like now to broaden the discussion out and for you to describe for me some good and bad experiences of the care that you have had both as an inpatient and an outpatient. If we could start with your experiences of being treated as an inpatient:

  • What is a good experience as an inpatient like?

  • What is a bad experience as an inpatient like?

  • What is a good experience as an outpatient like?

  • What is a bad experience as an outpatient like?

  • What does your CF unit do well?

  • What could your CF unit improve?

Closing remarks

We have come to the end of the focus group questions. Before we close, does anyone want to add anything to any of the issues that we have discussed?

Many thanks for your participation. The recording is going to be transcribed verbatim and a copy will be made available to you for your information.