Measuring prevalence, reliability and variation in high-risk prescribing in general practice using multilevel modelling of observational data in a population database

Commonly used explicit criteria

The Beers Criteria are the most commonly used explicit set of measures. The original Beers Criteria were a list of drugs which were judged by a Delphi panel to be potentially inappropriate for use in adults aged 65 years and over in the USA, and has since been updated on two occasions. 7,8,19 Included criteria cover both drugs which are judged to always be potentially inappropriate (e.g. amphetamines) and those which are inappropriate if patients have particular conditions (e.g. NSAIDs in people with a history of peptic ulceration). 7 The prevalence of potentially inappropriate prescribing (PIP) has been measured in multiple routine data sets in many different countries, and it is straightforward to show that PIP is common, although estimated prevalence varies widely depending on the exact subset of Beers Criteria applied and the setting. 20–26

However, the Beers Criteria have been criticised on a number of grounds. Many of the drugs being measured are only rarely prescribed, particularly those that are defined as potentially inappropriate in all older adults. In practice, therefore, the prevalence of PIP as assessed by Beers Criteria is driven by a small number of drugs, such as benzodiazepines. In addition, there is little evidence of an association between Beers ‘drugs to avoid’ and ADEs in clinical practice,27–31 where most serious ADEs (including those causing emergency hospital admission) are a result of drugs which are in fact recommended by guidelines12,13 such as aspirin, anticoagulants and antihypertensives (although there is some evidence that some individual measures within the Beers Criteria are associated with harm25,32). In addition, the focus solely on potentially inappropriate use of drugs that may be harmful (a sin of commission) has been criticised when as much or more harm (in terms of forgone benefit) may be caused by indicated drugs not being used (a sin of omission). 28,33,34

Prevalence of high-risk prescribing and change over time

A number of studies conducted in a variety of settings have investigated how commonly high-risk or otherwise potentially problematic prescribing occurs, with fewer studies reporting time trends. The focus here is on larger scale primary care studies, the majority of which were conducted in the USA, the UK and Sweden, with the remainder reporting data from the Netherlands, Ireland and Finland. 22,23,25,40–45

High-risk and potentially inappropriate prescribing summary

In summary, numerous studies using a range of indicator sets show that high-risk and PIP remains common in primary care. Notwithstanding its regular critique, most large-scale prevalence studies have applied the Beers list in patients 65 years and older and found that between 15% and 30% are affected, although a smaller study using START/STOPP criteria found a similar prevalence. Despite the growing emphasis on health-care safety in the last 15 years,9,10 and the current ubiquity of electronic prescribing with automatic interaction checkers in primary care, the majority of studies demonstrate little change in the prevalence of high-risk prescribing over time, irrespective of the specific measures used. The small reported changes in Beers drug prescribing in the three identified studies examining time trends may partly reflect a lack of evidence that the Beers list prescribing is associated with worse outcomes,21 although it is possible that prescribers have become more aware of high-risk prescribing over time, but that a parallel rise in the prevalence of polypharmacy and the associated increased risk of drug–drug interactions overburdens prescribers. Others have highlighted that physicians’ clinical practice styles in terms of referral, test ordering and resource use are relatively consistent over time,60 so the lack of change over time may also indicate scope for improving current mechanisms of translating prescribing safety guidance into clinical practice.

Nevertheless, there is evidence of statistically and clinically significant variation in practices’ prescribing patterns, even after adjustment for patient-level variables including polypharmacy, which suggests that some practices are more successful in avoiding high-risk prescribing than others. It is currently unclear, however, whether such variation is mainly because of individual prescribers (a ‘bad apple’ problem) or practices’ prescribing culture (a ‘spoiled barrel’ problem). The next section discusses the literature on variation between physicians and between the institutions or setting in which they work.

Variation at different levels of the health-care system

Since Wennberg published his seminal study of small-area variation in health services volume in 1973,61 there have been a large number of studies examining variation in health care. A recent systematic review of 836 such studies in Organisation for Economic Co-operation and Development countries that were published between 2000 and 2011 concluded that there was overwhelming evidence that practice varied between areas and institutions but relatively few data on the causes and consequences of such variation. 78 However, the studies cited primarily examine variation in hospital care or procedures, reflecting the fact that data on these are relatively more readily available, and most did not examine variation in ways which appropriately account for the effects of chance.

Fung et al. 79 carried out a systematic review of English-language literature identified from MEDLINE using multilevel modelling or similar techniques to examine variation between regions, institutions or providers in health service performance or which examined reliability of measurement at a higher/aggregated level (e.g. hospital) for an outcome measured at patient level. Such studies are not well indexed and the search strategy used combined broad medical subject heading (MeSH) terms with title or abstract keyword searches, for example Quality Assurance, Health Care (MeSH) and (hierarchical OR intraclass correlation coefficient OR Bayes) [title or abstract]. They identified 39 studies that explicitly partitioned variance in terms of the relevant importance of between-patient and between-region/institution/provider variation, with 30 studies explicitly using multilevel modelling approaches to do so. Thirty-two of the studies adjusted for patient age and sex, with smaller numbers adjusting for other sociodemographic variables such as socioeconomic status or ethnicity, or for clinical characteristics such as comorbidity. Few studies included explanatory variables at higher levels.

The most common form of reporting was to express variation between higher levels in terms of intraclass correlation coefficient (ICC), defined as the variance at a level divided by the total variance, and interpretable as the proportion of variation in patient outcome attributable to that level. Estimated variation attributable to higher levels varied considerably, from 0–19% in studies of physician-level variation and 0–10% at provider group level, to 0–51% at facility level and 0–3% at health plan level. Only a minority of studies presented data on absolute differences between higher levels, either in terms of crude and/or case-mix-adjusted variation or in terms of estimated residual variation at higher levels based on the multilevel model. The authors argue that, although ICCs are useful in examining variation, their clinical significance cannot be easily judged without understanding the magnitude of absolute differences.

Variation at higher levels above the patient was examined at different levels of aggregation, including physician, physician-group, facility (medical centre, hospital, nursing home), health plan and other levels (hospital ward, clinical team, geographical area). Most studies examined variation at only a single higher level, but seven examined two levels, three examined three levels and two examined four levels of care. The 12 studies examining variation at multiple levels are particularly relevant to our study, although none examined prescribing outcomes. Five examined satisfaction,80–84 three diabetes care processes,85–87 and one each preventative care,88 inpatient psychiatry length of stay,89 the number of physical therapy sessions for back pain90 and dialysis resource use. 91

The findings of these studies do not show consistently greater variation at one level over another, although a majority find greater variation at lower levels of aggregation (physician and facility) than higher (health plan and region). For example, two of three diabetes studies86,87 found greater variation in most process and intermediate outcome measures at physician than facility or hospital level (the exceptions being processes which are likely to be more under the control of the institution than the physician, such as eye screening, in which Dijkstra et al. 87 found greater variation at hospital level than physician level), as did the third85 for process measures although greater variation in costs and intermediate outcome achievement at facility level. In contrast, the number of physical therapy sessions for low back pain showed greater variation at practice (7.2%) than practitioner (4.4%) level,90 and there is much more variation in dialysis costs at facility than physician level. 91 Such inconsistency is not surprising, in that the processes and outcomes being measured vary in terms of the relative control over them exerted by individual clinicians versus organisations; dialysis costs, for example, are more likely to vary with decisions made at organisational level.

Twelve studies examined reliability, in the sense of the ability of the examined performance measures to distinguish true variation between higher level units from chance variation, usually focusing on variation between physicians, as, for a given ICC, reliability improves as the number of patients that a physician or other higher-level unit treats increases. Reliability is, therefore, usually more of a problem when measuring individual physicians rather than larger organisational units. Seven of the 12 studies used the Spearman–Brown prophecy formula for this purpose, in which it was most commonly suggested that a reliability of 0.8 was the minimum required for a measure to be used for physician or other higher level unit profiling (reliability measures vary between 0 and 1, where 1 is perfectly reliable). Using the Spearman–Brown prophecy formula, the authors estimated that, if the ICC was 0.20, then a physician would need 16 of their patients to be included in a performance indicator to have their performance measured with a reliability > 0.8, compared with 396 patients if the ICC was 0.01 (i.e. 1% of variation in patient outcome attributable to differences between physicians).

Only 2 of the 39 identified studies used a prescribing outcome. Normand et al. 92 combined observed and simulated data to estimate the minimum number of discharges with myocardial infarction a hospital would need to be reliably measured using four prescribing indicators, but did not report ICCs. Cowen and Strawderman93 examined variation in pharmacy costs using a range of models, concluding that multilevel regression models provided better estimates for physicians with smaller numbers of patients in particular than did single-level regression models. They examined multiple sets of aggregated pharmacy costs, but, for all costs, the estimated ICC was between 0.0009 and 0.04 depending on the population of physicians being examined. 93

Variation in primary care prescribing

As this systematic review did not include relevant primary care prescribing studies that we had already identified, we carried out an additional search for primary care studies of variation combining the MeSH term ‘Physician’s Practice Patterns’ and a set of terms to limit to primary care/family practice/general practice. This yielded 11,290 titles which were screened to identify papers explicitly examining variation in high-risk or PIP between settings and/or physicians using an appropriate method (i.e. not simply using a league table approach or using a crude comparison of the highest and lowest prescribers).

Background

A feature of the previous analysis is that it does not account for variation within practices in terms of who prescribes a high-risk NSAID. Linked to this, it ignores how individual prescriptions are created and issued, in that the outcome happens when a NSAID prescription is issued to a patient vulnerable to adverse drug effects (strictly speaking, the outcome happens if such a patient receives one or more NSAID prescriptions during the final quarter of 2006, as some patients receive more than one in that time period). A crucial distinction in GP IT systems is between acute and repeat prescriptions, because this determines who is allowed to issue a prescription, and the degree of oversight exerted by the clinician who signs the prescription.

Understanding how prescriptions are created and issued in the General Practice Administration System for Scotland

As part of the feasibility testing, we used a demonstration version of GPASS to explore the relationship between the way in which a prescription was created and the name printed on the paper prescription, on the one hand, and the name recorded in GPASS electronically, on the other.

Type of non-steroidal anti-inflammatory drug prescription

We defined four types of NSAID prescription according to whether they were acute or repeat, and new or subsequent. Acute and repeat prescriptions are defined by a field in the data supplied and are a key element in the way that the clinical system is set up and used. An acute prescription is a one-off issue from an encounter screen which can be reissued only in an encounter screen, and acute prescriptions are typically issued by clinicians. A repeat prescription is set up so that it can be reissued from the repeat prescribing screen, and this is typically done by administrative staff rather than clinicians. Each type of prescription is separately displayed/clearly distinguished in GPASS.

Based on the data, we then defined the following types of high-risk NSAID prescriptions:

• ‘new acute’: an acute NSAID prescription without a NSAID prescription of any kind issued in the previous 365 days

• ‘subsequent acute’: an acute NSAID prescription that was preceded by either an acute or a repeat NSAID prescription in the previous 365 days

• ‘new repeat’: a repeat NSAID prescription where there had not been a repeat NSAID prescription issued in the previous 365 days

• ‘subsequent repeat’: all other repeat NSAID prescriptions.

The distinction being made is between new decisions to create a particular type of prescription for the first time that year (new acute and new repeat NSAIDs) and a decision to reissue a recently issued drug (subsequent acute and subsequent repeat NSAIDs). Based on our knowledge of how practices organise this prescribing, subsequent decisions are subject to less scrutiny and clinical consideration than new ones. Creating new prescriptions is therefore a key decision point that partly determines future care.

Table 1 shows how common each type of prescription was in calendar year 2006, for each individual indicator and for the composite. For the composite indicator, acute prescribing accounted for 26.2% of all high-risk NSAID prescriptions issued, with new acute NSAIDs representing 10.6%. The creation of new repeat NSAIDs was relatively uncommon (1.2%) but subsequent issues of repeat NSAIDs represented nearly three-quarters (72.7%) of all NSAID prescriptions issued. There was some variation by indicator, where people with previous peptic ulceration and those with heart failure were somewhat less likely to be issued a repeat prescription, although overall the patterns were similar for all five individual indicators.

Linkage of high-risk non-steroidal anti-inflammatory drug prescription data to the encounter table and clinician data

The encounter data supplied to us contained a field called ‘encounter type’ with five values [C = ‘Co-op’ (i.e. out of hours), D = ‘data entry’, E = ‘encounter’ (i.e. a normal surgery encounter), P = ‘telephone’, U = ‘unknown’ (but actually a catch-all for other encounter types which includes a variety of values including ‘clinic’, ‘other practice’ and ‘home visit’)]. The encounter table also has a field for the user who recorded the encounter, which we received as an anonymised ‘clinician ID’ (although not all users are clinicians).

Table 2 examines the extent to which high-risk NSAID prescriptions for each indicator and the composite were linkable to the encounter table, and the proportion of linkable prescriptions for which encounter type and clinician ID were recorded. It shows that, of 20,759 high-risk NSAID prescriptions, 4543 (21.9%) did not have an encounter ID and so could not be linked to the encounter table. Of the 16,216 (78.1%) prescriptions that were linkable to the encounter table, only 64.9% had an encounter type recorded and only 52.4% had a clinician ID recorded, as these are not compulsory fields depending on how the record is opened. Linkage was similar for the five individual indicators, although somewhat better for peptic ulcer and somewhat worse for oral anticoagulants.

Table 3 examines linkage by type of high-risk NSAID prescription in terms of whether it was acute or repeat, and new or subsequent (as defined in Type of non-steroidal anti-inflammatory drug prescription). Acute high-risk NSAID prescribing was almost always linkable, and almost always had an encounter type recorded (> 99%). Of new acute prescriptions, 95.4% had a clinician ID recoded, versus 67.1% of subsequent acute prescriptions. By contrast, new repeat prescriptions were linked in only 76.3% and subsequent repeat prescriptions in 70.3% of cases overall, with over half not having a linked clinician ID. This reflects the fact that repeat prescribing can be managed either through encounter screens (which generate an encounter ID) or through the repeat prescribing screen (which does not).

On further exploration, it was clear that the encounter ID (and therefore clinician ID) for subsequent repeat prescriptions defaulted to the encounter and clinician who had initiated the repeat prescription, with all subsequent issues having the same encounter and clinician IDs. This reflects the data structure, where the first repeat prescription is recorded in the prescription table, which has encounter ID for linkage, but subsequent issues are recorded in a separate table, which records only the date of reissue. In practice, therefore, subsequent repeat prescribing has only the initiating GP’s identity recorded, but this is missing anyway more often than not.

We concluded that it was potentially feasible to examine acute but not repeat NSAID prescribing in a GP-level analysis, and the next section examines acute prescribing further.

Defining eligible encounters and eligible general practitioners

Table 4 shows the extent to which clinician IDs were missing for new and subsequent acute high-risk NSAID prescriptions by encounter type. Clinician ID recording was high (> 95%) when issued in normal surgery encounters (i.e. face to face in a normal GP surgery) or telephone encounters as well as for new acute high-risk NSAID prescriptions that were issued in other/unknown types of encounters. However, for data entry encounters, clinician ID recording was poor (< 50%) for both types of prescriptions, and especially for ‘subsequent acute’ prescriptions (28.1%).

For the planned analysis, it was also necessary to distinguish between clinical and non-clinical users, and between GPs and nurses. The data supplied to us contained a field called ‘clinician type’ with two values (G and X) which we were told was believed to distinguish GPs from other users, although there was ambiguity about whether or not ‘G’ clinicians also sometimes included nurses. We additionally extracted data about whether or not an encounter had an associated Read Code with a PTI modifier. All the practices contributed data to the NHS Scotland PTI data set, where face-to-face encounters with patients all have one or more Read Codes attached to identify the morbidity being dealt with, and these Read Codes have a specific modifier attached which identifies if they are a GP or a nurse consultation (the modifier allows data entry Read Codes, e.g. following a hospital admission, to be excluded from primary care workload analyses). Using this data, we identified clinician IDs who were GPs and nurses based on their recording of PTI-modified Read Codes.

During 2006, there were a total of 478 GPs who had a total of 181,263 encounters (normal surgery, telephone or unknown/other) with a patient at risk (i.e. vulnerable to adverse effects of NSAIDs by virtue of comorbidity or coprescription) at the time of the encounter, during which 2024 new acute NSAID prescriptions were issued. Of these, 1961 (96.9%) of new acute prescriptions had a GP ID. However, 80 GPs (who prescribed a total of eight new acute NSAIDs in 2006) had fewer than 10 encounters with at-risk patients, which is a problem when estimating variation in the prevalence of relatively rare events.

We concluded that it was potentially feasible to examine new but not subsequent acute NSAID prescribing in a GP-level analysis, and to examine new acute NSAID prescribing in normal surgery, telephone and unknown/other encounter types for GPASS users identified as GPs using analysis of PTI-modified Read Codes.

Do new acute prescriptions matter given that they are not the bulk of prescribing?

It was feasible to examine variation only in new acute NSAID prescribing in the analysis to examine variation between both practices and GPs but these prescriptions accounted for only 10.6% of high-risk prescriptions issued in 2006, raising the question of whether or not this was a important enough outcome to examine. Total high-risk prescribing in a practice will be determined by a number of processes including:

• New acute NSAID prescribing. Our expectation is that this would be the most frequent initiation event.

• New repeat NSAID prescribing without a preceding acute prescription. This will be rarer, but once set up is the gateway to regular issuing without much clinical oversight (although this is rare in this data set, with only one of 241 new repeat prescriptions in 2006 not being preceded by an acute prescription in the previous year, the exception having previously had a active repeat prescription issued just under 2 years earlier but subsequently cancelled).

• For those issued a new acute prescription, transition in some patients to receiving subsequent acute prescriptions.

• For those issued a new acute or subsequent acute prescription, transition in some patients to receiving new and then subsequent repeat prescriptions.

• Cessation of either multiple acute or repeat prescribing, either because patients’ symptoms resolve or because of review and active withdrawal by a clinician.

To examine the relative importance of high-risk new acute NSAID prescribing, we identified 441 patients eligible for one of our five measures who received a new acute NSAID prescription in Q4 2005, and examined high-risk NSAID prescribing to these patients in 2006. Five patients received a further new acute NSAID in 2006 (i.e. an acute prescription more than 1 year after the one issued in Q4 2005), 135 patients received a total of 279 subsequent acute NSAID prescriptions in 2006, and 21 received a new repeat NSAID with 203 subsequent repeat issues. In total, 143 (31.8%) patients prescribed a new acute NSAID in Q4 2005 received one or more NSAIDs in 2006, with a total of 513 prescriptions issued, representing 2.6% of all high-risk NSAID prescribing in 2006. As well as carrying risk in its own right, new acute NSAID prescribing is, therefore, an important initiation event for chronic prescribing.

Summary

For analysis of variation between GPs, attribution of a particular prescription to a GP is critical but is complicated for two reasons:

1. The decision to prescribe being made by more than one clinician, notably with regard to specialist recommended prescriptions issued by GPs. In this situation, GPs clearly have the legal responsibility for the prescription because they create and sign it but attribution is not as simple as one where the GP alone decides to prescribe.

We therefore defined a set of measures relating to high-risk NSAID prescribing where the first problem was relatively unlikely to arise, because almost all decisions to prescribe NSAIDs are made by GPs.

1. Not all prescriptions include a clinician ID, and, even if they do, the clinician ID may not be the correct ID for the clinician actually signing the prescription.

General practitioner systems distinguish between repeat and acute prescriptions. Repeat prescriptions are those for which a prescription is set up so that the patient can reorder the medication from reception staff, who print it off and pass it to a GP for signing, typically in a large pile of other repeat prescriptions. Acute prescriptions are one-offs which cannot be straightforwardly reissued by reception staff, and for which reissue is usually done by a clinician, meaning there is greater clinical oversight.

Unsurprisingly, repeat prescribing did not reliably have a clinician ID, and, where an ID was present, it related to the encounter where a new repeat prescription was originally created rather than to the signing of a repeat issue. In addition, we know that in many practices repeat prescriptions are often not signed by the GP whose name is on the prescription because the printed name is often automatically set to be the ‘duty doctor’ or the ‘registered doctor’. Hence, attribution of repeat prescriptions even in the presence of an electronically recorded clinician ID is problematic. Our planned analysis in this context was to examine cessation of high-risk prescribing after a medication review, but the data set did not have reliable recording of the person doing medication review so this was not possible. It was therefore not feasible to examine repeat prescribing at GP level.

For acute prescribing, we distinguished ‘new’ and ‘subsequent’ prescriptions (an acute prescription is a ‘one-off’ which cannot be straightforwardly reissued by reception staff and for which there is, therefore, greater clinical oversight). A new acute NSAID prescription was defined as an acute prescription where there has been no NSAID prescribed in the previous year. This is the prescription with the greatest clinical oversight, in that almost all this prescribing happens in face-to-face clinical encounters and is the key initiation event for subsequent prescribing, which is typically subject to less clinical oversight. One-third of new acute prescriptions were associated with additional high-risk NSAID prescribing in the subsequent year. Over 95% of new acute NSAIDs had a clinician ID associated with them, with most missing clinician IDs being in data entry encounters, where a clinician ID is not required. In contrast, only 67.1% of subsequent acute prescriptions had a clinician ID, reflecting the fact that they were much more commonly issued in data entry encounters, probably because they were telephone requests via reception rather than clinical encounters (reflecting the reduced clinical oversight involved, although oversight is still higher than for repeat prescribing because acute prescribing will typically be flagged as a special request to distinguish it from repeat prescribing).

Interpretation and implications for subsequent analyses

The feasibility analysis showed that we could reasonably robustly analyse new acute NSAID prescriptions in non-data entry encounters for GPs identified using PTI-modified Read Codes. The data set for analysis of variation between practices and between GPs, given the constraints described above, therefore included 181,010 (99.9%) encounters by 26,539 (99.97%) patients at risk in 2006 who consulted at least once, during which 1953 new acute NSAIDs were issued (96.5% of those with a clinician ID, 88.9% of all new acute NSAIDs).

However, problems of attribution limited the extension of this to other types of high-risk prescribing. First, the need to identify GPs using PTI modifiers means that extension to analyse new acute NSAIDs in non-PTI practices was not feasible. Second, repeat prescribing and to a lesser extent subsequent acute NSAIDs could not be reliably linked to a clinician ID. Although other clinical systems may always attach a clinician ID to a prescription issue, this in itself would not address the problem that receptionist-generated repeat prescriptions will not necessarily be signed by the GP whose name is on the prescription, which is likely to restrict analysis of prescribing at GP level to acute prescribing. In addition, many high-risk drugs were judged often to be initiated by both GPs and by specialists making recommendations to GPs. Feasible types of prescribing that could be examined are those where a large proportion of prescribing is acute rather than repeat and where the decision-maker is clearly the GP. However, in terms of high-volume drugs, this restricts the scope of analysis to drugs which are usually or often acute prescribed and initiated by GPs, such as analgesics, antibiotics and some psychotropic drugs including antidepressants and possibly hypnotics/anxiolytics (but not antipsychotics). To try to get round this problem of attributing the initiation decision to the GP who starts a repeat medication, we had intended to also examine drug cessation after medication review, which is more clearly attributable to the GP, but in GPASS medication reviews could not be reliably linked to clinician ID so this was not feasible (it might or might not be feasible in other clinical IT systems).

After discussion with the National Institute for Health Research (NIHR) Health Services and Delivery Research (HSDR) programme, it was agreed that we would complete the original study using the data on new acute NSAID prescribing from the 38 PTI practices and in addition examine change in high-risk prescribing over the period 2004–9 in 190 practices.

Data sources

Data were extracted by PCCIU for 38 practices participating in NHS Scotland PTI in 2006 (i.e. practices with higher than average coding quality, and where it is more straightforward to identify what kind of clinicians IT system users are and whether or not an ‘encounter’ is a clinical encounter as opposed to another kind of record opening). Data were complete up to 31 March 2007.

Outcome measures

As described above, we defined the five measures of high-risk NSAID prescribing in a way suitable for implementation in a cross-sectional analysis. Peptic ulceration (PCCIU supplied code-set) and heart failure (QOF-defined code set116) were defined as the presence of any relevant Read Code ever, and the date of the first Read Code was taken as the date of first diagnosis. Oral NSAIDs were defined as all drugs in BNF section 10.1.1. Oral anticoagulants were defined as relevant oral drugs in BNF section 2.8.2. Aspirin and clopidogrel were defined as relevant drugs in BNF section 2.9. 113

Patients were eligible for analysis if they were permanently registered with the practice continuously between 1 October and 31 December 2006, and if they were defined as being vulnerable to adverse effects of a NSAID on the basis of age, existing disease or coprescription. For age and existing disease, patients had to be aged 75 years or over or have a diagnosis of peptic ulcer or heart failure before 1 October 2006. For coprescription, patients had to have been prescribed an anticoagulant or antiplatelet drug either on the same day as a NSAID during the final quarter of 2006, or to have received an anticoagulant and antiplatelet drug both before and after the final quarter of 2006 (data for 1 January 2007 to 31 March 2007 were used to define coprescription to ensure that patients were on a coprescribed drug at the time of any NSAID prescription in Q4 2006). For the coprescription measures, NSAID prescription only triggered the outcome if the relevant coprescribed drug had been issued on the same day as the NSAID or in the 84 days before and the 84 days after the NSAID.

The outcome variable examined was a binary variable defined as the receipt of any oral NSAID between 1 October and 31 December 2006 inclusive. This replicates the methods used in our previous analysis of a broader basket of high-risk prescribing. 5

Additional data were extracted to examine how high-risk NSAID prescribing was associated with patient and practice characteristics:

• At patient level: sex, age, socioeconomic status (measured by quintiles of postcode derived Carstairs Score117), number of active repeat drugs (a measure of overall morbidity and resource use118–120) and the number of indicators that an individual was eligible for (a partial measure of how risky a NSAID might be for them).

• At practice level: list size (in quartiles), practice rurality/remoteness (three aggregated categories of the Scottish Executive Urban Rural Classification), whether or not the practice is accredited for post-graduate training of GPs, whether or not the practice is a dispensing practice, whether or not the practice holds a General Medical Services contract (the standard national contract) or a locally specified contract.

• At practice level: two practice-level variables were created by aggregating from patient-level data, namely the rate at which practices prescribed new acute NSAIDs to patient at risk in 2006 and the rate at which practices prescribed new repeat NSAIDs in 2006. Both variables were analysed by quartile of practice rate. The rationale for these is that three-quarters of high-risk NSAID prescribing in Q4 is a result of repeat prescribing, and first initiation of an acute NSAID and first initiation of a repeat NSAID are key transition points that lead to repeat prescribing, which is subject to less oversight than acute prescribing. These two variables therefore are measures of recent rates of such transitions at practice level.

Descriptive analysis and multilevel modelling

For each indicator and the composite we calculated the percentage [and exact confidence intervals (CIs)] of eligible patients with high-risk prescribing and how prevalence varied at practice level. Figure 1 illustrates the model structure, with the outcome measured at patient level (a light-green circle indicating that a patient did not receive a high-risk prescription and dark-green circle with a cross indicating that they did), and with patients clustered within practices in a many-to-one relationship (each practice has many patients, but all patients only have one practice).

FIGURE 1.

Two-level model of patients within practices. A light-green circle indicates that a patient did not receive a high-risk prescription and a dark-green circle with a cross indicates that they did.

As the outcome is binary at patient level (a patient particularly vulnerable to NSAID ADEs receives a NSAID either in Q4 2006 or now), we fitted a two-level hierarchical multilevel model of patients clustered within practices using the xtmelogit command in Stata Intercooled version 11 (StataCorp LP, College Station, TX, USA). 121–123 The empty model is defined as:

B_1j is the intercept which has an overall fixed component (B₁) but is allowed to vary randomly by GP (u_1j). u_1j is assumed to be normally distributed with mean zero and variance σ²_u1 which is estimated. e_0ij is the encounter-level residual and is assumed to have mean zero with a variance of π²/3. Fixed effects at either level are straightforwardly fitted as in single-level regression models.

The ICC at practice level is defined as the proportion of total variance that is at practice level:

The ICC was initially calculated in the empty model with no fixed effects fitted, defined as the level 2 (practice-level) variance divided by the total variance [the sum of the level 2 variance and the level 1 (patient-level) variance which in a logistic model is fixed at π²/3]. For each of the patient and practice characteristics described above, we calculated univariate odds ratios (ORs) and 95% CIs using the xtmelogit command to account for clustering, and then fitted an ‘adjusted’ model retaining only those variables which were statistically significant. Two adjusted models were fitted, one with only patient characteristics (a case-mix adjusted model) and one with both patient and practice characteristics. Selection of variables to include was based on testing the significance of individual variables, and on the overall fit of the model assessed using the Akaike information criterion (AIC) with a smaller is better criterion. Model assumptions were checked, including checking that the number of integration points used in the estimation was appropriate using the quadchk command (the Stata default of seven integration points were shown to produce stable estimates) and by examining graphically whether level 2 residuals were normally distributed. 122,123 Example plots for the main outcomes can be found in Figure 17 in Appendix 6. Variation between practices after accounting for patient characteristics was examined by estimating the ICC, calculating median odds ratios124 and by creating caterpillar plots of shrunken practice-level residuals with 95% CIs from the adjusted model including patient characteristics. 125,123 Initial data management and descriptive analysis was carried out in IBM Statistical Product and Service Solutions (SPSS) Statistics version 21 (IBM Corporation, Armonk, NY, USA) and regression modelling in Stata Intercooled version 11.

Reliability analysis

In this context, reliability is the extent to which the mean score of a patient-level outcome accurately measures an individual practice’s performance in terms of how well it differentiates between practices in terms of our confidence that observed differences between practices are a result of true differences in prescribing safety. 62,79,126–128 Reliability increases as variation between practices increases (measured by the ICC) and as the number of patients being measured in the practice increases. The ICC is estimated from all the data and does not vary by practice, but the number of patients being measured in each practice varies widely and an indicator may therefore be reliable for a large practice but not for a small one. Reliability varies between 0 (completely unreliable) and 1 (completely reliable). Values greater than 0.7 are generally considered to indicate acceptable reliability in the sense that observed differences between practices can be attributed reasonably confidently to true differences in quality or safety but for high-stakes evaluation such as for governance purposes, reliabilities of > 0.8 are generally believed to be required and > 0.9 may be preferable. In a two-level model, reliability can be estimated with the Spearman–Brown prophecy formula, which is commonly used in this context both to estimate the reliability of a measure for a given number of patients and to estimate the minimum number of patients that a practice must have being measured to achieve a prespecified level of reliability. 79,126

where n = the number of patients and ρ = the ICC.

The same formula can be rearranged so that for an observed ICC ρ, the required number of patients, n, to measure with a desired reliability, λ, is given by:

For the two-level models examined in this chapter, we used the estimated ICC to calculate the reliability for a practice with the median number of patients and to calculate the number of practices in the study for which the proportion of their patients with high-risk prescribing had reliability > 0.7, > 0.8 and > 0.9.

Prevalence of high-risk prescribing for all five indicators and the composite

Table 5 shows the prevalence of high-risk prescribing in the last quarter of 2006 for the five individual indicators and for the composite and the variation between practices. Between 4.3% and 11.1% of those patients particularly vulnerable to NSAID adverse drug effects as defined by the individual indicators received one or more NSAIDs, with 9.5% (95% CI 8.2% to 10.7%) of patients in any indicator receiving a NSAID. The highest prevalence was found for coprescription of a NSAID with aspirin or clopidogrel, and the lowest for coprescription of a NSAID with an oral anticoagulant.

There was evidence of quite considerable variation between practices, with a wide range in the prevalence of high-risk NSAID prescribing between practices, with the composite range being 4.2–21.0%, with an interquartile range of 7.3–11.2%. As is commonly found,79 ICC were relatively small, being < 0.05 for all indicators except for the NSAID and oral anticoagulation measure, where the ICC was 0.184 (95% CI 0.086 to 0.351). The interpretation of the ICC is that it shows the proportion of variation in patient outcome attributable to variation between practices. For example, the ICC for the composite indicator is 0.034 (95% CI 0.020 to 0.056), indicating that 3.4% of variation in outcome between patients is attributable to the practice. The remaining 96.6% of variation is either between patients or is random, but in a multilevel logistic regression model it is not possible to disentangle patient and random variation. 79,121,129

Composite indicator univariate associations

Univariate associations between high-risk NSAID prescribing measured by the composite indicator and patient characteristics are shown in Appendix 3, Table 22. Men were less likely than women to receive a NSAID (8.9% vs. 10.0%, OR 0.88, 95% CI 0.82 to 0.95), as were people aged under 50 years compared with older people. There was no association with socioeconomic deprivation but NSAID prescribing was less likely in people who had more than one risk factor (6.8% in those eligible for three or more of the five individual indicators vs. 9.8% in those eligible for one, OR 0.66, 95% CI 0.55 to 0.79). There was a strong association between high-risk NSAID prescribing and the number of repeat drugs that a patient is being currently prescribed, with the prevalence of high-risk NSAIDs increasing from 4.4% in those with no active repeats to 17.9% in those with 11 or more active repeats (OR 4.53, 95% CI 3.73 to 5.49).

Associations with practice-level characteristics were generally weaker (see Appendix 3, Table 23). None of the practice structural characteristics was associated with high-risk prescribing (list size, urban/rural location, type of contract, training or dispensing status). A practice’s rate of new acute NSAID prescribing earlier in 2006 was associated with any NSAID use in Q4 2006 [prevalence 12.2% in practices in the highest quartile of new acute NSAID prescribing compared with 7.1% in the lowest quartile, OR 1.83 (95% CI 1.37 to 2.43), and a consistent gradient]. There was a weaker association with practice rates of new repeat NSAID prescribing although this is a rarer event [prevalence 12.0% in practices in the highest quartile of new repeat NSAID prescribing compared with 7.9% in the lowest quartile, OR 1.42 (95% CI 1.03 to 1.96), but a less clear gradient across quartiles].

Composite indicator adjusted analysis

In multivariate analysis, only four variables were significantly associated with high-risk NSAID prescribing (Table 6) – patient age, the number of indicators patients were eligible for, the number of active repeat drugs and practice new acute NSAID prescribing earlier in 2006. After adjustment, there were significant differences only between the under-50-year-olds (reference category) and people aged 80 years and over, who were less likely to be prescribed a high-risk NSAID (OR 0.71, 95% CI 0.73 to 0.85, compared with the under-50-year-olds). The association with the number of indicators that a patient was eligible was strengthened somewhat (OR 0.49, 95% CI 0.41 to 0.59, in those with three or more risk factors vs. those with one), as was the association with the number of active repeat drugs a patient was prescribed (OR 5.62, 95% CI 4.59 to 6.89, in those with 11 or more active repeats vs. those with none). Practice rates of new acute NSAID prescribing earlier in 2006 remained significantly associated with any high-risk NSAID use in Q4 2006 (OR 1.80, 95% CI 1.41 to 2.30, in practices in the highest quartile compared with the lowest quartile).

The ICC in the empty model was 0.034 (95% CI 0.020 to 0.056), compared with 0.026 (95% CI 0.015 to 0.044) in the adjusted model with only patient-level fixed effects and 0.015 (95% CI 0.008 to 0.026) in the adjusted model with both patient-level and practice-level fixed effects. An alternative way of expressing variation between practices is the median odds ratio, which is on the same scale as the fixed effects and is the median odds ratio between a patient randomly selected from one practice and another randomly selected from another practice. 124 The median odds ratio in the empty model was 1.38 (95% CI 1.28 to 1.53) and in the fully adjusted model 1.23 (95% CI 1.17 to 1.33), indicating that variation because of differences in patient characteristics is much greater than variation owing to differences in the practice that a patient is registered with. Figure 2 shows variation by practice based on estimated shrunken residuals for each practice from the model adjusted for patient characteristics only. Six of the 38 practices had statistically significantly higher rates of high-risk NSAID prescribing compared with the population average, and nine had significantly lower rates.

FIGURE 2.

Residual variation between practices after accounting for patient characteristics significantly associated with any high-risk NSAID prescribing in Q4 2006 (practice-level shrunken residuals estimated from the ‘patient characteristics only’ adjusted model in Table 6).

Individual and composite indicator reliability

Table 7 shows the estimated reliability of the five individual indicators and the composite, both before and after accounting for patient characteristics. Reliability varies with both the ICC and the number of patients being measured in each practice. Reliability for a practice with the median number of patients was 0.9 or above for all indicators except for NSAID prescribing in heart failure where it was 0.67. All included practices could be measured with reliability > 0.7 for three of the individual indicators and the composite, but the majority of practices could not be reliably measured at this level for the NSAID in heart failure measure. All included practices could be measured with reliability > 0.8 for the composite indicator, and all but one for NSAIDs prescribed to older people and to people prescribed an oral anticoagulant. None of the individual indicators was reliable in more than three-quarters of included practices, but the composite had reliability > 0.9 in 35 of the 38. Adjusting for patient characteristics made no meaningful difference to the reliability of the composite indicator.

Summary of findings

Prescription of NSAIDs to patients at particularly high risk of adverse drug effects was relatively common, with just under 1 in 10 such patients receiving a NSAID in Q4 2006. In the full multilevel model, high-risk NSAID prescribing was associated with patient age (over-80-year-olds being less likely to receive a NSAID), the number of indicators that a patient was eligible for (those with more risk factors being less likely to receive a NSAID), and the number of repeat drugs a patient was taking (with a progressive increase in NSAID prescribing as the number of repeat drugs increased). At practice level, the only significantly associated variable was the practice rate of new acute NSAID prescribing earlier in 2006 (practices with higher rates of such prescribing had higher rates of total prescribing in Q4 2006).

Variation between practices was fairly large in absolute terms (varying from 4.1% to 21.0% for the composite) but fairly small when measured using the ICC, where less than 5% of variation in patient outcome was attributable to variation between practices. This applied to the composite (ICC in the empty model 0.035) and four of the five individual measures (ICCs varying from 0.026 to 0.043), although for NSAID prescription to people already prescribed oral anticoagulants the ICC was much larger (0.184). Of note is that when variation between practices is expressed as a median odds ratio (the difference in odds of receiving a high-risk NSAID between two patients randomly selected from different practices), then the fixed effects in the model generally had larger associations with high-risk prescribing than the practice the patient happened to be in (median odds ratio in the empty model was 1.38 and 1.23 in the fully adjusted model both smaller than the most weakly associated patient or practice characteristic). Our overall interpretation is that for total high-risk NSAID prescribing, there is small but statistically and clinically significant variation between practices.

Despite the relatively low ICC, four of the five individual indicators were reasonably reliable (the exception being NSAIDs in heart failure which was not) in their ability to distinguish practices from each other, at least to the level where the indicators could be used in more formative ways (reliability > 0.7), although only for NSAIDs in older people and NSAIDs coprescribed with oral anticoagulants was reliability considered adequate for higher-stakes evaluation (reliability > 0.8) in nearly all included practices. Reflecting the large numbers of patients eligible for the measure (i.e. particularly vulnerable to NSAID adverse drug effects), the composite indicator had reliability > 0.8 in all included practices and > 0.9 in 35 of the 38 practices.

Strengths and limitations

The analysis uses routinely collected data from GP electronic medical records, which has a number of strengths and a number of matching weaknesses. Key strengths are that GP records are generally near complete for prescribing and have reasonable accuracy for common diagnostic coding such as heart failure and peptic ulcer. Key weaknesses are that some prescriptions are hand-written and so will not be recorded electronically (particularly prescriptions written during home visits which are most commonly for frailer patients who are most at risk of ADEs), and that the practices being studied are relatively unusual in that they contribute to a national morbidity recording data set (although this does mean that their diagnostic coding will likely be of higher quality than average). However, the findings are similar to our previous work in a representative one-third of Scottish practices, in terms of the prevalence of high-risk prescribing measured by the indicators examined, the amount of variation between practices and the reliability of the indicators. 5 It is also important to note that the practice characteristics data that are available predominantly relate to practice structural characteristics rather than to indicators of how practices organise their prescribing systems or how practitioners within them conceive the risk of the measured prescribing. The partial exception to that is the measure of new acute NSAID prescribing in the previous year, which is associated with total prescribing in Q4 2006.

In addition, composite indicators have a number of potential disadvantages, including that they may combine indicators with very different implications and prevalences. In this case, the composite consists of people prescribed a single drug class where each indicator is clearly associated with risk, although given the numbers of people affected it will be little influenced by NSAID prescribing to people taking oral anticoagulation and to lesser extent NSAID prescribing to people with heart failure. Finally, we do not know the specific indication for the NSAID, which will of course be critical for understanding the risk–benefit balance, as the benefits in people with inflammatory arthritis are likely to be greater (and harder to achieve with other drugs) than in people with mild to moderate osteoarthritis.

Interpretation and comparison with existing literature

This analysis replicates our previously applied methods for the set of indicators and practices used in this study. Of note is that the lower rates of NSAID use in the oldest patients and those with more than one risk factor (i.e. included in more than one indicator) is consistent with there being a general recognition by prescribing GPs that NSAIDs are risky. Rising rates of NSAID use as the number of repeat drugs increases likely reflects greater need for analgesics in sicker people, although of course the risks also increase. The association between prior new acute NSAID use and total NSAID use in Q4 2006 is not unexpected, but reinforces the belief that new acute NSAID use is a reasonable outcome to use in the analysis which follows. Ideally, data would be available on how practices organise their prescribing systems and their knowledge of and attitudes to prescribing risk, although such data are unlikely to ever be routinely available.

The observed absolute variation between practices and the ICCs are similar to our previous study of high-risk prescribing,5 the exception being the NSAIDs coprescribed with oral anticoagulants measure, for which the ICC was 0.184 (95% CI 0.086 to 0.351) which is at the upper end of observed inter-practice variation in other studies, consistent with practices (or GPs within practices) varying considerably in their perception of the risk of this prescribing. 79 Small ICCs of the kind observed for the other indicators and the composite are the norm in studies of variation in quality, particularly for binary outcomes where the level 1 variance combines both variation between patients and chance variation. ICCs of this magnitude are not infrequently assumed to mean that there is little or only trivial variation in technical quality between practices or other higher-level organisations or areas. 62,85–87,130 However, as others have pointed out, small ICCs at practice or other higher level are quite compatible with large absolute variation between practices if total variation is also large,79,131,132 and the absolute variation in high-risk NSAID use between practices was large for all indicators including those where there were large numbers of patients being measured.

The reliability of an indicator is a measure of how well it distinguishes between practices, or more formally it is the proportion of the observed variation between practices which is because of true differences between practices. 62,79,126,128 Reliability varies between 0 (completely unreliable) and 1 (completely reliable) and it is an arbitrary decision as to how reliable a measure must be to be useful. Commonly applied rules of thumb are that reliability > 0.7 is adequate for formative evaluation, but values > 0.8 and ideally > 0.9 are required for summative or high-stakes evaluation, with Streiner and Norman128 suggesting that > 0.75 is the minimum requirement under most circumstances but also noting that few instruments have reliability > 0.9. In part, this depends on the context and in particular the use that a measure will be put to. For example, the reliability required for feeding data back to practices for reflection as part of a formative appraisal process is less than that required for a high-stakes evaluation such as identification of practices for clinical governance investigation or intervention or tournament-based pay-for-performance schemes where there are both winners and losers. 62,126

From this perspective, four of the five individual measures examined here would be suitable for formative assessment (the exception being NSAIDs in heart failure) with reliabilities > 0.7 in almost all practices, and the composite indicator would be suitable for high stakes evaluation with reliability > 0.8 in all practices and > 0.9 in all but the smallest practices. These high reliabilities are despite a relatively small ICC, reflecting that at practice level there are a large number of patients particularly vulnerable to NSAID ADEs, with reliability varying with both the ICC (the same in all practices) and the number of patients being measured (varying between practices). Adjusting for the available patient characteristics made little difference to the reliability, which would simplify implementation in NHS or other health-care settings.

Data sources

As previously described, data were extracted by PCCIU for 38 practices participating in NHS Scotland PTI in 2006 (i.e. practices with higher than average coding quality who routinely record Read Code diagnostic data during clinical encounters as part of contributing to a national data set). Data were complete up to 31 March 2007.

Outcome measures

We defined the same five measures of high-risk NSAID prescribing described above in a way suitable for implementation in an encounter level analysis over one calendar year (see Table 20 in Appendix 2). Peptic ulceration (PCCIU supplied code set) and heart failure (QOF code set116) were defined as the presence of any relevant Read Code ever, and the date of the first Read Code was taken as the date of first diagnosis. Oral NSAIDs were defined as all drugs in BNF section 10.1.1. Oral anticoagulants were defined as relevant oral drugs in BNF section 2.8.2. Aspirin and clopidogrel were defined as relevant drugs in BNF section 2.9. 113

An encounter with a patient was eligible for analysis if it occurred between 1 January 2006 and 31 December 2007 inclusive, if it was at a time when the patient was permanently registered and at a time when the patient was defined as being vulnerable to adverse effects of a NSAID on the basis of age, existing disease or coprescription at the time of the encounter and the patient had not had a NSAID prescribed in the year before the encounter (as it is then not possible for them to be prescribed a new acute NSAID according to the way we defined this type of prescription) and if the GP being encountered had at least 10 encounters with patients at risk during 2006. The data set for analysis of variation between practices and between GPs given the constraints described above included 181,010 (99.9%) potentially eligible encounters by 26,539 (99.97%) eligible patients at risk in 2006 who consulted at least once. During these encounters 1953 new acute NSAIDs were issued (96.5% of all NSAIDs prescribed to these patients in 2006 where there was a clinician ID recorded).

For age and existing disease, patients had to be aged 75 years or over, or have a diagnosis of peptic ulcer or heart failure recorded before the encounter date. For coprescription, patients had to have been prescribed an anticoagulant or antiplatelet drug in both the 84 days before and the 84 days after the encounter. One binary variable composite outcome was defined – receipt of a new acute NSAID during an encounter.

Additional data were extracted to examine how high-risk NSAID prescribing was associated with encounter, GP and practice characteristics:

• At encounter level: the encounter type, the number of indicators a patient was eligible for at the time of the encounter, and whether or not the patient had a ‘relevant diagnosis’ recorded at the time of the encounter (defined as a Read Code from chapter N musculoskeletal conditions, chapter R ill-defined conditions/working diagnoses, chapter S injuries and poisoning, and chapter 1 history/symptoms). 133 We additionally fitted variables which were ‘characteristics of the patient having the encounter’ including sex, age, socioeconomic status (measured by quintiles of postcode derived Carstairs Score117), and number of active repeat drugs at the start of 2006 (a measure of overall morbidity and resource use118–120). Such characteristics would be more appropriately fitted at patient level in a cross-classified model, but such a model was feasible. The main implication is that although the estimated association is likely to be unbiased, the standard errors and, therefore, the CIs estimated for these variables will be falsely narrow because the model is assuming that they are being measured more often than they actually are being (e.g. a patient who consults six times has only had their socioeconomic status measured once, but the model assumes it has been measured six times).

• At GP level, GP sex (recorded in the original data) and the number of encounters each GP had with patients at risk during the year (calculated from the encounter data and categorised into quartiles). There were no other data on GP characteristics available.

• At practice level for use in both analyses: list size (in quartiles), practice rurality/remoteness (three aggregated categories of the Scottish Executive Urban Rural Classification), whether or not the practice is accredited for post-graduate training of GPs, whether or not the practice is a dispensing practice, whether or not the practice holds a General Medical Services contract (the standard national contract) or a locally specified contract.

Descriptive analysis and multilevel modelling

For each indicator and the composite, we calculated the percentage (and exact CIs) of eligible encounters in which a new acute NSAID was prescribed and how prevalence varied at GP level and practice level. Figure 6 illustrates the model structure, with the outcome measured at encounter level (a light-green circle indicating that a patient did not receive a high-risk prescription and a dark-green circle indicating that they did), and with encounters clustered within GPs within practices in many-to-one relationships (each practice has many GPs, but all GPs only have one practice; each GP has many encounters, but all encounters only have one GP).

FIGURE 6.

Three-level model of encounters within GPs within practices. A light-green circle indicates that a patient did not receive a high-risk prescription and a dark-green circle with a cross indicates that they did.

As the outcome is binary at encounter level (a new acute NSAID is either prescribed in an eligible encounter or not), we fitted a three-level hierarchical multilevel model of encounters clustered within GPs clustered within practices using the xtmelogit command in Stata. 121–123 The empty model is defined as:

B_1jk is the intercept which has an overall fixed component (B₁) but is allowed to vary randomly by GP (u_1jk) and by practice (v_1k). u_1jk and v_1k are assumed to be normally distributed with mean zero and variances σ²_v1 and σ²_u1 which are estimated. e_0ijk is the encounter level residual and is assumed to have mean zero with a variance of π²/3. Fixed effects at any level are straightforwardly fitted as in single level regression models.

The ICC was initially calculated in the empty model with no fixed effects fitted. The ICC at practice and at GP level are defined as the proportion of total variance that is at each level:

For each of the encounter, GP and practice characteristics described above, we calculated univariate ORs and 95% CIs using xtmelogit to account for clustering, and then fitted an ‘adjusted’ model retaining only those variables which were statistically significant. Two adjusted models were fitted, one with only encounter characteristics (a case-mix adjusted model) and one with all of encounter, GP and practice characteristics. Selection of variables to include was based on testing the significance of individual variables and on the overall fit of the model assessed using the AIC. Model assumptions were checked, including by checking that the number of integration points used in the estimation was appropriate using the quadchk command (the Stata default of seven integration points were shown to produce stable estimates) and by examining graphically whether or not level 2 (GP) and level 3 (practice) residuals were normally distributed. 122,123 Example plots for the main outcomes can be found in Figure 17 in Appendix 6. Variation between practices and between GPs after accounting for encounter characteristics was examined by calculating median odds ratios124 and by creating caterpillar plots of shrunken practice- and GP-level residuals with 95% CIs from the adjusted model including encounter characteristics. 123,125

How GPs with statistically significantly higher than average high-risk new acute NSAID prescribing were distributed across practices which themselves had higher than average, average or lower than average high-risk new acute NSAID prescribing was then examined. We then repeated this to examine the distribution of the same GPs across practices with higher, average or lower high-risk prescribing of any NSAIDs in Q4 2006 as identified in the two-level multilevel model reported in the previous chapter.

Initial data management and descriptive analysis was carried out in IBM SPSS Statistics version 21 and regression modelling in Stata Intercooled version 11.

Reliability

Estimating reliability in a three-level model is somewhat more complex than a two-level model because the Spearman–Brown prophecy formula does not straightforwardly extend to this situation (although it has been used to estimate reliability for physician level measurement using ICCs estimated in three or more level models). We, therefore, examined reliability using generalizability theory (G-theory), which is an approach to partitioning variance developed in the context of educational assessment. 128 Classical test theory assumes that a test score is composed of a true score plus an error score because the instrument is imperfect. Reliability is then defined as the ratio of variance in the true score to the observed variance which is composed of both true and error variance. In practice, there are many different kinds of reliability depending on what is being measured, including scores made by one observer at two time points (intraobserver or intrarater reliability), scores made by different observers at the same time point (interobserver reliability), different items on a multi-item scale (internal consistency) and so on. In practice, there will often be multiple sources of error, and G-theory allows measured error to be partitioned in a G-study based on the actual data, and then to be modelled in a decision study which examines how reliability would change by altering the number of observations or other parameters. 128 The aim of D-studies is to inform consideration of the feasibility of implementing a reliable system to differentiate practices and doctors in terms of high-risk prescribing.

From this perspective, we can conceive of each encounter a GP has with a patient at risk as being a binary outcome (prescribed or not prescribed) test of their likelihood of prescribing a NSAID to a patient particularly vulnerable to NSAID adverse drug effects. We used urGENOVA programme and associated software (G_String_IV; The Program for Educational Research and Development, faculty of Health Sciences, McMaster University, Hamilton ON, Canada. Available from: http://fhspend.mcmaster.ca/g_string/) to conduct G-studies and associated D-studies. However, G_String_IV could not analyse the full data set, and we, therefore, used data for GPs with at least one hundred encounters to carry out a G-study, with the focus then being on the D-studies based on this analysis. These studies allowed the calculation of the different reliabilities achieved when discriminating between practices across a range of different possible numbers of encounters and GPs, and the reliabilities achieved when discriminating between GPs across a range of different possible numbers of encounters. 128,134 Only the results of the D-studies are presented.

Prevalence of high-risk new acute non-steroidal anti-inflammatory drug prescribing for all five indicators and the composite

Table 8 shows the prevalence of high-risk new acute prescribing in 2006 for the five individual indicators and for the composite, and variation between practices and between GPs. Between 0.20% and 1.59% of encounters with those particularly vulnerable to NSAID ADEs as defined by the individual indicators involved the prescription of a new acute NSAID, with 1.08% (95% CI 1.03% to 1.13%) of encounters with any patient at risk involving a new acute NSAID prescription. The highest rate of new acute NSAID prescribing was to people with a history of peptic ulcer disease and the lowest for coprescription of a NSAID with an oral anticoagulant.

There was evidence of variation between practices, with the minimum to maximum range of prevalence for the composite being 0.37–3.50%, and the interquartile range 0.76–1.51%. For two indicators, the minimum at practice level was zero (NSAIDs and heart failure; NSAIDs coprescribed with an oral anticoagulant), and for one the lowest quartile was zero (NSAIDs coprescribed with an oral anticoagulant). Variation at GP level was greater, with the minimum to maximum range of prevalence for the composite being 0–20.0%, and the interquartile range 0–1.50%. For all indicators, the minimum at GP level was zero, and for three individual indicators and the composite, the lowest quartile was zero (NSAIDs with peptic ulcer; NSAIDs and heart failure; NSAIDs coprescribed with an oral anticoagulant).

Figure 7 shows variation between GPs within practices. GPs are grouped within practices, and GP rates of new acute NSAID prescribing are plotted in ascending order of practice rate, and then ascending order of GP rate within practice. Visually, there is a reasonable amount of within practice variation across the entire range of practice variation. On the face of it, there is much more variation between GPs than between practices, but this would be expected to be at least partly driven by there being fewer encounters per GP than per practice. One hundred and thirty-three (33.4%) GPs had no high-risk NSAID prescribing and, not unexpectedly, the GPs with the highest and the lowest (zero) rates of prescribing are most commonly those with relatively few encounters, with, for example, the GP with the ‘highest’ prescribing (20%) having 15 encounters with eligible patients and prescribing three of them a new acute NSAID (Figure 8).

FIGURE 7.

League table of frequency of new acute NSAID prescribing in encounters with patients at risk by GPs within practices (in ascending order of practice rates, then ascending order of GP rates).

FIGURE 8.

Scatterplot of frequency at GP level of new acute NSAID prescribing in encounters with patient at risk and number of encounters with patients at risk each GP has.

Intraclass correlation coefficients estimated from the three-level empty model are shown in Table 9. These can be interpreted as showing the proportion of variation in encounter outcomes that is attributable to either practice or clinician. The proportion of variation attributable to variation between practices ranged from 1.9% (NSAIDs coprescribed with anticoagulants) to 8.7% (NSAIDs and heart failure), and was 5.5% for the composite. The proportion of variation attributable to variation between GPs ranged from 12.2% (NSAIDs and heart failure) to 17.1% (NSAIDs coprescribed with oral anticoagulants) and was 16.6% for the composite. The proportion of variation attributable to either practices or GPs ranged from 14.6% (NSAIDs and peptic ulcer) to 25.0% (NSAIDs coprescribed with anticoagulants), and was 20.5% for the composite. For every indicator, a higher proportion of the variation was attributable to GPs than to practices, ranging from 1.4 times more at GP level for NSAIDs in heart failure to 6.7 times for NSAIDs in peptic ulcer and 3.0 times more at GP level for the composite. Remaining variation is a mixture of variation between encounters and random variation but cannot be further partitioned. 123,125,129 At practice level, the estimated ICCs are similar to that found in other studies and at GP level the ICCs are at the upper end of observed ranges. 79

The median odds ratio is an alternative way of examining variation between GPs and between practices, although it is most useful in models with fixed effects fitted, as it allows comparison of the magnitude of fixed and random effects on the same scale. 124 Two median odds ratios in the various empty models are shown in Table 10. The GP-level median odds ratio is the median odds ratio between two randomly selected eligible encounters with the different GPs in the same practice. The practice-level median odds ratio is the median odds ratio between two randomly selected eligible encounters with different GPs in different practices. The practice-level median odds ratio should therefore be interpreted in terms of its difference from the GP-level median odds ratio, as it represents the additional variation in outcome from seeing a different GP in a different practice, compared with a different GP in the same practice. The median odds ratio at GP level varied between 1.95 and 2.29 for the individual indicators, and was 2.22 for the composite. At practice level, median odds ratios only increased in size by a small amount, varying between 2.04 and 2.71 for the individual indicators and 2.52 for the composite.

Composite indicator univariate associations

Table 24 in Appendix 4 shows how the percentage of high-risk new acute NSAID prescribing varies by encounter characteristics and the characteristics of the patient in the encounter, with the same for GP and practice characteristics shown in Table 25, Appendix 4. The equivalent multilevel univariate ORs are shown in Tables 26 and 28, Appendix 4, and the univariate findings are briefly summarised here.

Compared with normal surgery encounters (face-to-face in a normal surgery), high-risk new acute NSAID prescribing was less common in telephone or other/unknown encounter types [0.46% for telephone, 0.36% for other vs. 1.33% for normal surgery, OR 0.26 (95% CI 0.20 to 0.33) for telephone, OR 0.31 (95% CI 0.25 to 0.37) for unknown/other]. As in the previous analysis, the more risk factors for NSAID ADEs that a patient had, the less likely they were to receive a NSAID (0.48% in those eligible for three or more of the five individual indicators vs. 1.38% in those eligible for one, OR 0.35, 95% CI 0.28 to 0.42). Encounters where a ‘relevant diagnosis’ had been recorded were much more likely to have a NSAID prescribed (2.72% vs. 0.40% in those without a relevant diagnosis recorded, OR 7.12, 95% CI 6.48 to 7.97). Encounters with older people were much less likely to result in a NSAID prescription (2.18% in those under 50 years vs. 0.66% in those aged 80 years and over, OR 0.31, 95% CI 0.26 to 0.37), as were encounters with patients taking more repeat medication (1.76% in those on no repeat drugs vs. 0.62% in those taking 11 or more, OR 0.32, 95% CI 0.24 to 0.42). Encounters with a male patient were slightly more likely to result in a NSAID prescription (1.20% vs. 0.98% in women, OR 1.18, 95% CI 1.08 to 1.29), but there was no association with socioeconomic status.

Male GPs were more likely than women to prescribe a high-risk new acute NSAID (1.32% vs. 0.68%, OR 1.82, 95% CI 1.44 to 2.31), but there was no association with the number of encounters GPs had. None of the practice characteristics examined was associated with new acute NSAID prescribing.

Composite indicator adjusted analysis

In multivariate analysis, six variables were statistically significantly associated with high-risk NSAID prescribing (see Table 10) – three encounter characteristics (encounter type, the number of indicators the patient was eligible for at the time of the encounter and the presence of a relevant diagnosis code for the encounter), two characteristics of patients being encountered (patient age and the number of active repeat drugs the patient was on), and one GP characteristic (GP sex). Two models were fitted, one without GP sex and one with. Associations with encounter level variables did not meaningfully change with the inclusion of GP sex, so the text that follows refers to the ORs in the final model.

The most strongly associated variable was whether or not a relevant diagnosis had been recorded for the encounter (OR 7.03, 95% CI 6.32 to 7.82), which is not surprising (the variable was included primarily to account for case-mix variation between GPs94). Most of the other observed univariate associations weakened somewhat after adjustment, although all remained statistically significant. Compared with normal surgery encounters, the association between new acute NSAID prescribing and telephone encounters weakened somewhat after adjustment but remained statistically significant (OR 0.68, 95% CI 0.52 to 0.89) and the association with other/unknown encounters remained the same (OR 0.34, 95% CI 0.29 to 0.42). Associations with the number of risk factors a patient had also weakened but remained statistically significant (OR 0.61. 95% CI 0.49 to 0.76, in those eligible for three or more of the five individual indicators vs. those eligible for one). There were no statistically significant differences between encounters with those aged 50–59 years or 60–69 years compared with the under-50-year-olds, but encounters with people aged 70–79 years (OR 0.74, 95% CI 0.62 to 0.89, compared with the under-50-year-olds) and with those 80 years and over (OR 0.59, 95% CI 0.49 to 0.72) were less likely to result in a high-risk new acute NSAID. Encounters with people taking 11 or more repeats had half the odds of resulting in a new acute NSAID than those with people with no active repeat drugs (OR 0.51, 95% CI 0.39 to 0.68). GP sex remained statistically significantly associated with high-risk new acute NSAID prescribing (OR 1.73, 95% CI 1.39 to 2.16, for male GPs compared with female).

The practice-level ICC in the empty model was 0.055 (95% CI 0.029 to 0.102), compared with 0.042 (95% CI 0.021 to 0.083) in the adjusted model with only encounter-level fixed effects and 0.031 (95% CI 0.014 to 0.068) in the adjusted model with both encounter-level and GP-level fixed effects. Including the encounter-level fixed effects therefore slightly increased the estimated variation between practices, whereas accounting for GP sex slightly reduced it. The GP level ICC in the empty model was 0.166 (95% CI 0.135 to 0.197), compared with 0.142 (95% CI 0.114 to 0.173) in the adjusted model with only patient-level fixed effects and 0.131 (95% CI 0.103 to 0.161) in the adjusted model with both patient- and GP-level fixed effects.

The median odds ratio is an alternative way of examining variation between GPs and between practices, and expresses random variation on the same OR scale as the estimates of associations with fixed effects. 124 Two median odds ratios were calculated. The GP-level median odds ratio is the median odds ratio between two randomly selected eligible encounters with the different GPs in the same practice. The practice-level median odds ratio is the median odds ratio between two randomly selected eligible encounters with different GPs in different practices. The practice-level median odds ratio should therefore be interpreted in terms of its difference from the GP-level median odds ratio, as it represents the additional variation in outcome from encountering a different GP in a different practice, compared with encountering a different GP in the same practice. The median odds ratio at GP level was 2.22 (95% CI 2.00 to 2.50) in the empty model, 2.06 (95% CI 1.87 to 2.30) in the model with only encounter-level fixed effects and 1.98 (95% CI 1.80 to 2.21) in the adjusted model with both encounter and GP fixed effects. The median odds ratio at practice level was 2.52 (95% CI 2.15 to 3.09) in the empty model, 2.28 (95% CI 1.98 to 2.76) in the model with only encounter-level fixed effects, and 2.14 (95% CI 1.88 to 2.57) in the adjusted model with both encounter and GP fixed effects. Put another way, encountering a different GP in the same practice has a similar magnitude of association with outcome as the adjusted association between high-risk prescribing and age or the number of repeat drugs. Encountering a different GP in a different practice has slightly higher ORs but a very similar magnitude.

Figure 9 shows variation by practice based on estimated shrunken residuals for each practice from the model adjusted for patient characteristics only. Three of the 38 practices had statistically significantly higher rates of high-risk new acute NSAID prescribing than the population average, and two had statistically significantly lower rates. At GP level, 55 out of 398 GPs had statistically significantly higher rates of new acute NSAID, and 10 had statistically significantly lower rates (Figure 10). The majority of GPs in practices which had statistically significantly higher new acute NSAID prescribing did not themselves have statistically significantly higher rates of such prescribing. Of the 46 GPs in the three practices with statistically significantly higher new acute NSAID prescribing, eight had statistically significantly higher new acute NSAID prescribing, two had statistically significantly lower and 36 were not statistically different from average (Figure 11). Overall, therefore, of the 55 GPs with statistically significantly higher than average new acute NSAID prescribing, eight (15.7%) were in the three practices with statistically significantly higher than average prescribing (7.9% of practices with 15.3% of GPs), three (5.9%) were in the two practices with statistically significantly lower than average prescribing (5.3% of practices with 11.6% of GPs), and 40 (78.4%) were in the remaining 33 practices (78.4% of practices with 73.1% of GPs).

FIGURE 9.

Residual variation between practices after accounting for patient characteristics statistically significantly associated with high-risk new acute NSAID prescribing (practice-level shrunken residuals estimated from the adjusted model in Table 10).

FIGURE 10.

Residual variation between GPs after accounting for patient characteristics statistically significantly associated with high-risk new acute NSAID prescribing (practice-level shrunken residuals estimated from the adjusted model in Table 10).

FIGURE 11.

Variation in high-risk new acute NSAID prescribing between GPs in practices with higher than average high-risk new acute prescribing (practice-level shrunken residuals estimated from the adjusted model in Table 10).

Generalizability theory reliability studies

Table 11 gives the reliabilities achieved for the ability to differentiate between practices’ prescribing safety for different numbers of combinations of observation (encounters per doctor and number of doctors within practices). As expected, the ability to differentiate between practices increased with the number of encounters and the number of GPs. Reliability was good (G = 0.713) for practices with 15 GPs each having 200 encounters, which is adequate for quality improvement purposes, but it was only moderate for practices with five GPs and 500 encounters per GP (G = 0.526). Of note is that in the study data set, the interquartile range for the number of GPs per practice is 6–14, and for encounters per GP 90–766. Reliability at practice level is therefore relatively poor for the majority of practices.

Table 12 shows the reliabilities (ability to differentiate between individual doctors’ prescribing safety) for a given number of encounters for each doctor independent of practice. High variance between doctors (which contributed to the relatively poor reliability in differentiating between practices) resulted in moderate reliability (G = 0.699) for 200 encounters per GP and high reliability (G = 0.823) for 400 encounters per GP. In total, 61% of GPs in this data set had ≥ 200 encounters with patients particularly vulnerable to high-risk NSAID prescribing in 2006 and 44.7% had ≥ 400 encounters, indicating that in practice only GPs with relatively heavy clinical workloads could be reliably measured.

Summary of findings

New acute NSAID prescribing to patients at particularly high risk of NSAID-related ADEs occurred in 1.1% of eligible encounters in 2006, varying by individual indicator from 0.20% of encounters with people also prescribed an oral anticoagulant to 1.59% of people with a history of peptic ulceration. In the full multilevel model, high-risk NSAID prescribing was unsurprisingly much more common in encounters with a recorded diagnosis of a musculoskeletal or pain-related symptom (OR 7.03, 95% CI 6.32 to 7.82), and less likely to occur in telephone (OR 0.68, 95% CI 0.52 to 0.89) or other/unknown (OR 0.34, 95% CI 0.29 to 0.42) encounter types compared with normal surgery encounters (i.e. face to face in a normal surgery). Encounters with patients with more risk factors for NSAID ADEs were less likely to end with a new acute NSAID prescription, as were encounters with patients aged 70–79 years (OR vs. the under-50-year-olds 0.74, 95% CI 0.62 to 0.89) and aged 80 years and over (OR 0.59, 95% CI 0.49 to 0.72), and those prescribed more repeat drugs. Male GPs were more likely than female GPs to prescribe a high-risk new acute NSAID (OR 1.73, 95% CI 1.39 to 2.16).

There was evidence of variation between GPs and between practices, with the minimum to maximum range of prevalence for the composite being 0–20% at GP level (interquartile range 0–1.5%) and 0.37–3.50% at practice level (interquartile range 0.76–1.51%). One-third of GPs had no high-risk NSAID prescribing, although the extremes of the distribution at GP level were dominated by those with small numbers of encounters with eligible patients. Graphical examination of rates of high-risk NSAID prescribing was consistent with there being considerable variation between GPs within practices, as well as between GPs more generally. Using empty model estimates of the ICC, the proportion of variation in outcome attributable to variation between practices ranged from 1.9% (NSAIDs and peptic ulcer) to 8.7% (NSAIDs and heart failure), and was 5.5% for the composite. The proportion of variation in outcome attributable to variation between GPs ranged from 12.2% (NSAIDs and heart failure) to 17.1% (NSAIDs coprescribed with oral anticoagulants), and was 16.6% for the composite. For every indicator, there was greater variation at GP level than at practice level. At practice level, the estimated ICCs are similar to that found in other studies, and at GP level the ICCs are at the upper end of observed ranges in other studies. 79

The median odds ratio expresses variation between practices on the same scale as the fixed effects such as patient age or GP sex. The median odds ratio at GP level can be interpreted as the median difference in the odds of receiving a new acute NSAID if the same patient were to randomly encounter two different GPs in the same practice. The median odds ratio at GP level was 2.22 (95% CI 2.00 to 2.50) in the empty model, 2.06 (95% CI 1.87 to 2.30) in the model with only encounter-level fixed effects and 1.98 (95% CI 1.80 to 2.21) in the adjusted model with both encounter and GP fixed effects. The median odds ratio at practice level can be interpreted as the median difference in the odds of receiving a new acute NSAID if the patient were to randomly encounter two different GPs from different practices (but should be interpreted in terms of how different it is from the median odds ratio at GP level, as it includes variation between GPs as well as between practices). The median odds ratio at practice level was 2.52 (95% CI 2.15 to 3.09) in the empty model, 2.28 (95% CI 1.98 to 2.76) in the model with only encounter-level fixed effects and 2.14 (95% CI 1.88 to 2.57) in the adjusted model with both encounter and GP fixed effects. Variation between GPs is, therefore, of a similar magnitude as all of the encounter and GP characteristics included as fixed effects apart from an encounter for a relevant diagnosis.

Three practices and 51 GPs had statistically significantly higher than average high-risk new acute NSAID prescribing after accounting for encounter characteristics. However, most GPs in practices with higher than average new acute NSAID prescribing were not themselves higher than average, and 43 GPs with higher than average prescribing were in practices which were average or statistically significantly lower than average.

Reliability of measurement of high-risk prescribing was poor at practice level, with only the minority of practices with 15 or more GPs being able to be measured with reliability ≥ 0.7. As expected given the higher ICC at GP level, reliability of measurement was better for GPs with just over 60% having enough encounters to be measured with reliability ≥ 0.7 (which is adequate for comparative feedback for improvement purposes) but only 45% having enough to be measured with reliability ≥ 0.8 (which is required for higher-stakes evaluation).

Strengths and limitations

This analysis has the same general strengths and limitations as the analysis described in Chapter 3. It additionally has some important other limitations relating to the relative lack of data about GPs and about practices and to only being able to examine new acute NSAID prescribing.

As noted in Chapter 3, the practice-level data largely describe structural characteristics of the practice and lack information on how practices organise prescribing. Even more so, there are virtually no data available for GP characteristics beyond GP sex and the number of encounters that GPs have with eligible patients (which is a marker for part-time and/or temporary GPs). In particular, we cannot distinguish locum or temporary GPs from GPs in training or from more permanent salaried or partner/principal GPs. There are also no data available on GPs’ knowledge of risk or their attitudes to it, including their more general risk tolerance. This reflects both that routine data do not record such characteristics and the concerns of the data provider to ensure confidentiality, but is a major limitation when it comes to trying to explain why GPs vary. Finally, whereas we can be reasonably confident that a particular prescription was issued to a particular patient, because clinicians take care to ensure this, the attribution of that prescription to a particular GP relies on GPs not sharing user accounts. In general, this will not happen because GPs usually need to log in simultaneously with multiple colleagues working the same day, but it is likely that some locums log in either using a single locum user account or using the log-in of the GP for whom they are covering. Practices are likely to vary in how they organise this, but, without knowing more about the GPs being measured in this study, it is not possible to explore this further. In general, this would most likely cause a bias if locums had a different rate of high-risk prescribing than other GPs (although there was no association with the number of encounters each GP had which might have been expected to occur if this was the case).

It is also important to recognise that, while we believe that new acute NSAID prescribing is a key initiation event for all subsequent NSAID prescribing, most people exposed to high-risk NSAID prescribing at any one time have more chronic prescribing. As explained in Chapter 2, it is not possible with current data systems to attribute repeat prescriptions to individual prescribers. Our belief is that GPs will also vary in the degree to which they review the repeat prescriptions they sign (which in total will amount to hundreds of prescriptions per day in most practices), although whether or not GPs with lower rates of new acute high-risk NSAID prescribing also detect and avoid more chronic high-risk NSAID prescribing is unknown.

Interpretation and comparison with existing literature

Just over 1% of encounters with patients particularly vulnerable to a NSAID-related ADE led to a NSAID prescription, more commonly when there was a recorded diagnosis consistent with a musculoskeletal symptom or pain, in those prescribed more repeat drugs, in younger compared with older patients, and in those with fewer risk factors for NSAID ADEs. The first of these two characteristics is markers of need for prescribing, either directly (consulting with a particular diagnosis) or indirectly (number of drugs is a proxy for multimorbidity)119,135 and the second of the two is markers of risk of ADEs. The only non-encounter characteristic associated with high-risk NSAID prescribing was GP sex, with men more likely to prescribe than women, which is at least partially consistent with known gender-related differences in practice. 136

This analysis found that there was considerable variation between GPs in issuing new acute high-risk NSAID prescriptions, and that this variation was considerably higher than variation between practices. High-risk new acute NSAID prescribing is therefore more a characteristic of individual GPs than of practices. This is consistent with the literature that actions which are under the direct control of individual physicians vary more at individual than at institution or area level. 79,96,99,100,103,131 In contrast, it might be expected that repeat prescribing might be more associated with practice-level rather than GP-level variation, as it is organised as a single system for the practice albeit still subject to GP discretion whether to sign a prescription or not, although the data do not allow this to be examined. Reliability at practice level was poor, reflecting the considerable variation between GPs within practices and at GP level was only moderate with the majority of GPs only being able to be measured with reliability of more than ≈ 70%, which is suitable for formative but not high-stakes evaluation. Although the model fitted identified three practices and 51 GPs as being statistically significantly more likely to prescribed new acute high-risk NSAIDs, it would not be possible to identify high-risk prescribing GPs by examining high-risk prescribing practices because most GPs in high-risk prescribing practices are not themselves high-risk prescribers, and most GPs with high levels of high-risk prescribing work in average or lower than average practices.

It is worth reiterating that the analysis done could not be easily extended to most other high-risk prescribing because of problems of attribution (knowing whether the GP issuing the prescription is the primary decision-maker or is acting on the recommendation of a specialist) and of identification (having a GP ID clearly attached to the prescription). Variation in high-risk prescribing between GPs is therefore important in its magnitude, difficult to measure in general and difficult to measure reliably enough for high-stakes evaluation even if it can be measured.

Data sources

Data were available for 190 Scottish practices providing care to almost one million patients from the start of Q2 2004 to the end of Q1 2009.

Outcome measures

Statistical methods

For each indicator and for each composite, the prevalence of high-risk prescribing in Q1 2009 was calculated, and variation between practices in that quarter examined by calculating the mean, minimum, maximum, median and interquartile range of prevalence at practice level. The ICC at practice level was estimated using a two-level model of patients nested within practices using xtmelogit in Stata version 11, with normality of level 2 (practice-level) residuals checked graphically.

The prevalence of high-risk prescribing over time was then plotted for each indicator and each composite. For indicators where gastroprotection was included in the numerator definition, then prevalence of high-risk prescribing was plotted both accounting for and ignoring gastroprotection to check whether or not any observed change in the indicator was a result of less NSAID or antiplatelet prescribing or more gastroprotection prescribing. For each indicator and for each composite, we then used linear regression to estimate the slope of the trend over time. As the data is a time-series, we initially examined it for the presence of autocorrelation using relevant graphical plots of the autocorrelation and partial autocorrelation functions, and if necessary fitting lag terms for first order autocorrelation (which was present in some models). To estimate trends, we initially fitted linear models then explored whether or not the inclusion higher order quadratic and cubic terms improved model fit. Model fit was assessed using AIC, and the best-fitting model used to estimate change over time. In this context, autocorrelation is a nuisance term requiring adjustment and only main effects are reported.

Finally, changes in the variation between practices over time was examined using box plots of prevalence at practice level across the whole time series, and estimating ICCs for Q1 in each year using the same two-level model structure described in Chapter 3, Descriptive analysis and multilevel modelling. After checking for normality of the level 2 (practice-level) residuals, differences in variance at practice level was examined using an F-test comparing the variance in Q1 2009 with the earliest available variance in Q1 in a previous year (Q1 2005, except for the CKD indicators 11 and 12 and the renal composite, when Q1 2007 was used).

Prevalence of high-risk prescribing and variation between practices in quarter 1 2009

The prevalence of high-risk prescribing in those particularly vulnerable to ADEs varied from 0.17% (indicator 7: combined oral contraceptive pill in women with previous thromboembolism) to 20.2% (indicator 1: drugs to avoid in heart failure), with an overall prevalence for the composite of all indicators except CKD of 5.2% (95% CI 5.1% to 5.3%) (Tables 13 and 14). The range of high-risk prescribing across practices was typically large, although, as is common in a practice-based analysis, extreme values were usually found in small practices. Of note is that the minimum for every indicator is zero, which is invariably a single very small practice with few patients eligible for any measure. For the four indicators (7, 8, 10, 19) with the lowest prevalence, less than 20% of practices had any recorded high-risk prescribing (6.3% for indicator 7, 7.4% for indicator 8, 17.9% for indicator 9 and 16.8% for indicator 10). However, it was not the case that this generally zero prescribing occurred in the same practices, with 38.4% of practices having at least one patient triggering one of the four indicators, and no statistically significant correlation between the practice prevalences. For all four of these indicators, it was either not possible to fit a multilevel model to estimate the ICC or fitted models were unstable and had very non-normal distribution of level 2 (practice-level) residuals, violating a key model assumption (an example plot is shown in Appendix 6, Figure 17).

Intraclass correlation coefficients were estimated for the remaining 15 individual measures and the composite measures (see Tables 13 and 14). For the two indicators with the highest prevalence, the estimated ICC was statistically significantly different from zero but trivial in absolute terms, indicating minimal variation beyond chance between practices (indicator 1: drugs to avoid in heart failure; and indicator 12: drugs to avoid in CKD 4 and 5, both with prevalence > 20%). For three indicators, the ICC was large, indicating that a large proportion of the variation in outcome was attributable to variation between practices, being 0.320 (95% CI 0.239 to 0.415) for indicator 6 (methotrexate in mixed doses), 0.202 (95% CI 0.149 to 0.268) for indicator 3 (LABA use without ICS use) and 0.167 (95% CI 0.086 to 0.300) for indicator 13 (NSAID use without gastroprotection in people prescribed an oral anticoagulant). For all other individual indicators, the ICCs were in the range 0.023–0.087, which is similar to other primary care studies of between-practice variation and usually interpreted as indicating small to moderate variation between practices. ICCs for the composite indicators were in a similar range, varying from 0.025 to 0.058.

Change in high-risk prescribing and variation between practices over time

Figures 12 and 13 show time trends in high-risk prescribing for the individual indicators and the composites respectively. Tables 15 and 16 summarise analysis of change over time in prevalence and in variation between practices, with detailed analysis provided in Appendix 5. Overall, there was evidence that high-risk prescribing reduced between 2004 and 2009, with 8.5% of those patients particularly vulnerable to adverse events (measured by composite 6, which included all indicators except for the CKD indicators) receiving a high-risk drug in Q2 2004, compared with 5.2% in Q1 2009 (difference 3.3%, 95% CI 3.1% to 3.4%). However, this change was not linear, with an initial more rapid reduction and then subsequent partial flattening off. No reduction in variation between practices was observed for composite 6 (all indicators except for the CKD indicators). All of the other five composites showed reductions in high-risk prescribing, with composite 1 (NSAIDs), composite 2 (NSAIDs and antiplatelets) and composite 4 (drug–drug interactions) having the same pattern of initial more rapid reduction with subsequent partial flattening off, whereas the remaining two composites showed a linear reducing trend.

FIGURE 12.

Time series of changes in high-risk prescribing 2004–9. (a) Indicator 1: drugs to avoid in heart failure; (b) indicator 2: beta blockers in asthma; (c) indicator 3: LABA use without ICS use; (d) indicator 4: NSAID in peptic ulcer; (e) indicator 5: NSAID in over-75-year-olds; (f) indicator 6: methotrexate in mixed strengths; (g) indicator 7: COCP in previous thromboembolism; (h) indicator 8: oestrogens in previous breast cancer; (i) indicator 9: antipsychotics in dementia; (j) indicator 10: Parkinson’s disease; (k) indicator 11: NSAID in CKD 3–5; (l) indicator 12: drugs to avoid in CKD 4–5; (m) indicator 13: NSAID with OAC; (n) indicator 14: NSAID with aspirin or clopidogrel; (o) indicator 15: NSAID with ACE inhibitors/ARB and diuretic; (p) indicator 16: aspirin/clopidogrel with OAC; (q) indicator 17: aspirin and clopidogrel; (r) indicator 18: high-risk drugs with coumarin OAC; and (s) indicator 19: PPDE-5 inhibitor with oral nitrate. COCP, combined oral contraceptive pill; OAC, oral anticoagulant.

FIGURE 13.

Time series of changes in high-risk prescribing 2004–9 for composite indicators. (a) Composite 1: NSAIDs; (b) composite 2: NSAIDs and antiplatelets; (c) composite 3: drug–disease interactions; (d) composite 4: drug–drug interactions; (e) composite 5: renal adverse effects; and (f) composite 6: all indicators except CKD.

In total, 11 of the 19 indicators showed statistically significant decreases in prevalence (the seven NSAID indicators; the two antiplatelet indicators; indicator 1: drugs to avoid in heart failure; and indicator 3: LABA use without ICSs), seven showed no change and one showed a statistically significant increase (indicator 18: high-risk drugs with coumarin oral anticoagulant) increasing from 2.9% to 4.6%. All of the NSAID indicators had an initial more rapid reduction with subsequent partial flattening off, apart from indicator 11 (NSAIDs in CKD 3–5), although this indicator covers only the later part of the time series from Q1 2007 so is not directly comparable. For NSAID and antiplatelet indicators related to GI bleeding risk, reductions in high-risk prescribing could, in principle, be because of reduced rates of NSAID prescribing or increased use of gastroprotective drugs, but the same pattern of change was seen irrespective of whether or not gastroprotection was accounted for (see relevant panels of Figure 12), consistent with reductions being primarily driven by reduced NSAID prescribing.

Two indicators showed reductions between Q2 and Q3 2004, which did not fit the otherwise observed trend (indicator 3: LABA use without ICSs in asthma; and indicator 9: antipsychotic use in dementia). For antipsychotic use in people with dementia, we know from an analysis using this data set over a longer period of time that this is the tail end of a very large change in this prescribing associated with a regulatory warning in early 2004. 56 The large change in LABA use without ICSs is potentially the result of regulatory warnings and new guidance about avoiding LABA monotherapy issued in 2003. 139 However, the prior trend is unobservable in this data set, and, to our knowledge, there are no published studies that extend over a longer period, and so the first data point was excluded from regression models fitting a trend line.

In terms of change in variation between practices, of the 15 indicators for which an ICC could be estimated, variation between practices increased for five indicators, remained the same for five, and reduced for five. There was no clear relationship between change in prevalence and change in variation between practices (Table 17), although it was notable that 5 of the 11 indicators with a falling prevalence had an increase in variation between practices.

As we observed that changes in the overall composite appeared to be largely driven by reductions in high-risk NSAID and antiplatelet prescribing, we defined post-hoc a composite of all indicators which were not NSAID or antiplatelet related and examined change in prevalence and variation between practices. This composite was not prespecified but was defined in the same way as the other composite measures and consisted of indicators 1 (drugs to avoid in heart failure excluding NSAIDs), 2 (beta-blockers and asthma), 3 (LABA but no ICS in asthma), 6 (methotrexate mixed strengths), 7 (combined oral contraceptive pill and previous thromboembolism), 8 (oestrogens in breast cancer), 9 (antipsychotics and dementia), 10 (metoclopramide/prochlorperazine and Parkinson’s disease), 18 (high-risk drugs with coumarin oral anticoagulant) and 19 (PPDE-5 antagonists and oral nitrates). Consistent with overall changes in high-risk prescribing being NSAID and, to a lesser extent, antiplatelet related, there was no change in the prevalence of this composite indicator (Q3 2004 prevalence 3.3% vs. 3.3% in Q1 2009) or in variation between practices (ICC 0.023 in Q1 2005 vs. 0.021 in Q1 2009; F-test for difference in variances, p = 0.219).

Summary of findings

The prevalence of high-risk prescribing in Q1 2009 measured by composite 6 (all indicators except CKD) was 5.2% (95% CI 5.1% to 5.3%) and by composite 5 (renal adverse effects) was 9.0% (95% CI 8.7% to 9.2%). There was clinically important variation between practices in the absolute prevalence of high-risk prescribing, with the composite 6 range being 0–10.1% and the interquartile range being 4.1–6.3%. ICCs for the composite measures ranged from 0.025 (95% CI 0.020 to 0.033) for composite 6 to 0.058 (95% CI 0.046 to 0.073) for composite 1 (NSAID indicators), consistent with variation between practices being a small to at best moderate proportion of total variation.

Four of the indicators had low absolute numbers of patients receiving a high-risk prescription, and low overall prevalence, with no prescribing at all in > 80% of practices in Q1 2009 (and earlier). These were indicator 7 (combined oral contraceptive prescribed to a woman with a history of arterial or venous thromboembolism), indicator 8 (oral or transdermal oestrogen prescribed to a woman with a history of breast cancer), indicator 10 (metoclopramide or prochlorperazine prescribed to a person with Parkinson’s disease) and indicator 19 (PPDE-5 inhibitor prescribed to a person prescribed oral nitrates or nicorandil). For these indicators, it was not possible to calculate an ICC, but this prescribing seemed to be a result of rare and sporadic error rather than systematic differences, as most practices had no such prescribing, few practices had more than one patient with such prescribing and practices with any patient triggering one of these measures did not usually have any triggering other measures. Although still important to review from an individual patient perspective,38,39,140 the prescribing measured by these indicators will therefore not be a common cause of harm, and variation between practices in them is unlikely to be important.

Excluding these four indicators, the prevalence of high-risk prescribing in Q1 2009 varied from 1.0% (indicator 13: NSAID prescribed to person prescribed an oral anticoagulant) to 20.2% (indicator 1: drugs to avoid in heart failure). There was trivial variation between practices for two indicators (indicators 1 and 12), large variation between practices for three (ICC 0.167 to 0.320 for indicators 3, 6 and 13), and ICCs in the range seen in most other studies for the remainder (range in this study 0.023 to 0.087).

Overall, high-risk prescribing declined between 2004 and 2009, with 11 of the 19 indicators showing clinically significant reductions, eight of which were indicators involving NSAIDs and/or antiplatelet drugs, which were almost entirely responsible for the observed reduction in overall high-risk prescribing. Only indicator 18 (high-risk drugs with coumarin oral anticoagulants) increased in prevalence. The percentage of patients receiving any high-risk prescription in composite 6 (all indicators except for the CKD indicators) fell from 8.5% in Q2 2004 to 5.2% in Q1 2009, reflecting 7213 fewer patients receiving any high-risk prescription (21,848 out of 257,656 vs. 14,635 out of 280,461). Over the same period, variation between practices increased for five indicators, did not change for five and decreased for five (and could not be estimated for indicators 7, 8, 10 and 19; see Table 15). There was no clear relationship between change in prevalence and change in variation between practices.

Strengths and limitations

The analysis uses routinely collected data from GP electronic medical records for approximately 20% of the Scottish population. Such data have a number of strengths and a number of matching weaknesses. Important strengths are the size and scope of the data set and that GP records are generally near complete for prescribing and have reasonable accuracy for diagnostic coding for the common conditions included here. The partial exception to this is CKD, for which there were major changes to practice, with the routine reporting of eGFR being implemented across the UK in late 2005 and CKD becoming a domain the QOF from April 2006. This led to major changes in CKD coding during 2006, and analysis of the two CKD-related indicators (indicators 11 and 12) therefore only starts in Q1 2007.

Key limitations are that some prescriptions are hand-written and therefore will not be recorded electronically (particularly prescriptions written during home visits, which are most commonly given to frailer patients who are most at risk of ADEs), and some patients may obtain NSAIDs or aspirin over the counter from a pharmacy, although prescriptions are free to all over-65-year-olds and some younger people with chronic disease, making this less likely for many of the measures which predominantly target older people. As discussed in Chapter 3, composite indicators can be problematic to interpret if they combine individual indicators with different levels of risk or associated harm, or with different prevalence meaning that the composite is effectively dominated by a small number of the underlying individual indicators. In this case, the composites are less similar than in previous chapters, where they related to a single drug class (NSAIDs), and the underlying prevalence varies more widely. For this reason, interpretation of the composites should be cautious and made with reference to changes in the underlying individual indicators.

Interpretation and comparison with existing literature

There have been a number of studies of change in high-risk prescribing over time, although these typically examine the impact of specific interventions targeting one particular kind of prescribing, most commonly the impact of regulatory or other risk communications. 55–59,141,142 Such studies find typically small and somewhat variable impact of risk communication interventions with impact associated with clear recommendations for action, repeated warnings and larger reductions in new prescribing than prevalent prescribing (i.e. in people being prescribed the drug before the warning). 55,56 For the indicators examined here, there were Medicines and Healthcare products Regulatory Agency (MHRA) risk communications in relation to antipsychotics in older people with dementia in Q1 2004143 in relation to NSAID GI toxicity (focusing on the risks in older people and when coprescribed with aspirin) in 2002 and 2003,144,145 and COX-2 NSAIDs and cardiovascular risk in 2004146 and 2005147 (but which also reminded prescribers to use all NSAIDs at the lowest possible dose) and updated advice about cardiovascular risk of all NSAIDs in 2006, including that non-selective NSAIDs might carry cardiovascular risk. 148 In addition, the National Patient Safety Agency issued a risk communication and best practice advice in relation to methotrexate, including avoiding prescribing both 2.5 mg and 10 mg strengths in 2006. 149

We have previously shown that a very large change in prescribing of antipsychotics to older people with dementia happened in Q1 2004 (i.e. immediately before the start of observation in this study), and it is likely that the small out-of-trend fall in prescribing between Q2 and Q3 2004 is the tail end of this change. 56 Although the fitted trend in mixed-dose methotrexate prescribing across the whole period was linear, there is a visual indication of a possible change in trend in 2006 consistent with a small change in practice at the time of the National Patient Safety Agency risk communication, although formally testing this would require fitting a segmented regression or other appropriate model. 150,151 Using joinpoint regression, others have shown that NSAID prescribing (and particularly COX-2 NSAID prescribing) reduced after the MHRA warnings in 2004 and 2005, although high-risk NSAID prescribing in this study was falling from the start of observation period, which coincided with the first of these risk communications. It seems likely that the observed decline in high-risk NSAID prescribing from 2004 is at least partly because of MHRA warnings about GI risk in 2002144 and 2003145 and partly because of a spillover effect from risk communications about cardiovascular risk between 2004 and 2006. 146–148 Figure 14 shows the number of oral NSAID prescriptions dispensed by NHS Scotland from 2001 to 2013, distinguishing traditional NSAIDs from COX-2 NSAIDs. 152 This shows that the number of NSAID scripts dispensed rose steadily between 2001 and 2004 because of an increase in COX-2 prescribing which was only partly in substitution for traditional NSAIDs. Dispensing fell slightly in 2005, with a large fall in 2006 driven by large reductions in COX-2 dispensing with only partial substitution of traditional NSAIDs. There was then a smaller decline in total NSAID dispensing until 2008, at which point dispensing slowly increased.

FIGURE 14.

Number of NSAID prescriptions dispensed in Scotland 2001–13. 152

Few previous studies have examined change in variation between practices. In two studies in Sweden and the Netherlands examining health-care centre adherence to guideline recommendations to prescribe simvastatin for cardiovascular prevention, increasing mean adherence over time was associated with large decreases in between-practice variation. 96,153 However, similar to this study, an analysis of change in four indicators over 7 years in variation between physicians and variation between facilities found that quality improved and variation decreased for two indicators, quality improved and variation increased for one, and there was no change for the fourth. 131 This rather contradicts an assumption in some of the literature that variation between physicians or institutions is likely to be large when quality is low and consistently reduce as quality improves. 132,154 Although this must be generally true in the sense that as mean performance approaches 100% then variation must reduce, quality improvement is likely to have differential effects on different physicians or institutions, which very plausibly might increase variation when first implemented. Victora et al. 155 propose such a situation with their ‘inverse equity hypothesis’, which is that improved preventative care initially benefits the more affluent and so will initially widen health inequalities even if longer-term inequalities eventually narrow. The implication is that, although large variation between physicians or institutions is likely to indicate that the quality or safety of care being measured is improvable, decisions to try to improve should not be made solely on the basis of the absence or presence of variation. 131,132 However, quality or safety improvers should pay attention to variation and, if necessary, intervene to minimise it if improvement is widening inequalities.

Summary for objective 1 (original protocol): Chapter 2

Objective 1 was to define and operationalise prescribing safety indicators that can be applied at individual prescriber level and practice level. We concluded that it was only narrowly feasible to create prescribing safety indicators that could be applied at individual prescriber level because of the difficulty of attributing prescribing decisions to individuals. There were two main reasons for this. First, it is not possible in current data to identify who actually makes the decision to initiate many drugs, with GPs not infrequently prescribing on the recommendation or instruction of a hospital specialist (although the GP has no obligation to follow such a recommendation and retains legal responsibility for the prescription). Second, GP electronic data commonly do not have a clinician ID attached to them. After exploration of the data, we considered that new acute NSAID prescribing (the issue of an acute NSAID prescription when there had been no NSAID prescribing in the previous year) could be attributed to an individual in that almost all such prescriptions will be initiated by GPs, have a clinician ID attached and are likely to be issued and signed by the clinician identified because they are largely created in clinical encounters rather than through administrative processes. This method could be applied to some other high-risk prescribing but is neither generally applicable nor straightforward to implement.

The analysis additionally identified that the most high-risk NSAID prescriptions were issued as repeats, for which clinical oversight is often limited. New acute NSAID prescribing is an important initiating event as it is often associated with subsequent prescribing, with the shift from repeated acute prescribing to repeat being another likely key transition point that will influence practice high-risk prescribing rates.

Summary for objectives 2 and 3 (original protocol): Chapters 3 and 4

Objective 2 was to examine the prevalence of individual indicators and appropriate composites, associations with patient, prescriber and practice variables, and the relative importance of variation at prescriber level and practice level before and after adjustment for patient-level variables. Objective 3 was to measure the reliability of individual and composite indicators at prescriber level and practice level. Two multilevel modelling analyses are reported.

Summary for objective 4 (additional in revised protocol): Chapter 5

Objective 4 was to examine changes in rates of high-risk prescribing over time (2004–9) and variation between practices, using a basket of indicators with consensus validation. Nineteen indicators derived from UK-based consensus studies identifying valid indicators of high-risk prescribing were chosen as likely to be feasible to implement in routine GP data. 38,39 Changes in the prevalence of, and variation between practices in, each of these indicators were estimated. For the overall composite composed of the 17 indicators that could be consistently measured over the entire period, the percentage of patients receiving any high-risk prescription fell from 8.5% in Q2 2004 to 5.2% in Q1 2009, which was largely driven by reductions in high-risk NSAID and to a lesser extent antiplatelet prescribing. Only one type of high-risk prescribing statistically significantly increased in prevalence – high-risk coprescribing to people prescribed coumarin anticoagulants (which relates to antibiotic and antifungal coprescription). Change in variation between practices could be assessed for only 15 indicators, and it increased for five indicators, did not change for five and decreased for five, with no clear relationship between change in prevalence and change in variation between practices.

Measurement at general practitioner level is not routinely feasible for either governance or improvement activity

Although variation between GPs was estimated to be approximately three times greater than variation between practices, there is no feasible way of routinely measuring high-risk prescribing at GP level with current data because of attribution problems. These problems reflect both the difficulty of attributing the decision to initiate to the GP who issues the first prescription and the difficulty of identifying which GP actually issued a particular prescription using existing electronic data. The latter is potentially remediable, although is likely to require wholesale electronic prescribing where clinicians are identified by a unique ID used in all clinical data such as their General Medical Council registration number. The parallel is with the shift in patients having IDs that vary across settings (multiple hospital numbers, GP IDs that are only unique within one practice and so on) to having a single NHS number used across all settings. However, the problem of attributing the decision to initiate a high-risk prescription to a particular individual is more problematic, reflecting that similar to much of health care, prescribing is often a shared endeavour with several clinicians involved in decision-making (although legal responsibility primarily lies with the individual who signs the prescription). High-risk prescribing indicators at GP level are therefore unlikely to be usable for governance or wide-scale improvement purposes in the near future.

Measurement at practice level for governance purposes

It is feasible to measure total high-risk prescribing at practice level using the NSAID indicators examined in this study (and a wider basket of indicators in our previous work5), and these indicators distinguish between practices with high-reliability making them suitable for high-stakes evaluation as part of governance activity. In principle, therefore, prescribing safety indicators could be used to identify practices with high rates of high-risk prescribing to investigate for clinical governance purposes. However, it remains unclear why practices vary in their prescribing and such an approach would not intervene on the majority of GPs with high-risk prescribing (as most work in practices which would not be targeted) nor the majority of patients exposed to high-risk prescribing (as for all indicators for which high-risk prescribing is common, it happens in all or most practices).

Measurement at practice level for improvement purposes

An improvement approach could focus on higher-risk practices but, as indicated above, the majority of people with high-risk prescribing would not benefit. The alternative is to deploy improvement activity in all practices, on the basis that high-risk prescribing will sometimes or often be appropriate but needs regular review because appropriateness will change over time. These approaches could still use practice-level measurement as part of an intervention, for example for feedback of practice performance, which could include comparison against other practices or a benchmark. There are examples of such interventions being evaluated in trials, with a small number of feedback trials156–161 being based on prescribing safety outcomes,162 one large completed UK-based trial of a pharmacist-led intervention which did reduce targeted high-risk prescribing137 and two other UK trials (which have completed but not reported as of autumn 2014) using feedback163 or feedback combined with educational outreach and financial incentives. 164

The implications of this study is that research and service interventions targeting NSAID prescribing should carefully consider whether or not and how to combine review of existing prescribing (as repeat prescribing is the bulk of prevalent high-risk NSAID use), and understanding and intervention to influence the initiation of new prescribing and transitions from intended short-term treatment to long-term use of high-risk drugs. In terms of new prescribing initiation, variation between GPs was found to be large in this study. Although measurement of this variation using routine coded data is not feasible, as part of an improvement intervention practices could be encouraged to internally examine who initiates high-risk prescribing (both specialist and GP initiation) and the appropriateness of transitions from short-term to longer term treatment. In addition, improvement programmes should consider not only whether or not overall quality and safety is increasing, but also whether or not improvement happens across the board or is more piecemeal and therefore associated with increasing inequalities and consider whether or not more intensive intervention is required in some settings to reduce variation. 155

Causes of variation between practices

There is relatively little known about why practices vary in their high-risk prescribing, with only a limited range of practice structural characteristics examined in this and previous analyses of variation. It is likely that practice variation is driven by wider contextual factors (including variation between specialists and areas) and how practices organise their repeat prescribing systems. Although there are some examples of such work, there is a need for research to better understand how the organisation of high-volume, safety-critical systems influences safety and patient outcomes and how this organisation reflects wider practice culture. 108,109,111,165 In particular, research is needed to better understand how and why high-risk prescribing is initiated, how prescribing systems facilitate or hinder clinical oversight and how and when repeat high-risk prescribing is reviewed and managed.

Causes of variation between general practitioners

Although variation between individual clinicians is usually found to be larger than variation between the institutions or areas they work in, research is needed to better understand why this variation occurs. 79,96,102,153,166 There is some evidence from other contexts, including test ordering, that individual risk preference and risk tolerance are associated with differences in practice, but the extent to which this matters compared with differences in knowledge, for example, is uncertain. However, although the extent of variation seen is large, it is unclear whether or not and how to intervene to reduce variation between GPs without understanding what causes it.

Quantifying the risk of commonly used drugs

The type of variation occurring is what others have described as variation in preference-centred care where the evidence of benefit or harm is ambiguous or absent where physicians’ preferences play a large part in decision-making, at least partly because of a failure to allow patients themselves to make informed choices. 78 The risks and harm of prescribed drugs are often poorly quantified, with, for example, the risk of coprescribing with coumarin anticoagulants such as warfarin being remarkably uncertain for most drugs despite the length of time coumarins have been used and it being clear that drug–drug interactions are common and important causes of harm for the increasing number of people taking coumarins long term. 167 Although the indicators used in this study had all been validated in consensus studies based on existing evidence, there is a need to more systematically quantify the risk of commonly used drugs using robust pharmacoepidemiological methods.

Understanding high-risk versus inappropriate prescribing

The prescribing being measured is high risk but is not clearly inappropriate, and it is possible that variation in patient preference for risk plays a part in the observed variation between GPs and between practices. From this perspective, high-risk prescribing could easily be judged appropriate if patients make informed choices about the risks and benefits involved and choose to take a drug even if their risk of adverse events is higher than average (although, as noted below, the actual risks of some of this prescribing are remarkably poorly quantified; therefore, informed choices will not always be that easy to make). There is evidence from other contexts that prescribing decisions are often made without much active patient involvement and that doctors fairly commonly do not elicit or actively misunderstand patient perspectives. 168,169 Research to better understand patient perspectives on the risks and benefits of high-risk prescribing is needed, and how patient perspectives could be better elicited in consultations and incorporated into decision-making.

Interventions to manage and reduce high-risk prescribing

There is evidence from studies in the UK and elsewhere that high-risk prescribing in primary care can be improved, but it remains unclear what the most effective approach is. The completed PINCER study demonstrated the effectiveness of a 12-week pharmacist-led intervention combining single-round feedback of rates, educational outreach and facilitation of review by the pharmacist, but it is notable that there was evidence that reductions in high-risk prescribing at 6 months were only partially sustained at 12 months. 6,137 It is, therefore, unclear how to ensure longer-term prescribing safety, or how pharmacist-led interventions compare in effectiveness to simpler interventions using regular feedback alone163 or to other complex interventions prompting and facilitating GP review of high-risk prescribing164 and interventions that target patients with polypharmacy rather than focusing on particular indicators. 170,171 Computerised point-of-care reminders are also under-researched in the UK and may be an efficient way of improving prescribing safety, although ensuring that they are tailored to individual patient characteristics and avoid overloading prescribers with unnecessary alerts is challenging.

Research is also needed to examine the extent to which interventions which improve prescribing safety actually improve patient outcomes. Finally, it seems likely that the reductions in high-risk NSAID prescribing observed in this study were associated with a series of risk communications relating to NSAIDs in 2004–6. Of note is that these risk communications were about cardiovascular safety and largely related to COX-2 NSAIDs in the period examined, highlighting that although regulatory risk communications do sometimes impact on targeted prescribing, their effect is variable and often extends beyond targeted prescribing. 55,172 Given the importance of regulatory risk communications in disseminating knowledge of newly discovered drug risks and in recommending appropriate action, research to more systematically evaluate the impact of regulatory risk communications and improve their design to maximise effectiveness is needed.