The use of locum doctors in the NHS: understanding and improving the quality and safety of care

Data

We accessed several data sources to extract individual-level information on full-time equivalent (FTE) working hours (1 FTE = 37.5 hours/week), type of GP (i.e. locum GP, GP partner, salaried GP, GP registrar, junior GP and GP retainer), type of locum GP (i.e. long-term of infrequent locum) and GP characteristics (age, gender, country of qualification). General practice-level information was extracted on population age and gender, quality of care, morbidity burden, patient satisfaction, rurality, deprivation, single-handed practices and healthcare regulators rating for each general practice in England. Definitions and sources for all data are provided in Appendix 1.

Practitioner-level information from NHS Digital was extracted from practices at the last day of each reporting quarter, with 12 quarters available between 31 December 2017 and 30 September 2020. We restricted the time period window, as there were differences in the methodology used by practices to report locum data prior to December 2017. For the period of analysis, some practices did not provide valid or complete records, and this resulted in some data being recorded as missing or estimated. Even though we excluded these records from the analyses, coverage was very high with approximately 95% of all practices providing valid workforce data in December 2019. FTE for locum GPs was derived as an average of the total number of hours worked in each month over the reporting quarter. 37

From NHS Digital,38 we obtained information on achievement indicators for all long-term conditions in the UK’s Quality and Outcomes Framework (QOF), which we used to calculate morbidity burden and performance for each general practice in our data set. The QOF is a national pay-for-performance scheme in primary care that was introduced in 2004 with the aim to improve quality of care and linked financial awards to performance on achievement indicators. Lower-layer super output area (LSOA)-level deprivation, as measured by the Index of Multiple Deprivation (IMD) 2019,39 was available for all LSOAs (geographically defined neighbourhoods of 1500 people on average) and we assigned LSOA deprivation scores to practices based on the practice’s postcode. The IMD is a relative measure of deprivation for all 32,844 LSOAs in England where each LSOA is assigned a score on a continuous scale (i.e. 0–100) and a higher score corresponds to greater deprivation. We extracted data on patient satisfaction for all practices from the GP Patient Survey, data on rural/urban classification based on practices’ location40 and practice overall inspection ratings from the Care Quality Commission (CQC). All data were matched on practice unique identifiers [i.e. organisation data service (ODS)] and year. Data were publicly available and did not require ethical board review.

Statistical analysis

We plotted total locum FTE against total FTE for all types of GPs over time for the whole of England. We used violin plots to compare the age and FTE distribution of locums and other GP types, and by gender. We plotted cumulative FTE by gender and country of qualification for all GP types. We calculated the average rate of general practice locum use in 2019 defined as total locum FTE as a proportion of the total GP FTE and visualised geographical variation in locum use at the CCG level with the use of spatial maps.

We used mean-dispersion negative binomial models with robust standard errors to model locum FTE, with fixed-effect predictors for region (as categorical, to account for between-region variations) and time (as continuous, to account for time trends). We used the natural logarithm of the annual total GP workforce FTE count as offset. We control for several practice characteristics in all models: deprivation, practice CQC ratings, the proportion of practice’s female population, the proportion of practice’s patients aged over 65 years, single-handed practices, rurality, QOF performance, QOF morbidity burden, patient satisfaction and practice workload defined as list size over total GP FTE. We used one set of negative binomial regression models to investigate the relationship between locum FTE and practice and population characteristics over time (2018–9) and one set of models to investigate the relationship cross-sectionally (2019).

Stata v16.1 was used for the principal data cleaning, management and analyses. For the two primary sets of analyses, we used the nbreg command with the exposure option and the incidence rate ratio (IRR) specification. Practices with < 1000 patients were omitted from the regression analyses because these practices are opening, closing or serving specific populations.

Variation in locum use

Over time, reported mean locum use in England varied from 3.15% (1045.8 FTE) in December 2017 to 3.31% (1157 FTE) of total GP FTE in September 2020 (Figure 1). The proportion of practices that reported at least some locum use varied from 37.4% and 40.8% in 2018 and 2019, respectively. Most locums (74%) worked in long-term positions compared with infrequent locums (26%) (Figure 2). Long-term locum use remained stable over the study period, though there was a substantial 47% reduction in infrequent locum use between the first and the second quarter of 2020, indicating the impact of the COVID-19 pandemic.

FIGURE 1.

Variation in FTE by GP type over time, December 2017–September 2020.

FIGURE 2.

Variation in locum type over time, December 2017–September 2020.

Violin plots depicting the FTE distribution of locums and other types of GPs are presented in Figure 3. Median locum FTE in December 2019 was 0.09 FTE (0.7 sessions in a practice where 1 FTE = 8 sessions) and we observed a similar distribution in FTE for both male and female locums in contrast to other GP types (e.g. GP partners/salaried GPs) where we observed large variation in the distribution of FTE between genders.

FIGURE 3.

Full-time equivalent distribution of GPs in December 2019, by type and gender.

Locum characteristics

Locum workforce characteristics in terms of age are presented in Figure 4. In December 2019, the age distribution of locums shared similar characteristics with the age distribution of both male and female salaried GPs. Median age of female locums was 41 years [interquartile range (IQR) 35–48] and median age of male locums was 44 years (IQR 37–55). The median age for GP partners, junior doctors, GP retainers and salaried GPs was 49, 31, 42 and 39 years, respectively.

FIGURE 4.

Age distribution of GPs in December 2019, by type and gender.

Locum workforce characteristics in terms of gender are presented in Figure 5. Male locums accounted for 54.5% of total locum FTE. This was similar to GP partners who were mostly males (60%) but in contrast to registrars/junior doctors (40.6%), GP retainers (18.5%) and salaried GPs (30.5%) who were mostly females.

FIGURE 5.

Gender breakdown by GP type, December 2019.

Locum workforce characteristics in terms of country of qualification are presented in Figure 6. Most locums obtained their degree in the UK (63.8%), although this proportion was smaller compared with other types of GPs (82% for salaried GPs and 76% for partner GPs).

FIGURE 6.

Country of qualification breakdown by GP type, December 2019.

Geographic variation of locum use

We present variability in mean locum use at the CCG level across regions in 2019 in Appendix 1, Figure 28, with dark-shaded areas indicating higher locum use and light-shaded areas indicating lower locum use. Locum use varied substantially between regions (from 0.4% to 13.7%) with locum use accounting for 2.5% of total GP FTE in the North East and 7.4% in London. Descriptive statistics on locum FTE, population size estimates, number of practices, census information, deprivation and QOF population achievement for all English regions in 2019–20 are reported in Table 2. In Table 24, Appendix 1 we provide a table with 10 CCGs with the highest use of locums and 10 CCGs with the lowest use of locums in 2019.

Results from regression analyses

After adjusting for practice and population characteristics, large variability in locum FTE between regions persisted (Table 3). Using Midlands as the reference category, practices in London had the highest locum FTE [IRR = 1.369, 95% confidence interval (CI) 1.180 to 1.588], and practices in the North East and Yorkshire had the lowest locum FTE (IRR = 0.711, 95% CI 0.626 to 0.843).

Care Quality Commission ratings appeared to be a strong predictor of locum FTE, where practices rated as having inadequate (IRR = 2.108, 95% CI 1.370 to 3.246) and good services (IRR = 1.343, 95% CI 1.103 to 1.637) had higher locum FTE than practices that were rated as having outstanding services. Single-handed practices had substantially higher locum FTE (IRR = 4.611; 95% CI 4.101 to 5.184) compared with group practices. For practices in rural locations, locum FTE was 25% higher than for practices located in urban areas (IRR = 1.250; 95% CI 1.095 to 1.428). Practices with a higher proportion of female population had 3.3% lower locum FTE (IRR = 0.967; 95% CI 0.959 to 0.981) than practices that had a higher proportion of male population. A larger patient population in the over-65 age group was associated with 3% lower locum FTE (IRR = 0.970, 95% CI 0.950 to 0.984). Finally, patient satisfaction was very weakly associated with locum FTE, while deprivation, QOF quality of care, QOF morbidity burden and practice workload did not appear to have any discerning effect on practice locum FTE. Partial results from the overtime and cross-section regression models are reported in Table 3 and the full results are reported in Appendix 1, Table 25.

Summary

This work describes a methodological approach to capture and monitor the scale and scope of the GP locum workforce in English primary care. Our findings suggest that between December 2017 and September 2020, the proportion of GP locum work in the NHS has remained relatively stable, despite widespread perceptions that the number of locum GPs has risen. 3 Regarding regional variation and the characteristics of locums, we describe the intensity of locum use in general practice and how this varies across regions, as well as important information about the composition of the GP locum workforce. We identified substantial geographical variation in locum use between and within regions suggesting differences in the distribution of locums in England. Our comparisons of locums with other GP types showed that locums were more mobile, younger males of whom most had qualified in the UK, although a large percentage had qualified elsewhere. Most locums were employed in long-term positions and on average they did relatively few sessions. Our regression analyses showed that practice characteristics such as rurality, CQC ratings and whether the practice was single handed were stronger predictors of higher locum FTE than population characteristics.

Strengths and limitations

This analysis was conducted at the population level and allowed us to quantify and examine the scale and characteristics of the GP locum workforce compared with other types of GPs for the first time across general practices in England. We explored whether variation in practice and population characteristics explain variability in locum FTE to account for different health needs across different practice populations. The study has national scope and comprehensive coverage of the primary care population (95% of all general practices).

We used publicly available routinely collected data from NHS Digital. However, other databases on workforce report different estimates on the number of locum GPs. The GMC register and the National Association of Sessional GPs estimate approximately 17,000–18,000 GPs with a locum licence in England in 2017,3,41 while the NHS Digital data report only 5040 employed locum GPs in December 2017. There may be several reasons why these differences exist, one being that the NHS Digital data show a picture of the actual GP workforce at each time point rather than the prospective workforce that other databases report. Locum headcounts may overestimate locum use as some locum GPs may also be simultaneously permanently employed.

Second, the GP workforce data collected by NHS Digital have been subjected to changes in data sources and methodology over the years and also include estimates for practices where data are incomplete or have not been submitted. We restricted our time period to exclude data prior December 2017 when the infrequent locum category was first reported in the collection and excluded estimates for those practices that did not submit valid data. Third, the locum data are believed to be under-reported when compared with other types of GPs, mainly due to the infrequent locum category for which reporting may be lower than long-term locum data. In September 2020, NHS Digital switched from quarterly to monthly data collections of the GP workforce data; however, the transition to monthly collection led to a decrease in the number of FTE for infrequent locums. For this reason, the data collections were reverted to quarterly to allow practice managers to report infrequent locum data in time. 37

Interpretation of findings

Previous international evidence shows that the numbers of locums continue to rise,3,42,43 but our findings suggest that this may not be the case for GPs in England. Previous reports from the GMC and the National Association of Sessional GPs showed that the proportion of GPs with a locum GP contract had increased from 30% to 39% of all licensed GPs from 2013 to 2016 and was equivalent to approximately 18,000 GPs in 2018. 3

One recent study examined the geographical variation in the distribution of the GP workforce, including GP locums, across the 13 Health Education England regions using data from NHS Digital but did not make specific comparisons between locums and other types of GPs. 44 Our comparisons can provide a review of the locum workforce at a more granular level which is also particularly relevant to NHS organisations (e.g. CCGs). To our knowledge, no other studies to date have examined contextual factors and their association with locum use in general practices.

The accurate monitoring of the GP workforce may help policy-makers and commissioners to understand current challenges in primary care, including capacity and composition of the GP workforce and inform workforce planning. This can be particularly useful to meet local healthcare needs with sufficient resources for training and deployment of GPs which will help ensure that the targets set out in the NHS long-term plan are met. 45 For example, this research highlights elevated locum GP employment in practices in rural areas and those with inadequate CQC inspection ratings. These types of practices may face substantial challenges in recruiting and retaining permanent GPs and we could hypothesise that relatively high and sustained levels of locum use may be an indicator of wider problems which are affecting recruitment and retention.

Furthermore, this work lays the foundation for future analysis of other existing routine primary care data sets that contain information on service utilisation and patient outcomes. Additional work is needed to identify whether there exist differences in the clinical practice and performance between locum doctors and permanent doctors and also what consequences these may have for patient safety and quality of care. Future work should also aim to identify career intentions of locum GPs and what factors influence the choice to work as a locum. Furthermore, it will be important to understand what implications these career intentions and the observed locum workforce characteristics have on future workforce planning. As more data become available, the impact of COVID-19 on the use of the GP locum workforce should be examined.

Locum GPs have an important role in the delivery of primary care services, particularly in the delivery of out-of-hours care and in helping to address short-term workforce shortages. Despite expectations that locum GP numbers are rising, we found that locum use in primary care has remained stable over time though the use of locums seems to vary substantially across different practice types and areas of the country.

Data

Outcome measures

Analysis

Overall locum use

In 2019, total unadjusted locum use for all trusts in England was 2,004,485 shifts, of which 909,029 (45.3%) were bank shifts and 1,095,455 (54.7%) were agency shifts. Trusts requested 2,316,302 shifts with a trust mean of 208 per week [standard deviation (SD) = 258.3]. The completeness of the data was good with 99% of all trusts reporting at least some locum use in any week.

Locum intensity

Figure 7 plots the ranked mean locum intensity in 2019 for 219 NHS trusts in England showing significant variation in locum use across trusts. Mean locum intensity was 0.22 (SD = 0.16) (median = 0.195; 25th–75th centile = 0.11–0.31), indicating 0.22 locum shifts per permanent doctor FTE. Assuming five shifts per FTE, the locum intensity is equivalent to 4.4% of medical staffing provided by locums (25th–75th centile = 2.2–6%). Four ambulance trusts, three acute trusts and one community trust were not included in this analysis as they reported very low or zero permanent doctor FTE and therefore adjustments in their locum intensity could not be performed. The ranked rates of locum intensity in 2020 and 2021 are presented in Appendix 2, Figures 29 and 32. We report the 10 trusts with the highest and lowest reported locum intensity in 2019 in Appendix 2, Table 26.

FIGURE 7.

Locum intensity in 2019, NHS trust level.

Proportion of agency shifts

The proportion of locum shifts filled by locum agency staff (rather than from staff banks) ranked from low to high at the trust level is depicted in Figure 8. The use of agency shifts varied substantially across trusts in 2019 with a mean of 66.1% (SD = 28.5%; median = 68.9%; 25th–75th centile = 43.5–95.8%). Half of trusts51 reported 100% of shifts filled by agency staff at some point in 2019, of which 24 trusts reported that shifts were filled entirely by agency staff in every week. Eight ambulance, one acute, one mental health and one community trust reported zero agency shifts in every reporting week in 2019. We present the ranked proportion of agency shifts for 2020 and 2021 in Appendix 2, Figures 30 and 33.

FIGURE 8.

Proportion of agency shifts in 2019, NHS trust level.

Proportion of unfilled shifts

In Figure 9, trusts are ranked from low to high on their proportion of unfilled shifts. Overall, trusts were able to fill the majority of their requested shifts either via bank or agency staff, but we observed substantial variation. The mean proportion of unfilled shifts was 11.3% (SD = 11.9%; median = 7.23%; 25th–75th centile = 0–18.1%). Seven ambulance and one acute trust did not request any shifts at any point in 2019. The ranked proportions of unfilled shifts for 2020 and 2021 are presented in Appendix 2, Figures 32 and 34.

FIGURE 9.

Proportion unfilled shifts in 2019, NHS trust level.

Regional variation in locum use

In Table 4, we present descriptive statistics on outcomes and trust characteristics at the regional level for 2019. Figure 9 shows regional variation in outcomes at the trust level in 2019. At the regional level, median locum intensity varied substantially from 0.13 in the South West of England to 0.26 in the Midlands (Table 4 and Figure 10). We also observed large variation in the proportion of agency shifts across regions. Trusts in London filled the lowest proportion of shifts using agency staff with a median of 44.8%, while trusts in the East of England filled the highest with a median of 78.1% (Table 4 and Figure 11). Trusts in the East of England filled requested shifts more successfully with unfilled shifts of 3.25%, whereas trusts in London had unfilled shifts of 11.6% (Table 4 and Figure 12). Regional variation for the three outcomes in 2020 (Figures 35–37) and 2021 (Figures 38–40) is presented in Appendix 2.

FIGURE 10.

Regional variation in locum intensity, 2019. Figure includes data from 222 NHS trusts in 2019, adjusted for permanent doctor FTE.

FIGURE 11.

Regional variation in the proportion of agency shifts, 2019. Figure includes data from 229 NHS trusts in 2019.

FIGURE 12.

Regional variation in the proportion of unfilled shifts, 2019. Figure includes data from 229 NHS trusts in 2019.

We investigated spatial variation within and between regions using spatial maps at the STP level (see Appendix 2, Figures 41–43). Substantial variability was observed for all three outcomes both within and between regions. High levels of locum intensity were concentrated in the Midlands, the North East and Yorkshire and the North West. The South East and South West ranked among the lowest in terms of locum intensity. High proportions of agency shifts were observed in areas in the Midlands, the East of England and the North East and Yorkshire. London had by far the lowest proportion of agency shifts. The proportion of unfilled shifts was high in areas in London, the Midlands and the South West and low in the East of England.

Results from regression analyses

The regression analyses results using the three different outcomes are presented in Table 5. The results are reported as IRRs for the coefficients of interest. Sensitivity analyses, where we excluded ambulance and community trusts and examined the effects of deprivation on our three outcomes were nearly identical to the results from the main analyses. Deprivation did not appear to have any discernible effect on any of the three outcomes. The results from the sensitivity analyses are provided in Appendix 2, Table 27.

Locum use during the COVID-19 pandemic

Figure 13 shows the mean agency, bank, unfilled and total shifts per week at the trust level in 2019–21. Over time, the trust-level mean was 188.5 shifts per week (SD = 205.8), of which 95.2 (SD = 108.6) were agency shifts and 93.3 (SD = 135.8) were bank staff shifts and the mean of unfilled shifts across all trusts was 38.5 (SD = 85.2). Pre pandemic, we observed a small variability in the mean number of agency, bank and unfilled shifts. In March 2020, there was a steep decline (approximately 18%) in agency and bank shifts per trust as very few trusts reported locum use between March and April. In the third quarter of 2020, we observed an increase (approximately 15%) in agency and bank shifts per trust. In 2021, there was a steep steady increase in the mean number of unfilled shifts from 33.9 to 50.1 (47.8% increase) between May and June, which was sustained throughout 2021 and reached a peak of 69.2 unfilled shifts per trust in December 2021.

FIGURE 13.

Variation in mean number of locum shifts over time, 2019–21.

Summary

This study provides evidence on the extent of locum use and factors associated with locum use in NHS trusts in England for the period 2019–21. Our findings show that, on average, 4.4% of medical staffing in NHS trusts in 2019 was provided by locum medical staffing. Trusts with lower CQC ratings, acute trusts and smaller trusts had higher locum intensity. We observed moderate variability in locum use across regions and greater variability in the proportion of shifts filled by agency locums. During 2021, the proportion of shifts that were unfilled reached a 3-year high. Our findings can help inform NHS organisations about the extent of their locum use and can provide important information about the effective planning of the NHS workforce.

Strengths and limitations

The main strength of this study is the national scope and coverage of every NHS trust of England. For the first time, using routinely collected data on locum use, we quantified the extent of locum use, sourced from agencies or banks, across all NHS trusts for the period 2019–21. We also explored whether trusts were able to cover sufficiently for staff shortages and identified drivers of locum use at the trust level for the whole of England. We reveal the impact of COVID-19 on locum use in NHS trust. Our analyses allowed us to control for measured trust and population characteristics.

However, this study has some important limitations which should be considered when interpreting the key findings. First, the NHS Improvement data do not reveal information on locum use by specialty and there may be substantial variations across specialties which we could not identify. Second, although NHS Improvement collects data on the number of locum shifts, it does not collect the shift duration or locum FTE which would allow a more straightforward comparison with permanent doctor FTE. We had to assume that shift lengths for permanent and locum doctors were broadly equivalent in order to estimate the proportion of medical staffing provided by locum doctors. Third, there may also exist variability in locum use between locums of different types (e.g. infrequent or long-term locums) or durations apart from the agency/bank categories, which has been observed in general practice. 36 Some locums may be employed for several months52 often to cover a vacancy which has not been filled, while others may cover short-term absences such as illness for as little as one or two shifts and we did not have that information. Fourth, the data set has no information on how well NHS trusts use their locum workforce such as the provision of adequate induction, training, supervision and feedback in accordance with National Health Service England (NHSE) guidance. Prior work15 suggests that locum performance is driven more by organisational attributes such as these than by the characteristics of the locum doctors themselves.

Interpretation of findings

The use of locum doctors is important because of the high level of spending it entails and because of concerns about the quality and safety of locum staffing arrangement. 9 Our study shows that the actual level of locum use, as a proportion of overall medical staffing, is relatively low on average, but varies considerably, with some trusts having much higher use of locums and some trusts relying overly on more expensive agency locums rather than using staff banks.

Some of this variation may be explained by organisational characteristics. For instance, larger trusts may be more able to cover workforce gaps within their own staff without needing locums, and specialist/tertiary trusts may find it easier to recruit and provide attractive workplaces compared with general acute trusts. Mental health trusts may face particular staffing shortages, which may explain the high level of agency locum use.

Our results show significantly higher locum intensity in trusts with worse CQC ratings (inadequate or requires improvement). It may be that these trusts find it harder to recruit and fill workforce gaps, but it could also be hypothesised that sustained high levels of locum use may impact quality and safety and hence affect CQC ratings.

The introduction of the first UK lockdown brought significant reductions in the numbers of both bank and agency locum doctors employed across NHS trusts, due to cancellations in elective care. However, shortly after, trusts started employing more locums likely in an effort to tackle excessive workload and increasing demand for healthcare services during the pandemic. Furthermore, in 2021, we observed an increase in the mean number of shifts filled by bank compared with the previous years and this was accompanied with a stable trend in agency shifts and an increase in the number of unfilled shifts. This suggests that trusts were meeting the increased demand with bank staff, which is in line with the new agency rules enacted by NHS Improvement in 2019 that aim to reduce reliance on agency staff. 53 Despite the increase in the mean number of total shifts, trusts appeared to be less able to fill the number of shifts they were requesting over the second half of 2021. This may suggest a persisting high workload for permanent doctors that trusts were unable to address with the use of locum doctors over that period.

Questionnaire design

The survey was developed with input from stakeholders including a medical director, a research director, a senior leader in medical staffing, the chair of our PPI forum, a GP locum and a managing director of a locum agency. Drafts of the survey were sent out via e-mail and stakeholders responded with comments, and where possible in-depth discussions were conducted with stakeholders to gain further feedback. The research team discussed the comments received and made appropriate changes.

The study was approved by the Health Research Authority – National Research Ethics Service England, and the initial page of the survey stated that by completing the survey participants were agreeing to take part in the study.

An 89-question custom-built online open survey56 was generated using Qualtrics software. 57 We collected information about why locums were needed, how locums were recruited, supported, perceived and managed, how the work of locums compared with permanent doctors, experiences of locum agencies, familiarity with the NHSE guidance for supporting locums and how concerns about locums were dealt with. We also sought the views of NHS trusts about the advantages and disadvantages of locum work and how they see locum doctor work changing in the future. A copy of the survey is provided in Appendix 3.

For the purposes of this survey, we defined a locum doctor as a doctor in a temporary or fixed-term placement, engaged through a locum agency, internal locum bank or directly contracted by a healthcare organisation.

Survey distribution

This was a survey of 191 NHS trusts in England. Prior to distribution, we e-mailed ROs to make them aware of the research and to encourage engagement. A RO is accountable for the local clinical governance processes in their particular healthcare organisation, focusing on the conduct and performance of doctors. RO duties include evaluating a doctor’s fitness to practise, and liaising with the GMC over relevant procedures. 58 The survey was initially sent to trust ROs in England and periodic reminders were sent to non-responders. Due to a low initial response rate, we contacted non-responding trusts by telephone to identify appropriate contacts at each trust. The survey was then sent to each of the contacts provided, which included R&D departments, medical staffing departments and medical directors. The electronic link to the survey was active for 7 months between June and December 2021 to allow trusts the time to respond during the pandemic.

Survey analysis

We analysed numeric and Likert scale data from survey respondents using frequency tables. Comparisons between NHS trusts who responded to the survey and all other NHS trusts in England were performed using t-tests. Since most survey responses were not normally distributed, non-parametric tests were used. Differences were investigated using Mann–Whitney U-tests.

Three free-text questions were analysed using thematic analysis;59,60 these questions asked about the advantages and disadvantages of locum agencies, the advantages and disadvantages of locums and the future of locum work. Employing an inductive approach – coding and theme development – involved identifying patterns of shared meaning across responses and was driven by the content of the comments rather than a priori themes. The written responses mostly consisted of short sentences which provided additional contextual detail to the quantitative questions. The written responses were read and reread to become familiar with the content, and notes were made of any potential codes for each question by identifying recurring words or units of meaning. 61 Responses to the three free-text questions were combined and mapped into overarching themes which encompassed the main issues highlighted in the data (a list of these themes and illustrative quotes are shown in Table 9). One further free-text question, which asked for opinions about the NHS England and Improvement guidance about supporting locum doctors, was not included in the thematic analysis as it was specific to the guidance. Illustrative comments are included to provide contextual detail to the quantitative question asking about trusts familiarity with the guidance.

Respondent characteristics

We surveyed a total of 191 NHS trusts and we received 98 usable responses (response rate 51%); of these, 89 completed the whole survey and 9 answered half or more of the questions. The responses included 66 (67%) acute hospitals, 26 (27%) mental health and 6 (6%) community health providers. The survey was completed by 35 (36%) Medical Directors and/or ROs (including Deputies and Associates), 54 (55%) medical staffing (e.g. Temporary Staffing Manager, Head of Medical Workforce), 3 (3%) clinical staff and 4 (4%) other roles (e.g. Medical Human Resources Business Partner). One respondent did not complete the question about their job role. Compared with all other trusts in England, there was no significant difference in CQC ratings, reported extent of locum usage, permanent doctor FTE or deprivation, suggesting the responses were broadly representative of NHS trusts generally.

The need for locums

How the need for locum doctors is met

The most frequent method for engaging locums was locum agencies. Nearly all respondents used locum agencies that were ‘framework suppliers’ (on a national procurement list selected by Crown Commercial Services) and three-quarters of trusts felt that locum agencies matched their needs and provided accurate information about locums. Trusts also frequently used internal locum banks and doctors who have previously worked for the organisation. The use of Doctors Direct (NHS Professionals) and digital platforms such as Locum Nest were much less frequent. Digital platforms were used significantly more by acute trusts compared with community providers (p = 0.023) and mental health providers (p < 0.001). Internal locum banks were used significantly more by acute trusts compared with community providers (p < 0.001) and mental health providers (p < 0.001).

National Health Service England and improvement guidance about supporting locums

Familiarity with the NHS England and Improvement guidance about supporting locum doctors varied across trusts. Over half of respondents were either very or somewhat familiar but less than half of trusts were either slightly or not at all familiar.

In free-text comments, some respondents were positive about the guidance and reported that they followed the guidance in their organisation.

At [name of trust] we apply the principles of the guidance in providing our agency locum doctors with a service induction when they start from the clinical service where they are working. Prior to them starting we provide them with a welcome providing information of where the post is, access to parking along with who their contacts are whilst in post.

Trust 31

Other respondents emphasised that they follow the guidance where possible or that focus is given to certain aspects of the guidance.

Working short term has specific challenges. We are committed to have systems in place to support the professional development and governance of the Locum doctors. We have a dedicated Medical Lead to support and supervise the practice of the Locum Drs. Particular attention is given in pre-employment checks, occupational health and induction.

Trust 77

Respondents reported that the way medical staffing was organised and the cost of using locums can be barriers to implementing the guidance within their organisations.

The organisation does not have a temporary staffing team it also outsources the medical bank management however this is under review. Therefore best practice is not always adhered to at [Trust name].

Trust 87

Others felt that the guidance was unrealistic, contradicted other guidance requirements and did not recognise differences between trusts. It was also felt that these issues were exacerbated by current staffing pressures.

It’s perfect world stuff, the reality is it’s the Wild West and we are desperate to get people.

Trust 27

Difficult to do when we have to provide medical care and the substantive medical workforce are demanding locum cover … they do not read the guidance VS the GMC/CQC guidance.

Trust 53

I think it’s too generalised and doesn’t understand that each organisation works differently and not 1 size fits all.

Trust 25

Respondents commented that the guidance would benefit from being updated.

The guidance was put together in October 2018 so it is dated -query around some of the content around appraisals etc.

Trust 68

Application of the guidance

We asked trusts how frequently they followed different aspects of the guidance when locums were working in their organisation (Table 7). Key procedures such as verifying documentation and induction were conducted more frequently compared with, for example, providing feedback. Most trusts said they would report concerns about locums but support with annual appraisals (where appropriate) and revalidation was less frequent.

Trusts experiences of locum doctors

We asked trusts how care provided by locums compares with care provided by permanent doctors, in a number of areas (Table 8). Overall, care provided by locums was largely viewed as about the same as care provided by permanent doctors. Some trusts reported that care was worse, in particular in relation to continuity of care, but also adherence to organisational policies and guidelines, administrative errors and reporting of adverse advents. Mental health trusts were significantly more likely than acute trusts to report that workload for the permanent healthcare team was worse when care was provided by locums rather than permanent doctors.

Dealing with concerns about locums

We asked trusts what happens when there is a low-, medium- or a high-level concern about a locum doctor. We defined low-level concerns as: causing no harm to patients or staff and the doctor was not at any personal risk, medium-level concerns have potential for serious harm to patients, staff or the doctor was at personal risk and high-level concerns are when patients, staff or the doctor had been harmed. The action that trusts reported they would take increased depending on the severity of the concern (Table 10). The higher the severity of the concern, the more likely that the locums and locum agencies would be informed. It was common for contracts to be ended early when there were concerns and locums to not be used again.

Survey design

The study received research ethics approval from the National Research Ethics Service in England. It was explained on the first page of the survey that agreement to take part was implicit through completion.

Stakeholders who were included in the development of the survey included: a medical director, a research director, a medical staffing senior manager, chair of our Patient and Public Involvement and Engagement (PPIE) group, a locum GP and a locum agency managing director. E-mail drafts of the survey were distributed to stakeholders, to which they replied with feedback and further review and conversations with stakeholders about their input were performed as necessary. Stakeholder feedback was reviewed by the project team and the survey was adapted accordingly.

An online bespoke questionnaire56 consisting of 89 questions was produced using Qualtrics software. 57 Information was gathered about why locums were required, how they were recruited, supported, perceived and managed, how locums work compared with the work of permanent doctors, perspectives of locum agencies, familiarity with the NHS England and Improvement guidance for supporting locums and how practices resolved issues concerning locums. Perspectives were sought regarding the advantages and disadvantages of locums and any future change in locum work (see Appendix 4 for a copy of the survey).

We defined a locum doctor as a doctor in a temporary or fixed-term placement, engaged through a locum agency, GP chambers or locum bank, or directly contracted by a healthcare organisation.

Survey distribution

The survey was distributed via the National Institute for Health and Care Research (NIHR) Local Clinical Research Network (LCRN). LCRNs help increase opportunities for participants to take part in research. This method has been used before and secured a good response rate. 68 The survey link was distributed via e-mail to the 14 LCRNs in England who, in turn, sent it to practices in their region. Practices were selected for inclusion based on different criteria within each LCRN. Some LCRNs sent the survey link only to practices which had opted to take part in research. Other LCRNs sent it to all practices in their area. In most cases, the surveys were e-mailed to practices, but in one region it was distributed via a newsletter and in another only to practices which had expressed interest in taking part. We asked each LCRN how many practices the survey had been distributed to in their area; this totalled 3745 general practices out of 6822 practices in England. 69 Due to the pandemic and to allow practice time to respond, the electronic link to the survey was live from June to December 2021. LCRNs were prompted to send periodic reminders.

Survey analysis

The survey was analysed using frequency tables to describe numeric and Likert scale data. t-tests were performed to compare respondents with all other practices in England. Non-parametric tests were used because the majority of survey responses did not show normal distribution. Associations between responses and practice size and frequency of locum use in practices were investigated using Spearman’s correlation coefficient.

Thematic analysis was used to analyse the written responses to three questions [(1) the advantages and disadvantages of locum agencies; (2) the advantages and disadvantages of locum work; and (3) the future of locum work]. 59,60 Patterns of shared meaning were identified utilising an inductive approach where coding and theme development is based in the information found in the responses. Respondent’s free-text comments were brief and are used to supplement the findings from associated quantitative questions. Familiarisation with the free-text data was achieved through iterative reading and for each question possible codes were noted, taking account of repeated words or meaning. 61 All written comments were reviewed holistically and overarching themes were mapped, encompassing the key messages across the data (see Box 1 for a list of these themes and key sample quotes). Thematic analysis was not used with one question, asking for perspectives about the NHSE guidance about supporting locums; instead, key descriptive comments were included as examples to provide context to the quantitative question enquiring about practice familiarity with the guidance.

Respondent characteristics

We surveyed a total of 3745 practices and we received 605 usable responses (response rate 16.2%). The responses included 23 (3.8%) from single-handed practices, 203 (33.6%) from small practices (2–5 GPs), 238 (39.3%) from medium practices (6–10 GPs) and 141 (23.3%) from large practices (more than 10 GPs). The survey was completed by 205 (33.9%) GP partners, 14 (2.3%) salaried GPs, 9 (1.5%) in other clinical roles (e.g. advanced nurse practitioner), 323 (53.4%) practice managers, 39 (6.5%) non-clinical managers and 14 (2.3%) administrators. Compared with all other practices in England, responding practices were slightly larger both in terms of practice list size and number of GPs employed (for further details comparing responding practices and all other practices in England, see Appendix 4).

The authors acknowledge that the aim and methods for Chapter 6 are similar to those in Chapter 5. These surveys were conducted alongside each other and asked very similar questions of both NHS acute trusts and GP practices.

The need for locums

How the need for locum doctors is met

We asked practices about their methods for engaging locums. The most frequent method was the use of doctors who had previously worked for the practice, followed by word of mouth/personal recommendations. The use of Locum Chambers and digital platforms like Locum Nest was much less frequent. Locum Chambers is a model of working where locums are part of a local community of GP locums, who pool together resources to create a holistic support package for their work, supporting each other and the practices where they all work.

The use of locum agencies varied across practices with just under 40% using them often or sometimes but over a third never using them. Practices did not feel that locum agencies consistently matched their needs or provided accurate information. Practices with more GPs employed were significantly less likely to have their needs met by locum agencies r(338) = 0.18, p < 0.001 and report that they were provided with accurate information from locum agencies about locums r(338) = 0.22, p < 0.001. Practices with higher frequency of locum use reported that they were significantly more likely to have their needs met by locum agencies r(367) = 0.19, p < 0.001.

National Health Service England and Improvement guidance about supporting locums

Most practices were not familiar with the NHS England and Improvement guidance about supporting locum doctors. Some felt that it was not applicable to them, not a priority or they did not have the time to consider it.

I rarely have time to read guidance like this, with so much paperwork being sent to us about more critical issues

Practice 141

Some practices followed the guidance or aspects of it, and some had their own policies for supporting locums.

I have not read it – we are used to having locums and I have been a PM for 13 years, so I feel I/we know what we are doing and support our locums well – we follow our Locum policy

Practice 67

There were some positive perspectives on the guidance, with some respondents reporting that it was effective in controlling locum rates and encouraging doctors to take permanent posts.

I think the vision to bring down the rates paid for locum doctors is important as this pushes more into salaried positions and helps provide more stability for both the clinician and practice

Practice 75

Whereas others felt that the guidance was unrealistic, impractical or needed updating.

It is very ivory towers and idealistic, but not practical in real terms

Practice 7

Practice experiences of locum doctors

We asked practices how the care provided by locums compared with care provided by permanent doctors in a number of areas (Table 13). Generally, care provided by locums was viewed as about the same as or worse than care provided by permanent doctors. Practices with more GPs employed were significantly more likely to report that adherence to organisational policies and guidelines, providing continuity of care, and reporting of adverse events or untoward incidents was worse when care was provided by locums rather than permanent doctors.

Dealing with concerns about locums

We asked practices to tell us what happened when there were low-, medium- or high-level concerns about a locum doctor in their practice. We defined low-level concerns as: causing no harm to patients or staff and the doctor was not at any personal risk, medium-level concerns have potential for serious harm to patients, staff or the doctor was at personal risk and high-level concerns are when patients, staff or the doctor had been harmed. The action that practices reported they would take to deal with concerns increased depending on the severity of the concern (Table 15). The higher the severity of the concern, the more likely that they said locums and locum agencies would be informed. Practices reported that it was common for contracts to be ended early when there were concerns and locums to not be used again.

Characteristics of participants

Of the 130 participants interviewed, 88 were working in health care and 42 were patients. Participant characteristics are summarised in Table 17. The mean number of years of qualification for doctors who participated was 23.2 (SD = 11.0) and ranged from 1 to 48 years. Of the 59 doctors interviewed, 33 worked as locums, while 25 worked on a permanent basis and one worked as a locum in additional to permanent work. Of the permanent doctors, 10 had previously done locum work, 6 were working in primary care, 18 in secondary care and 1 in both sectors. Of the locum doctors, 19 were working in primary care, 12 in secondary care and 2 in both sectors.

In 2020, 38% of all licensed doctors in the UK were from an ethnic minority background. 82 The number of doctors from an ethnic minority background in our sample is 4% lower than the doctor population as a whole. Of the 59 doctors we interviewed, 20 (33.9%) were from ethnic minority backgrounds, while 34 (57.6%) were from a white background. Five doctors chose not to disclose their ethnicity (8.5%). Twelve doctors who took part were IMGs, and had been working in the UK for an average of 15.8 years (SD = 6.27). For seven of the IMG doctors, English was not their first language.

The ethnic breakdown of our locum sample more closely reflected the doctor population; of the 30 locums who reported their ethnicity, 18 (60%) were from White English or other white backgrounds, while 12 (40%) identified as ethnic minorities. The 24 permanently employed doctors who declared their ethnicity were largely in senior leadership positions (consultants, ROs, GP partners); 16 (66.7%) were White English or other white background, and 8 (33.3%) were from ethnic minority backgrounds. This is perhaps reflective of the fact that the ethnic diversity of the medical profession as a whole is not matched by diversity at senior leadership levels. 83

The ethnic breakdown of our patient population was largely reflective of the wider population in the UK where 82% of the population identify as white and 18% belong to an ethnic minority group. 84 In our sample, 34 (81%) patients identified as white, while 8 (19%) patients identified as belonging to an ethnic minority group.

In 2019, of NHS staff whose ethnicity was known, four out of five (79.2%) were white, and one in five (20.7%) were from all other ethnic groups. 85 The ethnic diversity of our qualitative sample of participants who represented professionals working as or with locums was more balanced; 57 (64.8%) were from a white background, while 31 (35.2%) where from ethnic minority groups. This is probably because of the number of doctors in the sample.

The gender breakdown of our doctor respondents was broadly reflective of the general doctor population,82 with women being slightly over-represented. Thirty-one (52.5%) participants were women, while twenty-eight (47.5%) were men. Women were slightly over-represented in our project as a whole.

We interviewed a broad range of participants from primary and secondary care as well as locum agencies. Healthcare organisation and participant details are listed in Table 18.

Interviews ranged in length from 23 to 171 minutes with the average interview being 59 minutes. Interviews were transcribed verbatim by a professional transcription company and organised into codes and themes using the software package NVivo (QSR International, Warrington, UK). 86

Thematic framework

Our findings are presented under seven broad and inter-related themes that examine how locum work relates to and impacts quality and safety. Themes were identified by their overall ability to answer the research question and how numerous the occurrences were of each theme across the data set. Frequency of the theme did not necessarily determine its importance; rather, the inclusion of a theme was determined by how relevant it was to the research questions. Themes were also guided by our previous narrative review on quality and safety and locum doctors9 as well as the ‘key messages’ participants had identified in their interviews.

The first theme describes how contextual familiarity was regarded as vital for quality and safety. The second theme discusses the impact of variations in familiarity and preparedness to with locums on efficiency and workload. The third theme continues to explore notions of familiarity and knowing and the implications of continuity and discontinuity for patient care. The fourth theme is about the balance between locum and permanent staff and what happens when organisations become overly reliant on locums. The fifth theme highlights the impact of perceptions of locums on how they were treated and viewed. The sixth theme continues to elaborate on the consequences of locum perceptions and how this is related to blame and defensive practice. In our final theme, we describe how locums and organisations engage with governance mechanism and activities. The themes are listed below:

• Familiarity and continuity: knowing who, where and how:

  • Locum understanding of the local population and staff, as well as how to access equipment/policies/pathways and how to carry out duties safely in unfamiliar environments.

• Efficiency, empowerment and displaced work:

  • The impact of locum working on the workloads of permanent staff and how this related to the organisational preparedness to empower locums to do their job safely and effectively.

• Continuity and discontinuity:

  • The positive and negative implications for continuity and discontinuity for patient care and how safety information was received and shared when doctors were transient.

• Balance of staffing and stability:

  • What happens to quality improvement and organisational and staff development when organisations become more reliant on locums.

• Collegiality, involvement and inclusion:

  • The implications for quality and safety when locums are/are not treated like other staff.

• Defensive practice, hedging and avoidance:

  • How temporary working impacts defensive practice and how locums attempt to practice safely and avoid complaints in complex and unfamiliar environments.

• Governance:

  • How responsibility for the oversight of locum doctors was organised and shared across organisations and how systems of governance worked for doctors who did not have a simple employed relationship with a single organisation.

Additional examples of participant quotations to illustrate each theme are provided in Appendix 5.

Familiarity and continuity: knowing who, where and how

Locum doctors faced distinct disadvantages in comparison to permanently employed doctors when they were unfamiliar with both the organisation and patients. Knowing the patients and staff, where to locate equipment and resources and how to escalate concerns and carry out processes was important for delivering care safely. The importance of local knowledge about patients and their communities was described by patients, locums and permanent staff. Familiarity increased with continuity of exposure, providing stability for teams and patients. Recruiting from the same pool of doctors and longer-term locum placements allowed for trusting relationships to be developed and also meant that locums were able to become familiar with the organisation and its staff, local policies, procedures and referral pathways, organisational layout and IT systems, as well as which services they could access.

And the locums we use now are people that we know. So you come on shift and its all regular locums that you’ve met before and you’ve worked with before … So it’s made a huge difference. You know what their skills are, you know what they’re capable of, you know that you trust them.

Interview 11, Clinical Director, Secondary Care

Locum participants described often working in unfamiliar environments, sometimes with minimal induction and varying levels of support. Organisational reluctance to invest in induction left not only patients vulnerable but also locums, who described being ‘set up to fail’. Induction was important given the large degree of variation in policies and procedures between organisations. Not knowing how to navigate IT systems, refer patients, order tests, find equipment and escalate problems, or locate who to go to for support could lead to operational failures or inadequacies and errors in locating information, supplies or equipment needed for patient care.

I was very rapidly discovering was every practice is different to the next practice by up to 500 different guidelines, policies, pathways and procedures … We can’t just be a bad GP in these positions enforced underperformance …

Interview 63, Locum GP, Primary Care

The absence of normal recruitment processes (involving meeting a doctor, carrying out an interview and following up on references) meant that recruiters were reliant on limited information on which to base their hiring decisions; meaning the knowledge and experience of locum doctors were not always formally assessed. Permanent staff with recruitment responsibilities described how the information provided by locum agencies and the format of CVs and references made it difficult to determine competency, scope of practice and suitability for a role. However, staff shortages and a requirement to meet safe staffing ratios meant that organisational leaders had little recourse of action if they were unsure about a doctor’s capability, which caused anxiety and frustration. This suggests that the provision of health care superseded ensuring safety standards and necessitated accepting one of two objectionable alternatives – accepting gaps in staffing that may jeopardise patient safety or accepting unknown staff that may jeopardise patient safety.

If a locum turns up and I have serious doubts about their ability to do the job to the required standard, I don’t have any recourse … And therefore I’m in a position where either I accept this locum or I don’t. There’s not much in the way of middle ground. Not accepting them is a really unpalatable choice because if I say look, I’m sorry, I don’t think you’re up to this, I think you should go home, that leaves me with a gap.

Interview 30, Consultant and Responsible Officer, Secondary Care

Other consequences of not knowing an organisation and its patients were risks to doctors’ professional reputation and registration and locum participants felt mistakes and complaints were more likely in unfamiliar environments. Doctors working as locums wanted to provide high-quality care but found some organisations were not adequately prepared to safely engage, include or support locums. Some locum participants described avoiding working in unfamiliar, chaotic or unsupportive organisations and restricted the number of organisations they worked in order to build relationships and familiarity and practice more safely.

The biggest sort of safety aspect that sticks in my mind, is that it is unbelievably frustrating to have to learn a whole new set of patients from day to day. And again, like I say, at the start, when I was signed up to four different hospitals, plus the locum agencies, I very quickly realised that not only is it the fact that you don’t know the patients from day to day, if you’re chopping and changing site the whole time, then store cupboard are laid out differently, ways of contacting relevant staff members are different, you’ve got to recognise what code to put in to bleep someone that’s different at every single site … So that’s why, pretty quickly in my F3 year, I decided, no, no, this is the hospital I’m going to work at, and I’m just going to stick with this hospital the whole of the way through, essentially. And that’s helped a lot with regards to that idea of consistency and knowing how to do what you need to do, essentially.

Interview 23, Locum, Secondary Care

International medical graduates faced additional difficulties adjusting to unfamiliar NHS systems and cultures when they first began working in the NHS. Locum doctors who trained outside of the UK described struggling to adapt and the added challenges of being naïve to the system; these locums described needing time and support to adjust. However, there were variations in terms of how organisations integrated and supported locum doctors who were IMGs; with some organisations providing additional supervisory support, while others expected locums, regardless of NHS experience, to ‘hit the ground running’. This meant that IMGs were particularly vulnerable to making mistakes. One participant described how they avoided using IMG locums without NHS experience; however, staff shortages meant that sometimes doctors with limited or no NHS experience were contracted.

I think when you get a lot of overseas … We try not to appoint many that haven’t got NHS experience but I remember when I was at the [hospital name] there were some when we were desperate and it’s just the, kind of … We expect them to come in as a locum and hit the ground running, provide a service and go … they were working at … kind of, put themselves down at a certain grade and they’d done that certain grade over in their home country but then they’ve come over here and there’s misinterpretation. They weren’t probably working at that level …

Interview 22, Medical Staffing, Secondary Care

I did actually study medicine abroad, so when I came to do F1 I was totally naïve to the system … I think there’s definitely challenges like with being overseas, you kind of feel a bit like a fish out of water in a way.

Interview 13, Locum, Secondary Care

Participants with recruitment responsibilities and ROs described difficulty in establishing level of qualification and competence when locums had trained outside of the UK. There was some uncertainty about the quality and safety of the practice of IMGs, as it was difficult to establish a clear picture of their clinical competency and capability and whether there were any governance concerns. While there was a reliance on GMC registration and licence to practise for assurance, it was recognised that governance systems were not completely failsafe and that there was some potential for doctors who were not appropriately trained to enter the system or for doctors to work beyond their scope of practice. For locum Ros, this was a challenge in that they were governing these doctors from a distance.

I have to say that our system is not tight, it’s not fool proof, and I find that we do accept doctors, for instance, from abroad, with no experience of the NHS, and actually with very little clinical experience, but to whom the GMC has given a Licence to Practise, and it therefore comes back onto us as the agency to decide whether the doctor is competent to be placed in an NHS position … There is also a very small sub-section of, dare one say, suspicious doctors who come from abroad, on whose governance we don’t have a clear picture other than nothing adverse has been declared, but conversely, we don’t know whether there are positive aspects of governance that should have been declared but are not. There’s a very small number of doctors, and then if they are accepted for registration and their compliance is correct, we fall back onto management of concerns as they may arise. But we are aware, or I am aware, of the very rare, but occasional possibility of either an incompetent doctor, or a dangerous doctor, or a fraudulent doctor that gets into the system. It is rare, it is rare.

Interview 47, Locum Agency RO

Locums managed the risk related to working in unfamiliar environments in a number of ways including avoiding organisations considered chaotic or unsafe or working below their grade. Some locums described working below their capability and experience to avoid having responsibility in unfamiliar organisations where they may not be supported or included in the team. Better networked locums were able to draw on their community to find out which organisations to avoid, while others learnt through experience

Most locums take jobs, locum work below their grade. So a person who’s at a registrar level would take a locum work as an SHO, because they don’t know the trust that well. And when they do that, they already are … have learning at a registrar level. But because they’ve taken a lower level job, and because they’re locum, they’re not included in part of the team at all.

Interview 55, Locum, Secondary Care

Efficiency, empowerment and displaced work

Employing locums meant vital services could be delivered despite staff shortages; however, variations in familiarity, workload and types of work locums did have potential negative impacts on efficiency, quality improvement and working relationships. Depending on contextual factors, such as how prepared an organisation was to empower locums to work safely and efficiently, how familiar the locum was with the team or organisation, length of placement or what terms and conditions the locum or organisation had negotiated, there were differences in terms of the types of work locums did, or were able to do, in comparison to permanent staff. These factors were related to how much additional work was created for permanent staff when locums were employed; for example, organisations with good processes of recruitment and induction, who used locums who were familiar with the organisations systems and processes, produced less work for other staff.

Regardless of their level of qualification and experience and support (or lack of) provided by the organisation, it was recognised that locums were unlikely to be able to perform to the best of their ability in unfamiliar settings and it was likely that they would need to depend on other staff for accessing IT systems, locating information or equipment and understanding policies and differences between hospitals for example. Unfamiliarity with organisational layout, and multiple inconsistent policies and procedures meant that locums felt that their performance could be negatively impacted. Inductions of variable quality were employed to mitigate unfamiliarity, but the responsibility for this fell on busy staff who described this as distracting from their own work, especially when the locum churn was high.

It can become a little bit like Groundhog Day can’t it, how do you keep your resilience up and keep engaged of having the same conversation, the same teaching every time … So you turn up to your shift, it’s busy. Let’s say you’re working on the acute medical unit, you’ve got 20 patients to clerk, there’s an empty shift and you’ve got a different locum. How do I get into the changing room, how do I do this, how do I do that? Oh, I’ve explained this so many times, you know. And do you get some of that, is some of it about resilience and some of it about you use up your usefulness, I don’t know. And again, that’s where it comes down to seeing this person as a person. They are a person, so it doesn’t matter how many times you hold the door open for someone, you continue to hold the door open for them and it just becomes part of what we do. And so it should be the same for these people we’re working with that just see them as the person rather than the locum.

Interview 19, Deputy Medical Director and Responsible Officer, Secondary Care

Locums were not always empowered to do their job efficiently and effectively because of lack of access to or restrictions on computer systems for example. Not providing locums with ready access to systems meant that they were not able to fully function in organisations until access had been arranged, which could take some time. This was particularly problematic for locums working on a short-term basis. Consequently, work that required access to systems, such as prescribing and referring, was sometimes carried out by other members of staff. Locums described having to rely on others as uncomfortable and stressful, particularly when it was perceived that locums were paid significantly more. There was a concern that relying on others could lead to care that was delayed, partially completed or not completed at all. This was also potentially divisive for working relationships between permanent and temporary staff. A workaround for a lack of access to computer systems for ordering tests and prescribing was to use the login credentials of other staff, meaning that prescriptions could not be traced back to individual doctors, which was a governance issue.

I did some A&E shifts in [hospital name]. But it was a logistical nightmare. It’s a different computer system. And by the time … I find it really tricky. As a locum, you’re there for one shift, you’re there for eight hours or 12 hours, however long it happens to be. By the time I found the toilets and the computer log in and managed to order an x-ray, it takes more than eight hours to do all those things. So I personally found it very, very stressful and didn’t like it. I felt like I asked too many questions, felt a bit helpless and hopeless. They probably thought I was similar as well. And so, yeah, never went back again, frankly. So I tend not to work in hospitals I haven’t worked in before.

Interview 53, Locum, Secondary Care

Continuity and discontinuity

Traditional notions of relational continuity were widely regarded as a thing of the past. Patient participants recognised that they were unlikely to be seen by the same doctors at a practice or in an acute setting, regardless of whether the organisation employed locums. Relational and management continuity was seen as difficult in all organisations due to the fragmented ways in which organisations were structured and changes in how doctors worked. Participants spoke about how continuity was generally no longer held by an individual doctor but in a computer system maintained by multiple doctors. Given that relational continuity was no longer the norm, good-quality management continuity was seen as essential. However, there were concerns that there were inconsistencies in terms of recording and sharing information and that the technological infrastructure was not robust enough to ensure continuity of information. Furthermore, locum unfamiliarity with varying local policies and procedures could increase the likelihood of information being incorrectly recorded or shared. A lack of continuity, both relationally and managerially, placed the onus on patient and carers to keep track of their condition and maintain good records. Patients and carers sometimes held the most complete story about what had happened previously; however, this also meant a reliance on memory and health literacy. Having to relay their history to the locum was frustrating for patients who expressed concern about who was ultimately accountable for their care.

Is it the individual and the continuity, or is it, you know, something else? And I think at the back of my mind … well, at the forefront of my mind, I desperately want it to all be about seeing the GP and having continuity, and them knowing me and my family inside out. But I do think we’re at the stage where that’s kind of like a media construct of a GP, it’s like a country practice version of a GP. And it hasn’t existed for like five decades. And we moved to this system where you see whoever you see, whether they’re just another GP in the practice or a locum or whatever. And the theory was that it shouldn’t matter who you see because what matters is the recording and the system and the process. But as I think we’ve all vividly described, we’re all on the same page really that the systems behind what we have now aren’t good enough. And that’s what the issue is rather than anything else.

Patient 2, Focus Group B

Locum participants reported that care could sometimes ‘slip through the net’ or be delayed because of discontinuity and unfamiliarity with systems and processes. There was a perception that patients were unlikely to come into direct harm because of locum working due to safety netting and the rest of the team compensating where there were deficits in knowledge for example. However, indirect consequences due to inefficiencies included, for example, delayed treatment or discharge or referral to the wrong service. A locum working ad hoc shifts described how a lack of continuity impacted handover and caused difficulties communicating information about patients delaying discharge.

So more often than not it’s when you step in for the back end of a week, and you’ll find these plans that the consultant, for example, implemented a plan on Monday, and then left it to a locum, or for example, left it to a junior on the Tuesday to enact, and then a locum on the Wednesday, and then you’ve stepped into their boots. And occasionally its things that do slip through the net … So it’s all delaying, particularly at the minute, delaying patient discharge is the big thing that’s rife with all the pressures and all the rest of it.

Interview 23, Locum, Secondary Care

While lack of familiarity was largely regarded as having negative outcomes, discontinuity was sometimes beneficial for patients as locums provided ‘fresh perspectives’ and could change the trajectory of their care. For patients, there were potential advantages of discontinuity as locums enabled a second or different opinion. Not having previous experience of patients meant that locums did not have pre-conceptions and could come to their problems afresh. The discontinuity locums introduced could be beneficial for patients as new perspectives led to different routes of treatment or management. Discontinuity introduced by locums could disrupt organisational cultures because locums did not have a relationship with the patient and were not perhaps as constrained by the practice culture or as familiar with policies. However, non-compliance, perhaps due to unfamiliarity, with organisational processes or meeting the expectations of the patient could be detrimental for locums in terms of their employment and reputation and potentially detrimental for patients.

I remember coming in to one of my hospitals … and they were giving a child phosphate. And I said to them, but that’s not how we do it anymore, there’s been a change in practice. And they looked at me as though I was completely bonkers … Then I discovered that one of the consultants, another one, not the one I’d spoken to, was aware of it, and together we got on top of it, and I helped them put together a new protocol. But it was finding the right people to support me. Because the first person I went to batted it away … So, it all depends on whether you can find the person that’s open to change.

Interview 56, Locum, Secondary Care

The discontinuity locums introduced could also be beneficial to organisations as locums brought new perspectives and their experience of seeing many types of practice in multiple organisations. However, this systems knowledge and experience was not always capitalised on. Opportunities to share were typically not invited or rejected if offered, meaning organisations did not take up opportunities to learn from the locum. Locums were sometimes reluctant to challenge the status quo because they recognised their status as a newcomer and a temporary member of team, and reflected on experiences when offering alternative solutions had not been welcomed in the past. This was not only with regard to general functioning of the organisation, in terms of processes that might be improved, but also when locums challenged the treatment of individual patients, which had potential consequences for patient safety.

Unfortunately what I’ve found, which again goes back to communication issues and ego problems, is that people don’t want to hear what the locum has to say in terms of like how things might be improved … there’s not that openness. And it’s also that, I think it’s an attitude thing as well, to think that the locum has nothing to really add. You’re only here to see our appointments, it doesn’t really matter what quality of care you offer.

Interview 70, Locum GP, Primary Care

If you see something being done that doesn’t seem quite right, it’s very hard to say something. Obviously, if it was dangerous, you would but you see sometimes, and you think, there’s a better way of doing that, but it’s not my job to tell them, because I’m only there for a bit, and I don’t want to upset the status quo.

Interview 56, Locum, Secondary Care

Balance of staffing and stability

The balance between locum and permanent staff had implications for quality and safety and efficiency of service delivery. Organisations that were heavily reliant on locums were likely to have gaps in medical leadership, meaning that quality improvement was slower or less likely to happen and that team working and trusting relationships between staff were harder to establish. Some participants felt that the balance between locums and permanent doctors was shifting and expressed concern about the long-term impact on the workforce, the perception of the profession and quality and safety. There was a perception that some organisations were heavily reliant on locums in order to maintain service delivery and locums had become ‘almost a workforce’ as they outnumbered permanent staff in some departments and organisations. One participant described how it was now uncommon to have a substantive medical workforce.

Locum work, my view on it is they’re there to fill a gap. They shouldn’t be relied upon to deliver a service Monday to Friday, day in, day out, week in, week out. And unfortunately my trust see it as that, though, that’s my worry that they feel they’re not just plugging a gap, they’re almost as a workforce …

Interview 84, Lead GP, Primary Care

A&E are obviously used to working with locums, they do it regularly, so it tends to be easier to locum there because there are so many other locums. I don’t know if there’s any permanent staff.

Interview 53, Locum, Secondary Care

The balance of permanent staff and locums was regarded as significant for a number of reasons. Locums were employed to deliver immediate services but were typically less likely to be invested in the team and organisational leadership and development and consequentially less involved in long-term planning, clinical governance and quality improvement. Service development, good governance and quality improvement were perceived as more likely when there were permanent staff to lead and facilitate this. Departments that were disproportionately locum dependent were perceived to be in danger of stagnation and lack of progression; for example, the implementation of new guidance tended not to happen or was slower and more difficult to implement.

So, we have some GP surgeries, which are frankly in a state of crisis at the moment … And any sense of stability is long gone out the door. So, when you get those practices … it probably takes two years to turn a practice around, and that’s with a substantive workforce. You’ve zero chance of doing it if your key players are coming and going because no one will lead, or no one will manage.

Interview 85, Medical Director, Primary Care

Well-functioning established teams were regarded as better able to incorporate a small number of locums without being significantly impacted by discontinuity, whereas GP practices run by locums, or small teams in smaller organisations were regarded as being vulnerable to instability and inertia when the majority of services were delivered by different locums. The expectation on permanent staff to continue service improvement without consistent medical leadership and support was regarded as ‘too big an ask’.

I think there’s much more service development and quality improvement with permanent staff. Again, there will clearly be exceptions to this, but locums tend to turn up, they do the shift, they go home. Whereas when you really want to drive improvements in patient care, it’s the permanent staff who do that. If you get a department that is disproportionately locum dependent, then it stagnates, it doesn’t progress … it’s really hard to expect that one person who’s trying to keep the whole service afloat and to a certain extent monitor the locums work and drive change and improvement. It’s just too big an ask.

Interview 30, Responsible Officer, Secondary Care

Locum participants also recognised that having ‘an NHS run by locums’ was detrimental to quality and safety and some avoided organisations that were locum dependent for this reason. A locum GP described their experience of working in an organisation run solely by locums and reported a target-driven culture based on maximising revenue and where follow-up was not carried out. Another locum GP avoided working in practices that were mostly run by locums and described ‘a total lack of continuity’ and a feeling of being vulnerable because of the impact of the lack of clinical leadership on quality and safety. A different locum felt that locums needed to be used ‘judiciously’ because of implications for continuity, quality and safety and efficiency; this participant wondered whether patients attending practices with a high proportion of locums would be more likely to attend A&E because of the lack of follow-up, for example. However, this participant recognised that they would not know what happened to patients because of their transient relationship with the practice.

I worked in a practice where there was one full time GP and everyone else was a locum, so a patient might see six different doctors for one problem over a period of six weeks or something. So if it’s like that and there’s a total lack of continuity …

Interview 59, Locum GP, Primary Care

A medical director articulated the difficulties in balancing temporary and permanent staff in an organisation that was struggling to recruit doctors. This organisation relied on locums in order to maintain service delivery and the medical director described frustration about a lack of supportive mechanisms for organisations that had no other choice but to rely on a temporary workforce in order to provide services. There was a perception that overuse of the locum workforce would be attributed to poor leadership, rather than a consequence of recruitment and retention problems.

What support or protection do organisations get when our staffing levels drop down that low to say, actually, we can’t deliver this service safely, or we’re trying to do it to the best of our ability … if I have locums coming in, suppose tonight, suppose the whole of my A&E department is staffed with locum doctors and something goes wrong, they’re going to say to me, what on earth were you doing, staffing the whole of your A&E department with locums. Well, where would you like me to start? You know, I don’t do it out of badness, you do it out of how do I keep the service running to the best of my ability.

Interview 32, Medical Director and Responsible Officer, Secondary Care

Collegiality, involvement and inclusion

Without exception, locum participants described negative attitudes and perceptions which had a bearing on how collegial colleagues were, and which in turn impacted involvement and inclusion in organisations in a variety of ways. Perceived disparities between pay, workload and organisational and team commitment between locums and permanent staff could be sources of resentment and influenced how locums were treated and viewed. The perception from some permanent staff that they were doing the same or more work for ‘a lot less’ pay threatened collegiality and could be damaging to working relationships and locum well-being. Exclusion from organisational and team processes not only had the potential to increase risk to patient safety, when information was not shared with locums, for example, but could also be damaging to the safety and well-being of locums themselves. Resentment played out in locums being scapegoated when things went wrong and excluded from organisational functions that were necessary for team function, learning and the delivery of safe care. Locums described a lack of collegiality which resulted in lack of support and during the pandemic some felt that they were exposed to greater risk than permanent staff. While there were some examples of inclusive and supportive practice, collegiality towards locums was sometimes motivated by a high need for locums and problems recruiting, meaning that some permanent staff felt a need to mask their resentment in order to fill gaps and not deter locums.

I guess like any temporary post really, you struggle to invest in them, don’t necessarily see them as being part of the team. Not very positive about them, particularly junior staff, particularly in the acute trusts. We’d have locums refusing to come back because of the treatment of the midwives … I’m not a midwife, but if you’re trying to birth a baby together, you’re reliant aren’t you, you’ve got each other doing a physical procedure, your hands in, it’s high risk. The mum might die, the baby might die, you really rely on each other, it’s very, very intensive.

Interview 86, Clinical Lead, Secondary Care

Locum work was judged differently because of perceptions around higher rates of pay and greater expectations were placed on locums, for example, a perception that locums could replace ‘like for like’ and ‘hit the ground running’ when they entered an organisation. A consequence of this was that locums were sometimes expected to work with minimal support and supervision, despite potential training and support needs, or deficits in organisational knowledge.

Not having supervision, not having anyone who you can go to about problems or just about having a rubbish time. I think that is a real problem.

Interview 36, Locum, Secondary Care

I think it’s the fact that you are a trainee, and there is someone who has to look after you, that kind of gives a bit of protection in terms of your learning, which you don’t get as a locum, obviously, ‘cause you don’t have a supervisor … In some ways you are expected to work very independently as a locum, I think. You can ask for help obviously, but I do think that there is kind of an expectation that you kind of just get things done.

Interview 13, Locum, Secondary Care

While different rates of work were tolerated from permanent staff due to an understood variability in complexity of patients and flow, there was less acceptance when it came to locums who were expected to see a greater number of patients faster. While some participants described their organisations as being welcoming and supportive, explicit and open hostility was described by one medical director who had witnessed staff being less than compassionate towards a locum doctor.

It is tricky, it is hard. I think I’d hate to be a hospital doctor thinking, gosh, I’m stuck here forever and these guys are just coming and going, living the dream, doing what they want to do whenever they want to do it. And that does build resentment and sort of breaks down that teamwork that you really need if you want them to learn more and be colleagues really. But I don’t know how you’d change that attitude other than maybe they earn less from being a locum. But I don’t think you’d fill the posts.

Interview 53, Locum, Secondary Care

Defensive practice, hedging and avoidance

Participants described instances of defensive practice or hedging-type defensive medicine which involved providing services (e.g. tests, referrals) usually to reduce perceived litigation or other adverse outcome. For locums, adverse outcomes could include a higher risk of complaints and potential termination of contract at short notice. Some locum participants described practising defensively to keep patients on side and prevent complaints because of the precarious nature of the locum contract and pervasive negative perceptions from patients and other healthcare professionals. Locum participants argued that locums were more likely to practise defensively because they were attempting to practise as safely as possible in complex environments where they were professionally isolated and may not be familiar or supported. Rationing resources was regarded as benefitting the organisation but not the patient or locum; this meant that some patients actively sought out locums who were regarded to be less constrained by the organisation. There was a perception that locums deflected some of their responsibilities because they were disempowered to make decisions, while others felt that some locums were simply avoiding work and evaded responsibility for patients by pushing work onto others or into the future, by which time they will have moved on.

I think decision-making can be very, very difficult as a locum. And also if the patient comes with some demands of sleeping pills, or painkillers, or things, anything. Then putting your foot down is difficult because the patients are more likely to complain about locums than about the partners. So often locums will be softer, and some patients actually know to get an appointment with locums because they can get the antibiotic they want, or whatever they want, kind of thing.

Interview 83, Locum GP, Primary Care

Well, the way things are now and how difficult it is to see any doctor, I’d see anybody … So when you get offered something, you take it. Even though it might mean it’s a complete waste of time, you see somebody who really doesn’t feel able or willing to treat you and advises you to come back and see a regular doctor, you’d do anything because you’re not being offered a choice even to have an appointment.

Patient 2, Focus Group A

In addition, locums described a sense of professional isolation as well as the perception that the ‘system will crush you’ if a mistake is made. Locums described how easily they could be scapegoated, meaning that some locums were reluctant to make decisions and deferred responsibility to others in the organisation. Participants gave examples of overly cautious management including, for example, carrying out more tests or referring patients for second opinion more than necessary and this was sometimes attributed to locums lacking confidence in their own abilities to diagnose without getting a range of investigations done.

And the irritation of partners is that a locum refuses to do this, they over-refer, they make sure that they’ve covered all the options so that nobody can come back to them and say, you didn’t do this right. And the errors will happen. I think that there is a defensiveness in medicine, which is increasing and really harming people. There is nothing else that you can take from the whole Bawa Garba incident other than the system will crush you if you make a mistake, if you don’t deal with an acute problem you can go to jail.

Interview 38, Locum, Primary and Secondary Care

Governance

The governance practices in relation to locums varied widely and were not generally regarded as being as robust in comparison to permanently employed doctors. Responsibility for locum doctors was sometimes unclear in terms of who was accountable for organising, providing or supporting governance mechanisms, such as performance feedback, supervision and oversight, educational opportunities and CPD, appraisal and involvement in quality improvement. While some organisations included locums in their governance activities, others regarded locum work as transactional, where the locum was there to provide a finite service and the organisation assumed no responsibilities for their performance, development or oversight. There were concerns that governance structures were modelled on and designed for permanently employed doctors, meaning locums were not fully integrated into them or able even to access informal and formal governance mechanisms.

Transience and distance meant judgements about clinical ability and systems such as appraisal and revalidation were reliant on information from actors who might have limited knowledge of the locum and their performance. Locum doctors revalidate in the same way as any other doctor, but they may experience challenges in finding a RO or collecting supporting information, particularly if the locum works for a locum agency. NHS ROs are more likely to be familiar with the organisation and the doctors who work there. Locum agencies’ ROs may carry out their role for a number of designated bodies (the employer or agency for which the doctor does the majority of their work). There is a risk that locum agency ROs have less direct involvement in the locum agency systems and with the board or governing body. ROs relied on concerns being escalated in order to keep patients safe, manage concerns, support doctors and make revalidation decisions. However, this became more difficult for locum ROs, as they were unlikely to meet the locums or their appraisers or colleagues. A locum’s relationship with their RO, appraiser, colleagues and patients could be more distant and removed from the usual social exchanges and mutual engagement that occur between colleagues working within an organisation, meaning it was difficult for others to make fully informed judgements about a doctor’s fitness to practice. Agency ROs described not knowing the locum doctors they were responsible for in the same depth as when they had performed the RO role in the NHS, where they knew the doctors personally; ‘they don’t know me and I don’t know them’. Locums could be unfamiliar to colleagues and contact could be limited and others in the organisation were less likely to have a detailed view of the competencies and capabilities of locum doctors. As a consequence, the level of oversight ROs and other colleagues had of locums was likely to be reduced in some circumstances and particularly when locums were very transient.

I think it’s a remote world. It’s like a cloud, you know, it’s like the cloud. We talk about the cloud when it comes to storing information. And I think locum world is a bit like that … And I don’t know the doctors anywhere like as much as I did when I was an RO in the NHS, I knew them all personally. If I used to have a problem, I used to get them in my office there and then, chat it all through, sort it. Can’t do that in locum world, it might take me four days to get hold of the doctor, some of them won’t respond immediately. I always will have a telephone conversation with them. They don’t know me and I don’t know them.

Interview 51, Locum Agency RO

Similar governance and information-sharing problems were described by locum agencies and NHS organisations; both described difficulties in sharing and gathering feedback. When concerns were raised, participants described receiving little or no response or assurances about whether governance information that had been shared across organisations was acted upon and participants were often uncertain as to what happened to the information they shared. Communication difficulties, in part due to the fluid nature of locum working, meant that robust oversight was less likely for doctors moving between organisations.

It would give you more confidence if you heard back. And sometimes I’ll pick up the phone and you try to do the best you can to make sure this information gets passed on. But I just have this nagging doubt that I’m not always convinced it does.

Interview 30, Responsible Officer, Secondary Care

It’s been bothering me for a little while that sometimes when you request for information from other responsible officers you don’t always get a response. Which, kind of, leaves a bit of an unknown, especially when you’re then coming up towards a revalidation decision, you know. It’s not a fool-proof mechanism.

Interview 48, Director of Quality, Locum Agency

To improve the flow of information between organisations, NHS England introduced a system to allow the organisation to share information about locums and doctors moving between designated bodies. The Medical Practice Information Transfer (MPIT) form is designed to support the transfer of information about a doctor’s practice to and from the doctor’s RO. However, organisations did not always complete or return MPIT forms, and one RO indicated that their locum agency did not use their resources in perusing them.

There is now a system in place, that the GMC introduced and we’ve instituted over the last year really called the MPIT, it’s an official form, official GMC form … so somebody signs on with us, a new doctor and connects as a designated body, we send the previous responsible officer this form, basically saying, is there any history with this doctor and their answer is either yes or no. The returns on these forms are not reliable because I would think 50 per cent of the ones, I send off don’t get returned yet but the system is, we got it running pretty much straightaway but I think there is a number of departments in places where the form goes into some address that maybe a person who can’t address it and it doesn’t get … the returns are pretty poor. Now we don’t chase them up because administratively it’s a difficult thing to do with our resources.

Interview 47, Locum Agency RO

Continuity of governance was difficult due to the transient nature of locum work and issues of information flow between organisations. There was a perception that some locums attempted to circumvent governance and revalidation systems by moving between agencies. It was highlighted that some agencies have higher governance standards than others and some locums avoided these agencies because of this. To mitigate this, one agency had a policy on discouraging connections in the period close to revalidation and deferring revalidation for some locums to allow time for gathering evidence. There was also a perception from some locum agency ROs that while the majority of doctors were excellent, there was a percentage of locum doctors that were isolated doctors with potential performance issues who were likely to be most in need of supportive organisational and professional support.

You have to accept that whilst within the agency world, 80 per cent of the doctors we place are excellent, and have no problems, and do a great job, perhaps 20 per cent are those that have shaken down to that 20 per cent in the agency world, because they’ve not succeeded in the NHS, they’ve not got a substantive place, they are lost souls. And they are less able to cope with the vicissitudes of busy clinical life and professional life within a large organisation such as the NHS.

Interview 47, Locum Agency RO

There was concern that governance structures were modelled on and designed for permanently employed doctors, meaning locums were not fully integrated into or able to access informal and formal governance mechanisms. There was broad consensus that locums were less likely to be involved in learning processes in organisations, including routine learning opportunities and communications and learning from mistakes. Locums were not included in in-house learning activities for a number of reasons; there was a perception among some participants that locums were primarily employed to deliver services and cover staff shortages; consequently, some organisations did not assume responsibility for providing learning, largely due to additional cost. Barriers to inclusion in organisational learning also included the additional administrative burden in already pressured environments, the transient nature of the locum workforce and variability between organisations in terms of ability to maintain accurate and current e-mail distribution lists. Resentment around pay and workload and othering was also a factor described by some participants, which meant that learning was sometimes perceived to be exclusive to permanent staff and responsibility for learning was assumed by the locum.

The trust doesn’t consider us to have any responsibility for them in regards to development or career progression.

Interview 7, Medical Staffing, Secondary Care

Locum attendance at in-house non-mandatory training and inclusion in routine communications and team meetings was variable and typically optional for both locums and organisations and, in part, dependent on the motivation and discretion of the organisation and the locum. In some organisations, locums were welcome to attend training (particularly if they were working on a longer-term basis), though sometimes with the proviso that they would not be paid for doing so. However, in organisations that were less inclusive, locum participants described hearing about changes relevant to their practice (e.g. significant changes to policy or practice) by chance or through word of mouth.

They don’t want to pay you for induction, they are not interested in you coming to any of their meetings or MDTs, and they will certainly not have any kind of CPD engagement with you unless you happened to be there and they happened to have it on already and it didn’t make a difference if you were there or not.

Interview 35, Locum GP, Primary Care

Locums did not typically have established relationships or access to support in comparison to permanent doctors. There were also ingrained negative perceptions relating to locum working. As a consequence, locums were likely to be blamed as individuals without taking into account systemic antecedents to problems, such as lack of induction, supervision or support, whereas permanent doctors who made mistakes were described as receiving remediation and support. Problems or mistakes were not typically addressed productively and collaboratively by organisations, meaning locums were sometimes denied the opportunity to learn when things went wrong. Locums did not always receive feedback essential to their practice and learning and end-of-placement feedback was not always provided. Worryingly, participants described instances of locum contracts being ended when there were problems, without the reasons why being communicated with the locum or other required parties. Lack of communication and feedback had implications for locums in terms of equipping them with the necessary information to carry out their role safely and effectively.

We had a big incident actually which was a consultant who tried to blame a locum for something that went wrong … And it was quite clear that it wasn’t the locum that caused the problem … and it’s so much easier just to blame the locum and then brush it all under the carpet because you can just end the locum’s appointment. Whereas actually, if there’s a problem with a substantive appointment, one problem with the NHS is, it’s very difficult to get rid of hopeless individuals.

Interview 24, Locum, Secondary Care

Recruitment

Patients were purposively sampled through patient and public contributor forums. Our PPI chair recommended that we recruit through Research for the Future and NIHR Clinical Research Network (CRN) which is an initiative to help recruit people to take part in health and care research using a ‘consent for approach’ model. Participants were paid £25 in vouchers or by bank transfer for their participation.

Data collection

Two authors (GS and JF) and two members of our PPI forum (MS and MM) carried out 5 focus groups with 30 participants. Focus groups involved between four and eight participants per group. GS and JF also carried out one-to-one interviews with 12 participants. Data were collected using video-conferencing software (n = 15) or over the phone (n = 2). Forty-one participants spoke about their own experiences as a patient; one participant was a carer who spoke about their experiences with the person they care for and also their own experiences as a patient (see Appendix 6 for additional participant quotations).

Analysis

Patient perceptions

Clinical acumen, enthusiasm for the role and the quality of supporting infrastructure were generally regarded by participants as more indicative of a doctor’s performance than employment status. Participants spoke about the variability in practice and personality between different doctors regardless of whether they worked as a locum or not. Participants who attended regularly were knowledgeable about the different GPs at the practice and knew who to avoid and who they preferred. Participants spoke about a false dichotomy between locum and permanent GPs and described positive and negative experiences of both. It was also recognised that the systems and environments that doctors worked in could potentially impact the locum’s ability to perform as well as their local knowledge and familiarity with patients, systems and care pathways.

That I think locums are no different than regular doctors and they find themselves sometimes in a difficult position because they are thrown into unknown place and they have to figure it out quickly. And as I said, they sometimes lack this confidence that a regular doctor has, just from being in single place for a number of years … locums shouldn’t be treated any differently than regular doctors, they’ve completed their education, their training, they want to help, they want to be professional, et cetera. Just perhaps the patients should be more tolerant.

Interview 78, Patient

You’ve got doctors which want to do their job and you’ve got doctors that are just there to get their pay check and clear off at the end of the day. It don’t matter whether they’re a locum or whether they’re full time.

Focus Group B, R8

Patient awareness

Participants were not always aware, in both primary and secondary care, whether their doctor was a locum. Participants mostly spoke about their experiences of seeing a locum at their GP practice, as they were more likely to be familiar with practice staff compared with a secondary care setting. Some participants felt that it was difficult to distinguish who was who at their practice when there were lots of doctors with different working patterns. The fact the doctor was a locum could be communicated by practice receptionists, and sometimes in an apologetic way.

I do get to see a regular GP. But if have to do a telephone appointment, especially if it’s an emergency type one, then I am advised over the phone that it’s a locum. And I have an occasion I’ve been and my regular doctor hasn’t been available and then they’ve said, oh, I’m sorry, the locum’s going to see you today.

Focus Group B, R3

There were differing opinions among participants about whether patients should be informed whether the doctor was a locum. Some participants felt it was important, particularly for those with long-term conditions or mental health problems, to be given information about who they were seeing, especially if they have not seen that doctor before.

That’s very important. Like as a patient, it should be explained under the charter of the service, what the whole purpose is of your treatment, wherever you’ll be seen by a junior doctor. So wherever you’ll be seen by a locum, or wherever you’ll be seen by another specialist. So I think it’s very important, you know, for either trusts or the NHS, to make people aware about what the service is, in the first instance.

Interview 95, Patient

I think when it’s long term patients, as it mostly is with mental health, I think they do need to know that it’s a locum, because it will be a long term relationship usually.

Interview 86, Clinical Lead Secondary Care

Participants perceived that locums and organisations may be reluctant to disclose to patients their doctor was a locum because of the recognised negative perceptions around locum working and the potential implications for consultations and doctor/patient relationships. There were also concerns that patients might not want to be seen by a locum.

I: Do you think that you should be told when you’re seeing a locum?

R: No, no, not necessarily, as I said, I don’t have a problem with that, but I think that some people may be prejudiced and may be, you know, throw a tantrum or something.

Interview 78, Patient

Why would you badge it with locum because I think if the reality is if you do use the word locum, I think most patients are familiar with the word locum and they’ll see it in a negative way as this is somebody that’s not going to know me, this is somebody that’s not going to be as close to my history and it’ll put those barriers up almost immediately.

Interview 27, Medical Director

Sometimes patients don’t react in the best way to knowing that it’s a locum doctor. Oh no, I want to see a ‘real’ doctor, and that is really difficult to manage from the agency doctor point of view ’cause that makes them feel not welcome, it makes them feel like they’re second class citizens, if you like, when absolutely they’re not. That’s difficult to manage from a patient point of view as well.

Interview 25, Medical Staffing, Secondary Care

While some patient participants felt that patients should know when a doctor was working as a locum, the general consensus among participants who were doctors or working in health care was that patients did not need to know. Participants perceived that patients in secondary care were less aware and concerned about whether the doctor was a locum. Participants felt that patients did not need to know their doctor was a locum because the standards of safety, patient care and governance processes should be the same regardless of the doctor’s contract. Some participants perceived that patients were not concerned if they were seen by a locum, as long as care was delivered by a doctor who was qualified and capable.

I: Do you think patients should know or need to know?

R: No, I don’t think they should or be told. I think the system ought to be robust and safe enough that it doesn’t matter.

Interview 30, Responsible Officer, Secondary Care

I: Do you think patients should know or need to know?

R: I would say if we’ve got the right governance in place and we’ve got the same level of assurance that they are safe, they are a quality practitioner and that they are up to date with what they need to be, I’m not sure patients do need to know ’cause there’s no difference in terms of they’ve got the required skillset, they’ve got the required training, they’ve got the required knowledge, I’m not sure there is a need to know.

Interview 25, Medical staffing, Secondary care

Continuity of care

Traditional notions of continuity of care and a family GP were recognised as a thing of the past. This was perceived to be a consequence of how modern general practice was structured and how GPs approached their working lives, with more GPs choosing to work flexibly.

I desperately want it to all be about seeing the GP and having continuity, and them knowing me and my family inside out. But I do think we’re at the stage where that’s kind of like a media construct of a GP, it’s like a country practice version of a GP. And it hasn’t existed for like five decades.

Focus Group B, R2

Communication and practice

Locum working was described as influencing how doctors communicated and practised, which could have positive or negative implications for patient care and outcomes. Defensive practice was described by patients, locums and participants who worked with locums. However, there were also positive outcomes for patients when they were treated by locums who were able to provide fresh insights and new perspectives.

Data

Clinical Practice Research Databank GOLD is a large, computerised database of anonymised primary care medical records. It contains complete electronic health records (EHRs) for over 14 million patients in general practices using the Vision system, with the healthcare events (diagnoses, treatments, referrals, tests and prescriptions) recorded using coding systems. 125 The database is broadly representative of the UK population, although larger practices are over-represented and the data have been shown in numerous validation studies to be generally of high quality. 126,127 Practices need to meet pre-specified data entry quality criteria to be defined as ‘up to research standard’, and for each study year, our main sample included all CPRD practices that were classed as such for the whole year in all UK countries.

We also obtained CPRD-linked HES data. The national HES data contain details of all admissions to NHS hospitals in England, including information on deaths in hospital. 49 Area deprivation, as measured by the latest update of the IMD (i.e. 2015), was available at the 2011 LSOA level. The IMD measures deprivation at area level based on domains, such as income, employment, health, housing and general environment and is the most complete and widely used approach to quantify relative deprivation and affluence for small areas in England. The IMD is assigned to every small area in England and ranks them from 1 (most deprived) to 32,844 (least deprived). 39

Linkage of CPRD GOLD data to the HES and IMD databases is available for those individual patients who are registered at practices in England and who have consented to data linkage. Our cohort was eligible for linkage to HES and IMD, and we provided pseudonymised patient identifiers to CPRD so that linkage could be performed. CPRD used a deterministic linkage algorithm to perform linkage between data sets on behalf of the research team. The algorithm was based on a patient’s exact NHS identification number, sex, date of birth and residential postcode and linked data were subsequently provided to the research team.

Analyses

Outcome measures for patient-level analyses

Practice-level analyses

The number of practices across all UK countries who participated in CPRD GOLD varied from 870 in 2010, to 716 in 2015, to 422 in 2021. Over time, data were available for 914 distinct general practices in the UK, of which 511 practices were located in England and its regions, 234 practices were located in Scotland, 127 were located in Wales and 42 were located in Northern Ireland. Of these, only 381 practices contributed data throughout the whole of the study period, of which only 42 were located in England and these are reported in Table 33. Our final cohort consisted of 8,133,324 patients with 184,269,687 recorded consultations.

We provide descriptive statistics for the different types of consultations by type of GP (permanent or locum) and over time in Table 20. We calculated the proportion of consultations done by locums in 2019, as a proportion of all consultations done by all GPs, in every practice in every year and provide descriptive statistics in Table 21. Our reference year was 2019, as that was the year before the onset of the COVID-19 pandemic which brought substantial disruptions in the delivery of NHS services and is generally considered a period of instability within the NHS. This process was followed for all types of consultations. Descriptive statistics for the different consultation types using only practices that contributed data throughout the period of study are provided in Appendix 7 (see Table 35).

Our practice-level analyses were descriptive and violin plots to depict the variation in the proportion of all, face-to-face (F2F) and telephone/online locum consultations over time are presented in Figures 14–16. Violin plots can be interpreted as box plots (including the median as a marker and a box indicating the IQR), overlaid with the density of the distribution for enhanced visualisation. The median rate of locum consultations as a proportion of all GP consultations varied from 5.5% in 2010 to 6.7% in 2021. For F2F consultations, the median rate varied from 5.9% in 2010 to 6% in 2021. Variation was more pronounced for telephone consultations, where the median rate varied from 0.7% in 2010 to 6.6% in 2021. The results from the sensitivity analyses using only practices that contributed to CPRD throughout the study period are provided in Appendix 7 (see Figures 44–46).

FIGURE 14.

All locum consultations as a proportion of all GP consultations.

FIGURE 15.

Locum F2F consultations as a proportion of all GP F2F consultations.

FIGURE 16.

Locum telephone and online consultations as a proportion of all GP telephone and online consultations.

Regional variation in all, F2F, and telephone/online locum consultations aggregated for the period 2010–21 for all UK countries as well as the English regions is depicted in Figures 17–19. Practices in Wales, Scotland and the South West of England had the largest proportion of care provided by locums and practices in the Midlands, East of England and the North East had the lowest. These findings were similar for F2F consultations but for telephone consultations, practices in the Northern Ireland, Wales and Scotland had by far the largest proportion of care provided by locums. The regional variation results from the sensitivity analysis using only practices that contributed data throughout the period of study are presented in Appendix 7 (see Figures 47–49).

FIGURE 17.

Regional variation in the proportion of locum consultations in 2010–21.

FIGURE 18.

Regional variation in the proportion of F2F locum consultations in 2010–21.

FIGURE 19.

Regional variation in the proportion of telephone and online locum consultations in 2010–21.

In Figure 20, we show the total number of consultations completed by all GPs and locums by day of week aggregated over 2010–21, and in Figure 21, we show the mean proportion of locum consultations by day of week, over 2010–21. Locum GPs appeared to do more work during the week where they provided on average 15% of total care. The proportion of care provided by locums during the weekend was 14%. The results from the sensitivity analysis using only practices that contributed data throughout the period of study are presented in Appendix 7 (see Figures 50 and 51).

FIGURE 20.

Total consultation volume by GP type and day of week, aggregated over 2010–21.

FIGURE 21.

Mean proportion of GP locum consultation by day of week, over 2010–21. Note: Bar labels show the average number of GP consultations done in each day across 870 UK practices.

Patient-level analyses

The number of practices in England participating in CPRD GOLD varied from 487 in 2010, to 228 in 2015, to 42 in 2021. Of these, only 42 practices contributed data throughout the whole of the study period. Our final cohort consisted of 3,591,367 patients with 13,696,455 recorded consultations between 407 practices across all years. For the second set of models, our cohort consisted of 547,146 patients with 7,623,205 recorded consultations, which varied by indicator and included patients from 407 practices across all years. We provide descriptive statistics for the outcome and some important practice and patient characteristics in Table 22. In Table 23, we summarise the numerators and denominators that allowed us to calculate the proportion of consultations that were exposed to potentially hazardous prescribing for each indicator. The numerator is the number of consultations that were exposed to each type of potentially hazardous prescribing, and the denominator is the number of consultations of patients at risk of exposure to the hazardous prescribing indicator.

Clinical practice analyses

In the first set of models investigating various clinical practice indicators, we randomly selected one consultation event for each patient within each financial year, aligning all the patient outcomes and covariates to that specific event date. This allowed us to give equal weights to patients and limited the potential for confounding introduced by higher-need patients who may be visiting numerous times within a year. Our exposure was a binary variable indicating whether the consultation was by a permanent GP or a locum GP. We were able to identify permanent GPs and locum GP through the staff role field which is available for every consultation. This approach was used for practice revisits; prescribing of antibiotics, strong opioids and hypnotics; tests and referrals; and hospital outcomes.

Prescribing safety analyses

In the second set of models investigating the PINCER prescribing safety indicators, for each indicator, we identified all consultation events with patients who could be exposed to potentially hazardous prescribing, because of a specific diagnosis or prescription on the day of the consultation (i.e. index event). These events were split into consultations by locum or permanent GPs. Second, for each index consultation event, we looked at consultation events during a pre-specified time window (which varied across indicators; Appendix 7, Table 36) leading up to the index event, to identify pre-existing prescriptions or conditions that would trigger a potentially hazardous prescribing outcome when combined with the index event. For each index consultation event, a binary variable indicated whether potentially hazardous prescribing was triggered. This allowed us to operationalise rates of potentially hazardous prescribing events for both locum and permanent GPs. Our exposure was again a binary variable indicating whether the consultation involved a permanent GP or a locum GP and we aligned patient covariates to the index consultation event using unique patient IDs.

For example, for indicator A we identified consultations for patients who were over 65 years, at which they were prescribed a NSAID. We then identified those patients who were not also prescribed the recommended proton-pump inhibitor (PPI) or H2 receptor antagonist at the consultation or in the preceding 3 months. The operational definitions for the PINCER prescribing safety indicators are provided in Appendix 7, Table 36.

Our regression models in Table 23 show that a consultation with a locum was 12% less likely to lead to a practice revisit within 7 days [odds ratio (OR) = 0.88, 95% CI 0.87 to 0.88]. A consultation with a locum was 22% more likely to involve a prescription for an antibiotic (OR = 1.22, 95% CI 1.21 to 1.22), 8% more likely to involve a prescription for an opioid (OR = 1.08, 95% CI 1.06 to 1.09) and 4% less likely to involve a prescription for a hypnotic (OR = 0.96, 95% CI 0.94 to 0.98). In terms of hospital-related outcomes, a consultation with a locum was 4% more likely to be followed by an A&E visit on the same day (OR = 1.04, 95% CI 1.01 to 1.08) and 5% more likely to be followed by an A&E visit between the 1st and the 7th day (OR = 1.05, 95% CI 1.02 to 1.08), but there was no impact on emergency admissions or ACSC emergency admissions. Consultations with locums were also 15% less likely to involve a referral (OR = 0.85, 95% CI 0.84 to 0.86) and 19% less likely to involve a test being ordered (OR = 0.81, 95% CI 0.80 to 0.82).

When comparing prescribing safety indicators for permanent and locum GPs, a consultation with a locum GP was 11.2% (OR = 1.12, 95% CI 1.08 to 1.16) more likely to involve the prescription of an oral NSAID, without co-prescription of an ulcer-healing drug, to a patient aged ≥ 65 years. But a consultation with a locum GP was 22.8% (OR = 0.77, 95% CI 0.64 to 0.93) less likely to involve the prescription of warfarin or a direct oral anticoagulant in combination with an oral NSAID, and 11.2% (OR = 0.89, 95% CI 0.85 to 0.93) less likely to involve the prescription of a long-acting beta-2 antagonist inhaler to a patient with asthma who is not also prescribed an inhaled corticosteroid. We did not find any significant differences between permanent and locum GPs across all other prescribing safety indicators.

We plotted the effects and the CIs of locum consultations on performance outcomes in Figure 22 and on prescribing safety outcomes in Figure 23 which enable comparisons across outcomes.

FIGURE 22.

Coefficient plot for locum consultations across all outcomes. Note: Results are expressed as OR and corresponding CIs. When the corresponding CIs cross the dashed vertical line, coefficients are not statistically significant.

FIGURE 23.

Coefficient plot for locum consultations across all prescribing safety outcomes. Note: Results are expressed as OR and corresponding CIs. When the corresponding CIs cross the dashed vertical line, coefficients are not statistically significant.

The full output from regressions in Table 23 is presented in Appendix 7, Tables 37 A–C and 38 A and B. We find that patients with increased multimorbidity had higher odds for a practice revisit within 7 days (OR = 1.20, 95% CI 1.19 to 1.20), a same day emergency admission (OR = 1.35, 95% CI 1.36 to 1.37), an emergency admission within 1–7 days (OR = 1.37, 95% CI 1.35 to 1.39). Patients with high multimorbidity also had higher odds of a same day A&E visit (OR = 1.19, 95% CI 1.17 to 1.20), an A&E visit within 1–7 days (OR = 1.19, 95% CI 1.18 to 1.20), a same day ACSC admission (OR = 1.34, 95% CI 1.32 to 1.36) and an ACSC admission within 1–7 days (OR = 1.38, 95% CI 1.36 to 1.39). The odds of an increase in the outcome reduced with deprivation, with the largest difference for opioid prescriptions, hypnotic prescriptions and emergency admissions observed between the 4th and the 5th quintile of deprivation. As deprivation reduced, we also saw an increase in the likelihood of practice revisits, A&E visits, ACSC admissions and tests, with the largest differences observed between the 3rd and the 4th quintile. Higher odds for antibiotics and referrals were observed with increased deprivation.

In Appendix 7, Figure 52, we show the mean and 95% CIs for all non-hospital outcomes (practice revisits, antibiotic prescriptions, opioid prescriptions, hypnotic prescriptions, referrals and tests) at the practice level by locum and permanent GPs in 2010–21. Over time, we observed small variability for practice revisits, hypnotics, referrals and tests; however, in 2020 there were sharp increases for practice revisits and a sharp decline for referrals, hypnotics and tests. We also observed a steady decline in the prescription of antibiotics and opioids over time. Differences between locums and permanent GPs were present for most outcomes and most noticeable for practice revisits and antibiotic prescriptions. In Appendix 7, Figure 53, we show the mean and 95% CIs for all hospital outcomes at the practice level by GP type over 2010–21. Overall, the six outcomes followed a modest increase over the study period and the differences between locums and permanent GPs were minimal. We repeated the analyses of patient outcomes and prescribing safety indicators, excluding all consultations with GP retainers and GP registrars as their clinical practice and prescribing behaviour may differ from permanent GPs. Overall, the results from these analyses were very similar to our primary analyses.

Summary

This study provides evidence on the extent of locum use in the UK primary care and evidence on the effects of locum working on patient outcomes for the period 2010–21. Our findings suggest that locum GPs provided around 10% of GP consultations (median = 6.7%, mean = 13.8%) and this figure remained relatively stable over time.

We observed moderate variability in the proportion of care provided by locum GPs across English regions and across the four UK countries. Locums appeared to do more consultations during weekdays compared with the weekend. In terms of the effects of GP consultations on patient outcomes, our primary outcome showed that patients were less likely to revisit their practice within 7 days following a consultation with a locum GP. For secondary outcomes, the largest effects were for antibiotics, referrals and tests, where locum GPs completed more antibiotic prescriptions, but completed fewer referrals and ordered fewer tests when compared with permanent GPs. On the PINCER prescribing safety indicators, again the results are mixed. On most indicators, there is no significant difference between locum and permanent GPs – and the differences we do observe on three indicators are not large and move in different directions. There certainly seems to be no basis to argue that locum GPs differ significantly from permanent GPs on these indicators.

Our findings provide useful insight on the extent of locum use in the UK primary care and have implications for the differences in the clinical practice and performance of locum doctors working in general practices in England.

Strengths and limitations

This is the largest longitudinal study of locum consultation rates in UK primary care. However, there are important limitations. First, any study of this nature is limited by the reliability and accuracy of the data in the patient’s electronic record. We are confident about the reliability of the recorded patient contact data and patient characteristics. However, less is known about the accuracy of the information on type of consultation, as primary care staff may be inclined to record F2F as the default option. If so, our data will have overestimated F2F consultations. 9 Second, we could not assess the recording of the staff role linked to each consultation. Practice managers may sometimes assign locum GPs a staff role value of a permanent GP. This may be more common with long-term locum GPs who tend to work in practices for prolonged periods. Third, CPRD collects data from practices using the Vision clinical information system and recording activity may differ for practices using other systems, although we would not expect that potential variation to affect our findings. Fourth, sex, age and years worked in practice are GP characteristics which are unavailable within CPRD, and we therefore could not control for these characteristics in our regression analyses. Fifth, our comparisons of the distribution of locum work during the week are limited by the complex working arrangements in general practice in the weekend. It may be that weekend work is the responsibility of a trust or consortium of GPs, which complicates comparisons of weekend working. Finally, CPRD GOLD is representative in terms of deprivation and population characteristics,125 collecting of data is available from a single clinical information system (Vision) and therefore contributing practices are not uniformly distributed across English regions, while its market share is in decline. 129 Thus, generalisability to every English region could not be achieved.

Interpretation of findings

The unique contribution of this study is the investigation of distribution and trends in the delivery of consultations by GP locums and the differences in patient outcomes when care is delivered by locums and permanent GPs. Our finding that the amount of care provided by locum GPs has been relatively stable over time confirms the finding in our previous study using NHS Digital data. 36 However, there are differences in the amount of care provided by locums between these two studies, which we attribute to CPRD measuring consultations and NHS Digital measuring FTE. We hypothesise that locums spend more of their time providing consultations than permanent GPs. Our qualitative work revealed that sometimes practice managers may record locums as permanent GPs, and some types of locum GPs, such as long-term locums, may be working long enough in a practice to be considered permanent members of staff. Therefore, the figure that we found in the analyses of NHS Digital data is likely underestimating the proportion of care provided by locum GPs.

In CPRD, recording is considered more accurate, as consultation events are central to how data are organised in the Vision computer system. However, in CPRD, it is not currently possible to identify a member of staff who has worked in multiple roles in a practice or in multiple roles across practices. It may be that salaried GPs are employed simultaneously as locum GPs in other practices and we would not be able to account for that as each member of staff has a staff ID which is unique to that practice. Despite these issues, our analyses can help with the understanding of the NHS workforce and can provide important information about the planning of the delivery of primary care services in the UK.

This work also provides evidence on differences in clinical practice and performance between locum doctors and permanent doctors which may have consequences for patient safety and quality of care. A consultation with a locum was less likely to lead to a practice revisit within 7 days. Locums were more likely to prescribe antibiotics and opioids, but less likely to order tests and referrals and prescribe hypnotics. A consultation with a locum was more likely to lead to an A&E attendance on the same day or within 1–7 days. Previous evidence suggests that the way locum doctors are recruited, employed and used by organisations may pose a higher risk of harm to patients, but our findings provided mixed results. In some areas, locum GPs performed worse and this may suggest that they need additional support from practices with improved induction and communication about practices’ approaches to prescribing and continuity of care. Recording of locum working across general practices should be improved and information should be linked across clinical systems and databases as this would help verify the findings of this chapter. Locum GPs face substantial challenges, with large variability between practices and areas, more acutely unwell and unfamiliar patients, limited communication, feedback and support and professional isolation. If local healthcare needs are to be met, locum GPs need sufficient resources for training and effective integration, which would be aligned with the aims set out in the NHS Long Term Plan. 45

Data

Analysis

Descriptive analysis was performed on the collapsed Bradford EPR data. We plotted the mean number of patients seen and orders created per doctor for each week of data. We calculated the total number of patients seen and orders created per doctor over the entire period, and presented the distribution of doctor workload across these measures of activity.

To identify working patterns for doctors, we focus on a 52-week period from 24 September 2017 to 16 September 2018. This period was selected to avoid the impact of COVID-19 and also a change in how the EPR was implemented: see Appendix 8, Figure 54 describing the whole period.

We calculated the number of doctors (unique IDs) who created an order in each week of the data, as a measure of the size of the active workforce over time. We also calculated the number of active weeks for each doctor as a measure of their length of activity for the 52-week period.

As doctor IDs did not distinguish between permanent doctors and locums, we attempted to identify short-term locums as doctors who are active for 4 weeks or less during the 52-week period. We do not identify a doctor as locum if they are only active at the beginning or end of our period, as these may represent longer periods of activity which we do not observe. We compare these short-term locums to all other doctors on the average number of patients seen and orders created per active week.

Results

In Figure 24 we present the mean patients seen and orders created per active doctor per week for 52 weeks from 24 September 2017 to 16 September 2018. For example, in the week beginning 24 September 2017, doctors who were active created orders for a mean of 6.6 patients. For these figures, active doctors are those who saw at least one patient in a given week. The mean number of patients seen was 7.7 and the mean number of orders created was 86.

FIGURE 24.

Mean patients seen and orders created per active doctor per week.

Between 24 September 2017 and 16 September 2018, there were 1461 doctors who were active in at least 1 week. Over all 52 weeks, the total number of patients seen and orders created per doctor is presented in Figure 25. Doctors created orders for a median of 78 (mean of 168) patients and created a median of 564 (mean of 1860) orders. There was significant variation in these values, with some doctors having extremely high levels of activity (e.g. the 95th percentile is 9000 orders created over 52 weeks). This distribution suggests that approximately 70 doctors created on average 170 orders every week, with many doctors creating almost no orders. This seems unlikely to represent true activity levels and may be driven by a mismatch between the doctor IDs within the EPR and with the doctors carrying out the activity. For example, the IDs associated with very high levels of activity may be used by multiple doctors.

FIGURE 25.

Total patients seen and orders created per doctor, September 2017–September 2018. Note: Dashed line for median, solid line for mean.

Doctors were active between 1 and 52 weeks during the period analysed. Figure 26 presents how many weeks doctors were active. For example, from 1461 doctors, 130 of these were active for only 1 week, which represents 8.9% of doctors. These findings suggest that a significant proportion of doctors are only active for a relatively low number of weeks (<10 weeks). This distribution is unusual and raises concerns about the validity of the data and doctor IDs. We do not observe a large group of doctors active for around 44 weeks, which would be expected for those working in full-time permanent positions. Furthermore, that the number of doctors’ active does not vary much between 10 and 52 weeks is also a concern. Some of these unusual patterns might be explained, partially, by junior doctors and their rotations on and off placements.

FIGURE 26.

Count and percentage of doctors by number of weeks active, September 2017–September 2018.

To investigate further and attempt to identify short-term locums, we focus on only those doctors who created an order in 4 weeks or fewer, and the active weeks for these doctors are shown in Figure 27 (274 doctors in total). For example, the first distribution shows doctors who were active for only 1 week, and in which single week they were active. For these doctors, we see a large proportion active in weeks 1 and 2. This is expected and likely capturing some doctors who were active before our observation period: similar patterns are observed for doctors working 2, 3 and 4 weeks.

FIGURE 27.

Count of doctors active in each week, doctors who are active for 1–4 weeks only.

Excluding doctors who work at the beginning and end of the period yields 163 doctors who have working patterns which are consistent with a short-term locum and may identify them as potential locums. These doctors work no more than 4 weeks in a 52-week period and do not work at the beginning or end of this period. The beginning and end periods are defined as:

• weeks 1, 2 and 52 for doctors working in 1 week only

• weeks 1, 2, 51 and 52 for doctors working in 2 weeks only

• weeks 1, 2, 3, 50, 51 and 52 for doctors working in 3 weeks only

• weeks 1, 2, 3, 4, 49, 50, 51 and 52 for doctors working in 4 weeks only.

These potential locums, on average, created orders for 1.8 patients and created 15 orders per week in the limited weeks they were active. This is a much lower level of activity than doctors we do not identify as potential locums, who doctors created orders for 7.6 patients and created 85 orders per week.

Over the 52 weeks in total, doctors identified as potential locums created orders for 494 patients and created 4133 orders in all the weeks they were active. These numbers represent approximately 0.15% and 0.20% of the total over the period.

The very low levels of activity, both per week and in total, suggest that these doctors are: (1) locums who create orders very rarely; (2) locums who create more orders but these are associated with other doctor IDs; or (3) not locums, perhaps temporary IDs which are used for other purposes. Given the lack of confidence in the ability to identify locums using EPR data, we do not progress with our further aims of comparing locum and permanent doctors on quality and safety.

Summary

Accessing any EPR data was challenging and ultimately we concluded that the EPR data we received from Bradford could not be used to robustly identify locums, and so it was not then possible to go on to explore differences in practice and performance (e.g. in test ordering, prescribing or other behaviours). This was due to irregularities in how the working patterns of doctors could be described by doctor ID and the frequency in how these IDs were used.

Strengths and limitations

Access to Bradford EPR is a strength of this study, as it presents the opportunity to understand more about how hospital EPRs can be used for research. We attempted to access EPR data from Salford Royal Hospital but faced significant barriers which ultimately prevented access. The EPR data at Salford Royal were managed by a data team whose purpose was focused on the operational needs of the hospital, and not capable of providing data for research in the timescales needed.

Our analysis of the Bradford EPR is limited as we were unable to identify locums directly via staff employment records, which would have greatly increased the confidence of the identification and allowed for a more robust analysis of their activity. Without better locum identification, we had to rely on looking at working patterns. Locum working patterns should be characterised by short and intermittent presence in the EPR data. However, our attempts have likely also identified some part-time doctors and doctors who are permanent but create orders in the EPR less frequently. The time period we could analyse was also shortened, only 52 weeks, due to the impact of COVID-19 and how the EPR was implemented in Bradford over our observation period.

However, the analysis shows the wealth of potential information contained within EPRs if they can be better used for research. To answer our research questions on the quality and safety of locum doctors, we feel a robust linkage between EPR and electronic staff records (ESRs) would be needed.

Accessibility

Our PPI forum (made up of two men and two women) collaborated with us in the design and development of materials used to recruit patient participants to the study to ensure our materials were inclusive and accessible. We offered a range of ways to participate, including providing written accounts, telephone or video calls and individual or group interviews.

The research team

Our research team comprised of five men and two women. We are all from white backgrounds and two of our group are not from the UK. Race inequality continues to be a perennial problem in all areas of higher education, including staffing, admissions and employment. We acknowledge the lack of diversity in our team.

There was a range of experience and expertise across the research team with more senior members mentoring and supporting more junior members. Development opportunities were provided and the research team were involved in mentorship both within the university and externally through Health Services Research UK. The research team also benefitted from attending national and international conferences and presenting study findings. The team have also been supported in developing their writing skills and writing for publication.

General practitioners workforce statistics data

The general practice data show numbers and details of GPs, Nurses, Direct Patient Care and Admin/Non-Clinical staff working in General Practice in England, along with information on their practices, staff, patients and the services they provide (available from https://digital.nhs.uk/data-and-information/publications/statistical/general-and-personal-medical-services).

All data are extracted quarterly from the National Workforce Reporting System (NWRS). This is an online system in which GP practices record and update staff details. Practices are asked to ensure that the information about their staff is up to date at the end of each quarter and NWRS automatically extracts the information. Furthermore, practices may make changes to individual records as appropriate, if they find mistakes in entries for the previous quarter. The NWRS has inbuilt validations to reduce data input errors and data are extracted every quarter, at the end of March, June, September and December, but for the October and November 2020 collections the data were extracted monthly at the end of each month. However, in the October and November collections, NHS Digital identified a number of data quality issues that caused a cautious interpretation of the data.

In particular, there was a shortfall in the GP locum data and NHS Digital included some estimations on FTE GP locum figures based on the previous quarter’s data. NHS Digital published only FTE figures for all practice staff, while they developed a methodology to account for missing headcount data. 2

However, the transition to a monthly collection in the autumn of 2020 led to a more wide-reaching decrease in the quality and completeness of the data, as practice managers did not record sufficient information on infrequent locums and the publication series was reverted to quarterly collections in December 2020. For this reason, we did not use the 2020 data in our regression analyses. The monthly series was preserved for transparency purposes, and it is advised that figures from these releases are not used in any further analysis and are not included in time-series tables. 2

In terms of recorded information for locums, the following definitions are used by NHS Digital to capture locums working in general practices:

• Long term locums (covering vacancy/absence/other): Are defined as those locums who work regularly at the Practice to cover long periods of time, such as maternity leave, long-term sickness or work regularly (e.g. every Monday).

• Infrequent locums: They are defined as GP locums who do not regularly work at a practice, may cover very few sessions and are typically employed on an ad hoc basis. Infrequent locum records were added to collecting tools in December 2017. The amount and detail of the data held by practices on infrequent locums are limited and the data collected for each infrequent locum reflect this, consisting solely of the total number of hours the infrequent locum worked at the practice during the reporting period, from which a weekly average FTE is calculated. Infrequent locums may not be working at the affected practice at the exact extraction date, but their inclusion improves the quality of the data collected for the period by providing valuable information about their usage.

However, the amount and detail of the FTE data held by practices on infrequent locums are limited and the data collected for each infrequent locum reflect this, consisting solely of the total number of hours the infrequent locum worked at the practice during the reporting period, from which a weekly average FTE is calculated. The infrequent locum data are considered to be of usable quality from the December 2017 release onwards, so it is advised that locum data are comparable from December 2017 onwards.

To conclude, we consider that data on FTE from December 2017 onwards are considered more reliable than that from earlier periods, as it followed the release after December 2016 of the improved guidance which emphasised the importance of recording locum data accurately and clarified how to do so and also it was the first time that infrequent locum data were added to the time series.

English indices of deprivation

Area deprivation was measured by the latest update of the IMD (i.e. 2019) and was available at the 2011 LSOA level. The IMD is a complete and commonly used method to quantify deprivation and affluence for small areas in England. The IMD measures levels of deprivation for all 32,844 LSOAs in England on a continuous scale of deprivation and most of the indicators are based on 2012 statistics. It represents a combined score of deprivation out of a total of 37 separate indicators which are grouped into 7 domains, income, employment, education and skills, health and disability, crime, barriers to housing and services, and living environment, thus reflecting a different domain of deprivation experienced by individuals living in an area. The overall IMD is calculated as a weighted mean across the seven domains and different weights are applied to each domain and the measure is standardised on the scale of 0–100. The IMD represents every small area in England and ranks them from 1 (most deprived area) to 32,844 (least deprived area).

Spatial maps

From the Office for National Statistics (ONS) open geography portal, we obtained. Digital vector boundaries for the 2019 CCGs, generalised to 20 m and clipped to the coastline to reduce size and improve visualisation. We inputted the vector boundaries in Stata using the shp2dta command and calculated the centroid for each CCG in the British National Grid format. The centroids were converted from British National Grid easting and northing to longitude and latitude in degrees.

Sex and population estimates

From NHS Digital, we obtained monthly general practice population counts for 2018 and 2019 at the general practice level (available from https://digital.nhs.uk/data-and-information/publications/statistical/patients-registered-at-a-gp-practice). The patients registered at a GP practice database is a monthly publication showing counts of patients at NHS Regions, Regional Local Offices, STP CCG and GP practice by gender and age. Data are collected at an aggregate level from GP practices by age and gender. The data are used by many NHS and local authority organisations as a denominator for other local analyses and are considered essential to maintaining this level of information. Data are extracted as a monthly snapshot in time from the GP Payments system maintained by NHS Digital.

Health regulators ratings

Health regulators ratings were obtained from the CQC (available from www.cqc.org.uk/about-us/transparency/using-cqc-data). CQC uses a statistical surveillance tool called Intelligent Monitoring (IM) to assess performance indicators based on routinely collected and available data about providers. The IM system for general practices contains 33 performance indicators concerning patient satisfaction, the management of chronic conditions, prescribing, disease prevalence and emergency hospital admissions. Together, these indicators are intended to support inspections with information on which practices to inspect and on which aspects of care to focus on. Performance indicators are aggregated into ratings of potential risk based on expected indicator values. Each year the CQC inspects general practices, and awards each of them one of four rating levels (‘outstanding’, ‘good’, ‘requires improvement’ or ‘inadequate’) in five domains (‘safe’, ‘effective’, ‘responsive’, ‘caring’ and ‘well led’), along with an ‘overall’ rating that summarised the domain ratings. From the CQC, we obtained published inspection ratings and corresponding inspection dates for 6587 of 6909 GP practices in 2018 (i.e. 95.3% of all practices in the GP workforce data set in 2018) and 6392 of 6606 practices in 2019 (i.e. 96.8% of all practices in the GP workforce data set in 2019) that were inspected between January 2017 and December 2019. Over the inspection cycle, some practices were reinspected and their ratings updated. Only the rating from the first inspection was used in this study, as subsequent ratings were likely to have been influenced by the previous inspection process and outcomes. Data on the most recent practice inspections are freely available online.

Patient satisfaction

The GP Patient Survey is an independent survey run by Ipsos MORI (London, UK) on behalf of NHS England (available from www.gp-patient.co.uk). The survey is sent out to over 2 million people across England and focuses on patient experiences of their local GP practice and other local NHS services, and it includes questions about general health. The survey includes questions about a number of issues. For example, how easy or difficult it is to make an appointment at your practice, patient satisfaction with opening hours and the quality of care received from the GP and practice nurses. To capture patient satisfaction, we use responses for the ‘Overall, how would you describe your experience of your GP Practice’. This is the only item relevant to experience that is consistent for all years. From the overall experience questions, we extracted data on the proportion of patients in each practice who responded that their experience was Very Good; Fairly Good; Not Bad Not Good; Fairly Poor and Very Poor. 135

Quality of primary care and practice morbidity burden (Quality and Outcomes Framework)

To quantify ‘health-need’ from chronic condition registers, we used administrative data from the QOF, for the financial year 2018–9 to 2019–20 (available from https://digital.nhs.uk/data-and-information/publications/statistical/quality-and-outcomes-framework-achievement-prevalence-and-exceptions-data). The pay-for-performance scheme, which was introduced in 2004, rewards clinicians for recording and managing numerous clinical domains, mostly chronic conditions. Although it has been downsized in recent years and has been withdrawn completely in Scotland, it has been at the heart of the GP contract for many years and there is evidence that the withdrawal of incentives does not affect recording practices, not in the short term at least.

In 2019–20, 21 clinical domains were included in the scheme: atrial fibrillation, asthma, cancer, coronary heart disease, chronic kidney disease (aged 18 or over only), chronic obstructive pulmonary disease, diabetes both types (aged 17 or over), heart failure, hypertension, osteoporosis (aged 50 or over), palliative care, cardiovascular disease primary prevention, peripheral artery disease, rheumatoid arthritis (aged 16 or over), stroke and obesity (aged 16 or over), dementia, depression (aged 18 or over), epilepsy (aged 18 or over), learning disability (18 or over) and severe mental illness. We used a subset of these conditions as only 14 had clinical indicators linked to them in both data years. The data range refers to the relevant financial year (1 April–31 March) and it is matched to the relevant GP Workforce publication (i.e. QOF 2019–20 data are matched to GP Workforce data from the financial year 2019–20).

Although access to health care is free in the UK, not all people in the UK are registered with primary care and not all general practices participate in the QOF. However, over 95% of the UK population is estimated to be registered with a practice, and over 95% of registered patients attend practices participating in the QOF. A total of 6873 English practices participated in the QOF in 2019–20 and for each of these practices we extracted the denominators from the condition registers. Unfortunately, there is no way to capture multimorbidity with these data, since the QOF is organised in condition silos and a multimorbid patient would be reported under all relevant conditions in the scheme. The total numbers for each of the three groups were also aggregated at the CCG level, with 135 entities covering the whole of English Primary Care.

To capture QOF performance, we used the percentage of patients receiving the intervention for the relevant population across QOF indicators as the main measure of quality. This measure is defined as PA=∑ Ni∑ (Di+Ei) where the numerator represents the sum of all patients who have actually received the care (Ni) described in the relevant indicator (indicator i), and the denominator represents the sum of the number of patients from the appropriate disease register who are eligible to receive the care described in the relevant QOF indicator (Di) and the sum of the number of patients who have been exception reported for the relevant QOF indicator is (Ei). This measure reflects the proportion of patients for whom a practice achieved a quality indicator rather than the number of QOF points scored. This measure also gives a more accurate indication of the rate of the provision of an intervention as the denominator for this measure covers all patients to whom the indicator applies, regardless of exception status (i.e. indicator exceptions and indicator denominator).

Indicators included in the composite measure of quality of care

We extracted data indicator’s numerator, indicators and exceptions for 15 conditions including atrial fibrillation, asthma, cancer, coronary heart disease, chronic obstructive pulmonary disease, dementia, depression, diabetes, heart failure, hypertension, osteoporosis, peripheral arterial disease, rheumatoid arthritis, serious mental illness and stroke and were aggregated into a single score. These indicators have been revised many times over the years, but their underlying aim has remained the same. Furthermore, some indicators were dropped while others were revised for the duration of the QOF. If an indicator was dropped from the scheme, we calculated population achievement for each year from the remaining indicators. If an indicator was revised, we did the revision based on the time span that measurement or treatment was taken or provided (e.g. for some indicators the time span was specified for 15 months but after the revision the time span was reduced to 12 months).

FIGURE 28.

Geographical distribution of locum use in 2019. Locum use is defined as mean locum FTE as a proportion (%) of total GP FTE. Population is divided into quintiles depending on levels of locum use. A higher level of the measure in the key represents higher locum use in the locality.

National Health Service workforce statistics database

National Health Service Digital collects monthly NHS Hospital and Community Health Service (HCHS) workforce statistics for staff in NHS trusts and CCGs in England. The data are recorded within the ESR, which is a payroll and human resources system and contains staff records for NHS-employed staff in England since 2008. Data are available as headcount and FTEs and for all months from September 2009 onwards and they represent an accurate summary of the validated data extracted from the NHS ESR system. We downloaded monthly data collections available from NHS Digital from December 2017 to December 2021 (available from https://digital.nhs.uk/data-and-information/publications/statistical/nhs-workforce-statistics). For each month and each NHS trust, we calculated the average total FTE across all available doctor categories in the NHS workforce database and matched the data to NHS improvement data for each trust in each month. The database also included monthly NHS trust level turnover data which were used in the negative binomial regression.

National Health Service vacancy statistics

From NHS Digital, we obtained trust level vacancy rates for the period January 2019–December 2019 (available from https://digital.nhs.uk/data-and-information/publications/statistical/nhs-vacancies-survey). The series refers to vacancy FTE from providers which are available on a quarterly basis but recorded on a monthly basis. This monthly rate is defined as the total number of unfilled posts reported at the end of each respective reporting month. A vacancy is defined as a post that is unfilled by permanent or fixed-term staff. Some vacant posts may be filled by agency or temporary staff, but these posts are still considered vacancies. The number of vacancies is the difference between the number of reported FTE permanent or fixed-term staff in post and planned workforce levels (i.e. the total funded or budgeted establishment on a FTE basis). 50

Health regulators ratings

Health regulators ratings were obtained from the CQC. Each rating is based on the assessment of the evidence against the key lines of enquiry in the assessment framework for healthcare services and, for relevant non-specialist acute trusts, the use of resources assessment framework. The trust level ratings refer to the trust’s overall quality, based on findings under five key quality questions that CQC inspects (safe, effective, caring, responsive and well led). Where applicable, the CQC also awards a combined rating at the trust level, based on the findings of the five trust-level quality ratings plus a use of resources rating. Each year the CQC inspects NHS trusts, and awards each of them one of four rating levels (‘outstanding’, ‘good’, ‘requires improvement’ or ‘inadequate’) in five domains (‘safe’, ‘effective’, ‘responsive’, ‘caring’ and ‘well led’), along with an ‘overall’ rating that summarised the domain ratings. From the CQC, we obtained published inspection ratings and corresponding inspection dates for all 229 NHS trusts in 2019 that were inspected between January 2019 and December 2019. Over the inspection cycle, some NHS trusts were reinspected and their ratings updated. Only the rating from the first inspection was used in this study, as subsequent ratings were likely to have been influenced by the previous inspection process and outcomes. Data on the most recent practice inspections are freely available online from www.cqc.org.uk/about-us/transparency/using-cqc-data.

English indices of deprivation

Area deprivation was measured by the latest update of the IMD (i.e. 2019) and was available at the 2011 LSOA level. The IMD is a complete and commonly used method to quantify deprivation and affluence for small areas in England. The IMD measures levels of deprivation for all 32,844 LSOAs in England on a continuous scale of deprivation and most of the indicators are based on 2012 statistics. It represents a combined score of deprivation out of a total of 37 separate indicators which are grouped into 7 domains, income, employment, education and skills, health and disability, crime, barriers to housing and services, and living environment, thus reflecting a different domain of deprivation experienced by individuals living in an area. The overall IMD is calculated as a weighted mean across the seven domains and different weights are applied to each domain and the measure is standardised on the scale of 0–100. The IMD represents every small area in England and ranks them from 1 (most deprived area) to 32,844 (least deprived area). These area measures were assigned to trust based on trusts’ location. Trusts in our analysis are allocated an IMD score based on the mean IMD score of all admitted patients using data from HES admitted patient care database 2017.

Spatial maps

From the ONS open geography portal, we obtained digital vector boundaries for the 2019 STPs, generalised to 20 m and clipped to the coastline to reduce size and improve visualisation. We inputted the vector boundaries in Stata using the shp2dta command and calculated the centroid for each CCG in the British National Grid format. The centroids were converted from British National Grid easting and northing to longitude and latitude in degrees.

FIGURE 29.

Mean locum intensity (bank and agency shifts combined) at trust level in 2020. Dashed line indicates the median (0.170) locum intensity per week per FTE across 216 NHS trusts in 2020. Eight trusts were excluded from due to very small or zero denominators (i.e. permanent doctor FTE).

FIGURE 30.

Mean proportion of agency shifts at trust level in 2020. Dashed line indicates the median (62.9%) proportion of agency shifts per week across 224 NHS trusts in 2020.

FIGURE 31.

Mean proportion of unfilled shifts at trust level in 2020. Dashed line indicates the median (8.12%) unfilled shift rate per week across 224 NHS trusts in 2020.

FIGURE 32.

Mean locum intensity (bank and agency shifts combined) at trust level in 2021. Dashed line indicates the median (μ = 0.193) locum intensity per week per FTE across 208 NHS trusts in 2021. Thirteen trusts were excluded from the analyses due to very small or zero denominators (i.e. permanent doctor FTE).

FIGURE 33.

Mean proportion of agency shifts at trust level in 2021. Dashed line indicates the median (μ = 55.7%) proportion of agency shifts per week across 221 NHS trusts in 2021.

FIGURE 34.

Mean proportion of unfilled shifts at trust level in 2021. Dashed line indicates the median (μ = 12%) unfilled shift rate per week across 221 NHS trusts in 2021.

FIGURE 35.

Regional variation in locum intensity (bank and agency shifts combined) at trust level in 2020. Figure includes data from 218 NHS trusts in 2020, adjusted for substantive doctor FTE.

FIGURE 36.

Regional variation in the proportion of agency shifts at trust level in 2020. Dashed line indicates the median (μ = 55.7%) proportion of agency shifts per week across 221 NHS trusts in 2021. The thick blue line represents the IQR (25th–75th centile). The thin line represents the rest of the distribution with upper/lower adjacent values. The white dot represents the median. The distribution shape of the data is based on a kernel density estimation where wider sections of the plot represent a higher chance that members of the population of interest will take on a given value and where thinner section represents lower chance. Eleven trusts were excluded from the analyses due to non-availability of data for permanent doctor FTE.

FIGURE 37.

Regional variation in the proportion of unfilled shifts at trust level in 2020 (figure includes data from 224 NHS trusts in 2020). The thick blue line represents the IQR (25th–75th centile). The thin line represents the rest of the distribution with upper/lower adjacent values. The white dot represents the median. The distribution shape of the data is based on a kernel density estimation where wider sections of the plot represent a higher chance that members of the population of interest will take on a given value and where thinner section represents lower chance. Eleven trusts were excluded from the analyses due to non-availability of data for permanent doctor FTE.

FIGURE 38.

Regional variation in locum intensity (bank and agency shifts combined) at trust level in 2021. Figure includes data from 210 NHS trusts in 2021, adjusted for substantive doctor FTE. The thick blue line represents the IQR (25th–75th centile). The thin line represents the rest of the distribution with upper/lower adjacent values. The white dot represents the median. The distribution shape of the data is based on a kernel density estimation where wider sections of the plot represent a higher chance that members of the population of interest will take on a given value and where thinner section represents lower chance. Eleven trusts were excluded from the analyses due to non-availability of data for permanent doctor FTE.

FIGURE 39.

Regional variation in the proportion of agency shifts at trust level in 2021. Figure includes data from 221 NHS trusts in 2021.

FIGURE 40.

Regional variation in the proportion of unfilled shifts at trust level in 2021. Figure includes data from 221 NHS trusts in 2021. The thick blue line represents the IQR (25th–75th centile). The thin line represents the rest of the distribution with upper/lower adjacent values. The white dot represents the median. The distribution shape of the data is based on a kernel density estimation where wider sections of the plot represent a higher chance that members of the population of interest will take on a given value and where thinner section represents lower chance. Eleven trusts were excluded from the analyses due to non-availability of data for permanent doctor FTE.

FIGURE 41.

Spatial map of locum intensity at the STPs level, England 2019.

FIGURE 42.

Spatial map of agency shifts at the STPs level, England 2019.

FIGURE 43.

Spatial map of unfilled shifts at the STPs level, England 2019.

Focus group patient and carer schedule

Note to facilitators: Please refer back to these notes just before the group is due to meet to refresh your memory.

One member of the project PPI forum will lead the group and ask the questions, while a member of the University of Manchester research team, Jane Ferguson or Gemma Stringer, will support the interviewer. Jane or Gemma will take responsibility for the recruitment of participants, issuing participants with participant information sheets and consent forms, taking consent from focus group participants, handling the recording equipment, making field notes and transferring sound files.

Remember to take a note of the date and case study site that the focus group is related to and record on the audio.

It is important to remember that you are seeking to explore the viewpoint of the group. You should try to get everyone involved in the discussion; however, this does not mean that everyone must have the same view. You need to explore both majority and minority views.

Before the group assembles

Jane or Gemma will ensure participants have received and completed the relevant paper work before the focus group meets, for example participant information sheets and consent forms, P20 forms so they can claim payment. They will test the recording equipment to make sure it is working and that the sound is recording at an acceptable level.

Preparing to start the session

If the focus group is taking place by a platform such as Zoom or MS Teams, to encourage discussion, ask all participants to keep their microphones and cameras on wherever possible. To facilitate discussion virtually, focus groups will be kept to a maximum of four participants.

Once people are settled, check with the group whether they all know each other. If not, start by going round the group and getting everyone to introduce themselves.

Make sure that everyone is comfortable before you start and that everyone can see and hear each other. Read out the statement on confidentiality:

Opinions expressed will be treated in confidence among project staff for the purpose of understanding patient and carer experiences and views of locum doctors. All responses will remain anonymous. Please don’t refer to any individual patients or doctors by name or name a specific hospital or practices. Also, please don’t describe any serious unsafe practices that led to serious harm that haven’t already been dealt with through the proper channels as we will be obliged to report it.

Check that there are no objections to the use of the audio recorder; then Jane or Gemma will switch it on.

Introduction to the session

Start off by reiterating the purpose of the focus group:

I’m very grateful to you all for sparing time to talk about your views and experiences of locum doctors and how locum working might impact care. The purpose of this focus group is to better understand patients’ and carers experiences of locum doctors and their views of locum working. This understanding will help to find ways to improve the working arrangements for locum doctors and the quality and safety of patient care that they provide. I would like to concentrate on discussing what you think about locums before going on to discuss any experiences you might have had. There are no right or wrong opinions; this discussion is confidential and we would like you to feel comfortable saying what you really think and how you really feel.

Perceptions of locum working

What comes to mind when you think about locum doctors?

Prompt: What do you think a locum doctor is? (i.e. how would you define one).

Accessing care

Do the group think patients and carers generally know when they are seeing a locum?

Prompt: How did you know they were a locum?

1. Do you think you should be told when the doctor is a locum? If so, why?

2. Have you ever made another appointment with your regular doctor after seeing a locum? If so, why/why not?

3. Have you ever refused an appointment because it was with a locum? If so, why?

4. Have you ever not gone to the doctor because your regular doctor wasn’t available? If so, why/why not?

5. Are there particular circumstances when you would prefer to see your regular doctor or permanently employed doctor rather than a locum? Which circumstances would you be happy to see a locum, and when would you rather see your regular doctor?

Critical incident question: experiences of locum doctors

The overall aim of this research is to better understand the quality and safety of locum practice and how locum working might affect care. Can you describe a time, whether it be positive or negative, when care was provided by a locum?

Do you think there are any differences in how care is provided when permanent staff are employed in comparison to when locums are employed? Can you describe any differences and what these differences have meant for you or the person you were caring for?

Prompt: If there was a problem, what happened next? How was it dealt with?

Challenges and opportunities

Why do you think some organisations are more reliant on locums than others?

What challenges do you think a locum doctor might face?

What would a good consultation with a locum look like?

Because they might not know you, are there any key questions the locum should ask?

How do you think locums should be supported to provide safer care?

What do you think should change to make care safer for patients when locums are delivering care?

Anything else

Has taking part in this focus group changed how you think about locum doctors/changed how you might make appointments in the future?

If there is anything we haven’t covered that you would like to talk about?

Summarise the discussion and check in with participants if there are any key issues that study team should focus on.

Ending the session

Finally, thank participants for their time and reiterate that the discussion is confidential.

Jane or Gemma will submit the recording to the transcription service.

Locum doctor interview schedule.

Thank you for agreeing to take part in this project. I’d like to confirm that you have read the participant information sheet and check whether you have any questions about the study? This interview/focus group should last approximately 1 hour. We’d like to remind you that the information collected from interviews and focus groups will be kept strictly confidential, and your responses will be pseudonymised, so you will not be identifiable in any published data. Any discussions that take place during the study are confidential. However, if you were to tell us something that could put someone at risk of harm, or reveal unsafe practice that has not been reported through the usual procedures, the researcher might be professionally obliged to report the incident through the normal risk management procedures. Information that indicates harm to patients or professional misconduct will be disclosed by the research team as part of a safeguarding process, in accordance with established good research practice and with the University of Manchester’s own policy on whistleblowing and public interest disclosure. If this happens, the interview will be stopped and we will discuss with you what we intend to do.

Narrative question

Could you spend 5–10 minutes telling me about your career?

What are your plans for the future?

Prompt: Can you tell me more about your decision to become a locum doctor?

Finding work and induction

How do you find work? For example do you work with an agency? Word of mouth? How do you choose the agencies you work for?

Tell me about the places you work in? How do you choose them?

Are you typically offered induction? Are you typically offered a paid induction?

Tell me about your experiences of induction, for example are you provided with details of where things are, who people are, login details and/or more complete induction processes

What does a good induction look like?

What should be included in induction to make it safer/easier for you to work?

Experiences of locum working

What would you say the key differences were between working permanently and working as a locum?

Prompts: What are the advantages and disadvantages of locum work for you? What are the advantages and disadvantages of locum work for patients?

What have been your pre-COVID and during the pandemic experiences of being a locum?

Do patients know you are a locum? Do you tell them?

How do patients usually respond to you as a locum?

How do staff respond?

Are you usually included in multidisciplinary team (MDT) activities, meetings and/or educational activities?

What types of staff training if any are you offered? For example statutory/mandatory, for example information governance; general professional basics, for example safeguarding and basic life support; general CPD opportunities.

Who do you revalidate through? Do you feel well supported in this? Has this been impacted by the pandemic?

For a provider where you have not worked before do you undertake any preparatory work?

How are you supported in your clinical work? For example peer clinical advice and/or navigating the local system.

Prompt: Who do you usually go to if you need support?

What would happen if there was a clinical problem with your work?

What has happen if there was a general problem with your work?

Do you get to hear about complements and complaints?

What happens when you reach the end of placement? Do you get feedback?

Critical incident question

The overall aim of this research is to provide evidence on the quality and safety of medical locum practice and the implications of medical locum working for health service organisation and delivery. Can you describe a time, whether it be positive or negative, when working as a locum, or how your work was organised by the organisation you worked for, had implications for the quality and safety of care?

How does your job relate to locum working?

Tell me about what you do and how your job relates to locums? For example do you have hiring responsibilities, governance responsibilities; do you work alongside locums?

(Depending on the nature of the involvement with locum doctors) Can you describe the process of recruitment/induction/governance/integration with the team?

Perceptions of locum working

When you think about locums, what comes to mind?

Why do you think doctors choose to work as locums?

Would you say there were different ‘types’ of locum?

Have you ever worked as a locum? If so, why?

Would you ever work as a locum? Why? Why not?

How do staff respond to locums?

Why are locums needed where you work?

Why are locums needed in your organisation?

Where are they most likely to work?

Does the need for locums differ across specialties? Why?

What types of work do they usually do?

Patients and locums

How do patients respond to locum doctors?

Do patients know when they’re seeing a locum?

Do you think patients should know or need to know?

Do locums typically find out if patients complement them or complain about them?

Critical incident question

The overall aim of this research is to provide evidence on the quality and safety of medical locum practice and the implications of medical locum working for health service organisation and delivery. Can you describe a time, whether it be positive or negative, when locum working, or how a locum was engaged by your organisation, has had implications for the quality and safety of care?

Do you think there are any differences in how services are delivered when permanent staff are employed in comparison to when locums are employed? Can you describe any differences and the implications?

What happens when things go wrong or right?

If there is a problem with a locum, what happens next?

Prompt: What happens if there is a complaint or a serious untoward incident (SUI) involving a locum? What do you do in this situation?

Would you say there are any typical issues when it comes to locum working? What are they?

Does the locum typically find out if they were involved in a SUI?

Prompts: Who deals with this?

What benefits do locums bring to your organisation?

How do you capture their knowledge and share it with others?

Governance and support for locums

How does your organisation support locums?

What sort of support do you think locums should have?

Are locums included in CPD where you work?

Do locums attend MDT meetings and events?

Impact of the pandemic

How has the pandemic impacted on locum working where you work?

Initiatives

Are you aware of any policies or initiatives that are being used or developed to improve how locums are used by organisations?

What would you like to see included in any initiatives?

Anything else?

Is there anything else you’d like to add or anything that we haven’t covered?

Thank you for taking part

Additional participant quotations