An evaluation of foundation doctor training: a mixed-methods study of the impact on workforce well-being and patient care [the Evaluating the Impact of Doctors in Training (EDiT) study]

NHS policy context

The health and well-being of NHS staff has been of great interest to policy-makers in recent years, with a growing acknowledgement that good levels of health and well-being are likely to have benefits for organisations and patients. The role of organisations in contributing to the health and well-being of staff is recognised as key, with support structures aimed at improving the health of staff likely to positively influence staff retention, sickness absence, productivity and also, potentially, patient satisfaction and quality of care. 1,2

The publication of two major reports has increased the focus on health and well-being in the UK workplace and NHS organisations. 3,4 The Black review3 examined the health of the UK working-age population with a focus on the large-scale problem of sickness absence and reduced productivity (including the role of common mental health conditions). There is evidence that reduced well-being is one of the major causes of reduced productivity for individuals in work. Alongside this, a growing literature links morale and job satisfaction with health outcomes and performance. Although individuals may differ in the importance they attach to issues such as salary or level of responsibility, this review identified key job-related characteristics that influence well-being at work, such as employee autonomy and adequate social support. Good management and leadership also play a vital role in promoting well-being and improving performance. 3

Following on from the Black review,3 the Boorman review4 examined issues of health and well-being in the NHS workforce. The focus on staff well-being is explained by the continuing high rates of sickness absence in the NHS, with over 10 million sick days lost annually, equivalent to 45,000 whole-time equivalent staff,5 with over one-quarter of absences caused by stress, depression and anxiety. The NHS review of health and well-being4 found links between the well-being of staff and key performance indicators such as patient satisfaction and trust performance, with trusts with lower rates of sickness absence and turnover more likely to score highly on indicators of patient satisfaction and quality of care. The report4 recommended that organisations develop strategies and provide services to NHS staff to prevent and treat sickness, including work-related stress, anxiety and depression, and that management be assessed on their contribution to staff health and well-being.

These reports are consistent with pledges made in Lord Darzi’s 2008 report6 regarding the need for a broader commitment to health and well-being in workplaces and the recognition that the health and well-being of NHS staff was an important component of the commitment of the NHS to provide high-quality care. National Institute for Health and Care Excellence (NICE) guidance has also been produced,7 which recommends a strategic approach by employers to the well-being of staff. This includes ensuring that job design, selection, recruitment, training and appraisal promote well-being and that assessment of the well-being of employees is undertaken to identify areas for improvement.

Although the link between staff development, motivation and well-being and patient care is recognised as important,8,9 the impact of staff stress, depression and other aspects of well-being on patient care has been generally under-researched. Evidence demonstrating a link between indicators of well-being and indicators of patient safety, experience and quality of care is rare and has primarily been collected in nurse settings in the USA. There is also a lack of good-quality evidence from data collected longitudinally. 10

Training doctors in the NHS

Training and appraisal have been identified in the literature as important elements of appropriate people management, impacting on knowledge and skills, job satisfaction and well-being, which in turn may influence patient outcomes. 11 Previous studies have demonstrated relationships between the quality and extent of training and appraisal and the well-being of staff and better patient care. 12–16 Recently, postgraduate medical training has undergone changes in response to long-standing criticism of its suitability in a modern, patient-centred NHS. A report by the Chief Medical Officer17 highlighted a number of perceived problems with the job structure, working conditions and training opportunities in postgraduate medical education, with the balance between medical training and service provision weighted too heavily in favour of providing for service delivery at the expense of a well-structured and well-planned training programme for postgraduate trainees [senior house officers (SHOs)]. SHO training placements were perceived as short term and stand-alone and not part of a clearly structured training programme.

These issues called into question whether doctors were being appropriately trained to meet the demands of a modern, patient-centred NHS. 18 Postgraduate training was also criticised for failing to provide more trained specialists for a consultant-led NHS. 2 The report recommended the introduction of a new programme-based system of postgraduate training [foundation training (FT)] that would provide broad-based specialty experience and flexible training arrangements.

Foundation training

The new model of FT was piloted in 2004 and introduced nationally in 2005. The model introduced a fixed 2-year Foundation Programme to address the perceived deficiencies of the previous postgraduate training grades (pre-registration house officer and SHO). Postgraduate training was structured around a formal programme with a national curriculum and structured assessment of clinical competencies (see Appendix 2).

The first year of FT (foundation year 1 or F1) focused on developing the skills and competencies learned during undergraduate medical training. The completion of core competencies was required during F1 to achieve full registration with the General Medical Council (GMC). The second year of FT (foundation year 2 or F2) was designed to enable doctors to become functioning members of the health-care team, competent in the management of the acutely ill patient and with key skills in team working, time management and communication with both professionals and patients.

Foundation training generally consisted of 4-month placements in a variety of specialties to give postgraduate doctors sufficient experience of different areas of medicine. Modernising Medical Careers (MMC) also provided a clear structure for post FT with ‘run-through’ specialist training following on from FT. 19

As well as introducing new structural arrangements, FT changed the delivery of training within placements. For the first time doctors were explicitly required to demonstrate competency to practise through the completion of a range of specific assessments (see Appendix 2). These assessments were based on a new Foundation Curriculum. 20 The Foundation Programme also required the designation of educational supervisors (ESs) and clinical supervisors (CSs), charged with ensuring that foundation doctors were meeting their educational and training goals.

The well-publicised problems with selection processes in the early stages of MMC reform led to an inquiry into MMC, which also examined FT. 21 The inquiry highlighted a number of areas of concern with the FT model, including the insufficient breadth of clinical experience in foundation placements; a lack of flexibility in programmes; and the length of programme placements. This contributed to a perception that foundation doctors were not reaching appropriate levels of clinical responsibility compared with their SHO predecessors. The report also recommended greater clarity about expectations of the role of F2 doctors in the health-care team and what their service contribution should be.

A more recent evaluation of the Foundation Programme reported improvements including a well-defined curriculum, trainees exposed to a wider range of medical specialties and implementation of a comprehensive programme of trainee assessment. 22 However, the report stated that the programme still lacked an articulated purpose, found that there was confusion over the role of the F2 doctor and questioned the ability of placements to accurately reflect the current and future needs of the NHS. Further, the assessment process placed excessive loads on ESs and there were safety and quality issues in the learning environment.

Reduced hours of working and the European Working Time Directive

As well as changes to the structure of postgraduate training, there has been a major change to the conditions of work for postgraduate foundation doctors. The New Deal for junior doctors, published in 1991,23 highlighted the need for improved working conditions for this workforce group primarily focused on working hours. It was widely acknowledged that excessive hours of practice amongst foundation doctors was a risk to patients. In 2003 the working week was limited to 56 hours and the European Working Time Directive (EWTD)24 further limited the hours that medical staff could work to a maximum of 48 hours. This was implemented in stages with the 48-hour limit enforced by law in 2012.

Health service staff motivation and well-being

High levels of stress among health-care professionals has been recognised as a problem for some time. A review of doctors’ stress levels found that between 28% and 30% of doctors had above-threshold levels of stress compared with about 18% of the general population. 16 A survey of over 11,000 NHS staff5 found staff reporting high levels of stress and that they did not consider that senior managers took a positive interest in their health. Some studies have also reported high levels of depression amongst doctors. 25,26

Studies have also reported higher levels of stress among doctors (both consultants and junior doctors) working in emergency medicine (EM), with above-threshold scores for around half of the respondents from each group. 27,28 These levels of stress are again higher than might be expected among the general population. 29 However, the impact of stress, depression and well-being on patient care has been generally under-researched. 10

Foundation doctor well-being

There is a limited literature examining the well-being of doctors in training. One study looked at levels of psychological distress in SHOs working in the ED. 27 SHOs were selected from six EDs in London and received questionnaires to measure psychological outcomes and coping strategies. Over half of respondents scored above the threshold for psychological distress on the General Health Questionnaire. 30 Higher levels of anxiety and depression were related to a venting style of coping (such as expressing negative feelings) whereas lower scores for these outcomes were associated with a more active coping style (such as devising strategies to cope with stressors). Another study followed junior doctors for 3 years after their graduation from medical school finding that first-year postgraduates reported levels of depression of 29%, dropping to 10% by their third postgraduate year. 26

Foundation doctors and quality of care

Studies evaluating the impact of foundation doctors on the quality of patient care have evaluated the following outcomes: (1) numbers of patients seen; (2) reattendance of patients in the ED; and (3) confidence and competence in managing conditions.

A prospective observational study compared the productivity (numbers of patients seen) of F2 doctors and SHOs working across two EDs in Scotland. 31 Both groups demonstrated a significant rise in productivity between the first and last months of their attachments. There were no significant differences in productivity between the two groups of doctors over the 12-month study period. However, there were concerns about a reduction in the percentage of patients seen by junior doctors overall and an increased need for senior review of patients. Further analysis by Armstrong et al. 32 investigated the number of patients seen by all junior doctors (SHO/F2) over a 3-year period. The study found a 4% decrease in the number of patients seen by junior doctors in this period. In addition, there was a significant reduction of 16.6% in the number of patients seen per hour (an indicator of work rate).

A study at an inner-city ED in England also found no significant differences in the mean number of patients seen by F2 doctors and SHOs over a 12-month period. 33 Individual doctor performance had a greater influence on the number of patients seen than type of doctor (either F2 or SHO), with a small number of F2 doctors seeing considerably more patients than their SHO colleagues.

A study by Whiticar et al. 34 compared reattendance rates of patients to the ED over a month in 2006 by grade of doctor assessing the first presentation. Junior doctors (SHOs and F2 doctors) had higher reattendance rates (2.83% vs. 2.32%; p = 0.52) than middle-grade doctors and nurse practitioners (although the result was not statistically significant).

Croft and Mason35 assessed levels of confidence in foundation doctors’ management of common minor clinical presentations in an inner-city ED. Foundation doctors’ confidence in treating minor injury patients was identified as a problem, and a lack of exposure to minor injuries during daytime hours was cited by doctors as a possible cause.

A further study36 evaluated junior doctors’ experience in performing practical procedures in an ED. Two cohorts were measured: trainee doctors in the ED in June 2005 and June 2006. The study found that doctors in the later cohort reported significantly less experience in each procedure.

One study37 measured SHO and pre-registration SHO knowledge of basic acute care in 12 topics. A total of 185 junior doctors from six UK hospitals were included in the study. This study found that knowledge was poor across a range of basic acute care topics and that junior doctors were poorly prepared to identify and treat critically ill patients.

Overall, these studies raise questions with regard to foundation doctors’ confidence and performance in the ED which require further investigation.

NHS staff motivation, well-being and patient care

There is limited evidence of a direct association between factors that affect performance and outcomes in health care, which would be important to take into account when studying a changing workforce. In one study,15 which sought associations between organisational practice and clinical outcomes, it was possible to demonstrate a linkage between good human resources practice (such as appraisal and training) and effective teamwork and reductions in measures of patient mortality. A further study in a non-health care setting demonstrated that organisational climate (e.g. skill development, concern for employee welfare) was significantly associated with productivity and profitability, and that the relationship was mediated by employee job satisfaction. 38 There is an increasing literature on links between patient safety and organisational culture and climate, with a range of tools and interview methods proposed. 39

The link between staff development, motivation and well-being and the influence of these factors on patient care is recognised as important. 8,9 In one review of the literature,10 a significant linear effect was found between levels of nurse stress and burn-out and patient outcomes (patient satisfaction, medication errors and patient falls). However, the cross-sectional designs of the studies and lack of control of confounding variables (such as doctor sickness absence)40 limits the usefulness of these findings. 10

Few studies have evaluated the consequences of well-being for foundation doctors in terms of confidence, competence or patient outcomes. A study of SHOs working in 27 hospitals evaluated the relationship between psychological distress and confidence in performing clinical tasks. 41 The questionnaire was administered four times during the 6-month rotation. Overall, confidence levels in carrying out a range of practical and clinical tasks (recorded on a visual analogue scale) increased significantly between the first and fourth months of the SHO training rotation. SHOs with higher psychological distress scores at the end of months 1 and 4 had lower confidence scores. Factors associated with greater psychological distress were organisational, such as workload, certain clinical presentations and consultation issues such as communication.

Phase 1

• To conduct a national and regional consultation exercise with training stakeholders to:

  • describe the national strategic view of the aims of delivering FT, with a particular focus on the role of training in supporting the well-being of doctors

  • assess how the national view is implemented on a regional basis through the postgraduate deaneries and identify any regional variation to implementation within the specialty of EM

  • undertake a scoping exercise to identify factors contributing to the well-being of F2 doctors in training within up to four EDs to develop measures to inform a quantitative evaluation of foundation doctors in phase 2 of this study.

Phase 2

• To undertake a longitudinal study using a structured survey to assess F2 doctors in terms of their well-being, motivation, confidence and competence at four time points over a 12-month period.

• To conduct a survey at four time points (at the end of F1 and then after each F2 placement) to assess the level of and change in F2 doctor well-being, motivation, confidence and competence. One of these placements will be in EM and the impact of this placement can be assessed in relation to the study outcomes.

• Assess patient safety and quality of care by F2 doctors by reviewing the clinical records of patients receiving emergency care from F2 doctors and evaluating routine ED data to link workload and mean time with the patient for each of the participating F2 doctors.

We will examine the findings from phase 2 to:

• evaluate whether there is a relationship between F2 doctor well-being and motivation and patient care

• identify best-practice models of F2 doctor training, which might be generalised and implemented across the NHS to promote a healthy and productive foundation doctor workforce

• provide a starting point for the development of a tool that can be used to monitor the well-being, motivation and training of doctors in EM and other specialties.

Ethical and governance arrangements

Ethical approval for phase 1 was received in May 2009 (ref.: 09/H1307/27).

Approvals from non-NHS organisations and research governance approvals from participating NHS trusts were obtained between June and November 2009.

Consultation exercise

To understand the strategic aims, national structure and implementation of FT, national and regional stakeholders from key postgraduate educational organisations involved in the delivery and implementation of FT were interviewed by telephone or videoconference. A semistructured interview schedule was designed around the aims and objectives of the study, with a particular focus on identifying potential variation in the implementation of FT and provision within training of an appreciation of the well-being and motivation of F2 doctors (see Appendices 3 and 4 for interview schedules).

A letter of invitation and information sheet about the Evaluating the Impact of Doctors in Training (EDiT) study were given to interviewees in advance, along with the interview questions. Written, informed consent was received from each participant before the interview (see Appendices 6–8 for the letter of invitation, information sheet and consent form respectively).

Scoping study

A scoping study involving interviews with TLs and focus groups with F2 doctors was carried out in four EDs to evaluate the impact of training in the ED on both F2 doctors and other health-care staff. The four EDs were selected from 15 EDs recruited for the phase 2 longitudinal study. There was a particular focus on well-being and motivation of F2 doctors in order to develop measures to inform the quantitative evaluation in the phase 2 longitudinal study.

Interviews took place with TLs in EDs. A semistructured interview schedule was designed to assess the training role of doctors, the impact of training on staff, the well-being of training doctors and the quality of care (see Appendix 5). An Information sheet and interview questions were given to interviewees in advance (see Appendix 7). Written, informed consent was received from each participant before each interview.

Focus groups were held with four groups of foundation doctors in their ED placement (mainly F2 doctors, although some F1 doctors were present in two groups). A semistructured schedule (see Appendix 9) was also designed for the focus groups, which included issues around well-being, confidence, competence and ED training experiences. Written, informed consent was received from each individual before the focus groups.

Data analysis (consultation exercise and scoping study)

Three researchers were involved in the analysis of interview and focus group data in order to gain multiple perspectives and insights into the data collected and ensure inter-rater reliability. Interviews and focus groups were not recorded to maintain anonymity of the participants; however, data collected from all participants in the consultation exercise and scoping study comprised thematic accounts and reflections from each of the researchers,42 the content of which was validated with the participants to gain a full understanding of meaning. 43

One researcher produced templates for the four participant groups, aggregating themes at the group level for (1) national PESs, (2) regional PESs, (3) emergency department training leads (EDTLs) and (4) F2 doctors in the ED primarily by using the interview/focus group questions as a priori coding. 44,45 Each template was then examined by the other two researchers, checking for inter-rater reliability to reduce any potential bias. Agreement was high (92%), with issues of terminology being the main areas of correction; any areas of misunderstanding or possible bias were corrected on each template.

Individual templates derived from the interviews and focus groups were also reviewed by the three researchers for salience (agreement of themes across stakeholders) and difference (individual perspectives that add value to the enquiry) and key overall themes were derived for each group of participants. A summary template was then produced to bring together the similarities and differences between key themes across the groups (Table 1).

Sample achieved

National stakeholder interview findings

In the following sections the numbers in parentheses indicate the numbers of participants agreeing with a particular statement. Statements without a number are the comments of one participant only.

Regional stakeholder interview findings

The seven regional PESs in our study were involved in programmes that offered places for between 200 and 550 postgraduate doctors. The smallest programme started in 2008 and worked with nine trusts, expecting to expand in the following year. The largest programme worked with 24 trusts and 19 community placements.

Training lead interview findings

The interviews with the eight EDTLs elicited information about the implementation of FT in the ED, including the role of F2 doctors in the ED, changes in training provision, the evaluation of the well-being and motivation of F2 doctors, preparation for the delivery of patient care, and assessments.

Focus groups with foundation year 2 doctors

Focus groups were held with F2 doctors who were on placements in four of the EDs participating in this study (25 F2 doctors in total). The focus groups discussed issues of confidence and competence, anxiety and general experience in EM.

Aims

• To measure levels of (and change in) F2 doctor well-being, motivation, confidence and competence over the period of their F2 training.

• To measure the impact of placements in EDs on F2 doctor well-being, motivation, confidence and competence.

Objectives

• To undertake a 12-month longitudinal study to assess F2 doctors’ experiences of working in terms of their well-being, motivation, professional identity, confidence and competence.

• To implement a survey at four time points during the longitudinal study (at the end of the F1 period and then after each of the F2 placements). One of these F2 placements will be in the ED and the impact of this placement on F2 doctors’ experiences of working will be assessed.

Ethical and governance arrangements

Ethical approval for the phase 2 longitudinal study was received in September 2009 (ref.: 09/H1300/80).

Approvals from non-NHS organisations and research governance approvals from participating NHS trusts were obtained between November 2009 and June 2010.

Survey design

We used a ‘closed’ online survey design because in the phase 1 focus groups F2 doctors expressed a preference for online surveys rather than paper versions. F2 doctors were eligible for this study if they had a placement in the ED during their F2 rotations. The online survey was accessible via a portal on the EDiT study website,48 with eligible doctors sent a link taking them to the appropriate part of the website to access the survey (see Appendix 13 for a screenshot of the website). The EDiT website provided information about the study, such as news and updates, and also included an interactive element (a medical casebook quiz). The website was designed to be informative and attractive to potential participants and thus increase recruitment and retention over the period of the survey. Visitors to the website could examine the information provided without being obliged to complete the survey itself.

Participants

As the focus of the study was examining the impact of placements in the ED on the FT experience of F2 doctors, the sampling frame (eligible doctors) for the survey was all F2 doctors in England who had a placement in the ED as part of their F2 training year. Eligible doctors were identified following discussions with PMDs and EDs.

Development of survey measures

In the first instance a pilot questionnaire was designed to measure a range of work-related outcomes and job-related characteristics by adapting well-validated scales. The questionnaire was piloted with a small sample of F2 doctors and following feedback some minor amendments were made to the final questionnaire (see Pilot study of questionnaire). The pilot questionnaire and the final questionnaire are provided in Appendices 17 and 18 respectively.

The content of the final questionnaire is detailed in the following sections.

Sample size

We planned to include a sample size of 210 F2 doctors from across EDs in England. We hypothesised that the job-related well-being of F2 doctors, measured using the six-item anxiety–contentment and six-item depression–enthusiasm dimensions of the Warr54 job-related well-being scale would change over the 12-month study period. Assuming a correlation of r = 0.4 between the baseline and 12-month test scores, a sample size of 210 cases was required to detect a 0.2-unit change in anxiety–contentment at the p < 0.05 level of significance with 95% power.

We envisaged an average of 24 F2 doctors in each participating ED during the 12-month period of the study. We assumed a 20% non-response rate within each ED and a further 20% loss of paired cases between pre- and post-placement surveys across the sample, giving us a final required sample of around 14 F2 doctors per ED. To satisfy the power analysis above we aimed to recruit doctors from at least 15 EDs.

Pilot study of the questionnaire

We conducted a pilot study in June 2010, primarily to test the reliability of the survey measures, particularly those developed by the research team, but also to examine:

• the usability of the questionnaire (e.g. issues of length)

• the feasibility of using FSAs to forward e-mails to F2 doctors on our behalf, with links to the online survey

• the functionality of the online web-based platform for the questionnaire.

Survey implementation

Following the pilot study the final questionnaire was confirmed and administered identically at the four time points (T1–T4), with the exception of background information, which was collected at T1, and the development of professional knowledge scale, which was added at T4.

The survey utilised survey-design software (SurveyGizmo, Boulder, CO, USA) to enable online usage. The survey measures were grouped together in four sections, with the demographic data collected only the first time that a participant received the survey. The initial survey consisted of 31 items presented over eight pages. All items presented on one page had to be completed to proceed to the next page of the survey. A ‘back’ button allowed participants to go back through the survey and amend answers if they wished to do so.

Cohort sampling points

We aimed to administer the survey measures at up to four time points (T1–T4), capturing F2 doctors’ working experiences in four Foundation Placements, the final F1 placement and the subsequent three F2 placements (Table 3). Doctors were recruited initially at T1, although a small number did not join the study and respond to the survey until T2.

Analysis

Using longitudinal multilevel modelling, the pattern of change in mean sample scores on each of our survey measures was assessed over the duration of the study. After the initial measurement of mean scores at the end of F1 (see Table 3), variation in mean sample scores on the survey measures, by time of placement in the ED, was also assessed by categorising each member of the sample as belonging to one of three groups, depending on whether they had completed their second-year placement in the ED as their first, second or final F2 placement (Table 4). Grouping the sample in this way allowed the measurement of the impact that a placement in the ED had on doctors’ role characteristics and work-related outcomes.

Response rate

Overall, 217/654 eligible F2 doctors completed/partially completed the questionnaire for at least one of the four time points, a response rate of 33.2%. Table 5 details the number (%) responding at each of the four time points.

In total, 87 (40.1%) doctors completed the survey at all four time points, 26 (12.0%) completed the survey at three time points, 56 (25.8%) completed the survey at two time points and 48 (22.1%) completed the survey at one time point.

Sample characteristics

The mean age of our study doctors was 27 years, 58.2% were female and they had been qualified as a doctor for a mean of 1.5 years. A total of 68.7% were white British, 8.5% were black, 10.9% were South Asian, 6.0% were East Asian and 6.0% were of mixed race. UK-qualified (at undergraduate level) F2 doctors made up 83% of the sample.

Response by trust

Details of the number of F2 doctors recruited from each participating deanery and ED are provided in Appendix 19 (see Table 21).

Emergency department rotation groups

As detailed in the analysis section, we classified participants into three ED placement groups depending on the timing of their F2 placement in the ED (whether carried out as rotation one, two or three) (Table 6).

In total, 26 participants (12.0%) also reported completing a placement in the ED as part of their F1 training. We analysed this subgroup to see whether they differed in any way from those without previous F1 ED experience.

There were no statistically significant differences between the three ED groups in terms of age, sex, length of time qualified as a doctor and scores on either the trait personality measures or the coping strategy measures (see Appendix 19, Tables 22 and 23).

Work-related outcomes

Confidence in managing common conditions

We measured levels of and change in mean sample scores for:

• confidence in managing individual acute common conditions

• overall confidence in managing acute common conditions (calculated across the 23 conditions).

Confidence was measured from 1 (lowest level of confidence) to 9 (highest level of confidence). Details of the 23 conditions are provided in Box 1.

Mean confidence scores for each of the 23 acute conditions varied significantly across the four time points, with a statistically significant increase in confidence for managing each condition between T1 and T4 (see Appendix 19, Table 24).

Mean overall confidence (across the 23 conditions) varied significantly across the four time points (F_3,309 = 86.0, p < 0.005), with an increase in mean overall confidence found over the course of F2 training (Table 7).

TABLE 7 - Overall confidence over time in managing patients with different illnesses/conditions

SD, standard deviation.

ED group	Time point of response
	1		2		3		4
	Mean	SD	Mean	SD	Mean	SD	Mean	SD
1	5.96	0.96	6.80	0.78	6.80	0.73	7.10	0.78
2	6.25	0.80	6.30	0.86	7.24	0.62	7.33	0.49
3	6.10	0.90	6.41	0.87	6.63	0.89	7.36	0.71

Box 2 provides a quote from a free-text comment made by a F2 doctor on the questionnaire.

BOX 2 - Quote from a free-text comment made on the questionnaire

I really enjoyed this year; it was hard work but it helped me move on from stress of all the practical things to learn the bigger picture of patient-centred care and being part of a team.

T4 participant

Impact of placement in the emergency department

Participants were divided into three groups depending on the rotation in which they had completed their second-year placement in the ED (see Table 4).

For 14 of the 23 conditions (shortness of breath, collapse – unknown cause, acute mental health problem, elderly fall, chest pain, back pain, abdominal pain, acute allergic reaction, acute stroke, overdose – paracetamol, diabetic ketoacidosis, acute confusion, seizure and rectal bleeding) there was a statistically significant interaction between change in confidence and group (i.e. time of placement in the ED), with the biggest increase in mean confidence for each group of doctors associated with the placement in the ED compared with other placements (see Appendix 19, Table 24 for mean scores over time by condition and ED group).

There was a significant interaction between change in confidence and group (i.e. time of placement in the ED) when predicting mean overall confidence across the 23 conditions (F_6,309 = 9.3, p < 0.005), with the biggest increase in confidence for each group of doctors associated with their placement in the ED compared with other placements (Figure 1).

FIGURE 1.

Interaction between time and time of ED placement and mean overall confidence in managing patients with different conditions.

Box 3 provides some quotes from the free-text comments made by F2 doctors on the questionnaires.

BOX 3 - Quotes from free-text comments made on the questionnaires

A&E was a very good clinical experience, and I am glad that I had the opportunity to undertake this placement.

T4 participant

ED is an excellent training ground for F2s for any future field, given the sheer volume of patients, good support and regular teaching.

T4 participant

Experience in performing practical procedures

We assessed the levels of and change in mean sample scores for:

• experience of doctors in performing five common practical procedures (defibrillation, arterial blood gas analysis, suturing, ECG interpretation and radiograph interpretation)

• overall experience across the five common practical procedures.

There was a statistically significant increase in reported mean experience over time for defibrillation (F_3,344 = 23.3, p < 0.005), suturing (F_3,323 = 61.4, p < 0.005), ECG interpretation (F_3,315 = 27.2, p < 0.005) and radiograph interpretation (F_3,362 = 19.1, p < 0.005). Mean scores across the four time points for overall confidence are shown in Table 8.

TABLE 8 - Overall experience over time in managing patients with different illnesses/conditions

ED group	Time point of response
	1		2		3		4
	Mean	SD	Mean	SD	Mean	SD	Mean	SD
1	5.96	0.96	6.87	0.96	6.35	1.55	6.99	1.05
2	5.96	0.95	6.29	1.19	7.28	0.81	7.44	0.72
3	5.72	1.20	5.77	1.46	7.18	0.76	6.02	1.40

The coefficient alpha for mean overall experience in performing practical procedures ranged from 0.60 (T4) to 0.80 (T3). Mean overall experience (across the five different practical procedures) varied significantly across the four time points (F_3,367 = 61.1, p < 0.005), with a statistically significant increase in experience over the course of F2.

Impact of placement in the emergency department

There was a statistically significant interaction between change over time and group (i.e. time of placement in ED) when predicting the improvement in mean experience scores for arterial blood gas analysis, suturing, ECG interpretation and radiograph interpretation. For each of these techniques the biggest increase in mean experience was associated with the placement in the ED compared with other placements (see Appendix 19, Table 25 for mean scores over time for each procedure by ED group).

There was a significant interaction between change over time and group (i.e. time of placement in the ED) when predicting mean overall experience across the five practical procedures (F_6,367 = 10.3, p < 0.005), with the biggest increase in perceived experience reported by each group of doctors associated with their placement in the ED compared with other placements (Figure 2).

FIGURE 2.

Interaction between time and time of ED placement and mean overall experience in performing practical procedures.

There were many positive free-text comments made on the questionnaires about the learning experience of working in the ED (Box 4).

BOX 4 - Quotes from free-text comments made on the questionnaires

I learned an incredible amount clinically in knowledge and practical skills.

T2 participant

Experience was rewarding as I gained skills and experiences that I could not have got elsewhere.

T2 participant

Going forward, I feel more confident clinically for the experience.

T3 participant

A&E was a very good clinical experience, and I am glad that I had the opportunity to undertake this placement.

T4 participant

Well-being

Anxiety and depression

Levels of and change in mean sample scores for anxiety and depression were calculated. Scores for each ranged from 1 (lowest level) to 5 (highest level). The coefficient alpha for anxiety ranged from 0.85 (T4) to 0.91 (T2) and for depression ranged from 0.87 (T1) to 0.93 (T3).

There were no statistically significant variations in mean sample scores for either anxiety or depression across the four time points. The mean scores for anxiety and depression over time are shown in Appendix 19 (see Table 26).

Impact of placement in the emergency department

There was a significant interaction between time of placement in the ED and mean anxiety (F_6,390 =7.9, p < 0.005) with all three groups of doctors showing the biggest rise in mean anxiety during their placement in the ED (Figure 3). There was no difference between ED placements and other placements for scores on depression. Mean scores for each ED group by time point can be found in Appendix 19 (see Table 26).

FIGURE 3.

Interaction between time and time of ED placement and anxiety.

Several free-text comments were made about the nature of the ED experience and the impact on participants’ well-being, with examples provided in Box 5.

BOX 5 - Quotes from free-text comments made on the questionnaires

ED was stressful at the beginning.

T2 participant

A&E is a stressful job but the most rewarding I have done so far.

T3 participant

A&E is stressful and always very busy.

T3 participant

Comparison with other occupational groups

We compared F2 doctors’ mean levels of anxiety [mean 2.10, standard deviation (SD) 0.61] and depression (mean 1.68, SD.59) at each of the time points with those of a comparison group of 2909 professional and technical workers (mean anxiety 2.4, SD 0.83; mean depression 1.88, SD 0.87). F2 doctors reported significantly fewer anxious and depressive symptoms than the comparison group at T1, T3 and T4 and similar levels of symptoms as the comparison group at T2.

Job satisfaction

Levels of and change in mean sample scores for job satisfaction were calculated. As well as an overall mean job satisfaction score, the 15-item overall score was divided into two separate scale mean scores for intrinsic job satisfaction and extrinsic job satisfaction. All items on the job satisfaction scale were scored from 1 (lowest level) to 7 (highest level).

The coefficient alpha for mean overall job satisfaction ranged from 0.86 (T1) to 0.90 (T2), which compares favourably with the reliability of 0.87 obtained in a study of 20,694 NHS trust employees. 59 The coefficient alpha for mean intrinsic job satisfaction ranged from 0.80 (T1) to 0.86 (T2) and for mean extrinsic job satisfaction ranged from 0.72 (T1) to 0.78 (T2).

All measures of mean job satisfaction increased over the time period of the study; this was significant for overall job satisfaction (F_3,397 = 7.2, p < 0.005) and intrinsic job satisfaction (F_4,410 = 9.5, p < 0.05).

Mean scores for job satisfaction at each time point are shown in Appendix 19 (see Table 27).

Impact of placement in the emergency department

There was no statistically significant interaction between time of placement in the ED and mean sample scores for overall job satisfaction or intrinsic job satisfaction.

There was a statistically significant interaction between time of placement in the ED and mean extrinsic job satisfaction score (F_6,380 = 3.72, p < 0.005), with two of the three ED groups reporting a decline in mean extrinsic job satisfaction immediately after their placement in the ED (Figure 4). Mean job satisfaction scores for each ED group at each time point are shown in Appendix 19 (see Table 27).

FIGURE 4.

Interaction between time and time of placement in the ED and mean extrinsic job satisfaction.

Several free-text comments were made about levels of satisfaction with working in the ED, with examples given in Box 6.

BOX 6 - Quotes from free-text comments made on the questionnaires

Thoroughly enjoyed.

T2 participant

I would thoroughly recommend this job to any FY2. Although very demanding it was very rewarding.

T2 participant

I thoroughly enjoyed my second placement and learned a great deal.

T3 participant

Thoroughly enjoyed my A&E placement, really grew up as a doctor.

T3 participant

However, participants were not happy with the shifts and working hours (extrinsic aspects of the role) (Box 7).

BOX 7 - Quotes from free-text comments made on the questionnaires

Objected to working 12 out of 16 weekends when others worked eight or nine.

T2 participant

Pretty antisocial shifts of mostly afternoon and evenings to cover service needs.

T3 participant

Excessive working hours.

T3 participant

Rota was tough and life was put on hold.

T3 participant

Shift work was difficult and work life balance was almost non existent.

T4 participant

Comparison with other occupational groups

We compared F2 doctors’ mean levels of job satisfaction (overall mean 5.13, SD 0.55) at each of the time points with those of a comparison group of 2616 doctors and NHS staff (mean 4.55, SD 0.80). F2 doctors were significantly more satisfied than the comparison group of NHS staff at all time points.

Motivation (effort)

Levels of and change in mean sample scores for motivation were calculated, with scores ranging from 1 (lowest level) to 5 (highest level). The coefficient alpha for motivation ranged from 0.83 (T1) to 0.91 (T3).

There was no statistically significant variation in mean motivation scores over the four time points of the survey (see Appendix 19, Table 28).

Impact of placement in the emergency department

There was a statistically significant interaction between time of placement in the ED and mean score for motivation (F_6,384 = 4.1, p < 0.005), with all three groups of doctors showing the biggest rise in motivation during their placement in the ED compared with other placements (Figure 5). Mean motivation scores for each ED group at each time point are shown in Appendix 19 (see Table 28).

FIGURE 5.

Interaction between time and time of placement in the ED and motivation.

Several free-text comments were made about the effort that F2 doctors expended, with examples given in Box 8.

BOX 8 - Quotes from free-text comments made on the questionnaires

I worked very hard to fully clerk patients.

T3 participant

The pressure of 4 hour target made it difficult to feel you had completed everything before patient left dept.

T3 participant

Job-related characteristics

Role characteristics

We measured levels of and changes in mean sample scores for work demands, role clarity and task feedback, with scores ranging from 1 (lowest level) to 5 (highest level). We also measured mean hours worked in a typical working week and shift changed at short notice in the last 4 weeks (‘yes’/‘no’).

The coefficient alpha for work demands ranged from 0.81 (T4) to 0.88 (T2), which compares well with the reliability of 0.80 quoted by Dwyer and Ganster. 80 The coefficient alpha for task feedback ranged from 0.82 (T1) to 0.90 (T3, T4). This compares well with the reliabilities of 0.80–0.87 with health-care samples in the study by Haynes et al. 81 The coefficient alpha for role clarity ranged from 0.85 (T1, T4) to 0.87 (T2). This compares well with the reliabilities of 0.76–0.90 with health-care samples in the study by Szilagyi et al. 82

There was evidence of statistically significant variation across the four time points in mean scores for both work demands (F_3,393 = 10, p < 0.005) and role clarity (F_3,389 = 6.5, p < 0.005).

There was no statistically significant variation in mean hours worked or shifts changed at short notice between placements. The mean scores at each of the four time points and overall for work demands, role clarity, task feedback, hours worked in a typical working week and shift changed at short notice in the last 4 weeks are provided in Appendix 19 (see Tables 30 and 31).

Impact of placement in the emergency department

There was a statistically significant interaction between time of placement in the ED and task feedback, with each of the doctor groups reporting the biggest decline in mean task feedback score after their placement in the ED (F_6,371 = 3.24, p < 0.005) (Figure 6). Box 9 shows some free-text quotes from comments made by F2 doctors on the questionnaires.

FIGURE 6.

Interaction between time and time of placement in the ED and task feedback.

BOX 9 - Quotes from free-text comments made on the questionnaires

I tried to learn but lack of feedback on whether management of cases were satisfactory or not was the biggest problem.

T4 participant

A wealth of experience; did not get much feedback on what we were doing – I’m not sure the job can ever offer this.

T3 participant

There was also a significant interaction between time of placement in the ED and mean hours worked in a typical working week, with the biggest increase apparent for each group of doctors associated with their ED placement (F_6,416 = 7.5, p < 0.005) (Figure 7).

FIGURE 7.

Interaction between time and time of placement in the ED and hours worked.

There were many free-text comments made on the questionnaires about working hours and shift patterns, with examples given in Box 10.

BOX 10 - Quotes from free-text comments made on the questionnaires

Long hours and close to EWTD limit of less 70 hours per week; but not taking ruling at its value of avoiding exhausted staff.

T2 participant

Pretty antisocial shifts of mostly afternoon and evenings to cover service needs.

T3 participant

Excessive working hours.

T3 participant

Shift work was difficult and work life balance was almost non existent.

T4 participant

Relational characteristics

Levels of and change in mean sample scores for support from CSs and colleagues were assessed on two scales. Extent of support on each scale was scored from 1 (‘to a very little extent’) to 5 (‘to a very great extent’).

The coefficient alpha for CSs support ranged from 0.90 (T1) to 0.94 (T2, T3, T4). The coefficient alpha for colleague support ranged from 0.90 (T1) to 0.94 (T2, T3). This compares well with quoted reliabilities for each relational scale of 0.90–0.92 with health-care samples. 81

There was no statistically significant variation in mean sample scores for reported support from either CSs or colleagues across the four placement periods (see Appendix 19, Table 32).

Impact of placement in the emergency department

There was a statistically significant interaction between time of placement in the ED and mean support from colleagues (F_6,390 = 3.4, p < 0.005). This was mainly caused by a large increase in mean support from colleagues reported by one ED group immediately after their ED placement (Figure 8).

FIGURE 8.

Interaction between time and time of placement in the ED and colleague support.

There were several free-text comments made about support during the ED placement, but mostly from staff and supervisors, with examples given in Box 11.

BOX 11 - Quotes from free-text comments made on the questionnaires

Fantastically supportive staff.

T2 participant

Medical staff were approachable and supportive.

T2 participant

Highly trained nurses who encouraged junior doctors a lot.

T2 participant

I always felt there was someone to ask if I needed to check a decision I made.

T3 participant

Well supervised.

T3 participant

Good support and regular teaching.

T4 participant

Emergency department-level comparison of role characteristics and work-related outcomes

We also investigated whether non-trivial variation existed in respondents’ role characteristics and work-related outcome measures across the participating EDs. This analysis was cross-sectional in nature as respondents’ role characteristics and work-related outcomes relative to the specific ED in which they did their ED training were typically measured just once, immediately after their ED placement (i.e. respondents did not typically do their other F2 placements in the same department).

The majority of the variables showed little or no variance that could be explained by the ED in which the F2 doctors were working during their ED placement. The percentage variance in scores for each of the role characteristics and work-related outcomes that existed at the ED level exceeded 10% for CS support, number of hours of one-to-one contact with supervisor, number of hours of close working contact with supervisor and job satisfaction.

Summary of results

1. F2 doctors reported a significant increase in confidence in managing 23 common acute conditions over the course of F2 training, with the biggest increase in confidence arising immediately after their ED placement.

2. Experience managing common procedures improved significantly over the four time points (with competence in interpreting arterial blood gases improving non-significantly between T1 and T4), with the biggest increase in competence in all five procedures arising immediately after the ED placement.

3. In terms of well-being, F2 doctors showed similar (T2 comparison) or better (T1, T3, T4) levels of anxiety and depression than a comparison group of professional and technical workers. Levels of anxiety and depression did not vary significantly over time but the biggest rise in anxiety was associated with an ED placement.

4. Job satisfaction is another aspect of well-being and F2 doctors reported significantly higher levels of job satisfaction than a large comparison group of doctors and other NHS staff. A significant improvement in overall and intrinsic job satisfaction was seen during F2. However, in two of the three groups studied the ED placement was associated with significantly lower scores for the extrinsic aspects of job satisfaction (e.g. pay, hours of work).

5. Motivation was examined in terms of effort and this did not vary significantly across F2. Reported levels of effort were similar to (T3) or lower than (T1, T2, and T4) those in a comparison group of managers. However, a steep increase in effort was associated with the ED placement.

6. Examination of work characteristics showed that there was significant variation in work demands and role clarity across the various placements. However, the ED placement was associated with a significant reduction in task feedback and an increase in social support from colleagues and hours worked in a typical week (work demands increased slightly and role clarity improved but not significantly).

7. There was some weak (cross-sectional) evidence of variation in work characteristics across trusts associated with variation in hours of one-to-one contact and close working contact with CS, work demands, job satisfaction, amount of teaching and support from CSs.

Recruitment of emergency departments/sites and reviewers

The aim was to recruit 12 EDs to the quality-of-care study with the expectation that this would provide access to 144 F2 doctors. Initially, all 30 participating EDs were contacted by e-mail in November 2010 to assess whether their ED information systems recorded the identifiers of the primary caregivers (in this case F2 doctors) for clinical episodes in the ED. Identifying ED episodes for which the F2 doctors were the primary caregiver was a necessity in the records review of F2 doctor quality of care. Nineteen of the 30 responded, detailing that their ED information systems had this facility. These 19 were then contacted by e-mail in March 2011 with an outline of the study and an invitation to take part. Fourteen of the 19 EDs contacted expressed interest in participating, but four of these were unable to complete the reviews within the required time frame for the project. A total of 10 EDs took part in the study and provided access to 74 F2 doctors. These 10 EDs represented six of the nine deaneries where F2 doctors carried out their placements in EM.

Recruitment of the EDs also relied on the recruitment of sufficient numbers of middle-grade doctors (specialist registrars) in EM to review the selected records at each site. Reviewers were recruited through the lead consultants at each site; a total of 28 reviewers were recruited across the 10 participating EDs. A copy of the reviewer information sheet is provided in Appendix 21.

Choice of clinical conditions for review

The choice of clinical conditions for review involved the initial identification of conditions that F2 doctors could be expected to see with reasonable frequency during their time in the ED, that are reasonably common in presentation, that present in adults, that will differentiate performance and that have well-defined clinical guidelines to support the criterion-based review. Following discussions between the project team and local ED consultants, six potential clinical conditions were identified:

• overdose

• head injury

• COPD

• asthma

• back pain

• gastrointestinal bleed.

The consultant leads at the 14 EDs that had expressed an interest in participating in the quality-of-care review were asked to rank the six conditions in relation to their suitability and likelihood that F2 doctors at their site would have responsibility for the care of patients with these conditions. In total, 10 of the 14 sites and two F2 doctors provided rankings for the provisional list of conditions. Three sites suggested chest pain as an additional condition but it was felt that there would be less scope for variation in practice. Taking an average of all ranks, overdose, head injury and COPD were ranked highest across all sites. COPD and head injury were the two conditions chosen for the study on the basis that they best met the criteria above. These two conditions also had the advantage that they were quite ED specific and different from the range of conditions included in the longitudinal study assessment of confidence in managing conditions across all placements.

Development of criterion-based review form

Criterion-based review requires the reviewer to identify and record specific items of care that are consistent with established guidelines on quality of care. The review criteria in this study were developed using established methods for developing explicit evidence-based review criteria from clinical guidelines as detailed in Hutchinson et al. 95 For each of the two conditions, the first drafts of the criteria were developed using relevant national clinical guidelines96,97 and local ED guidelines; these were subsequently validated by the project team and ED clinicians.

Clinicians in the project team in consultation with ED colleagues identified a subset of criteria that might be useful in the study. Refinement of the criteria involved three stages. Initially, the criteria were reviewed to consider whether the necessary information was likely to be available in the case notes. The draft lists of criteria were then sent to the consultant leads at participating EDs to classify the criteria as essential, desirable or non-essential and to elicit comments on the structure and wording of each item. A final review by the project team sought to ensure that all criteria were clear and logical. This process resulted in 15 head injury criteria and 20 COPD criteria. The criterion-based review items and scores for head injury and COPD are provided in Appendices 23 and 24 respectively.

Development of the holistic review form

The structured review form used for the holistic review was consistent with that in previous studies using this approach88,94 in providing a framework to structure implicit judgements with a view to maximising inter-rater reliability. Unlike the criterion-based approach, reviewers were not provided with any specific criteria for current best practice and were asked to use their professional judgement. The holistic review form provided reviewers with a limited structure to enable different levels of health-care quality to be identified – from excellent to poor care across different aspects of care (see Appendix 25). This approach has been used successfully by the study team in previous research examining quality in emergency care. 88

The reviewers were asked to rate the quality of care actually provided (as documented in the clinical notes) in relation to three key aspects of care (assessment of the clinical problem, investigations performed and patient management) and overall care on a numerical scale (1 = unsatisfactory, 6 = very best care). Reviewers were provided with written guidance to aid consistency in the interpretation of the numerical quality-of-care scale (Table 9). In addition, reviewers were asked to provide textual comments regarding the overall quality of care.

Data analysis

Data from the SurveyGizmo database were transferred to IBM SPSS version 19 (SPSS Inc., Chicago, IL, USA) for statistical analysis. Statistical analysis examined inter-rater reliability, the quality of care delivered by F2 doctors during their ED placements and the relationship between work characteristics, well-being and quality of care.

Scoring of criterion-based data involved calculating a total score for each head injury and COPD patient record based on the total criteria met per record (see Appendices 23 and 24 for criterion scores for head injury and COPD respectively). To compare mean criterion scores for the two conditions, a score for the proportion of the criteria met was also calculated by dividing the total score by the maximum potential score. Analysis of holistic scores for head injury and COPD cases used the actual ratings from each of the holistic rating scales, which are comparable across the two conditions. Mean criterion-based and holistic scores for the two conditions were calculated for use in assessment of the relationship between quality of care, job-related characteristics, well-being and motivation.

Intraclass correlations (ICCs) were calculated to assess the reliability of ratings between pairs or groups of reviewers within participating EDs. The ICC gives the correlation between reviewer ratings for the same set of case notes and is based on continuous data, unlike kappa statistics, which require the data to be categorical.

Descriptive statistics were calculated for criterion-based and holistic ratings to examine quality of patient care delivered by F2 doctors during their ED placements.

Mean scores were calculated for criterion-based and holistic ratings across three levels of case complexity (low, average and high) and statistical tests were conducted to assess the statistical significance of differences in scores [analysis of variance (ANOVA), t-test].

Pearson’s correlation coefficients were calculated to assess the relationship between criterion and holistic ratings.

Inter-rater reliability

Table 12 provides the ICCs used to assess inter-rater reliability; these are based on the actual (absolute) scores rather than the ranked (consistency) scores as the actual ratings are used to examine performance. The single-measure ICCs are presented as the quality-of-care analysis uses ratings from single reviewers rather than averaging multiple ratings for each record. In general, these will be lower than the reliability that might be expected from using the average of several raters (Table 13).

The correlations for the criterion-based reviews range from 0.65 to 0.94 and represent strong agreement. The correlations for the holistic review are lower and more variable (0.08–0.66) but are reasonably consistent with those from other studies. 93,94 These studies also report lower ICC statistics for holistic review than for criterion-based review.

Although an ICC of 0.8 is regarded as indicative of good agreement, this generally relates to data having a clear right and wrong answer and for which 100% agreement is possible. Higher levels of agreement are more achievable with criterion-based review using well-established criteria, whereas in the holistic review approach, based on subjective judgements, such a high level of agreement is less likely.

Appendix 26 provides ICC analyses based on the ranked review scores rather than the actual (absolute) scores (Tables 36 and 37).

Quality of care delivered by foundation year 2 doctors during emergency department placements

Table 14 presents the mean criterion-based and holistic scores for all F2 doctors. The mean scores for the different aspects of care assessed by holistic review are all > 4 on the six-point rating scale. There was a significant difference between the mean scores for ‘investigations’ [t(739) = 2.04, p = 0.04], with head-injury case notes scoring higher than COPD case notes.

Appendix 27 provides details of the proportions of review criteria assessed as being met in the head injury and COPD patient case notes. This gives an indication of the relative strengths and weaknesses of F2 doctors’ performance and quality of care for these two conditions. Appendix 28 shows the distribution of holistic review scores for head injury and COPD case notes across the six-point rating scale.

The more detailed information on proportions of review criteria being met (see Appendix 27) highlights omissions in recording information. A common issue across both the head injury records and the COPD records appears to be the level of detail provided. Criteria relating to documenting the mechanism and details of head injury are met by 93% and 80% of case notes respectively; however, information is lacking in relation to loss of consciousness and the reason for organising a computerised tomography (CT) head scan. Similarly, for the COPD case notes, details about shortness of breath and respiratory rate were recorded by most but there was variable recording for other criteria.

Summary

Intraclass correlations calculated to assess inter-rater reliability showed a high level of agreement amongst reviewers for the criterion-based review (0.65–0.94). The holistic review ICCs were much lower (0.08–0.65) and varied across sites. Taking an average measure of quality across reviewers produced higher ICCs for criterion-based (0.79–0.98) and holistic (0.20–0.85) review.

Mean scores for the proportion of head injury and COPD criteria met were 50.78% and 54.88% respectively. Mean scores for the different aspects of care assessed by holistic review range from 4.14 to 4.61 on the six-point scale. Analysis of the criteria met identifies strengths and weaknesses in relation to quality of care, specifically the extent of information recorded in case notes.

Findings from an analysis of case mix for head injury and COPD cases found no significant differences in relation to quality of care and complexity of clinical presentation, indicating that any observed differences in quality of care are not attributable to case mix variation.

Correlations between criterion-based and holistic quality-of-care ratings show associations between all review measures, but the intercorrelations between the holistic measures are somewhat higher (0.64–0.93) than those between the holistic and criterion-based measures (0.44–0.64).

Analysis

Detailed information on how the work-related outcomes and job-related characteristics were calculated is contained in Chapter 4. In summary, measures of work-related outcomes (well-being and motivation, intention to quit, confidence in managing acute conditions) and job-related characteristics (e.g. work demands, task feedback, role clarity) were analysed for 55 F2 doctors from the time of their placement in the ED.

Detailed information on how the quality-of-care outcomes were calculated is contained in Chapter 5. In summary, two approaches were used, criterion-based review and holistic review. Criterion-based review involved calculating a total score for each head injury and COPD patient record based on the total number of criteria met per record. The holistic review provided reviewers with a numerical scale to rate the quality of care actually provided (1 = unsatisfactory, 6 = very best care). For this analysis we used four outcomes – two criterion-based scores for head injury and COPD and two holistic overall care scores for the two conditions. Holistic overall scores were calculated by combining holistic scores for assessment, investigations and management.

Correlations between mean scores for the work-related outcomes and job-related characteristics relating to the time point at the end of each respondent’s ED placement and the four quality-of-care outcomes were calculated using Pearson’s correlations. A statistically significant level of association was set at p < 0.05. The correlated variables are listed in Box 12.

Quality of care

The only statistically significant association at the p < 0.05 level was the association between task feedback and holistic overall care. However, there was a clear pattern of small- to medium-sized correlations between levels of motivation and two quality-of-care outcomes (head injury – holistic overall care and COPD – holistic overall care) such that higher levels of motivation were more likely to occur amongst those doctors with higher scores for these two quality-of-care outcomes.

A similar pattern of effects existed, albeit in the opposite direction, for work demands and these two quality-of-care outcomes such that respondents with higher work demands were more likely to have lower quality-of-care outcomes for head injury – holistic overall care and COPD – holistic overall care (with the latter approaching statistical significance, p < 0.051). Indeed, when we examined the association with the single holistic measure of overall care, the association between work demands and overall care was statistically significant.

The correlations between all job-related characteristics/work-related outcomes and quality-of-care outcomes are provided in Table 18.

Routine performance data

Data collected

All data were routinely collected by the participating EDs. The following individual-level data were requested from all participating EDs:

• The total number of patients seen by each F2 doctor during their ED placement.

• The percentage of patients seen by each F2 doctor within the 4-hour ED performance target.

• The number of unplanned patient reattendances within 7 days per F2 doctor.

• The mean doctor episode time per F2 doctor (over placement in the ED). Episode time was defined as the time interval from when the patient first saw the F2 doctor to discharge/referral from the ED.

• The mean total time per F2 doctor (over placement in the ED). Total time was defined as the time interval from patient arrival in the ED to discharge/referral from the ED.

In addition, data were collected at the ED level as follows:

• Mean waiting time for each month during the period of the longitudinal study. Waiting time was defined as the time interval from when the patient arrived in the ED to when he or she left the ED.

• The percentage of waiting times in the ED that were within 4 hours for each month during the period of the longitudinal study.

Analysis

Pearson’s correlations between mean scores for the work-related outcomes and job-related characteristics relating to the time point at the end of each respondent’s ED placement and performance outcomes were calculated. A statistically significant level of association was set at p < 0.05.

Results

As in the analysis of quality of care, no statistically significant associations existed between well-being and motivation and scores on the performance outcomes at the p < 0.05 level (Table 19). There was a clear pattern of small- to medium-sized correlations between both anxiety and depression and two of the performance outcomes such that:

• respondents with reported higher levels of anxiety or depression had higher mean episode times and higher mean total waiting times

• respondents with reported higher levels of anxiety or depression were more likely to see a smaller percentage of patients within the 4-hour ED performance target.

Postgraduate medical training: the theory and the reality

Postgraduate medical training has changed from the previously unstructured and service-led experience of the ‘house officer’ years to a more structured programme of FT accommodating over 8000 trainees in England. Collins22 undertook an evaluation of this change and this report became available during the course of this study, allowing some comparisons. FT was established in 2005; our stakeholder study examined the position of the programme in its fifth year of operation and the longitudinal study examined the reality for the trainees in its sixth year of operation.

We found that there was a strong national framework for FT across England and Scotland which was well understood by national and regional stakeholders, demonstrating the successful change to a structured programme of postgraduate development and agreeing with the Collins evaluation. 22 However, examining the realities of implementation at regional and trust level revealed several differences in terms of placement quality and the amount of supervision and assessment available for trainees. There was evidence that the training programme had good quality-assurance processes in place and the regional and national stakeholder groups were aware of some of these differences, but issues of communication and involvement of NHS trusts in placement planning and delivery were highlighted as reasons for implementation difficulties. There was also concern expressed at the trust level in relation to the capacity of trainers to deliver the volume of training and assessments required by FT, and that this often conflicted with service delivery commitments.

We found that there was a variety of innovative placements being employed (such as general practice and palliative care or highly specialised roles such as medical biology and neonatal orthopaedics) and there was a slight variation in the length of ED placements, with some regions preferring the use of 6-month rotations to 4-month rotations; however, this variation occurred in a minority of cases. This agreed with the pattern reported in the Collins evaluation. 22

We found that one of the main areas of variation was the educational philosophy of the FT programme held by the educators and the programme implementers at trust level. National and regional stakeholders communicated a view of the programme that built on current problem-based medical education with experiential learning in the workplace, enabling development of the competency and confidence of junior doctors. By contrast, several TLs and some trainees emphasised the importance of ‘specific training sessions on ED-related topics’ to develop trainee competency and confidence. These differences would seem to undermine the purpose of FT, requiring the espoused needs of the programme to be fully and widely articulated and debated both at trainee and at trust level to move FT forward. This issue has been noted previously100 and would appear not to have been resolved. Collins22 concluded that the FT programme lacked an articulated and accepted purpose, especially in its second year, and we would contend that differences in educational philosophy and implementation of agreed learning outcomes are a major reason that the purpose of FT is not clearly understood.

This study shows that, according to the perceptions of F2 doctors, there is improvement in undertaking their work role throughout the F2 period. The time spent in the ED had the greatest impact on their perceived confidence and competence compared with all other placements that they experienced during the year. However, the influence of the FT programme itself is unclear given that we did not include any control areas not delivering FT for comparison. However, it is clear that, overall, F2 doctors perceived that their skills improved over time. Our study has moved the understanding of F2 doctors’ training forward by exploring development across a range of placements over time, in which F2 doctors experienced various work roles in diverse patterns (such as day, night, on-call or shift working). By looking at several cohorts of F2 doctors we have been able to enlarge the scope of enquiry of postgraduate medical education compared with studies considering only a single specialty or work pattern; this was called for in the review by Scallan. 101 In addition, we have developed a simple, self-report measure to assess perceived confidence and competence during training, called for by Miller and Archer. 102

Variation in implementation across organisations

Our study demonstrated cross-sectional evidence of variation between trusts in the number of hours of one-to-one and close working contact with CSs, work demands and amount of teaching and support from CSs. These findings validate the comments of regional stakeholders who noted variation in supervision and those of national stakeholders who were concerned that not all trainees were getting constructive feedback on their work. In their focus groups trainees demonstrated the importance of adequate supervision in enabling them to gain confidence in their decision-making abilities, especially early in a placement, and comments about the quality of support and supervision were frequently included in the questionnaires – with examples of excellent and poor supervision being given. These data build on reviews and studies (e.g. Scallan,101 Marteau et al. 103) offering clear support for the need for and value of close, supportive supervision, including feedback for trainees on their abilities, to enable them to develop confidence and competence as doctors. However, the study was not sufficiently powered to reveal any objective evidence that reduced supervisor support impacted negatively on the perceived clinical competence and confidence of the doctors when directly questioned.

It would appear that two factors stand in the way of developing a strong framework of supervision within the workplace. The first is the difference in educational philosophy between medical educators, supervisors and trainees that we have already discussed. Second, but probably the most important, is the fact that supervision and support are taking place in the workplace and supervisors have to find the time and opportunity to undertake these activities, often during periods of busy service delivery. This is particularly an issue in time-pressured specialties like EM. This is an important resource issue associated with workplace education that seems to rest solely with trusts, who are also responsible for service delivery. It is hard to ignore the comment made by a busy consultant looking at the impact that FT has had on his department: ‘We are not broken yet but it is not far off – we just need more senior staff.’

Clearly a more joined-up resource between medical training and service delivery needs to be explored, particularly with senior NHS trust staff, to enable the appropriate level of supervisory support to be delivered.

Assessment of competence

An obvious focus of the support and development of trainees is the completion of their e-Portfolio of competence. This study, along with the evaluation by Collins,22 shows the development of a national assessment programme. However, the stakeholder groups describe variation in implementation. National and regional stakeholders commented that the implementation of assessment was far from consistent across the regions and varied in quality between departments and assessors. Regional stakeholders noted that e-Portfolios were being used by most trainees, although there were some challenges with regard to the availability and use of information technology in various trusts. TLs recognised the importance of assessment in experiential learning but noted that feedback required delivery on a 24-hour basis, necessitating the involvement of senior and middle-grade medical staff along with senior and practitioner nursing staff. However, the value of actually being able to observe the trainee in structured situations was acknowledged. Trainees held a balanced view of the national assessment process: some saw the value of gaining direct feedback on their competence and the opportunity to discuss individual cases with their supervisor; others felt that the format was ‘too tick boxed’ and, although allowing the examination of routine clinical cases, did not allow discussion of complex cases that might provide greater opportunities for learning. These findings are supported by the study by Hrisos et al. ,104 which examined learning e-Portfolio use in one deanery between 2004 and 2005. Although direct comparisons with this study are difficult, it would appear that trainees and supervisors have developed a little more faith in the recording of experiential work-based learning in the e-Portfolio. Clearly more development work needs to be carried out with assessments to examine the balance between feedback and case discussion that is possible in a busy working environment.

One of the main criticisms of FT was the lack of a consistent framework to assess the completion of the F2 period of workplace training. Unlike the end of F1 training, which is marked by registration with the GMC, a strong outcome as it involves a shared responsibility between the deanery and the trusts,22 the end of F2 training relies on the ‘sign off’ by TLs. National stakeholders fear that this end stage lacks focus and is not a sufficiently significant milestone for the junior doctors. Regional stakeholders are concerned that there is not a consistent approach, as with assessment, and it is likely that different sign-off criteria are required by different assessors. Qualitative comments from trainees would encourage a view that they have self-evaluated their learning over the F2 period and have acknowledged the experiences that have benefitted their practice (see comments in The impact of the emergency department placement). There was little evidence from any of the stakeholder groups that F2 sign-off led to the identification of specific issues or ‘doctors in difficulty’. It would seem that more work is needed to develop a nationally agreed scheme that marks the end of F2.

Well-being and motivation of foundation doctors

National and regional stakeholders and TLs confirmed that there were no systems in place to identify and support periods of overwork and strain for F2 doctors in general but that there was provision for those few who had been designated as ‘doctors in difficulty’ during their training programme. Our study was the first to systematically examine a sample of trainees at the end of their F1 training and throughout the F2 programme.

On average, trainees reported levels of anxiety and depression that were similar to (T2) or better than (T1, T3, T4) those of a normative group of professional and technical workers. Levels of anxiety and depression did not vary significantly over time. Reported levels of job satisfaction were, on average, significantly higher than those of a normative group of doctors and other NHS staff, and overall and intrinsic job satisfaction (associated with aspects of work such as freedom to choose their own method of working and opportunity to use their abilities) were seen to increase over the period of their FT. Trainees reported lower levels of motivational effort than a group of general managers and this did not vary significantly across their training. This study has demonstrated that it is possible to systematically record levels of well-being of trainee doctors and compare these with levels found in other normative studies, enabling appropriate interpretation. It is likely that these measures could be incorporated within trainees’ e-Portfolios.

Robust and stable levels of well-being of foundation doctors from a number of NHS trusts are demonstrated in this study, describing a more positive picture of trainee well-being than has been found in previous single-centre or single-measure studies (e.g. Brennan et al. ,105 Yates et al. 106).

We examined various job-related characteristics associated with well-being across the F2 period and found that there was significant variation in work demands and role clarity across the various time points and, therefore, placements. These data were supported by qualitative comments from trainees who describe a variety of work roles: in some they are being stretched with a heavy workload (e.g. being on-call at night) whereas in others they have nothing to do and are not involved in decision-making about patients. For example:

I feel that my first F2 placement was not representative of my experiences as a doctor to date, as it was a particularly difficult job. As the SHO on ‘take’ for 3 busy admissions units I could take up to 40 referrals per 12 hour shift, and I spent a substantial amount of time doing paperwork, answering the bleep and organising the list rather than seeing patients. When the opportunity arose to learn a new skill (e.g. lumbar puncture) I was constantly being called away to do mundane tasks and training suffered as a result. The consultants, registrars and other SHOs on the unit were excellent – very approachable and supportive – it was just the nature of the job that ground me down – it was a relentless stream of clerking without any feedback as to whether your initial management was correct. This was not the fault of anybody, it was just the nature of the work, but I did find it very dispiriting. I did learn from the 4 months, but I felt that my job was 98% service delivery with very little training, hence why I have given quite negative feedback about it. By contrast, currently I am working on POSU [postoperative surgical] and my job is exactly what I hoped for. I am learning new skills, I have time to do tasks properly and give patients high quality care, the team are excellent, and I feel very well supported with plenty of teaching and other opportunities for development including ring-fenced training time.

T2 participant

I very much enjoyed my time in this placement, but the hours were just very unsociable. I think that more could have been learned if the department was not so busy – every patient that I saw, I felt I could learn something from, but there was not the time for much teaching on the shop floor.

T3 participant

I had a rotation in XX was quite frankly, dreadful. I had three consultants (one working 50% for the trust) who I would see for about an hour a day. I had no junior colleagues, hence no support. I had minimal teaching, certainly no formal teaching in the job. I was largely on my own, in a meaningless role (there is not much an F2 can do in a XX department and certainly very little responsibility or decisions you can take). I had no patients.

T2 participant

Similarly, in some placements trainees are clear about what is expected of them within their work whereas in other placements there is little direction as to what they are expected to do and ambiguity over whether or not they are part of the medical team responsible for service delivery. Examples show that some ongoing monitoring of the trainees’ work environment, as we have shown in this study, would identify issues of work demands and role clarity and, if measured at an appropriate time, enable changes to be made in placements reporting less than satisfactory learning outcomes.

From this we can conclude that the average junior doctor is likely to have comparable or better levels of well-being than other NHS workers and, on average, to expend a lower level of effort than those in managerial roles. However, before we conclude that trainees cope well with their FT, we should bear in mind that these are averaged data and within these groups there may be specific individuals who require additional support at specific times and/or in certain placements. A monitoring process using measures such as those within this study could be incorporated into the e-Portfolio, fulfilling recommendations made by Borman4 and Darsi. 6

Application of these findings would encourage regular monitoring and review of junior doctor well-being (we note that there is a general ‘job satisfaction’ survey within the trainee annual survey used by deaneries; however, this would seem to be concerned with specific training-related outcomes). We see the value in two levels of assessment: at the placement level associated with the e-Portfolio and at an organisational level within the NHS trust. This study has demonstrated an accurate and valid method of assessing trainee well-being and motivation, the outcomes of which are required by both supervisors and trainees in order to adapt and develop placements as appropriate learning experiences. Therefore, these data should not be kept at deanery level but communicated locally and swiftly to supervisors and trainees. A useful method of assessing well-being is the trust’s annual staff survey (provided by the National NHS Staff Survey Co-ordination Centre). By adapting this survey to include specific categories for F1 and F2 doctors, these doctors could be identified at trust level and their results compared with those of other doctors within that specific trust; in addition, national and between-trust comparisons would also be possible. Incorporating assessment of junior doctor well-being within the human resource framework of each trust may increase local ownership of FT.

The impact of the emergency department placement

We were particularly interested in the impact of ED placements on trainees’ competence and confidence and well-being and motivation. The longitudinal design of the study provided the opportunity to study a group of trainees who encountered the ED during their first, second or third F2 placement.

This study has shown that trainees display a significant increase in competence and confidence managing common conditions and performing routine procedures across F2, and the biggest increase in competence and confidence occurred immediately after their ED placement, regardless of when in the year the ED placement occurred and what other placements they experienced during the year. This finding was validated by qualitative comments from trainees who, although noting difficulties relating to the high work demands, long working hours and difficult shift patterns associated with the service delivery role of EM, also described the excellent supervision, fantastic learning experiences and excellent teamworking that increased their confidence as doctors. For example:

Despite how stressful I found A&E in the beginning, on the whole it was a fantastic experience and I am definitely glad to have done it. The consultants and SpRs [specialist registrars] at XX A&E are without exception brilliant doctors to work with.

T2 participant

A&E was a very good clinical experience, and I am glad that I had the opportunity to undertake this placement. It was difficult at times, particularly with regards to the rota/hours, but I have learnt a lot.

T4 participant

Emergency department placements were also associated with changes in well-being, motivation and the way that trainees worked. On average, the biggest increase in effort by trainees was shown in relation to their ED placement compared with all other placements during the year, indicating that they had to use up greater reserves from their energy pool during their ED placement. Further, although not significantly different, on average trainees’ biggest increase in anxiety was associated with their ED placement. Qualitative commentary in the questionnaires and material from trainees’ focus groups suggest that this anxiety stems from worry about making the right decision to admit or send home a patient, and it is this that trainees want the most feedback on and support for. In terms of work characteristics we found a significant reduction in task feedback during the ED placement, which was not matched by the trainee need for feedback on their decisions from senior staff. This reflects the busy and often unplanned nature of the workload in EM, which often makes planning one-to-one time with trainees challenging. Overall levels of job satisfaction and intrinsic job satisfaction were not seen to decrease during the ED placement whereas some participants reported a significant decrease in levels of extrinsic job satisfaction at this time. Extrinsic satisfaction is related to aspects such as pay, hours and conditions of work and these factors received a good deal of commentary from trainees, particularly with regard to long shifts and antisocial hours of work, supported by data revealing increased hours worked per week. The contrast between the demands of service delivery in EM and those in other placements is often great, with some trainees coming into EM having never experienced working night shifts, for example. It is, therefore, perhaps not surprising that there are some reported changes in job satisfaction as a result. Trainees clearly cope well with these difficulties as, with the exception of extrinsic satisfaction, there were no other significant differences in well-being, which remained favourably comparable with that of other health-care workers. Increased social support from colleagues during their ED placement would appear to help trainees, as would the clarity of their role in the ED, helping them know how to direct their efforts. Unfortunately, these factors would seem to go hand in hand with the nature of emergency working, which for some trainees would have been their first taste of service delivery and a contrast to their previous year of training.

This study has shown that there are considerable benefits of the ED placement in terms of competence and confidence gained by junior doctors. However, this does come at a cost of slightly increased anxiety and decreased extrinsic job satisfaction, although the levels of these are comparable to those of other doctors and health-care workers. Careful monitoring of trainees and good supervision with direct feedback on the quality of referral decisions are vital at this time. However, it should be noted that this burden will fall on hard-working staff with few additional resources to offer. If the ED is adopted as a fixed rotation for F2 training, additional senior resources in the ED should be carefully considered.

Quality of care provided by foundation year 2 doctors

The stakeholder study showed a level of disagreement with regard to the quality of care provided by F2 doctors. Regional and national stakeholders suggested that F2 doctors were able to provide adequate patient care and were better prepared in terms of their communication and procedural skills than postgraduates predating MMC. However, supervisors working in trusts found F2 doctors underprepared for service demands, having had little previous experience of decision-making with acutely ill patients. Trainees recognised their supervisors’ concerns, admitting to anxieties relating to decision-making, but agreed with the educational stakeholders that they were able to deliver adequate patient care at the end of their 4 months of training.

Given these varying views of the adequacy of the quality of care provided by F2 doctors we carried out a clinical case notes review of doctors involved in the longitudinal study for their ED placement. Following consultation, two conditions were selected for the case notes review: head injury and COPD.

Overall, the quality of care provided by the cohort of F2 doctors in this study during their ED placements appears good, with mean scores from holistic review ranging from 4.14 to 4.61 on a scale from 1 to 6. Examination of the spread of holistic ratings across all records does highlight some scope for improvement in a small proportion of cases. The mean score for the proportion of review criteria met was > 50% for both head injury and COPD cases. Examination of the proportions of review criteria met highlights aspects of care for which there is scope to improve the level of detail recorded in patient case notes. Given the importance of patient case notes for communication, audit and legal purpose, it is important that all relevant information is clearly documented.

Assessment of quality of care can focus on care outcomes or the process of care. Process measures such as case notes review are recommended as being more suitable for judging quality in health care. 83 The literature suggests that both criterion-based and holistic review methods have strengths and weaknesses and should be regarded as complementary. 93 The findings show stronger correlation coefficients for intercorrelations between the various holistic measures than for correlations between criterion-based and holistic review measures, despite the criterion-based review being carried out first for each record. This appears to support the view that holistic review is assessing aspects of quality not reflected in criterion-based review94 and a thorough assessment of performance should combine the two approaches. Furthermore, the inter-rater reliability results illustrate the benefit of enhanced reliability when using two reviewers and taking an average measure of quality of care. The web-based data-collection tool proved very efficient for collecting the review data such that the addition of automated data analysis and output functions could support performance review and feedback by CSs and potentially peer and even self-review by junior doctors.

When we examined the relationship between well-being and motivation and the quality-of-care outcomes there were no statistically significant associations found, although small- to medium-sized correlations were noted between motivation and work demands and holistic care of head injury and COPD cases. Junior doctors reporting greater effort during their ED placement were likely to have better quality-of-care outcomes for these two conditions. However, junior doctors who reported higher work demands were likely to have lower quality-of-care scores for these conditions. Detailed examination of performance data for 74 junior doctors working in the ED revealed no statistically significant associations with well-being; however, there was a suggestion of small- to medium-sized correlations between anxiety, time spent with patients and percentage of patients seen. This suggests a possible relationship between anxious F2 doctors and longer patient episodes and seeing fewer patients within their ED placement. Although none of these relationships is statistically significant, they are plausible in busy working conditions and are in keeping with the comments of the supervisors during phase 1 of the study. This study indicates that these relationships should be investigated in more depth in large-scale studies. However, these findings (length of patient episode and number of patients seen) may offer a useful and easy rubric for supervisors to use to identify trainees who are struggling with the EM working environment. There is already anecdotal evidence (S Mason, emergency medicine consultant, Sheffield Teaching Hospitals NHS Foundation Trust, 2011, personal communication) that such measures are already being used informally to chart trainees’ progress in the ED.

Background

1. Briefly describe your own role in relation to FY2 training? (3 minutes)

Prompt: Can you describe the input that you have in the planning and implementation of FY2 training.

Approach to postgraduate medical education

2. Can you describe to what extent there is a national strategic approach to FY2 training in England/Wales/Scotland? (5 minutes)

Prompt: Who is responsible?

Prompt: What mechanisms exist for implementing a national approach?

Prompt: Policies, policy documents, conferences and curriculum?

Prompt: Nationally how many trainees are there? Is this sufficient?

3. What are the challenges that exist in the implementation of a national strategic approach for FY2 training? (5 minutes)

Prompt: What are the likely successes? How are these measured?

Prompt: What are the likely failures? How are these measured?

Prompt: Issues specifically for the ED?

Variation in training

4. To what degree is there variation in the implementation of postgraduate training at the regional level? (5 minutes)

Prompt: Is there variation at foundation school level?

Prompt: What do you think are the causes of variation in implementing training?

Quality

5. What are the criteria for success for FY2 doctors’ training? (7 minutes)

Prompt: How are these measured?

Prompt: How well are these met? (in terms of patient care and performance)

Prompt: Are they sufficient? (in terms of patient care and performance)

Prompt: Is training providing FY2 doctors that are fit for purpose?

6. What quality assurance mechanisms exist for FY2 training at a national level? (5 minutes)

Prompt: e.g. Feedback from health organisations, doctors?

Prompt: Are these sufficient? If issues are noted what changes are made?

(35 minutes to this point)

(Sue takes over)

Well-being and motivation of FY2s

7. How is the well-being and motivation of FY2 doctors evaluated in their training? (8 minutes)

Prompt: How is this assessed, when and by whom? (Is it part of the assessment process?)

Prompt: If there are issues of well-being or motivation how are these addressed?

Prompt: Differentiate between local and strategic responses.

Prompt: Is the potential impact on patient care considered?

Career development

8. Do you feel there is sufficient career development built in to the training so that FY2s can build their careers on a firm basis? (7 minutes)

Prompt: Is there limitation of choice for doctors?

Prompt: Are they specialising too early?

Prompt: Are doctors ‘slotted in’ to operational arrangements and miss out on the bigger picture?

(50 minutes to this point)

Future developments

9. Can you describe any developments that will occur in the near future that may affect FY2 training? (5 minutes)

Prompt: Regional or national basis?

Prompt: Are there any changes you would personally like to see made to postgraduate medical training?

Ending

Is there any information you would like to add that would enable us to understand the current and planned future state of postgraduate medical training?

Thank you for your valuable time – we will be sending you a copy of our final report. (5 minutes)

Background

1. Briefly describe your own role in relation to FY2 training? (2 minutes)

Prompt: Can you describe the input that you have in the planning and implementation of FY2 training?

Planning and implementation (regional level)

2. How is FY2 training planned and implemented at the regional level? (5 minutes)

Prompt: Individuals responsible, policy?

Prompt: Is this approach consistent with the national strategic approach or does it differ in any way?

Prompt: How many trainees are there? How many trusts are involved? Is this sufficient for your regional area?

3. Are the placement organisations involved in the planning of training? (3 minutes)

Prompt: If so, how?

Prompt: Meetings, visits, joint bodies?

Variation

4. Is there variation in the implementation of foundation training across the regions? (5 minutes)

Prompt: What are the points of variation (e.g. placement length)?

Prompt: What are the causes of variation?

(20 minutes to this point)

Quality

5. What are the key training outcomes for FY2 doctors? (5 minutes)

Prompt: How useful is the mentor/educational supervisor role for you in making an assessment of the junior doctors?

Prompt: Is there variation in educational supervisors?

Prompt: Are these the same criteria for signing off the successful completion of a placement? If not, why not?

Prompt: Is the e-Portfolio used?

(Sue takes over)

6. To what extent is FY2 training providing doctors that are fit for purpose? (5 minutes)

Prompt: How useful are the formal assessments to assess FY2s’ ability?

Prompt: Are doctors ‘slotted in’ to operational arrangements and miss out on the bigger picture? This repeated later in career development?

Prompt: Are there any specific issues for the ED?

Prompt: What type of issues are you finding with FY2 doctor training? Did these happen before foundation training? (if a specific issue is mentioned ask them to describe the example fully)

7. What quality assurance mechanisms exist for foundation training in your region? (5 minutes)

Prompt: e.g. Feedback from health organisations, doctors?

Prompt: Are these sufficient? If issues are noted what changes are made?

(35 minutes to this point)

Well-being and motivation of FY2s

8. How is the well-being and motivation of FY2 doctors evaluated in their training? (10 minutes)

Prompt: How is this assessed, when and by whom? (Is it part of the assessment process?)

Prompt: In your experience what motivates FY2 doctors?

Prompt: What aspects of work do FY2s struggle with?

Prompt: How is the potential impact on patient care considered?

Career development

9. Is sufficient career development built in to the training so that FY2s can build their careers on a firm basis? (5 minutes)

Prompt: Is there limitation of choice for doctors?

Prompt: Are they specialising too early?

Prompt: Are doctors ‘slotted in’ to operational arrangements and miss out on the bigger picture?

(50 minutes to this point)

Future developments

10. What developments will occur in the near future that may affect foundation training? (5 minutes)

Prompt: Are there any changes you would personally like to see made?

Prompt: Changes that may influence patient care?

Prompt: Changes that may influence FY2s well-being and motivation

Ending

Is there any information you would like to add that would enable us to understand the current and planned future state of postgraduate medical training?

Thank you for your valuable time – we will be sending you a copy of the Executive Summary of our final report. (5 minutes)

Background

1. Briefly describe your own role in relation to FY2 training please? (3 minutes)

Prompt: How are you brought into contact with FY2s working in your department?

Training

2. Can you describe the training FY2 doctors receive in your ED? (4 minutes)

Prompt: What is the role of the junior doctors in the ED?

Prompt: How many trainees are there? Is this sufficient or is it too many?

3. Has there been any changes to the way training in the ED has been provided in recent years? (3 minutes)

Prompt: What is the impact of changes to training (e.g. changes to way you deliver care, workload and supervision)?

4. Do you think the FY2 doctor training is adequate preparation for them to deliver the care they are expected to provide? (5 minutes)

5. In what way do you think training can be improved? (5 minutes)

(25 minutes to this point)

Quality

6. How well do assessments measure the ability of FY2s to deliver good clinical care? (5 minutes)

7. How are FY2s assessed? (5 minutes)

(35 minutes to this point)

Well-being and motivation of FY2s

8. How is the well-being and motivation of FY2 doctors evaluated in their training? (10 minutes)

Prompt: How is this assessed, when and by whom? (Is it part of the assessment process?)

Prompt: In your experience what motivates FY2 doctors?

Prompt: What aspects of work do FY2s struggle with?

Prompt: Is the potential impact on patient care considered?

Prompt: Do FY2s integrate into the ED?

9. What clinical support or supervision do the FY2 doctors get in the ED? (5 minutes)

Prompt: Do you consider the clinical support to be adequate or is too much support/supervision required?

Prompt: What other types of support do FY2s receive (peers, occupational health)?

Prompt: What support should they receive?

Impact on care

10. To what extent do you feel FY2s in your ED contribute to the provision of a good quality service? (5 minutes)

Prompt: Issues such as workload, work rate, independent working.

Prompt: Are there any improvements that could be made to the way FY2s work in your ED? Give examples.

Ending

Is there any information you would like to add that would enable us to understand the current and planned future state of FY2 training?

Thank you for your valuable time – we will be sending you a copy of the Executive Summary of our final report. (5 minutes)

Participant information sheet

We would like to invite you to take part in a research study. The study is evaluating the experiences of foundation year 2 (FY2) doctors and the impact on their well-being and the quality of care they provide. The evaluation is being carried out in England. Before you make any decision regarding participation you need to understand why the research is being done and what it would involve for you. Please take time to read the following information carefully. Talk to others about the study if you wish. If you have any further questions, please contact the project manager (contact details provided at the end of this leaflet).

Participant information sheet

We would like to invite you to take part in a focus group for a research study. The study is evaluating the experience of foundation year 2 (FY2) doctors in England and the impact on their well-being, motivation and confidence in their role. Before you make any decision regarding participation you need to understand why the research is being done and what it would involve for you. Please take time to read the following information carefully. Talk to others about the study if you wish. You may wish to speak with your FY2 representative about participating. Alternatively, if you have any further questions, please contact the project manager (contact details provided at the end of this leaflet).

Participant information sheet

We would like to invite you to take part in a research study. The study is evaluating the experience of foundation year 2 (FY2) doctors in the emergency department (ED) and the impact on their well-being and the quality of care they provide. The evaluation is being carried out in a number of EDs in England. Please take the time to read the following information which explains why the research is being done and what it would involve for you. Talk to others about the study if you wish.

You may wish to speak with your FY2 representative (contact details to be included) about participating. Alternatively, If you have any further questions, please contact the project manager (contact details provided at the end of this leaflet).

What is this survey?

This is a survey of your views and opinions of your current F2 PLACEMENT and of the department as a whole where you are undertaking this placement. This is not a test. There are no right or wrong answers.

We want to know your personal views on the issues raised in the questionnaire. You will have received an information sheet about why this questionnaire is being administered. If you want further information then please visit the study website at http://edit.group.shef.ac.uk/.

Who will see my answers?

The information you give is totally confidential. Findings will be made available to all who participate, but in such a way that it is not possible for individuals to be identified. The research team at the University of Sheffield will be the only organisation to have access to the questionnaires completed by individuals.

How do I fill in this survey?

Please complete the questionnaire for your current role. The survey will take about 15–20 minutes to complete.

How should I respond?

For each statement you are asked to select one response that best fits your views. Please answer all the questions as openly and honestly as possible. Respond according to your first reaction. Do not spend too long on one question.

Please enter your email address.*

____________________________________________

About your job:

30. Year of qualification:*____________________________________________

31. Place of qualification:*

( ) UK

( ) Non-UK

About you:

32. Age:* ____________________________________________

33. Are you:*

( ) Male?

( ) Female?

34. What is your ethnic background? (Please select the answer that best describes your ethnic background)*

( ) White – British

( ) White – Irish

( ) White – Other

( ) Black – British

( ) Black – Caribbean

( ) Black – African

( ) Any other Black background

( ) Asian – British

( ) Asian – Chinese

( ) Asian – Indian

( ) Asian – Pakistani

( ) Asian – Bangladeshi

( ) Any other Asian background

( ) Mixed – White and Black British

( ) Mixed – White and Black Caribbean

( ) Mixed – White and Black African

( ) Mixed – White and Asian

( ) Any other mixed background

( ) Any other ethnic group (please specify): _________________

Hours of work:

35. In a typical working week in your current role, how many hours are you contracted to work?*

____________________________________________

36. What is the total number of extra hours worked in a typical week?*

____________________________________________

37. In the last four weeks of your current role have you had a shift changed at short notice (less than five days)?*

( ) Yes

( ) No

Foundation training experience:

38. Please indicate your first, second and final F1 placements:*

39. Please indicate your first, second and final F2 placements:*

Future plans:

40. What do you plan to do after your F2 year?*

( ) Take a short-term contract in medicine

( ) Work abroad in medicine

( ) Take some time out from medicine

( ) Don’t know yet

( ) Go onto specialist training

What specialty will you be moving into?

( ) General practice

( ) Hospital medicine

( ) Hospital surgery

( ) Emergency medicine

( ) Obstetrics and gynaecology

( ) Radiology

( ) Psychiatry

( ) Paediatrics and child health

( ) Trauma and orthopaedics

( ) Medical education

( ) Academic based

( ) Laboratory medicine

( ) Other (please state): _________________

What is this survey?

This is a survey of your views and opinions of your current F2 PLACEMENT and of the department as a whole where you are undertaking this placement. This is not a test. There are no right or wrong answers.

We want to know your personal views on the issues raised in the questionnaire. You will have received an information sheet about why this questionnaire is being administered. If you want further information then please visit the study website at http://edit.group.shef.ac.uk/.

Who will see my answers?

The information you give is totally confidential. Findings will be made available to all who participate, but in such a way that it is not possible for individuals to be identified. The research team at the University of Sheffield will be the only organisation to have access to the questionnaires completed by individuals.

How do I fill in this survey?

Please complete the questionnaire for your current role. The survey will take about 15–20 minutes to complete.

How should I respond?

For each statement you are asked to select one response that best fits your views. Please answer all the questions as openly and honestly as possible. Respond according to your first reaction. Do not spend too long on one question.

Please enter your email address.*

____________________________________________

About your job:

30. Year of qualification:*

____________________________________________

31. Place of qualification:*

( ) UK

( ) Non-UK

About you:

32. Age:*

____________________________________________

33. Are you:*

( ) Male?

( ) Female?

34. What is your ethnic background? (Please select the answer that best describes your ethnic background)*

( ) White – British

( ) White – Irish

( ) White – Other

( ) Black – British

( ) Black – Caribbean

( ) Black – African

( ) Any other Black background

( ) Asian – British

( ) Asian – Chinese

( ) Asian – Indian

( ) Asian – Pakistani

( ) Asian – Bangladeshi

( ) Any other Asian background

( ) Mixed – White and Black British

( ) Mixed – White and Black Caribbean

( ) Mixed – White and Black African

( ) Mixed – White and Asian

( ) Any other mixed background

( ) Any other ethnic group (please specify): _________________

Hours of work:

35. In a typical working week in your current role, how many hours are you contracted to work?*

____________________________________________

36. What is the total number of extra hours worked in a typical week?*

____________________________________________

37. In the last four weeks of your current role have you had a shift changed at short notice (less than five days)?*

( ) Yes

( ) No

Foundation training experience:

38. Please indicate your first, second and final F1 placements:*

39. Please indicate your first, second and final F2 placements:*

Future plans:

40. What do you plan to do after your F2 year?*

( ) Take a short-term contract in medicine

( ) Work abroad in medicine

( ) Take some time out from medicine

( ) Don’t know yet

( ) Go onto specialist training

What specialty will you be moving into?

( ) General practice

( ) Hospital medicine

( ) Hospital surgery

( ) Emergency medicine

( ) Obstetrics and gynaecology

( ) Radiology

( ) Psychiatry

( ) Paediatrics and child health

( ) Trauma and orthopaedics

( ) Medical education

( ) Academic based

( ) Laboratory medicine

( ) Other (please state): _________________