The 10-year impact of a ward-level quality improvement intervention in acute hospitals: a multiple methods study

Frameworks for understanding local approaches to Productive Ward implementation

Implementation science is the scientific study of methods to promote the uptake of research findings into routine health care in clinical, organisational or policy contexts. One such commonly applied framework in the health-care context is the Consolidated Framework for Implementation Research,29 which outlines four activities within an implementation process: planning, engaging, executing and reflecting, and evaluating.

Informing comprehensive frameworks such as the Consolidated Framework for Implementation Research are specific lessons from the broader implementation/improvement science literature. Several of these merit particular consideration in the context of our study as they relate to important features of the recommended approach for implementing PW (see Earlier research into the adoption and implementation of PW in the period 2009–12). First, Dixon-Woods et al. 30 and Powell and Davies31 highlight how securing the support of one professional group – nurses, in the case of PW – can lead to the alienation of others (as cited in Kislov et al. 32). Second, Lozeau et al. 33 raise the possible ‘contradiction between rhetoric of empowerment and the command-and-control procedures for auditing the performance data representing the managerial agenda’ (as cited in Kislov et al. 32). This was certainly one potential scenario in relation to PW, given the heavy emphasis on data collection and audit in the programme combined with the tension between the rationales of productivity and ‘releasing time to care’. Third, Kislov et al. 32 themselves suggest that ‘formidable contextual influences can significantly distort improvement approaches, activities and techniques’ over time, something that, given the duration of the period we studied, was likely to be highly relevant in acute trusts.

Fourth, Kislov et al. 32 argue that:

. . . interest in quality improvement significantly varies between different organisations (Krein et al. 201034), with ‘early adopters’ being more likely to be recruited in the initial phases of new improvement projects (Walshe 200935).

Kislov et al. 32

In doing so, they garner the most resources and support to implement an initiative. Finally, it is a common critique of Lean-based approaches that, at least in implementation, they fail to take due account of the customer (the patient in the health-care context). 36 This was an issue that we took an interest in during our fieldwork.

A recent classification of implementation strategies has been developed by Powell et al. 37 and Waltz et al. 38 Powell et al. ’s37 study used a three-round modified Delphi process to generate expert consensus on a common nomenclature for implementation strategy terms, definitions and categories. A panel of stakeholders with expertise in implementation science and clinical practice reached consensus on a final compilation of 73 discrete strategies; Waltz et al. 38 later organised the 73 strategies into nine categories. We have used this categorisation to explore the similarities and differences between the implementation approaches used in our case study sites (see Chapter 3).

The next section outlines a framework for considering what happened to PW after the initial implementation phase, namely how it was, or was not, assimilated into routine practice.

Frameworks for understanding assimilation processes

Here we are concerned with what happens after the adoption and implementation of a service innovation such as PW. Wiltsey Stirman et al. ’s27 review of empirical studies found that ‘many of the innovations that are initially successful fail to become part of the habits and routines of the host organisations and communities’. As Greenhalgh et al. 39 note, empirical evidence strongly suggests:

. . . an organic and often rather messy model of assimilation in which the organisation moved back and forth between initiation, development, and implementation, variously punctuated by shocks, setbacks, and surprises.

Greenhalgh et al. 39

Certainly, over time, adaptation and/or abandonment40 were both potential outcomes of implementing PW.

Scheirer and Dearing41 outline how one detailed line of research defines sustainability as the institutionalisation or routinisation of programs into ongoing organisational systems. With this perspective, the maintenance of programme activities without special external funding, which was time-limited (usually up to 2 years) in the case of PW, is most likely to occur if the programme components become embedded into organisational processes. While this was the intention of designing and implementing PW, if this happens, researchers may no longer be able to identify a specific ‘programme’, as the activities have become a part of the organisation’s core services. These concepts are well developed in Yin’s42 concept of routinisation and are summarised by Shediac-Rizkallah and Bone,43 who explain that:

. . . a process of mutual adjustment occurs such that both the innovation and the organisation change to adjust to each other. The innovation eventually loses its separate identity and becomes part of the organisation’s regular activities, a process that has been referred to as ‘routinizing’ or ‘routinization’.

Shediac-Rizkallah and Bone43

Building on the work of Lozeau et al. ,33 Kislov et al. 32 argue that a potential ‘compatibility gap’ between a set of assumptions underlying the design of a managerial intervention (such as PW) and the actual cultural, structural and political characteristics of an adopting organisation can result in one of the following, each of which we view as a different form of assimilation or routinisation:

• customisation, which involves both adapting the managerial technique and adjusting unit processes (i.e. the mutual adjustment described in quotation above) and recognising that adaptation, reinvention and ongoing development are part of continuous improvement

• loose-coupling, whereby the technique is adopted only superficially, in a ritualistic way, with the functioning of the unit remaining largely unaffected

• co-optation or corruption, whereby the technique becomes captured and distorted to reinforce existing roles and power structures

• transformation, whereby the adopting unit modifies its functioning to fit the assumptions behind the managerial technique and the actual use of an innovation does not significantly differ from its intended use.

While Lozeau et al. 33 originally presented the different forms of assimilation outlined above as distinct scenarios, Kislov et al. 32 suggest that these could be viewed as ‘temporal stages of a broader evolutionary process’ (i.e. they are stages of the same process rather than distinct independent categories). Kislov et al. ’s32 own longitudinal case study of ‘facilitation’ as a managerial technique (in the context of health-care QI) found that the technique moved through transformation, customisation and loose-coupling, to corruption (where it was co-opted for producing outcomes prioritised by the most powerful stakeholders). We have applied Kislov et al. ’s typology to PW in a similar way.

Frameworks for understanding the sustainability of service innovations

In their systematic review of the diffusion of innovations in service organisations, Greenhalgh et al. 39 defined sustainability as ‘making an innovation routine until it reaches obsolescence’ (while noting an ambiguity in the notion of sustainability, namely that the longer an innovation is sustained, the less likely an organisation will be open to additional innovations). Shediac-Rizkallah and Bone43 similarly argue that if a programme (such as PW) has been sustained as originally planned, then the system will have stopped evolving and adapting. In their review of empirical studies of sustainability, Wiltsey Stirman et al. 27 found that partial sustainability was more common than continuation of the entire programme or intervention, even when full implementation was initially achieved, and that, in studies that employed independent fidelity ratings to assess the sustainability at the provider level, fewer than half of the providers sampled continued the practice or intervention at high levels of fidelity.

The term ‘sustainability’ is rarely used in the mainstream literature to relate to the diffusion of innovations, and is a contested theme in discourses on innovation in organisations. 39 Shediac-Rizkallah and Bone state:

Little consensus exists in the literature on the conceptual and operational definitions of sustainability. Several terms have been in use to refer to the phenomenon of program continuation. Among these are: program ‘maintenance’, ‘sustainability’, ‘institutionalization’, ‘incorporation’, ‘integration’, ‘routinization’, local or community ‘ownership’ and ‘capacity building’.

Shediac-Rizkallah and Bone43

However, ‘sustainability’ is widely and commonly applied in discussions of the ‘success’ (or otherwise) of QI interventions in health-care organisations. Historically in this context, the Modernisation Agency, the precursor to the NHSI in the NHS, defined sustainability as ‘when new ways of working and improved outcomes becomes the norm’ (cited in Greenhalgh et al. 39).

A (simplistic) starting point for how we are conceptualising sustainability in our study is that the term ‘refers to the process of changing organisational goals and procedures that are maintained beyond the initial introductory period’. 44 Alternatively, Martin et al. 28 defined ‘sustainability’ heuristically and inclusively as ‘continuation of a service beyond its initial pilot funding’, making no judgements about fidelity to original intent nor categorising the specific nature of what was intended to be sustained. Wiltsey Stirman et al. ’s27 review found that the most frequently cited definition of sustainability was Scheirer’s,41 which was, in turn, based on a framework set forth by Shediac-Rizkallah and Bone. 43 This framework identified (1) continued benefits, (2) continued activities and (3) continued capacity as three elements of sustainability. We find Shediac-Rizkallah and Bone’s43 definition – in the context of public health interventions – most pertinent to PW, namely that (1) it is a multidimensional concept of the continuation process encompassing a diversity of forms (i.e. an entire programme may be continued under its original or an alternative organisational structure; or parts of the programme may become institutionalised); and (2) continuation may occur at a number of levels (individual, network or organisational).

We have therefore adopted this definition as the starting point for how we conceptualise sustainability in our study.

The important point in the context of our study, as also alluded to in Martin et al. ’s28 conceptualisation, is that rigid definitions of sustainability (i.e. maintaining the original goals of the programme) should be rejected in favour of a more organic notion of continuing, ongoing (adaptive) change in a positive direction. Our goal in reconstructing the story of the PW was to recognise that, as some aspects of any programme must be abandoned with time while others must expand and adapt, in evaluating the sustainability of PW we will be required to make complex judgements about an evolving programme-in-context.

National surveys

We carried out online surveys of DoNs and PW leads in English acute trusts. The objectives of the two surveys were to:

• identify cohorts of non-adopting and adopting acute trusts of PW in England from 2006 onwards

• explore how PW has been adopted, implemented and assimilated in English acute trusts and how this may have changed over time

• collate perceptions of the nature and scale of any impacts that PW may have had.

All acute NHS trusts in England were eligible to participate in both surveys, with the DoN (or someone to whom the DoN delegated responsibility) and the current or most recent PW lead being eligible to complete the DoN and the PW lead survey, respectively.

The survey was designed as a two-step process. In the first step, DoNs at each acute trust in England were identified and invited to complete a short online survey. That survey included a request to provide contact information for the PW lead at the trust. In the second step, PW leads were contacted and invited to take part in a longer online survey. Both surveys were set up and distributed via SurveyMonkey^® (Palo Alto, CA, USA; www.surveymonkey.co.uk). For each survey, two reminder e-mails were sent at fortnightly intervals to non-responders, with final follow-up telephone calls when necessary.

In formulating the survey, we received feedback on the wording of the DoN survey from three former DoNs and on the PW lead survey from three former PW leads, all of whom had agreed to act as critical friends. Revised versions were then circulated to the Project Advisory Group (PAG), and we received further feedback from five PAG members. After further refinement, colleagues carried out tests of the surveys to check that technical aspects were functioning. Finally, two DoN critical friends, one PW lead critical friend and a member of the study team, who was a former PW lead herself (RC), piloted and timed their completion of the online version to ensure that it did not take too long. As a result, the length of the DoN survey was found to be acceptable, but some questions were removed from the PW lead survey.

The final DoN survey (see Report Supplementary Material 1) consisted of up to 16 questions covering implementation, current use, perceived impact, reporting, influence on QI strategy, and future plans. The more detailed final PW lead survey (see Report Supplementary Material 2) had up to 40 items, covering implementation; feedback on modules and tools; staff, patient and carer engagement; use over time; costs and resourcing; learning and dissemination networks; data collection and reporting; perceived impact; personal and professional impact; and legacy. The surveys used a combination of open- and closed-response questions, including questions and statements with Likert scales. The surveys used skip logic to route respondents as appropriate. The only mandatory questions were the routing questions.

Different recruitment strategies were used for the two surveys, and each evolved over time. For the DoN survey an invitation to participate was sent to DoNs listed as in post as at April 2016 in NHS acute trusts in England according to a publicly available database. DoNs in all eligible trusts were e-mailed an invitation to participate, which included study details and a link to the online survey (see Report Supplementary Material 1). DoNs new in post during the lifetime of the survey were also identified through refreshed databases from the provider in July and November 2016 and through professional networks. In those cases where a new appointee was identified for a non-responding trust, an invitation was sent to the new appointee. Following an initially low response rate to the DoN survey, items publicising the survey, inviting DoNs to contact the researchers, were published in two online newsletters. Where Clinical Research Networks and/or R&D departments of eligible trusts offered to facilitate recruitment, or where DoNs asked us to resend the invitation to participate, we e-mailed a Microsoft Word (Microsoft Corporation, Redmond, WA, USA) version of the covering e-mail, containing a generic link to the DoNs survey. In all of these cases, the respondents were asked to enter the trust name in the survey so that we could identify them.

Given that not all DoNs provided us with contact details for a current or most recent PW lead, we applied for, and received, a substantial ethics amendment to diversify recruitment paths for the PW lead survey. Once this was received, we published an article in Nursing Times, drawing on preliminary analysis of the DoN survey, which invited PW leads to contact the researcher if they were interested in taking part in the study. PAG members who had worked in the NHSI also e-mailed former PW lead contacts to publicise the survey, and others passed on contact information to help us identify PW leads. A general publicity flier with contact details was circulated at relevant events and the survey was also publicised using Twitter (Twitter, Inc., San Francisco, CA, USA; www.twitter.com). The researcher also contacted trust QI teams (or equivalent), where possible, in an attempt to identify PW leads.

The DoN survey was conducted between June and December 2016 and the PW lead survey was conducted between September 2016 and January 2017.

Organisational case studies

Based on Rogers’45 theory on the diffusion of innovation, we planned to categorise all survey responses as being from one of five types of trust:

• ‘early adopters’ (Rogers’ ‘innovators’, ‘early adopters’ or ‘early majority’) that:

  • implemented PW on all their wards (through either whole-hospital implementation or planned roll-out)

  • implemented PW on some of their wards

• ‘late adopters’ (Rogers’ ‘late majority’ or ‘laggards’) that:

  • implemented PW on all their wards (through either whole-hospital implementation or planned roll-out)

  • implemented PW on some of their wards

• ‘non-adopters’ that had never implemented PW on any wards.

We then planned to conduct organisational case studies46 in six adopting trusts. In sampling for our six case study sites, our strategy was to sample using the following criteria in order of priority: time of adoption (prioritising as ‘early’ sites the trusts that participated in our earlier study), implementation strategy and other characteristics. These criteria are described in more detail in the next paragraph. A list of eligible sites was drawn up, along with information on the sampling criteria, and a ‘top six’ was decided in discussion between team members; it should be noted that the process of recruiting the six sites then took 11 months. A significant time-lag often occurred between inviting trusts to participate as case studies, receiving agreement in principle from the DoN, receiving local R&D approvals and receiving the information we required to start data collection. This meant that, in practice, we had to readjust our ‘top six′ sample as we went along, each time applying the selection criteria in their order of importance in the light of existing participating sites.

Our primary criterion was initially to sample trusts as distributed across Rogers’ adoption categories from the survey results alone. We encountered several difficulties with this. First, our sample was not large enough for us to assume that it represented the national picture on time of adoption. Second, year of adoption data proved unreliable, with DoNs and PW leads from the same trust often giving conflicting reports. We therefore drew on four sources of data – (1) public data for early pilot sites, (2) the date (if any) provided by the PW lead, (3) the date provided by the DoN and (4) the earliest date that the trust had purchased a PW box set from the NHSI – to reach our ‘best guess’ on adoption year for 141 English acute trusts identified by any or all of these means as having adopted PW. We ranked these sources in the order given above according to perceived reliability. The data provided by DoNs were considered less reliable than those given by PW leads because many DoNs would not have been in post at the time of adoption. The box set purchase data (provided with permission of a PAG member who had worked in the NHSI) were the least reliable because the purchase may not have been followed by implementation. We used the derived ‘best guess’ dates for all adopting English acute trusts (Figure 3) to try to sample case sites estimated to have adopted PW in different years, with the aim of attaining maximum variation.

FIGURE 3.

Rate of PW adoption per year using Rogers’ classification of adopters. 45

In selecting the three ‘early’ adopting trusts, we ideally hoped to recruit three of the five trusts in which we have previously conducted in-depth case studies; this would have enabled us to draw on both the national picture of early adopters from our earlier survey study in 2008 and an existing qualitative data set (comprising 58 transcribed qualitative interviews and documentary materials from 2009 to 2010). These existing data sources would have provided significant contemporaneous insights into the local approaches to early implementation of the PW in these five trusts. However, we were able to recruit only one of the sites that had participated in the earlier study; two of the others had not taken part in the survey and the other two declined our invitation.

As a secondary consideration in sampling our sites, we also hoped to recruit two sites that implemented PW on a whole-hospital basis (the ‘big bang’ approach); two that implemented PW in selected wards only, with no plans for whole-hospital roll-out; and two that had planned for a whole-hospital roll-out from the start. This would have enabled us to compare and contrast what we assumed were likely to be significantly different local approaches to implementation. This was not possible because only two trusts responding to our survey reported implementing PW in some wards only (without a view to further roll-out); they did not agree to be contacted with a view to participating as a case study site. Two trusts reported implementing PW in all wards at once, although, as they were both early adopters, they were not approached, as trusts that had participated in the earlier study were prioritised.

Third, we also considered the following selection characteristics: geographical (e.g. urban/rural), size of trust/number of wards, Care Quality Commission (CQC) rating and age of hospital estate. Within a relatively small sample frame, we sampled for diversity with respect to region and type/size of trust as far as possible, but did not consider other characteristics. Our final set of organisational case studies with sampling characteristics is shown in Table 15.

Within each of the case study sites we randomly selected two wards in which to carry out interviews and observations. The rationale behind random selection was to be able to explore the story of PW in wards that were not suggested or selected on the basis of ‘successful’ implementation, or atypical legacy. Eligible wards were those that had implemented the three PW foundation modules and at least one process module. Additionally, we knew from our earlier research3 that > 59% of implementing wards were medical (24%), surgical (21%) or care of older people (14%). We therefore planned to sample three medical, two surgical and one care of older people wards from the three ‘early’ adopting trusts, with a similar sample in the three ‘late’ adopting trusts. The fact that we recruited sites over a protracted period, and were, therefore, unable to create a single sample frame with precise details of eligible wards, meant that it was impossible to recruit such precise numbers of each ward specialty without restricting choices unreasonably in sites recruited later. Therefore, we randomly selected eligible wards at each site as we went, from a trust-specific sampling frame consisting of eligible wards within these specialties, and ensured that the second ward selected was of a different specialty from the first ward.

Within each randomly selected ward, we purposively sampled a range of staff who would give a range of perspectives on PW implementation and legacy. We interviewed ward managers and other ward staff, focusing, where possible, on those who had been in post at the trust at the time of implementation. We were guided by the ward manager and other ward staff in identifying these longer-serving staff members. When possible, we approached staff directly to ensure that consent to participate was given without obligation; some ward managers preferred to pass on our request.

All interviewees at the sites were given (by e-mail or face to face) a participant information sheet, and at the start of the interview they were asked to sign a consent form. Interviews were recorded (with express permission) and transcribed verbatim. Interviews lasted between 7 and 78 minutes, with a mean length of 33 minutes. The organisational case studies were undertaken partly to allow the findings from the surveys (see National surveys) to be explored within a local context, essentially retracing the ‘story’ of the PW in each case study site. We conducted the following in each of our six case study trusts:

• Semistructured face-to-face or telephone interviews with key staff or PPI representatives identified through purposive sampling47 to give us a range of perspectives on the implementation, assimilation and legacy of PW from board to ward, and within wards, from ward manager to health-care assistants and auxiliary staff (see Report Supplementary Material 3 and 4).

• Structured observations of the ward environment using an observation guide (see Appendix 2) along with informal conversations with ward staff to note legacy on processes that were observable without intrusion (mealtimes, medicines round, patient observations) and evidence of material legacies of the PW (PSAG noticeboards, Safety Crosses notices, medicine trolleys, etc.) and (with permission) to photograph these. The observation template was derived from the ’10-point checklists’ at the end of the PW modules, which encapsulated their goals.

• Short questionnaires to ward managers derived from the ’10-point checklists’ at the end of the PW modules, covering modules and aspects not suited to observation (see Report Supplementary Material 5).

• Collation of local contextual data, including documentary (including electronic) materials relating to PW implementation, and monitoring of impact.

• Completion of a brief pro forma questionnaire (see Appendix 3) to identify the use of ward-level metrics to monitor the impact of PW, followed by a focused assessment of available data.

Fieldwork was carried out between March 2017 and February 2018.

Telephone interviews with early Productive Ward leads

Some of the impacts of PW are likely to inhere in individuals involved in it. To capture these, we carried out telephone interviews with people who led PW at an early stage. Early PW leads were identified from an existing national database of contacts compiled as part of earlier studies of PW at NHS acute trusts carried out by members of the current research team;3,4 snowballing; personal contacts; and Twitter posts. Topics covered with these early PW leads (see Report Supplementary Material 6) included their past involvement with PW, their views on the programme, their experiences of implementation and sustainability, and any impacts that their involvement had on them personally or professionally.

We e-mailed an invitation to participate, along with a participant information sheet, to all contacts. One follow-up e-mail was sent to non-responders. Consent forms were sent ahead of time to all those agreeing in principle, and potential participants were invited to ask any questions beforehand, or at the start of the telephone call, before consenting. Signed consent forms were returned or, if preferred, verbal consent to each item on the consent form was audio-recorded.

Appointments for interviews were made at participants’ convenience. Interviews were carried out by telephone and were audio-recorded. Interviews lasted between 15 and 42 minutes, and on average 25 minutes. Audio files were transcribed verbatim by a third party and then checked for accuracy. Data were collected between February and December 2017.

National surveys

For each survey the following method of analysis was used. The SurveyMonkey data were exported as a Microsoft Excel^® (Microsoft Corporation, Redmond, WA, USA) spreadsheet, which was prepared for analysis by removing blank surveys and by adding identification numbers and descriptive data (region, trust type, bed numbers) to identifiable trusts.

The analysis of the closed questions used descriptive statistics from the summary report generated by SurveyMonkey, and ranges and means were calculated for quantitative questions.

Open-comment fields were coded thematically. Initial broad themes were those that had formed the themes of the survey questions, both open and closed (i.e. implementation type and process; use over time of PW modules and toolkits; patient and carer involvement; perceived impact, etc.) Subthemes, such as difficulties in attributing impact, or inconsistency of impact, were identified through inductive analysis of open responses. This analysis was carried out by Sophie Sarre. A comprehensive report, based on all relevant data under these themes, was reviewed by the research team to allow agreement or refining of themes and to generate any additional questions we might explore.

Organisational case studies

To analyse the interview data, we used the Framework method. 48 Initial themes for the framework were developed from the theoretical literature, the topic guide (itself reflecting theoretical and empirical literature), familiarity with interviews, and the coding of four transcripts by Sophie Sarre. With regard to the theoretical literature, as an integral component of the coding framework, we used three existing frameworks to explore the (1) implementation,38 (2) assimilation32 and (3) sustainability41 of PW in the six case study sites. Each of these frameworks has been described in Chapter 1.

Data from each transcript were summarised under each theme, with the related line number from the transcripts given. Quotations were also included, along with initial interpretive comments. This process meant that data analysis was a combination of deduction (theory-driven exploration) and induction (data-driven generalisation). It also allowed us to examine similarities and differences of views within and between sites and participants.

The initial framework was refined through the following process. The framework was represented in a Microsoft Excel spreadsheet that included a key with definitions of the themes and subthemes. Each of the other four members of the study team (JM, RC, PG and GR) coded a different transcript, drawn from two study sites. Any inconsistencies were discussed to refine definitions, and any additional codes required to capture the full range of data were identified and added. Following this process, Sophie Sarre analysed 12 transcripts from one site using the revised coding frame, and this analysis was presented to the PAG for comment. In addition, a further transcript, from a third site, was coded by three of the four team members (JM, RC and GR). The results were then re-examined for consistency. Any inconsistencies were discussed between Sophie Sarre and Glenn Robert until consensus was reached and the description of that theme was refined. An additional subtheme was added (‘training’). A final, third version of the coding framework was then developed, and tested on one further transcript by two members of the research team. This version was found to be robust and comprehensive enough for use in analysis, and the remaining transcripts were analysed end entered into this coding framework.

The completed coding framework was used to write up descriptive working papers of each of the sites. At this stage, documentary, observation and ward manager questionnaire data were included in the thematic narrative. Analysis occurred through this writing process. Summaries of findings were sent to PW leads at the case study sites for member checking, and any corrections were made to the working papers. At this stage, we also mapped each of the nine Waltz et al. 38 categories (encompassing 73 strategies) for each of the six sites. The longer working papers and our implementation map were used as the basis for the cross-case analysis.

In each of the case study sites, we also reviewed how metrics have been used locally to determine the impact of PW. These reviews were based on our secondary analysis of documentary sources and our earlier review (see Chapter 1, Earlier research into the adoption and implementation of PW in the period 2009–12), and our analysis included collating all local PW-related data in each case study site, judging the rigour and robustness of these data for (1) QI and (2) evaluation purposes, and assessing whether or not they showed improvements over time on the relevant PW modules (and related measures). Our assessment was partly informed by our earlier work, which included empirically-based recommendations relating to the measurement of impact of PW. 3,5 Based on our findings, we present revised recommendations for future data collection in Chapter 9.

We asked each site to complete a short questionnaire about the availability and use of data at a ward level to measure the impact of PW (see Appendix 3). We focused on asking about outcomes, measures or indicators that were either specified or alluded to in PW materials, with additional items specified based on items mentioned in reports, survey responses or common measures of nurse-sensitive indicators (Box 1).

For each of the metrics we asked a series of questions to determine the data that might be available at ward level that had the potential for analysis that moves beyond the simple ‘before and after’ comparisons that are prevalent in the literature on the topic:

• Has the trust collected these data at ward level?

• Has the trust analysed these data specifically to monitor the impact of PW?

• Are there data at more than one time point before implementation?

• Are there data at more than one time point after implementation?

• Has ward-level analysis been reported, within or beyond the trust?

Where trusts indicated that data might be available that would permit a secondary analysis beyond before-and-after comparisons, we enquired further and requested/inspected samples of data. We ultimately determined that additional analysis was possible with data from one trust in which monthly records of module completion and audits were recorded over an 18-month period. These data, and our subsequent analysis, which was exploratory rather than planned, are described in detail in Chapter 6.

Telephone surveys with early Productive Ward leads

An initial coding framework was based on the topic guide and early familiarisation with interviews. A sample of three transcripts were then analysed by a researcher and a member of the research team to check for fit. Any discrepancies in coding were discussed until agreement was reached. At this stage, coding descriptions were clarified and an additional subtheme was added. Coding was then carried out in Microsoft Excel using a Framework approach. 48

Triangulating and combining data

We triangulated data at the points of data collection and analysis. When collecting data, we used case study interviews to explore survey responses for each trust and some of the issues raised across the piece. In the one case study site for which we had existing data, we also re-familiarised ourselves with the earlier analysis and with key transcripts from the previous study, and used interviews with the PW team to follow up on what had happened with respect to work and plans reported in earlier interviews. In case study interviews and telephone interviews, we used our emerging findings in probes and prompts as we went along.

When analysing data, we looked for similarities and differences (1) between the DoN and the PW lead survey responses; (2) between survey findings, case study findings and telephone interview findings; and (3) between case studies. The first two of these are noted in the discussion (see Chapter 8). The cross-case analysis is the subject of Chapter 5.

Study governance

All parts of the study were approved by the Health Research Authority London – Stanmore Research Ethics Committee (reference 16/LO/0918). We made three amendments: a non-substantial amendment to add one trust previously missing from our database of NHS acute trusts; a substantial amendment to widen recruitment routes to the online survey; and a substantial amendment to add observations and the use of photographs in our case study sites. Local R&D approval was not required for the survey or for telephone interviews. It was sought and given in the case study sites.

The PAG was chaired by an independent chairperson and included two PPI representatives, who also gave feedback on the wording of the online surveys, the participant information sheet and the consent form for patients. The PAG convened four times during the study. The research team met approximately every 6 weeks to review progress, agree on plans and contribute to analysis. All members of the research team contributed to ongoing discussions and provided input into the drafting of this report.

Introduction

Our findings are based on descriptive analysis of responses to closed items in the DoN survey, and thematic analysis of qualitative data from the open-response fields (see Chapter 3) and are organised around five phases of the ‘story’ of the PW:

• adoption

• implementation

• assimilation

• sustained impact

• sustainability.

The survey response rate was 37% (56/153). This includes DoNs from four trusts that responded to our survey asked were not identifiable. Because the survey used question routing, not all of the 16 questions were put to all respondents, and not all questions presented achieved a full response rate. We received responses from each of the 13 commissioning areas in England. The representativeness of the sample in terms of the four commissioning regions is shown in Table 4, which also shows the proportion of respondents in terms of trust type.

Adoption

We invited responses from trusts irrespective of whether or not they had ever used PW. We received only two responses from trusts that had never used PW, and from one trust where the DoN did not know whether or not it had been used. The two trusts that did not adopt PW attributed this to using other approaches and having other priorities. Given these other priorities, one elaborated that the project management required for PW was a deciding factor in choosing not to adopt. No trust reported beginning its involvement with PW later than 2012 (Table 5).

Implementation

Thirty-eight (81%) trusts had initially implemented PW in selected wards within a site, and nine (19%) had implemented PW in all wards across a whole hospital or trust (Table 6); the remaining 10 trusts did not respond to this question.

Among the trusts that reported originally implementing PW in ‘some’ wards, the majority (33/35, 94%) said that this was with a view to further roll-out (three trusts did not answer the question on roll-out, and two trusts recorded ‘don’t know’); in 26 of these trusts (79%), this roll-out was reported as having happened (with five reporting that it had not happened and one recording ‘don’t know’).

The criteria on which initial wards were selected were overwhelmingly based on positive ward attributes, predominantly wards where ward managers were keen to use it, wards that were thought able to rise to a challenge, and wards embracing innovation (in one case already using another Lean approach). Few trusts (n = 5) included one or more negative attributes in their selection criteria (wards not doing well in terms of staff well-being, patient outcomes and/or patient experience). No responding trusts relied solely on negative criteria for selecting initial wards.

Just over one-quarter of responding trusts (n = 11, 26%) said that impact measures had regularly been reported to the trust board.

Perceived impact on patients and staff

We asked respondents to indicate the scale of impact of PW (on a five-point scale from ‘a significant negative impact’, through ‘no impact’ to ‘a significant positive impact’) on a number of items. Some items were related to the four core objectives of the PW: patient safety and reliability of care, patient experience, efficiency of care and staff well-being. We also included items related to the wider legacy of PW in terms of staff leadership skills; and increased QI awareness, knowledge and skills. Responses are shown in Table 7 and Figure 4.

FIGURE 4.

Directors of nursing: perceived impact of PW.

Overall, PW was reported as having a positive impact, although for each item a relatively high proportion (between 12% and 36%) of respondents noted no impact arising from PW. Twenty-one of the 41 DoNs reported at least one area in which PW had had a significantly positive impact. A negative impact was noted by three respondents (in three different trusts), and these related to time savings and staff morale. Conversely, one respondent remarked that the PW processes were ‘meaningful to staff at the front line’ (D039). The following open-ended response similarly suggests a sense of purpose engendered by PW as well as the encouragement it provided for networking and sharing between trusts:

Productive Ward was a very exciting project that supported staff to get involved and gave them the tools to make changes. It encouraged networking and sharing between trusts. Staff that were involved with Productive Ward still remember and talk about what they did as well as pointing out the changes that have been sustained and developed further.

D049

Others also praised the clear tools and the way in which PW involved front-line staff.

One of the key mechanisms of impact appears to be standardisation. Displaying standardised information (in this case, information pertaining to safety issues) was an important element:

The Productive Ward was implemented in 2009–2011 across two sites. All wards implemented the 3 foundation modules which consisted of safety cross boards with standardised information, PSAG with standardised information and magnets which significantly assisted with handover and highlighting safety issues to nursing and medical staff. The Well Organised Ward enabled standardisation of clinical rooms, bay cupboards and colour coded labelling of clinical areas. This helped save money, reduce stock and increased the time nurses spent with patients. Several wards implemented other modules, Mealtimes, Handover, Medicines being favourites.

D142

One trust, which used staged implementation during a site renovation process, used the WOW and KHWD modules on initial wards to identify ‘best practice’, which was then used in new wards as they opened. The DoN went on to explain:

This allowed for a degree of standardisation in core support areas such as treatment/clinical rooms and dirty utility rooms. Patient Status at a Glance enabled an agreed way forward for the patient information boards.

D064

The centrality of evidence was noted by another respondent as supporting impact:

PW was an excellent initiative [. . .] It enabled us as an organisation to standardise processes and offered evidence to rationalise decisions and changes that were being made.

D084

Although evidence may have been used to rationalise decisions at either ward or board level, as reported above, only 11 (26%) said that impact measures had been regularly reported to the trust board. In 22 (51%) trusts, such reports had not been made [with one-quarter of DoNs (n = 10, 24%) not knowing]. Some acknowledgement was made that it was difficult to accurately attribute outcomes to PW, especially over time:

Some of the longer-term impacts are difficult to attribute directly to Productive Ward methodology as a number of other factors have been overlaid over time.

D064

One such factor, mentioned by a number of DoNs, is that PW was used alongside other QI initiatives.

Impacts were not consistent within trusts. Open-ended responses point to variation in impact between wards:

Productive Ward wasn’t implemented very successfully or consistently, therefore it is difficult to assess its impact on the organisation as a whole. Certain processes around materials management were improved, and there may have been particular practices that were embedded in some areas, but this is inconsistent across the Trust.

D018

Assimilation

Length of PW use ranged from > 1 year to 7 years, with the average being 3 years (mean, median and mode). In 27 trusts (61%), DoNs thought that staff were no longer ‘regularly using PW modules or toolkits to make improvements in the way they organise and provide care’.

Relevant respondents were asked why the trust had stopped regularly using PW, and given the option of selecting multiple responses. The most common reason given was a change in QI approach (n = 16, 59%). It is clear from open-ended responses throughout the survey that this switch could include adapting the PW methodology, and incorporating it in a wider improvement initiative (see Sustained impact). Resourcing was also an issue, with just over half of those stopping (n = 14, 52%) saying that they had no longer been able to resource it. All but one of these trusts had adopted PW in or before 2009, when central government funding had been available. Two trusts elaborated that PW ceased after the PW leaders left or were made redundant. Other common reasons given were that enthusiasm was too difficult to sustain (n = 10, 37%), and that other work became more of a priority (n = 8, 30%). One respondent elaborated that other priorities took over once they stopped measuring the outputs, and another said that the trust had considered relaunching PW but that other cost-improvement programme projects had taken priority. One respondent suggested that PW had become less relevant as the wider health service policy landscape changed:

This was a good initiative at the time, however things have moved on in terms of embedding assurance processes, change management, model hospital, regulation processes etc. (for example CQC) so this doesn’t have the same impact as it did at its initial launch.

D133

This may also relate to another respondent’s view that PW is ‘an outdated programme’ (D017).

Sixteen trusts (30% of adopting trusts) reported currently and regularly using PW modules or tools. Other trusts wrote about a different type of regular usage, namely the continuing use of display boards or everyday practices developed as part of implementing PW at an earlier time. PW methods could also be revisited as part of a new problem-solving drive.

Sustained impact

We have already noted that the most common reason for ceasing regular use of PW was a shift towards other QI approaches. However, in some trusts PW was still operating in tandem with other QI methodologies, sometimes in an adapted, integrated or possibly synthesised form:

Patient Status at a Glance has developed continuously as part of the wider patient safety work, reduction in length of stay initiatives and the development of ‘board rounds’ to improve communication across the multi-disciplinary team. Much of the quality improvement skills that underpin Productive Ward are utilised in these initiatives but are not discretely labelled as being ‘productive ward’.

D064

Some responses suggest that what remains of PW is not necessarily the precise tools or nomenclature, but its underlying principles:

The Productive Ward paved the way for the Patient First improvement strategy that we have embarked upon. The wards learnt about waste reduction and Lean ways of working, as well as de-cluttering and observation of processes. There are many similarities between the Productive Ward and the work we are now doing, so staff are not starting from the beginning, but building on the work that has been done in the past. It is apparent that those involved in the Productive Ward ‘get’ the Lean ways of working and are able to grasp the principles easily.

D145

The survey explored the influence of PW on trusts’ QI strategy. Twenty adopting trusts (47%) said that PW had informed their QI strategy, while 18 (42%) said that it had not (the remainder did not know). In one trust the PW nursing quality indicators had informed its quality strategy. In two trusts, QI methods drew on PW modules. Another trust reported that PW had featured in the first trust quality strategy but had since been superseded by other strategies that focused more on patient outcomes; another suggested a similar shift in focus.

Sustainability

Dedicated staffing resource was seen as an important factor in sustainability:

There was a Productive Ward Lead who was seconded for 12 months. However, much of the work ceased when the secondment ended and the staff member left the Trust.

D018

Having funding to pay for additional nursing cover (‘backfill’) while ward staff spent time on PW activities is a significant facilitator of any QI programme, albeit an expensive one. However, backfill was not in itself sufficient to maintain enthusiasm without the commitment of ward managers:

It was very challenging because even with backfill staffing levels meant that worked planned had to be rescheduled.

D049

Previous research has shown that support at board level is important. One response implied that, despite this support, the fact that PW effectively operated in ward silos necessarily limited its sustainability, and it perhaps needed a longer-term commitment from the board. With ongoing staff turnover, the need to train and involve new staff in the ways of PW was also a challenge.

Over one-third of trusts (n = 17, 39%) had plans to continue using PW and/or to incorporate elements of it into their QI strategy. One trust was considering applying PW principles in new clinical areas, and one had plans to roll out PW to other sites within the trust. It appears that a priority for trusts is to have a QI strategy that can be applied more broadly across a trust, a demand that some DoNs felt PW’s focus on wards did not meet.

Introduction

As with the DoN survey, our findings are based on descriptive analysis of responses to closed items in the PW leads survey, and thematic analysis of qualitative data from the open response fields. The findings are organised again around five phases of the ‘story’ of PW:

• adoption

• implementation

• assimilation

• sustained impact

• sustainability.

A total of 57 PW leads working in eligible trusts were identified and invited to complete the survey. We received 35 completed surveys, a response rate of 61%. Table 8 shows the characteristics of participating trusts, and indicates their representativeness in terms of region and type. The sample was slightly over-representative of trusts in the north of England, and slightly under-representative of trusts in all other regions. It was less representative in terms of hospital type, over-representing teaching hospitals.

In terms of PW lead respondents, their personal involvement ranged from approximately 1 to 8 years at the time of the survey, with a mean of 3.6 years. Those in the sample were currently working in a role that was primarily ‘nursing care’ (n = 9, 26%), ‘service improvement’ (n = 8, 23%), ‘practice development (n = 5, 14%), ‘patient quality/safety’ (n = 4, 11%) and ‘other’ (n = 2, 5%); seven (20%) did not respond to this question.

Adoption

The majority of trusts were reported to have had first used PW in 2009 (n = 16, 46%) or 2008 (n = 11, 31%). The earliest reported use was 2007 and the latest was 2012.

Implementation

In the vast majority of trusts (n = 31, 89%) PW was implemented using a phased approach (i.e. on some wards but with further roll-out planned). Two trusts implemented PW in all wards at once, and two implemented it only in selected wards with no intention of further roll-out. To capture the rationale behind a phased implementation approach, these 31 respondents were given four closed options and an ‘other’, and invited to select all that applied. The results are shown in Table 9.

Open responses pointed to a phased approach being chosen because of (1) the capacity needed to support staff in implementing PW and/or (2) the desire to use the learning from initial wards to facilitate PW during roll-out. In 26 (84%) of the trusts using a phased approach, roll-out across the whole trust was achieved.

Respondents were presented with a list of possible criteria and asked to indicate which ones were used to select initial wards. Ward selection was most commonly based on positive attributes such as keen ward managers, wards known for embracing innovation, and wards thought able to rise to a challenge. In a minority of trusts (n = 3), wards initially selected for PW were those not doing well in terms of one or all of the following: staff well-being, patient outcomes or patient experience. Open responses showed that some trusts used a competitive process, and selected or shortlisted wards based on their enthusiasm for PW. Three trusts used a strategic approach (i.e. including wards from different clinical areas to gauge the requirements of different specialisms). One trust was undergoing structural and organisational changes, and used PW on the wards that were due to be in the vanguard of the new set-up.

Staff involvement

Productive Ward was designed to be used by all levels of ward-based staff, and supported by management. The box set contains guides for executive leaders, project leaders and ward leaders, as well as the module guides and toolkit to be used by ward staff. Some of the actions that staff using the PW might decide to implement could involve non-ward staff such as medics, allied health professionals (e.g. physiotherapists, speech and language therapists), administrative and clerical staff, ancillary staff (e.g. porters, kitchen staff, domestic staff), estates staff, or health scientists such as phlebotomists.

The survey asked about the extent of involvement of different staff groups. Ward-based staff were the most involved, with senior nurses (bands 7 and 8), other nurses (bands 5–6) and health-care assistants (in that order) being described as ‘very involved’, followed by senior managers (e.g. DoNs). The involvement of medical staff was relatively low and behind that of estates, ancillary staff, allied health professionals, and administrative and clerical staff (Figure 5).

FIGURE 5.

Extent of involvement of different staff groups. HCA, health-care assistant; RN, registered nurse.

Perceived impact on patients and staff

We asked respondents to indicate the scale of impact (on a five-point Likert scale from ‘significantly negative’, through ‘no impact’ to ‘significantly positive’) on a number of items. Some items were related to the four core objectives of the PW: patient safety and reliability of care, patient experience, efficiency of care and staff well-being. We also included items related to the wider legacy of PW in terms of staff leadership skills, and increased QI awareness, knowledge and skills. Responses are shown in Table 11 and Figure 6.

FIGURE 6.

The PW leads: perceived impact of PW.

No negative impacts were noted. For each item, some respondents (between two and eight, 7–27%) reported no impact. Twenty-one PW leads described at least one module as having had a significantly positive impact.

The most positive impact reported was on the physical environment and on patient care-related factors (i.e. patient safety, patient experience, nursing time spent with patients, quality of care and care processes). Although ‘nursing time spent with patients’ was reported as having been affected positively by 28 PW leads (93%), there was less impact on the quality of patient care. Nearly one-quarter of PW leads (n = 7, 23%) said that there had been no impact on this, and 23 (77%) reported a positive impact.

The group of indicators with the next largest positive impact was savings (waste reduction, and material and time savings). The lowest impact was on staff. Although eight (27%) reported an impact on staff QI skills, there was little impact on leadership skills, possibly reflecting the fact that there were relatively few members of staff who would have been in a leadership role. The impact on staff morale was relatively low.

A separate question, using statements and a five-point Likert scale, included five statements related to impact. The results of these are shown in Table 12. Over three-quarters of PW leads (n = 23, 77%) agreed or strongly agreed with the statement: ‘The Productive Ward has had some positive impact wherever it has been used in the trust’. A similar number (n = 24, 80%) agreed/strongly agreed that PW ‘has worked well for some wards in the trust’.

Assimilation

Sustained impact

Sustainability

We presented respondents with a number of statements about their enthusiasm for PW and the current situation regarding support and engagement for PW at their trust. They were invited to indicate their level of agreement or disagreement using a Likert scale: strongly agree, agree, neither agree nor disagree, disagree, strongly disagree or don’t know. Thirty PW leads responded to each statement (Table 13).

Despite a majority view that PW fitted well with trusts’ aims and objectives, examples of current financial support and other sources of support for PW appear rare. In the majority of trusts, PW leads agreed (n = 15, 50%) or strongly agreed (n = 3, 10%) with the statement ‘The Productive Ward fits well with what we want to do in this trust’. There was also strong disagreement with the statement ‘Productive Ward was a waste of time and energy’. Only two PW leads agreed with this statement (one strongly), whereas 25 (83%) disagreed or strongly disagreed. However, it is also true that the majority of trusts no longer had the funding, or a PW champion, to support PW. Twenty-six PW leads (87%) indicated that funding to support PW was not currently available in the trust. The remaining four neither agreed nor disagreed (n = 2), or did not know (n = 2).

In only six trusts (20%), PW leads reported a clear PW champion, while there was no such person in 22 trusts (73%). In a minority of trusts (n = 3, 10%) PW is currently reported to be facilitated by the trust’s ‘change team’. Over half of the PW leads agreed/strongly agreed that they would ‘like to see PW re-invigorated at the trust’ (n = 16, 53%) but many (n = 7, 23%) were non-committal, and five (16%) disagreed or strongly disagreed.

Current engagement with PW was also weak. Although five trusts reported involving patients and carers at some point (see Implementation in this section), no trusts claimed strong patient and carer involvement in their current implementation of PW, with 27 (90%) denying that there was currently strong patient and carer involvement. Only three PW leads (10%) indicated that their trust was ‘currently sharing ideas and knowledge with other hospitals implementing PW so that we all benefit from each other’s learning’ (see Sustained impact in this section); PW leads from 20 trusts (67%) disagreed or strongly disagreed with this statement.

Implementation strategies

As described in Chapter 1, the PW toolkit included detailed guidance on various strategies for implementing, spreading and sustaining the PW. In this section, we use Waltz et al. ’s38 nine ‘clusters’ (see Chapter 1, Frameworks for understanding local approaches to Productive Ward implementation), encompassing 73 discrete implementation strategies, to compare and contrast the respective approaches taken by each of the six case study sites as they implemented PW. In Chapter 8, we reflect on both how closely these approaches followed the PW implementation guidance and the extent to which the differences in approaches may explain how well PW was assimilated into routine practice and sustained. Our overall findings relating to each of the nine ‘clusters’ of strategies are outlined in the following sections.

Resourcing

In both the theoretical literature and our analysis, various aspects of resourcing appeared highly significant in terms of how PW was implemented, and the nature and extent of any sustained legacies over time. Sites A, B, C and E received external funding from either NHSI (if it was a pilot site) or their SHA. In terms of the level of funding and support from both external and internal trust resources, the two early adopter sites (A and C) received significantly more of each and over a longer duration (see Table 17). The late majority (site D) and laggard (site F) sites received much less in terms of internal resources (and no support at all from external sources). Internal trust funding indicates a level of commitment from the trust that might bode well for later assimilation and sustainability. On the other hand, a complete lack of external funding may mean that overall resources remain insufficient. PW lead and PW facilitator posts were funded for between 1 and 2 years initially. Beyond the initial implementation period, two sites (C and E) extended the life of the PW implementation team by gaining further external funding, and, at a third site (A), via internal funding. Across our sample, a smaller or larger PW ‘team’ remained in post for 2, 2.5, 3 and 4 years.

As Table 17 shows, the total number of staff employed to implement PW at any one time varied. While there was a marked difference in resourcing between the two early adopter (sites A and C) and the four remaining sites, it should be noted that there was large variation in the number of wards in the roll-out plan, the number of modules that wards were expected to implement, and the duration for which implementation teams were funded (Table 18). Accounting for these factors, PW leads in sites A and B were at opposite ends of the scale: those in site A were the most stretched, whereas site B (a specialist trust), with far fewer wards and fewer modules, had a proportionally higher level of resourcing. It is also worth noting that in sites B and C, where PW leads had some of the lightest workloads, ward managers had protected time to implement PW (in site B this was as much as 1 day per week for 1 year). Some wards in site C received practical help from their practice educator.

‘Implementation experts’ are suggested to be another key resource. 37,38 Five of our sites received external support from implementation experts, either specifically in PW (sites A, B, C and E) or in Lean (site D); site A described this support as ‘instrumental’ in its achievements. Site B purchased a support package from NHSI, which included 4 days’ training for the PW team, the executive sponsor for PW and eight ward managers (i.e. nearly all of the wards in its roll-out plan). Site D was unique in that non-nursing staff received training. In sites E and F, no such expert training was received. Of all of the sites, site F had the least support as, unlike site E, there were not even any SHA-run networking events with other implementing trusts for it to attend and learn from.

Use of evaluative strategies

All sites used some form of evaluation to allocate wards to their PW implementation cohorts, but these strategies varied. Sites A and B used the NHSI selection criteria (see Chapter 1, The ‘Productive Ward: Releasing Time to Care’™ programme), but only Site A invited all interested wards to apply to be ‘showcase’ wards. Site B relied on recommendations from matrons, whereas site C – which implemented in selected wards only – first targeted wards with high levels of emergency admissions, and then randomly selected wards within these; and later selected wards that had critical incidents, as these were thought to provide valuable learning opportunities. Site E started with wards with stable staffing, while site D and F selected the wards that had expressed the most interest. In site D, one PW lead thought, in retrospect, that in their roll-out plan they had not taken sufficient account of ward readiness or of what wards needed for successful implementation.

Sites ‘RAG (red, amber, green)-rated’ wards on a spreadsheet to evaluate their progress with implementing at least mandatory modules (Figure 7 provides an example from site F). Site B monitored tasks to complete, obstacles and actions, but sites D and F were particularly strong on evaluating implementation over time. This is likely to reflect that those leading on PW were already a QI team (site D) or situated within a change management team (site F), which PW leads at site F credited with giving them good project management skills. Site D created an audit system (described in the following paragraph). Site F carried out baseline Activity Follow measures, which categorised activities, counted interruptions and tracked mileage, for almost every ward in the roll-out plan. (Ward managers were tasked with carrying out follow-up Activity Follows, and, according to implementation data provide by the trust, completion of follow-up Activity Follows faltered considerably in cohort 3, and was not done at all in cohorts 4–8.) The PW facilitators at site F also prepared monthly progress reports, identifying key issues, critical tasks to be accomplished by set dates, and risks (see Appendix 10). Other evaluation activities included an evaluation form given to ward staff, and inspections of store rooms and cupboards by the PW facilitators. The implementation strategy at site F was strong on testing and on standardisation. They tested modules on one ward, and then a couple of wards, using Plan, Do, Study, Act (PDSA) cycles, and then imported the output to all wards. [After the implementation period in site C, once it had developed its own adaptation of PW (see Assimilation), its evaluation system became very robust.]

FIGURE 7.

A RAG-rated roll-out plan (site F).

Of the six sites, only site D attempted to build in sustainability from the start of its implementation strategy. After each ward had implemented the mandatory modules, the PW team would introduce ‘sustainability tools’. This consisted of carrying out monthly spot checks and scoring wards against audit sheets created by the implementation team. A ‘leaderboard’ created an element of competition between wards, and the results were reported to the head of nursing, who supported Trust Way (the trust’s adaptation of PW; see Tailoring implementation: adapting Productive Ward); she reported to the DoN, who reported to the executive board. The team would issue actions to improve based on the scores. They later introduced quality control boards, which were described as ‘our ultimate sustainment tool’ (D01). The boards provided another system of audit and surveillance:

The Quality Control Board had five levels of checks. On a daily basis, the ward staff on sort of the shop floor were asked to check things, again for example has the Comms Cell been filled in today or have the stock levels been maintained. And then on a weekly basis the ward manager would check that, monthly basis the matron would check that, quarterly basis the head of nursing would check that and on 6-monthly basis the director of nursing would check that.

D01

This tiered system of auditing and reporting continued for another year after the last cohort had implemented Trust Way.

Developing stakeholder relationships

There was no evidence of a steering group in site D. We do not have consistent data on the composition of steering groups in the remaining five sites. However, we do know that in site C the group was chaired by the chief nurse and that the attendees included a patient representative and representatives from participating wards; and that site A included the chief executive and site F included some central services (facilities and pharmacy). In sites A and B, PW’s implementation progress was reported on an ongoing basis to the executive board, and board members visited wards to see and talk to staff about PW progress. This did not appear to be the case in the other four sites (although our data on this implementation aspect are not robust; see Chapter 8, Limitations).

Sites F and D were unusual in having strategic relationships with central services. In site F, as mentioned, facilities and pharmacy had been included in the steering group, and, in site D, pharmacy, procurement and IT also attended the Lean training. In other sites the pattern was for PW teams to engage central services over time. This was done either when they were deemed necessary or, as happened frequently, after making changes had led to angry responses from the services or teams affected (e.g. pharmacy being inundated with out-of-date stock during a WOW-related clear-out, or the infection prevention and control team disapproving of noticeboards or equipment introduced as a result of PW). Relationships with central services were most successfully developed in site A, particularly under the later trust-wide implementation model that they adopted (see Tailoring implementation: adapting Productive Ward), which enabled central service leads to meet staff from all wards at once. This allowed ward staff to understand the processes beyond the ward, and to make changes in the light of these.

The earliest adopting sites (sites A, B and C), which had either helped develop the materials/toolkit or received extensive training from the original designers of PW, had better engagement across staff bands and groups than other sites. Some wards in sites A and B were successful in involving allied health professionals (AHPs) at ward level in implementing the PSAG board. By contrast, in site C, AHPs were not involved in designing the PSAG, and had started using it only once auditing of their compliance had started. In site F, one PW facilitator reflected that they should probably have made more effort at the start to engage non-nurse members of the multidisciplinary team:

Given that this was a ward-based initiative, I think we did include as many ward staff as we could, so we tried to include therapists where we could. I wonder with hindsight whether – and given the impact it did have, whether it should have had more of a MDT [multidisciplinary team] launch.

F07

It appears that medical staff were not on the steering group in any of the sites and, although there were other intervening factors, such as turnover of junior doctors, the schedules of surgeons and attributed cultural differences, this was likely to be reflected in their lack of engagement at ward level, which was a consistent – and important – finding across all six of our sites. A few interviewees across sites reported that PW was perhaps regarded, as one interviewee put it, as ‘a very nursey nursey project’ (C07), which might have inhibited broader engagement and affected sustainability.

Site D was unique in that the team implementing Trust Way (its adaptation of PW; see Tailoring implementation: adapting Productive Ward) were not nurses but rather an integral part of the wider QI team within the organisation. However, in all of our sites, ward staff reported a preference for nurses in the PW team (and, later in the implementation, site D did second a clinical lead to work on specific modules of Trust Way). As one nurse remembered:

The people who came in with the initiative they were like mainly office-based people and they’d come in and said ‘Right, we’re going to do this’. And a lot of people were like ‘Well you’ve come from offices. NHS doesn’t work like that, it’s a completely different set-up’. [. . .] They came in and started putting up boards everywhere, stickers on the floor and we were like ‘what are you doing?!’

D11

Sites A and B (and, to a lesser extent, site E) fostered collaboration and knowledge exchange between wards. In site B – uniquely – regular group meetings were held between the PW lead and all ward managers; these occasionally included central services.

Sites and wards varied in the reach of engagement within nursing teams in terms of bands. Interviews with ward staff suggested that there had been minimal engagement with more junior nurses and health-care assistants in sites C, D and F, which had implications for the legacy of PW.

Engaging patients, family members and the general public

There was PPI representation on the steering group in sites A, C and F but not in sites B or E. There was no steering group in site D, nor was there strategic PPI by any other means. The two PW PPI representatives in site F were asked what information should be displayed on the ward. They also carried out unstructured observations on wards, looking at what staff were doing (task allocation, and whether tasks overlapped) and communication with patients, and providing an outside perspective on the placement of storage. The patient representative we interviewed thought that she had helped staff to see things from a patient’s perspective (i.e. why patients should have a protected mealtime, or why they might not want details of their care discussed in front of a visitor). The patient representative we spoke to thought that they were appreciated by the ward staff and the PW team, one of whom commented that the representative helped to keep a focus on direct care time:

They kept us focused on the direct patient care time. I could have spent 3 years just talking to pharmacy, just dealing with logistics, or dealing with NHS procurement. I could have spent 3 years doing that. And actually [the PPI members] brought us back to ‘How have we released the time and can we quantify this time? And what are the nurses doing?’

F01

Patient and public involvement was relatively strong in site A, which was said to have a history of PPI. The PPI representative said that the scope of involvement (in terms of both trust-wide scope and the scope of the things they were looking at) was innovative, and she felt that patients ‘had a big influence’. The PW PPI representative fed back patient views up to the steering group, and steering group issues down to the patient participation group. For instance, the patient participation group was asked for its input on PW posters and was invited to review the KHWD boards. The representative could also mobilise patient groups in different areas, so she could get an ‘army’ of volunteers when necessary. Patient representatives would go on to wards, meet patients and carers and have ‘before and after PW changes’ conversations. They asked for feedback about the KHWD boards: what do you think, do you read them, do you ask questions about them? The findings from the patient participation group and ward visits were fed back to the PW steering group, which, the PW PPI representative felt, gave them a voice ‘right up at the top’. At the PW PPI representative’s suggestion, they carried out several patient-focused Activity Follows alongside the staff ones, at different times of day.

Except for a small number of wards across sites, there was no or minimal involvement of inpatients or visitors by ward teams. Examples of the latter were using general patient feedback (from ad hoc conversations or Friends and Family Test comments) to inform PW; inviting comment on the KHWD board; and, in a very few cases, surveying patients. We interviewed one member of the public who had spent 7 months staying on what we knew was an implementing ward in site B as a carer for her son (who was the patient), as well as having multiple shorter stays. She had never heard of PW before being contacted by the study team. She would have been a prime candidate for enrolment in PW, and had ‘laboured over’ making some suggestions for improving patient and visitor experience, but she and her husband had never been approached for input. She explained:

I was really surprised when you said about the Productive Ward because I thought I’ve been on this ward for longer than some nurses, I’ve never heard of this word. And I’m thinking why haven’t I heard about it because I’m not a nurse on that ward, but it would have been really good if the staff who were doing this Productive Ward had got my husband and my ideas on how things could be improved [. . .] We were 7 months in [name of ward], and there were so many things, you know, during the time which I just thought I can’t believe that I can’t suggest something, you know? [. . .] We tried to improve things but nobody ever came and said ‘it would be really helpful after your 7 months here, now you’re going home, have you got any suggestions?’ We would have had heaps of suggestions, you see. They didn’t ask.

B15

Several interviewees, when reflecting, said that this lack of involvement of inpatients or visitors was an opportunity lost:

I think the bit that I don’t think we could say hand on heart we certainly got in any organised way was actually ‘Have the patients noticed a difference?’

B05

Even in site A, where the involvement of inpatients was highest, this was felt to be wanting.

Training ward staff

As well as direct input from experts to at least some ward teams and/or ward managers in sites B, D and E, training of ward teams and/or ward managers was also part of the remit of the internal PW teams. Site E’s training model was to provide classroom-based training to ward managers or nominated ward leads, and to give presentations on wards. All other sites began by going on to wards and training and (see Providing interactive assistance) supporting ward managers and (to varying degrees) other staff. After 2 years, sites A and C moved to a trust-wide system, whereby all wards were trained together in one module at a time. Accounts of the detail of training given and received vary between participants, with ward staff often not having had or not remembering training that PW leads reported that they had provided. However, there are indications both that (1) training delivered to wards diminished over the course of the roll-out and (2) training in PW principles – as opposed to training in how to use tools – was relatively weak. If we adopt ward staff engagement in improvement thinking as a proxy for an understanding of PW principles, then site B may be an exception:

Is it because of Productive Ward that we got into this new way of thinking that actually what else can we streamline? And it changes your mind thought doesn’t it, and thought processes? [Makes you] more flexible . . . it’s very easy to get stuck in a rut isn’t it with your routines and things and actually to look at a new way of working and think actually yeah we’re open to change, bring it on!

B07, R3

This may well reflect that 8 out of 10 ward managers had received direct training and also the relatively light workload of the PW lead. The reach of training (whether training was given only to ward managers and deputies, or whether all bands of staff were trained) is also important. Either by design (sites C and E) or in practice (sites D and F), training was not delivered to all bands by the PW team; the evidence suggested that a reliance on ward managers passing on training was not successful. Similarly, in three of the four sites where wards were supported to implement only a limited number of modules, and were then expected to carry on on their own (sites B, C, E and F), this rarely happened; site B was the exception. One PW lead reported:

We did the three foundation modules and then Medicines and Meals and then, theoretically, the wards would have the skills to be able to go on and do the remaining modules without that intense facilitator support . . . but then it became quite clear that as soon as we’d come away, that just wasn’t really carrying on.

C01, R2

Providing interactive assistance

Productive Ward teams in all of the six case study sites provided interactive assistance to wards, which was a fundamental element of the PW approach to implementation. Among the sites for which we have data on the amount of time PW leads or facilitators spent with each ward, this time ranged from 2 to 12 weeks. The nature of PW makes the boundaries between training and interactive assistance somewhat blurred. The assistance given comprised introducing PW, engaging ward staff, helping ward staff to carry out activities using the PW tools (i.e. 5S game, Process Mapping, Video Waste Walk, timing activities, Activity Follow), observing processes, giving suggestions, and providing hands-on help with WOW. In site C, some wards received additional help with these activities from their practice educators. In site F, where implementation was driven by the impending move, the PW facilitators took on a particularly heavy burden of the work by carrying out the vast majority of baseline Activity Follows, and, with respect to WOW, labelling, colour-coding, and getting rid of out-of-date stock. Although ward staff said that this had made implementation fairly rapid, some worry was expressed about whether the facilitators became too involved in the mundane work associated with PW to the detriment of ward teams’ investment in the programme:

I remember going to the orthopaedic ward where we spent an awful lot of time on storage and labelling and trying to empower staff to think about ways of how to do the storage. But of course they were all very busy and because [PW facilitator 1] was a doer, [PW facilitator 1] would do it for them. And that made me anxious because there’ll be lack of investment then from the team if someone comes in and does it for you.

F05

We do not have sufficiently consistent data on the detail of this support to draw strong inferences but, as with formal training for ward managers and ward staff, the assistance offered to wards at all sites seems to have declined significantly as the roll-out continued, either through moving to a module-by-module trust-wide system training that reduced the presence of PW leads on wards, or simply because of lack of time:

So what we did [with WOW] was we gave them a day of our time. So we would have both gone, that was the day that we did it. The other ones, like Knowing How We’re Doing and PSAG . . . some of them [wards], it was like ‘This is a general introduction what we’re going to do and we need to change it’, and every once in a while ‘Actually this has got to happen, we’re delivering a board’. So some of the later ones [wards] we [just] dropped off the board.

C07

Tailoring implementation: adapting Productive Ward

Waltz et al. 38 use the unintuitive term ‘accept and tailor to context’ to describe the ‘cluster’ of strategies that include tailoring the implementation strategies to address barriers and leverage facilitators; and identifying the ways in which the innovation (in our case PW) can be tailored to meet local needs, and clarifying which elements must be maintained to preserve fidelity (as well as using data experts to inform management on the use of data generated; and integrating data systems to facilitate spread within and between organisations). In the story of PW, the first two of these are relevant, and we discuss them here. (The slightly different use of data experts to support ward staff is discussed in Developing systems to support ward staff.)

Around 2 years into implementation, site A, a large trust, changed its implementation strategy, partly in response to the reduction in the number of PW facilitators to two (the other two having been moved on to adapting PW for use in the emergency department and implementing The Productive Operating Theatre), and partly because it wanted to take a whole-hospital approach to improvement. From this point, instead of facilitators working with and on individual wards to implement modules of their choosing, the new strategy was to deliver trust-wide training en masse, module by module (only process modules were delivered at this stage), with representatives from all wards getting 3 days’ training per module over several weeks. This enabled better relationships between wards and central services (see Developing stakeholder relationships), and they started ‘looking at whole hospital issues rather than just looking at what affected the wards’ (A14). Mass training was also useful for introducing trust-wide policies (i.e. Protected Mealtimes, Patient Observations using the Early Warning Score). What was lost, according to one PW facilitator, was a tailored approach to individual wards. According to the one interviewee we spoke to about this training model, it was not helpful for ward staff. This ward manager felt that there was an assumption that they knew what it was all about:

My staff and my deputies sort of went along to some of the sort of initial induction days, or whatever you call them, and the feeling was ‘well you know what you’re on with’. But our feeling was well no we don’t. We didn’t feel that we had a great deal of guidance with it. [. . .] We came to Productive Ward late and there was kind of an assumption that you know what you’re on with, which we didn’t because we hadn’t undergone it ourselves.

A06

Elements of PW were tailored. First, with respect to the integrity of PW modules as a programme, site A was the only site where implementing all of the PW modules was made mandatory for all wards in the roll-out plan. Although wards were initially allowed to choose the sequence in which they implemented the process modules in order to best suit their needs, this changed when site A moved to the mass, module-by-module, training model. In the remaining five sites, not all of the process modules were mandatory (although sites C, D and F did make some process modules, or aspects of them, mandatory). In sites B and E, wards were allowed to choose whether or not and which process modules to implement (and which elements). Second, changes were made to the use of the Activity Follow observation tool, which was a key tool in PW implementation (see Appendix 1). It was designed to capture how much staff time and movement was spent on direct care time and on other ward activities by measuring the task, activity and location of a staff member each minute during an observation period. This required two people: one to time activities and the other to map movement using a spaghetti diagram. It was prescribed as forming baseline data for the KHWD module; and one member of staff was to be followed for 12 hours initially (and 8 hours thereafter). Activity Follow was clearly extremely resource-intensive, and not all staff could see the point of it. In response to these concerns, PW teams in sites C, E and F reduced the number of hours required for Activity Follows, and/or followed different members of staff over the observation period. Some wards took it upon themselves to make adaptations, such as using student nurses to carry out Activity Follows, and the lack of follow-up Activity Follow data across sites suggests that they simply stopped doing it. This was understandable, according to one executive interviewee:

There’s no getting away from it, if you’re not going to resource that specific requirement in terms of a particular way of monitoring something [such as Activity Follows], a particular way of reporting it, then that’s likely to fall by the wayside.

B05

Site D had a unique story with respect to adapting PW. Accounts suggest that, acting with no reference to the nascent implementation of PW, the trust contracted Unipart Expert Practices (Oxford, UK) to work with the QI team on a transformation programme, ‘Trust Way’, based on Lean. Working on Trust Way, the QI team was visiting wards to look at pathways and processes. The two worlds of nursing and service improvement practically bumped into each other on the wards and realised that they were doing very similar work:

Productive Ward had kind of come in from a very nursing level onto the wards but we [QI team], at that stage, we hadn’t really had much to do with it. But it became apparent very quickly that actually the work that we were looking at with Unipart was very, very similar to the Productive Ward.

D04

The overlap in aims, and the shortage of resources, resulted in bringing PW under the Trust Way banner. Those wards that had implemented PW were told by the DoN to ‘put things on hold’ until Trust Way was rolled out. PW was seen as useful for the purposes of presenting Trust Way to ward staff. According to the QI manager, Trust Way was to be presented ‘under the guise of Productive Ward because that’s what the nursing staff were familiar with’ (D04). She went on to explain:

It was much better that it was Productive Ward, not Unipart, because it was already owned by the nursing staff [. . .]. Productive Ward translated a lot of the theory into practical.

D04

It was also felt that PW was more credible to nursing staff, as it was rooted in health care, not industry; and it was being used nationally. Nevertheless, Unipart had its own tools and language, some of which Trust Way adopted. The end result was that Trust Way was a merger of Unipart and PW.

More generally, with respect to tailoring to local needs, there is an inherent tension in PW between the aims to standardise systems and the aims to give ward teams the autonomy to find their own solutions to local problems. This often played out as a tension between trusts’ desire for standardisation and wards’ desire to tailor PW outcomes to their local contexts. Uniquely, in site F, all aspects of implementation were driven by the logic of standardisation from the outset (as a result of the merger of the two hospital sites as the raison d’être of the project); there was no attempt to allow individual wards to develop their own solutions. Site B’s desire to standardise increased over the course of the roll-out. In site D, some wards imported solutions from others. Several of the specific outputs from PW implementation also lent themselves naturally to standardisation; for example, the design and use of WOW was standardised across all six sites. PSAG was a more complex story, whereas in site B standardisation of PSAG did occur over time, at the request of senior nurses, in site D PSAG became less standardised over time. In site F, the standardisation of PSAG was still being resisted at the time of fieldwork:

[WOW] was the bit everybody liked – they could walk into a cupboard and find what they needed to. That well-organised ward works. The tension comes when you’ve got things like Patient Status at a Glance boards. Some wards really don’t care if a social worker’s involved because they’re admissions. They don’t need it. You’re only there for 12 hours. But what they need to know is that somebody’s been clerked. So standardisation doesn’t work for things like that and we are still, we’ve still got teams involved making those wards bespoke. [. . .] The tension comes when you try to standardise where there isn’t a standard, and information’s one of them.

F01

In sites A and F in particular, there was a move towards more prescription over outputs over time, which was, at least in part, related to the clock ticking:

So our wider brief initially was the three baseline modules plus one process module. All wards eventually did the three baseline modules either by willing to be involved with us or it just happened top down from the directorate because we were running out of time to get stuff done in time for the hospital move. So things like we standardised the PSAG boards and it came a point well actually this has got to just happen. So although that goes against the Productive Ward bottom up theory, once we’ve tried and tested it in a couple of areas, it was just bang, out it goes.

F07

Developing systems to support ward staff

Site A was unique in employing a project support officer principally to develop a dashboard to allow wards to input metrics data, and which allowed them to generate graphs to see patterns over time. (Although the data were collated centrally and reported to directorates, in practice data collection and analysis was primarily regarded as a way for wards to spot their own weaknesses and successes.) Site D developed a quality control board for wards to use to sustain Trust Way (see Use of evaluative strategies). Site D also developed an electronic PSAG board. In sites A and F, an electronic PSAG board had been developed after the initial implementation period; in site F this was a sophisticated e-’flow’ board.

Changing infrastructure

Four sites had existing service improvement/transformation/change management teams that supported PW implementation in some way. Site D was unique in that the PW team members had QI, not nursing, backgrounds, and were drawn from the existing QI team. In site E, a member of the QI team provided some additional ‘light touch’ training. In site F, the team sat within a ‘change’ team that was formed to work on the trust’s move from two sites to a new-build, single-site hospital. This was seen to encourage good governance and reporting practices:

When we were running the PW project we were working out of a project office, in amongst all the other change programmes leading up to the move, and it was run along strong project guidelines. This I suspect generated good project structure in the form of highlight reports and steering groups.

C07

Overview of implementation strategies

The timing of PW adoption had significant consequences for resourcing implementation. Trusts that had adopted PW earlier had access to much greater, particularly external, resources to aid implementation. There was very large variation between our six case study sites in terms of the number of modules that wards were expected to implement; the ‘person-hours’ dedicated to implementation; and, hence, the workload of PW teams.

Applying Waltz et al. ’s38 categorisation of implementation strategies revealed that, although some elements of local approaches were common (i.e. providing interactive assistance and low levels of PPI), there were significant variations that had important consequences for the nature of the assimilation of PW into routine practice and, subsequently, for the sustainability and legacies of the programme over a decade. Fidelity to the approach to implementation recommended by the NHSI was remarkably similar across most items (including a lack of fidelity regarding the strategic engagement beyond nursing in the steering group). Sites differed in three key areas. First, site A was the only site to follow guidance in its methods for selecting early ‘showcase wards’. Second, the earliest adopting sites, which had either helped develop the materials/toolkit or received extensive training from the original designers of PW (sites A, B and C), had better engagement across staff bands and groups than other sites. Third, it is worth noting that, although not expressly addressed in the guidance, there was an implicit assumption that all modules should be implemented. However, only two sites (A and E) included all modules in their roll-out plan.

Some trusts situated their PW teams in wider QI/change/transformation teams, which enabled better monitoring of implementation progress. Nevertheless, only one of our six case study sites explicitly considered PW sustainability as part of its implementation approach.

It is important to note that there was as much variation within sites (i.e. between wards) as there was between the six sites. This seemed to be a temporal issue. Quite quickly, implementation efforts became laboured and PW lead teams became weary as they recognised that they were (1) running out of time to implement all of the mandatory modules in their targeted number of wards, and (2) had typically (we can surmise in most sites) left the most challenging wards (and often the most challenging modules) until last. As a consequence, implementation strategies changed over time; for example, site A had moved away from providing intensive, tailored support to small cohorts of wards and towards (much less in-depth) mass training for 72 wards, module by module. In the majority of sites, there was a noticeable shift away from empowering ward staff to take ownership of the programme and implement the modules themselves (a fundamental design principle underpinning PW) and towards a range of short cuts driven by time constraints and the logic of standardisation.

Physical environment

The most instantly recalled and most common impact of PW was improvements to the physical environment of participating wards. All sites introduced like-with-like colour-coding systems for storage areas (some wards used photographs to show what went where); and most wards in all sites introduced labels and/or floor stickers to signal designated areas for equipment. These were credited with making items easier and quicker to find, particularly for new or temporary staff, and had the added effect of reducing interruptions of other staff. Wards got rid of out-of-date stock, and some rationalised medicines in medicine trollies. In many wards across sites, additional equipment was purchased, for example extra observations machines, trollies or bedside lockers to reduce time going to and fro.

Visual signposting was one of the underlying principles of PW. The PSAG module, which all sites implemented, led to the introduction or development of PSAG whiteboards, and in later-adopting sites D and F to at least the start of the development of these into electronic systems. An electronic PSAG was also later developed in site A. PSAG boards were intended to make key information about patients available to all staff ‘at a glance’ in order to improve patient safety, prevent interruptions of nurses, aid admissions processes and planning of discharge, and reduce length of stay. As noted above (see Tailoring implementation: adapting Productive Ward), there was some tension between wards and trusts with respect to wards’ autonomy to tailor PSAG boards to the requirements of their own patient groups, but typically PSAG boards displayed patient name/initials; initials of the patient’s medical and nursing teams; whether or not initial assessments had been done; icons signalling patient safety risks; tests or procedures planned; expected discharge date; whether or not take-home medications had been ordered and received; and whether or not transport home had been booked. Some wards across sites reported introducing additional boards beside patients’ beds and/or in the central area, signalling, for example, the names of medical and nursing teams, any dietary requirements, and patients’ preferred form of address.

Other changes to the physical environment often came from staff suggestions and were very simple. One ward manager had kept these suggestions in a PW file, which she brought to show the researcher:

Some of them were really simple. ‘Please could we have a really big calculator in the treatment room and try to have it in a place where it normally lives’. So we put a big calculator laminated on the wall and when people were doing their drug calculations it will be just there in the treatment room. A win–win. Nobody would have to walk around with their big calculators in their pocket to do that. So, yeah. So [looking through file], oh ‘Urgent. Patient collapsed today, had to rush to a treatment room for small things required. Wasted precious moments. Luckily patient was OK. We desperately need a small trolley at the nurses’ station with syringes, flushes and everything’.

B12

Data collection and display

There were some conflicting accounts within sites about the impact of PW on the collection of ward-level data on key patient safety metrics and unplanned staff absence (as a proxy for staff well-being). Most interviewees thought that PW was the first time that ward-level data had been collected in real time by ward staff. One aim was for wards to use the data to identify problems in a timely fashion. As one PW facilitator noted:

In the old regime, you know, ward sister would get e-mailed 2 months down the line to say here’s your MRSA [meticillin-resistant Staphylococcus aureus] data for whatever, January, and you’re thinking well what use is that? Whereas now you’re collecting the data yourself . . . it’s more for them there and then, for them to know how safe their ward was really.

A01

Wards were also expected to use data to look for patterns over time. Safety Crosses and the auditing associated with PW helped ward managers review practices and training, think critically about how to avoid incidences and, ultimately, improve patient safety. As one ward manager described:

I suppose as a team it gave us a greater awareness and the ability to review practices and education and training and those sorts of things . . . If there was a pattern or a theme that emerged, we reviewed it at ward level and made some amendments from that . . . So it did provide us with evidence and improvement and, at the time before the Safety Thermometer and all the new scoring and management came in, you know, we were recording improvements in compliance rates for observations and management of those and things like that.

E09

We were given very few examples of how wards responded to patterns over time.

Aside from the recommended metrics, sites and wards varied with respect to the data collected.

Productive Ward was universally credited with the introduction of the display of such data, such as Safety Crosses showing daily incidents of key PW metrics and/or KHWD boards summarising ward data. Despite worries about patients’ responses, accounts varied about whether patients and visitors paid any attention to KHWD boards or Safety Crosses. No one reported any negative effects on patients.

Data collection and display was thought to have improved ward team’s knowledge of their ward data. The clear visual of Safety Crosses was regarded as particularly helpful:

The Safety Crosses and things, that again it’s a good visual, it’s there, it’s so obvious it hits you in the face almost . . . As a manager . . . it was an easy alert rather than having to root round or dig around or rely on that they’ve left you a message about it . . . and it allowed you to then look into things as a manager to see if there was a pattern or, you know, to make sure everything had been done appropriately as well after the incident had occurred.

D05

In site D, the template for Safety Crosses, some of which were based on trust policies such as single-sex accommodation, included standard actions that needed to be taken, which helped nurses to use data and to comply with trust protocols:

The biggest tool for me was the action plan, driving change from the care metrics. That made the most change for me. Yeah, that was a very powerful tool.

D03

I mean like obviously the falls things was very helpful, you know, the falls risk, because then you’d have something to tick off, whether the relatives had been informed and, you know, had the falls been updated and things like that. So that was very helpful.

D14

Data collection on compliance with care pathways and trust aims in site D also provided ward managers with a useful record on the practices they were responsible for:

It was a more formal way of checking. Because although you would do it and at a glance you would know things are being done, there was no record of that. So I think this was about us being able to evidence that ‘Yes the staff are doing this and we are checking that the staff are doing this’.

D10

Site A reported data from patient observation audits to a trust-wide safety committee. However, in the early days at least, and despite the example just given, here and elsewhere PW data were not systematically collated and used at trust level. One PW facilitator thought in retrospect that data collection had potentially become an end in itself, and that the PW team should have played a greater role in using the ward-level data for service improvement.

Processes

We remind the reader that only in site A were all modules mandatory (see Table 18). Here and in other sites, interviewees reported impacts on only a limited number of processes. We cannot tell if this is a function of limited impact or of recall.

Sites sometimes used PW in tandem with other interventions. Protected mealtimes was a common example. It was not itself part of PW, but PW was associated with protected mealtimes in all sites (including those in which the Meals module was not mandatory). Where protected mealtimes had not previously been implemented, sites used PW to introduce it. Other sites used PW as a focusing, incentivising device for the existing policy. As one interviewee explained:

We had protected mealtime in place but it probably wasn’t as protected as it should be. But again this [PW] kind of highlighted it. And it just I think some of this should already be in place, but sometimes things slip. And when you have a project like this, it helps you bring it into the forefront and think ‘Actually that is important’.

B08

In both cases, ward staff used PW as a tool for legitimising action, apparently enabling them to implement protected mealtimes in the face of opposition from other staff, predominantly medics. Many wards across sites also reported introducing red trays to signal patients who needed help with eating or close monitoring of nutrition, and streamlining the system of delivering meals to patients.

Where wards had attempted to implement the ward round module, the impact of this was weak. This was attributed to difficulties engaging AHPs and medical staff in implementing PW.

Sites D, A and F used PW to implement trust-wide changes to shift handovers. In site D, handover was to be preceded by a safety brief for all nursing staff, alerting them to the safety issues of all patients on the ward, which was said to be ‘invaluable’. Sites A and F imposed bedside handovers on all wards; in site F, this was in anticipation of the move to the 75% single-room new building. Other details of shift handovers varied within and between sites, and continued to change over time, but changes were aimed at making the process slicker and improving patient safety.

One ward in site E, and the whole of site F, used PW to look at whole systems, and changed the roles and responsibilities of staff. Site E’s randomly selected ward 1 gave additional responsibilities to the housekeeper and the ward clerk, and enlisted additional volunteers. Site F shifted the putting away of supplies and stock control from ward staff to central services, namely the laundry service and materials management.

Several other changes to processes were reported by individual wards in all sites.

Efficiency of care: length of stay, costs and direct care time

In site D, the PW team had collected data that were reported to show an overall increase in direct care time and a reduction in patient falls. A reduction in length of stay in a number of wards was also reported. These data were no longer available. Only two sites could provide us with any written reports on trust-level outcomes. Site F’s end-of-implementation report claimed that the PSAG boards had contributed to a reduction in length of stay by highlighting delays. Site B reported to the SHA that although they had spent £68,000 on PW, the saving realised from better stock management was estimated to be a minimum of £132,000 from September 2008 to July 2010. Direct care time had increased from a mean of 42% at baseline to 60.1% shortly before the end of implementation. Site F’s end-of-implementation report claimed that, based on before-and-after Activity Follow measures in cohorts 1 and 2, the proportion of direct patient care time increased from an average of 40% to 55%. This included time spent on bedside handover. According to another internal report, some of the time released at site F was spent on the trust policy of ‘intentional rounds’ (also known as care rounds or comfort rounds: regular rounds by nursing staff to check on patients and ensure that their fundamental care needs are met). A PW facilitator in site C said that when the site first implemented PW, direct care time had been around 24%, and that this had nearly doubled.

We explore such claims in much more detail in Chapter 6 – particularly in relation to site A, which gave us direct access to what we found to be the most robust data set – but Figure 8 shows screenshots of a 2011–12 data spreadsheet and graph to illustrate how site F carried out their calculations. Figure 8 comprises baseline and a ‘re-measure’ of ‘direct patient care time’ on eight wards as measured by Activity Follow. The percentage change in direct patient care time, the nurse time released in minutes, and what the released time was invested in are also detailed. With regard to these eight wards, it is stated that a total of 463 minutes have been released between the baseline and re-measure, which was predominantly ‘invested in’ discharge planning (as well as admissions and intentional rounding).

FIGURE 8.

Site F baseline and follow-up data for initial wards.

Figure 9 provides a graphical representation of these data as they relate to the percentage change in ‘direct patient care’ on the eight selected wards.

FIGURE 9.

Site F reported impact on ‘direct patient care time’ in first eight wards.

Perhaps unsurprisingly, few staff we interviewed could recall any specific quantifications of the impact on ‘direct care time’. However, interviewees who remembered PW commonly stated that changes made to the physical environment and ward processes had either ‘increased direct care time’ or at least ‘saved time’. (We noted a general tendency to conflate time saved with time spent directly with patients.)

Patient outcomes

We discuss patient outcomes with respect to first patient safety and reliability of care, and then patient experience more generally. No written reports give hard data on patient outcomes, so this section draws on reports from interviews only.

The implementation teams in sites A and D reported a reduction in patient falls overall. As a result of a rare case of the strategic use of data, site A responded to audit data showing poor performance on patient observations by introducing hand-held devices for patient observations throughout the trust, which was reported as having improved the reliability of patient observations (and, by implication, patient safety).

Aside from these, we have to rely on accounts from or about individual wards. In site F’s end-of-project report, the missed dose rate on the ward that implemented the Medicines module was reported to have dropped, although there was only one time point before and one after. Although many interviewees talked about the usefulness of ward staff collecting and displaying patient outcomes data, it was difficult to tease out examples of how this had led to improved patient outcomes on any of the key PW metrics. In site A, a postnatal ward had used PW to look at instances of mothers dropping their babies, and as a result had bought cots that could be brought right up to the bed. In site B, an example was given of one ward that looked at the pattern of falls to identify the change that had caused it. On another ward, PW had apparently made staff more vigilant about care, for example searching for pressure sores.

One ward manager in site B reported that auditing patient observations had ‘helped us fix a few problems we were having with observations’ (B12).

In site E, one ward manager said that systematic data collection was helpful for highlighting weaknesses that they might not have otherwise noticed:

It was kind of an eye-opener, you know, especially with falls because orthopaedic ward, you know, you don’t realise how many falls you do have until it’s actually written up. [. . .] We could see it in black and white and what we needed to improve on. Because it’s like everything, you can talk and say till the cows come home, you know, ‘We’re doing a really good job’. But until you actually see it written down, ‘Yeah we’re doing a good job on A, B and C. But on D we need to improve on and here is the data’, you know, so we used that quite a lot.

E02

As a result of their falls data, the ward purchased chair alarms and bed alarms and moved high-risk patients closer to the nurses’ station. Another ward manager at site E had started to take a more proactive approach to risk assessment and prevention, and had, for instance, improved the system for identifying which patients should be given beds closer to the nurses’ station in order to prevent falls.

With respect to patient experience, the one ward manager that collected this systematically as part of PW said that patient experience was improved as a result of higher staff morale:

Certainly the feedback from the patients, and looking at the patient surveys that we did, you could see the positivity coming through that. But could I say that the impact of it reduced pressure ulcers? No, because we didn’t have huge amounts anyway. So yes indirectly because of the improvement to the team.

E09

Nurses having more time was said to have improved patient experience (as patients liked having more time with nurses or nurses being more available). Changes to mealtimes were said to have made mealtimes more peaceful, and allowed staff to spend time helping patients to eat (aiding nutrition). At least some wards in sites B and F had used PW to give patients more choice of meals. Bedside handovers (introduced throughout sites A and F as part of PW) helped to involve patients in their care.

In site D, the matron seconded to work on discharge, shift handover and ward rounds reported that criteria-led discharge had had the greatest impact of the three, and in site F the nurse-led discharge system introduced in the surgical directorate was deemed a success. One ward manager in site D used audits on care pathways to see which parts were being missed out, and changed the documentation to help ensure that each process was completed. For instance, she added mandatory questions to the admissions form, which helped staff to trigger the correct pathway for patients with pneumonia, and to identify patients with dementia. She felt that communication and patient care had improved as a result.

Staff outcomes

There were few data available on hard outcomes for staff, and none on unplanned staff absence (the proxy measure recommended for measuring staff well-being). Site F reported that ‘time spent in motion’ reduced from an average of 14% to 11%, and that ‘mileage walked’ was reduced from 4.5 to 3.1 miles per shift.

Site A, perhaps because it was a pilot site working closely with the NHSI, stood out as a site where the implementation team possibly overclaimed on what PW might achieve, which left some staff disillusioned, as the following quotation illustrates:

It was sold that ‘oh yes this’ll revolutionise you and your ward and patient care’ and so on and I think that’s probably why we got a little bit disillusioned with it as well in the sense of ‘well that’s no different to what we’re already doing’. But then it may be that we were doing some things and we just tweaked them.

A06

Across sites, the Activity Follows and audits associated with PW created work, and, without dedicated time to do them or to think how to improve things, these could be a source of frustration or resentment. We were also given examples of ward teams feeling debilitated after suggestions that they had made had been resisted by infection prevention teams, or had not been executed by Estates (reflecting poor or tardy engagement at a strategic level with central services in most sites; see Developing stakeholder relationships).

However, overall PW was reported as having positive impacts on staff. An inter-relationship was apparent between increasing direct care time, improved patient experience and improved staff experience because nurses liked having more time with patients; and several interviewees spoke of the personal satisfaction they had got from working on PW and seeing its results. Changes that improved nurses’ working environment (through physical changes or changes in process) also made the working day less frustrating. PSAG boards, which were introduced in all sites, were a useful tool for staff that helped them to deliver better care for patients. To use site D as an example, PSAG boards were apparently ‘used massively’ by different staff groups. They helped staff to see which patients needed prioritising; they helped medics to prioritise their workload; nurses used them to allocate special beds (i.e. respiratory beds); and they reduced the time taken handing over information. On some wards they were used for discharge planning.

Productive Ward was reported to have improved team working. It was also valued for giving staff, including junior staff, the opportunity to put forward ideas for QI. This was described as empowering:

I think one of the things that Productive Ward did do, particularly with the foundation stuff, was to say to people actually you can make this change. You don’t need to ask permission. This is your ward and OK so there’s a ward manager there and you’ve got your senior staff but, you know, you don’t need the director of nursing or the chief operating officer or the board to say ‘thou shalt’. Just go and do it.

B05

However, it must be said that reports of the impact on junior staff were not always mirrored in interviews with junior staff themselves, particularly in sites C, D, E and F, where training in PW was given only to ward managers. In site D, this was somewhat mitigated by the communications aspect of safety briefing introduced in ‘Trust Way’, which allowed a more formal space for staff to raise ideas and have a group discussion on ways forward than there had been previously. Similarly, reports of impact on staff QI knowledge and skills showed that this had not permeated far below ward manager level on any of the sites.

For members of the PW implementation team, PW had a positive impact on their own learning and personal satisfaction, and had given opportunities to raise their profile within the trust and to build relationships with ward teams and central services. For one facilitator at site F, it led to her taking on a patient safety and quality role at regional level.

Organisational capacity

Aside from the impacts on individual staff members and teams (discussed in Staff outcomes), PW had some impact on organisational capacity. The collection and display of ward-level data became normalised. The experience, confidence, QI and leadership skills of ward managers increased. PW introduced a general familiarity with Lean thinking among ward staff. PSAG boards were introduced in all sites, and the development of electronic systems for tracking patient journeys began. In site A, PW prompted the creation of an IT infrastructure to collect and report ward data. At site F, PW was said to have been pivotal in the trust’s decision to set up a QI team of three, who train staff in QI methodology.

Active use of Productive Ward resources

Site B was an exception in that there was still some active support of PW in terms of personnel. A senior nurse had PW as part of her remit at one of the two trust hospitals; and two members of ward staff there still had some time (1 day every 4 weeks) ring-fenced to lead PW on their wards. Only the two ward leads who still had protected time for PW had referred to the box set or toolkit since the implementation period. One liked to ‘keep dipping into it’ and had used the process mapping tool; the other had used the box set to refresh her memory of PW when she took over as ward lead for PW and, again, before a CQC inspection.

In site A, the principles of PW continue to underpin its current QI programme. In site C, the TQB was still actively used, with a new TQB module being implemented at the time of fieldwork. As part of another initiative (the NHSI Safer Staffing work stream), the trust used Activity Follow to measure direct care time in five randomly selected wards annually.

In other sites, implementation teams reported that some ward managers still used some of the QI skills first introduced to them through PW (process mapping being the most frequently cited), but few ward managers we spoke to referred to using these skills. This may reflect terminology rather than practice. Specific PW resources (module booklets or tools) were very rarely used on wards. Referring to the PW box set, one ward manager commented:

Since we originally did it I’ve probably moved about five times. And since we never access it, it will probably have been binned!

A06

Physical environment: storage, data display and Patient Status at a Glance

Table 20 shows whether or not we found evidence of the 26 potential PW legacies on either of the two randomly selected wards in each of our six case study sites (12 wards in total). This table gives headline findings from our more detailed observation tool (see Appendix 2). The legacies are categorised as ‘artefacts-cum-practices’, as we were interested in noting not just whether, for instance, data were on display, but whether they were up to date and actively used. The other processes (see Process legacies) related to the three most observable PW process modules (Meals, Patient Observations and Medicines).

The most common artefact we observed was ‘neat, tidy, clearly labelled (as appropriate) storage’, on 9 of the 12 wards. On one randomly selected ward in site A, equipment was being left in the store room, in an old toilet and in the physiotherapy room (Figure 10); on the other randomly selected ward, storage was still neat, tidy and accessible, although colour-coding had ceased, and not all storage was labelled (Figure 11). Figure 12 shows examples of ‘ideal’ PW storage (transparent drawers, colour-coding of labels or drawers, and like-with-like storage) still in use. Although a common (somewhat pejorative) view of PW is that it was ‘just about tidying the cupboards’, it is worth noting that, even up to 10 years after its adoption, the maintenance of well-organised and easy-to-use storage in ward stock rooms is likely to result in not-insignificant time savings (given that ward staff are more likely to be able to find equipment quickly).

FIGURE 10.

Site A (ward 2): equipment stored inaccessibly in old toilet.

FIGURE 11.

Site A (ward 1): neat and tidy storage, not colour-coded, labelling variable.

FIGURE 12.

Colour-coded, like-with-like, labelled storage: (left) site B (ward 1); and (right) site C (ward 2).

The ideal physical location of storage being clearly marked was observed in only 3 out of the 12 wards. Figure 13 shows the floor maps in site D, ward 2. Floor maps had been used in more wards in the past, but we were told that these had worn away or that new flooring had been put down and wards had not replaced the maps. The other two wards used notices on the wall to signpost equipment location.

FIGURE 13.

Site D (ward 2): floor maps.

We found little evidence for the following legacies, which related to data display:

• up-to-date KHWD measures boards on display (artefact)

• clear and up-to-date Safety Crosses on display (artefact).

Knowing How We Are Doing boards were on display on or outside 11 out of 12 wards, but on only 2 of the 12 wards did we note that these were kept up to date. Most of the data on display were generated centrally from locally inputted data. They tended to be monthly scores, and could take more than 1 month to be returned to the ward. Often the data displayed on these boards were uninformative or incomplete. For example, on ward 2 in site D, although the board was up to date (the photograph was taken on 13 July 2017), the ‘number of days since last fall’ is shown as zero but no specific date is given of when the entry was completed (Figure 14).

FIGURE 14.

Knowing How We Are Doing board in site D (ward 2).

Data display was not always clear, suggesting a lack of understanding or commitment to the principles of using data to clearly signal performance. For instance, on ward 2 in site C, the KHWD board comprised a rather unclear bar chart displaying the number of ‘admissions; discharges; plaudits; medication [errors, presumably]; slips, trips and falls; tissue damage; VTE [venous thromboembolism]; and security’ (Figure 15); and on ward 1 in site B, although the KHWD board was up to date (photograph taken 26 April 2017), the data were, again, rather unclear (Figure 16).

FIGURE 15.

Unclear data display in site C (ward 2).

FIGURE 16.

Unclear data display in site B (ward 1).

A final example of a KHWD board is from ward 2 in site A. This comprised Safety Crosses for falls, MRSA, C. difficile and acquired pressure sores that were 2 days out of date (photograph taken 20 April 2017), with the ‘date of last fall’ not updated and a cleanliness score for February (Figure 17).

FIGURE 17.

Out-of-date Knowing How We Are Doing board in site A (ward 2).

Knowing How We Are Doing scores per se were rarely discussed with nurses below deputy ward manager level and not discussed with health-care assistants. However, incidents were investigated with individual staff and sometimes discussed with nurses at safety briefings. This means that the focus is on negative events.

We found that 4 of the 12 wards had ‘clear, up-to-date Safety Crosses on display’. Figure 18 shows that, on ward 1 in site B, although the Safety Crosses data were up to date (photograph taken 26 April 2017), they were kept in a folder and not on general display, and how on ward 1 in site C a Safety Cross relating to staff sickness was on display only in the staff office. On some wards, Safety Crosses were displayed but were not used in a thoughtful way. For instance, staff were instructed to fill them in only when an incident occurred (leaving it open to interpretation whether a day had been incident free or whether the cross had not been filled in).

FIGURE 18.

Safety Crosses at (top) site B (ward 1); and (bottom) site C (ward 1).

In some sites, more evolved online systems were now available for incident reporting [e.g. Datix^® (London, UK); www.datix.co.uk], which included ‘process mapping’ data of the circumstances surrounding an incident and what had been done as a result. Although they had many advantages, these systems lacked the clear visual message of Safety Crosses, and emphasised negative events only.

In order to be effective, these legacies require timely and ongoing updating and maintenance, and an understanding of the principles behind them. The continuing use of artefacts and processes without regard to their usefulness in terms of QI is an issue we return to in Discussion.

Some kind of PSAG board was on display and used on all of the randomly selected wards, except in site A. Here, the trust had moved to an electronic board system, which was available to staff on a hand-held device if they logged in, and could be displayed on a large screen. This provided information on investigations; safeguarding; medical, nursing and social plans; mobility; and diet. It was relevant to shift handovers, multidisciplinary ward round, patient flow, bed management and discharge. We were told that when the screens had broken down they had not been replaced. On ward 1 there was one screen that could be viewed communally in the multidisciplinary team room. This was used by visiting staff; for a board round attended on by the ward manager; and for the 10-minute safety briefing prior to bedside handovers using the hand-held device. So the elements of PSAG pertaining to having patient information available ‘at a glance’ remained, but they relied on the technology working, having access to at least a hand-held device, and staff being able to log in. We did not manage to ascertain exactly which staff groups had hand-held devices or log-in permissions, but we know that the original PSAG boards were immediately visible to all staff, and we know from other sites that they reduced interruptions. A nurse on ward 2 said that the electronic system was less useful for the ward than the original PSAG, but more useful for the trust. In site D, the PSAG board on display supplemented an e-system of patient information. In site F, the e-system was the only system, but it was permanently visible, and located by the nurses’ station. PSAG boards or e-systems on all wards were used for Admissions and Planned Discharge (Figure 19).

FIGURE 19.

Patient Status at a Glance used for admissions and discharge planning: site C (ward 2).

We found no evidence of artefacts that were very narrowly tied to PW: up-to-date Visit Pyramids on display (one of the PW tools to encourage surveillance of PW compliance and connect ward to board); and Ward Visions on display and known to staff. Often, Ward Visions had been agreed during PW and put up for display, but had been taken down during ward refurbishment, visual decluttering or an infection control drive, and staff no longer remembered what they had been. On the one ward where there was a Ward Vision on display the researcher was told that there was no Ward Vision.

Process legacies

We begin by making some general comments, and then we briefly discuss legacies on two of the observed processes: meals and medicines. As Table 20 shows, there is very little variation between wards, and this is particularly so for legacies related to the three processes: meal rounds, medicines rounds and patient observations. The following seven legacies were found to be present on 10–12 wards according to our structured observations:

• menu process done outside mealtime (meal round)

• responsibility for patient observations defined and understood (patient observations)

• frequency of patient observations based on condition (patient observations)

• frequency of patient observations easily accessible (patient observations)

• the person responsible decided in advance (medicine round)

• equipment in same location, ready to go (medicine round)

• patient needing observations known before the round or clearly identifiable at bedside (medicine round)

• system for flagging patients who have missed patient observations

• system for flagging patients who have missed medications

However, it is important to note that, on all wards, flagging patients who had missed either observations or medicines was dependent on the e-systems now in use. E-systems were not a visual flag ‘at a glance’, which was characteristic of the PW method, and they signalled missed processes only when actively consulted (likely to be the next observation or medicine round). On only one ward (site E, ward 2) was it accepted practice to supplement this with the visual clue of leaving the observations folder on the bed of a patient missing observations. Rarely did individual staff use a reminder system such as this, or by setting an alarm on their phone. For missed meals, where there was no electronic system, only one ward (site F, ward 2) marked this on a board in the kitchen. For all of these missed processes we were told that ‘the nurse remembers’. The universal practice of allocating nurses to certain bays or rooms (which also clarified responsibility for certain processes) may make remembering easier. However, in a busy ward environment, relying on individual staff to (1) remember, (2) notice the return of the patient, (3) take action or (4) remember to hand over such information if they leave the ward is not an efficient or reliable process. The ward manager questionnaire (Table 21) also showed that only one ward used a system for flagging and following up any patients missed from ward round.

It was rare for written standard operating procedures to be on display. These were another key component of PW, underpinning standardisation. Some ward staff said, and others ventured a guess, that a standard operating procedure might be available on the intranet. In site C, ward 2, the standard operating procedures covered protected mealtimes only.

Medicines

In site E, the photographs introduced in medicines trollies to show what went where remained on at least two of the wards in our study. However, in ward 1, we were told that the trollies were often over-full, and would get sorted occasionally over a quiet weekend. The two observed wards still used ‘do not disturb’ tabards for nurses on medicine rounds to prevent interruptions. So did some wards in site F, where on one observed ward there were also notices to this effect on the medicines trolley (Figure 20).

FIGURE 20.

Medicines trolley with notices to prevent interruptions at site F (ward 2).

Initial outcomes sustained

As discussed above (see Efficiency of care: length of stay, costs and direct care time and Patient outcomes), very few hard data were available on the impact of PW, even during or at the end of implementation, and only site C had undertaken any post-implementation measurements. Its annual Activity Follow data on a random sample of five wards (different wards each year) indicated that direct care time had been maintained at just under 50%. In the rest of this subsection, we can report in only general terms what people said about whether or not outcomes had been sustained.

With respect to time saved, as discussed in Resourcing, for the most part things were still in place that had initially saved time. Wards in site F, whose move to a building based on a racetrack design, with storage off the wards and with mostly single rooms, had increased the amount of time taken, had worked hard to make minor adjustments over time and to buy some additional equipment to regain some of the time lost. Where PSAG boards of some description were still on display, they continued to reduce the number of times that nurses were interrupted by requests for information.

No wards reported re-running modules in the intervening period, despite acknowledged changes in context. The most obvious example was when wards changed location or specialty. Although they maintained the general principles of reducing unnecessary time spent moving around the ward, and ‘a place for everything and everything in its place’, they had not re-run the WOW module to examine their new needs. This appeared to be the case even in site C, which was still using its PW-based TQB programme. Here, the impact of earlier work on infection rates, which over time had included new foci such as hand hygiene and environmental audits, had apparently been sustained. In sites A and E, a ‘stay in the bay’ policy, whereby computers were introduced in bays to allow nurses to spend more time there, presented an intervening variable likely to affect falls and direct care time. In site F, the move into the new building was said to have considerably added to the mileage nurses walked as a result of the physical layout, and predominance of single rooms made nursing activities more time-consuming.

More generally, imputed gains in direct care time or time saved arising from PW had been gradually reduced by increasing demands for data collection and the introduction of computerised systems. The growth in the number of audits and other paperwork required by trusts and by the NHS over time has placed a considerable burden on senior ward staff, as reported in this nurse’s powerful account:

I might be wrong, but the way I remembered it, it was about releasing time to care. And there’s still a continued fight for releasing time to care. And that frustration, I feel a bit emotional almost, is just never-ending. Because I spend most of my days messing about when really what I’d like to do is go and care for my patients and [I] spend so much time hanging over a computer . . . that and you just think to yourself well actually I just want to do a good job.

A05

Information technology (IT) was reported as a challenge. Apparently many support staff were not proficient in IT; but even for IT literate staff, computerisation relied on having access to sufficient (working) IT infrastructure. Increased patient complexity had also eroded gains in direct care time.

Principles remain: Lean/service improvement principles; staff voice; nursing role in quality improvement

There was no explicit set of principles written up in PW guidance, but several can be inferred:

• standardisation

• a ‘Lean’ mindset, cutting out waste and improving flow

• empowering nursing staff at all levels to put forward suggestions for QI

• giving staff QI tools and skills

• using QI tools for continuous improvement.

We discuss these briefly here. Apart from the standardised colour-coded storage systems introduced during implementation, standardisation had waned. KHWD boards standardised across the entire trust existed in sites C, D and F. There was little standardisation of processes.

Lean thinking appeared to have remained in all sites, and was strongest in sites B and F:

I’ve got seven sisters within my area that I manage and they all know that ‘that’s not Lean’. That’s the wording that will come out. So from the whole Productive Ward series, the elements that seem to still flow through are the Lean element.

F01

In site F, one interviewee was working on a project to streamline documentation throughout the trust. In the physical environment that worked against care at the bedside, ward managers continued to purchase trollies and storage solutions to reduce time and distance, and to bring care nearer to the bedside. In site B, one senior nurse talked about a legacy of thinking about flow:

I think everybody who’s involved with it, we now really think about the flow of our work. I don’t think we ever did before, we just kind of, because as nurses it’s all there in front of you, there’s your six patients, there’s your one patient and the work kind of comes to you and you don’t need to organise yourself in an intelligent way. [. . .] But this kind of made us think about where people walk to and how far they have to walk and especially in a hospital like this [. . .] there’s a lot of time spent walking up and down. So any little benefit can be a big benefit.

B11

In terms of current reach of staff involvement in QI, site A maintained the system of shared governance that some staff attributed to PW (see Assimilation). In site E, both randomly selected wards had continued to make improvements as a result of suggestions from ward staff at all levels. Examples were better signalling staff on duty and bay allocations, changing window covers to ensure patient privacy, and making changes to shift handovers. However, one interviewee attributed this culture primarily to ward teams, rather than being a legacy of PW:

I wouldn’t say as a whole this Productive Ward has changed this ward, I would actually say it’s the management and the staff that we’ve got. We still to this day have people that come up with ideas and they are used, which is good, you know. Obviously there’s some that don’t work but then, you know, we can try again.

E06

In site C (which was still implementing TQB), senior nurses working across wards reported wide staff involvement in QI, but they were uncertain of its legacy. One directorate was currently trialling a ‘shared governance’ initiative, and individual wards had their own mechanisms for gathering staff ideas. In other sites (B, D and F), the current reach of staff involvement in QI was difficult to ascertain. Staff often said that they felt that they could put forward ideas to the ward manager, but none of them could give concrete examples of having done so. There was a sense that lower bands of ward staff in sites D and F were particularly disengaged from the QI process. This is likely to reflect the low levels of their engagement in implementing PW (see Developing stakeholder relationships). This remark from a band 5 registered nurse was fairly typical:

So have you been involved at all in improving, would you say, any aspect of the care on the ward?

Not directly, no. My manager is but, you know, she just pass it on to us ‘Oh this is what you need to do’. I’m just a good subordinate so do as you’re told!

Current use of QI tools and skills was fairly restricted to ward managers and more senior nurses who had been involved in implementation. It was strongest in sites B, E and F. For instance, in site F, the manager on ward 1 was in the process of trialling new storage in a number of rooms on the ward. A ward manager in site B spoke about drawing up business plans, which she had first done as part of PW. In site C, the implementation team reported that a PDSA improvement methodology had been absorbed into ward managers’ common practice. In sites D and E, there was little or no evidence of ongoing continuous improvement. This could be attributable to implementation being seen as more of a finite project there than in sites A, B and C. Site F was anomalous in that implementation had been very much geared to the move into a new site. Yet, there too, there was strong evidence of continuous improvement.

For PW leads and ward managers, their involvement in implementation continued to have an effect on their skills and working practices:

I think, for me, it helped me to [. . .] just look at a process. Before it was very easy to move into a team and be told ‘This is how we do this’ and then just think OK so that’s the team’s process, that’s how I’ll do it. Whereas now I might look at something a little bit more critically and think OK that is how we do it and that works but we could do it like this and it might work a little bit better, which I probably would never have done before.

E01

Networks/partnerships/coalitions sustained

Productive Ward leads continued to benefit from the connections they had made with central services and relationships of trust that they had built with ward teams. This was perhaps most important for the QI team in site D, who were not nurses. Networks with other trusts, which NHSI and SHAs had originally brokered, had not lasted after these central organisations had ceased to support them.

Productive Ward replicated/used elsewhere: Productive Ward adapted for use outside ward?

All but site C had implemented The Productive Operating Theatre shortly after implementation of PW began. Site E had also implemented PW in its community hospitals and productive community services. Site B adapted PW for use in its catheter laboratories. In site B, aspects of The Productive Operating Theatre and their catheter laboratory programme were still being used. No further replication of PW had taken place in the intervening period.

Organisational capacity

In site A, the whole-hospital QI programme based on PW was still active, and the trust had implemented a shared governance system of capturing QI suggestions from ward staff, which some attributed to PW. In site C, at the time of fieldwork, the trust was building on its earlier work and implementing a ‘safety learning’ system of incident reporting that allowed trust-wide learning from incidents, and also streamlining its data reporting systems. In site D, the QI team was still using similar principles to PW/Trust Way, and continuing to offer Lean training for 5S and Comms Cell to all staff. One interviewee, who had implemented ‘PW pure’ and Trust Way when she was a ward manager, was currently leading her own workstream as part of the QI team. The QI team set up as a result of PW implementation was still operating in site F.

Intervention

Productive Ward, some parts more than others, addressed a need felt by ward staff, and those were the components that were sustained. However, a criticism of PW was that it was very much rooted in wards and, as a result, had a limited effect on whole-hospital systems. Site A, by developing its own whole-hospital system, was addressing this concern. A senior nurse in site B said that PW had not had much impact on hospital systems that did not sit within wards, which was a factor limiting its sustainability. She attributed this to PW having been regarded as ‘a nursing issue’:

I think that’s often the case when something gets perceived to be a ward, ergo a nursing issue, and not a wider one. I’m always interested, was and am interested, in things that engage the whole team in whatever it is you’re trying to change. Productive Ward wasn’t like that.

B17

At the time of fieldwork, site B had been working to a ‘transformation’ agenda. A PW facilitator in site F also noted that PW being nurse-focused was a limiting factor:

My recommendation is something like this [PW]. A service improvement programme based on this . . . But I think it needs to be a wider scale thing. And to include other bits other than just nurses or nursing processes or nursing things.

F07

Efforts to implement PW in some non-ward areas in sites D and F (late adopters) perhaps reflect a realisation that, from a trust perspective, a wider-reaching transformation programme was desirable.

Site D’s Trust Way incorporated a sustainability plan from the start, which was based on a system of audits (Figure 21).

FIGURE 21.

Site D’s ‘Trust Way’ Lean sustainment audit template.

The design of TQB at site C, both from the start and over time, included several elements to bolster its sustainability. These were:

• Incorporating overlapping QI programmes that might otherwise have competed for resources.

• Providing a unified methodology for these QI programmes.

• A system of continuous review.

• A dashboard visible at trust level to monitor ward metrics for all wards, and baseline and follow-up scores and annual Activity Follow data for a small number of randomly selected wards. This dashboard is used to prompt wards to re-run modules.

• Implementing new and adapted modules relevant to wards.

• A sustainability module (Figure 22).

FIGURE 22.

Site C’s sustainability module questionnaire.

The TQB team has adjusted modules (including each of the PW foundation modules), and run the new versions as the context has changed, to keep the modules relevant.

Organisational

Sites B and C still had dedicated resources to ongoing implementation. In site B, leading PW was part of the remit of one senior nurse; and two wards had protected time for ward leads to continue implementing PW. Site C was still actively implementing TQB (its evolution of PW) and had two full-time TQB leads. In site F, there was a desire among at least some senior managers for cost savings. When these were not realised, there was no appetite for investing any more resources in sustaining PW. Staff turnover risked the loss of PW or general QI knowledge and skills. The reach of staff engagement in implementation could, in theory, mitigate the loss of knowledge inherent in individuals. Trusts often moved on to new innovations. The resilience of TQB (site C) was in its absorption of a number of overlapping initiatives.

Environment

External audits (e.g. by the CQC) helped to sustain data display, storage and cleanliness. NHSI had provided a valuable networking opportunity between trusts, which no longer existed. The training available via licensees was felt to be prohibitively expensive, and none of the sites had accessed it to train new staff (a particular issue for trusts with high staff turnover). Environmental changes that eroded direct care time included expansion in nursing role (in procedures expected, in admissions and discharge paperwork, and in safety monitoring and reporting) and patient complexity. The ‘busyness’ of the ward environment, which was related to patient complexity and acuity and, in some cases, staff shortages, meant that wards could make only piecemeal changes:

I think it’s a shame that Productive Ward seems to have – we all know, its legacy lives on, we all know about it, remember it, we talk about it. [. . .] But I look at this ward, as it is now, with the changes that we’ve done and I just think oh if only we could do that, you know, if only we had that invested time to look at it again. [. . .] It’ll end up being me coming in on a weekend with a team of people and we’ll do what we can in what time we’ve got and hope that that – because it’s always going to have an impact if we don’t do that on patient care. So if you pull people off the ward to do things then it’s going to impact on them and we can’t do it. So, yeah, I think like a rolling cycle would be good.

F06

There was some support for a wholesale re-running of PW:

I think the key thing is that there is a lot of value in it and I think it’s something that we probably should re-promote because actually there’s always stuff you can improve in each one of those sections. I think the national picture where we are about productivity and getting the right people to do the right things, at the moment everybody’s doing everything. I think there’d be a lot of value in re-thinking the advertising of it and I think the NHS would benefit from it right now.

F13

Outcomes

The four measures we identified in these data resulted from audits conducted according to methods informed by the PW materials:

1. Patient satisfaction – based on a satisfaction questionnaire. The scores are presented as a percentage of the maximum with a multi-item questionnaire comprising ordinal scales giving a score based on responses, but the sample size is unclear. While guidance suggests this is 10, the number achieved is not reported. The sampling strategy in unspecified/unclear.

2. Staff satisfaction – based on a satisfaction questionnaire (as above). The scores are presented as a percentage of maximum, but the sample size is unclear. The sampling strategy is unspecified/unclear.

3. Patient observations – audit of patient observations records against 10 criteria. Ten patients per month with a score based on the number of records completely compliant (i.e. number of patient records who fully met 10/10 criteria). The sampling method was unspecified.

4. Direct care time – based on the Activity Follow methods as described in the PW materials, although the achieved sample sizes/observation periods are not specified.

For each ward and for each month, there was a record of the number of modules planned (the target number of PW modules for that ward) and the number completed.

Analysis

We undertook a basic descriptive analysis using Microsoft Excel to explore progress through modules and overall trends in audit scores over time. Because scores are nested in wards and because different wards participated in the programme at different times, we further explored the relationship between undertaking Productive Ward modules (number of modules completed or completion of specific modules) and outcomes (audit scores) using multilevel models.

A Stata^® (version 14, StataCorp LP, College Station, TX, USA) mixed-effects generalised linear model was used for all models. For observations compliance, we modelled the outcome (the number of records that were fully compliant) using Poisson regression. As all counts were based on the same number of opportunities to observe, we did not include an offset in the model (Gaussian). As these analyses were exploratory, we used default options in Stata for integration (with 7 points), standard errors (observed information matrix) and covariance matrix structure for random effects (independent). We assessed the sensitivity of models to these assumptions by applying alternative options on initial models (laplacian approximation, robust standard errors, including an offset for the number of observations and unstructured covariance matrix) and found the coefficients we sought to estimate to be almost identical and substantive conclusions unchanged. Model fit was marginally worsened [increased Akaike information criterion (AIC) and Bayesian information criterion (BIC)]. We attempted to estimate negative binomial models but could not get models to converge, although we had no basis for suspecting overdispersion.

Ward was included in all models as a random effect. We added a time variable to the models to account for the possibility that the effect might simply be due to a trend to increased compliance over time, and selected models based on minimising the AIC and BIC, which given indications of relative (rather than absolute) model fit.

No models showed evidence of multicolinearity (default checking and omission of collinear variables during estimation and post-estimation variance inflation factor < 5).

Results

The trust had begun to implement PW in 2007. Data were provided for 16 months, commencing October 2009. We received 1234 records (wards × months) from a total of 94 wards/units across seven directorates; 62 of these (66%) provided data for all 16 months, with a mean of 13.4 months’ participation. The minimum was 2 months.

At the start of the data series in October 2009, 65 wards provided data, with a mean 7.4 PW modules planned (range 3–12 modules) and 6.4 modules started (range 0–12 modules). By the final month, 89 wards had provided data, with a mean of 7.9 modules started. Across all wards, the number of modules started rose from 412 (October 2009) to 698 (January 2011) (Figure 23).

FIGURE 23.

Progress through PW modules over time.

In general, wards had already completed the three core modules (KHWD, WOW and PSAG) when they first contributed data. Of the 1234 wards × months available, these modules were recorded as complete on between 99.8% (KHWD) and 95% (PSAG) of occasions. By contrast, the process modules ‘Procedures’, ‘Admissions’ (which site B separated from Discharge) and ‘Ward Rounds’ were the least likely to be undertaken (reported as complete on between 20% and 26% of occasions; Figure 24).

FIGURE 24.

‘Exposure’ to PW modules (ward months).

In total, 2190 audits were reported, with an average of 1.8 of the possible four audits reported per ward per month (Figure 25).

FIGURE 25.

Audits per ward per month.

Most wards reported one or more audits for most months. We received one or more audits on 84% of all possible occasions. Compliance varied by audit, with the observations audit most commonly reported (72% of occasions), followed by staff satisfaction (49%), patient satisfaction (39%) and direct care time (18%). The expected frequency of reporting direct care time is unclear and so this figure may under-represent compliance. There was a decrease in compliance over time, with a mean of 2.4 audits completed per ward in the first month and 1.5 in the last month (Figure 26).

FIGURE 26.

Mean audit scores by month.

There was no consistent trend in the mean audit scores over time (Table 23 and see Figure 26) other than an increase in staff satisfaction.

For staff satisfaction, there appeared to be a gradual increase from 67% to 73%. Other scores varied from month to month with no clear trend. Mean observation audit was 71% in the first month and 76% in the final month, direct care time was 43% in the first month and 38% in the last, and patient satisfaction was 4% in the first month and 85% in the last month. However, this is a potentially misleading picture as new wards were entering the study and gradually moving through more PW modules, and so a decline from one month to another could be due to the entry of new wards on which less progress had been made with PW. Therefore, we explored these relationships further, using multilevel models to assess how progress through the PW modules was associated with these measures.

Results of multilevel models

There was no clear evidence that patient or staff satisfaction increased as wards completed more PW modules (Table 24). Patient satisfaction was not significantly associated with the number of modules completed (p = 0.772). Time trend was not significant and adding it to the model reduced fit (AIC and BIC increased). Staff satisfaction was significantly associated with the number of modules completed (beta = 0.009; p < 0.001), but this was reduced and became non-significant (beta = 0.004; p = 1.39) when the time trend was added to the model, as staff satisfaction showed a significant increase over time, independent of the number of modules (p < 0.001), resulting in improved model fit.

However, there was clear evidence that care processes changed as the number of modules completed increased, and this was not accounted for by an overall time trend (see Table 4). Direct care time increased significantly as the number of modules completed increased (p < 0.001). Each module completed was associated with an increase of 1.3% in direct care time. Each module completed was associated with a 3% increase in the rate of compliant records in the observation audit [incidence rate ratio (IRR) 1.03; p < 0.01]. Time was not significant in either case, and adding time to the model did not improve fit or substantially change the coefficients.

Module specific effects

For those audits where we identified an effect from the number of modules, we explored further to see if this could be linked to any specific modules. Because the observations module could be linked to a specific audit that measured an intended outcome, we modelled the association between completion of the observations module and compliance measured by the observations audit. Wards that had completed the observations module had a 21% higher rate of compliant records (IRR 1.21; p < 0.001) (Table 25). Model fit was improved compared with the model with number of modules completed. Time was not significant and model fit was not improved by adding it as a variable to the model.

Although we identified an effect of the total number of modules completed on direct care time, there was no one-to-one mapping between direct care time and the expected outcomes of particular modules. Therefore, we considered completion of all modules as potential predictors. We omitted the KHWD and WOW modules from consideration, as nearly all wards had already started these by the time they began providing data. In order of magnitude of effect, the handover, admissions and ward round modules were all associated with significant increases in direct care time, with an increase of between 8% and 5% associated with having completed the module (p < 0.001 – p = 0.029) (Table 26). However, completing the observations module was associated with a 3.5% decrease in direct care time (p = 0.026). There was no clear advantage or disadvantage for this model compared with models based on the number of modules completed (AIC smaller, BIC greater).

Learning about quality improvement

Participant experiences of PW had taught them about various aspects of QI: the use of evidence, the importance of engaging a variety of stakeholders, and how to engage staff. It also made participants think about fidelity of implementation, and the benefits and disbenefits of local adaptations over standardisation.

Using tools such as process mapping and PDSA cycles taught some interviewees the importance of not skipping steps but rather thoroughly diagnosing a problem:

Productive Ward really gave the tools to enable me and my colleagues to try and understand what the issues are before we try and fix it.

5TI

There was widespread learning on the importance of stakeholder involvement in making positive changes to improve patient care:

I really enjoyed from a nursing point of view seeing what changes you could make that can affect patient care. But [also] getting staff to do it, you know, involving them in the whole process and patients and relatives, asking them what they think we should do about something.

8TI

Matrons were found to be important stakeholders who could support or undermine the engagement of ward staff. One criticism of PW as a whole was that for an intervention aimed at improving patient care there was too much focus on staff in wards, and not enough focus on patients or the patient journey. Furthermore, the narrow focus on nursing staff did not provide a good match for the reality of the complexity of teamwork between different staff groups.

Productive Ward also left personal legacies on PW leads in terms of soft skills and ways of working, including leadership skills, organisational skills, the ability to identify problems on a ward and increased confidence. All but one interviewee stated that they had used their PW skills and experience in subsequent roles.

The impact of Productive Ward on career pathway, promotion and work practices

All but one interviewee stated that PW had had a positive impact either on their career pathway or on promotion. The majority of interviewees came from a nursing background with little to no experience of QI. After involvement with PW, many reported an impact on career pathway. Most interviewees went on to work on new Trust QI programmes.

I think probably it had a big impact on me personally, because it almost changed my steer of my career [. . .] Productive Ward kind of ignited a, a flame really. And I’m still working in Improvement albeit Strategy.

6TI

One interviewee directly attributed being made lead nurse for quality and safety to their work with the KHWD module. Another saw their knowledge of change management as leading them to a career within IT. Other examples of QI roles participants went on to work in are director of improvement, senior nurse in clinical procurement and productivity, and head of QI. One interviewee went on to co-found their own QI consultancy company. There was also an example of secondment with the SHA, where an interviewee went from PW at trust level to working regionally, working on both PW and Productive Community Services. A few interviewees went on to write published articles and research about QI. Some went on to further education studying health and social care management generally, or postgraduate degrees in leadership management and change management, using their experience with PW as examples in their degree work.

As well as changing career paths, many felt that PW had had a direct impact on promotion. Several participants reported that either they or PW colleagues had been promoted to senior nursing posts, including matron, head of nursing and DoN, as a direct result of PW.

Views on the Productive Ward product

One interviewee felt that PW was ahead of its time and would be more useful in the current NHS context of financial and staffing constraints. For those who valued fidelity of implementation, having a standard methodology was helpful. There was praise for PW’s practical nature, but, on the whole, interviewees were somewhat critical of the PW product.

Several interviewees stated that the PW materials were difficult to comprehend. One trust adapted PW from the start, feeling the need to simplify it and turn it into more of a trust ‘product’. Scepticism was expressed about the accuracy of Activity Follow data. One issue was whether or not people acted differently when they knew that they were being watched, and whether what was observed was truly representative of what happened in the ward.

Staff engagement

Productive Ward relied on engaging ward staff, as they had a role in identifying problems and potential solutions, collecting data, and helping to organise the ward. Interviewees spoke about the importance of engaging ward staff and other staff groups, how this was best achieved and the challenges that this could present.

Most ward staff were reported to have connected with and valued PW because they could see things that were wrong with the existing system (such as waste, harm and delays). One interviewee did not appear to regard staff engagement as an important part of her role, but the majority talked about the importance of motivating and empowering staff to make a change. Implementing change with ward staff involvement was regarded as more effective than a top-down approach, and increased staff’s pride in their ward.

Several people spoke about variability in staff engagement between wards. Interviewees also spoke about what helped staff engagement in PW. This included an emphasis on how PW would save time and improve the delivery of patient care. Showcase wards were used to promote PW to the rest of the trust. One interviewee used a variety of techniques to engage staff: before and after photographs that acted as visual evidence, personal testimonies demonstrating PW successes, and evidence of time saved (from Activity Follow and spaghetti diagrams). It was also thought helpful if the PW team appeared to understand nurses’ work and to be on the side of the staff:

I insisted we stayed in uniform. I think the senior team would have preferred us out of uniform, but the reason why I stayed in uniform was I thought it would be easier to engage the nursing team if they could see that we were nurses and we were on their side. [. . .] Because even the nurses that were resistant saw that we were nurses trying to do things to improve their life but also their ability to care for patients.

14TI

Poor staff engagement was often thought to be because staff could not identify that change was needed. Support from a ward manager was also seen as key to implementation. Where ward managers were less proactive, staff were not given the opportunity to engage fully and so implementation was harder. Ultimately, staff were thought to get involved only if the ward manager was behind it.

Matrons were another important staff group with respect to implementation of PW. In practice, activities or changes that involved medical staff were found to be difficult to implement, as nursing teams and PW teams lacked leverage over them. It was commonly felt that support from senior staff in the trust was instrumental in transformational change. Backing from those in managerial positions was thought to be necessary so that, if needed, they could use their authority to get things done. To gain support from executives, PW needed to be linked to the trust’s goals and strategy.

Resources required for implementation

Successful implementation relied on a number of resources: an understanding of the QI principles underlying PW, input from a named PW lead (which in practice diminished over time), dedicated ward staff time to work on PW and money to implement improvements.

Understanding of quality improvement principles

The majority of interviewees had had no experience of leading a QI programme prior to leading PW. One trust rejected the tried-and-tested PW programme, and insisted on adapting it from the outset using multiple methodologies. This re-design was left to one individual, who admitted not knowing what they were doing:

I was given free rein, completely free rein to design something with quite limited experience, and not to use the Productives. [. . .] We didn’t use the Productive measurement tools and I think that was again naivety and trying to do our own thing. And actually there was a tool there that was perfectly adequate and we should have and could have used that but we decided to try and use our own method.

7TI

At ward level, sometimes enthusiasm for working through the box set overshadowed the rationale of what staff were actually doing:

What I observed, was on occasions, they were missing the point of what they really needed to be doing. So, they were working on the admissions module, because that was the next one in house, but really their problem on their ward was falls.

3TI

In another trust, staff reduced time spent on the handover without critically assessing its quality, which the PW lead felt had decreased. This tendency was in part attributed to the PW design. Both the PW roll-out template and the sustainability tool were criticised for focusing on compliance by scoring whether or not a module had been completed, rather than evaluating the outcomes. The focus on saving time rather than on using time constructively was also criticised.

Input from a Productive Ward implementation team

Input from a PW team (in terms of time, energy and enthusiasm) was a key resource. One interviewee identified that a significant factor in the success of implementation was the position of wards on the roll-out timeline, as support from the implementation team tended to diminish as roll-out progressed:

Some wards – those who got all the attention at the beginning – absolutely loved it. [. . .] Other wards, that weren’t so keen on Productive Ward because they’d not been included and supported at the beginning, would – and this was actually literally what I saw sometimes – they would use the box set to raise their computer screens up. It was literally like not used at all, other than as a door stop or a computer thing.

3TI

The lack of support as implementation progressed owing to the PW team capacity being stretched (or no longer being available at all) was exacerbated by the fact that later wards tended to be those that needed more support.

Ward staff time and financial resources

Ward resources, in terms of time released to implement PW, were also required:

The areas that were well staffed and given time to implement Productive Ward properly did very well on it. There was a number of other areas that struggled with it and I think, from an organisation, although the trust board wanted it rolling out, they weren’t prepared to give the staff the time to actually implement it properly.

10TI

Financial support was another significant factor.

I think perhaps we didn’t have the commitment or the finance from the trust to perhaps implement it as we would have liked to have done.

11TI

Trusts that received external funding found that it covered only the wages of the implementation team and a few essential items to help wards. PW seemed to do better on wards that could access alternative financial support.

Ward ownership versus standardisation

There was a tension between ward ownership and standardisation across the trust. Some interviewees found that QI initiatives are not always transferable from one clinical area to the next. PSAG boards presented a case in point. One interviewee thought that PW was a useful methodology for piloting innovations in some wards but that the results should then be rolled out as gold standards across the trust.

Summary

Participant experiences of PW had taught them about various aspects of QI: the use of evidence, the importance of engaging a variety of stakeholders, and how to engage staff. Their experiences had also prompted participants to think about fidelity of implementation, and the benefits and disbenefits of local adaptations over standardisation.

For the vast majority of interviewees, their experience of leading PW had had significant impacts on their career pathway. Starting with little or no QI experience, many went on to work on other QI initiatives within their trusts, or to work in QI at regional or national level within the NHS or in the private sector.

Interviewees were fairly critical of the PW booklets and tools. In its favour some praised PW’s practical nature, and the standard methodology was said to help fidelity of implementation.

Interviewees spoke of the importance of engaging ward staff and other staff groups. Involving ward staff in implementing change was regarded as more effective than a top-down approach. In practice, wards had varied in terms of staff engagement. As well as ward managers and ward teams, other stakeholders who could block or ease implementation were identified, namely medical staff, matrons, managers and executives. Medical staff had not been successfully enrolled in PW, and this had made some elements of PW difficult or impossible to implement.

Successful implementation relied on a number of factors: an understanding of the QI principles underlying PW, input from a named PW lead, dedicated ward staff time to work on PW, and funding to implement improvements. In practice, support for wards tended to diminish over time, as central resources became ever more stretched as more wards implemented PW.

Sustainability had not been built into the implementation plan in any of the trusts represented in telephone interviews; this was regarded as a key flaw.

Adoption

Site A was a very early adopter of the PW. A large teaching hospital, it was one of the whole-hospital pilot sites in autumn 2007, and had begun implementation even before all of the PW modules had been finalised. One stated reason for adoption was to facilitate the previous merger of two trusts; PW was seen as one mechanism to facilitate the ongoing integration between the two.

Implementation (2007–11)

Site A received external funding from the NHSI for 2 years for four PW facilitators, and also appointed a trust-wide PW lead and a project support officer, all working full-time on PW. The NHSI also funded the provision of external support from a delivery partner who helped to plan the implementation strategy, trained the PW team, and offered face-to-face support and challenge. Two of the four PW facilitators remained in post for a total of 4 years, latterly funded by the trust.

Reported impact

Four of the PW metrics were captured at trust level: direct care time, patient observations, patient satisfaction and staff satisfaction. These four metrics resulted from audits conducted according to methods informed by the PW materials:

• Patient observations – audit of patient observations records against 10 criteria. Ten patients per month with a score based on the number of patient observations completely compliant; sampling method unspecified/unclear.

• Patient satisfaction – based on a satisfaction questionnaire. The scores are presented as a percentage of the maximum with a multi-item questionnaire comprising ordinal scales giving a score based on responses, but the sample size is unclear. While guidance suggests that this should be 10, the number achieved is not reported. The sampling strategy in unspecified/unclear.

• Staff Satisfaction – based on a satisfaction questionnaire (as above). The scores are presented as a percentage of the maximum, but the sample size is unclear. The sampling strategy in unspecified/unclear.

• Direct care time – based on the Activity Follow methods as described in the PW material, although the sample sizes/observation periods are not specified.

These could be linked in our analysis to a record of each ward’s progress through the PW modules over 16 months, giving the potential to demonstrate changes associated with the implementation of the PW over time. The rate of compliance with the observation audit is increased by 24% (IRR 1.237) after the observation module is completed; overall mean is 6.5 observations fully compliant when the module is not completed, compared with 8.1 when it has been completed (in addition, the more modules that had been completed, the more patients were assessed as having complete observation records). The number of modules completed was a significant predictor of direct care time and staff satisfaction but not patient satisfaction; each module completed increased the direct care time by 1% and staff satisfaction scores by 1%. Adding a test for linear trend over time rendered the results for direct care time non-significant, although the direction of the coefficient was unaltered. The association between modules implemented and staff satisfaction remained significant.

Safety Crosses data on falls, infection and pressure sores were not collected centrally, but used at ward level as audit tools. However, multiple interventions at the same time (e.g. a pressure sore initiative – React to Red – and a trust initiative on mealtimes) prevented staff from accurately ascribing impact to PW. Similarly, trust policies that were still in place at the time of our fieldwork, such as hourly rounding and ‘cohorting’ (putting patients at risk of falls or with dementia into bays together and ensuring that there is always a staff member in the bay), were felt to have increased direct care time, independent of PW.

Monthly spreadsheets showing the progress of PW implementation and results on PW metrics were sent to each directorate but our interviewees felt that neither wards nor the PW team paid sufficient attention to these data; PW was felt to have just ‘faded away’ rather than being robustly evaluated (there was no formal decision made to end support for its ongoing implementation).

Although PW was perceived as time-consuming, leading to staff disengagement when ideas were not acted on (e.g. by the Estates department in the trust), and some disillusionment when excessive claims were made as to its likely impact, overall, staff reported that PW had been worthwhile.

The training given to the PW team and their experiences in leading the implementation of PW – which included widening their networks within the trust – had a significant impact on their working practices and future careers. Two of the facilitators subsequently led other large-scale QI programmes in the trust. More generally, ward managers were given the confidence, skills and opportunities to make changes; nurses commented on how the principles of PW had made an impact on their and colleagues’ ways of thinking.

Assimilation

Material changes (e.g. to storage systems, location of items, additional equipment, changes to the physical environment) did become embedded in day-to-day practice, as did improvements to mealtime processes (e.g. protected mealtimes, red trays for patients who needed help with eating, and greater involvement of ward staff in mealtimes).

There were also several examples of how initiatives inspired by, or at least related to, PW have led to longer-term continuous improvements. For example, the PSAG boards evolved over time into an electronic trust-wide system in 2009, and 5–6 years after initial implementation the KHWD boards were re-evaluated and made more relevant and understandable to patients and visitors. Other examples included the move to hand-held devices for patient observations across the trust and the shift in handovers to ‘accountability handovers’.

Sustainability

Legacy

The trust’s QI programme today remains based on the same principles as PW. For example, site A’s whole-hospital improvement programme continues to use process mapping, which was first introduced to the majority of staff through PW. However, no one has used any of the PW tools or module guides since initial implementation in 2007–10 (even if they had revisited processes subsequently). In terms of specific PW interventions/improvements:

• Knowing How We Are Doing boards are still used in all/majority of wards to display data.

• Wards still display Safety Crosses or safety calendars.

• A minority of wards still use the data dashboard to input ward-level data and generate graphs of metrics over time.

• Changes made to ward layouts have been maintained and designated places for equipment remain on some wards (although floor maps or photographs of what went where have declined over time).

• Storage with colour coding of shelves and drawers is prevalent but is not in all wards.

• Protected mealtimes and red trays remain (Mealtimes Matter is still a trust policy).

• Patient Status at a Glance boards are not on permanent display in any of three wards we visited (had been replaced with an electronic board system that was less visible to staff).

• Some wards still maintain audits of store rooms.

• Few documented audits of processes (mealtimes, patient observations, shift handover, nursing procedures, patient hygiene, medicines round, 5S).

Adoption

On the DoN’s initiative the trust signed up to the PW ‘accelerated membership’ support package in the 2007–8 financial year; at that time, the DoN perceived PW to be a programme that her trust should be seen to be doing (as opposed to being an evidence-based intervention). According to an internal document, PW was seen as a good fit with the trust’s modernisation agenda, running since 2001, which had developed Lean techniques and process improvement tools (i.e. process mapping) and set up a number of process improvement groups. The DoN did not take her decision to adopt PW to the board, regarding it as within her remit of responsibility for nursing and governance. However, the DoN and deputy DoN did seek, and receive, reassurances from the chief operations officer that he would not seek to use any time savings to reduce nursing hours.

Implementation (2008–10)

Reported impact

Internal reports note a mean increase in ‘direct care time’ from a baseline of 42% to 60% at the end of the implementation period, as measured using Activity Follow. However, our review of these measures shows that this was based on only one ‘before PW’ or ‘after PW’ measure; and interviews revealed that Activity Follows followed only one (sometimes different) member of staff. Interviewees spoke in general terms, focusing almost solely on direct care time, patient experience and staff mood. Most did not quantify time savings, but several said that direct care time had increased as a result of time savings achieved from changes made to the physical environment and ward processes. Increased direct care time was said to improve patient experience (because patients liked having more time with nurses, and/or nurses being more available) but others spoke in more general terms. Examples given were more peaceful mealtimes and the introduction of meal vouchers to give patients more choice (as a result of PW feedback gathered on a long-stay ward).

Other impacts reported included financial gains from stock management [a £132,000 saving from September 2008 to July 2010 (thought to be the minimum saving)], financial gains from standardising procurement (e.g. new storage systems allowed bulk buying), and an improved relationship between wards and materials management service. Such impacts were enabled, for example, by presenting Activity Follow data to the trust’s modernisation board in order to secure funding for new ‘see-through’ storage racking on all wards. A standardised system of like-with-like, alphabetised and colour-coded storage was then introduced, making it easier for new or temporary staff to find items quickly, without the need to interrupt another staff member. Pod lockers were also introduced to minimise time spent going from the bed to the medications cupboard. Other changes included an index in store rooms, and the use of photographs and floor maps showing what went where.

Although accounts varied about whether or not ward-level data had been collected prior to PW – although this may simply reflect variation in practice – PW did mark a ‘sea change’ in relation to data. First, it was the first time that data had been put on public display on wards. Second, both the collection and the public display of key PW metrics on safety and reliability of care became trust policy. While PW was reported to have made data intelligible to ward staff, there was a tendency for interviewees to conflate the collection and display of metrics data (an output of the PW) with the actual impact on patient safety and reliability of care.

The decision not to invite all wards to apply to be showcase wards meant that selection relied on the reputation of wards or ward managers. This strategy potentially restricted capacity-building, as the opportunity for ward managers who were relatively unknown (for whatever reason) to put themselves forward to be at the vanguard of PW at the trust was denied to them.

Assimilation

The trust considered continuing funding for PW after the 2-year implementation period, but other priorities took precedence. The few remaining wards implemented PW after the 2-year period (with no formal resources). A ward manager thought that impacts had been sustained for about 1 year after the 2-year implementation period but that some impacts had been lost after that time. Factors that appeared to have hindered assimilation included that medical staff were not successfully engaged and that a loss of protected time meant that momentum was lost. Other factor were staff turnover and a sense that the initial implementation had skimmed over, rather than resolving, underlying issues.

Sustainability

Given that new initiatives were always being introduced, it was difficult to maintain the momentum of PW. Staff turnover in particular was associated with attrition of PW and service improvement knowledge and skills.

Legacy

Productive Ward is still part of the remit of one member of the nurse development team who works almost exclusively on one of the trust sites. At this site there are still ward leads for PW on two wards who have 1 day every 4 weeks as protected time; only these ward leads referred to the PW box set or toolkit (one liked to ‘keep dipping into it’ and had used the process mapping tool; the other had used the box set to refresh her memory of PW when she took over as ward lead for PW, and again before a CQC inspection). On some wards there was a strong culture of service improvement, and staff involvement in spotting problems and putting forward solutions. This was not directly attributed to PW, but changes had been made to ward processes (i.e. handovers) and layout, based on staff suggestions, since PW implementation, and continued to be made. Nonetheless, staff who had arrived at the trust since the implementation period tended not to have heard of PW. There was variable evidence of a service improvement culture on the wards at both sites.

In terms of specific PW interventions/improvements, the following legacies were evident:

• Changes to physical infrastructure (e.g. taking doors off bay desks, creating a discharge lounge with a computer) and additional equipment purchased on the basis of PW (e.g. lockable bedside cupboards where medication could be secured, and additional medicines trolleys).

• Knowing How We Are Doing boards were still used in all/majority of wards to display data.

• Floor stickers on many or all wards had disappeared over time as wards were refurbished.

• Store cupboards still used see-through storage for small items, used like-with-like storage and they were clearly labelled, and there was some use of colour-coding where helpful.

• Evidence of wards continuing to use daily and weekly checklists with respect to keeping store rooms in order; these were audited weekly (although standards in terms of 5S were reported to have declined to a greater or lesser degree in some store rooms over time).

• Improvements in stock management from PW had been maintained across the trust; at one site there was an audited system for night staff to check and restock various trolleys (although the stock cupboards on the ward were not included).

• Standardised PSAG boards were still in place and used to visually flag key patient information related to risks, needs, treatment, admissions and discharge and as part of Ward Rounds and Board rounds; some wards used the PSAG board at shift handover while others used handover sheets.

• Some wards on each site still used and displayed daily Safety Crosses data; not all ward-level data are currently routinely analysed above ward level to look at patterns over time, although – apparently as a result of Safety Crosses data collection – the trust now has special steering groups on, for example, pressure ulcers, which do use the data.

• Some processes had stayed in place (protected mealtimes, audits of patient observations and pressure sores); some had evolved over time, suggesting a continuous improvement mentality (handovers); and others had declined (use of a red peg on curtains and trolleys to remind other staff about privacy, dignity and interruptions). At least in some wards, PW processes were business as usual.

Adoption

There were no data on reasons for adoption other than that the chief nurse at the time of adoption was reported to be forward-thinking and saw PW as a means to improve the quality of care on the ward.

Implementation (June 2007–August 2010)

Reported impact

The trust used PW to standardise storage. Colour-coded, see-through, labelled storage drawers were introduced throughout the trust from the time it moved into the new building.

When the trust moved into the new building, the PW Meals and Medicines modules were used to decide the best processes for delivering meals and medicines in the new physical structure, and (for meals) under a new catering system. The trust also used PW to introduce protected mealtimes in all wards, red trays and a system for managing menus.

The requirement for wards to collect data, and early difficulties with that, had led to developments to improve data collection systems. Prior to PW, data were apparently only available to, or used by, ward managers. Over time they had been made visible to, and were used strategically by, the trust, and the ward team’s knowledge of their ward data also improved.

Average direct care time was reported to have nearly doubled, from a baseline of 24%. Accounts of impact on direct care time from individual wards varied. Where it was thought to have increased, this was as a result of improvements to the layout of the ward and the placement of stores and equipment. Changes made to mealtimes were thought to have improved patient experience, as had a number of changes made in individual wards. For example, there were changes to admissions documentation on one ward; and one division had introduced hygiene packs for patients arriving on the ward from accident and emergency without any toiletries. The women and children’s division acted on patient feedback to set in place twice-yearly open evenings for parents and young people, to improve patient experience.

As well as educating staff on, for example, particular aspects of patient care or trust resources, such as interpreters, PW was said to have improved QI capacity in several ways. It had made staff stop and think about whole processes, look at how the ward was performing and identify areas for improvement, and become better at looking for causes of problems and devising solutions.

Looking at stock levels as part of WOW had improved stock control such that staff less often had to go off the ward in search of stock. WOW had also effected cost saving through standardising equipment and stock.

Assimilation

In September 2010, three and a half years after adoption, and just over a year after moving into the new building, the trust ‘drew a line in the sand’ with PW. According to the NHSI case study, this was because different wards were at different stages of implementation; and the changes to ward teams meant that, even in wards that had implemented PW modules, many of the new team members had not been involved. Working with NHSI they developed a QI ‘bundle’. This used PW as the framework. It used the PW methodology, was modular in structure, used the imagery of the PW house and included the three PW foundation modules (now with slightly altered objectives) and six PW process modules (Patient Observations, Medicines, Patient Hygiene, Ward Round, Shift Handover and Nursing Procedures), again with slightly altered objectives. But it also included six new modules and a reconfigured module (Admissions and Communication, in place of PW’s Admissions and Planned Discharge), making a total of 17 modules. All modules incorporated the relevant elements of other service improvement programmes, namely High Impact Actions, Essence of Care, Energise for Excellence and the trust’s own values. This was known as TQB.

Each module of the TQB consists of a closed-question baseline questionnaire sent to ward managers. This is similar to the PW ‘10 point checklist’, although the items are slightly different. The aim is to help ward managers to identify areas for action, and to provide a baseline ‘score’. Following the completion of the baseline questionnaire, the issue under investigation is to be process mapped and an action plan devised. At the end of 30 days, a review questionnaire using the baseline questions and any additional questions based on the ward’s action plan is then to be completed and RAG rated on progress. The penultimate module is also reviewed and RAG rated at this time, to check whether outcomes have been sustained.

By June 2017, the TQB consisted of 30 modules. In addition to the initial 17 modules in TQB, there were another two ‘review’ modules; a sustainability module, aimed at ensuring awareness of and involvement in the TQB by all ward staff, and ongoing use of the PDSA improvement cycle applied to ward processes. The remaining modules covered new processes and new concerns; or focused on cross-cutting issues (i.e. Patient Experience) or on a reconfigured combination of issues (WOW and General Ward Management).

Sustainability

Legacy

Wards still had the additional equipment they had bought earlier on in implementation, and had continued to buy additional equipment where necessary, to reduce time spent and distance travelled. Apparently there was variability between wards with respect to whether they still had colour-coded, like-with-like storage.

All wards continued to have a KHWD board on display containing items standardised across the trust, although these were not all up to date. Safety Crosses data had been displayed until recently, when the trust had shifted to a reliance on data entered on the Datix reporting system. The PSAG was still used in the majority of wards to display some patient information and to aid discharge, alongside an e-system. Some wards continued to use PSAG boards as part of handover and/or in their board rounds.

Continuing Activity Follow data had shown that ‘direct care time’ had been maintained at just under 50%, although it was reported that some of the impacts on time-savings accruing from the initial implementation of WOW may have been eroded on those wards that had changed location or specialism and had not re-run WOW to examine their new needs.

Resourcing for implementation

Productive Ward was poorly resourced, with no dedicated PW lead. The implementation of Trust Way was supported by two full-time leads (one member of trust staff and one from Unipart), with support, when required, from a further three team members for the first year, and continued without the Unipart post for the remaining 2 years. Wards did not receive any funds to pay for backfill. No external funds were received for implementing either PW or Trust Way.

Implementation

Productive Ward was used strategically in the implementation of Trust Way, as it was thought to have more credibility with nursing staff. When training staff, the Trust Way team referred to the PW manual and used some of the PW tools, but also used Unipart tools and language. The end result was that Trust Way was a merger of Lean tools from Unipart and PW. Trust Way consisted of three core elements:

1. PSAG. This was based entirely on the PW PSAG module, and was used to move from existing ‘bed state’ boards to electronic PSAG boards.

2. 5S. This equated to PW WOW, and the language is that of the 5S game in the PW toolkit – sort (sort out), set (an efficient place for everything and everything in its well-signposted place), shine (cleaning and tidying at least daily), standardise (consistent and standardised work practices with delegated responsibilities), sustain (making changes business as usual and continuously improving standards). There were audit sheets for 5S.

3. Comms Cell. This equated to PW’s KHWD in that it included gathering daily evidence on ward metrics and using Safety Crosses as a visual representation. In addition to daily data on the standard PW safety metrics (falls, pressure ulcers and infection rates), Trust Way gathered daily data on single-sex accommodation breaches; bay nurse attendance at ward round; discharge before 11:00; venous thromboembolism assessment within 6 hours of admission; and staffing per day. An important difference between KHWD and Comms Cell was that Comms Cell was also aimed at creating a formal space for reviewing performance against metrics; and gathering ideas and planning specific, delegated actions for continuous improvement. Regular Comms Cell meetings were to be used for safety briefings at shift handover, and to inform staff of any key messages from the trust or ward.

Implementation strategy

Important components of site D’s implementation strategy were:

• The early adaptation of PW.

• Implementation of Trust Way was the responsibility of the QI team. None of the Trust Way team (the two leads and their manager) had nursing backgrounds.

• The central role played by the QI team, the inclusion of heads of central services in training provided by Unipart, and the inclusion of non-ward areas in the rollout plan suggests a broader focus than ward teams.

• A ‘sustainment’ plan was implemented from the outset.

• The remit of Trust Way was limited to the three foundation modules, and ‘Protected Mealtimes’; although some wards chose to try other elements of PW, and some introduced specific changes (i.e. tabards, ‘do not enter’ pegs on bay curtains, producing a ward vision) without going through any QI processes.

• One clinician was engaged at a strategic level and engaged in implementing Trust Way on the first ward, but there was little other engagement by clinicians beyond that.

Staff engagement

Although comments were made about referring to PW to better engage nursing staff, Trust Way was still reminiscent of the Unipart model and, as such, initially met with some resistance. From interviews with staff on the two randomly selected wards, all of whom had been in post at the time of implementation, it also appeared that the reach and scope of ward staff, especially junior staff, was limited. Ward staff were reported to have been more engaged in certain aspects of Trust Way, namely having input on which ward-level data were useful to them (although this decision appears to have been restricted to ward managers); prioritising areas for action and working out action plans to disseminate to the team (ward managers and deputies); and how they wanted their wards organised in terms of placement of equipment and items (health-care assistants, housekeepers). No interviewees below ward manager level on randomly selected wards had been involved in using any of the tools, apart from the 5S game, to identify room for improvement on the ward, or in making any suggestions or strategic decisions about changes on the ward other than changes to storage. Interviewees either knew nothing about it or had implemented 5S only, by clearing out unwanted paperwork or stock.

Interviews with ward staff suggest that, despite the training plan reported by members of the implementation team, there was very little use of any of the tools for wards to go through the processes of QI. Although reports differed about wards’ freedom to shape the measures they collected, the impression from interviews with ward staff was that wards were presented with PSAG boards, metrics forms to complete and instructions on how to use Comms Cells and told to use them. In the drive to improve discharge, ward rounds and shift handover, a member of clinical staff seconded to work with the Trust Way team observed processes on a select number of wards and then suggested improvements for wards to implement.

Patient and public involvement

There was no strategic PPI. One participating ward reported using (general) patient feedback to shape improvements.

Physical environment

Storage systems were standardised across wards by using a standard colour-coding like-with-like system, with supplies labelled and stored alphabetically within coloured themes. Wards got rid of old stock, and 5S (the Trust Way WOW) also played a part in cost saving by optimising stock levels. Patient notes were better organised. Some wards reorganised where things were kept, and signs and floor maps to the location of equipment were introduced.

Changes to processes

A Protected Mealtimes policy was rolled out trust-wide. As part of Trust Way, the system for handover was introduced, which used a verbal handover supported by a printed-out sheet that nurses could annotate. Handover was to be preceded by a safety brief for all nursing staff, which alerted nursing staff to safety issues of all patients (not just the patients in their bay). The safety brief took part as part of a Comms Cell meeting so that, as well as issues relating to current patients, any key messages from the ward manager or trust could be given, and any improvements in metrics could be shared among the team. No other changes to shift handovers themselves were reported.

Data

Productive Ward was apparently the first time that daily ward-level data had been collected by and visible to ward staff below ward manager level. The data collected covered safety incidences, breaches from protocols, compliance with care pathways and various trust aims, and other ward-specific data thought useful. Consultants sometimes added to the list, requesting data collection on specific conditions. One interviewee felt that the Comms Cell required too many data. Safety Crosses were introduced; in Trust Way these were accompanied by standard actions that needed to be taken, which were described as very effective in driving change.

Impacts on patients

It was reported that direct care time had been measured and had increased, but no data were available. Interviewees thought that protected mealtimes had improved patient experience. The Trust Way team reported a reduction in patient falls overall, and a reduction in length of stay on a number of wards. Individual wards reported making changes that improved patient safety and/or experience, for example improving compliance with care pathways by changing the documentation to help ensure that each process was completed; improving the quality of patient observations by introducing spot checks; improving pain assessment scores; reducing falls by 37% in the first year of using Safety Crosses for falls; and improving the discharge process by making changes to documentation.

Impacts on staff

Involvement in Trust Way increased the skills of the implementation team, who were relatively inexperienced at the time. Changes to processes were reported positively, and made life easier for staff. PSAG boards were introduced under Trust Way. They were reportedly used by different staff groups; helped staff to see which patients needed prioritising; helped medics to prioritise their workload; nurses used it to allocate special beds (i.e. respiratory beds); and they reduced the time taken in handing over information. On some wards they were used for discharge planning. Staff gained satisfaction from feeling that patient care had improved. With respect to staff voice, the safety brief allowed a more formal space for staff to raise ideas and for a group discussion on ways forward than there had been previously. In terms of ward staff QI knowledge and skills, the impact was uneven.

Intervention

As wards completed implementation of the foundation modules and Protected Mealtimes, the Trust Way team would execute their sustainment plan. This consisted of the team creating audits, carrying out monthly spot checks and scoring wards. They created a leaderboard to introduce an element of competition between wards, and the results were reported to the head of nursing, who supported Trust Way, and she reported to the DoN, who reported to the executive board. The team would issue actions to improve based on the scores. Later, they introduced quality control boards, which provided another system of audit and surveillance.

Interviewees working in QI commented that PW/Trust Way would still be highly relevant and useful. For ward staff, the usefulness of any particular element of Trust Way was the key to its sustainability. An example of this was that tying actions into Safety Crosses (as in Trust Way) appeared to increase the usefulness of the tool by allowing ward staff and managers to chart the actions taken following an incident and to review practice. This is likely to explain the continued use of Safety Crosses with actions.

The model for improvement the trust was using at the time of fieldwork was felt to have an advantage over PW/Trust Way in that it was applicable to all areas and processes.

Organisational

The constitution of an almost entirely new Executive marked the end of the trust’s investment in Trust Way. One senior interviewee attributed this to the new members wanting to make their own mark. At the time of fieldwork the trust was about to embark on piloting the Model Ward initiative, for which they planned to look at elements such as WOW, Protected Medicines Rounds and Protected Mealtimes.

Ward or trust champions helped sustainability. With staff turnover, limiting engagement to ward managers risked losing impetus if that person left.

Data collection and staff involvement in QI were supported by wider trust initiatives. Staff leadership courses available at the trust required nurses to implement a QI project. At the time of fieldwork, the trust was using the Model for Improvement, which a member of the QI team likened to PW, albeit that it was more applicable to non-ward areas.

Achievements on stock levels were not maintained. This was attributed to the fact that Stores were not properly engaged during the implementation of Trust Way. However, the introduction of ward budgets had more recently incentivised wards to reduce waste.

Environmental

Nursing staff are now more familiar with measurement and QI, as these have become part of nursing culture. We were told that if PW were to be re-launched it would be important to make sure it was not seen as cost saving, and that people were given time to implement and embed it.

Resourcing for implementation

The trust seconded staff to make up a three-strong PW team of a PW lead, a PW facilitator and a PW administrator. The PW lead and the PW facilitator had nursing backgrounds. These posts were funded by the SHA for 18 months. The PW lead and the managers of the showcase wards attended training by NHSI and SHA networking events. There was no external funding for backfill for ward staff to participate in PW activities. The trust paid for materials (including PSAG and KHWD boards). Shortly after the launch, there was some limited additional input from a member of the QI team who provided some Lean training. After the 18 months, charitable funds were awarded to cover the three posts for a further 18 months, to November 2011.

Implementation

The project team provided classroom-based training to ward managers or nominated ward leads, and made presentations on each ward. These introduced Lean principles and the three foundation modules. Wards were expected to implement the foundation modules, and were strongly encouraged to use PW to at least evaluate all of the PW processes. It was up to wards to decide the order of the process modules and what they wanted to concentrate on within any module. The project team provided on-ward support to implement modules, but this was primarily seen as the responsibility of the ward manager or nominated ward lead. After implementing in three showcase wards, cohorts of around seven wards were launched around every 6 months.

The showcase wards carried out their own Activity Follows. On rollout, at least on some wards, these were done by the PW team in order to relieve the pressure on staff time. The PW implementation team also did a lot of work monitoring patient observations, creating a checklist for ward staff to use, and continuing to audit patient observations. In addition, they went on to wards, observed processes such as mealtimes, and made suggestions. Ward staff remembered them rationalising the stock in the medicines trollies and producing a picture to go in the trolley, showing the layout.

Variability of implementation between wards was thought to have been, at least in part, a result of the re-organisation of some wards (in terms of location, specialism and ward teams), and ward specialism.

Implementation strategy

Important components of site E’s implementation strategy were:

• All grades and types of ward staff were invited to generate ideas and implement changes, although it was felt that greater efforts should have been made to involve junior staff in a meaningful way.

• The role of the PW team as doers rather than facilitators went hand in hand with a low level of training of ward staff in the PW tools and principles.

• The PW team tried to integrate PW with other QI initiatives being implemented at the trust.

• Implementation of PW was tied to the trust’s ward accreditation scheme.

• There was no strategic PPI, and little engagement at ward level.

Staff engagement

Buy-in at divisional chief nurse and matron levels was always an issue, which hindered the project. There was no strategic engagement with medical staff, which affected implementation at ward level. During implementation the team worked with central services and cross-ward teams.

Making things better for patients and making life easier for staff were seen as important incentives. The layout of wards in one building (i.e. long and thin) had been problematic for ward staff, and the promise of reducing the time spent and distance travelled was appealing, as was giving ward staff ‘ownership’ of their ward, implying a sense of responsibility for aspects of improvement. It was widely reported (from the PW team, ward managers and ward staff) that all grades and types of ward staff were invited to generate ideas and implement changes, but it appeared that ward staff were rarely, if ever, given any direct training in PW by the PW team or the Lean facilitator. Rather, ward staff below ward manager level were asked for ideas, followed for Activity Follows and timed, and told what to implement.

Work on and achievements of PW were made part of the trust’s ward accreditation scheme. Wards had to collate and analyse data and present their ward improvements. This encouraged more strategic involvement and learning by at least some staff.

Patient and public involvement

There was no strategic PPI. It was generally reported that, on most wards, inpatients were informed about PW but were not involved further; and all but one interviewee said that neither patients nor visitors had been involved in implementation. The exception was one ward that used a patient survey to gauge awareness of PW and to obtain feedback relevant to PW outcomes.

Physical environment

Productive Ward was used to introduce standardisation of the physical environment with respect to like-with-like colour-coded storage; PSAG boards with standard icons; ‘do not disturb’ signs for bay curtains; and photographs inside medicines trollies showing the location of items. Some wards also organised and rationalised paperwork; made changes to the infrastructure of the ward; purchased additional items of equipment to speed up processes; and introduced bed maps at the entrance to each bay with patients’ names to help, especially, doctors.

Changes to processes

Productive Ward was used to standardise processes with respect to introducing: having a protected mealtimes policy; having a patient observations checklist; using ‘do not disturb’ aprons for nurses on medicines rounds; using the SBAR (situation, background, assessment, recommendation) effective communication tool for shift handovers; and implementing PUPS (Pressure Ulcer Prevention Strategy) – pressure ulcer magnets and intentional rounding for patients at risk of developing pressure ulcers. Individual wards also introduced changes to processes.

Data

Accounts varied in whether or not ward-level data had previously been collected, but PW introduced the display of these data. The vast majority of wards used Safety Crosses for falls, pressure ulcers, MRSA, C. difficile, staff absence and protected mealtimes. Some wards used them for other things, including instances of aggression towards staff and interruptions during medicines rounds.

Impact on patients

Overall, direct care time was said to have increased. The extent of improvement varied by ward, and at least in part reflected their starting positions. WOW was said to be the main contributor to increased time. Safety Crosses made people more aware of falls, pressure ulcers and infections, and reportedly helped ward managers to review practices and training, think critically about how to avoid incidences and, ultimately, improve patient safety. The PW facilitator could not remember whether or not it had had an impact on incidences overall. No quantifiable outcome data were available. With respect to patient experience, the ward manager who collected these data systematically said that patient experience had improved as a result of higher staff morale.

Impact on staff

PW was thought by some to have raised staff morale. Factors behind this were better team working, satisfaction from working productively, a sense of purpose, and a sense of empowerment and ownership (in terms of which it had been sold to ward managers and ward teams). Some ward managers and deputy ward managers involved in implementing PW valued the opportunity it gave to review processes and to think differently. Some, although not all, had learned some QI skills.

Intervention

No comments implied anything about PW with respect to sustainability.

Organisational

The trust restructuring and the end of the PW team that affected the assimilation of PW almost from the point at which implementation ended was a key factor in sustainability.

The expectation that once training had been given to ward managers they would have the skills to train their staff and keep it moving forward turned out to be unrealistic. It relied not only on the keenness of ward managers, but also on their having time to train their staff.

Elements of changes were sustained where there were named people with responsibility for driving and sustaining improvements, such as housekeepers being responsible for keeping the ward clean and tidy, and link nurses whose responsibility covers, for example, nutrition (protected mealtimes) and tissue viability (pressure ulcers).

Stable staffing helped wards to carry on with PW ideas. Two wards where deputy managers at implementation had become ward mangers carried on using PW principles. Conversely, staff turnover meant that participating wards had few staff who had been in post during implementation.

In tying PW into the ward accreditation scheme, the trust increased surveillance of ward activity with respect to PW (and other QI initiatives). The ward accreditation scheme, which was still running, served to underpin some of the PW principles by taking into account innovations, patient satisfaction and ward metrics. In their preceptorship year, new staff were obliged to carry out a small QI project.

Environmental

A CQC visit lay behind the reintroduction of ward data display as trust policy.

Resourcing for implementation

As the trust was a late adopter, no external funds or resources available were from the SHA or NHSI for either training or implementation. The trust’s only contribution was the salaries of the PW team. The management structure for PW included a PW lead who took overall responsibility but did not have any direct engagement with wards. The day-to-day work of implementation was carried out by two PW facilitators who were seconded for 2 years. They reported to the PW lead, and were given mentoring support by a head of nursing. The head of nursing reported to the board on the progress of PW. The PW facilitators sat within a change management team responsible for preparing for the move to the new building. As neither the PW facilitators nor their mentor had any experience of QI, the lack of investment in training was significant. The lack of resources also affected other aspects of implementation, such as learning from other trusts and the scope of possible changes to estates. The PW facilitators did not have time to provide thorough training to ward teams.

Implementation

The perceived difference in aims between members of the PW management team – between cost saving and QI – played out during implementation. Implementation was also hindered by a number of competing initiatives that distracted ward staff. Despite the decision to implement only the three foundation modules across all wards, and up to two process modules in some wards, it was felt that the PW facilitators were still spread too thinly (given the number of wards they had to work with). As there was no funding or other external support, the work of implementation on the ward was completed in addition to everybody’s day job, which made it difficult for ward staff, and prompted the PW facilitators to do a lot of the work themselves (see below). The previous implementation of PW in 2008–9 caused some resistance to the 2011 implementation. In response, the team adjusted the roll-out plan for the second cohort to start with enthusiastic directorates as they sought a better reputation to encourage wards in other directorates. In the end they were victims of their own success. The roll-out plan had included only specified wards (mostly wards that were due to move to the new building) but other wards also wanted input. The facilitators were conscious that the wards omitted from the roll-out plan felt sidelined, so they did end up giving some time (a day or half a day) to the off-plan wards, which stretched their resources even further.

Initially, wards were acting alone in choosing which process module they wanted to implement. But according to the end of project report, by the end of cohorts 3 and 4, all wards from the Surgery and Neurosciences directorates were included in the programme, and rolled out a process module in each (Admissions & Discharge and Shift Handover, respectively).

At the end of the 2-year implementation period, 36 of the 40 wards in the rollout plan had implemented the three foundation modules, and introduced bedside handover (although they did not necessarily go through the shift handover module). Interviews implied that some areas may have had the outcomes of modules imposed on them. For example, PSAG boards were introduced in all wards (regardless of whether or not they were in the roll-out plan), WOW was applied in non-ward areas, referred to as ‘Well Organised Workspace’, and KHWD boards were installed in outpatient areas. According to internal reports, 16 wards across the five directorates (between one and seven wards per directorate) did one process module: Shift Handover, Meals, Medicines, Admissions and Discharge or Ward Round. Again, not all wards went through the process of implementation, but rather adopted the end result. The trust used PW theory as a basis for creating its own discharge module as part of a long-term conditions project. Central services, which worked across wards, helped to ‘pollinate’ wards with productive practices, and working with bodies outside the ward was seen as crucial, albeit time-consuming. Data from trust records suggest that although activities may have been carried out, there was not a good understanding of the principles underlying PW. Some of those we interviewed, who had been involved in one or more aspects of evaluation, felt that the trust had ultimately treated PW as a project, rather than as a vehicle for ongoing QI.

Implementation strategy

Important components of site F’s implementation strategy were:

• The PW steering group comprised two patient representatives, the DoN, the director of facilities, and pharmacy. It did not include representatives for medical staff (doctors, consultants or surgeons) or AHPs.

• Having the director of facilities as a sponsor helped to facilitate working relationships with central services and, given the lack of dedicated resources, his involvement was important.

• Selected modules were to be implemented in 40 wards that were due to move to the new shared site. All implementing wards were to do the foundation modules and shift handover, in order to move to bedside handover before the move to the new 75% single rooms building. Wards could decide which other process module(s) they wanted to implement.

• The roll-out plan was a cohort of five new wards per month, and for three months per cohort. Each head of nursing was asked to nominate a ward in their area with which to start. Early wards appear to have been selected for their reputation for testing innovation.

• PW facilitators and ward staff worked with central services and other staff groups. As mentioned, pharmacy was represented on the steering group, and the PW team and wards implementing the Medicines module worked well with them, particularly collaborating towards a Commissioning for Quality and Innovation54 on missed doses.

• The directorate implementing the Meals module invited input from the director of facilities, the catering manager, the dietitian and nutritional specialists.

• The PW facilitators took a ‘hands-on’ approach to working with wards. As well as carrying out the bulk of baseline Activity Follows, ward staff commented on the amount of work done by the facilitators, particularly with respect to WOW (e.g. labelling, colour-coding, getting rid of out-of-date stock). Ward staff appreciated the work done by the PW facilitators, and said that it made implementation fairly rapid, but there were concerns about whether or not the facilitators got too involved in doing the mundane work associated with PW to the detriment of ward teams’ investment in the programme.

• Baseline Activity Follows were carried out, which categorised activities, counted interruptions and tracked mileage for almost every ward in the rollout plan. The length of observations was 7–8 hours (rather that the recommended 12 hours), as this was more manageable and was thought to be sufficient. Ward managers were tasked with carrying out follow-up Activity Follows but, according to implementation data provide by the trust, completion of follow-up Activity Follows faltered considerably in cohort 3, and was not done at all in cohorts 4 to 8.

• The PW project was run from a project office, in among all the other change programmes leading up to the move to the new single-site hospital; it was run along strong project guidelines. The PW facilitators prepared monthly progress reports identifying key issues, critical tasks to be accomplished by set dates, and risks. Other evaluation activities included an evaluation form given to ward staff, and inspections of store rooms and cupboards by the PW facilitators. The PW facilitators wrote interim and final reports, which reported on roll-out and presented findings based on before-and-after data for wards 1 and 2. It also made suggestions for the sustainability of PW and predictions of the challenges that lay ahead.

Reported impact

The final trust PW report stated that, based on before and after Activity Follow measures in cohorts 1 and 2, the percentage of direct patient care time had increased from an average of 40% to 55% (including time spent on bedside handover). The PSAG boards (according to the report) contributed to a reduction in length of stay by highlighting delays. According to another internal report, some of the time released was spent on the trust policy of ‘intentional rounding’. One interviewee reported that PSAG had made the discharge process smoother, and the nurse-led discharge system introduced in the surgical directorate was deemed a success. Changes made to the mealtime routines through PW allowed staff to spend time helping patients to eat while the food was still hot, which helped patients’ nutrition. The bedside handovers introduced through PW were said to be more patient-centred, and helped to involve the patient in their care. In the end of project report, the missed dose rate on the ward that implemented the Medicines module was reported to have dropped, although there was only one time point before and one after.

No claims were made by any of the interviewees about measurable impact on hard outcome measures related to patients. However, Safety Crosses data were reported by many as raising the awareness of ward staff regarding patient metrics (previously visible only to managers). Ward staff would be quizzed about their scores by matrons or visitors doing walk arounds. There appears to be variability between wards in whether or not ward staff below ward manager level became more involved in carrying out audits. The data were said to have been used by wards, and were reported to heads of nursing, who would include them in their reports. One interviewee said that KHWD data were useful to wards for showing them which groups of patients were staying in hospital longer, and in her area for flagging infection rates. The rates were then improved through changes outside PW processes (e.g. commode cleaning and having more domestic staff). Despite the impact on awareness, the scope and strength of the impact were questioned.

Ward staff felt that PW had made their work more ‘efficient’, and processes ‘slicker’. The internal end of project report in 2013 reported that, among the wards that had before and after Activity Follow measures, time spent in motion had reduced from an average of 14% to 11%, and the mileage walked reduced from 4.5 miles to 3.1 miles per shift. Interviewees reported that time was saved as a result of:

• changes to the physical infrastructure of some wards

• better labelled storage, which saved time looking for things or telling visiting staff where things were

• better stock-taking systems, which prevented staff running out of supplies.

Changes were also made to the allocation of tasks between nursing and other staff (although it is not entirely clear whether this change was during the implementation period or later). These changes shifted responsibility for putting clean laundry away from nurses to laundry staff, and passed responsibility for stock-taking, ordering stock and stocking cupboards to materials management staff.

Patient Status at a Glance was reported as useful for bed managers, nursing staff, therapists, case managers and receptionists, and for patients and visitors. It was used for identifying empty beds, finding patients, quickly referring to what procedures had been completed or were awaited, and discharge planning. PSAG was said to reduce interruptions and save staff time, but the data on interruptions in the end of project report suggest that this is not clear-cut. One interviewee said that having patient information in numerous places (PSAG, bedside board, handover sheet) meant that there were a lot of different things to update, and not all of them would be updated consistently.

Assimilation

Although it was not originally part of a sustainability plan, at the end of the implementation period the PW facilitators prepared a handover document for each ward, detailing the person responsible for PW at ward level, baseline Activity Follow data, progress to date, specific next steps, and a template for reporting outputs and outcomes to the PW operational group.

One year after the end of PW implementation, the trust moved to its new PFI site. This was a major undertaking, with important consequences for the assimilation of PW. The physical infrastructure of the new building caused problems, increasing mileage and ‘wasted’ time. As a result, direct care time was reduced and, because the new building had off-ward storage (which increased the distances travelled and the time taken to get things), wards had to buy extra equipment to keep care nearer the bedside. The PFI building made it prohibitively expensive to make minor changes to the ward environment (e.g. putting up whiteboards or shelving).

However, two examples of successful assimilation over time stood out. First, PSAG developed over a 5-year period to a touch-screen electronic board that has ward-facing details on permanent display and more patient details at the touch of a screen, and is part of a trust-wide bed management system. Second, the admissions and discharge work was incorporated into the discharge bundle and came under the FLOW project, a trust-wide real-time bed state project linking the PSAG boards and the discharge process.

The time frame of implementation was felt to have been too short to thoroughly embed the principles of PW. One ward manager said that since the move to the new single site the general increase in standardisation across the trust (not specifically about PW processes), with staff being told how they must do things, had left nurses feeling disempowered. The focus on audits, and everything needing to be the same across the trust, had left little room for imagination or experimentation.

Sustainability

Given the lack of sustainability planning, there was a strong reliance on legacy in individual staff. However, a high proportion of implementing staff left in one swoop at the time of the move or shortly after, and there were also changes in the senior team. According to one interviewee, this staff turnover meant that, even where practices were maintained (i.e. bedside handover), staff did not necessarily understand the theory underpinning the practices. Other accounts suggest that there had never been a deep understanding of QI theory or methods.

Legacy

It appeared that ward staff below the level of ward manager had never used or seen the PW box set. The ward managers who had used it did not do so any longer. Some wards still had Safety Crosses on display, although our observations showed that these were not always clear or regularly updated.

Observations on wards showed that they remained free of informational clutter. The standardisation of all wards with respect to where things are indicates that WOW principles continue to have an impact. The off-ward stores are now filled and managed by materials management staff. However, there is no longer a colour-coded storage system, and interviews and observations indicated that there is no clear signposting of where things can be found.

Electronic PSAG boards are still in place and every ward had a KHWD board outside with a standard set of trust-required data. Our observations showed that the data were not always up to date or particularly relevant to that specific ward. It was reported that ward managers understood the data, and that was helped by the fact that they now had a supervisory role. Trust-level metrics are e-mailed to all staff on a daily basis, but it is unclear whether or not or how this is used by ward staff. The twice-daily safety briefings cover individual incidents of, for example, falls and infections, and are sometimes used to discuss data at a more strategic level.

Broadly speaking, the system for meals set up after the move is still in place: meals were dispensed room by room; a hot trolley moved from one end of the ward to the other; and magnets and diet boards were used to signal dietary restrictions or help required with eating. However, one ward manager thought that they were no longer good at preparing patients before mealtimes; observations confirmed this to be the case. Interviews and observations showed that there was no system for flagging patients who had missed a meal. A policy of protected mealtimes with respect to staff is still in place, but the trust now allows visitors onto wards throughout the day.

The system for medicine rounds that was put in place after the move is still in place, as is the system for flagging medicines for missed doses when it arrives on the ward. Some wards still use a red tabard for medicine rounds to discourage interruptions.

Bedside handover, with a standard paper handover sheet, combined with a safety briefing, is still in place, although it was felt that some of the principles of bedside handover (interacting with the patient, involving them in their care) have been lost.

Staff involvement in continuous improvement was evident. On the basis of her own ideas and staff suggestions, one ward manager had made a series of improvements since PW. She had introduced a magnet system to signal when people were on breaks (so that no one wasted time looking for that person), and checklists for admissions, skin damage and discharge. Another ward manager was trying to integrate medical and nursing handover but continued to face difficulties engaging surgeons. Although the monthly ward meetings with staff on another ward had ceased when the trust moved to the new building, staff (from health-care assistants to senior nurses) felt that they could still put improvement ideas forward to the ward manager. However, particularly in the light of the ways in which the new building took up nurses’ time, there was little time for taking a holistic review of the ward.

At trust level, PW was said to have been pivotal in the trust’s decision to set up a QI department, which trains staff in QI methodology. The impacts on the PW team shaped their future careers. The senior nurse on the PW team went on to lead the QI team formed after PW implementation. One PW facilitator went on to take a patient safety and quality role at regional level. Another had been asked to lead Lean training across the trust, and she was regularly consulted on Lean methodologies by ward managers.