Knowledge exchange in health-care commissioning: case studies of the use of commercial, not-for-profit and public sector agencies, 2011 14

Norchester

To recruit Norchester, LW opportunistically met an influential Norchester GP at a conference. This academic GP agreed to an interview and then liaised with the chief operating officer of the CCG on our behalf to recruit the CCG.

Norchester CCG was a highly research-aware organisation, largely because of the involvement of this GP academic. Although Norchester was in an affluent area in England, the CCG was facing considerable financial challenges with targeted savings of over £20M by 2014. Nationally known for its pioneering initiatives, the CCG’s primary, somewhat high-risk, strategy for reducing unsustainable spending was to modify about 40% of their contracts by (1) rewarding providers on the basis of achieving outcomes such as fewer deaths in hospital rather than activity (e.g. number of procedures) and (2) commissioning a lead provider who then subcontracted other providers. Information and knowledge to support this work came from local CSU analysts employing Swallow’s risk prediction tool and several commercial and not-for-profit agencies with expertise in Public Health, contracting, business case development and condition expertise.

Norchester was split into localities of different sizes; the largest locality covered a population of 150,000. The CCG was in considerable flux during the first period of fieldwork (summer 2011), which had stabilised by the second period of fieldwork (December 2012 to May 2013), shortly before authorisation. Public Health and CSU colleagues (formerly known as the ‘Decision Support Unit’) were colocated in the same building, although some Public Health professionals also worked in local authority premises. Participants for Norchester were selected on the basis of involvement with Swallow and its tools and role in organisation (e.g. senior leaders). Again, snowball sampling was the principal method of identifying appropriate candidates. Data collection is itemised in Table 1.

Carnford

We were directed to Carnford via a Swallow NHS participant. Carnford covered a 210,000+ population with generally low deprivation in England, and was one of several CCGs operating in the same county. They were under serious financial pressure, having received one of the lowest allocations in England, and felt that ‘operating at the edge of bankruptcy’ had been normalised in their area. There was a lot of interest in data and tools as a route to finding solutions to their problems, and those offering knowledge based on research evidence were enthusiastically received. Public Health and the local CSU were both working closely with the CCG to find ways to provide the knowledge and support they wanted, but the CCG had also engaged an external provider with an accountancy background (Bullfinch: pseudonym) to work on a number of specific projects.

Our first interview was with a GP who was using the Swallow risk prediction tool for end-of-life care. This GP was a member of the Unscheduled Care Board (UCB) that reported to the ‘Clinical Cabinet’, which was described as the CCG’s ‘engine room’ where clinical priorities were identified and work streams were monitored. Perhaps because our entry point was via the UCB, data for this CCG were more operational than strategic. Participants for Carnford were selected on the basis of role in the organisation (e.g. senior leaders, purveyors of research or data) and involvement with Swallow and its tools. Observation at meetings and snowball sampling were the principal methods of identifying appropriate candidates. Data collection ran from May 2012 to May 2013 and is itemised in Table 2.

Penborough

Penborough CCG had contracted Heron for 3 years. This CCG covered a population of about 160,000 and was in the most deprived quintile of local authority areas in England. NHS commissioners in Penborough had a history of working in close partnership with the local authority to achieve an integrated commissioning strategy for health and social care. With the Health and Social Care Act 2012,4 this partnership was maintained and many key staff from the former commissioning structure were retained. Working collaboratively with providers, the local authority, the community and other CCGs was an ethically driven approach, but was also seen as the most efficient and effective way of commissioning. They were idealistic, positive and regarded themselves as an effective commissioning organisation. They also had the confidence to openly admit in CCG board meetings and interviews where improvements could be made.

As stated previously, our entry point to this CCG was two Heron consultants. In keeping with a CCG that prioritised patient involvement and integration with social care, this case had the broadest range of participants. These were selected to include senior leaders, and those involved with Heron and its tools, but also to reflect the organisation’s focus on including knowledge from clinical, managerial and community perspectives. Observation at meetings and snowball sampling were the principal methods of identifying appropriate candidates. Data collection took place from February 2012 to May 2013 and is itemised in Table 3.

Deanshire

Coterminous with its local authority, Deanshire was a county-wide CCG serving a largely rural population of approximately half a million. The practices making up the CCG were organised into localities, each of which nominated a representative to sit on the Clinical Operations Group, the ‘engine room’ of the CCG. Deanshire had a somewhat conservative ethos in ensuring that the CCG ticked the right boxes and was seen to be above board, possibly to maintain its reputation as a leading commissioning organisation, but there were also several examples of innovative, inspiring projects focusing on improving patient care carried out by committed staff.

Like Penborough, Deanshire had formerly contracted Heron but our entry point to this CCG was through the director of commissioning development, who suggested that we focus our enquiry on two commissioning initiatives: the reablement project and the acute stroke project. Work on the reablement project began in 2010 and rollout of the redesigned reablement service across the county was ongoing at the time of data collection. The acute stroke project was at a much earlier stage and grew out of broader work around stroke, which had been instigated by the CCG in response to poor outcomes in Deanshire.

Heron’s presence in the data collected from the CCG was minimal, largely because the contract had stopped several years prior to fieldwork, but other commercial providers of commissioning expertise were visible. These included a commercial company that had worked on the reablement project and a project management contractor who was working on the acute stroke project. Participants for Deanshire were selected basis of role in the organisation (e.g. senior leaders) or in one of the two projects which had been selected as foci. Observation at meetings and snowball sampling were the principal methods of identifying appropriate candidates. Data collection took place from January to April 2013 and is itemised in Table 4.

Definitions of commissioning

A literature review by the University of London found many definitions of ‘commissioning’, which varied across public sectors. 30 In asking for definitions of commissioning from our study participants, a NHS commissioning director said that ‘ideal’ commissioning was the ‘right balance’ between ‘strategic focus on needs assessment and service strategy delivered through contracting’ utilising Public Health and analysts to understand needs, commissioning staff with service specific knowledge to ‘build up a picture of what the whole system needs to look like’ and contracting staff ‘with technical skills to make sure that we use the contracts to deliver that’ (Paula, NHS senior commissioning manager).

Other answers included ‘everything but provision’ (Donald, CCG chairperson) but others argued that provision was also a form of commissioning, as every time a GP issued a prescription or made a referral it had commissioning implications (Jen, commercial consultant). A long-term conditions study also concluded that the ‘strict separation’ of commissioning and provision was notional. 31 The definition of commissioning has even inspired several YouTube™ videos (search www.youtube.com under ‘what is commissioning?’).

Commissioning frameworks

An early, simple conceptual framework of purchasing (the forerunner of commissioning) was suggested by Øvretveit and colleagues using the plan-do-study-act model. 32 Several years later, the Department of Health developed a much more complex model to include functions such as assessing needs, designing services, managing demand and managing performance (Figure 1).

FIGURE 1.

Commissioning framework. Source: Health Reform in England: Update and Commissioning Framework. 33 Copyright Yorkshire and the Humber Public Health Observatory.

The Institute of Public Care split commissioning activities into commissioning and procurement. They expanded and refined the framework to include other common commissioning activities such as gap analysis, business case appraisal, service evaluation, development of service specifications, contract management, performance management and resource allocation Figure 2. Clearly, over the past 20 years, the role of commissioners has evolved substantially.

FIGURE 2.

Institute of Public Care model of commissioning. Reproduced with permission from Developing Intelligent Commissioning Yorkshire and the Humber Joint Improvement Project. Commissioning Model: IPC Joint Model for Public Care. URL: www.yhsccommissioning.org.uk/index.php?pageNo=594. 34

The literature

In comparing these frameworks with ‘real-life’ commissioning, recent studies found that the work of commissioners is complex, disjointed, mutable and fast paced. Moreover, high-quality, effective commissioning is difficult to achieve, not least because the link between commissioning processes and outcomes is tenuous.

In their study of health-care middle managers in four PCTs,11 Checkland and colleagues concluded:

The generic managerial work undertaken by PCT middle managers was messy, fragmented and largely accomplished in meetings . . . [There was] evidence of confusion and overlap between the various commissioning teams and groups. Managers struggle with this and appear to compensate by dividing up their personal responsibilities into ‘pieces of work’.

p. 11

The aim of an action research study led by Smith and colleagues was to learn more about the ‘nitty gritty’ of commissioning long-term conditions in three PCTs. 12 Based on their findings, plus previous work, they suggested that there were two cycles of commissioning. The first was the annual contractual cycle, which was labelled as ‘transactional’. The second cycle was ‘interpersonal’, built on trust, common values, and established and new networks. They identified nine activities of effective commissioning (e.g. getting the balance right between interpersonal and transactional aspects, strong focus on monitoring and using information to inform review). The researchers also detailed seven key themes:

1. The ‘commissioning cycle’ is a misnomer as developmental commissioning running over many years ran in parallel with annual contractual aspects of commissioning.

2. An ‘extraordinary’ level of effort went into commissioning, and this often seemed ‘disproportionate’ to the anticipated or actual outcomes.

3. Many different individuals carried out commissioning tasks including managers and clinicians from providers, GPs, voluntary sector representatives and PCT commissioners. Clinicians’ role was primarily as champions for change.

4. Money was at times peripheral, with the majority of spending remaining in block contracts.

5. Changes brought about through commissioning were incremental. Success was more likely with ‘bite-sized’ completion of tasks within a wider plan.

6. External drivers such as national guidance played a powerful role.

7. Commissioners within this study worked within a context of uncertainty, as fieldwork took place while PCTs were winding up and CCGs emerged.

In our study, participants also mentioned the fast-paced nature of commissioning, whereby those providing support had to be highly flexible to changing needs.

You have to go in with a blank sheet of paper and almost listen to their requirements from the ground up again and see has the landscape changed? Commissioning requirements are a moving feast. Does your tool still – is it still relevant?

Randall, freelance analyst

Our study participants highlighted several processes, skills and viewpoints necessary for commissioning not previously mentioned. With their strategic overview, commissioners took ‘a whole systems approach to the way we develop services and pathways’ (Jane, NHS commissioning manager), because providers may be overly focused on ‘their services, their staff, their accountability’ (Abbie, NHS commissioning manager). Furthermore, leadership and persuasion skills were crucial to link providers together and manage tensions that competing agendas invariably generated. Commissioners, however, did not just have to influence providers. Successful commissioners also continually worked with and competed against their commissioning colleagues. For example, in one CCG we observed a commissioner present a business case for a lymphoedema service to the CCG board. The board had to consider the merits of funding this business case compared with several others recently submitted by other local commissioners; the lymphoedema business case was in direct competition with other proposals.

Having outlined the processes that commissioners engage in and provided a flavour of the shifting, challenging nature of their role, the rest of this chapter focuses on results from our study. The next section covers the reasons that prompted commissioners to look for information and the various pressures they needed to satisfy.

National policy

Although various policy changes have aimed to involve clinicians in commissioning, the 2012 Health and Social Care Act4 brought about a seismic change by establishing over 200 CCGs to replace PCTs across England. Clinical in this case means GPs, as the majority of CCGs were led by GPs,37 although each CCG had a secondary care consultant on its CCG board and a nurse lead for quality. In April 2013, GPs became responsible for 60% of the NHS budget, principally to commission acute and community services. There was substantial variety in how CCGs were configured. For example, some CCGs were a small ‘fleet of foot commissioning body buying in what it needs’ (Simon, CCG chairperson) and some had their own in-house finance and analytical departments.

Although all general practices were legally obliged to belong to a CCG, CCGs were membership organisations, where interested clinicians could opt in (or out). 38 As one chairperson said:

Being part of a CCG and being involved in the CCG are two separate things.

Malcolm, CCG chairperson

The hope was that as local general practices participated in decision-making, they would abide by those decisions, change their behaviours and ‘play by the rules’ (Tom, CCG chairperson). As one CCG chairperson mentioned in an account of dealing with unengaged GPs, ‘you keep looking at the CCG as this third party over there when actually the CCG is you’ (Malcolm, CCG chairperson). However, during fieldwork, several participants worried that general practices would withdraw, unless the ‘central command and control’ style of the NHS was eased to permit more local autonomy.

What we have tried to do is generally take on board the expectations of this redesign, which was to put clinicians at the heart of it, and so we have embedded clinicians throughout the organisation at pretty much most levels, to lead and drive the commissioning. And, as I said, it’s a challenge for us, because whatever is being said, there is still a top-down command and control, a tight grip wanted from the centre at the NHS management line, to ensure and assure that we are going to deliver 20 billion pounds savings. And I absolutely understand that. Versus the expression ‘going to let 1000 flowers bloom’ to allow the GPs to expand and be creative. And I think management at the local level has got that difficult relationship to manage. Because it won’t take much to pee the GPs off and make them walk away from this.

Sarah, NHS commissioning manager

The contribution of clinical commissioners

This section refers mainly to the contribution of GP in their role as commissioners rather than as providers of care. According to one participant, the same GP ‘noisy lights’ still held commissioning roles, as had previously been the case (Patricia, commercial consultant). However, other long-serving commissioning managers from more than one study CCG case site noted that CCGs were more clinically driven than previous commissioning organisations. One commissioner with over 25 years’ experience commented that at board level the clinical commissioning had wrought ‘much more conversation and thinking about . . . clinical knowledge’ (Carla, NHS commissioning manager). One GP clinical lead believed that clinical commissioning meant that raw data were interpreted with a ‘clinical eye’, which would reduce instances of drawing misleading conclusions that might lead to unfair resource allocations to general practices (Patrick, GP commissioner).

We regularly observed clinical input to commissioning decisions in observations of CCG meetings. For example, in one CCG the group were looking at an unscheduled care dashboard to attempt to identify patient groups that could be diverted from hospital. Two GPs had prolonged discussion about cellulitis, applying their clinical knowledge to interpret information from the dashboard.

[Anthony continues getting Teresa to open different bits on the dashboard. He says there are lots of areas which appear to be ‘low hanging fruit’, and mentions cellulitis. He says ten people a month were being admitted with that, and asks why they weren’t being cared for in the community [he says something about what treatment would be], and also says something about the mindset around admissions.]

Be a bit careful about whether it is actually cellulitis they are being admitted for – often it is because there has been a social care breakdown.

But they still should not be an acute admission – in that case they should be admitted to a nursing home.

It is still not as simple as putting in a venflon. Anthony says that this kind of data still gives them a way to look at things which need sorting. He says DVTs are another example.

Meeting observation

Several participants, including GP commissioners, commented that an advantage of involving GPs in commissioning was that GPs were ‘closer’ to patients, which made decisions more ‘real’. GPs also had useful local knowledge of how the health-care system worked. Although during fieldwork the specific contribution of GPs was still being worked out, several participants mentioned that GPs were well suited to developing services and designing patient pathways. We observed many examples of clinical commissioners involved in service redesign and provision, such as establishing a GP role in hospital emergency departments. In addition to service development, GP commissioners were identified by participants as having an important role in challenging their hospital consultant colleagues.

It’s so predictable . . . you always go through the same process that they [hospital] are particularly hard done by, they have cut everything to the bone, and there’s all these vital initiatives that [the local health-care economy] must have, and we’re so far behind everywhere else, and GPs are wonderful, they only refer things that absolutely need doing, and we never see anybody for a follow up, and all this stuff, you know, this comes out. And finding places, clinical areas where you can begin to change behaviour is hugely difficult. So what I think we have to do, as GPs, is to try and anticipate and understand those areas. So this is what I regard as almost the only bit that GP commissioners add value, is that we hopefully can see through the guff that our clinical colleagues in specialties will put up and begin to challenge some of that. But of course that’s quite difficult. They are the experts and so on.

Angus, former PCT chairperson and GP

To change GP behaviour, this participant said that GPs liked information that suggested they were ‘in the middle’ of the pack and that if they were not, their behaviour could be shifted, but only if the data were trustworthy (Angus, former PCT chairperson and GP). Another agreed saying ‘If there’s a really good arguments, if you can see the graphs, see the numbers people will almost change overnight because they generally want to do the right thing’ (Patrick, GP commissioner). One chairperson from another CCG recounted that ‘one of the powerful phrases’ he had heard from a GP was ‘show me the data and I’ll sort myself out’.

So a lot of the time people just don’t know that the way they’re behaving clinically is at variance with their colleagues. They may not know that they ask for 50% more ultrasound scans than anybody else. And if you point that out to them, then they can actually say, ‘Oh OK, right, every time I write ultrasound scan perhaps I should go and talk to a colleague and say would you have done that, should I be doing something differently?’ So it’s not necessarily being done in a judgemental way; it’s being done in a sort of actually just to inform you and see if you can solve the problem yourself.

Tom, CCG chairperson

Another CCG chairperson recounted two examples of behavioural change due to peer review. Once was when data suggested that his own behaviour with X-ray referrals meant that his practice was the second highest user of X-ray services in the CCG, and the other instance was with a GP colleague.

We’ve got a GP who has been known about by the PCT for years, who spends exactly twice as much on drugs as anyone else in our group. [PCT chief executive] would go round and plead with him. He’d be very polite, give him tea, coffee, biscuits, anything he liked, but what really mattered [was] controlling his prescribing expenditure. Since he’s had peer review, and charts up on the wall showing where he is with his peers, he’s accepted mentorship from one of our GP leads, you know, and he’s starting to make some progress. In other words, he’d flick a V sign at the, you know, the managers, but he doesn’t do that to his peers.

Martin, CCG chairperson

Challenges

During fieldwork from 2011 to 2013, several challenges were noted facing the introduction of CCGs. GPs carried out their commissioning duties in addition to their ‘day job’ as GPs. In working two jobs, interview participants had widespread concerns that GPs had insufficient time for commissioning. Moreover, several voiced fears that if pushed to choose, GPs would keep their clinical jobs because their commissioning activities were extraneous. At least one GP commissioner voiced the attitude of many in stating that he was still mainly a ‘jobbing GP’ (Anthony, GP commissioner).

Several participants noted that GPs needed to shift from their ‘provider’ stance to a wider population perspective, mindful of the wider system. Without this wider system view and an understanding of the impact of their decision-making, GP commissioners could inadvertently make the situation worse. For example, one participant was concerned that GPs might not have sufficient understanding of the financial implications of service changes. He gave a complicated account of how if some hospital services were moved into the community or private sector, the hospital would still have the same overheads and this would be passed on to commissioners’ bills. Commissioners would then become trapped because the costs at the hospital would stay the same (or increase) and they would also be paying for a similar service elsewhere, in effect paying twice. He believed that the only way to actually save any money was to take entire services out of the hospital and commission them elsewhere, such as the British Pregnancy Advisory Service (Joel, commercial analyst). This was quite a sophisticated argument, possibly unknown to many GP commissioners (or, indeed, NHS commissioning managers).

Another commissioning manager gave a different example of how inexperienced GP commissioners could accidentally create more problems.

I think – engineering analogy coming again – it’s really easy to try and fix something at one part of the system and make it worse somewhere else. So you can focus around a particular issue, let’s imagine you look at a rapid response community service, and you employ a team of nurses to respond to medical crises that would normally have ended up in A&E [accident and emergency], and it looks like you’re doing a really good job. You’re recording the fact that you’re avoiding all of these admissions, but what you’re not recording is what happens to those people in 30 days, 90 days, whether they’re going back in twice as many times. So that’s just an arbitrary example of how easy it is within all of this to try and focus on something and fix something, but actually make it worse somewhere else.

Alan, commissioning manager

Given these concerns, fears were commonly voiced from NHS commissioning managers, commercial providers, Public Health professionals and the GPs themselves that GPs lacked the necessary skills.

I mean where do we get the expertise? . . . You know, are GPs born managers? . . . I now find at the end of my career that slightly perversely I’m having to sort of learn new skills to supposedly support the care of my patients.

Roger, GP commissioner

An experienced commissioning manager identified those skills as understanding legal duties and responsibilities such as budgets, managing contracts and poorly performing providers and identifying and interpreting information, especially around costs and cost utilisation (Paula, NHS senior commissioning manager). Although some GPs had previously been involved in commissioning, it was a major leap from project-based PBC to strategically managing million-pound budgets across whole health economies in CCGs. For some, amassing this expertise within 2 years was daunting. In fact, when asked what he would do when the PCT dissolved, one CCG chairperson responded candidly, ‘Cry’ (Martin, CCG chairperson). However, these data were collected from 2011 to early 2013, when CCGs were still in the authorisation phase; their confidence may have grown as they moved from shadowing PCTs to full responsibility.

Having worked internationally, analysts from one commercial provider expressed several other concerns about the implications of the policy of clinical commissioning on data collection and interpretation. One pointed out that with the slimming down of commissioning, those left had to be more multiskilled. An American analyst commented that by shifting financial responsibility to providers, which was also a trend in the USA, ‘there should be a pretty heavy incentive for these groups to have analytics so they can manage their populations’ (Sergio, commercial analyst). Another analyst agreed noting that GP commissioners were not going to understand cost and utilisation modelling and so there was a ‘kind of commissioning support service that will be probably more essential now than even it was before’ (Brenda, commercial analyst). Tim thought that the advent of CCGs meant lots of smaller contracts with smaller data sets, which were more difficult to manage. This would increase the amount of analyst time needed to reconcile those data sets (Tim, commercial analyst).

Definitions and challenges

The second model identified from our data was integrated health and social care commissioning. Often known as joint commissioning, this has been a policy feature since the split between Health and Local Authorities in 1991. Some policy initiatives focused on pooling health and social care budgets, initially with the Health Act Flexibilities in 1999 and later in 2006. 39 Most recently in the 2013 Spending Review, the government committed £3.8B to the ‘Better Care Fund’ by ‘top slicing’ a proportion of funding allocated to CCGs into pooled funds for health and social care. 40 This was not new money.

In general, the NHS has been responsible for health care, while local authorities deliver social care, for example help with bathing and dressing for the frail elderly living at home. However, clarifying needs as health or social care can be challenging, with some care recipients experiencing duplication while others fall through the gaps. A major reason for disagreement is that health and social care have separate budgets, and so the classification of a particular need as ‘health’ or ‘social’ care has financial implications.

One participant, with a specific remit to develop joint commissioning, commented that health and social care were not ‘easy bedfellows’, partly because their approaches to commissioning came from ‘medical research’ and ‘social science’ mind-sets, respectively. He noted that despite ‘desperate’ attempts to integrate health and social care, ‘it’s very, very hard because . . . it’s two silos with a lot of aggro’, which was mirrored from government level to the local system (Percy, former PCT chief executive officer and Department of Health senior manager, now freelance consultant). Moreover, a recent study found that despite widespread perceptions of the benefits for patients, there was limited evidence that joint commissioning improved outcomes,41 although a former King’s Fund fellow argued that such evidence was difficult to generate, partly because other initiatives might also have had an impact on outcomes. 42

Examples from the Clinical Commissioning Group case sites

The level of involvement of local authorities into NHS commissioning strategically varied across CCG case sites. In one, a Social Services representative sat on one of the key CCG decision-making forums but did not make any comments in any observed meetings. In two others, Social Services representatives attended and made several contributions. The fourth was striving to be a fully integrated organisation and not only included local authority representatives (who appeared to be highly influential in observations of meetings), but also articulate elected councillors. At operational level within the CCG case sites, integration of health and social care took forms such as integrated health and social care teams, jointly funded collaborative projects and overarching programmes.

In one commissioning case site, the integrated services programme was repeatedly discussed at meetings of the UCB. This programme sought integration of different health sectors (i.e. acute and community) as well as health and social care. It emphasised operational changes, such as improving information sharing so that social care needs could be identified and addressed within health assessments. An integrated care team made up of a GP, a community matron, a community nurses and an adult services social worker aimed to offer ‘joined up care’ to those with long-term conditions (press briefing). Another prominent initiative integrated a walk-in centre and a GP out-of-hours service with the emergency department so that all urgent care was located in one place. A joint discharge scheme had been set up and the first few patients had been through this successfully. The UCB were considering joint care packages to prevent the delay of discharge from hospitals because of questions about the appropriateness of health or social care.

With regards to impact, at one UCB meeting a ‘taking stock’ discussion noted positive culture change, for example with providers from hospitals and community sharing information, but joint commissioning was not yet a reality. Instead, in an interview, the freelance consultant leading this work commented that the intermediate aim was to develop joint service specifications, which he coined as ‘complementary commissioning’, before pooled budgets were achievable. Participants were generally enthusiastic about this programme, but more progress was needed.

In a second commissioning organisation where we found evidence of integrated health and social care commissioning, 11 organisations from across voluntary, health and social care sectors worked together to support individuals ‘to get back on their feet following treatment in an acute or community hospital’ (briefing paper, January 2012). The impetus for this initiative was feedback from patients that their needs were not being met. With help from a commercial agency using a methodology focused on patient experience, the project team mapped current services and detailed five patient pathways. They also carried out service user consultations, stakeholder engagement and shadowing exercises. The shadowing included professionals from different organisations observing each other at work and teams of professionals from both social and health care (including senior managers) visiting service users. This was particularly insightful in clarifying service user needs.

So we took a group of professionals, occupational therapists, physiotherapists, social worker, care worker . . . So they went into somebody’s home . . . they learnt [that] having a professional doing something led to a greater quality of interaction. So you can sit down with a cup of tea and say, ‘I’m here to listen to you. Tell me about yourself,’ but they got a lot more in terms of understanding what was going on with that person from working with them around helping them to walk or helping them to do something, that was consistent. It just was a more natural conversation. They’d bump into the MBE medal on the wall and have a conversation around that, and there were things that helped them [professionals] to understand what mattered to them [service users].

Daniel, commercial consultant

From these activities, the team reported to the CCG board:

Assessment and treatment was undertaken in silos – the results of this validated the observations from our initial shadowing work that service users felt passed from service to service without being fully understood or [having] the things that mattered to them addressed in a meaningful way. The consequences of this for the system are a huge amount of unnecessary cost, duplication, waste, inefficiency and low morale with services responding to a significant amount of failure demand, seeing the same service users again and again.

Briefing paper

The intervention designed was an integrated service consisting of NHS staff (occupational therapist, physiotherapist) and social services professionals (occupational therapist, social worker, care worker supervisor). This was piloted at a GP surgery and then spread across the locality with five further integrated teams. The integrated teams were ‘not constrained by organisational barriers and systems’ and were encouraged to develop innovative approaches to problems (briefing paper).

An evaluation of the service was carried out by the local Public Health director comparing a cohort of service users with non-service users (control group), with the intended outcomes of reduction in hospital admissions and social care packages. The results were a small reduction in the hospital admissions between the 12 months before and 3 months after the intervention for the service users (no mention of comparison with control group). The cost of social care packages, however, went up for both the service user and control groups, although it was significantly less for service users (briefing paper). Using a researchers’ lens, the conclusions would be that the effect of the intervention was weak, as there was no reduction in hospital admissions between the intervention and control group and the costs of social care packages increased. (NB: we did not have access to the original report.) Interestingly, these findings were reported differently at the CCG governing board meeting.

The principal outcomes of the study indicated: a significant difference between the patient groups in the number of hospital re-admissions and a significant reduction in social care costs.

Meeting minutes

Based on the data we have, in being transmitted to the board the impact of the intervention became substantially overstated. These minutes also mention that the occupational therapists had reported a ‘re-discovery of their skills’ and GP feedback was positive (meeting minutes). The project was recommissioned at £250,000 annually on a recurring basis. Interview participants had widespread enthusiasm for this project, citing how it had improved morale.

For another commissioning organisation, integration of health and social care commissioning occurred strategically and operationally and permeated every aspect of their approach. It appeared to have unanimous support, although dissent on this front might be difficult in an organisation so committed. Health and social care had pooled their entire budgets several years ago, partly in a ‘defensive’ move to stop the local PCT from being amalgamated with neighbouring PCTs in early reforms. They made infrastructure changes, such as moving Public Health and children’s services to the local authority and shifting adult social care services into health. The aim was to create an ‘integrated system as opposed to an integrated single organisation’, although one participant noted that this was difficult when issues of ‘self-preservation’ arose (Alan, NHS commissioning manager). Decisions were taken with ‘a no detriment advantage [in] that everything we do is about no detriment to the other party’ (George, Social Services manager). This meant that in making commissioning decisions, the board, including the highly vocal locally elected councillors, deliberated on the effect of actions both on key health outcomes such as hospital admissions and on social care indicators such as residential care placements.

A CCG director described the social care voice as ‘very strong’ (Carol, NHS commissioning manager) and interview and observation data suggested that those from the local authority were highly influential. Moreover, lay members and patients played a major role. The CCG chairperson was a lay representative, with a GP as vice-chairperson. Small, three-person teams, consisting of a clinical lead, a service manager and a lay person, led each of their seven work streams (e.g. unscheduled care, disabilities, older people). The CCG had set up a community forum whose members worked in various capacities across the CCG. Furthermore, ideas for service changes were put to a ‘Dragon’s Den’ panel of service managers, clinicians and lay people.

The impetus for such high levels of community involvement appeared partly as a result of ethical values, but also because of a desire for informed community debate on the most difficult issues facing the commissioning organisation. Community knowledge was important in challenging clinical knowledge and priorities and engagement also sought to achieve acceptance of difficult decisions and public behaviour change. However, such openness to public involvement also brought difficulties; for example, a CCG board member had accidentally caused an outcry from remarks made in a radio programme and we observed a member of the public heatedly challenge the CCG board in a public meeting. (NB: the CCG members addressed this individual by his first name and invited him to talk with the deputy director further, and so presumably he was well known.)

Outsourced contract

For the third model we have coined the term ‘commercial provider commissioning’, which applies only to instances where the commissioning is done exclusively by the external provider. Although in this study commercial providers mainly had a role in advising and supporting clients through the deployment of tools and commissioning expertise, we found one example of ‘commercial provider commissioning’, where commissioning had been completely outsourced to external providers. The commercial provider had taken over all aspects of the commissioning process, including contracting and performance management. (NB: the use of the term ‘commercial provider’ in this context does not refer in any way to commercial providers of actual health-care services such as private hospitals).

This started in 2009. The commercial provider team managed the contracts of a group of large, powerful acute providers, which were ‘very expensive and quite difficult to control’ (Joel, commercial consultant). The remaining NHS staff managed another set of contracts with acute providers in a different geographical area. The commercial provider team consisted of a programme manager, nurses, administrators and ‘lots of analysts’. Although the NHS and commercial provider teams were colocated on the same site, there was a clear split in their remits and significant hostility from the NHS team, which may be a reason why only one operational NHS team member was willing to participate in this study.

Definition of commissioning by commercial providers

While participants from advisory commercial providers defined commissioning in a variety of ways, often comparable with NHS study participants, the definition offered from the company with the outsourcing contract was consistently in terms of managing supply, demand and consumer engagement. Managing supply was defined as getting the right providers, contracts and management processes, which one participant thought was ‘simple’ (Larry, commercial consultant). Managing demand was ensuring that providers deliver the right health care, which another said was critical to ensure that ‘people are only passing deeper into the system at the appropriate point’ (Jemma, commercial consultant). Managing the health-care consumer was about interacting with patients and the public to navigate their way through the system.

The importance of data

Crucial to the commissioning ethos of this commercial provider was to ‘use data to drive decision making’ (Kristen and Thomas, commercial consultants). They largely saw the ‘problem’ as a NHS driven by politics and people rather than by data. Several participants talked about the provision of data to ‘move you beyond that purely political/anecdotal approach’ (Jemma, commercial consultant). Another said ‘we have always said data are the lifeblood of any organisation’ (Dennis, commercial consultant). They wanted to ‘integrate the system’ by linking medical records, through better analysis of data to understand ‘what’s driving your costs’ and by engaging patients to change their behaviour. Moreover, the aim of their interventions was to develop or find a ‘single source of the truth’ (i.e. reliable, consistent data). This phrase emerged repeatedly in interview and observation data.

Analysts for this outsourced service undertook ‘forensic investigation of the data’, mainly finding errors in coding leading to overcharging (Joel, commercial analyst). However, when the commercial provider team started the contract, there was ‘pretty much no data’ and so the hospital providers ‘kind of managed themselves with little contract management’ (Harlow, commercial analyst). Only 2 of the 17 hospitals supplied hospital (Secondary Uses Service) data, which:

. . . gives you granularity that finally you can start unpicking exactly what’s going on for the patient, you can start looking at pathways and so on . . . [otherwise you are paying] for something that you don’t even know what it is.

Dennis, commercial consultant

The approach the commercial provider took with hospital providers was to say:

‘If you don’t supply us with this data, we can’t validate our patient activity, therefore we are not going to pay for it.’ So slight – at times, very antagonistic approach. And I remember we sent out these letters and we were lambasted and lambasted by the PCTs, lambasted by the providers.

Dennis, commercial consultant

In an attempt to reduce the hostility between the commercial provider, acute providers and PCTs, a NHS commissioner was seconded for 1 year to the commercial provider mid-contract. During fieldwork, several participants noted that relationships had since significantly improved. This ‘firm but principled’ stance of ‘no data, no payment’ (Dennis, commercial consultant) suggests that the commercial provider was willing to sacrifice comfortable relationships for the provision of good-quality, useable data. This is not to say that all commercial companies would prioritise data and relationships similarly.

Accountability and performance management

In addition to prioritising measurement and data capture, another characteristic of commercial provider commissioning was the emphasis on accountability and performance management. For this contract, the commercial provider placed nurses into hospitals to verify patient notes against invoices, as often commissioners have limited ways of checking the accuracy of claims. Feedback from the hospitals was that clinicians were essential in this role to make clinical judgements. A commercial provider thought that embedding the nurses in the hospital teams was ‘invaluable’ in identifying anomalies because ‘every single patient going into that trust is being reviewed by our nurse on site’ (Dennis commercial consultant). For example, if there was any bed blocking, the nurses would ‘pick up the phone’ to contract leads and investigate further. The nurses also regularly reported back performance management issues to the PCTs. Another participant was less positive about the embedded nurses, seeing it as a lost opportunity, as, apart from ‘one or two significant issues’ of quality and safety, the nurse did not capitalise on driving up quality or challenging clinical behaviours. Measuring the impact of the nurses was also difficult, and this initiative was stopped.

Knowledge needs for clinical commissioning

Within clinical commissioning, local GP knowledge of the health economy and service provision was important. As key gatekeepers, they were assumed to provide an understanding of how best to deliver care to patients. However, GP commissioners were cautious about relying on their local knowledge alone. In our observations of commissioning meetings, clinical commissioners repeatedly called for evidence to inform which services to commission anew and which services to recommission. This did not have to be research based, but clinical commissioners were clearly uncomfortable making decisions in a vacuum. Clinical commissioners were also wanted robust evaluation ‘to say “yes” and it is delivering what we expected – or it isn’t’ (Clara, NHS commissioning manager). For example, one CCG chairperson mentioned that they had about eight different initiatives to address problems around delayed transfer of care, but no mechanism to find out if any were ‘contributing well or contributing badly to that final outcome’ (Simon, CCG chairperson).

In terms of information, NHS commissioning managers and commercial providers commented that clinical commissioners needed information faster than PCTs, as GPs were used to ‘real-time’ data with general practice information technology (IT) systems. In one commissioning study site, extensive work was undertaken to update a pathway tool with locally informed pathways, which, the lead was pleased to report in a board meeting, appeared increasingly used during consultations. In another commissioning organisation, several participants talked about wanting the ‘holy grail of a very simple, understandable dashboard for the general financial picture that is useable and understandable to GPs’ (Craig, GP commissioner). Ideally, they wanted this tool to track real-time activity and financial flows to find out how money was spent.

We observed anxieties around discrepant data in several commissioning meetings and this concern also surfaced regularly in interview data. For example, one CCG chairperson said that disagreements about data were a ‘fundamental flaw of our system’ (Simon, CCG chairperson). Another GP commissioner was keen to get consistent data because of the situation of ‘oh God, we went to this meeting with one set of figures, now I see another set that tell a completely different story’ (Angus, GP commissioner). One GP clinical lead commented:

I think GPs have a very limited tolerance for being given information in data, and as soon as you present, you have two different sets of data, they typically devalue both of them and believe neither of them.

Craig, GP commissioner

This issue was of high priority to clinical commissioners, as they were well aware that without consistent data, behavioural change among their colleagues was unlikely. However, several participants also mentioned that the lack of consistent data or repeated requests for further information was sometimes an excuse for clinical commissioners to delay or avoid taking difficult decisions.

I sometimes get quite frustrated at people’s constant requests for more and more and more local data, and more and more and more analysis, when really it’s quite clear what we need to do, and actually more data is not going to change the actions that you need to take . . . actually, you know, it’s almost like that’s just a mechanism for not taking [laughs] any action.

Sandra, Public Health consultant and GP

Knowledge needs for integrated health and social care commissioning

The knowledge privileged in health and social care commissioning in two of the three examples was largely that of service users. In the integrated project, service users were the reference point in understanding what worked (and what did not) and their perspectives were gathered in various ways (pathways, consultations, shadowing). Similarly, mechanisms for incorporating service user views and their expertise were instituted in the third commissioning organisation including service users as strategic leads, elected councillors as board members, and a community forum. In comparison, the data for one CCG suggested service user involvement were minimal.

Do you see any patients around here today? No. So I think it is a bit about getting patients involved, both in advising, helping with decision making and then communicating decisions to the public. I think to have laypeople communicating messages would be very powerful alongside clinicians or board members or politicians or whatever. And at the moment we have very little of that.

Anthony, GP commissioner

External expertise also contributed to integrated health and social care commissioning initiatives. For example, a large international accountancy consultancy helped with the urgent care initiative and was rated highly by the CCG chairperson. Elsewhere, a commercial consultancy was brought in to use their systems methodology to map and understand ‘customer’ demand. (NB: in briefing papers, terminology such as ‘failure demand’ appear regularly, suggesting that at least some commissioners had become highly adept in its use.) Several participants were positive about the contribution of this consultancy. In another example, Public Health led on the evaluation of an initiative. Finally, in one commissioning case site, a commercial provider supplied expertise to set up a ‘Community Forum’ and small group structures, which was also appreciated.

Knowledge needs for commercial provider commissioning

Given the emphasis on good-quality data, unsurprisingly analysts’ expertise was highly valued in the commercial provider commissioning model. This was crucial to feed into and inform decision-making. Pulling together data from disparate sources and increasing its quality had been a major task. Keeping on top of the contract also required excellent analytical skills. A NHS client said that the key to this contract was that the commercial provider did ‘the basics really well’, which resulted in savings estimated at over £1M in the first year.

Initially within the contract, a clear knowledge translation strategy had been costed in by the commercial provider so that a NHS team could pick up the skills of the commercial provider, but this was eliminated early on by the NHS client to reduce contract costs. Therefore, scant ‘legacy planning’ was in place and the costs of the contract were likely to increase again, once the commercial provider left. However, there was little evidence that the commercial provider would leave, as the initial contract for 3 years was renewed for 2 more, and then after a further tending exercise the commercial provider was awarded another 3-year contract with an extended 2-year option. By 2019, the commercial provider may have run this outsourced commissioning service for 10 years with no mechanisms in place to transfer skills into the NHS/CSU. This further deskilled local NHS/CSU analysts and commissioners and effectively made the NHS clients completely reliant on commercial provider support.

Box 4 summarises the key points in this chapter.

Classification of sources

In building a cohesive case for action, practising the art of commissioning and manoeuvring their proposals through the system, commissioners sought knowledge and information from many sources to determine, clarify, substantiate and defend their position. An overview of those sources commonly named by commissioners and observed in commissioning meetings is detailed in Tables 7–10, categorised as ‘people’, ‘organisations’, ‘tools’ and ‘research based’. These tables have been constructed by a commissioner working with our research team using the categories commonly understood by commissioners, and are not exhaustive, as other sources and types of knowledge undoubtedly exist and these sources offer other types of knowledge not named by participants. As this is not a quantitative survey, we make no claims to the representativeness of this classification, nor do we attribute weightings to the particular factors. However, Clarke and colleagues conducted a survey on sources of evidence used by commissioners,43 which offers additional useful information.

Which sources are commissioners more likely to trust?

Among national sources, commissioners appeared to trust certain organisations more, namely the National Institute for Health and Care Excellence (NICE), The King’s Fund and NHS Improving Quality (formerly NHS Institute for Innovation and Improvement). Participants viewed NICE as providing rigorous, but not always easily applicable, information. NHS Improving Quality and The King’s Fund were useful sources of best practice guidance and new ways of commissioning. In contrast, several participants, especially GP commissioners, seemed sceptical about the rigor of the information from the Department of Health.

Local information often trumped generalised research-based knowledge or information from other localities. For example, ‘telehealth’ was a national ‘must do’, but clinicians were wary about its local applicability. In one CCG, a local Public Health consultant compiled a briefing about the academic research evidence behind ‘telehealth’. This briefing appeared to reconfirm, and perhaps even strengthen, the CCG’s stance that ‘telehealth’ was not a useful intervention. One month later, a health-care provider implementing ‘telehealth’ presented data on the first eight patients presenting at their local service and noted the reduction in hospital admissions. His conclusion was ‘the data is showing that it’s really starting to make a benefit’. Although the committee considering ‘telehealth’ did not completely reverse their earlier position, they became more positive.

We’re not entirely sure but it seems to have some evidence that something happening.

Can you tell us the key areas where it’s starting to show benefit? COPD [chronic obstructive pulmonary disease] isn’t.

Heart failure.

It’s soft measures such as empowerment.

Telemedicine helping people to feel they can take control but it’s hard to measure.

Marvellous!

So watch this space.

Meeting observation

Although many factors might have contributed to the group’s more welcoming response to ‘telehealth’, the presentation of positive local data appeared persuasive enough to start to reverse previously held opinions substantiated by research evidence.

Verbal

The mechanisms whereby commissioners acquired knowledge included conversations, stories and documentation.

Verbal exchange was particularly well suited to the fast-paced, rapidly changing environment in which commissioners worked. Conversations were an important way of getting information quickly through chance encounters, formal meetings and informal gatherings. The contingent nature of these exchanges, however, meant that if a different combination of individuals had happened to meet, different knowledge would have been acquired and perhaps a different set of decisions would have been made.

. . . popping in to see various GPs on my way somewhere else because they had 5 minutes to spare and wanted to chat. But we’ve come up with these ideas, and it’s just been because we’ve sat there and had the time to chat.

Randall, freelance analyst

Knowledge was also acquired through stories. Sometimes stories were told to substantiate a viewpoint, and sometimes stories about patients and services were recounted to appeal to ‘common sense’ and/or make an emotional impact.

So the example that we used, and that really hit the team, was a chap who was having problems with dexterity in his hands – has anybody related this story to you already?

No, no, not this one.

. . . And the clothing side, he was struggling to do his buttons and to do the collars of his shirt. They spent some time with him and, you know, the house was clean, he was a very proud person. And the old approach would either be to put in a package of care around help to dress himself, or making him wear some sort of T-shirt or something. Actually we were able to take his own formal shirts, and behind the buttons put in some Velcro, so that he could wear his own shirts and have that sense of self-esteem, and not have somebody coming in every day which, you know, the learning from the team was people lose their independence the more they have people doing for them. So those types of simple things, but you can imagine, in a normal system, if you said, ‘Well I need to put some Velcro in,’ you know, they’d be hunting for the Velcro budget, we don’t have one, put in a ‘package of care’ around clothing!

Daniel, commercial consultant

Stories could be powerful in influencing commissioning decision-making. For example, a commissioning manager said that stories were important to persist with proposals through the lengthy, repeated decision-making cycles.

And I have often thought in the past you need the story of the change. Because ideally from the time you’ve gone through health scrutiny committee, the CCG, other local groups, other stakeholder groups and especially if you get to a procurement exercise where actually you might draw services to a close and you know people may be TUPE’d [Transfer of Undertakings] or made redundant and there’s some heavy duty consequences for people, you need a compelling story. And often that is much more powerful than data that you want to throw at people, and so having the clinical stories is really important and the patient’s story is really, really key.

Harry, NHS commissioning manager

But not everyone was in favour of stories. A Public Health consultant noted that GP commissioners tended to be ‘overly swayed by these hugely dramatic stories that clinicians would come in with based on individual patient anecdotes’ (Mary, Public Health consultant).

Documentation

Although substantial knowledge acquisition occurred through informal conversation, more formal verbal exchanges such as meetings were recorded, to leave a paper trail documenting discussions for accountability purposes. Commissioners had substantial access to other documentation, much of which was unsolicited. Documentation was often sent electronically. Documentation included performance, activity, financial and referral data from a range of health-care providers, directives and guidelines from the Department of Health and regional bodies, meeting papers, business cases, reports, patient satisfaction surveys, guidelines and pathways. Often key points were summarised on a side of A4 or an executive summary, possibly because the volume of reading was unmanageable. For example, governing board members had usually at least a dozen documents of several pages each to read before monthly meetings. Presumably to make this task easier, many documents had standardised cover sheet with information such as title, purpose of the paper and action required. In some cases, executive summaries of one page or shorter directly followed these cover sheets.

Despite this ‘overabundance’ of information, which meant that CCGs were ‘absolutely swimming in data’, a CCG chairperson said they had ‘a staggering lack of . . . intelligent data’ (Simon, CCG chairperson). One way of finding missing but desired information was through the internet. GP commissioners and commissioning managers recounted how they used Google™ or Google Scholar to find the relevant information to contribute to discussions with colleagues, inform thinking about service provision and substantiate the decisions already made.

I go onto Google Scholar or Google, and it’s not very difficult to type in key words like ‘CCG’ or ‘PBC’ or ‘PCT budget allocation evidence’, and Google is phenomenal, and Google Scholar will obviously just give you the articles, and you can usually get there on a single page, obviously go a little bit further, read some of the subreferences, etc., extract the key data and some of the graphs, put it onto a PowerPoint, and either present it at a board meeting or in one-to-one discussion with colleagues.

Patrick, GP commissioner

The process mentioned above of reading the subreferences, extracting the key data, entering findings onto a PowerPoint presentation and presenting the data to colleagues clearly illustrates the simultaneous processes of knowledge acquisition and transformation, as sifting, selecting, rejecting, synthesising and incorporating takes place to reshape the original knowledge (see Chapter 6, Ways knowledge is treated).

Useful but overlooked sources

Although where and how commissioners acquired information was important, the obvious sources that were overlooked were also interesting. For example, several participants from one commercial provider pointed out the vast number of good-quality data that commissioners did not use, including Public Health data sets. Another former commissioning manager recounted how she had been unaware of the wealth of information available to her from internal sources when she had worked within the NHS, and so had not known what to ask for.

. . . when they [commercial providers] came in and said, ‘Who is driving your costs?’ at which point I was like, ‘What do you mean? I don’t understand that question’. I’ve got this demographic. I’ve got this population, deprived population here and here. I’ve got this overall growth in population, and this wodge of money at that hospital, and that wodge of money at this hospital. That was my data points. And they came in and said, ‘Well who is driving your costs?’ and came back with a pile of paper this high saying, ‘All these people have been in hospital more than three times in the last 12 months, they’re driving your costs’. I didn’t know I had that data, I didn’t know to ask for it, and now I know.

And where did they get that data from?

From my own IMT [information management technology] department!

Kirsten, commercial consultant

Implicit use of research evidence

For commissioners, the word ‘evidence’ often meant any source of information other than personal experience and anecdotes. When asked about the use of evidence in commissioning decisions, participants often mentioned best practice guidance, hospital and primary care data and Department of Health documentation.

Academic research made a fleeting appearance and was usually embedded within other forms. Most participants did not talk about research evidence unprompted and research evidence was only formally presented at two of observed board meetings (of the same CCG). Instead, participants at another commissioning organisation talked about how research was implicit ‘in the system’ (Alan, NHS commissioning manager), although there was no formal mechanism or any obvious attempt to check if it was being appropriately and systematically adduced. Instead, clinicians and commissioners were ‘expected to keep on top’ of their area (Alan, NHS commissioning manager) sometimes by having their ‘ear to the ground’ with national and local networks (Vidur, GP commissioner). The expectation was that clinical commissioners would bring a research perspective with up-to-date, research-based assessments. In an observation of one CCG meeting, this higher level of understanding of research was visible when a GP commissioner explained ‘funnel plots’ to his colleagues, although we do not know if GP commissioners as a profession were this knowledgeable.

Difficulties with using academic research

When asked about academic research, participants mainly talked about the difficulties in accessing and using research. These included ‘academically very robust’ interventions not working in clinical practice (Jane, NHS commissioning manager), the difficulties in finding applicable, relevant research or tools, the challenge in drawing conclusions from literature reviews when there were substantial variations in the intervention, lack of commissioner time and skills to do comprehensive reviews, difficulties in interpreting the ‘spin’ within abstracts (Mary, Public Health consultant) and the length of time for locally commissioned research to produce meaningful outputs.

One Public Health consultant discussed the difficulties in making research digestible for consumption by commissioners.

So there is an issue around the evidence, and often it’s very sort of generalised, the conclusions can be quite vague, so it’s quite difficult to turn that into kind of an operational plan. Then you need to pull the evidence together, and there’s always the risk that you’ve missed out a really critical recent study which contradicts everything else. Then you’ve got to find a receptive audience, and often it will come across as being quite kind of ivory tower, and is this really of direct relevance to what we’re trying to do?

Rick, Public Health consultant

Explicit use of research (or research-like) evidence

However, research-based information was sometimes evident. Two commissioners (from different CCGs) mentioned conducting literature reviews for specific projects, or contracting commercial providers to review the literature for them. Many participants mentioned NICE guidelines. Although NICE guidance was described as ‘hard edged’ with stronger links to academic research (Karen, Public Health consultant), it was not mandatory to apply, and locally implementation was sometimes problematic if local services did not exist to support the guidelines. Another GP commissioner made the point that NICE guidance was ‘absolutely crucial’ to decision-making but ‘we know we would go bust if we implemented all of them’. So, they picked the ‘best ones’, defined as the most ‘do-able’ (Patrick, GP commissioner).

Local service evaluations were mentioned by participants from three CCGs, when they were asked about research. These included a controlled cohort study of the reablement project, a cohort study mapping the service usage of 50 patients at a local hospital and a controlled cohort study on the impact of case management on hospital usage. The controlled cohort study was led by a local academic GP and was mentioned by several participants, as the findings of this study had a clear impact on commissioning decisions because the CCG decided not to expand the case management service; the evaluation had found that little difference between the case and control groups. However, the nature of the contract was such that the CCG could not recoup the costs of the service, although they did decide not to increase their investment.

Public Health was cited in response to queries about accessing research evidence, for example the Public Health consultant who drafted a briefing on ‘telehealth’ for one subcommittee, but carrying out a robust evidence review was time-consuming. Nonetheless, the subcommittee appeared enthusiastic about this these evidence appraisals.

Teresa comments that this presentation is very, very helpful. People are interested in more reviews of research evidence around unscheduled care, to help with things like education and self-management . . .

Value of this is that if we think of new initiatives we want you to be critical . . .

. . . Can we give things to you (Public Health consultant) to check out?

That is the sort of input we want, and have not had in past.

Meeting notes

Yet despite this enthusiasm, the subcommittee members appeared at a loss in applying the information, especially as much of the research evidence was about ineffective interventions. For example, in observing a meeting where an evidence review on interventions to prevent unplanned hospital admissions was presented, the committee scanned through the interventions, a short discussion ensued on the benefits and disadvantages of one intervention with inconclusive evidence (‘telemedicine’) and the document was then put to one side. We do not know if the document fed into their future thinking, but its use seemed cursory. This group did not appear to consider the disinvestment opportunities highlighted by these reviews. The Public Health consultant who prepared these reviews was aware that the group struggled to apply the information usefully.

Another issue I think, so for instance Sarah Purdy’s work [on unscheduled care], an awful lot of it was like, ‘Well there is no evidence that this works and there’s no evidence that that works.’ And that, it’s really interesting, but then you think, well if we are trying to get people who have these really kind of strict targets, real pressure to reduce costs, and to just come in every month saying ‘We’ve looked at this and it doesn’t work,’ then that’s quite a difficult position to be in.

Rick, Public Health consultant

Evidence-based commissioning

Although in three CCGs research evidence was not particularly drawn on, the ethos of one CCG was explicitly grounded in evidence-based policy-making. This was visible throughout the organisation. In meetings, participants, including lay representatives, often asked ‘where’s the evidence?’ and in interviews participants from this CCG regularly talked about how the application or lack of research evidence affected decision-making.

I’ve had conversations [with colleagues] about, ‘Well, you know, we shouldn’t be putting that down to say it will make savings because there’s no evidence that it will,’ versus me saying, ‘But actually we’ve still got a statutory responsibility to deliver a balanced plan, and if I take those savings out they need to come from somewhere else.’

Carla, NHS commissioning manager

One GP commissioner from the same CCG noted that the repeated refrain for evidence for which they are ‘always duty bound’ can stop decision-making by commissioners.

. . . [they become] frozen because you can’t make pragmatic decisions. Because it’s like America is out there but they will not set sail until they know it’s there . . . So with interventions like ‘telehealth’, ‘they’ll say ‘well we are not doing that.’

Patrick, GP commissioner

Briefings

A formal mechanism for improving the evidence-based practice of GPs was clinical briefings drafted and sent out by a GP commissioner from one study CCG. Initially, these clinical briefings just went to one practice, but then the CCG disseminated the briefings across the patch. The briefings evolved from focusing on clinical evidence to including updates from local services, such as feedback from local hospitals on problematic referrals. Again, in the act of acquiring information for the briefings, the GP commissioner was already transforming knowledge as some information was discarded and other information was selected, reshaped and synthesised to reflect local circumstances.

Within the commercial providers, regular briefings were drafted and disseminated. The information from these briefings was sometimes passed on to the NHS clients via e-mail. One commercial consultant had a self-appointed task of pulling together regular briefings with research from ‘think tanks’, NHS Improving Quality, Public Health observatories and ‘leaders of thinking’. He also included information from Deloitte on health-care innovation (North American), McKinsey, the Mayo Clinic and Dartmouth Atlas to ‘bring in stuff that might be a bit different from what you normally see on the Department of Health websites’ (John, commercial provider). Once John left the company, however, no one appeared to take up this role – another example of the way in which knowledge acquisition was dependent on the happenstance of who was present. Conversely, in the other commercial company, this role was formalised, as a designated consultant sent out weekly bulletins. If this staff member was unavailable, another was appointed to this task. The consensus from commercial provider staff at both companies was that these briefings had helped them become more evidence based.

Commercial providers and academic research

Many software tools marketed by commercial providers were predicated on academic research. For example, one tool arose from decades of academic research initiated by clinical teams. Development of the tool was ongoing and one external provider encouraged researchers around the world to carry out independent studies using the tool. Results were published on its website ‘to impart knowledge of how it’s been applied in different areas’ and establish ‘rigour’ (Stan, commercial analyst). Consultants from the company regularly attended academic conferences where they gave presentations on the tool and made ‘opportune’ contacts (Katie, commercial consultant). Engaging academics was important to the success of rolling out the tool.

And how important do you think universities are in terms of expansion?

Extremely. Absolutely, that’s the key, and that’s what differentiates us from [competitors] that we are research oriented, and that’s what the [tool] is based on.

Katie, commercial consultant

In summary, commissioners defined evidence in many ways, but not usually in terms of academic research. Academic research was occasionally explicitly sought, but usually it had been digested, transformed and embedded into forms such as NICE guidance, software tools, the clinical knowledge of GP commissioners and local briefings, presentations and conversations. Importantly, commissioners did not usually check or confirm the standing of rigorous evidence in such sources, but took it on trust. When commissioners had more direct access to research evidence, they had reservations about and difficulties in applying it to local circumstances. Ironically, a negative consequence of evidence-based commissioning was that without robust evidence commissioners became stalled in their decision-making. The general interest in using research evidence was encouraging, but difficulties in finding applicable, user-friendly research and its inconsistency in providing useful, accessible conclusions meant that the impact of academic research evidence on commissioning decisions was limited.

Box 5 summarises the key points of the chapter.

Interpersonal relationships

Interpersonal relationships came into play when commissioners sought information held by others with whom they had ongoing relationships. For example, one commissioning organisation contacted consultants from a local not-for-profit organisation to ask for possible external organisations with specialised skills to inform a contracting initiative. Because professionals from the two organisations had worked together before, the commissioners knew that the not-for-profit agency was well placed to advise them. Of all the conduits, interpersonal relationships appeared the most influential and were implicitly nested into people placement and product deployment.

Interpersonal relationships were visible at different times and played out in key relationships. Some key relationships were positive and some less so, which led to varying levels of impact. For example, one commercial provider allocated consultants with experience in commissioning to help staff from commissioning organisations to interpret data from software tools. A few commissioning consultants were ‘very, very good individuals within [commercial provider] who added a huge amount of value in terms of helping people with their commissioning problems’ (Alfred, NHS client). Others appeared less useful to commissioners and made less progress.

In one commissioning organisation, interpersonal relationships were closely linked to colocation and the face-to-face, informal contacts that colocation permits. In interview, a GP commissioner noted the number of desks he sat away from the Director of Finance (two) and the Director of Public Health (eight) in their open-plan office, and credited this proximity with the high level of collaboration. A freelance analyst working with this CCG was also aware of the importance of informal, face-to-face encounters with local GPs.

If you’ve got trust and if you’ve got experts in both domains really closely engaged, literally, and by that I mean literally sat at the same computer fiddling around with stuff on the screen, bouncing ideas off each other, that for me was where all these light bulb moments came.

Randall, freelance analyst

One Public Health consultant talked about how through building relationships, Public Health had transformed commissioners’ understanding of the commissioning cycle, highlighting the importance of needs assessment. This was particularly important given how thinly stretched the Public Health team was across five CCGs.

I suppose what we’ve done a lot over the last two years has been around really relationship building. Because one of the problems with having five CCGs and a sort of relatively lean Public Health team is actually, you know, physically being able to be a presence which, I mean I know that just being somewhere is not the same as imparting but if you’re never there and they can’t see you, that is a problem . . . I think we’ve been very successful there with the CCGs actually supporting them to understand the commissioning cycle, and that it starts with a needs assessment.

Sandra, Public Health consultant

With commercial providers, interpersonal relationships sometimes appeared to depend on a perception of shared ‘public sector values’ such as partnership, knowledge freely exchanged and the primacy of patient benefit (Alfred, NHS client). We observed several instances where commercial consultants attempted to capitalise on their NHS credentials and knowledge to build relationships.

. . . to get credibility if they’ve not seen your CV [curriculum vitae] beforehand, is that I’ve worked in the NHS, so I’m clinically intelligent in that sense, and I’ve been in their shoes most of the time, so I’ve often been in the wards with people that I’m working alongside and I try to understand what their issues are. But I can also work with them as a person, so who they are and what issues they are facing. And I mean I know that’s very effective, because I get an awful lot of disclosure very early on in terms of what the real issues are for people.

Jessie, commercial consultant

In summary, interpersonal relationships manifested themselves in different ways, but often appeared dependent on informal, face-to-face encounters and demonstrating shared values through claims of belonging to the ‘NHS family’.

People placement

Another conduit, closely linked to the above, was people placement, whereby the necessary skills, expertise, background and experiences were embodied in the professionals themselves. Ideally, these individuals were embedded within a client organisation. An example of people placement was when a commercial company located two nurses at acute hospitals to check invoices against patient notes. Commonly, the intention behind people placement was to inject knowledge into the health-care system and to foster interpersonal relationships.

Several types of external providers employed the conduit of people placement. For example, a CSU provided regular senior manager input at senior management team meetings and an analyst embedded within the CCG.

Well, we have a very good contact at the CSU, Mike. He comes to our senior management team meetings now once a fortnight. And what we’ve managed to do with his help and support really is get his team to see themselves as welcome within our building . . . And so we need to build up some of the personal relationships which always make these things easier. And we need some of the ease of communication which comes from somebody being on the next desk rather than at the end of the telephone or e-mail.

Tom, CCG chairperson

Although it was relatively rare, some commercial companies managed to embed consultants into NHS organisations, but only if commissioning organisations were receptive.

It was very much a place of work as well, so we stayed up there during the week, so it wasn’t a transient thing either we absolutely became part of their organisation. And we were invited to everything that was to do with that organisation, weren’t we? Whatever happened we were – one of us was involved in something that was going on in there. And so because we were there, and we were very visible, and we were in the same building as the four commissioning group leads, directors, we were very available, weren’t we? (Yeah) And we had lots of drop in questions, so they regularly came to us with questions that weren’t necessarily related to the piece of work that we were doing with them at the time. So they would come for advice and help, wouldn’t they?

And I think because we had built that relationship, we were then seen as an organisation coming in that they didn’t feel threatened by.

Patricia and Helen, commercial consultants

Former public sector organisations such as CSUs employed the people placement conduit with perfunctory matching of skill sets to client needs. With commercial providers, the process of people placement appeared more sophisticated, as the matching of consultant and NHS client was fundamental to success. Good matching considered discipline (e.g. analyst to analyst), previous experience in the care sector (e.g. mental health) and skills and expertise (e.g. knowledge of contracting). One commercial provider, either accidentally or intentionally, also allocated consultants whose personality type filled gaps on the management team (e.g. ‘completer/finishers’ vs. ideas initiators). Some clients wanted international experts rather than ‘recycled NHS managers’ (Kristen, senior commercial consultant), but other clients preferred working with those with NHS background. Softer skills were also necessary.

The front man, Tim, was very good, he knew his stuff, he could hold his own with the GPs, he worked well with the colleagues, didn’t wind anybody up and we had a very productive year out of that working relationship.

Kurt, NHS analyst

Commercial consultants were allocated to projects on the basis of client needs, which were often unclear and highly changeable. Commercial consultants could suddenly be reallocated, leaving NHS clients somewhat bereft.

There’s a piece of work that the PCT want to do which is more about looking at different kinds of contracting options for the future. And we’ve got an expert in that, and it seems absolutely daft me spending ages reading up and scratching my head and only being able to give them part of the solution, when actually John can do it all. And he’s really wanted to get his teeth into something like that. So I’ve explained to them [the clients], ‘This is our expert. I will be moving back. John is going to be doing this piece of work’. But, they’re not happy – I think they’re not happy about me stepping back, but they know that they’ll get what they need and at the end of the day, that’s what they’re paying for.

Jessie, commercial consultant

Although the conduit of people placement relied on the knowledge embodied within individuals, those individuals often chose to relocate. For example, about 20% of study participants at one commercial provider left before fieldwork stopped 6 months later. Many went on to work for rival commercial companies, taking their expertise with them. We also encountered NHS participants who straddled the NHS/commercial provider divide by working concurrently for both. During the data collection phase, no one voiced concern about this intermingling and exchange between the NHS and commercial companies. Subsequently, however, a programme broadcast on BBC Radio 4 questioned the impact of secondments of NHS staff into large commercial providers and vice versa. 44

Governance

Another conduit through which knowledge was obtained and transformed was governance, which we use here in the very broad sense of the structures and processes which shape ‘the way we do business’. Governance is a similar conduit to the others in that knowledge is exchanged and transformed between two or more parties, sometimes from national/regional bodies such as the Care Quality Commission or NHS England and sometimes between different local committees. Knowledge to inform governance came from sources such as national ‘must dos’ and guidance, local financial data and patient-level data. Unlike the other conduits whereby the commissioner actively sought information, with this conduit the commissioner was often the (unwilling) recipient. The governance conduit was visible in processes and structures adopted by CCGs and the way decisions were made. For example, with clinical commissioning medical input was now much more likely to inform decision-making.

In the old days decisions made by the PCT would be made according to what the Department of Health required, the Strategic Health Authority required, the managers thought was appropriate and then we’d also consult with the clinicians . . . and that’s how it would go through. Now it starts with a clinician, the whole innovation, the whole idea. The clinicians drive it through and they bring it to the governing body and say, ‘This is what we want to happen – is it feasible to make it happen?’ The governing body then, as the name suggests, looks at the governance of it, and that’s the right way. That’s a big shift in where the responsibility, where the accountability, where the innovation and the service provision comes from.

Malcolm, CCG chairperson

Because probity and accountability were so important to CCGs, proposals went through several decision-making cycles before ultimate approval. This often frustrated commercial providers and freelance consultants, who complained of slow NHS decision-making, such as Brian:

Think about the decision route. From the project board to the Operations Group – the papers are already out for the March Operations Group meeting.

Why does the Operations Group need to make the decision?

It needs to be the whole community, can get it into the March agenda. [Len is Operations Group chairperson.] [Kevin asks about terms of reference and whether they have delegated decision-making powers.]

Can’t possibly expect every decision to go to them? This isn’t a major decision.

This is our top priority.

The more you push it up the governance ladder the harder it is to make a decision.

The expectation was that we were coming back with a recommendation.

So we will inform them and explain why. Not the same as asking ‘do you agree’?

We are asking for an endorsing decision from them.

Meeting notes

Interestingly, Brian, the freelance consultant above, was a former PCT chief executive, so he was presumably well accustomed to the exigencies of public sector accountability. But a recent study suggests that these structures have become even more prolific with the advent of the Health and Social Care Act 20124 and CCGs. 45 Because proposals were repeatedly discussed in decision-making cycles across several decision-making groups, it was often hard to determine what and when decisions were taken, who was involved or what information had contributed. Nonetheless, as public money was involved, proposals were necessarily subjected to substantial scrutiny.

Besides decision-making processes and structures, the governance conduit was evident in other aspects, including setting strategy, finance, risk and information management. The conduit of governance was most visible in our observations of CCG board and subcommittee meetings. For example, we observed a freelance consultant presenting her report on the Francis Inquiry to a CCG governing body, which considered the implications of the report operationally and strategically. This report necessitated a response from every CCG as a national mandate. The consultant, who was an ex-NHS colleague, took the 290 recommendations of the Francis report, synthesised them into a few categories and benchmarked local performance against these categories. CCG board members found this report ‘really helpful’, especially as the consultant’s assessment was that the commissioning organisation was ‘not starting from zero’ (Hannah, commissioning manager). This then influenced their subsequent decisions on developing a strategy for quality assurance.

The governance conduit was visible in policies around data sharing, which often had a major impact on limiting or preventing knowledge transformation. The desire to use patient-level data to link across data sets was increasing, as commissioners from several of the study CCGs developed patient pathways to inform their commissioning decisions. But this coincided with greater stringency around data protection.

For example, one commissioning organisation wanted to link up their dashboard with hospital data to the risk prediction tool at patient level to learn more about where in their pathways patients stopped using NHS services. Information governance stipulations were that linkage of patient-level data could occur only if there was direct patient benefit to the patients involved. But in the case of the dashboard, direct patient benefit could not be demonstrated, so this linkage was not permitted. Distinguishing between data linkage for ‘direct patient benefit’ versus ‘commissioning’ was not straightforward.

Think that would be great and if the hospital could tell us what happened to them. Can we share that data between hospital and community (i.e. trace a patient journey through)?

[Yvonne says no but Sally says yes.]

Because patient’s been discharged we couldn’t argue that it’s for the benefit of that patient. But you could anonymise. But you couldn’t then link without an identifier to community provider data.

Couldn’t someone call the patient and ask if they could use their data?

If we want to use data for performance we can take to (something).

As long as anonymised that’s fine.

Meeting notes

Moreover, although a participant said that GPs were less enthusiastic about using patient data for ‘planning’ purposes rather than direct patient outcomes (Sanjay, GP commissioner), when GP commissioners realised the restrictions around use of patient-level data for commissioning, there was considerable outrage.

You can’t beat actually following case notes.

That was shot down due to information governance.

Rubbish!! [He is really indignant].

If it is a clinician to clinician discussion you’ll be fine.

In that case I’ve breached information governance lots of times.

There is a distinction if you are looking at the data for a cost saving versus a care issue. General group response is that ‘it is all patient care’.

Meeting notes

In another commissioning organisation, data sharing with commercial providers was an important concern. To populate software tools, commercial providers needed access to patient data sets from different health-care sectors. For the risk prediction tool, all the data for the local general practices and hospitals were held in a data warehouse by a third-party commercial company. This same company also held national NHS data, and so NHS data governance requirements were all met. Nonetheless, several GPs voiced concerns about releasing their data without patient permission, as they were alarmed about data handling.

I know a lot about it, because I’m extremely sceptical because I did not like the idea of [commercial provider] having all of our data. People completely do not understand the data flows and the IT implications of what they were suggesting. So it takes primary care data, merges it with secondary care data, then what people didn’t really realise is that there is a third data set which had no information governance surrounding it whatsoever, apart from [commercial provider] could use it for whatever they wanted and sell it to the highest bidder.

Really?

Well I’m exaggerating slightly.

Patrick, GP commissioner

One practice manager said that he and his colleagues had similar concerns about the association of the risk prediction tool with a commercial company, as practice managers were beginning to see themselves as ‘guardians of the patients’ data’ (Adrian, practice manager). Once the contract with the commercial provider ended, however, the CCG was ‘desperate’ to share patient-level data and roll out the tool, but it was tied up in information governance (Patrick, GP commissioner).

Not all CCGs were concerned about data sharing with commercial providers. An analyst at the same CCG as above noted that data sharing was hampered because influential GPs were ‘at the cautious end of the spectrum in terms of data sharing’ (Charles, NHS analyst). He commented that GPs in a neighbouring county had 100% uptake of the risk stratification tool, because there was not the same level of concern and the other county also had ‘people at very senior level who were driving the value of the risk prediction tool’ both clinicians and managers. Another reason Charles thought practices were reluctant to sign up was that the sign-up documentation was lengthy and technical.

In summary, the governance conduit was mainly visible through decision-making and information governance. With decision-making, because proposals were considered by many different committees, the knowledge was constantly reshaped. Information governance could halt knowledge transformation by forbidding linkage of patient data sets and because of alarm around commercial providers having access to patient data.

Copy, adapt and paste

Another conduit was ‘copy, adapt and paste’, whereby innovations, ideas, tools, pathways, guidance, service redesign and so forth were taken from one context and adapted to a different programme, project, organisation, health-care economy or sector. One commercial consultant, who was a former public sector employee, commented that there was an ‘almost pathological desire’ to ‘copy and paste’ in the NHS (Felicity, commercial consultant). One GP commissioner noted that commissioners were good at ‘foraging around to find something roughly right’, but then had difficulties in testing the robustness of the borrowed concept (Ciara, GP commissioner). To make the imported knowledge work, commissioners and clinicians had to contextualise and adapt the concept locally.

We found many examples of copy, adapt and paste across the case site CCGs. For example, one CCG adopted ‘copy, adapt and paste’ with a commercial pathway tool. Based on NICE guidelines and tailored to reflect local pathways, this electronic pathway tool was intended to sit on GPs’ desktops to provide evidence-based pathways accessible during consultations. This had been adapted in a neighbouring county, to great acclaim. A CCG borrowed and paid for their neighbour’s modified pathway tool and adapted it further to meet their requirements by adding information about local services.

‘Copy, adapt and paste’ was encouraged by copious documentation on best practice, disseminated from the Department of Health, The King’s Fund, NHS Improving Quality and other organisations. One commissioning organisation was constantly ‘horizon scanning’ to look out for pioneering initiatives to appropriate ‘because there are loads out there’ (Carol, NHS commissioning manager). Sometimes, they contracted external commercial providers to ‘bring in research from other areas that says over here they are doing this and how that might be adapted or adopted here’ (Hilda, lay representative). But often they conducted those searches themselves.

Research in this modern world is very quick, you know, it’s really very, very easy to find out who’s doing what. So if you look at A&E and put in GPs in there, you know, I know there’s been national research done, I know there’s The King’s Fund stuff, but in 10 minutes I could go online and find out which other hospitals are publishing information about trying it. So it’s relatively easy to pick a subject matter and say what’s going on out there.

Alan, NHS commissioning manager

In summary, ‘copy, adapt and paste’ was so ubiquitous in the case sites that it was sometimes hard to trace back to the original source of the innovation. Few ideas or ‘innovations’ seemed genuinely original. Moreover, each time an innovation was appropriated into a new context, its content was further modified.

Product deployment

Another related conduit of knowledge was product deployment. Product deployment consisted of the production and dissemination of information via tools, often combined with training in their use that was produced by the external contractors. In one commissioning organisation, the tool was a system-wide business method. Developed from managerial research into organisational behaviour from the 1980s, the aim was to move organisations from a ‘command and control’ ethos to focusing on customer needs. The commercial provider contributed the method and the client brought local knowledge. Although the idea of seeing the organisation from the service user perspective is not new, the combination of the knowledge of the consultants (who were often ex-clients of the method) with an overarching method that prioritised ‘principles’ over ‘rules’ (e.g. downgrading box ticking to meet governance requirements) seemed to change the way that service redesign was viewed.

And I think what I’ve learned, which actually, in some ways, when I say it out loud seems so simplistic, is that actually, if we do the right thing for our patients, then it actually is cheaper, quicker, more efficient, more effective.

Jane, NHS commissioning manager

But a non-electronic tool was unusual. In this study, product deployment usually involved software tools developed by commercial providers for invoice validation, scenario modelling and risk prediction. Risk prediction tools were most common. These tools aggregated hospital and primary care data to work out the current and predicted costs of patients based on their conditions. This information was usually used to identify patients at high risk of hospital admissions for interventions such as case management by community matrons, although one CCG was piloting use of the tool for resource allocation to general practices.

Training NHS staff in operating the tools was part of product deployment. One tool provider set up virtual online webinars, whereby NHS clients were taught advanced operational skills by international experts in the risk prediction tool (see Chapter 8, External provider 1). Another risk prediction provider organised face-to-face multidisciplinary sessions, either in GP practice premises or at the headquarters of the commercial company. Participants at these training sessions tended to be practice managers, community nurses, pharmacy leads, commissioning managers and GP clinical leads. The format of the training sessions was interactive PowerPoint presentations followed by a demonstration of the tool populated with local data.

A critical moment in the sessions was the identification of local high-risk service users, as this was a test of the tool’s accuracy and usability. If ‘the usual suspects’ (i.e. known patients) (Ginny, external consultant) were identified and no new information was provided, NHS clients might conclude that the tool offered no added value. But if too many patients unknown to the clinical team were identified, the conclusion could be that the tool was inaccurate. In comparing data output from the tool with their clinical and practice-based knowledge, clinicians formed judgments about the usefulness of the tool.

Two commercial providers used a strategy of training NHS/CSU trainers who in turn trained other NHS/CSU staff. The NHS/CSU trainers came from different backgrounds, including primary care commissioning and information. They taught general practice staff and community matrons and others how to use the tools through group presentations and ‘sitting by Nelly’ methods. However, several NHS trainers noted that those trained to use the tools quickly forgot those skills, as neither the CCG nor other general practice staff requested any output. The skills were not drawn on frequently enough to embed the knowledge (see Chapter 8, External provider 1).

NHS/CSU analysts were key to the implementation of these software tools, as analysts tended to have day-to-day operational responsibility. But the capacity of analysts available within the NHS was low, especially during fieldwork when PCTs were losing staff in the turmoil following the 2012 Health and Social Care Act. 4 Instead, commercial providers trained anyone who showed up from the commissioning organisation to training sessions, which might include those who were uncomfortable with anything more complicated than spreadsheets.

But in one commissioning agency, the NHS analysts were an equal match for the sophisticated expertise of commercial providers. In working closely with the commercial providers to run and test a scenario-generating tool, the NHS analysts identified several ways that the tool could be improved and made these adaptations themselves within the software. These suggestions were then incorporated into the next version of the tool by the commercial provider and rolled out to other clients. Some commercial provider consultants, mainly those with lead responsibility for a particular tool, actively sought these opportunities to enhance tool performance. This was an example of genuine knowledge exchange whereby both commercial provider and NHS client benefited.

Contextualisation

Contextualisation involved taking information from elsewhere and applying a local lens or filter. Commissioners had an essential role in contextualising the knowledge to local circumstances, but sometimes external providers had to undertake contextualisation as well so that the knowledge was fit for commissioners. When appropriating information from elsewhere, one commissioning manager commented that someone ‘always says our system is not like that’ (Clara, NHS commissioning manager), and so the knowledge needed to be contextualised to overcome this hurdle. One commissioning manager said that contextualisation was the ‘crux’ of commissioning.

I think that’s the crux of our job. It’s really interesting, because you read what you read, and you find out what you can, but then it has to be applied locally. And all localities are different, you know. If you look at [our CCG], for instance, we’ve only X population in our entire area. Geographically it’s relatively small and urban compared to say our neighbours in [county name]. So that alone says something about how you start to think about things. And a lot of activity, research, JSNAs, a lot of Public Health driven sort of surveys about where your deprivation is – all that information is available. And so you have to then balance best practice against what’s reality locally, what you’re doing already good and bad locally.

Alan, commissioning manager

The inherent transferability between the original context where the information was generated and the local situation was important and participants varied in views on how close that matching had to be.

If evidence or trials show that it works elsewhere then we have to believe that it can work here too.

Agree but we can’t assume that it will all work here because the data elsewhere may say 200 but it won’t be 200 here. Need to look at all of the pieces not just one part.

Meeting notes

Contextualisation repeatedly appeared in product deployment. The type of knowledge and information built into software tools was often based on academic research evidence or expert consensus that was not UK based. Often, tools needed ‘Anglicisation’. This contextualisation was crucial, as without high NHS applicability clients tended to dismiss the tool. Some contextualisation was undertaken by external provider staff, but, to make some tools useful, NHS analysts provided further contextualisation for the local health-care economy.

Moreover, NHS clients had to know what to do with the data and needed help applying data outputs to commissioning decisions. Ideally, commissioners had access to an ‘interpreter’ who was known and trusted to assist with this type of contextualisation. For example, in one CCG, a CSU analyst attended meetings where she had a regular slot to present a dashboard and work through the implications of the data with committee members. In another CCG, a GP commissioner talked about his concern that, without a trusted interpreter, commercial providers might take advantage of GP commissioners.

But, you know, what do you do with that data? We know that it must be saying something to us, the fact that a little old lady has had three falls, the ambulances have been round a few times, that probably is telling us something about her, that she’s not very stable. But it doesn’t tell us that she’s somebody that necessarily needs to be assessed by the team. And when our team resources are really very, very stretched, we’re in a bit of a dilemma. We’ve now got a waiting list for people to be assessed, and half of them, I suspect, will be assessed and it will be decided that they hadn’t really got a big problem at all. So somebody coming in and saying, ‘Oh yeah, we recognise all of that, we can do something with that,’ sounds superficially very appealing. Now some of it may be complete bullshit and they may just be angling to get into the markets and that’s what worries me.

Roger, GP commissioner

Contextualisation was integral to the ‘copy, adapt and paste’ conduit and manifested in interpersonal relationships and people placement, where through relationships and by combining knowledge and expertise, knowledge could be transformed into a more useable, applicable form. Training on the tools alone did not appear to be sufficient in helping commissioners to maximise their use of the tools. The outputs had to be interpreted and contextualised by those who understood the tools, so that local commissioners could digest and apply the information. Otherwise, NHS clients did not know what to do with the data. Contextualisation was less evident in governance processes, although translating national and regional mandates required some contextualisation.

Engagement

Another knowledge transformation process was engagement. Engagement was about taking transformed knowledge and exposing and refining it further by involving the ‘right people’. The right people might have important information or perspectives, be positioned to instigate behavioural change (or show reason why that could or should not happen) and/or could tap into local or national networks to make the initiative a success. Again, commissioners were crucially placed to undertake local engagement, but external providers also actively used engagement strategies.

Local commissioner-led engagement was evident with service redesign initiatives, whereby commissioners drew on the experience of those ‘round the table’ (Abbie, NHS commissioning manager). Moreover, engagement of clinicians into commissioning initiatives was discussed across all case sites. For example, in one CCG meeting the group talked about progress on engaging ophthalmologists in developing a community service. Engaging GP practices was also a common topic.

And one other positive thing I think is that in previous times it’s been incredibly difficult to actually engage general practice in some of the quality improvement work because, understandably, you know, independent contractors, getting GPs to come to meetings and do things if you can’t fund them and things is quite difficult, and now obviously that is a role [for GP commissioners].

Sandra, Public Health consultant

With appropriate governance, engagement was visible to ensure that the right people were involved in decision-making. This signalled that the relevant organisations had been consulted and took responsibility. Being perceived as engaging appropriately was a high priority with one CCG, where very large meetings with many different organisational representatives were held, although input from some was minimal (i.e. Public Health, Social Services). In another, the process of engagement led to a different (and smaller) mix of elected councillors and a lay representative as chairperson.

In one contract, engagement by the external provider was clearly implicated in the vicissitudes of the contract’s success. The commercial provider first deployed a team of analysts with little commissioning expertise or NHS knowledge and minimal focus on relationship building and engaging local commissioners. The NHS client complained. The commercial provider then allocated a cohort of new management consultants, many with a background in the NHS, with a remit of ‘commissioner engagement’ to help to interpret and use the data. Some successfully developed interpersonal relationships. The NHS clients were happier. However about a year later, changes within the commercial provider meant that the emphasis shifted and engagement moved more to the background. Moreover, the commercial provider blocked NHS clients from having any direct contact with subcontractors of the tools because of concerns that the NHS clients would drop the main contractor in favour of the subcontractors. This stifled knowledge exchange. The contract was renegotiated; the termination date was brought forward by several months and the NHS clients directly contracted further training from a subcontracted company. Changing levels of commissioner engagement had an impact on the success of this contract.

In summary, engagement helped to spread, and further transformed, knowledge. Engagement was particularly necessary with product deployment, as ample data from this study suggested that without engaging the right people such as influential GPs and practice managers, as well as commissioners, the roll-out and use of software tools was frustrated. Interpersonal relationships enhanced product deployment and appeared to facilitate greater understanding (and possibly use) of the tool outputs generated. Engagement was highly visible within governance.

Model of knowledge conduits and transformation processes

Figure 5 represents the findings of this chapter visually, setting out the relationships between the different knowledge conduits and commissioners’ knowledge transformation processes. In Chapter 8, details from vignettes will be added to the template below to provide examples.

FIGURE 5.

Template of conduits and transformation process. CAP, copy, adapt and paste; G, governance; IR, interpersonal ralationships; PD, product deployment; PP, people placement.

Classification

During the two-and-a-half years of fieldwork from 2011 to mid-2013, our four case study CCG commissioners worked with many different external providers. We have listed these below, categorised as public, commercial, not-for-profit and voluntary sector, along with the work these organisations (or individuals) were engaged in, where known. Some functions were carried out by multiple organisations; for example, evidence reviews were conducted by commercial companies and public health departments. This list is not exhaustive, as occasionally participants mentioned that external providers had been contracted, but the provider was not named and no further details were available.

The breadth of external providers and their level of involvement suggest that there was substantial external provider presence in NHS commissioning during the fieldwork period, some of which pre-dated the 2012 Act. 4 Table 11 gives an overview of the range and type of external providers working across the four CCGs.

Commissioning support units and study Clinical Commissioning Groups

The four case CCGs contracted three different CSUs. The CCGs had quite distinct relationships from their CSU.

One CCG had a large internal team to avoid dependence on the CSU, which they felt was too removed. A participant intimated that the ‘best’ former PCT staff had been cherry picked by the CCG and those without posts then went to the CSU. This CCG bought financial services and ‘transactional’ activities such as payment of invoices, but the CCG still carried out internal finance reporting such as budget reports and contract monitoring. Another participant was clear that he would prefer CSUs not undertake service design, as local knowledge was needed. The general sense from this CCG was that the CSU was an external organisation.

Commissioning Support Unit staff embedded within this CCG included a pharmacy advisor, human resources support and a small team for extra contractual activity. A participant said that embedding CSU staff within the CCG was key, as GPs liked to work with those they knew. Without embedding CSU staff, the likelihood was that the CCG would create internal support services duplicating those offered by the CSU.

GPs tend to work on a one-to-one relationship, and better the devil they know sort of example. So to work with an anonymous organisation, they will find a challenge. And I can see Clinical Commissioning Groups up and down the country wanting to recreate lots of commissioning support services within their own environment, because they’ll be wanting to build relationships with people they know.

Sarah, NHS commissioning manager

In another CCG, the CSU and CCG worked more closely together. This particular CCG was keen on using data and tools to inform commissioning. The CSU carried out routine data analyses, but they also built bespoke tools and supported software tools from commercial and not-for-profit providers. The CSU ‘offer’ included media support, responses to Freedom of Information requests, public and practice engagement, human resources and organisational development, recruitment, workforce planning support, risk management, information governance, IT support to the CCG and general practices, horizon scanning and financial activities such as managing staff payments. The CSU also offered a scheme to fill ad hoc capacity gaps in the CCG.

The CCG had a close relationship with a senior CSU manager and a CSU analyst; the latter ‘had been around for donkey’s years’ and took the role of ‘principal analyst’ (Connie, CSU analyst). She was highly regarded and well embedded within the commissioning organisation with her own desk. Because of this long-standing relationship, this analyst believed that she was not seen as a threat when requesting commissioning plans or making suggestions. Her role was to liaise with the CCG and the team back at the CSU offices that did ‘a lot of the actual [number] crunching’ (Connie, CSU analyst). CCG staff could also use her as an access point to find relevant expertise within the CSU. Another important role was interpretation, as this analyst helped CCG commissioners to understand outputs from the CSU.

Some of the reports can be quite monstrous. And, you know, they were bravely wading through it all, which was frustrating for them, and it wasn’t fair on them, because there should have been someone there to assist them.

Connie, CSU analyst

In observations, the reliance of the CCG on this principal analyst was notable. During one meeting of a subcommittee, she was not able to attend and the members had no access to the dashboard tracking unscheduled admissions. This caused significant consternation and some negativity.

[Morris asked if it was possible to have an update on the gaps in the dashboard.]

We have no dashboard.

The difficulty is that the information service hasn’t been able to complete the work we wanted them to do because they were doing the QIPP [Quality, Improvement, Productivity and Prevention] information plan and HRGs, so staff were pulled off our stuff. We are not happy about this and are trying to get it resolved.

We felt we were getting real traction in November.

Difficult times at the moment, changes in the system are having an impact but we are where we are . . .

Very frustrating.

Don’t give up hope. Connie will be back on it.

Presumably we are paying. If she is not available then they have to get someone else.

Meeting notes

Commissioning support units and commercial providers

Having looked at the relationship between CSUs and the study CCGs, we now discuss general findings about the interactions between CSUs and commercial providers.

Commissioning Support Units had complicated relationships with commercial providers, working sometimes in collaboration and sometimes in competition. For example, one CSU manager commented that in her experience CSUs and commercial providers best worked together in partnership whereby CSU staff offered local knowledge to sense check and interpret the data.

[Commercial provider] have come in and developed the data warehouse to take the SUS [Secondary Uses Service] data. But the data warehouse has only become useful because our data managers have got involved in looking at that and interpreting the data for them to say, ‘Right well you’ve got um – um you’ve matched this particular data item wrong,’ or, ‘You’ve got to filter for this,’ or, ‘You need to be aware of that,’ and then quality checking the information, so that they’ve actually set up the data warehouse to process the information correctly.

Joan, CSU manager

Another CSU manager planned to offer a mediation service, whereby CCGs could go to CSUs to identify commercial and not-for-profit providers for other work. But there were signs in some CCG–CSU relationships that CCGs might turn to commercial providers out of dissatisfaction with CSU performance.

I think that’s bizarre. At the end of the day, if we’re having this sort of contract arrangement, then if we’re saying to an organisation, ‘We’re paying you X number of pounds to provide us with the data,’ they can’t come back and say, ‘Well we can’t do it because we’re too busy doing something else,’ having agreed to undertake that body of work. They’re essentially in breach of contract . . . I think what’s probably going to happen is I think they’re going to shoot themselves in the foot potentially. Because I think people will go outside of the CSU when they’re allowed to.

Anthony, GP commissioner

Competition between CSUs and commercial providers became increasingly common over the study as they bid for the same business. One commissioning manager recounted an experience when a CSU did not bid for a contract because among the competition was a commercial company with whom the CSU had previously worked. His interpretation was that the CSU knew that they would struggle to win the contract, because the analytical service of the commercial provider was superior.

In April 2014, we conducted our final interview. This was 1 year after CSUs had been launched and 18–24 months after our initial data collection. Although the viewpoint of this particular commissioning manager might be uncommon, he mentioned that in tendering exercises the quality of bids from the CSUs had been disappointing.

And I suppose there is a sort of basic commercial lesson to be learnt out of this in that if the CSUs are moving as they are towards a place where they need to go out and win business, they need to raise their game substantially both in terms of their customer orientation, just the whole sort of attitude, but also in terms of the amount of careful, competent work that they need to put into preparing bidding documentation, because some of it was just woefully inadequate really.

Jacob, NHS commissioning manager

In summary, the relationships between CCG and CSUs varied from distant to quite close, with some CSUs offering a broad spectrum of services. In general, CCGs trusted that the CSUs had the interests of CCGs at heart, but in practice there were concerns that the CSUs struggled to manage the demands of multiple commissioning clients and produce useful output. Embedding CSU staff within CSUs was common. The relationship between CSUs and commercial providers could be collaborative or competitive and there were some concerns that CSUs were not competitive enough and/or not adept at bidding for contracts.

Reasons for working for commercial and not-for-profit providers

The rest of the chapter covers commercial and not-for-profit providers. First, we will present reasons that NHS professionals moved to commercial and not-for-profit providers and then discuss cultural differences between the NHS and commercial sectors, reasons for contracting commercial external providers and attitudes towards the use of commercial and not-for-profit providers.

In interviews, we found that many consultants from for-profit and not-for-profit providers had ex-public sector or NHS backgrounds. Ex-public sector professionals said that they joined for-profit and not-for-profit companies for several reasons. Two former information managers believed in the product of a not-for-profit and wanted to promote it more widely. Another ex-information manager moved to commercial consultancy because he became ‘fed up with the NHS’. He thought he would have more influence outside than in (Rhett, commercial analyst). After over 15 years in the NHS, Jessie also felt that she could make more of a difference from the outside. Inside the NHS, she often felt ‘constrained’, whereas now ‘there’s more opportunity and I’ve found that I’ve been able to be more effective in driving change and changing people’s behaviours and attitudes than I was when I was an [NHS] employee’ (Jessie, commercial consultant).

For analysts, the differences between employment in the NHS and commercial sector were particularly stark. The main reasons for departure from the NHS were career progression and job satisfaction. An analyst said that in the PCT he felt like ‘a small piece in a larger clockwork’, whereas with the commercial provider he had ‘free rein’ to make decisions which made him feel ‘more important’ (Ranulph, commercial analyst). After 12 years in the NHS, Doug left because of lack of progress in IT project management. He was interested in independent contracting or interim management but needed more private sector experience. Only one participant raised the issue of pay, claiming that salaries in the NHS and commercial sector were commensurate, although this is difficult to believe given the tendency for NHS analysts to be graded at bands 5 or 6 (currently £21,478–34,530 annually).

Joel appreciated that commercial analysts were allocated work on the basis of their skills and the ‘whole emphasis is a lot less about hierarchy’ (Joel, ex-NHS commercial analyst). In the NHS, the quantity and quality of analysts was substandard, as ‘one of the frustrations with the NHS is that they don’t reward excellence and knowledge’ (Joel, commercial analyst). Because analytical staff had limited prospects, good analysts tended to leave or went into management, which meant that they were not benefiting the ‘coal face’. This is a familiar management problem.

A NHS commissioning manager who had worked with both NHS and commercial analysts thought that the NHS analysts were ‘a fairly dejected bunch who are not very loved or cared for’. This was reflected in the quality of their work.

I think the NHS, they don’t have the same motivation to trawl through the detail that [commercial provider] have. My experience of information staff in the NHS is they’re kind of a fairly dejected bunch who are not very loved or cared for. And I think that sort of is reflected in the quality of the work they produce. They sort of churn out the monthly monitoring reports or a standard monthly interpretation of ‘We’re over-performing in cardiac, we’re under-performing in urology,’ and that’s about it. And the [commercial provider] analysts seem much more interested in understanding what the data is telling them, so moving on from just analysing data to the sort of information intelligence insight end of the spectrum.

Paula, NHS commissioning manager

Several former NHS analysts now working for commercial companies agreed that NHS work was dull and the focus was not on interpretation, whereas commercial analysts were more knowledgeable about data application.

I was in a meeting this morning with um – I won’t name the PCT, but it was a PCT. I spoke to two analysts and the programme manager, and the programme manager knew what they were doing, the analysts were contractors who were just brought in and knew the nuts and bolts of it, but didn’t have any knowledge, and were just there to churn out the information. And I think the difference is that our analysts actually know a bit more than just churning out the data, because they can apply more knowledge around the business and the NHS.

Howard, commercial analyst

Cultural differences between the NHS and commercial providers

Background

External provider 1 was a small international company that marketed a software tool developed and refined from decades of academic research. Academics carried out studies with the tool in different countries, tested its viability, amended the algorithms and presented findings at conferences, which helped to contextualise the tool, provided credibility and reached new markets.

It’s often that the academicians (sic) through publications, through presentations and conferences and so on that proves the [tool’s] viability within a particular country or setting and demonstrate its value. And then the government gets – you know – it gets their attention. But it has already the acknowledgement and the credits that come from having been already vetted out in an academic setting.

External provider 1’s business model was based on keeping their company small and relying on other larger locally based companies with licences to provide software, training and the ‘wrap arounds’ (or ‘front end’) which were software/user interfaces. The advantage of working through suppliers was that these larger companies could ‘exploit the system’ by ‘react[ing] much more strongly to a market or emerging market’ and were big enough for initiatives such as FESC and the Lead Provider Framework. As an international company, external provider 1 also valued the local knowledge these larger suppliers had and received ‘added value’ such as the ‘report generator, additional analytical tools or consulting that they [the suppliers] offer the user’.

A disadvantage of working through local suppliers was that the licensed suppliers were selective about the level and type of knowledge that filtered through to end users, as the aspects of the product that local suppliers highlighted varied. For example, external provider 1 noticed that one supplier was ‘much more interested in the delivery of information, how it looks and feels and so on’ because they were an informatics company, while another prioritised the ‘managerial aspects’ because it was a ‘classic management consultancy’. This meant that the management consultancy ‘struggled with not being able to understand the subtleties in the technology and how sometimes a particular report in terms of the way it looks is actually important to whether it gets adopted or utilised in any way’.

Knowledge exchanged

This vignette begins with a NHS team acting on behalf of a consortia of commissioning organisations contracting a licensed supplier of external provider 1’s tool. The terms of that particular contract were that no direct contact was allowed between external provider 1 and NHS staff; all contact had to be filtered through the licensed supplier. (The latter was concerned that, with unmediated contact, the clients might decide to contract external provider 1 directly and sidestep the local supplier.) Consequently, NHS end users were unable to develop more advanced skills, as they were working with a supplier with less understanding of the tool than the clients would have liked. However, the contract with the licensed supplier was eventually renegotiated by the NHS contracting team, lifting the clause of no contact. External provider 1 was then contracted directly.

To facilitate knowledge exchange, external provider 1 arranged for experts in designing the software tool to teach selected NHS end users more advanced skills. Known as ‘superusers’, the NHS participants worked across several different PCTs, were chosen by their local PCT and had a variety of backgrounds including analysts, commissioning managers and project managers. The contract between external provider 1 and the NHS was for several modules of advanced training in the tool. The experts were located in North and South America and were chosen because they had developed particular aspects of the tool. At a pre-arranged time, the superusers would log on to a webinar. The expert could not be seen but his/her voice would talk users through a series of slides. Simultaneously, superusers could write notes to each other or the expert or ask questions by raising their virtual hand. Sometimes there was a long enough time lag until the expert noticed the question that the webinar could already be on a completely different topic. At the end of the set of modules, the superusers sat an exam and received a qualification.

The training was technical and oriented to the US health-care system, although a consultant from external provider 1 told us that each module was revised prior to the webinars to contextualise for the particular country. To help contextualise and embed the knowledge further, they used several mechanisms, for example:

1. Two knowledge brokers were present at every session: a consultant from external provider 1 and an information manager who worked for the NHS team that set up the contract and who knew the superusers well.

2. Superusers were given homework. Homework questions for module 2 included ‘describe the risk factors that are included’ as well as an opportunity to identify ‘take home messages’ and further questions. To improve their understanding, superusers were allocated time to compare their homework when they met at other forums.

3. The superusers were sent the slides a few days before along with some background reading to prepare for online sessions.

4. One participant organised a full-day revision session, when the course was nearly completed.

5. External provider 1 supplied manuals to users about the system with every variable described, but other manuals could also be downloaded.

Knowledge transformation

The tool necessitated GP practice-level data and generated output on patients that were at higher risk of using health services. One superuser’s role was to embed the software tools into GP practices and encourage GP practice staff to use them. In transforming her learning about the software tool to GP practices, she selected aspects of the webinar learning that were germane to practice managers and administration staff and dejargonised the language.

There are a number of reports within the tool, so I focus on two reports that they need. The first one is the case management report, which has their entire practice population within it. And then you apply a series of filters to only have those patients that you want to look at. And that’s taking a very broad approach. And then there’s also an approach where you can concentrate on specific disease conditions. So I concentrate on those two reports mainly. And then I go into the report and I go through column by column so they understand what each column means. And I put it into terminology that they can understand. And then, while I’m there, we actually run a report, we put it into Excel, I go through the filters so I’m showing them how to do it, and actually often I get them to do it. So I get them to do the picking rather than me, so that when I leave they’ve actually got a list of patients. And I think, at the point I leave, they do understand. I think the problem comes because they don’t access it often enough, so by the time they need to go back into it they’ve forgotten. So what I’ve done is produce a PowerPoint presentation with step by step instructions so they can refer to that.

We obtained a copy of this PowerPoint presentation and each slide is a screen shot of a menu on the tool with a red circle around the option to generate case management reports for the Quality and Outcomes Framework. Although this superuser said that general practices did not want to learn how to operate the tool as they had little time, tool operation was the focus of her teaching. Instead, general practices wanted the superuser to analyse the data and send practices the results, and then practices could ‘get on with it’. But information governance stipulations meant that neither the PCT nor this superuser had permission to access to the relevant practice data.

Another superuser who was trained in advanced skills had an analytics background. A major initial challenge was that he had no access to the tool, so it was like ‘trying to be a car salesman and sell a car by describing it to people. We were on a hiding to nothing’. A further complication was that a tool with the same function from a different provider had been operational for over 5 years locally and this was ‘not always flavour of the month’ with GPs sceptical about case management. To remove the association with case management and market the tool more successfully, this superuser showed differences in standardised mortality rates and type of disease by practice to identify novel ways of allocating resources to general practices. This strategy was apparently successful, as after one presentation one locality increased its take up from three to all 10 practices. In training general practice staff, he tended to ‘dive straight into using the tool’ with a case report and explain terms and concepts as he went along.

As a primary care commissioner, another superuser said that the tool was used in several ways. The first was to identify a case list for the community matrons, which longer term could feed into an ‘overhaul’ of the community matron contract and a change in referral criteria. Second, she said that ‘a lot of’ commissioners ask her to run the tool to see if there is ‘anything interesting’ about a particular cohort of patients. For example, to reach elderly individuals who needed some care but were not housebound or known to health-care providers, the CCG had set up drop-in clinics in church halls and sheltered accommodation staffed by community matrons, pharmacists and voluntary sector staff (e.g. Age Concern). To ‘populate those clinics’, the superuser devised an algorithm and ran it in the tool, for example over age 60 years, several comorbidities, ‘a couple of hospital counts’. This list of patients identified was further refined when passed on to the GP practices who then decided if the tool had identified the right people. This ‘did actually throw up people they [general practices] might not have thought about’. The feedback the superuser and her team got from the practices about using the tool in this way was ‘quite, quite promising’.

Challenges

One challenge was that some general practices were not motivated to use the tool. A superuser reported that, once general practice staff were taught, those skills were forgotten, as the tool was infrequently used. Another superuser noted that, because practices were not asked to identify patients as a part of any programme or initiative, the tool was not used routinely. That particular version of the tool also did not allow for stripping out of certain patients (e.g. pregnant women, those with transplants), which meant lists of patients were produced that the practices could not manage. Moreover, trend data did not exist to track patients over time, thereby determining whether or not the tool and consequent interventions made a difference. For these reasons, practices sometimes did not ‘see the point’ in investing their time in the tool.

Another challenge was around data accuracy. For example, one superuser recounted how raw data were extracted from primary care, linked with secondary care data, cleaned and ‘put through the grouper’ (the algorithm). But the result wasn’t ‘right’. The superuser and her team identified that the problem was not with the tool, but with the ‘wrap arounds’ from the other suppliers. Consequently, the CCG had considerable concerns about using the tool and had not launched the ‘charm offensive’ with the general practices. Moreover, the superuser and her colleagues had to learn about technical aspects of the tool which they ‘shouldn’t need to know but we’re feeling we do need to know because we can’t ask the right questions’. However, overall, this superuser was positive about the benefits of this tool; she just thought that the package with multiple suppliers was too complicated. As these data were collected in 2012 and several versions of the tool will have been rolled out since then, these problems may now be corrected.

Views on the training

Satisfaction with the training depended on expectations. One superuser with a commissioning background identified her objectives from the training as:

Technical expertise about how the data is extracted, how it’s cleaned, the algorithm that’s supplied and the outputs that the [tool] gives and in addition to that they [external provider 1] will be in the best position to understand how best to use it.

She said the training left her ‘better informed’ but ‘more confused’, because of its technical detail. When probed about ‘how best to use it’, this superuser said that there were lots of ways to ‘cut the data’ and at the moment a ‘generic approach’ was employed, but she wanted someone from external provider 1 to work with her and a commissioner for long-term conditions ‘to actually think about the best way to get maximum results’, perhaps by developing a strategy focusing in-depth on one condition such as chronic obstructive pulmonary disease or diabetes. She wanted the tool contextualised to her immediate concerns, which was difficult without a tool expert on hand. She also wanted an opportunity to blend her local knowledge with expert knowledge to identify where the tool could make a difference. The absence of a strategy for using the tool with general practices meant that the superuser did not have much clarity about the direction she should be taking with practices and, in turn, practices were less receptive to ‘getting on board’. Her learning from the training did not address this.

Another superuser found external provider 1’s training interesting, not least because she learnt what the tool could do without the ‘front end’ (i.e. ‘wrap arounds’) provided by other suppliers. This superuser thought that the original training by the local supplier did a ‘good job in trying to get across the things we needed to know at the time’ and it was ‘a very useful introduction to what actually is quite complicated’. Through actual use of the tool over time, this superuser found that the implications of some of the training were not appreciated until she was working with live data. She thought that ‘the training is obviously very necessary, but also there’s a period of time of actually using data and reflecting back on things that’s also very important’.

Benefits and disadvantages

At the time of fieldwork in 2012, the benefits of the software tool appeared to be more theoretical than actual, perhaps because superusers had just been trained. A member of the NHS team that procured the tool said he ‘absolutely’ believed that the tool could be useful for commissioners, but thus far making a difference was ‘more the exception than the rule’. Other mechanisms were needed to capitalise on the tool, such as good-quality community matrons, ‘integrated’ working relationships between community matrons and GPs and regular patient review to ensure that the information generated was useful. This participant identified many difficulties, including the inherent challenges in changing the status quo, the complicated environment, the range of stakeholders across multiple organisations who did not necessarily buy into the change objectives, and the short-term focus on ‘today’s problem today’ leading to managing the ‘minutiae’, which militated against longer term strategic thinking, induced a lack of relevant skills among managers in interpreting information, and limited leadership at senior management level to champion the tool.

Another superuser thought that a potentially powerful way of using the tool for practices would be in ‘case-mix adjustment’. Currently, practices were judged against an overall average (i.e. average number of hospital admissions by practice). Using data from 10 practices in one locality, he identified the practice that had the greatest opportunity to lower admissions, which would not have been selected through usual methods. This example was also brought up by a local GP clinical lead, who felt that this application was potentially very powerful. He was also relatively positive about the tool because it had addressed some long-standing problems about resource allocation to GP practices that the clinical lead had been aware of since fundholding days.

As both of these examples suggest, in general, participants identified how the tool might influence commissioning, rather than examples of actual impact.

Box 8 summarises the key points from the first vignette.

Figure 7 explicates visually a few of the transformative processes in this vignette. Chronologically, this starts at the bottom with product deployment.

FIGURE 7.

Transformation processes in vignette 1. CAP, copy, adapt and paste; G, governance; IR, interpersonal relationships; PD, product deployment; PP, people placement.

Table 12 illustrates how the differing agendas and pressures of the key parties affected their knowledge exchange activities.

Figure 8 illustrates how the tool, the knowledge embedded in it and the information derived were transformed by the key players in the knowledge exchange conduits. The ways that these key actors ‘framed’ that knowledge are indicated by the changing shape of the knowledge that each set of actors had about the tool, for example the shift from the tool being framed at its inception as evidence-based knowledge with deep and wide academic credentials to a deliberately simplified and, therefore, useful piece of saleable software.

FIGURE 8.

The transformation of knowledge in vignette 1. CAP, copy, adapt and paste; IR, interpersonal relationships; PD, product deployment.

The software tool

External provider 2 was a large UK-based company that supported several software tools, some developed in-house and some developed elsewhere. Consequently, they had a large team of analysts. The tool in this vignette was developed by another commercial company. It used ‘evidence based protocols’ to assess whether or not a patient or cohort of patients were in the best place of care for their particular needs; for example, those in hospital might be better placed in community settings. According to the product guide, the ‘evidenced based protocols’ that informed the tool were ‘developed and maintained by a Global Clinical Panel of over 800 independent experts including UK based clinicians’. These guidelines were ‘compared with the clinical evidence within Map of Medicine and NICE guidelines and found to be 97% compliant’ (product guide). No further information was available about where or how these guidelines were generated. However, there were references to academic research within the tool.

Although local commissioners had not specifically requested or sought this tool, it was available for regional use. The local hospital had higher than average emergency hospital admissions and the decision was made to conduct audits using this software tool to determine if patients were misplaced in acute settings. Essentially, the audit consisted of clinically qualified staff (i.e. ‘reviewer’) entering patient data by working through a number of screens checking boxes. The reviewer would enter supplementary notes to explain reasons for his or her decisions. The tool then reported whether or not the patient ‘qualified’ by meeting the criteria for care in that particular setting. As the audits took place in ‘real time’, care providers would be available to answer any queries of the reviewers arising from patient notes.

The first audit

A first audit of the local hospital using the software tool was carried out solely by staff from external provider 2 in the autumn of 2010. One NHS client said that ‘the first audit was a disaster really’. He identified several problems, the first being the lack of detail. For example, the tool gave the proportion of those who could have been given care elsewhere without any details about why or where.

It gave us no details. So when we got that we felt that that was unhelpful because a percentage figure didn’t give us any idea of what was required, who those patients were, were there any themes in that? And what facilities were required? So if it was say falls, you know, what’s required? What would we have to put in? Because they assumed that there was a physician who would go and visit the patients at home, and constant nursing support at home, which wasn’t available.

Once the notes were reviewed, they found that ‘whoever applied it, applied it without any common sense at all’, giving them ‘ridiculous’ results. This was thought to be a problem not with the tool, but with its application. Hospital clinicians retested the patients who had been ‘non-qualifying’ and found that nearly all qualified. Examples of those initially classified as ‘non-qualifying’ included a pregnant woman who was ‘essentially killing her baby’, which led to a statutory legal admission, and a man with repeated chest pains who needed a coronary angiogram which could not be administered at home; the tool said that he was suitable for home care. These anomalies were not because the tool was North American (although several participants commented about the challenges in applying a North American tool) but because ‘it was just a complete misunderstanding or very, very poor assessment’. We do not know who within external provider 2 carried out the assessments.

The hospital wanted the PCT board to consider the revised results, but the PCT commissioners were reluctant.

They had just fed back and signed off the protocol . . . One of the PCTs said that they didn’t really care whether it was accurate or not, it gave them the message they wanted . . . they actually refused to look at it.

Eventually, the PCT and external provider 2 were persuaded to examine the revised data and they agreed that the quality of the audit was poor.

And they said ‘Look if you really want us to see your data, if it would help for us to go through your data, we’ll set up a meeting’. And I said, ‘Look, it’s not my data, it’s your data. It’s not going to help me in any way at all. I’ve read it, I understand it. It’s just that you’re planning all around this, and you haven’t looked at the data yourself, and consistently won’t look at it’. So they at that point said, ‘All right we will look at it. And we’ll bring it there with external provider 2’. And I brought the notes and we went through it. And at that point the PCT were completely shocked. So external provider 2 said they just didn’t understand why it had been so poor. And the PCT, I could see they were quite shocked, because I think they at that point felt that I just didn’t like the result of it and was arguing on the edges or the extremes, but in truth it was really, really very poorly done.

The second audit

The experiences of the first audit obviously had a major impact on the planning of the second. Commissioners leading on the second audit did lots of ‘one to one talking, listening to their concerns’ to get the right people back on board. The argument used was that the second audit was using internal people with local knowledge of organisations that external provider 2 did not have. Moreover, the second audit would have joint PCT–care provider ownership with adequate clinical leadership. In addition, the settings expanded to include acute, community and social service premises, including the medical admissions unit, community hospital wards and social service wards. The population under study was all those aged over 70 years currently located in those settings during the 7 days of the audit. The knowledge wanted from the review was:

Are patients being admitted appropriately?

For the patients who are in beds who could go home today?

For the patients who are in beds who could receive a different level of care from another provider?

Meeting minutes

There were some concerns about whether or not the software tool would provide all the necessary information needed, especially around capacity. A bespoke questionnaire was to be administered in tandem with the tool when auditing. Additional information collected by the questionnaire would include:

Is there an existing care package?

Did the patient have a ‘telecare’ service available?

Does the patient live alone?

Meeting minutes

Staff from the hospital meant to develop this further, but a participant subsequently said this questionnaire was never devised. It was not entirely clear if an explicit decision was taken or if the questionnaire dropped off the agenda. Regardless, these data were not collected.

For the health-care staff to undertake the audit themselves, training was necessary. The tool developer (not external provider 2) carried out the training. The training was intended for 5 days, but local reviewers could not be released for that amount of time, so the training was cut down to 2 days. Reviewers included two managers with nursing backgrounds, a consultant community nurse, an acute hospital nurse, a discharge liaison officer without a clinical background and a social worker. Only two reviewers were in current clinical practice. Seven reviewers were trained but five carried out the audit. As part of the training, the reviewers developed a glossary to translate American nomenclature into terms suitable for this particular health-care economy. While entering the data into the tool, consultants from external provider 2 were on hand within the setting to answer any questions. At the end of every day, the reviewers met a senior hospital consultant to discuss any cases found to be ‘disqualifying’ and analytics staff (not clear if PCT or commercial based) then generated data reports.

Knowledge transformation

Two key streams of knowledge were exchanged in this vignette. The first was about the tool and was between external provider 2 (including their tool developers) and local stakeholders and reviewers. The medium was formal meetings with PowerPoint presentations and didactic, face-to-face training of local reviewers both delivered by commercial consultants. But one reviewer said that the real learning came from using the tool because of misconceptions of how the tool operated.

Because we didn’t have an understanding of how the tool actually worked, when we set up that workshop. We were very much posing the questions of: what should be included in the audit, what shouldn’t be, what types of questions should we be asking in the audit? Thinking that an audit would be questioner-based, that you would present – you know, if you’ve got these clinical notes in front of you, you’d also have a list of questions at the side of you and you would answer those questions, if that makes sense. Where the audit tool itself isn’t like that at all. So I think we had a misconception of what the tool looked like.

This comment is slightly confusing, however, as several workshop participants had been involved in the first audit and so, presumably, were familiar with the tool. Nonetheless, the second audit generated ideas among commissioners, community and hospital providers about how to use the tool next time around. One was to follow patients through their journey from day one of admission to test suitability daily to inform length of stay and another was as a 24-hour ‘snapshot’ of A&E.

The conceptual model of the tool was especially useful to hospital staff. They replicated the approach (without the tool) in ‘several audits of little areas’.

I think the whole question of looking at admissions and what was required, and what services could be put around it, is one that is so obvious that actually we weren’t thinking about it. So although the audit didn’t achieve that, or the first one didn’t do it, and the methodology seemed so obvious, the questions seemed so obvious, we weren’t doing it, so I think it was quite useful for the initial concept, because it is a simple concept . . . and I think they’ve [commercial provider] brought that.

This conceptual shift had generated substantial enthusiasm, but neither the hospital nor commissioners said that they would use the tool itself again. Moreover, the hospital would not work with external provider 2 again.

A second stream of knowledge centred on outputs from the tool and how those outputs informed commissioners’ decision-making, as a NHS participant said that a key problem was acting on the findings. Despite numerous requests, we never obtained the final report from either the first or the second audit. However, we did collect a PowerPoint presentation for an audit using the software tool at a neighbouring hospital a year earlier. This provided an account of the tool outputs.

This report presented separate findings into acute admissions and long stay with ‘headline’ information for each. This information was then broken down further for each category, into ward, ward type, specialty, patient age and admission day. Additional information on patient characteristics (e.g. lives alone) and admission source (e.g. GP) was given for long stay. Alternative levels of care were identified for all three categories (e.g. acute admissions, long stay). Results from qualitative interviews with clinical and managerial staff were then presented in terms of ‘opportunities identified’. The report gave separate recommendations for commissioners and providers, for example ‘Review sub-acute provider admission criteria and admission protocols in consultation with acute provider. Implement monitoring arrangements to ensure that these are adhered to and remain based on level of care need’. The report finished with ‘next steps’ (report).

Although we did not obtain a copy of the final report for the audits under study, a participant said that 28% of patients could have been cared for elsewhere in the first audit, while the second audit found that 24% of patients were in a suboptimal care setting. Although the difference between the two audits was minimal, the results of the second audit were widely accepted, whereas the first audit was highly contested and led to the disengagement of the hospital. Consequently, the first audit did not generate much knowledge transfer. A participant in both audits said that the second was more successful than the first because of local involvement across providers and the commissioning organisations. This created ‘a much better understanding of what was actually happening in hospitals’ across the health economy, which was ‘invaluable’. Local engagement led to local ownership.

In the findings presented to the PCT board, recommendations included conducting an audit with the 50- to 60-year-old age group, given that fewer over-70s were in hospital and community settings than anticipated. This also meant that commissioners needed to look elsewhere in the system to reduce hospital demand. However, it is unclear whether or not these recommendations were taken on board, as a subsequent e-mail in late 2011 said that ‘the results have not been used in any commissioning decisions’. The timing of the publication of the report coincided with the height of the turbulence from the 2012 Act and the departure of the PCT champion to a new job, so possibly other factors contributed to its limited impact on commissioning.

Benefits and disadvantages

A participant said that the local stakeholders could have conducted the audit without the software tool, but it would have been ‘very qualitative’ and the same quality of data would not have been produced.

Generally, local stakeholders found the second audit ‘very useful’, but not because the tool gave much insight into unplanned hospital admissions, nor did it provide information to feed into planning for future care. This was partly because the questionnaire to collect these data was not developed but also because of ‘gaps’ in the information provided by the reviewers, including the lack of recording of NHS number. Without current clinical backgrounds and possibly because of the abbreviated training, some reviewers ‘struggled’ and the quality of data was poorer than anticipated.

Nonetheless, the audits had an entirely unexpected outcome, in that relationships between the hospital, community providers and commissioners improved, particularly through the fieldwork phase of the second audit when local reviewers, commercial consultants and hospital staff met daily to consider the doubtful cases. Several participants stressed how important this improvement in relationships was, as previously the hospital had not worked well with the commissioners or the community providers, ostensibly for over 20 years. A participant said that ‘it made us start talking about data rather than prejudiced opinions’, which meant that formerly fractious conversations were now based around ‘evidence’.

I’d say [I had a] very negative experience from working with this tool and the way it was applied. But ultimately it might have been the trigger to really make some very big changes in the health economy, in terms of making us realise we have to work better together. And that probably was more valuable than the tool ever could be. So paradoxically [the commercial company] helped. And I think they helped because it was the beginning of saying, ‘We’re going to talk about evidence rather than about opinions’.

In sum, the application of the tool mended historically fragmented relationships, built trust within the local health economy and helped the hospital to consider other ways of grappling with a longstanding problem. But the commercial provider played a minor role, mainly in offering a forum for the stakeholders to work together and providing knowledge about the tool.

Box 9 summarises the key points from the second vignette.

Figure 9 below illustrates a few of the transformative processes in this vignette. Chronologically, this starts with number (1). Table 13 details the motives behind these transformations further. Figure 10 summarises a few of the crucial transformations that the key actors in this vignette undertook to reshape the tool, its embedded knowledge and the information derived from it. It also illustrates the way they framed the status of that knowledge as a result of their contextual demands and constraints, and their worldviews.

FIGURE 9.

Transformative process in vignette 2.

FIGURE 10.

The transformation of knowledge in vignette 2. EP2, external provider 2; CAP, copy, adapt and paste; G, governance; IR, interpersonal relationships; PD, product deployment; PP, people placement.

Background

External provider 3 was a UK subsidiary of a North American health-care corporation with an international portfolio. Although in the UK external provider 3 employed over 100 staff, they had access to several thousand North American employees, who acted as international experts, developed software tools and sometimes filled UK staff shortages.

In 2008, external provider 3 won a 2-year contract to carry out several specific pieces of work for a PCT. During the first year of the contract these were renegotiated and four main work streams emerged: (i) support for WCC; (ii) complex case management, including the introduction of a software tool; (iii) development of a ‘savings cabinet’; and (iv) consultancy/coaching work with newly appointed directors within the local commissioning structures. This vignette focuses on work streams (i) and (ii). Various extensions meant that the contract eventually lasted 3 years.

External provider 3 had five to six whole-time equivalent staff working on the contract, but this involved eight main individuals. As well as a programme manager and a clinical lead, there were two North Americans who spent a substantial amount of time on site, an associate programme manager, one or two people for ‘actuarial-type work’ and an analyst. Some of these staff were ‘coming and going’, but a feature of the work seemed to be the continual presence on site over a long period of time of the clinical lead and the associate programme manager.

Knowledge wanted

The commissioners in this vignette worked within a strong local culture of collaboration, innovation, public engagement/accountability and transparency. They were very open to working with external agencies to support their commissioning activities, and did so regularly because sometimes they were unable to attract candidates with the necessary experience. Being able to recognise where external support would be helpful was an area where the NHS clients felt skilled. During fieldwork, several NHS staff spoke about the importance of building up a network of external organisations and individuals who had the right knowledge and skills. The expense, however, was mentioned as an important constraint on accessing external support.

At the time of tendering for this contract, which pre-dated the 2012 Act by several years, the commissioners were ‘trying to go from good to great as a commissioner’. To achieve this, they wanted to look outside the NHS as ‘we wanted to set our aspirations greater than just what was available nationally and look at some of the good practice internationally.’ Contracting a company with a North American base was, therefore, particularly appealing. Consultants from external provider 3 described the commissioners at this particular NHS site according to Belbin’s team roles. 51

So they were the plants and shapers, but they weren’t the completer finishers. I would say that that was evident when we were working with them, that they had a huge amount of ideas. Lots of shaping, lots of meetings, huge meeting culture, and then the actual discipline of completing it and measuring was not there.

The external provider 3 consultants assigned to the contract saw themselves as ‘completer finisher’ personalities which complemented the missing skills within the PCT. Both external provider and NHS staff spoke about the degree of focus on performance management as one of the key differences between their organisations. Another commissioner summarised the knowledge and skills which he thought external provider 3 had brought to the NHS client organisation as good, real-time analysis, programme management, engagement and coaching.

World Class Commissioning work stream

External provider 3 described a ‘gap analysis’ based on the success criteria for WCC as ‘hard going’ for both sides, but also ‘the key’. Through the gap analysis, the external providers sought to help the commissioners understand the assurance process. The consultants went through each commissioning competency and challenged the commissioners to ‘demonstrate that you actually do that. Give me the tangible evidence’. The gap analysis also facilitated communication within the NHS client organisation, leading to the discovery of duplicated work.

A ‘massive programme of work’ was then undertaken to prepare for the next WCC assurance day. An early task was to work with commissioners to identify who should be part of the WCC process ‘not just having the same people that put their hands up, but actually who does need to be involved’. This included identifying the stakeholders external to the commissioning organisation: ‘if you don’t start engaging with them as of day one, when they do get contacted they are going to give a very different message to what you think they will’. The external providers said that the commissioners recognised that project management was not one of their strengths, and so training was provided: ‘a lot of the stuff that we did was around developing staff to take on projects’. They also provided information about tools which could be used for particular tasks, and further training around these.

The commercial consultants mainly drew on best practice examples nationally and internationally, identified from existing and new contacts to inform this work. Some of this knowledge came from colleagues working at other NHS sites who had already gone through the WCC process. The external consultants on site brought in experts from external provider 3 to the NHS client site, including North American staff who provided some training. But they also made contacts based on publicly available information and facilitated exchange of knowledge between NHS organisations.

Case-management work stream

A NHS commissioning manager described the second work stream as ‘a project directly to support adult social care and community nursing to think about what we needed to make that work here’. First, a software tool was used to identified the ‘riskiest population’ in each of the localities, and then the external providers ‘sat down with the community matrons’ to compare the lists of patients produced by the tool with current patient lists (presumably based on clinical judgement). They found little overlap. External provider staff said that this had ‘really started [NHS clients] to ask questions’. Agreeing how to proceed had been a long process, necessitating going ‘back quite a few steps’ to discuss the basic principles of case management and the definition of a ‘complex’ patient. External provider staff felt that NHS staff sometimes did not have thinking space or ‘someone next to you challenging’ your assumptions. The software tool, and the external provider staff supporting it, had provided this challenge. This led to changes in the implementation of case management.

As part of this work stream, the clinical lead from external provider 3 was involved in project meetings with the leaders of the NHS clinical teams and she also ran sessions directly with nurses and social care workers. This consultant said that within the existing case-management programme ‘there was nothing to say whether what they were doing was actually adding value anywhere in the system’. So, as evidence to evaluate the impact of the existing case-management programme was not being generated, this was something else offered by external provider 3.

In addition, the personal qualities of the external provider lead were appreciated. A NHS client said that the external provider clinical lead was ‘a skilled programme manager’ and had clinical knowledge and experience as a nurse, a combination missing from their own organisation at that time. Actually having carried out case management elsewhere, and speaking from that experience of changes, the various benefits and pitfalls, was key to the external provider’s value. The NHS commissioning lead was also a clinician and this similarity in backgrounds meant that the external provider clinical lead had ‘very ably’ supported him to get other clinicians on board.

Challenges

The main challenges, which were raised by both sides, were those of defining what work was needed and ensuring the work remained relevant and useful in a constantly changing commissioning environment. A NHS commissioning manager felt that several factors had affected the relevance and usefulness of work done by various external providers for her organisation. She felt that either partner could be ‘at fault’, with the brief sometimes not having been that good, but also ‘what is brought is not quite as relevant as it could have been’. The political climate had changed after work was commissioned, meaning that it was no longer as acceptable. The contract was sufficiently flexible for changes to be made when needed, although this did take a good deal of time and energy. The long timescale of the contract facilitated this, as did the mutual goodwill to engage.

Views on the work

The NHS commissioners expressed positive views about the quality of the work conducted by external provider 3. The clinical lead assigned to the project was rated particularly highly, and was seen to have good credibility based on her ‘sound knowledge’. A commissioning manager said that the external provider had ‘much better horizon scanning, because they were looking internationally’, while another manager emphasised how the external provider had been skilled at interrogating data, identifying areas which needed work, and coming up with ‘good tools which we could use’. Another commissioner described external provider 3 as having ‘both the tools and the understanding of those tools’. The manager who had been closely involved in the case management work was very satisfied.

What we’re buying is a polished product for something that we want to do anyway. But of course you then come into the aspect of how they support us to get going, and that’s a standard supplier/provider thing, and we’re very happy with that, I think they do a very good job. If I didn’t, they wouldn’t get any more business.

A GP commissioner was less sure about whether or not the contract had been a good use of resources:

They did some work here but for what we paid them I’m not sure. It was somewhat difficult to understand exactly what they achieved in the end. So I think if you’re going to do that you’ve got to be very clear about getting value for money because it can be very expensive . . . you know, well it’s the cynical idea, isn’t it? You get management consultants in to tell you what you know already. They just go round, you know, finding, you know, and producing, probably hopefully producing solutions that you may not have seen, but probably you have seen, but not been really able to put into effect.

The quality of the work was not the only factor affecting how useful it ended up being, and this senior manager’s view that some streams were very useful and others less so was a fair reflection of our data overall.

There were some really good pieces of work where we were looking at things such as risk stratification and programmes of work going forward, and those worked quite well . . . I think both of us [external provider and clients] probably felt, at the end of the day, that we’d got some really good things out of it but perhaps not as much as we had hoped.

Benefits and disadvantages

The external provider staff assigned to the contract felt that it had gone particularly well, saying that their organisation was really proud of this project. They described knowledge transfer as having been a pillar which stood alongside all of the work streams, and that this was important to their professional integrity. The benefits perceived by the NHS clients included those linked to specific projects but also more general skills and approaches which they felt they had acquired through the work.

And they have supported us in making us better at doing the evidence. Because external provider 3 will never do anything without having some data analysis, they’d have almost a hypothesis to do something and then they’d test that out . . . So I think they have made us more organised in that way.

A commissioning manager referred to a ‘commissioning toolkit’ of ‘those types of technical things that we would need to support us going forward, which we’ve used’. She felt that the external providers had led by example but had also set up the systems and processes that ‘then enabled us to see and learn how to do that type of process’.

The WCC work stream had been very effective in moving the NHS client up the national rankings, but it was not clear what the impact was on their actual commissioning at the time. A minority of NHS clients were uncertain about the ongoing value of the work a few years down the line and in a very different commissioning landscape.

The case-management work stream had challenged how community services were being targeted, and had engaged clinical, social care and management staff in negotiating agreed definitions of targets and objectives for case management. A system for measuring outcomes was also introduced. However the ultimate outcome of the work was beyond the control of both external provider 3 and the commissioners, as it required the service providers to deliver the benefits. Restructuring of community services owing to national level initiatives was seen by a commissioning manager as having limited the impact. This was ‘absolutely nothing to do with the expertise we brought in’.

Yeah I suppose, hmm, [long pause] I mean to some extent, you know, to some extent we’ve gone as far as we can at the moment on that particular case-management initiative. And I would point to it and say that was a result of their involvement.

Nevertheless, this work was described by another commissioner as having been ‘the biggest gain’ on the contract. He commented that the external providers were ‘very good’ at working with professional groups and felt that an opportunity was missed in not bringing the external providers together with clinicians earlier. Another commissioning manager also felt that the clinician engagement had worked well, and that it was where some of the legacy lay, as it was ‘real for people, and it’s not just feeling as if it’s a theoretical exercise, so people can see that there has been a change that are using the systems’.

The WCC work stream is depicted in Figure 11. Although the numbers indicate possible chronological order, this was not entirely clear from the data. Moreover, because the WCC aspect of this vignette is derived mainly from the experiential knowledge of external consultants, using data to develop tables and models tracking the transformation processes of that knowledge was difficult. So, no table or figure equivalent to those in External provider 1 and External provider 2 accompany this vignette.

FIGURE 11.

Transformative proccesses in vignette 3.

Box 10 summarises the key points of the third vignette and Box 11 summarises the key points in the chapter.

Commissioner/analyst silos

The gap between analysts and commissioners also created difficulties in measuring the impact of contracts between the NHS and external providers. A major product of commercial/not-for-profit providers in this study was the generation of good-quality data using sophisticated tools. But within the NHS generally, data were not used to generate change. Therefore, this affected the assessment of the impact of these contracts. Moreover, because interventions were software-tool based, NHS/CSU analysts were more likely to benefit from contracts with commercial and not-for-profit providers through more powerful, easier ways of generating data. But benefits such as easier data generation often were not noticed by commissioners, as commissioners had little knowledge of the work of the analysts. The schism between these two professional groups meant that an appreciation and understanding of the impact of these contracts might not have percolated through the system. However, sometimes commissioners did notice. In one contract, NHS commissioning clients clearly identified how detailed analysis of data from invoice validation software had led to million-pound savings.

Dependency on other factors outside control

A further difficulty in measuring the impact of contracts was that often the change indicated was outside the influence of the external provider (and sometimes the commissioner).

For example, the purpose of many software tools was to identify populations for care interventions. Assessment of impact depended not only on the quality of the software tool, but also on the care intervention. If there was no care intervention, a poor-quality care intervention, or one that affected few people, then the benefits of the tool were blunted. Moreover, the benefits often were not realised for a long time, which frustrated short-term assessments of the impact.

The question that I get asked most often . . . is, ‘Well where’s the evidence that having [software tool] makes a difference?’ And er it’s an incredibly difficult question to answer because – and my stock response is, ‘Well actually having a system that combines data, runs it through an algorithm and provides you with a set of information on its own isn’t going to make any difference. But if you’ve got good-quality community matrons, if you’ve got GP practices that have an integrated care team, where GPs and community matrons and possibly social workers meet together on a regular basis and review patients from the [software tool] list, and you’ve then got a willingness to change the care pathway for those patients, then you will see a change, and you will see an improvement’.

And is that happening anywhere?

Um yeah there’s er – I think it’s probably more an exception than the rule.

Alfred, NHS analyst

Sometimes NHS clients accepted that the impact of the tools was dependent on the behaviour of others and these did not negatively influence their view of the commercial provider. Elsewhere, commissioners were less sanguine.

And they spent £5,000,000 on it and they were busy, I went down to the meetings down in [X] on a regular basis, and every time, every meeting I went to they were saying, ‘Oh well, yeah, it’s all rolled out across [County A] and [County B] you know, I can’t understand why you don’t use it.’ I said, ‘Well what have they achieved?’ They said, ‘Well they’re finding – they’re finding out lots of things.’ I said, ‘Yeah but what have they actually achieved?’ and nobody could tell me a single thing that had happened as a result of it.

Patrick, GP commissioner

Changing priorities

Another challenge in measuring benefits was the changeable nature of commissioning. Both commercial providers mentioned contracts that initially had substantial knowledge exchange mechanisms and that were subsequently dropped from the contract when more pressing needs arose.

And one of the biggest challenges for us around the skills transfer is the recognition of what our clients need to do to be ready to take it. So going back to the [software] tool we are using, the process has been designed whereby we would do the first round of audit in the setting. The next round 6 months later, we would coach and mentor the PCT staff to do that in a community setting, so they learn how to use the tool with a member of our team sat next to them. And they would be trained in how to get the outputs from it. What happened in practice was the PCT said, ‘We don’t have the staff to be able to do that so what we’d like to do is pay you extra to do the audit for us again’. So I’ve written and said, ‘Okay, we could do that, but you’re not going to get the transfer of knowledge and skills’. So we have to be really, really careful. Our whole ethos is that they get this knowledge and skills so they get the benefits but if a client chooses not to do what they need to do, we have to be very, very explicit. Because if we get to the end of this and they say, ‘Well you haven’t told us how to do this; it all gets very, very muddly’.

Lana, commercial consultant

In some situations, the commercial providers genuinely wanted to maximise knowledge exchange, but the NHS clients were not very interested. In others, the commercial providers did not appear to value knowledge exchange. One contract became particularly difficult because the NHS clients believed that the commercial providers deliberately avoided transferring knowledge to keep the clients dependent.

Isolating desired consequences

Another difficulty in assessing the benefit of contracts with external providers arose from the complexity of the NHS and the wider health and social care system. Benefits from one intervention might easily create unintended, negative consequences elsewhere. Moreover, isolating the benefits of one particular intervention was difficult if multiple changes were taking place within the health economy concurrently.

Sometimes you might have a very successful project, but it will be swamped by something else. For instance, we had a pathology lab project, and I believe it was quite successful on the elements that they were targeting, but the overall pathology results showed a huge increase because there had been a new policy put in around dementia testing, which involved eight different pathology tests.

Connie, CSU analyst

Benefits slow to materialise, subtle and difficult to measure

Benefits were often slow to be realised. For example, an experienced senior commissioner was brought in about 18 months into a 3-year contract to give notice of its termination as the NHS clients thought it was not working well. This manager thought that the difficulties were because the contract was underfunded and forecasting of overspend ‘tended to bounce around quite a bit from one month to the other’ (Jacob, commissioning manager). He convinced the NHS clients to continue with the contract, arguing that 18 months was too early to reap the benefits. The volatility calmed down over the next few months, as the commercial provider had predicted. In the following year, the full budget was allocated, the savings target was delivered in full and the ‘overall contract envelope came in pretty much on the button which is quite an achievement really I think for [commercial company] given the amount of change and sort of chaos frankly that was going on in the system at the time’ (Jacob, NHS commissioning manager).

Although the success of this commercial provider was recognised, a few commercial consultants and a Public Health consultant mentioned that sometimes the benefits from their contributions might be overlooked. This could be because as knowledge was transformed, tracing it back to the source was challenging, but also because often benefits were subtle, such as changes in attitudes, relationships or dynamics. For example, a commercial consultant recounted how the use of a software tool had improved local relationships:

I mean I think that we have added value to the organisation, and added skills and expertise. I’m trying to think, if you went in there, would they say, ‘Oh, you know, this is what we did before and this is what we did now’. I think they would see some changes in terms of just working around some of the day-to-day commissioning issues. I don’t think what we have affected is some of this – well I think we have affected the dynamic relationship with the provider actually. I think we’ve certainly added some expertise and some clinical expertise in terms of the work around [software tool]. So an area where they thought they had a problem but there had never been the evidence to have that kind of discussion with the clinicians. Yeah so I think they might say that – you know, or they didn’t have the analytics or the – so I think they would probably say that some of their – they are now almost better equipped to make those decisions or to have the discussions.

Betty, commercial consultant

Interestingly in this case, the perspectives of NHS clients who used this commercial provider concurred with regard to the development of better relationships with acute providers and production of evidence informed discussions, although they categorically stated that the commercial provider had not brought clinical expertise (see Chapter 8, External provider 2). Neither have they elected to buy the tool again.

Moreover, even when changes could be identified and agreed, quantifiable measurement was open to extensive negotiation. For example, in calculating the savings accrued by deploying a software tool, a participant said that nurses who were embedded into acute hospitals to check invoices had looked into 150 cases and ‘made real patient impact’. A monetary figure was put against this in terms of the savings realised from earlier discharges and reductions in pre-operative lengths of stay. But this figure was not convincing enough; the nurses were pulled from the hospitals and moved back into the internal audit team because it was ‘very hard to quantify those savings’ (Dennis, commercial consultant).

In summary, there were many reasons that made demonstrating the benefits of contracting external providers challenging. These included:

• the split between analysts and commissioners which left commissioners uninformed about the value of external provider input

• the dependence of benefit generation on other professional groups within the system who were outside the influence of the external provider

• changing commissioning agendas, which led to the prioritisation of different benefits

• the complexity of the NHS, which meant that benefits in one area could lead to disadvantages elsewhere

• the usual challenges in outcome measurement, such as difficulties in attributing causality, or in distinguishing and quantifying the impact of earlier or more subtle inputs.

Generally, perceptions of change depended on the agenda and position of the informant. Commercial consultants and the NHS clients responsible for their contracting understandably claimed greater evidence of added value to justify the expense, while those with more negative views of external providers or decision-makers who did not see directly the contribution of the external providers were more likely to downplay or overlook external provider contributions.

Aggressive, strategic marketing

Several factors could negatively affect the contractual relationship. As mentioned in Chapter 7, several participants had concerns about the profit-making agendas of commercial companies. For example, a Public Health consultant talked about attending a conference where commissioners were ‘pressurised’ to purchase their products. Another participant was suspicious about the claims about software tools, because he thought that these might stem more from a desire for profit rather than usefulness.

And I think there is huge value in going to the States and looking at those tools and saying, ‘Well which one of these works in the UK?’ Not, as the management consultants are doing at the moment, looking at them and saying, ‘Which one can we make (the biggest) profit margin on in the UK?’ And I suspect (why) [software tool 1] have been touted fairly heavily by [commercial provider A] is because [software tool 1] come with a much lower price tag than [software tool 2] which, you know, are owned by [commercial provider B] and – and [commercial provider B] don’t give things away.

Harold, freelance consultant

Limited desire for external provider support

A major factor influencing success was the desire for external provider support. Being told what to do was endemic in the NHS, and NHS professionals were used to having to try to find some merit in hierarchically imposed interventions, programmes and tools that were unsolicited or appeared not to meet their needs. In one contract, after a tendering process that took nearly a year, the client needs had changed and so the ‘solution’ no longer addressed a clear ‘problem’. Moreover, it was never clear whether or not frontline, operational analysts, on whom the success of the venture depended, had been consulted or engaged. One freelance consultant with a view from the sidelines, said that he thought that this particular contract was ‘appalling and shambolic’ on both sides. For example, he claimed that NHS staff ‘intentionally sabotaged projects’ (although details were not provided), because of ‘anti-private sector sentiment’. Meanwhile, there were competence issues with the commercial provider who could not deliver or ‘get the requirements quite right’ because they had a ‘less than helpful customer’. In his view it was unsurprising that the success of this contract had been ‘patchy’ (Randall, freelance consultant).

Participants discussing other contracts noted the difference between those clients who were ‘forced’ to work with external providers because they were poorly performing and those who were more ‘open to knowledge’ (Joel, commercial analyst). Hostility from NHS staff stopped knowledge transfer in several contracts, as this process was possible only if both sides were receptive.

Potentially I think what’s been challenging across the project generally is a little bit of hostility initially from the NHS staff definitely, all blame on their side I would say. But that, I think, has created quite a difficult relationship, and I think that is impeding what could otherwise be really good knowledge transfer.

Paula, NHS commissioning manager

Lack of clarity around the brief

If a contractual relationship had not been desired or sought, or even when it had, a related difficulty was lack of clarity about how to maximise the contribution of the external provider. A commercial consultant thought that the NHS clients were not getting the best out of one contract because PCTs did not know what they wanted.

I’m probably being unfair, but from the PCT perspective, I don’t think they’ve actually sat down and thought, ‘How can we really get the best value for money from this particular contract in terms of sharing knowledge?’

Georgia, commercial consultant

A CCG chairperson also felt that the reason this contract had not turned out as intended was because ‘there was not enough intelligence in the commissioning of that set up in the first place. In other words, the commissioners didn’t really know what they wanted’. This meant that the private providers were not clear on what they were supposed to deliver ‘and there was not then the engagement between the intelligent commissioner and the intelligent provider to actually deliver a product that was really useful’ (Simon, CCG chairperson).

A NHS commissioning manager from another CCG thought that commissioners overfocused on the ‘holy grail’ among tools at the expense of thinking through what information tools might provide to create change.

Actually these things are tools and they’re not perfect. They’re giving you a clue and pointers, but at the end of the day you have to interpret them and apply [laughs] a bit of common sense, and actually take some action. And I suppose that’s all it is. I think people spent such a lot of time talking about tools and not enough time in actually thinking about systems and what we needed to do.

Sandra, Public Health consultant

This lack of clarity about the purpose of external provider support was a common refrain across the case sites. Partly this stemmed from minimal knowledge about what external providers could offer, but the rapidly changing needs of NHS clients also exacerbated this difficulty. To redress this confusion, help clarify the brief and generate enough work to justify the costs of the contract, one external provider held meetings with senior commissioners and analysts attached to the contract across several organisations. The aim was to identify three clear projects for each organisation, develop an action plan and track progress. This resulted in several impressive, colour-coded documents with progress quantified in percentages for each project. Moreover, the office walls of the commercial provider were covered in different coloured Post-it^® notes with information about progress and the occasional headline, for example ‘All project deliverables for April achieved!’ However, 3 months later when enquiring about these action plans, we were told that they were out of date and no longer in use.

Sometimes another contributing factor was that the NHS client who first contracted the external provider was no longer available, leaving a gap in terms of understanding the purpose and vision behind the contract. This was especially true in contracts that relied heavily on one manager to develop the tender.

One of the critical things that happened, in fact it was their start day was – I became seriously ill and I was taken out of the system for 6 months. So having done all the work up, all the negotiation . . . So there was a steep learning curve, having built that relationship with me, and working with me and understanding what I wanted, and I went off on the day that they arrived, and other people had to try and pick that up. But I would say, regardless of that, I’m not saying that there wouldn’t have been difficulties even if I had been there.

Sarah, NHS commissioning manager

Without a vision about the purpose of the brief, clients often struggled to know how best to interpret and apply the output, so little happened. Sometimes the presentation was a ‘glossy’ document that was not amenable to application, perhaps because of limited resources dedicated to implementation.

But I think a lot of these large consultancy firms come in, they go through the data which is corrupt and needs uncorrupting, and then they give you a load of information. But when you try and put it in, it doesn’t do anything. So I mean [commercial provider] so they’re very good at coming and giving you a false sense of security as to what you’re doing . . . But they’re also not given the time to actually make the change, you know. So they’ll keep coming in to give you a diagnosis, but it tells you what you already know. But they’re not really given the opportunity to implement . . . there’s lots of money been spent on that sort of information, but it doesn’t get transferred to the organisation.

Percy, freelance consultant

Output of limited value

NHS clients also tended to not value the contributions of external providers if external provider output was not very helpful. In addition to the many examples reported earlier, analysts from one CCG compared the positive predictive values of several different risk predictions tools using local data. They found the results were ‘pretty disappointing’, as in general these tools identified only one ‘true’ positive for about ‘five to six’ false positives (meaning that for every five to six patients that the tool suggested needed interventions, only one genuinely did). They went on to calculate the costs and found that given the number of false positives, any intervention, to be cost neutral, would have to save around £700 a year per patient, which was not feasible. This contributed to their lack of engagement with the tool. Reportedly, those who undertook this work said the results were fed back to the external providers who ‘didn’t want to know’, although there could be many reasons for this, including concerns about the robustness of the analysis.

In summary, unhelpful factors included:

• aggressive marketing by commercial providers

• limited interest in external provider support from NHS clients

• lack of clarity and vision around the contract’s purpose

• limited impact, for example owing to restricted resources, an unclear brief or departure of the client who originally let the contract

• limited emphasis on implementation in favour of strategy

• outputs of limited value

• lack of early involvement of those expected to use the external services

• changing commissioning environment and priorities.

Understanding clients and the market

Having discussed challenges, we now focus on factors that facilitated contractual relationships between the NHS and external providers.

In comparing and contrasting various providers, one participant said that some commercial providers ‘really got it’, meaning that client needs and the health-care market were understood.

We worked with [commercial provider A] and it’s a struggle, if you know what I mean. And the reason I’m thinking it’s a struggle is I’m thinking that these guys just don’t know health care . . . They’re trying to be management consultants, they’re trying to fit – you know, they’re trying to support CSUs as if we just ought to be generic businesses: we’re not generic businesses, we’re health businesses and you need to understand health care. My challenge is the support we need needs to make us the best in giving commissioning support. We need to be able to articulate the question, you know, we think commissioning support in the future will be all about X or Y or Z or what does transformation mean, what does outcomes based commissioning mean? And actually when you speak to this particular advisory, they can’t really answer those questions.

Kurt, NHS analyst

He went on to say that external providers who met clients’ needs and had insight into the ‘market’ were more likely to produce something useable and thought provoking. This participant, as well as others, seemed to suggest that when contracting external providers, NHS clients were looking for something that went over and above what was generally available.

Given the rapidly changing nature of commissioning, external providers who regularly revisited objectives, adjusted to the changing environment and contextualised their support appropriately were better placed to meet or exceed expectations.

That was one of the things I really found was that, you know, it seemed to be that every 3 months there was a different tune in town, you know, there was a completely different emphasis on what needed to be figured out or worked on now and so it’s quite easy for something to be the flavour of the month at one time and then for it to be almost accepted as standard or potentially not to be relevant any more.

Randall, freelance analyst

Helping clients to do the work themselves

Other participants mentioned that helping clients to do the work themselves, rather than doing the work for them, was beneficial. Of course, clients needed to co-operate for this to occur. For example, one participant mentioned a previous experience working with commercial providers on market analysis whereby the management consultants ‘upskilled’ the NHS staff through the use of tools ‘and we had to do it ourselves’. She thought that this work was ‘really excellent’ (Paula, NHS commissioning manager). A commercial provider echoed that embedding knowledge transfer through upskilling enhanced the value of their work.

You asked about what else do we bring to the table. I think it is that piece of our knowledge transfer. It’s that we do like to be an organisation that it’s not that we go in there, we deliver and we walk away, but we go in there, we deliver, we show them how to do it and then we walk away. And that was very much a focus at [CCG].

Yeah that’s a really good point actually.

Even though we had four work streams eventually, there was this pillar next to all those four work streams that we had to be able to, as an organisation internally demonstrate knowledge transfer.

And we know they’ve continued to do the work that we started there. And in fact we’re going back to do some more work with them. I’m going back to do some training with them at the beginning of March. So we do know that they are continuing on that work . . . And that’s really – and I think, personally for us two, we were really keen on that when we were coming towards the end of the contract, that we’d left everything that we could possibly leave with them.

Patricia and Helen, commercial consultants

There was an interesting paradox in this account, in that the commercial providers wanted to avoid dependency but simultaneously mentioned being recontracted for further work as evidence of success.

Developing non–threatening relationships

Not all successful contracts appeared to depend on developing longer term relationships, but some did. Non-threatening relationships were particularly helpful.

I think sometimes we don’t have the – I don’t wanna call it a luxury – but sometimes you don’t have the time to build that relationship. We were able to build, as a team, not just myself and Helen, but as a wider team we were able to build a relationship with the people that we were working with and others, so it gave us a greater understanding of being able to identify what would help them and what wouldn’t, and then to challenge that when needed . . . And I think because we had built that relationship, we were then seen as an organisation coming in that they didn’t feel threatened by.

Patricia and Helen, commercial consultants

However, not all longer term contracts resulted in better relationships. Another commercial provider with a 3-year contract kept switching embedded consultants, depending on the changing needs of clients’. This interrupted developing relationships and left some clients ‘not happy’ (Jessie, commercial consultant).

An ideal contractual relationship?

One particular commercial company stood out in that their NHS clients were uniformly very pleased. A NHS commissioner said that the partnership with the commercial company worked well and was different from other external providers because they went at the slower pace of the NHS and did not say ‘you’ve got it all wrong’.

I think what made the difference was the specific individuals understood that we couldn’t change the whole of the NHS overnight because some of their observations about how health and social care work . . . [showed that] this is a big, enormous beast. It’s a bit like the cruise liner; you can’t turn it on a coin very quickly. And I think sometimes the management consultants, people coming externally from private organisations feel very frustrated and tied by what we’re trying to work within, and our ability to make big change quickly . . . So, I think what made it work with [commercial company] was that they were willing to go at our pace, and understand that there was some parameters we couldn’t change rather than coming in saying, ‘oh, you’ve got it all wrong’.

Jane, NHS commissioning manager

In contrast, an ex-NHS freelance consultant working with the same CCG was struggling, perhaps because he was given to challenging the client.

One of the things that I’m trying to do within the organisation, which has got nothing to do with this project, is I’ve said to Abbie and to Kevin who is the MD [managing director], ‘Your method of delivering projects isn’t working’. You know, ‘You just don’t have a project – a change of culture in the organisation, and you need to think about that, and this is how you can do it’.

Brian, freelance consultant

Not only was the approach of the successful commercial company less confrontational, but they also motivated senior leaders to shadow clinical teams to have ‘normative’ experiences with patients themselves to learn the reasons for change. Patient stories were also effective in communicating difficulties with the current system back to the organisation, as the lesson of these stories was that the organisation was delivering poor experiences and outcomes in exchange for highly expensive interventions from health and social care. In addition, the success of the commercial company was contingent on local commissioners doing the work themselves and not ‘being done to’.

Yeah I think the key point here is often with external organisations there is an expectation that they will come and do it to you . . . our approach is very much that we have expertise around the design and management of work but we have no understanding of your particular area, and it is your responsibility to improve your system, but hopefully we can help with that. But it’s about you doing it to yourself.

Daniel, commercial consultant

The example of this commercial provider seems to suggest that the key ingredients for success were going at the pace of the organisation, taking a piecemeal approach, involving senior leaders ideally through arranging encounters between senior leadership, clinical teams and patients, using patient stories to illustrate the case for change and ensuring that the client organisation does the work. Elsewhere, we found that commercial providers drew on some of these principles but not all.

In summary, the key feature of successful contractual relationships between NHS clients and external providers were:

• the providers’ ability to understand and respond to the clients’ needs and market

• flexibility as circumstances change

• facilitating activities so that clients do the work themselves and gain skills

• non-threatening, non-confrontational approach of the external provider

• working to the time frames/constraints of the clients in a piecemeal approach.

Box 12 summarises the key points in the chapter.

Commissioning managers

1. Before purchasing a tool or external support, a clear problem of importance to commissioners and/or analysts needs to be identified with a shared understanding of how the tool or external support can resolve the problem and contribute to the solution.

2. Those drawing up contracts with external providers need to fully engage from the start the staff who will be the recipients of the external expertise or expected to utilise it, ensuring that all parties understand and agree about the nature the problems being addressed and the proposed approach to their solution. Contracts with external agencies coproduced by all of the actively interested parties may have a greater chance of successful results.

3. Commissioners may need to take note of, and make better use of, the knowledge conduits and transformation processes that we have elicited (i.e. interpersonal, people placement, governance, ‘copy, adapt and paste’ and product placement).

4. Careful thought is needed when embedding individuals, considering not only skill set, experience and knowledge but also personality type. Commissioners need to be clear about what type of individuals are missing or poorly represented from their organisation.

5. With many service change and transformation projects as much focus needs to be given to implementation as to the process and deployment of the tool itself. Commissioning organisations need to acknowledge this in their plans with their knowledge exchange partners (e.g. external providers, hospital managers, clinicians, etc.) and make decisions with this in mind.

6. Commissioning organisations need to build in knowledge transfer as a key requirement to contracts with external providers, which may help to retain and develop in-house skills. This is a vital step for commissioners using external providers to take full advantage of the potential for knowledge exchange and training on offer.

Commissioning support units

1. Commissioning support units or equivalent public organisations with a mandate for ‘business intelligence’ need to develop and maintain cultures that value staff – focusing on individuals, teams and organisations in terms of drive, performance and contribution, to foster healthy conduits for knowledge exchange.

2. External providers, and particularly CSUs, need to help their commissioner clients define the scope and detail of the support they require. Having loose service-level agreements based on individuals and long-term support to programmes is unlikely to deliver the outcomes needed or sustainability for commissioners.

3. The role of commissioning analysts may need to be reconfigured to ensure that their training and activities includes interpreting and contextualising data outputs. This may help to make NHS analytics a more credible profession with their commissioner colleagues. There is also a need to upskill commissioners so that they know how to use their analysts better. Better structures and processes to encourage interpersonal interactions between the two ‘camps’, including widespread use of people placement, may be a useful way to help achieve better mutual understanding, if not actual ‘coproduction’ or ‘engaged scholarship’ in the knowledge transformations that result.

4. Commissioning support units should understand that most valuable knowledge transfer occurs via good personal relationships, and set up their structures and processes in such a way as to ensure that their staff can develop such relationships with their local clients.

5. It may also be useful to prioritise knowledge management and knowledge exchange within organisational development work, ensuring that attention is paid to this when reorganisations are planned and rolled out.

External providers

These findings are relevant for all types of external providers we encountered within the study, be they freelance, not-for-profit, commercial or CSUs.

1. Ideally, knowledge transfer drawing on the five conduits should be a core offer from external providers.

2. It would be helpful to provide interpretation of data and practical (not merely technical) assistance with tools to drive change. We found that transferring operational or technical knowledge of a tool alone is often not enough.

3. When working in the NHS, external providers might be more successful if they adapted their expectations to fit clients’ reality and negotiated mutually acceptable understandings and time frames about the use of the techniques and knowledge they are purveying. Pressing the contractor’s own values and time scales onto the client rarely yielded the desired result.

4. It would be useful if external provider staff, including CSUs, regularly asked clients to define (and redefine) the problem and intended outcome and were amenable to adapting and adjusting the project timetable accordingly.

5. Where possible, external providers could helpfully ensure that the work was carried out by or jointly with clients, so the clients took ownership, and skills were more likely to be transferred. Although this may appear to undercut the future for the external provider’s services, it is also likely to increase the chances of the outputs being trusted and useful, which may actually increase the prospects of repeat business.

6. Costing in the resources needed for genuine knowledge exchange and implementation of change might mean more contracts were successful, for example by acknowledging and allowing the time needed to build local relationships.

7. The commissioning organisations in our case studies preferred the following from external providers, who may consider making them a core part of their work with the NHS:

   1. long-term contracts/relationships

   2. continuity of workforce/relationships

   3. flexible and adaptable capacity

   4. experienced staff

   5. clinical knowledge/subject matter experts with good interpersonal skills rather than just technical or project management skills

   6. contextual understanding.

Replication

There would be some merit in repeating this work, given that it was carried out at a time of particular turmoil in the NHS, to see the extent to which those organisational exigencies affected some of the findings.

Knowledge exchange

• To what extent do clinicians and managers from primary, secondary and community care draw on the conduits and transformation processes identified in our study? What is their impact and how is that affected by different circumstances?

• Are these knowledge exchange conduits amenable to deliberate and productive manipulation and, if so, to what extent do such interventions improve the process and outcomes of knowledge exchange?

Commissioning

• Can and should NHS and commercial health-care providers be commissioned in the same way?

• What are the ‘core skills’ required for the commissioning of the health service and where should they reside?

Public Health

• What are the characteristics of public health departments that successfully meet commissioners’ needs?

Commissioning support units

• How do the role, training and skills of health-care analysts need to change to make working in CSUs an attractive option?

• How can CSUs and commercial/not-for-profit providers work together to maximise knowledge exchange?

Commercial, not-for-profit and freelance providers

• What are measurable outcomes of utilisation of commercial and not-for-profit agencies and how do they compare?

• Given that this study focused mainly on commercial providers who offered software tools, what is the contribution of commercial providers that primarily provide management consultancy?

• How successful is the ‘Lead Provider Framework’ in identifying and securing effective collaborations?