Notes
Article history
The research reported in this issue of the journal was funded by the HS&DR programme or one of its preceding programmes as project number 09/1002/37. The contractual start date was in December 2011. The final report began editorial review in February 2014 and was accepted for publication in November 2014. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The HS&DR editors and production house have tried to ensure the accuracy of the authors’ report and would like to thank the reviewers for their constructive comments on the final report document. However, they do not accept liability for damages or losses arising from material published in this report.
Declared competing interests of authors
none
Permissions
Copyright statement
© Queen’s Printer and Controller of HMSO 2015. This work was produced by Rushmer et al. under the terms of a commissioning contract issued by the Secretary of State for Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science Park, Southampton SO16 7NS, UK.
Chapter 1 Introduction
What is the problem and why should we care?
Across the world, considerable time, effort and money invested in health research are creating a better understanding of effective and efficient health interventions (the evidence base). National and international policy documents clearly reflect the need to use, where possible, the evidence base to inform effective practice. 1–5 These policy calls, to ‘close the gap’ between what we know works (the evidence base) and frontline practice, occur across multiple policy areas as diverse as health, planning and information technology (IT) development,6–9 attesting to the prevalence, complexity and persistence of this problem. Evidence from the late 1990s suggested that, on average, it takes 17 years for evidence to shape frontline practice8 and, 12 years on, despite more than a decade of translational research efforts in health-care research, the same conclusion was reached by different authors. 10 The problem is one of wasted resources (money and effort) creating an unused evidence base, suboptimal care provision (based on less effective care or poor delivery) and a moral argument that so much time and effort is expended in doing the ‘wrong thing’ to achieve ‘less than the best’ results. This seems to suggest an ideal where research evidence, policy drives and practice align to deliver optimal care, and where practice does reflect a strong evidence base, we know that outcomes can improve. 11 However, in practice, this may be simplistic. It is to address these issues that this project was undertaken.
In public health there are specific challenges to research utilisation. The very nature of public health problems is that they are complex, multifaceted, interlinked and difficult to address with simple interventions. 12 Solutions may need to address health, social and political factors, take time and require support from various stakeholders, making it difficult to establish clear evidence of effectiveness. 12 Specific difficulties are threefold.
First, the evidence base is often patchy or not proven and may be contested. 13 Key stakeholders may believe that local conditions are counterindicative. 13,14 Additionally, public health agendas span health, local government and voluntary organisations with distinct research cultures, governance and procedural processes and different ‘employee’ members. Efforts to join up public sector services to enhance coherence and co-ordination in the pursuit of ‘joined-up services’ have proved difficult. 15,16 We know that it is still particularly hard to get research information to flow over professional and organisational boundaries. 17–20 All of this suggests that gaining agreement on priorities and actions may be challenging. 14,20,21
Second, the evidence base may not address the questions that practitioners, managers and commissioners (given the exact task in hand) want answered, when they need answers: here the evidence base appears to be less than useful, untimely and largely irrelevant, as Lomas writes:
Decision makers – the patients, the care providers, the managers, and the policy makers – tend to see research as a product they can purchase from the local knowledge store, but too often it is the wrong size, needs some assembly, is on back order, and comes from last year’s fashion line.
p. 13019
Third, a range of contextual factors suggest that the evidence base may not be as objective as it is claimed. Culturally, how participants make sense of their activities22 can inform what counts as evidence (e.g. home births are considered safe in Canada but the USA favours hospital births). 23 In this instance it may be financial and legal concerns, and not research evidence, that define acceptable practice. 23 Politically, evidence emerging from research funded by powerful groups (the drug companies; legislative groups), can squeeze out opportunities to collect competing evidence. 24 Conventionally (see typical online sources that offer guidance to practitioners),25 the ‘hierarchy of evidence’ privileges quantitative, statistical randomised controlled trials (RCTs), largely ignoring plentiful observational and narrative data regarding what the public prefers or what is culturally acceptable and might work. 26 Furthermore, the introduction of evidence-based practice may be seen as an attempt to impose management control on wieldy and costly processes to standardise practice around a politically driven agenda27 rather than public choice or public need. It is not just practitioners who may feel that research evidence can be abused; researchers may feel that evidence is misused or used selectively to support favoured policy positions or practices. 28
The above illustrates that implementing an evidence base and research utilisation is a complex, dynamic process subject to social, contextual and political influence14,17,18,29 and it becomes clear that services do not fall out of policy directives in any simple, straightforward or automatic way. The research reported here was designed to be sensitive to these social processes. Two in-depth, mainly qualitative, case studies were undertaken, working in cocreation with the research participants rather than doing research on them, respecting their situated expertise and mindful of their contexts.
The contextual backdrop to this project
Two factors inexorably shaped this project as it took place. The proposal was written, funded and empirical data collected at a time of international financial downturn, resultant financial constraints and the dawning of an ‘age of austerity’ in the UK. Alongside this, the reforms to the public sector of April 2013 and the transitional period leading up to this created churn, instability and uncertainty within services trying to carry out business as usual. Some of the difficulties and disquiet surrounding the reforms was contextually ‘swept up’ with other data and our findings cannot be fully understood without reference to this troubled context. The influence of this context appears throughout the report.
The project was designed to cross-compare research utilisation across policy contexts (commissioning in England vs. joint planning in Scotland) and this remains the case (see Chapter 3). However, the parallel decision to design and operationalise this project in cocreation with co-applicants and research participants also left a deep footprint on the findings as they are reported here. In particular, in an effort to research a topic of most pressing need, salience and usefulness to our case study participants (and to secure ‘buy-in’), our case study partners picked the topic of the process case study themselves. When each picked a different topic (licensing vs. maternal alcohol consumption), this made little difference to the core conceptual design (investigating evidence use), but substantially impacted on aspects of the proposed work that needed more traditional approaches. The small health economic component of the project was largely neutered, with no like-for-like health issue to compare relative performance on; it became solely reliant (partially as envisaged) on measures of overall organisational performance. However, existing performance measures also changed during the life of the project, making it difficult to tell a longitudinal story around the organisational performance of our case study sites (see Chapter 8, Limitations of the study).
Structure of the report
This report lays out the design, operationalisation and findings of the project. The first three chapters give the background. Chapter 1 introduces the topic area and its importance; Chapter 2 reviews the literature to summarise what we already know about the topic areas, including the ‘promise’ of cocreation; and Chapter 3 outlines the methodology of the project including the ‘plan’ for cocreation, the five workstreams and the methods of data collection and analysis. The next three chapters report on the findings and are interspersed with analysis, comment and discussion. Chapter 4 reports on evidence use in the alcohol licensing process in the Scottish case study site and Chapter 5 reports on evidence use in commissioning to reduce harms related to maternal alcohol consumption in pregnancy in the English case study site. Chapter 6 reports on the findings from the national seminar and the questionnaire issued to explore how the findings from the in-depth case studies play out elsewhere. The final two chapters discuss the significance and implications of the findings, with Chapter 7 reflecting on the ‘experience’ of cocreation and Chapter 8 providing a high-level synthesis of ‘what works’ in relation to the dominant programme theory: Research evidence will enable public health functions to be met more easily. We conclude that research evidence is necessary and useful, but that it is only one part of a complex set of considerations.
Chapter 2 Literature review
Preamble
Best et al. 30 outline three phases through which the conceptualisation regarding what it takes to get evidence into practice has developed: from linear discrete models; through an appreciation of the quintessentially relational nature of the process; and on to a growing understanding of its largely unpredictable and complex nature. One conceptual framework has not replaced the others, and all three understandings, to a certain extent, coexist. We begin this short review of the extant literature with a look at the linear model of knowledge translation and its rise to favour in the evidence-based practice movement of the 1990s, using this as a springboard to explore the developing theoretical understanding of the broad area of translational research.
Methods
We searched academic literature databases: Cumulative Index to Nursing and Allied Health Literature (CINAHL); The Cochrane Library; Google Scholar; GoPubMed; Scopus; Social Science Research Network (SSRN); and Web of Science. We used (but were not limited to) the following search terms: translational research; knowledge transfer; knowledge translation; knowledge exchange; knowledge exchange and linkage; knowledge integration, evidence-based medicine (EBM); evidence-based commissioning; participatory research; action research; cocreation, co-production; complex systems; and complexity linked to health and public health. The search terms ‘research utilisation’ and ‘public health’ identified 973 papers. We cross-referenced these papers with those identified via other combinations of search terms and 729 papers appeared more than once. We hand-searched all 973 titles (and then abstracts if the titles appeared to be relevant) and 393 papers were of some relevance. We also searched for ‘alcohol’ and ‘licensing’ (availability, affordability, public health and interventions), ‘pregnancy’, and ‘alcohol-related harm’ (health care professionals’ attitudes, fetal alcohol syndrome) linked to public health and interventions. What follows is not a systematic review, nor does it limit itself to one evidence type. It draws on qualitative studies and grey literature (in particular, policy documents, professional guidelines and advice), and reports findings from quantitative controlled studies to give an overall narrative review reporting on relevant findings. In addition to this search approach we also drew on the principal investigator’s knowledge of relevant papers, authors and debates in the area to include papers from their personal archives not directly surfaced via the search terms. This is an approach advocated by Øvretveit31 as more inclusive of cognate conceptual issues and empirical findings in related, but parallel, disciplines.
Producing and getting evidence used: the linear model (push and pull)
Historically, within Western societies universities were acknowledged as ‘the first and foremost knowledge producer’. 32 Dissemination followed a top-down approach with the expectation being that, once produced, knowledge was automatically implemented. 29,33,34 Early models of EBM were based on positivist, quantitative and biomedical modes of understanding (see discussion in Blevins et al. ,33 Golden-Biddle et al. ,34 Øvretveit35 and Goldenberg36).
This process was seen as largely linear, one-way, rational and unproblematic. The aims and purpose of EBM are articulated well by Dopson et al. , who argue that EBM:
. . . is about creating a culture where practitioners automatically think in an ‘evidence’-based way every time they see a new case, where it becomes instinctive to seek out research evidence and base treatment decisions on that evidence.
p. 31737
So, if EBM and other linear models worked well, it would not simply result in the ‘push’ of evidence into practice by evidence producers but would also result in practitioners’ desire to ‘pull’ evidence into their practice for themselves. 38,39 Davies et al. 39 conceptualise knowledge transfer as including three activities: knowledge push (from evidence producers to evidence users), knowledge pull (from users) and linkage and exchange (engagement between producers and users). 19,39 Kerner et al. 40 suggest that passive diffusion is generally ineffective in changing practice, and that effective dissemination should be a push–pull process whereby those who can adopt innovations want and are receptive to them (pull) and there are, at the same time, clear efforts to assist the adopters in implementing the innovation (facilitation and support). Furthermore, they suggest that tacit knowledge based on experience drives pull while explicit knowledge from research drives push. Tetroe et al. 41 note that ‘push’ involves diffusion and attempts to disseminate information to a broad audience, ‘pull’ focuses on the needs of users and creates an interest and desire for research results and ‘linkage and exchange’ exchanges knowledge and ideas by building and maintaining relationships through which information and ideas can flow.
It became clear that EBM could not be simply a matter of applying recipes or predetermined guidelines because their appropriate application required the use of clinical expertise in addition to clinical evidence and was heavily influenced by professional and peer opinion,21 although latterly within EBM it is acknowledged that there is considerable variation in what is considered ‘evidence’ (anecdotes, clinical opinion, qualitative data, etc.)42 and not all evidence is seen as equivalent. 37,43 The dominant view of the capacity of different types of evidence to faithfully tell us about the world is reflected in the ‘hierarchy of evidence’, with RCTs at its pinnacle and, more latterly, the meta-synthesis of RCT evidence (see discussion in Lambert44 and Petticrew and Roberts45). At this point in time research evidence (and in particular research evidence produced through particular methodologies) was considered to present a privileged view of the world and offer a superior take on ‘what works’ regardless of context. These views were not without challenge, with others considering that the selection and use of specific methods and research designs was a matter of ‘horses for courses’, with differing approaches being useful for answering different research questions (RQs). 45
A changing view of evidence
The rise of EBM led to calls for such an approach to be adopted in policy and health-care management settings. 46,47 Others have argued that attempts to translate the principles of EBM into evidence-based policy have misunderstood the policy process. They argue that the policy process is already evidence based, but that it draws on a different kind of evidence:
Policy decisions incorporate evidence as to whether a policy will be implementable in practice, and whether it will be politically acceptable. Any policy which cannot meet these criteria is not worth pursuing, whatever the research evidence says.
pp. 325–637
In policy development, the social, political and economic climates in which organisations (and vested economic interests) operate may also be significant considerations. 48 Sanderson49 highlights that, in practice, the complexity and messiness of policy decision-making raises the issue of the role that value judgements play, and suggests that researchers should consider adopting a broader perspective on what counts as ‘evidence’ by investigating how ‘valid knowledge’, as well as research evidence, informs policy-making. 49 In their discussion of the role of evidence in health promotion and public health, Sanderson49 and Armstrong et al. 50 suggest that the complexity of decision-making means that the development and use of evidence is difficult and contentious. Both argue that a collaborative approach is needed for gathering and applying evidence, which requires health practitioners and academics to work across sectors.
What is developing is a growing understanding that evidence and other forms of information flow two ways, influenced by context and the prevailing social dynamics within it, and that to fully understand these issues, understanding context becomes important. The literature begins to acknowledge the way in which different types of evidence are varyingly and competitively used, perhaps with different stakeholders keen to demonstrate that they have (politically) better traction on what will actually work in situ. While Contandriopoulos et al. 51 suggest that for researchers it is the internal validity of a process that leads to ‘best evidence’ (i.e. evidence produced through a recognised research process), they question whether or not this holds the same importance for practitioners. They argue that as evidence users are exposed to diverse forms of information, only some of which has been produced through scientific methods, they are unable to weight information on the basis of internal validity alone. Tacit knowledge, personal values and experience may all carry significant weight. 50 Kothari et al. suggest that tacit knowledge is widely embedded in the planning of programmes and services in health-related areas. 52 In these contexts, practitioners may draw on both personal and professional experiences in the construction of ‘what they know about the world’ (knowledge frameworks), and tacit and professional knowledge may influence practice to a greater extent than evidence based on specific research findings.
Increasingly nuanced views of what constitutes knowledge, evidence and research utilisation appear where practitioners are no longer considered merely consumers of university-produced knowledge. 34 Knowledge produced ‘in the sterile laboratory of the isolated scientist’53 (p. S2.2) is no longer considered ‘real knowledge’53 (p. S2.2), and academic research, which rarely meets the direct needs of policy and practice,51 is often ignored. 30 Tacit knowledge, previous experience and the influence of colleagues, however, have become recognised as powerful guiders of action. 21,32,52,54–56 Boaz and Ashby57 argue that in order to facilitate the development of evidence-based practice and policy, a broader conception of research quality is needed, which includes its fitness for purpose rather than just traditional academic measures of quality. On a more pragmatic level, it has been suggested that commissioners need three types of knowledge: knowledge from research (‘evidence’), knowledge from data analysis (e.g. statistics) and knowledge from clinician and patient experiences. 58
Specific challenges of getting evidence used in public health
Getting evidence from public health research into decision-making and real-world practice is not straightforward;33 it is not the linear process it is sometimes believed to be, and may bring its own challenges. 28 This results in part from the diversity of topics, organisations and environments that form public health,59 and in part because there is little consensus around what constitutes either knowledge or research utilisation in these contexts. 51,60 The issue is further complicated by the increasing range of terms used to describe these processes. 51,61 The terms are often used interchangeably, with an oft-implied assumption that everyone has the same understanding of their meaning.
These debates attest to the view of evidence (and its use) as being not a simple, identifiable ‘thing’ (a product that is portable), but rather a dynamic process which may vary according to what evidence is being used, by whom and for what purpose. What counts as evidence on any one occasion may, in fact, be largely ‘worked up’ in the flow of the setting of its use. What counts as evidence may be what those involved can be persuaded is evidence (proof) on that occasion. 62 Research utilisation may be not simply the instrumental, goal-orientated action that the linear model suggests, but that all types of evidence may be used tactically, strategically and politically to steer action in certain directions, or stall and silence as well encourage debate, and as ammunition in securing the imposition of the desired actions of a few. 22,63
Traditional interpretations propose that barriers to research utilisation exist because researchers and practitioners occupy differing ‘communities of practice’,64 each with their own perspectives of knowledge and utilisation,32,65 where discussion and information sharing flow freely among members but where flow occurs less across community boundaries. 17,32 Each community works to its own identified time scales and priorities, and these may not be well synchronised. 19 However, the interlinked, multifaceted nature of public health issues themselves, the responsibility for which is shared across sector boundaries, may present unique and substantial barriers to evidence use. It can take many years for any health gains to be seen. Decision-makers can feel that the time scales they work to are often too short to ‘prove’ the benefits of using research findings. 66 Furthermore, practitioners have reported that longer-term planning was required in order to have a positive impact on the wider determinants of health or to enable broader societal changes, but that this was constrained by the 4-yearly political electoral cycle in which the short-term political and financial cycles were embedded. 66 The frequent NHS reorganisations also inhibited long-term planning, with high levels of change, inconsistent and disruptive work patterns, a lack of leadership and the loss of tacit knowledge and organisational memory all leading to deflected primary care trust (PCT) focus and lost momentum in relation to commissioning. 67
Greenhalgh et al. 68 suggest that politics, and the way these impact on innovation as well as power relations, can be critical for research evidence uptake. It is argued that health is a ‘political world and while evidence can be brought to the table, there is certainly no guarantee that it will be acted upon’. In light of these multiple social, relational, political and contextual influences on the use of various types of evidence, researchers have been criticised for having an unrealistic view of the importance of research evidence, ‘which is after all just one piece of a rather complicated jigsaw’ (p. 381). 69 Previously, researchers were also criticised for prioritising publication in peer-reviewed journals,20,28 seen as more relevant to other academics, over engaging users and promoting utilisation. 20,32 Criticism is similarly levelled against health professionals; although practitioner involvement in research is recognised to positively influence utilisation,70 the use of research is hampered by bureaucracy, centrally directed organisational structures and the need to provide services on time and within budget. 14,28,39,69–71 In this way, the contemplative nature of research can be considered ‘the opposite of action not the antidote for ignorance’20 (p. 129). 20 The general picture is ‘one of poorly connected worlds lacking knowledge of (and often respect for) each other’ (p. 129). 20 We might speculate that the changes introduced by the most recent research excellence framework assessment (REF 2014) to include impact case studies might prompt closer engagement across the two communities.
Although the two communities theory65 offers some insight into barriers that inhibit research utilisation, its inherent assumption – that researchers and practitioners are homogenous groups – presents an overly simplistic interpretation, in several ways especially when applied to public health. Firstly, within public health, individuals and organisations working in the same fields often come from diverse disciplines and have differing perspectives and values which influence how knowledge is selected and interpreted. 51 Similarly, given the increasing diversity, plurality and multiculturalism of contemporary society,28,40,70,71 many of the issues faced within public health, especially those within the inequalities agenda, are complex, multidimensional and defy the imposition of single or simple solutions. Stakeholders identify different causes of the problems and correspondingly see solutions in different actions. Such ‘wicked problems’72,73 are multifactorial, with any unidimensional solutions offering only ever a partial solution (the ‘problem’ simply morphs to resurface slightly differently elsewhere). For example, if laziness appears to be the cause of unemployment, forcing the idle to take up work is the solution; alternatively, if the lack of locally available jobs is seen to be cause of the problem, relocation may be the solution, or if there are no jobs, then job creation is the preferred solution. No single approach is either fully correct or incorrect, nor will it ‘solve’ unemployment forever. Weber and Khademian74 suggest that in order to tackle wicked issues such as health inequalities it is necessary to synthesise complex sets of evidence from across different disciplines and methodological divides, and to understand the process and context of interventions while using such syntheses to inform real-world decisions.
Lastly, the notion of flow of evidence across two communities ultimately rests on the prevailing metaphor of evidence as a ‘thing on a journey’, from evidence creators to evidence users, with the success of the journey being reliant on the uptake of the evidence in an instrumental way. We have seen from the above that research utilisation is not ‘recipe following’ but ‘cooking’ informed by tried and tested techniques and approaches, with mixing and blending equivalent to drawing on and working around the issues that the current context provides as a backdrop. We move on to consider the use of research evidence in different ways.
Non-instrumental use of research evidence
The Canadian Institute for Health Research (CIHR) argues that all translational research has, at its heart, the desire to accelerate the rate of change. 38 Yet rather than leading to radical shifts in practice, the impact of evidence is more likely to contribute to a general shift in thinking. This may range from major conceptual shifts or simple low-level awareness raising. The new understanding created may remain unspoken and unused, or become gradually absorbed into familiar practices rather than directly influencing new ways of working. 21,22,28 Ward et al. encourage a need for:
moving away from narrow descriptions of knowledge transfer towards a broader sociological explanation of the process, testing the adequacy of alternative models of knowledge transfer, and refining and testing tools for designing and evaluating interventions.
p. 15761
These new approaches to research and data collection would need to look for more subtle and indirect use of research evidence and the evolution of impact rather than seeking out overnight revolution as ‘proof’ of research utilisation. This, of course, takes time and presents problems in directly attributing cause-and-effect inferences to evidence use, perhaps stretching our understanding of what scientific enquiry is and how to understand and ‘measure’ impact.
Kothari et al. 75 suggest that the importance of the end-user’s underpinning value base is key to determining utilisation. Within the context of multidisciplinary working, as is typical within public health, the differing value bases of differing stakeholders may wrestle for credence in relation to a specific issue. A better understanding of the role evidence plays on values would be useful. Contandriopoulos et al. 51 state that it is the extent to which any group working around a specific issue has reached consensus regarding the issue problematisation, the importance of the problem and the measurements of success of potential solutions (how well a group works to achieve issue polarisation) that impact on research utilisation. Best et al. 30 write of ‘knowledge integration’, as opposed to research utilisation, suggesting that it is the ability of the new evidence or knowledge to become part of the culture (values and identity) of the users that influence its use. In this way researching how evidence shapes cultures is also important.
Exchanging knowledge, relational approaches and complexity
The debate has largely moved away from rational, instrumental uses of evidence to a consideration of more subtle effects of evidence as thought-shaping, (re)defining values, and shaping cultures by forming collective views on what is important and how things should be done. Many have noted that interpersonal trust promotes communication21,76 while repeated communications encourage trust. 76 Mitton et al. ,77 too, note that communication, time and timing, context, the quality of relationships and trust can be very important. Greenhalgh et al. 68 also identify that meanings can be re-evaluated and reframed, that the decisions to use research evidence should not be viewed in isolation from other factors, and that it is difficult to predict the size, nature and direction of any changes in advance. 69,78 In summary, research utilisation may accelerate change, but it may not be possible to identify in advance (and therefore control) the direction this takes.
Recent literature has emphasised the importance of interactions between researchers and practitioners75,79 and there has, increasingly, been a move towards collaborative approaches to knowledge translation and research utilisation. It has been argued that the use of a more participatory approach enables a more nuanced understanding of context, a shared enactment of facilitation and transformative leadership. 80
Researching in new ways: participatory approaches and the promise of cocreation
Collaborative research (including participatory action research) describes a partnership between researchers and practitioners to undertake research. Partnerships may be based on many forms of involvement and engagement. Each needs time and genuine resource commitments from all involved in order to succeed. 81 Participatory approaches aim to enhance both the credibility and ownership of research, addressing barriers that inhibit utilisation and allowing each party to develop a greater understanding of each other’s world. 32,39,82–84 To understand the aspirations for cocreated research, we need to refer to Van de Ven’s definition of engaged scholarship:
Engaged scholarship is [. . .] a participatory form of research for obtaining the different perspectives of key stakeholders [. . .] in studying complex problems. By involving others and leveraging their different kinds of knowledge, engaged scholarship can produce knowledge that is more penetrating and insightful than when scholars or practitioners work alone on problems.
p. 982
Collaborative research can enable greater insight into the interpretation of data3,4,82–86 by adding a contextual perspective to findings,85 supporting implementation14 and enabling the development of more focused real-world RQs. 28,85 While outwardly appearing to dissolve the barriers of cross-boundary working, collaborative research is not without its challenges. Researchers and practitioners often do approach issues from differing agendas and bring competing (and complementary) skills to partnerships. 86 Questions have been raised regarding the objectivity and potential bias of collaboratively generated research and also the viability of investing typically large amounts of time developing and sustaining partnerships. Despite the challenges and the professional and organisational compromise that is often needed to develop effective collaborative partnerships,77,87 researchers and practitioners alike report benefits. 53,82–84 Typically it is the strength of the partnership, the people involved and the working relationships developed that are key to determining effectiveness: ‘the best processes in the world are unlikely to produce results without the right people to work within (or sometimes around) them’ (p. S2.5). 53 This suggests that successful relational approaches are partially dependent on structural mechanisms to facilitate and support engagement. For example, the structures (e.g. strategic partnerships committee structures and their prescribed membership, and the softer structures such as job descriptions, roles and reporting channels) can present opportunities to engage across professional and sector boundaries, but may not. Even if these structural opportunities are present, it still takes the agency of those involved to use the role and access created to engage with and promote the use of evidence, with sensitivity to all aspects of the context.
Calls for collaborative working largely assume that practitioners wish to collaborate in the research process and the creation of evidence. However, while research and evaluation skills are key public health competencies,88 for many practitioners they are perceived as additional responsibilities within already constrained professional remits88 and are not necessarily welcomed. In reality, organisations and individuals tasked with commissioning often require support to develop capacity and capability to strive to meet such aspirations. What is needed is not only more accessible and better public health research but commissioners and public health managers able to use the research and skilled in translating it to their local setting. 28,89 In summary, cocreation and other participatory approaches may enhance both research utilisation and evidence creation in multiple ways, but equally they may be as welcome as ‘your cat bringing you a dead rat’. 35
Concluding remarks and the focus of this project
Overall, what we know is that knowledge transfer, exchange and research utilisation are embedded social processes encompassing complex, sometimes context-specific interactions. 14,17,18,51,61,76 In a cross-cutting discipline such as public health, some authors conclude that taking decisions is a balancing act:
The public health approach is not an exact science but more an art, balancing competing voices in decision-making such as the evidence of efficacy and cost effectiveness of interventions, patient demand, clinician or speciality interests, financial constraints, collaborators’ and other stakeholders’ agendas, quality standards, targets and so forth.
p. e38790
However, as yet we have few studies of how evidence is used to address live issues in situ, as the action unfolds to provide empirical examples of how the conflicting factors are pragmatically resolved and how the complexity is pragmatically closed down to achieve action in commissioning and planning decisions. As Dobbins et al. note:
The influence of: key stakeholders, organizational [sic] culture and values, individual decision-making styles, research evidence, and the importance of the decision itself have yet to be comprehensively studied and understood.
p. 15791
While many theories and models seek to explain how and why knowledge is or is not utilised,70,86 gaps still exist,86,92 and sometimes precious resources are spent on research that is never utilised. 60,61 Innvær et al. 70 suggest that much literature focuses predominantly on instrumental use of evidence, while Ward et al. 61 suggest that how knowledge is actually used is often overlooked in discussions, especially in models of knowledge translation, and little exists that identifies specific approaches that make utilisation more or less likely in particular contexts. 61,77,92 It was on addressing these gaps that our research focused. We considered that varying organisational structures (commissioning across a purchaser–provider split vs. joint planning) created by differing policy contexts would provide varying managerial mechanisms for mobilising and using evidence. In turn, we also considered that these managerial mechanisms would simultaneously affect the relationships between those involved and what could be achieved (see Chapter 3).
We now proceed to give a brief synopsis of the extant literature relating to the substantive topic areas selected by our case study partners: alcohol licensing and alcohol consumption in pregnancy. We begin with a short overview of the area of alcohol-related harms as a public health issue.
Alcohol misuse: a clinical and public health issue
In 2002, in developed countries, alcohol ranked third after smoking and hypertension as a leading cause of ill health and premature death. 93 A key indicator of alcohol harm in the population is measured by mortality due to liver disease. UK cirrhosis mortality rates have risen sharply over the past 30 years, especially in Scotland (104% increase in males, 46% for females). In England and Wales there was a 69% increase in males and 44% for females. 94 What makes this rise more poignant is that corresponding figures from the European Union (EU) over the same time period fell by 30%. 94
For some years, UK drinking patterns have appeared to be out of step with those of the rest of Europe. In the UK, consumption of alcohol has increased by roughly 50% since 1970, compared with France and Italy where consumption has more than halved over the same period. 95 In particular, the number of women who are drinking at harmful levels has increased in some areas of the UK. This has led the Scottish Government96 to report that alcohol-related mortality among Scottish women is now higher than that of English men for the first time. Substantial differences between regions in the UK are noted, with the greatest increases in alcohol-related deaths being seen in Glasgow, north-east England and north-west England, including among younger women, which have led to calls for targeted action. 97 Differences in patterns of drinking have been noted, with older, affluent women tending to drink more frequently than less affluent women. 98 The wider health, economic and social effects of increasing alcohol consumption are well documented elsewhere99,100 and efforts to reduce the harm caused by alcohol101 would result in benefits for women of childbearing age as well for as the wider population. 102
At a population level, ‘alcohol problems’ range further than alcohol dependency or alcoholism. Heavy drinking is believed to contribute to many social problems (e.g. antisocial behaviour and crime, domestic violence, drunk-driving, accidental injuries, assault, and poor street and environmental cleanliness). 103 Heavy drinkers outnumber people with alcohol dependency by 7 to 1. 104 Consequently, the greatest impact in reducing alcohol problems can be made by focusing on prevention rather than the treatment of ‘alcoholism’ (the so-called preventative paradox)105,106 and making alcohol abuse a mainstream public health issue.
Alcohol misuse interventions
In cultures of heavy drinking, the most cost-effective approaches to prevention focus on regulating the environment, particularly focusing on the price, availability and marketing of alcohol. 101,107 Typically, this requires legislative change; however, concerted community-based programmes can be effective at reducing alcohol problems in specific areas following similar strategies. 108 These approaches may include education and information campaigns, controls on selling and other regulations reducing access to alcohol (supported by surveillance and law enforcement), and may be led through community organisation and coalitions. The second most cost-effective strategy is for public sector services (particularly the NHS) to deliver systematic programmes of screening and brief intervention. 107 Here, typically, a conversation between a practitioner [most often in primary care or accident and emergency (A&E) settings] and a heavy drinker takes place. Current alcohol use is discussed and the future consequences of alcohol misuse are explored. There is a large and robust evidence base supporting brief alcohol interventions,109 and this approach is a key component of alcohol misuse prevention guidance published by the National Institute for Health and Care Excellence (NICE). 110
In addition, the Department of Health advises a co-ordinated approach to reducing alcohol-related hospital admissions, advocating what are called ‘high-impact changes’ (HICs). 102 The seven HICs have been extensively used across the NHS and local government to highlight practical measures that can be implemented at local level. Three HICs suggest how efforts can be best organised to achieve success: work in partnership; develop activities to control the impact of alcohol misuse in the community; and influence change through advocacy. The remaining four HICs suggest services, interventions and activities that can be commissioned to reduce alcohol-related harm and reduce the rate of the rise in alcohol-related admissions. These four HICs are improve the effectiveness and capacity of specialist treatment; appoint an alcohol health worker; offer brief intervention to provide more help to encourage people to drink less; and amplify national social marketing priorities. 102
Alcohol policy and licensing
Anderson111 argues that New Labour alcohol policy framed alcohol-related problems in individual terms, was developed in partnerships with the alcohol industry, ignored the social context of alcohol-related issues or problems, and has increased alcohol-related harm, whereas an effective alcohol policy should flatten trends and reduce harm. 111 In England and Wales, the Licensing Act 2003112 restricted the power of licensing to ‘the prevention of crime and disorder, public safety, the prevention of public nuisance, and the prevention of children from harm, and not with public health’. 111 The Act did not offer any licensing responsibilities for the behaviour of customers after they had left the licensed premise. Moreover, it stipulated that licensing should not interfere with the free market and its understanding of ‘need’ was attached to the commercial demand for an additional premise. 111 In Scotland, the Nicholson Committee report published a review of licensing law in 2003,113 exploring ways in which licensing legislation could be used to modify drinking cultures and binge drinking in particular. Below, we provide more detail of the Scottish licensing policy context in the light of our case study (see Chapter 4).
Licensing law in Scotland has previously been based on the 1973 Clayson Report114 and Licensing (Scotland) Act 1976,115 which had recommended longer opening hours and the option of premises seeking special dispensation for extended hours of opening. 116 At the same time, alcohol became more affordable, with a 54% reduction in real cost between 1980 and 2003117 and a 70% change in affordability between 1980 and 2010. 118 The Nicholson Committee recommended the abolition of statutory permitted hours, including those for off-licences and supermarkets, with Local Licensing Boards agreeing opening hours for specific premises and Liquor Licensing Standards Officers supervising and monitoring the system’s operation in each Licensing Board’s area. 116 It was suggested by the Nicholson Committee that Licensing Boards would have to introduce policy statements to outline both their expectations and their mode of operation. Furthermore, a Licensing Forum, with a broad membership, would review the Licensing Board’s activities and highlight any local issues or concerns. These two structures were intended to provide local accountability for the Licensing Board’s decisions and would sit under a national Licensing Forum headed by the Ministers for Justice and Health, which would review practice and advise on problems, with national guidelines for practice and training. 116 The range of allowed objections and objectors was narrow, covering nearby residents, local community councils, some officers of the local council, chief constables and health and safety officers. 116 Medical and health perspectives were, at that time, potentially absent from this, as the staff of these sectors were not generally included in the permitted objectors.
What ‘works’ in the context?
In the Scottish context, it is suggested that a range of measures are effective in alcohol policy, including regulation, early detection and interventions, treatment and support in addition to education, with controls on price and availability, drink-driving legislation and brief interventions being most effective. 117,118 Alcohol brief interventions are a key element in Scottish alcohol policy, with suggestions that these reduce alcohol consumption for up to 1 year in harmful or hazardous drinkers, and in 2008 a target was set for a total of ≈150,000 interventions by 2011, with each health board achieving an agreed number in line with Scottish Intercollegiate Guidelines Network (SIGN) guidelines. 118 It is suggested that front loading (drinking alcohol at home before going out to a licensed premises) is an increasing trend among under 25s and, consequently, more focus needs to be put on off-sales. 117
The 2005 Licensing (Scotland) Act was based on five licensing objectives: (1) preventing crime and disorder, (2) securing public safety, (3) preventing public nuisance, (4) protecting and improving public health and (5) protecting children from harm. 117 It is interesting that while public health is a key objective, the decision-making structures around Licensing Boards do not incorporate health representation. The updated report13 reviewed actions relating to protection and controls, prevention and education, provision of services and ‘getting the framework right’. There is also some discussion of the impact of alcohol and of changing alcohol cultures. Within ‘protection and controls’ intended actions are summarised in relation to licensed premises, actions in the community and action with the UK government. Intended actions in the ‘prevention and education’ category cover those relating to general information and communication, plus those targeted at schools, workplaces and the community. Intended actions relating to ‘provision of services’ are discussed at three levels: nationally, in the community and in primary and secondary care settings. The aims pertaining to ‘getting the framework right’ are discussed in relation to structures, information and evaluation and training.
Further priorities and actions are suggested in the Scottish Government’s 2008 discussion paper96 and the ensuing framework for action. 119 These documents identify targets in relation to the Scottish Government’s three overarching priority themes: ‘improving our productivity and competitiveness’, ‘increasing our labour market participation’ and ‘stimulating population growth’. Alcohol-related actions are outlined in relation to the themes of reduced consumption, supporting families and communities, ‘positive attitudes, positive choices’ and improved support and treatment. Some key actions for consideration in these reports include modifying the Licensing (Scotland) 2005 Act to end the option for off-sales premises to supply alcohol free of charge with a purchase (e.g. ‘buy one get one free’ schemes); ending reduced price sales of alcohol; introducing a minimum price per unit as a condition of both premise and occasional licences; consideration of raising the minimum age for alcohol purchases; applying a social responsibility fee; reviewing test purchasing; improving enforcement; focusing on early intervention through the Community Initiative to Reduce Violence; restricting the display of marketing material and encouraging responsible marketing; working with health and industry partners; introducing mandatory product labelling (ideally as part of a common system across the UK); establishing a working group to update core services for alcohol treatment; supporting the evaluation of brief interventions pilots; supporting NHS boards in reaching brief intervention targets; and exploring the opportunities for developing psychological therapies. The issues of minimum pricing and trade discount bans in Scotland are discussed more thoroughly in a University of Sheffield report,120 which offers a detailed conceptual and statistical discussion and models potential scenarios. At the date of writing, the Scottish Government’s efforts to introduce minimum unit pricing for alcohol is being challenged in the courts.
European policy on alcohol and licensing
While national policies are probably most significant with regard to approaches to alcohol problems and licensing, there are also intersections with European policy. The European Commission’s 2006 Communication on alcohol used research about the effectiveness of different policy measures pertaining to the reduction of alcohol consumption and harm, and identified five priorities for best practice: (1) protecting young people, children and the unborn child, (2) reducing injuries and deaths from alcohol-related road traffic accidents, (3) preventing alcohol-related harm among adults and reducing the negative impact on the workplace, (4) informing, educating and raising awareness about the impact of harmful or hazardous alcohol consumption and highlighting appropriate consumption patterns, and (5) developing and maintaining a common evidence base at EU level. 121 Guidance on advertising has also been set at the European level. It is noted by Gordon and Anderson121 and Anderson and Gual122 that the commission places significant emphasis on education, but this is the least effective alcohol policy option. Moreover, the impact of alcohol misuse on mental health and well-being has been significantly undervalued. 122 Issues such as cross-border shopping are also very relevant, and may become more so with the divergence of English and Scottish policies.
Next, we provide an overview of what is known about women’s alcohol use in pregnancy and explore some of the key themes identified to support the findings from Rosetown (see Chapter 5).
Prevalence of alcohol consumption in pregnancy
First, it is difficult to know the size of the problem. According to national surveys, around one-third of women in the UK drink more than medically recommended levels. 123,124 Estimates of alcohol consumption during pregnancy vary widely, with some national surveys estimating that 5% of women drink alcohol during pregnancy,125 compared with 54% in other studies. 126 However, estimating the amount of alcohol women drink is complicated and inexact, with considerable variation between research studies and countries127 and little standardisation about what constitutes heavy, moderate or light use. This results in imprecise definitions and difficulties of interpretation and comparison. 128 Poor understanding among the population of units and measures of alcohol129 may also lead to under-reporting. It has been noted that alcohol use, particularly in pregnancy, remains a socially stigmatised activity, and surveys of drinking behaviour in pregnancy may underestimate the true extent of alcohol consumption130 through fear of social disapproval. 131
A Swedish study found that older age, living in a major city with low social support and using tobacco during pregnancy, as well as pre-pregnancy drinking, were predictors of women’s drinking in pregnancy. 132 It has been suggested that increased awareness of the dangers of drinking in pregnancy has resulted in changes in women’s behaviour,133 but this may simply be a case of women reporting less alcohol consumption.
Effects of light to moderate drinking in pregnancy
Second, it is difficult to estimate the harms caused. There is considerable debate about the effects of light to moderate drinking in pregnancy in published literature and media reports (e.g. Taylor 2012134). A 2006 systematic review found no robust evidence of poor outcomes among women consuming moderate amounts of alcohol while pregnant,135 whereas a 2009 review from the Swedish National Institute of Public Health found impaired cognitive and socioemotional development in children aged 3–16 years in three of the six studies reviewed. 136 Advice to avoid alcohol in pregnancy is recommended following a study of the Danish National Birth Cohort, which found a strong graded association between alcohol intake and risk of miscarriage in the first 16 weeks of pregnancy. 137 In Australia, no independent effects of light to moderate drinking were found on birth weight or head circumference at 5 years of age. Similarly, a large-scale Danish study (1628 women) found no significant impact on preschool child intelligence. In the UK (Millennium Cohort Study) Kelly et al. 138 suggest there is no increased risk of behavioural or cognitive deficits at the age of 3 years for children whose mothers drank within recommended limits compared with children whose mothers did not drink. Indeed, boys born to light-drinking mothers were less likely to have conduct/hyperactivity problems, showing some apparently protective effects of alcohol, with similar findings in Australia,139 although there may be important confounding factors as light alcohol consumption is noted as a marker of relative socioeconomic advantage, which can influence children’s social and emotional behaviours. These effects were shown to continue until children reached 5 years of age. 140
Elsewhere it is also argued that the risks are not clearly established, and there is a need to understand possible risk factors mediating the relationship between drinking and outcomes. 141 Factors might include difference in drinking patterns (e.g. frequency, quantity, variability and timing) as well as absolute levels of alcohol exposure in utero,142,143 which have all been shown to affect the functioning of young children. Although the likelihood is that individual differences in alcohol metabolism protect most women, it is not possible to predict who is and is not at risk. A recent finding from a large population-based study found variants in genes involved in alcohol metabolism among children and mothers who had drunk in moderation during pregnancy, associated with lower cognitive ability in the children at the age of 8 years. 144 This suggests that in some cases even small amounts of alcohol in utero can affect future cognitive outcomes, leading some authors to conclude that we may never be able to conclusively prove whether or not there are safe levels of alcohol consumption in pregnancy, making it morally and ethically unacceptable to suggest otherwise,143–147 a message at odds with current UK guidelines.
Effects of heavy drinking in pregnancy and fetal alcohol spectrum disorder
We know that fetal alcohol spectrum disorder (FASD) is a set of conditions which are poorly understood by the general public and health-care professionals, limiting opportunities for appropriate diagnosis, prevention, early intervention and treatment. 130 Children affected by maternal alcohol consumption suffer a range of primary and secondary disabilities, the effects of which are often seen, although not well understood, by education providers and early-years practitioners. 129
Numerous studies have shown the harmful effects associated with heavy drinking in pregnancy,130 with heavy alcohol consumption increasing the risk of low birthweight and preterm birth. 148 Experts agree that there is a dose-dependent effect of alcohol on fetal and child development. 98
The term fetal alcohol syndrome was first coined in 1973 by Jones and Smith in the USA, but as research has accumulated, terms have changed, and they remain inconsistent between countries and institutions. 149 FASD is an umbrella term that covers fetal alcohol syndrome, alcohol-related neurodevelopmental disorders, alcohol-related birth defects, fetal alcohol effects and partial fetal alcohol syndrome. Symptoms include changes in facial appearance, hyperactivity, impulsivity, difficulty with abstract concepts, poor problem solving and social skills, and difficulty learning from consequences,150 leading to developmental, health, behavioural, intellectual, learning, emotional and transition-related difficulties. 129 Disabilities range from mild to very serious, and affect individuals throughout their life course.
There are currently no reliable UK data on the incidence and prevalence of FASD, as routine data are not collected, definitions differ and under-reporting is likely. 151 Estimates of the economic impact of FASD are acknowledged as scarce, with studies limited to the USA and Canada, and this calls for a standardised methodology to allow for proper comparisons across countries. 128 One study estimated that the cost of FASD annually to Canada of those affected from birth to the age of 53 years was CA$5.3B, providing a strong rationale for commissioning prevention programmes. 149
Assessment and detection of problem drinking in pregnancy
Third, there is difficulty in knowing where and when to target interventions. Pregnancy is considered to be a ‘teachable moment’, a time of increased motivation to learn and eliminate unhealthy behaviours (including excessive alcohol consumption). 127 Routine screening tools have been developed which are quick, inexpensive and shown to be effective at identifying problem drinkers in the pregnant population in Canada [e.g. the Alcohol Use Disorders Identification Test Consumption (AUDIT-C), comprising the first three questions of the full Alcohol Use Disorders Identification Test (AUDIT) tool]. 152,153 In both Australia and the Netherlands, while women found screening acceptable, midwives were reluctant to discuss alcohol with women in their care and felt that they needed additional training in asking difficult questions and in managing those who disclosed that they were continuing to drink alcohol while pregnant. 154,155 A small-scale qualitative study of the factors that influence women’s disclosure of substance use in pregnancy in Australia found that a non-judgemental rather than confrontational approach encouraged disclosure. Midwives reported a good rapport and that trusting relationships were essential; for women, direct questions, continuity of care and addressing child protection issues early and honestly all helped. 156 The booking visit when women first come into contact with antenatal services has been identified as important, alongside multidisciplinary collaboration and co-ordination of services for pregnant women, who may experience embarrassment and stigma regarding alcohol use. 157
Alternatively, given that up to half of all pregnancies are reported as unplanned,133 with many women consuming alcohol unaware that they are pregnant, alcohol consumption in the first trimester is likely to be common, suggesting that intervening at the first midwifery booking appointment may be too late. The latest national alcohol strategy158 recognises that FASD can be caused by mothers drinking before they know they are pregnant, and so preventing them is strongly linked to reducing the levels of heavy drinking in the population as a whole (universal as opposed to targeted interventions). There may be arguments in favour of continuing with both universal and targeted interventions. Although many women are known to reduce their alcohol consumption in pregnancy, a sizable minority who continue are thought to drink moderate amounts or to binge drink. 127 In a systematic review of 14 international studies, women’s pre-pregnancy alcohol consumption appeared to be consistently associated with drinking during pregnancy, alongside exposure to violence and abuse,159 leading the authors to highlight the importance of antenatal providers assessing these factors to improve the detection of women at continuing risk.
International and UK guidelines on alcohol and pregnancy
The official recommendations regarding alcohol consumption in pregnancy differ from country to country. The USA and Denmark advise pregnant women to abstain from alcohol. Since 2009, the Australian government has advised women to abstain, a move away from previous guidelines which suggest that, if they chose to drink, they should consume fewer than seven drinks per week and never more than two drinks on the same day, and never get drunk. 146
In the UK before 2008, NICE suggested that pregnant women limit alcohol consumption to no more than one standard unit per day. Currently, pregnant women and women planning a pregnancy are advised to avoid drinking alcohol in the first 3 months of pregnancy if possible, because this may be associated with an increased risk of miscarriage. 100 The guidance goes on to state that if women choose to drink, they should be advised to drink no more than one to two UK units once or twice a week. [Note: one unit equals half a pint of ordinary-strength lager or beer, or one shot (25 ml) of spirits. One small (125 ml) glass of wine is equal to 1.5 UK units.] NICE110 guidance states that although there is uncertainty regarding a safe level of alcohol consumption in pregnancy, at this low level there is no evidence of harm to the unborn baby. 110
The Department of Health’s guidelines160 endorse this approach, stating that pregnant women or women trying to conceive should preferably avoid drinking alcohol altogether. The recommendations suggest that, if they do drink, women should not exceed the limits outlined above and not get drunk, as getting drunk or binge drinking during pregnancy may be harmful to the unborn baby. After a 2012 review of the evidence base by the UK Chief Medical Officer, the government concluded that ‘current guidance adequately balances the scientific uncertainty with a precautionary approach’. 99 This view is not universally held. NHS Health Scotland161 and the British Medical Association Board of Science130 have produced more conservative guidelines, recommending that women who are pregnant, or who are considering a pregnancy, should be advised not to consume any alcohol at all, as there is no known safe level of alcohol consumption in pregnancy.
A Royal College of Midwives (RCM) guidance paper on alcohol and pregnancy162 (p. 3) recommends that midwives advise pregnant women about the risks of consuming alcohol and to avoid alcohol while pregnant and breastfeeding. The RCM encourages midwives to understand the evidence base, and adopt an individualised approach, which will enable them to discuss the implications of alcohol use. Midwives are expected to support pregnant women who continue to consume alcohol and encourage them to seek further help if abstaining from alcohol is problematic. Overall, evidence on the effects of drinking alcohol in pregnancy is contested and contradictory, resulting in different national and international guidelines. This gives conflicting and somewhat confusing messages to women and to the health professionals who provide their care.
A wider understanding of the stigma and shaming of alcohol use in pregnancy
So far, this short literature review has taken a medicalised view of alcohol consumption, largely ignoring the leisure, pleasure and cultural aspects of drinking alcohol, whereas for most people this may be the only aspect of alcohol consumption that makes sense to them. It has allowed us to focus on the research evidence of alcohol-related harms, but it is guilty of treating alcohol in the ways that sometimes ‘food’ is treated in the obesity literature – as a ‘medicine’, without reference to the pleasure, comfort and social inclusion that eating brings. 163,164 The extant public health literature views alcohol as a toxin and, in keeping with this, continuing to drink during pregnancy is considered irrational, illogical and wilful. Unless we open our understandings to see a competing view that culturally links alcohol with sociability, with ‘having a good time’ or with relaxation (its social and cultural significance) then we may miss some of the reasons why many people drink, where they drink, and why women to continue to drink in pregnancy.
A growing body of literature in the social sciences questions the ways in which pregnant women and their fetuses have become such ‘a potent focal point for regulation, monitoring and control’. 165 Concerns have been expressed about the punitive and value-laden language that is used to describe the exposure of fetuses to damaging substances and the moral assumptions which underpin these discourses,166 generating something of a moral panic. 167 Drawing on Foucauldian concepts of the neoliberal government of citizens, Lupton165 notes that pregnant women are encouraged to take ethical responsibility for themselves and their fetus by avoiding exposure to harmful toxins such as alcohol and tobacco. Failure to respond to expert advice about appropriate health promoting behaviours in pregnancy risks exposing women to critical public gaze, judgement and recriminations for lack of self-discipline and self-sacrifice. 168 Some see policy-makers who advocate abstinence without a clear evidence base as creating new definitions of risk, formalising a connection between uncertainty and danger, transforming health advice into rules ‘less connected to a balanced assessment of evidence’. 169 Empirical research in Australia found high levels of guilt and anxiety reported by women about the welfare of their children, with marked social class differences observed. 170 Interviewees were aware of the judgemental attitudes of others, including other mothers, towards their efforts to conform to the ideal of the ‘good mother’. These findings are important to note in the context of efforts to raise awareness among women of the risks of drinking in pregnancy and enable women to access appropriate services in a timely manner. It may also explain why the routinely collected self-reported data on alcohol consumption during pregnancy are viewed as unreliable (and often are not collected at all).
Chapter 3 Methodology
Overview
The study design, RQs and proposal were a collaboration effort (and jointly written) across the research team. Half of the research team members are senior practitioners in the NHS, local authority (LA) or other national public sector organisations and it was their choice to investigate public health initiatives to address alcohol-related harms. Two in-depth, largely qualitative, cross-comparison case studies (in two sites) were undertaken. Each process case study was designed to investigate research utilisation in a live, real-time process of the commissioning cycle (or joint planning in Scotland) start to finish, in relation to services or interventions to reduce alcohol-related harms. With a strong focus on collaborative working and knowledge cocreation, this study draws primarily on qualitative methods, although a small amount of basic quantitative work was undertaken to help to contextualise the qualitative data. To see if the findings were typical and applied elsewhere, a two-stage Delphi questionnaire was designed and issued to interested stakeholders (followed by the wider issue of this questionnaire; see Exploring the transferability of the case study findings: overview and Delphi questionnaire design). A national workshop was held to share and discuss the findings.
The aim, RQs, objectives, data collection methods and fieldwork details are discussed in more detail below, starting with the overarching analytical (realist) framework and the impact that concurrent public sector reforms had on the project, its design and execution.
Overarching analytical framework
The overarching analytical framework is provided by Pawson and Tilley’s notion of realistic evaluation (that mechanism plus context informs outcomes) to ask the fundamental question: ‘what works where and under what conditions?’. 171
Programme theories
Behind commissioning (and joint planning) processes are programme theories – that is to say the assumptions (or ideas) about how things work or what the active ingredients are in a process that makes it work. A key task was to identify these programme theories and look at how they played out in different contexts. For example, a programme theory may be supplying research evidence into the commissioning (or joint planning) process will ensure that research evidence is mobilised in the decision-making process to secure evidence informed commissioning (or joint planning) decisions. The realist approach asks under what conditions (where and for whom) does supplying research evidence into the commissioning process secure evidence-informed commissioning: where and how does this hold true and why? We were looking for the key components (active ingredients – sociocultural mechanisms) that made this work, or not. Central to the realist approach is an exploration of the context–mechanism–outcome interactions, and this informs the design of this project. Different policy contexts in the UK (English and Scottish health policy) require managers to operate within different managerial mechanisms (commissioning vs. joint planning) to secure public health services and interventions. The realist task is to investigate how and in what ways the interplay between context and mechanism influence the use of research (and other types of) evidence (the outcomes) in order to build mid-range theories. These theories in turn may be modified and developed through their application across additional contexts.
The key to understanding the differences will lie in exploring the extent to which different NHS contexts provide varying managerial mechanisms for effecting change. The potentially hard contractual lever of commissioning across a purchaser–provider split (or joint commissioning across the NHS and local authorities as is more common for cross-cutting public health issues) is the predominant English managerial mechanism. In Scotland, however, single outcome agreements seek to establish a shared commitment to priorities across partners at the level of each LA (with no purchaser–provider split or commissioning arrangements). A pejorative understanding of the difference may be expressed in this way: England – commissioner’s contract, provider’s work; Scotland – shared work. Within these two contexts, the mechanisms managers have available to them may strongly influence what action can, and should, be taken, as well as how that can be achieved. Each context may offer different opportunities (barriers and facilitators) for research utilisation, allowing events to play out in different ways.
A look across contexts will permit initial observations of how these things play out in practice and implications for commissioning approaches. It might allow us to see if forceful commissioning is essential to drive change but potentially damaging to long-term purchaser–provider relationships (or unduly legalistic). The predominant English managerial mechanism is the potentially hard contractual lever of commisioning across a purchaser–provider split. For public health issues this more often takes the form of joint commissioning across the NHS and local authorities.
Conceptually clearing the ground
In line with the realistic approach, it is necessary to ‘conceptually clear the ground’171 and to clarify the wider context of this research. Without wishing to open a complex debate on epistemology, we take a pragmatic approach to defining our terms; throughout the project we use the term ‘evidence’ in its widest sense to mean:
Evidence noun 1 information or signs indicating whether something is true or valid. 2 information used to establish facts in a legal investigation or acceptable as testimony in a law court. verb be of show evidence of: the city’s economic growth is evidenced by the creation of new jobs.
PHRASES in evidence noticeable; conspicuous.
ORIGIN Latin evidentia. 172
Within this definition what counts as evidence is diffuse: facts, information, signs, indications, testimony, documents and material objects may all, in the appropriate context, be considered ‘evidence’ that something is true or valid. We proceeded on the understanding that research utilisation is a socially situated activity – so that what counts as good (useful) knowledge and evidence will depend on the audience and the context – and that this in turn informs how it can be used (or abused?) and to what end. To understand events, we locate them within the context that gave them meaning and made them possible. We adopt this diffuse definition of evidence, as it is the one routinely used by our research participants to inexorably shape their understandings of what counts.
However, where useful, we contrast this lay understanding of ‘evidence’ with ‘research evidence’, to explore the issues raised when an externally coded evidence-base that is widely accepted in rational western society as a privileged way of understanding the world attempts to trump lay and situated understandings of what counts in a given context. By ‘research evidence’ we mean the available body of facts or information indicating whether a belief or proposition is true or valid as it results from the research process and as analysed through recognised methods. That is:
. . . the systematic investigation into and study of materials and sources in order to establish facts and reach new conclusions. 173
To focus the research gaze within the process case studies (and avoid looking too widely at all types of evidence used in all parts of the process), we concentrated on the times and places where overt use was made of research evidence (and other types of knowledge). For other definitions see the Glossary. 38
The context: researching in a changing public health landscape
The fieldwork for this project (January 2012 to November 2013) coincided with the public sector reforms introduced by the Health and Social Care Act 2012. 174 During this period, public health commissioning in England moved from the NHS to LA control. Between 2012 and 2013, public commissioners (i.e. our research participants) were in transitionary arrangements, and after April 2013 they became LA employees, although some [e.g. the Director of Public Health (DPH)] had already held a joint NHS/LA post since 2007. For much of this time (and in the contracting period before the project officially began) the precise details of the reforms, new organisational structures, and funding arrangements, including ring-fenced funding for public health services, were still being negotiated. It was unclear what public health services and interventions would remain in place post reforms. In addition, the global financial downturn and the resultant public sector financial constraints introduced by the 2010 UK government formed a much wider backdrop of uncertainty regarding the nature and longevity of public health provision, especially given that funding to the NHS was protected, whereas LAs had been subject to substantial reductions in funding.
To cope with this level of uncertainty and to take what steps were possible, at that time, to ‘future-proof’ the project to ensure that it was not derailed by as-yet-unseen changes resulting from the reforms, the research team decided that:
-
The data collection would ‘follow-the-action’175 and not study case study sites (i.e. physical entities which were being dissolved) but follow the process of commissioning (or joint planning) wherever the action was taking place. In accordance with this, as public health moved to the LA, the project moved with it and carried on.
-
To ensure that the commissioning process around the intervention or service being tracked would still exist post reforms, a high profile public health issue was identified. It was decided that reducing alcohol-related harms was likely to remain a priority concern regardless of other organisational or political priorities.
These changes were incorporated into the project design and the project protocol. The case studies became process case studies to follow research utilisation (and use of other types of knowledge) in public health commissioning (or joint planning) to reduce alcohol-related harms.
Design
A qualitative methodology, to focus on the social and situated process of evidence use and the meanings and significance this holds for participants, was selected. Two interpretive, in-depth, multimethod, cross-comparison process case studies (one in each of two sites) were undertaken. 176,177
Key aim
To work collaboratively with research participants to explore and understand how research is utilised and knowledge mobilised by NHS and LA managers (and others) in the commissioning and planning of public health services to reduce alcohol-related harms.
As an exercise in the cocreation of knowledge,82 it is important for us to be able to show how we worked with our research participants to achieve our aim. In order to map this out clearly, we have added details of the ‘research engagement process’ to the customary study aim, RQs and key objectives.
Research questions
-
How, when, where and by whom is research utilised and are other forms of knowledge mobilised in the commissioning and planning of public health services? What is the perceived impact?
-
What is involved in working collaboratively with research participants to cocreate knowledge?
-
What are the individual and organisational factors that support or hinder research use in the commissioning and planning of services?
Key objectives
-
To engage research participants in the research process (cocreation of knowledge) (meets RQs 1, 2 and 3).
-
To track the commissioning process (and joint planning) in public health for research utilisation (meets RQs 1 and 3).
-
To identify knowledge management roles (and the opportunities and challenges posed by these roles) (meets RQs 1 and 3).
-
To explore individual and organisational learning around research utilisation (embedding and sustaining research utilisation) (meets RQs 1, 2 and 3).
-
To explore any link between research utilisation and organisational performance (the impact) (meets RQs 1 and 3).
-
To explore and discuss transferability of the findings (meets RQs 1, 2 and 3).
Research engagement process
-
To negotiate entry, identify mutual RQs within the research brief and agree the process case study178 (meets RQs 1, 2 and 3).
-
To engage participants in the research process (e.g. to allow them to contribute to recruiting research participants, data collection and write-up of results if they so wish) (meets RQ 2).
-
To hold bimonthly (or as otherwise agreed) feedback and joint data interpretation meetings (meets RQs 1, 2 and 3).
-
To engage a wider set of stakeholders in the interpretation of the findings via local workshops, a modified two-stage Delphi process, the UK national workshop, and the dissemination of results through professional and policy networks (meets RQs 1, 2 and 3).
Operationalising the project
A study designed to cocreate knowledge
In order to ensure at least a minimal degree of collaborative working we required two things from our participating sites:
-
That the sites were keen to take part, and that they picked the specific topic (the commissioning or joint planning cycles we were to follow within the study parameters, that is to reduce alcohol-related harms). This devolved decision-making was to help to ensure local buy-in, to ensure that the research addressed issues that were timely, useful and relevant to services needs, and to show our commitment to work together on areas of mutual interest.
-
That the sites take part in regular feedback and data review sessions to encourage joint interpretation of the data and its importance, and to explore possible iterative responses to it (changing practice). The establishment of a joint interpretation group on each site would allow collective reflection on the findings in terms of accuracy, completeness and usefulness and identify next steps for the research. These reflections also informed recruitment of respondents and questions for the follow-up interviews.
These two requirements formed the core conditions of cocreation. If, at the point of negotiated entry, our proposed participants were not able to offer to try and meet these, then we would be unable to proceed working with that site. In addition to this, beyond these core requirements, we offered our participants the opportunity to add extra RQs and to participate in data collection and report writing, if they so wished. In forming these working patterns we hoped to build strong relational ties with our collaborators, permitting a deeper level of trust, and leading not only to better access (and, therefore, to richer data), but also to a greater understanding of what it takes to work collaboratively across the academe–practice boundary. An extended period of negotiated entry (4 months) was built into the project time scale to accommodate the needs of the cocreation process. Documents explaining the cocreation approach were produced and shared to support the entry process.
Implications of allowing the case study sites to pick the commissioning topic
We anticipated that there would be two main implications of allowing the sites to select their own topic for the case study. Firstly, there would be an initial period of uncertainty; that is, only after agreeing the specific commissioning/planning process would it become clear who the specific stakeholders were (e.g. for interviews), or what the most appropriate routine data sets to consult for the quantitative impact analysis would be. We had considered that this would be a temporary state for which a carefully negotiated entry process would help to clarify and establish the final study parameters.
Secondly, across our case studies we accepted that we would be unlikely to be comparing like-with-like cases in the traditional sense (comparing the same public health intervention). We therefore constructed our sampling strategy not to allow comparison of the topic area or health outcomes but to contrast the managerial mechanism for enabling the use of evidence.
Sampling
The focus of comparison across our two process case studies became an identification of how research evidence (and other types of knowledge) was being used under either commissioning (across a purchaser–provider split) or joint planning across unified organisational arrangements. Therefore, a case study in England (purchaser–provider split) and Scotland (joint planning) was selected. The varying English/Scottish contexts provide a natural experiment across which to sample these differences. Our sampling strategy can be seen in Table 1.
| Context | Case study process | Case study site |
|---|---|---|
| Managerial mechanism | Commissioning with purchaser–provider split | English PCT in transitionary arrangements with the local HWB (note: latterly with the LA) |
| Joint planning (no purchaser–provider split, no commissioning) | Scottish Health Board with partners organisations (note: ultimately within the LA) |
Selection of sites
After careful discussion between the research team and the advisory group both the English and the Scottish site were identified by their similar population profiles (large, post-industrial urban conurbations). Previous nationally funded public health research had been undertaken across these locations, and we sought to add to this evidence base. In both sites the DPH was known to members of the research team, and this facilitated the initial approach. The sites themselves were keen to take part.
Work streams and methods
Data were gathered through five distinct strands of work. These are listed below (i–v) with the specific methods associated with them.
Two in-depth process case studies
The first case study was undertaken in an English site (Rosetown), with the process case177 following research utilisation (and use of other types of knowledge) in a commissioning process to address alcohol-related harms. The second process case was undertaken in a Scottish case study site (Thistletown) following research utilisation in a joint planning process across unified organisational arrangements, again to address alcohol-related harms.
(i) In-depth qualitative interviews with those carrying out the commissioning or joint planning to explore research utilisation: the rationale, opportunities, challenges and perceived impact of the process at operational, tactical and strategic levels. Repeat interviews (with those directly involved) after milestone events to gather developing views.
Observation of commissioning (or joint planning) meetings: to observe live (as opposed to reported) research utilisation and the mobilisation of other types of evidence.
Documentary analysis of the written documents that record process activity, for example minutes, commissioning (joint planning) agreements and strategic documents.
(ii) In-depth qualitative interviews with those holding a knowledge management role within the two case study processes (i.e. those who gather, synthesise and supply evidence into the process) to investigate the opportunities and challenges posed by their roles.
Interviewees were recruited via purposive sampling (those directly involved in the commissioning or joint planning process under study) and via snowball sampling (additional people identified as active agents in the process by these initial interviewees, including those people holding a knowledge management role). Interviews followed a semistructured topic guide (see Appendices 1 and 4); however, content was iterative, as both data collection and data analysis took place concurrently and followed the method of constant comparison,179 with data analysis from earlier interviews informing topics discussed within later interviews. In this way the interviewers were able to increasingly hone in on issues of key interest.
All interviews were digitally recorded and transcribed verbatim (by an external agency at another location in the UK to preserve anonymity). All identifying details were removed at the point of transcription. A coded list linking participants to their transcripts was kept so that first interviews could be linked to subsequent interviews. Following transcription, the digital recordings were erased. Electronic and hard copies of the transcriptions have been retained in accordance with Teesside University Ethics Committee policies and procedures.
(iii) ‘Impact’ analysis: an impact assessment in the strict understanding of the technique was not possible in the time scales of this project, nor was a cross-comparison of ‘performance’ on the same public health issue across the sites (except on the broadest level). Our aspiration was to look at measures of overall organisational performance to see if there was any reason for considering these organisations to be typical (or outliers) in what they are able to achieve. Although the data would not establish causal links, they might have enabled us to see whether the organisations are good, fair or failing to help in the interpretation of the qualitative findings and provide a way of locating the significance of what was found – does its evidence use show up in the larger measures? Does it not make any discernible difference? (See Chapter 8.)
The plans for quantitative data gathering are given below in Tables 2 and 3, which identify the data sources scoped out in the two specific topic areas. The results of these searches and a more full account of the quantitative work (largely from secondary reports) can be seen in Appendices 9 and 10, and are explained in Chapter 8.
| Data source | Data domains | Data features |
|---|---|---|
| NHS Scotland | Alcohol-related admissions and deaths by intermediate zone and CHP/sector | Cross-sectional data 2011 and 2012 |
| Health and Well-being Survey | Alcohol consumption levels | |
| The Scottish Public Health Observatory (ScotPho) | Scottish Health Survey (2012)180 | Self-reported answers to annual questionnaire |
| Alcohol Profiles | Epidemiological data comparing local alcohol profiles with Scottish national averages | |
| Information Services Division Scotland | Alcohol licensing statistics by area | Personal and premises licences granted and refused 2010–11 and 2011–12 |
| Literature | Various peer-reviewed academic papers and evaluation reports |
| Data source | Data domains | Data features |
|---|---|---|
| Local Alcohol Profiles for England (LAPE): Rosetown Public Health Observatory |
|
Annual Trend data 2002–12, comparator cities, comparator local areas, comparator PCTs and LAs 2008–13 |
| Rosetown LA/PCT data | HRG inpatient data | 035.4 Suspected damage to the fetus related to alcohol P04.3 Fetus and newborn affected by maternal alcohol use Q86.0 Fetal alcohol syndrome |
| Rosetown Integrated Commissioning for Addiction and Offender Health | As above | |
| Rosetown University Centre for Public Health | Monitoring data for the commissioning group | |
| Rosetown hospital | A&E data | No data |
| Rosetown Mental Health Trust | Inpatient data Psychiatric liaison Community teams (including monitoring data from Rosetown Community Alcohol) Service |
Drugs only/detox figures for alcohol |
| Rosetown ambulance service | Alcohol-related calls | No data |
| Rosetown maternity hospital (foundation trust) | Sociodemographic data on women during pregnancy and a record of reported alcohol consumption from antenatal booking form | Total number of births per annum, local data on fetal alcohol syndrome, numbers of women known to have alcohol/substance misuse issues in contact with midwives. Booking question numbers |
| National Drug Treatment Monitoring System with Manchester University’s NDEC | No data |
The data sources identified for the overall gauge of ‘organisational quality’ were to be performance on world-class commissioning measures (and similar performance measures across the Scottish site in a shared baseline data set) and patient-reported outcome measures. These measures were to be used to give an indication of the overall standing of the organisations we were researching. We wanted to know if they were typical, or if there was any reason for thinking that they were in any way different (better or worse) than similar comparable organisations. Would the qualitative case study data shed any light on overarching quantitative differences?
Exploring the transferability of the case study findings: overview
The design of this study permitted an in-depth, situated and rich account of research use (and other types of evidence) in one commissioning and one joint planning process, at a specific point in time. It was important to understand to what extent the findings from the two case study processes applied in other contexts. As a first step, local feedback workshops allowed us to check our interpretation of the findings with local stakeholders. These stakeholders included commissioners in other public health topic areas locally, to see if the findings applied in other areas of commissioning locally. Second, a modified Delphi process and an interactive workshop were designed to enable us to ‘road-test’ the findings to explore their relevance to other settings, and gain insight into whether the case study findings were context specific or transferable to other settings. (Note: we detail the aspirations for the Delphi process, but also include details of the steps we took when the response rate to stage 1 of the Delphi was very low.)
(iv) Modified two-stage Delphi process using a Behaviourally Anchored Rating Scale (BARS) questionnaire (see below): 10 of the main findings emerging from the case study processes (that illustrated similarities as well as differences across the cases – nomothetic and idiographic data) were selected for inclusion in the Delphi process questionnaire. One other question on cocreation was included. A 9-point BARS questionnaire was used (described in more detail below and in Chapter 6, including the modifications to the proposed Delphi process). The first round of the Delphi was issued with the invitation to the national workshop.
(v) The interactive dissemination workshop: this offered the opportunity to both disseminate and road-test emergent findings and to explore transferability of the findings through discussion and debate. The workshop invited those with an interest in alcohol misuse and related issues – practitioners, managers, commissioners and those from the voluntary sector – to attend. Results of round 1 of the Delphi questionnaire prompted debate to get ‘expert/stakeholder’ comment on the emergent findings. A second-round Delphi vote was taken. The data from the modified Delphi and field notes on the workshop, detailing the issues raised in the discussion, are incorporated into the findings of this report. As an additional unforeseen step (because numbers attending the national workshop were so small), we issued the BARS questionnaire from the Delphi process to a wider national audience to boost the numbers (as a one-stage process). This allowed us to gather the views on the applicability of the 10 main findings more widely. We analyse this questionnaire (as outlined in the section below) initially as a two-stage Delphi process (but because the numbers are so small this is only included in the appendices for interest) and a one-stage questionnaire (using exactly the same questionnaire, based on the main 10 emergent findings) to test the applicability of the views more widely. These responses form the bulk of the analysis in Chapter 6.
Delphi questionnaire design
A modified Delphi process with a questionnaire using a BARS questionnaire was used to road-test the findings. The Delphi process itself was designed to maximise the stakeholder input and expertise to the debate without incurring any negative face-to-face group influences (i.e. peer-group pressure, minority influence). 178 A questionnaire is issued to ‘experts’ (often geographically dispersed) for their views, feedback on the results is given, least popular answers are removed, and second, and possibly subsequent, questionnaires are issued.
Behaviourally Anchored Rating Scales are a form of Likert scale first used in organisational psychology,178,182–184 typically in instruments designed to standardise descriptions of employee performance appraisal (i.e. what would an ‘excellent’ performance as opposed to a ‘poor’ or ‘average’ performance actually look like). As such, they were said to help to ‘ground’ the scale in actual (recognisable and meaningful) behavioural descriptors to add clarity and enhance the validity and acceptability of the instrument. 178,183–186
The aim in modifying the BARS questionnaire and in creating the anchors for the questionnaire was, as far as possible, to use the terminology of the respondents and illustrate the choices (dilemmas?) they face in their daily practice as decision-makers trying to use evidence. We provided competing behavioural anchors at either end of the scale (that we recognised from our data) to tease out the conflicting pull of each choice across the issue (see Table 10). Where respondents placed their ‘X’ represents for them the resolution between the two competing anchors. Examples of a similar method being used in health-care research can be found in Rushmer et al. ,56 Guthrie et al. ,187 NHS Research Authority188 and Ritchie and Spencer. 189
Respondents were also asked to indicate on a 3-point scale, ranging from not important to very important, how important they thought each pair of statements was. In addition, the following bibliographic data were collected: job title, organisation name and work location. This was done to enable comparison across different groups of respondents (e.g. NHS employees vs. LA staff). The instrument was piloted with a small number of researchers for readability and ease of use. (For the full questionnaire, including rubric, see Appendix 1.)
Research governance, project governance and patient and public involvement
Ethics committee approval was granted by Teesside University. In consultation with the local Integrated Research Application System (IRAS) committee and in accordance with the Governance Arrangement for Research Ethics Committees (GAfREC) agreement,190 it was agreed that full IRAS committee approval was not needed, as only professional staff were taking part in the research, in line with their professional roles. In Rosetown, research and development (R&D) approval was gained from six NHS trusts, and in Scotland R&D approval was granted from the host health board. No additional research approvals were needed for engagement with the LAs in either of the two case study sites.
Governance for the project was provided by an advisory group consisting of two external academic members: a commissioning lead for alcohol and the public health advisor for Voluntary Organisations North East Network (VONNE). This member was able to provide a public voice and comment on the project and interim findings from a voluntary and community sector perspective.
Data collection
The Scottish case study process
Cocreation: negotiated entry and selecting the process case study
Discussions began within Thistletown in mid-2011 and were initiated by the co-applicants, more than one of whom knew key stakeholders in the site in their professional capacity. An initial face-to-face meeting with the DPH, head of service improvement and head of information services introduced the team and the study (aims, objectives, RQs, etc.) and introduced the notion of cocreation and the opportunity for the site to choose the topic that they wished the research to address (in relation to joint planning for the reduction of alcohol-related harms). The opportunity to be active in the research process and the necessity of establishing the joint interpretation group (or, as they called themselves, our ‘adoptive family’) was raised. The resulting discussion clarified the nature of the project and stakeholders were left to consider the topic they wished to select. A second meeting (supported by intervening e-mail correspondence and shared documents (e.g. the lay summary, etc.) took place in early 2012. In addition to the original stakeholders, people attended who represented public health, social care and the corporate planning office of the LA. At that meeting it was formally decided (with input from the health board, LA and the research team) that the project would focus on research utilisation in the alcohol licensing process. We further reflect on this period of negotiated entry and the choice of case study process in Chapter 7.
The time we had spent in establishing relationships with public health staff at the health board was lengthy, but not wasted; however, identifying the new stakeholders who were involved in the alcohol licensing process took time. Then we began a second period of negotiated entry, this time with the LA. An initial meeting in spring 2012 was set up with representation between the LA’s corporate planning office, the elected member (who served as chairperson of the Local Licensing Board) and a legal representative of the council. Once again, details of the project, cocreation aspirations and entry requirements were all shared and discussed. The legal nature of the process and the limitations it created for access to information was clarified. The principal investigator agreed not to issue a ‘freedom of information’ request to draw down further information in support of the project (beyond the data collected through the data collection process). In consultation with the co-applicants and the advisory group, it was agreed to proceed with this case study process.
Fieldwork
Fieldwork began in February 2012, with periodic updating until September 2013. Tables 4 and 5 identify the interviewees by role. Table 6 gives an overall summary of all the research activity in both sites.
| Number of interviewees | Category |
|---|---|
| 5 | Health |
| 5 | Legal |
| 9 | LA |
| 8 | Other |
| Number of interviewees | Category |
|---|---|
| 1 | Elected member |
| 3 | Commissioners |
| 6 | Clinicians/service providers in primary and secondary care |
| 7 | Public health |
| Research process | Thistletown | Rosetown | Totals |
|---|---|---|---|
| Negotiated entry meetings | 3 | 2 | |
| Meetings with the joint interpretation group | 4 | 3 | |
| Scoping interviews | 4 | 14 | |
| Interviews | 15, plus five research managers | 17, plus six research managers | |
| Focus groups | 2 (n = 5 and n = 6) | – | |
| Follow-up interviews | – | 5 | |
| Observations | 6 | – | |
| Local dissemination and shared data interpretation workshop | 1 (June 2013) | 1 (May 2013) | |
| Delphi questionnaire | |||
| First round (1 August to 4 November 2013) | 34 | 34 | |
| Second round (5 November 2013) at workshop | (10) | ||
| Extra first-round replies to end of November 2013 | 39 | 73 | |
Observations
Observations were made of the Local Licensing Board and Local Licensing Forum meetings. As these meetings are open-access public meetings, Thistletown (LA) specified that no formal consent was needed from those attending in order for the research team to observe. However, without formal consent, we have not included direct quotations from the meetings, but have elected to use this information to contextualise our findings and aid our interpretation of them. The researcher attending meetings kept detailed observation notes and reflections within a research diary.
Documentary data
In Thistletown the research team had access to previous minutes, agendas and documents presented at Licensing Forum meetings and Licensing Board meeting documents, all of which are publicly available on the council website. The team also had access to Thistletown Licensing Board policy statement (2010; full reference withheld) and overprovision statement (2010; full reference withheld) and, as presented within Licensing Forum meetings, the draft Licensing Board policy statement (2013) and draft overprovision statement (2013; full reference withheld) presented during pre-consultation.
The English case study process
Cocreation: negotiated entry and selecting the process topic
Initial discussion began with Rosetown in December 2011, following e-mail introductions. Two on-site meetings with the DPH, associate DPH (health intelligence), associate DPH (health outcomes), strategic lead for alcohol and one alcohol commissioner from Rosetown allowed the research team (principal investigators and research assistants) to explain the project and its approach to cocreation and to answer any questions. A local joint interpretation group, which our research partners named the alcohol and pregnancy research group, held meetings in March and August 2012 and May 2013. At the first of these meetings, our research partners decided that the alcohol topic they wanted to focus on was reducing maternal alcohol consumption before, during and after pregnancy. This included improving guidance and education, identification and screening for alcohol (mis)use during pregnancy and improving access to alcohol services. In discussions at this meeting, the under-reporting of fetal alcohol syndrome and the impact of alcohol on children and families were highlighted as important local issues. This fitted with the strategic priorities identified by the Health and Well-Being Board (HWB), and an awareness of the need for targeted work with children, young people and families. Clearly this topic was wide ranging and our case participants asked that we focus on their commissioning of some local insight work (primary data collection following a market segmentation approach) to gauge the views of local women and midwives on the acceptability and feasibility of interventions to address maternal alcohol consumption during pregnancy. The decision to focus on this topic was ratified by the project advisory group.
Fieldwork
Fourteen scoping interviews were needed to map out the organisations and systems involved in the chosen commissioning process. These were undertaken between May and September 2012. It became clear that stakeholders were spread over several NHS and LA organisations (commissioners and providers), and this was reflected in our need to gain R&D clearance from six NHS trusts (see Tables 4–6).
Observations
Two observations of the Alcohol Strategy Group were undertaken in 2012 and 2013, one of an alcohol and pregnancy workshop in April 2012, one of a Health and Well-being Strategy workshop in May 2012, and one of a celebration event for alcohol advice and brief intervention work in early 2013. Observation of one Alcohol Strategy Group meeting was missed owing to severe weather conditions in late 2012. In November 2013 a meeting of the Insight team (the commissioned market research organisation to report on findings from the commissioned process) was also observed. Written, informed consent was obtained from all of those attending each meeting we observed. Two of the planned meetings of the Alcohol Strategy Group and one of the alcohol and pregnancy research group meeting were cancelled as participants gave their apologies citing pressures created by the public sector reforms and new organisational arrangements that made attendance impossible.
Documentary data
In Rosetown the research team had access to the local Alcohol Strategy Group minutes 2011–13; the Health and Well-being Strategy 2012–15; the Joint Strategic Needs Assessment 2011; the LA and PCT Alcohol Strategy 2011–14; the Insight Team Service Evaluation specification (2011); local health inequalities and local Public Health Observatory data; a university partner’s research; and local professional newsletters (full details withheld).
Research managers
Research managers (i.e. those people whose role it is to gather, collate and supply evidence to the case study process, or perhaps to create that evidence) across the two sites worked in a range of job positions across sectors, including:
-
substance misuse specialists and research programme managers based at local universities
-
heads of library services/specialist librarians and public health consultants based in the NHS
-
public health intelligence managers, strategic policy officers and senior marketing executives within city councils
-
operations or information managers within strategic partner organisations, such as crime and safety partnerships.
Data analysis
Qualitative data
All qualitative data were entered into NVivo 10 (QSR International, Warrington, UK). Using a focused framework analysis approach191 but also being open to emergent and unexpected themes and issues, the research assistants met frequently to discuss the programme theories, emerging themes, categories and competing interpretations (as per Geertz’s thin and thick descriptions192) and to foster inter-rater reliability. Early analysis was further tested in fortnightly meetings with the principal investigator and in other meetings with members of the research team, and consensus was reached. Overall, this built in multiple chances to identify known issues, explore newly emerging findings and look for generic and transferable messages. It also allowed the team to identify when data saturation had been reached.
Quantitative data
The quantitative data were managed within Excel (Microsoft Corporation, Redmond, WA, USA) and SPSS (IBM Corporation, Armonk, NY, USA) (as appropriate) after being cleaned.
Data and analytical synthesis
Each data stream (case studies, interviews, observations, documentary analysis, two-stage modified Delphi survey, national interactive seminar) was subject to its own analysis and this yielded varying (but partial) perspectives on research utilisation in the commissioning (or joint planning) process. To synthesise and combine data from these multiple sources we conducted a thematic high-level ‘read across’ to identify recurrent themes and commonalties across the case studies (and other data sources) as well as to highlight places where managers’ use of research varied (within that context) and to explore why. An analysis of the programme theories permitted the testing of the assumptions behind how research utilisation ‘works’.
Chapter 4 The Scottish case study: evidence use in the alcohol licensing process
Background and context
Alcohol use is recognised as a key feature within Scottish culture;119 it is also a key problem. 193 Evidence identifies that, in recent years, levels of alcohol use among the population and levels of use above recommended limits have increased, as have levels of alcohol-related harms. 193–195
Alcohol is associated with health harms that can occur in the immediate short term, such as intoxication, accidents, poisoning and increased vulnerability. It is also associated with poor mental health and is a key risk factor for many chronic long-term conditions. 194,196 Although many regular drinkers experience no noticeable problems in the short term, a number of alcohol-related health conditions are complex and can develop unnoticed over time.
In Scotland, liver disease is growing at one of the fastest rates in the world, while alcohol-related mortality has doubled within the last 15 years119,196 and 1 in 20 deaths results from alcohol-related conditions. 195 The personal, social and economic costs of alcohol to Scotland are estimated at over £7B. 194
Alcohol misuse does not only affect a minority of the population; it impacts across society. 119 It is ‘no longer a marginal problem, with up to 50% of men and up to 30% of women across Scotland exceeding recommended weekly guidelines’ (p. 7). 119 The scale of this issue could be broader, as people underestimate their alcohol consumption and self-reported data are typically under-reported. 119,193
Increasing levels of population alcohol use, the associated health, social and economic harms coupled with continuing high levels of alcohol-related hospital admissions, and mortality rates present key public health challenges for Scotland. 197 The increased availability and affordability of alcohol and the normalisation of regular drinking and home drinking are factors which have contributed to this increase in population consumption. 121,198,199 Alcohol was 66% more affordable in 2009 than in 1987. 196 In recent years alcohol purchases from off-sales, often purchased from supermarkets at lower prices, and home drinking have increased, and there has been an increasing concentration of drinking venues in city centres. 200
Some authors suggest that the harms caused by alcohol can be curbed only if population alcohol consumption is addressed, with measures introduced to address the affordability and availability of alcohol. 194 In an attempt to achieve lasting social and cultural change, the Scottish Government has consciously adopted a population-level approach alongside targeted interventions for specific high-risk and vulnerable groups. 119,121 At the time of writing, Scotland’s decision to introduce a nationwide policy on minimum unit pricing of alcohol is being legally challenged by the Scotch Whisky Association.
Nationally and internationally, the research evidence is extensive: the density of alcohol outlets, extension of hours of sale for both on- and off-trade, and changes in drinking patterns, combined with the extensive advertising and promotion of alcohol, have been linked with increased consumption and associated harms. 199 Mobilising and using this evidence effectively to inform licensing decisions offers the opportunity to restrict alcohol availability and address public harm. For example, by controlling the overall numbers of premises, hours of opening and business operations, alcohol licensing is an example of a population-level approach that addresses the availability and affordability of alcohol. 197 Yet as we will illustrate, public health faces a key challenge in effectively applying what is largely population-level evidence at the localised level needed for licensing decision-making,197,198 amid a backdrop of powerful vested interest, differing notions of ‘well-being’ and the constraints of legal process.
Public health involvement in alcohol licensing is supported by legislation in Scotland. 201–203 The public health licensing objective was introduced in the Licensing (Scotland) Act (2005). 201 This objective was designed to enable public health evidence to enter the consultation process and be considered in licensing decision-making alongside other evidence. This chapter presents findings on the use of public health evidence within the alcohol licensing decision-making process in the Scottish case study site (Thistletown).
Policy context: the process of joint planning for alcohol services
In Scotland, responsibility for health and for NHS services is a devolved matter. The Scottish Government sets national objectives and priorities for the NHS in Scotland, signs delivery plans with each NHS board, monitors performance and supports boards to ensure achievement of key objectives. NHS boards work closely with their partners, including patients, staff, local authorities and the voluntary sector, to deliver services and to safeguard and improve the health of their residents. Since devolution there has been strategic development in support for the principle of the NHS and LAs working together in the design and delivery of services. This is embodied in the development of Community Health Partnerships (CHPs) in the early 2000s but the principle also extends to other complex and interconnected public health issues, such as health inequalities and alcohol.
At local level, CHPs [or Community Health and Social Care Partnerships – CH(C)Ps] manage a range of local services delivered in health centres and clinics, the community and homes across Scotland. The inception was a response to the challenge of:
. . . improving joint working to deliver modern and effective person-centred services, to identify options for charging for home-based care, and sharing good practice. The focus was initially to be on older people, but eventually to move on to other client groups, including children. 204
Currently, these are formal structures that ensure close involvement of LAs, patients and the public. These structures support the integration of health and social care planning and delivery. They also allow a co-ordinated focus of prevention at local area levels.
The principle of aligning planning and delivery between LAs, the NHS and other local partners is embodied in the establishment of Alcohol and Drug Partnerships (ADPs). The National Framework for Alcohol and Drug Partnerships (2009)205 set out the new arrangements for the strategic governance of alcohol and drugs services across Scotland. The framework provides an emphasis on joint working across a wide range of agencies involved in tackling alcohol and drugs issues at a LA level. In particular, there is a focus on ensuring participation from partner agencies to ensure that there is commitment to taking forward the strategic direction of the development of alcohol and drug-related services. The ADP is the structure that allows this, embedded within LA Community Planning arrangements; they involve not only the local CH(C)Ps but also community groups and organisations, education, licensing boards, social work, the police, registered social landlords, fire services and voluntary organisations.
Alcohol and Drug Partnerships develop local strategy to address policy outlined in Scottish Government documents119,206 and also single outcome agreements (agreements between the Scottish Government and Community Planning Partnerships which set out how each will work towards improving outcomes in a way that reflects local circumstances and priorities). Within the Scottish Government’s alcohol strategy documents, the need for joint working between the NHS and LA partners is made explicit in the requirement to invest in prevention and treatment services alongside building an environment that supports cultural change.
The process of alcohol licensing in Scotland
Below, we give a generic overview of the licensing process in Scotland, as identified in the data, before going on to present our findings. The alcohol licensing process is a complex, legally governed decision-making process. Information and evidence from many diverse and sometimes competing sources enter and flow throughout the system, ultimately leading to a decision made by the Licensing Board on whether or not an application for a specific premise’s licence to sell alcohol, or to change the terms of an existing licence, is successful.
The legislative framework
The alcohol licensing process is heavily regulated. The process that is mandated creates prescriptive opportunities for evidence use but also constrains the freedom of those involved to request evidence outside this process. The current legal framework and policy guidance for alcohol licensing has been set out by the Scottish Government. This includes:
-
Licensing (Scotland) Act (2005)201
-
Changing Scotland’s Relationship with Alcohol: A Framework for Action (2009)119
-
Criminal Justice and Licensing (Scotland) Act (2010)202
-
Alcohol etc. (Scotland) Act (2010)203
-
Alcohol (Minimum Pricing) (Scotland) Act 2012. 207
Statutory bodies: the Licensing Board
The Licensing (Scotland) Act (2005)201 stipulates that every LA must have a Licensing Board (the Board) and a Local Licensing Forum (the Forum). The Board is made up of between 5 and 10 elected members. Licensing Boards are a separate legal entity from councils and have their own constitution and statutory procedures that differ from those applicable to councils and council committees. The Board is required to be an independent and impartial tribunal. The role of the Board is to regulate premises that sell alcohol to the public. Legal advice is provided to the Board by the clerk to the Board, a legally qualified member of staff within the council.
Statutory bodies: the Local Licensing Forum
The Forum is a multidisciplinary group comprising key stakeholders with an interest in the sale of alcohol and its consequences. Forum membership differs from locality to locality; however, it typically includes representatives from the alcohol trade, adult social care, police, communities, environmental health and the legal profession. Forums were introduced within the Licensing (Scotland) Act (2005),201 which stated that each LA should have a Forum and that membership must include a licensing standards officer. The Alcohol etc. (Scotland) Act (2010)203 first stipulated that a Health representative (or someone nominated by Health) should be included in the Forum membership. To ensure that this was actioned, Forum membership could be increased by one or two places to accommodate Health representation.
The Forum meets quarterly. The role of the Forum is to review the operation of the licensing system within its locality and to give advice, make representations and provide recommendations to the Board on general licensing issues but not in relation to specific cases. The Board is not required to act on this advice or these recommendations; however, if it chooses not to, the Board must provide the Forum with the reasons underpinning this choice. Legislation stipulates that the Board and the Forum must hold at least one joint meeting per calendar year.
The Licensing Board policy statement
The Licensing (Scotland) Act (2005)201 requires each Licensing Board to develop and review (every 3 years) a statement of licensing policy that will guide the work of the Licensing Board. Policy statements can be reviewed and amended within this 3-year period.
Each licence application and variation is considered on its own merits; however, the policy statement sets out how the Board will meet the five licensing objectives in order to provide applicants, the public and responsible authorities with information in advance of the Board’s general approach to licensing. The Licensing (Scotland) Act (2005)201 requires Boards to carry out an assessment of overprovision and include this within policy statements. In determining overprovision, Boards must consider the numbers and capacity of licensed premises within the area. This statement of overprovision further guides decision-making.
In this way, typically, once every 3 years, current research evidence is gathered and embodied in the policy statement. The evidence is applied to the local area to map out areas designated as overprovided. The Board must operate within the bounds of its licensing statement when considering licensing decisions; however, Board hearings are civil not legal processes. This difference is significant and will be explored later.
Within legislation, there is a rebuttable presumption that a licence will be granted unless there is significant information to the contrary that is brought forward or unless it can conclusively be shown that to grant this licence would contravene at least one of the five licensing objectives within the Alcohol etc. (Scotland) Act (2010). 203
Making licensing decisions: the legal process
The Licensing (Scotland) Act (2005)201 identified five licensing objectives. These objectives form the foundation for Boards to develop policies that will promote safe, social drinking. The application of licensing objectives is key to a Board’s ability to carry out its functions within legislation. While the policy statement sets out a Board’s general approach to decision-making, it does not counter an individual’s right to make a licence application.
Information and research evidence that is brought forward to challenge a licence application can only be done so under the five licensing objectives. These five objections are as follows:
-
preventing crime and disorder
-
securing public safety
-
preventing public nuisance
-
protecting children from harm
-
protecting and improving public health.
[Note: in England within the Licensing Act (2003)112 there are four licensing objectives. There is no public health licensing objective in England.]
For each licensing application or variation received, the Board must seek information and evidence in writing prior to the licence hearing from a number of statutory consultees. Details regarding each individual licence application are also posted in local communities and any person or organisation may lodge an objection to a licence being granted, under one or more of the licensing objectives. Statutory consultees include:
-
Police Scotland
-
Licensing standards officer
-
Environmental health
-
Health.
Health was appointed as a statutory consultee within the Alcohol etc. (Scotland) Act (2010). 203 As such, a locally designated department within the health board receives a written request to respond to all new and major variations to premises licences. For a response to be considered by the Board, the response (research evidence, data and information) must be formally presented in writing, and in person if desired, within the public domain of a Board hearing. Case law, prior legal precedent, current legislation and local knowledge are also factors that influence alcohol-licensing decision-making. Research evidence, data and information presented in person are open to questions from members of the Board and from those present at the hearing. If a statutory consultee does not make a response then the Board is unable to request this information as, under the legal process, this could be considered prejudicial to the applicant.
Once furnished with all the information, the Board retreats from the public meeting and convenes (in private) with the clerk to the Licensing Board (who is legally trained) to consider the material presented and come to a decision regarding each licence application. If a licence is refused, or if conditions are applied to a licence, the applicant may challenge the decision by lodging an appeal in the Sheriff Court. This is a legal process and can be costly if the Board’s decision is overruled, with the council bearing the costs from the public purse. (Note: to preserve confidentiality we refer to the Scottish site as Thistletown).
The alcohol licensing process in our Scottish case study site
Below, we systematically go through the process of granting licences in our case study site to explore where and how research evidence and other types of evidence enter the process. To do this, we overlay the simple diagram (Figure 1) with a series of subprocesses, represented diagrammatically (Figure 2). These diagrams are heuristic to illustrate the process, the groups and statutory bodies involved, and the way evidence enters and passes between them.
FIGURE 1.
Summary of the core process for alcohol licensing in Thistletown.
FIGURE 2.
Wider types of knowledge that form a backdrop and enter the process. Arrows push evidence in and bean men pull evidence in.
The licensing process is, of course, embedded in wider socio-politico-cultural, economic and legal processes both within and across local government, health and other statutory bodies. The context creates both structural and person-dependent opportunities for, and constraints on, what can be achieved (structure and agency). As well as the formal statutory bodies and the legislated ways in which they can and should operate, information and knowledge pass through the process informally. The prevailing beliefs, customs and ideas of what is valuable and constitutes a ‘good outcome for those involved’ enter the process of decision-making in the expectations, relationships and actions of those involved. With members drawn from multiple interest groups, it is not unreasonable to expect that these interest groups have their varying ways of seeing the world and hold differing values and views. Below, we illustrate how this wider canvas of knowledge and beliefs enters the system both as a backdrop (context) to the system (structure) and via the people involved (agency).
In Figure 3 we illustrate where information flows around the licensing process, and how it is pulled between the different bodies and pushed into them via the communication mechanisms they have. Outputs from the Board (issues they were unable to address locally), flow to the Scottish Government the Board pulls information in from consultees, it has information pushed to it from trade bodies, and they both push and pull information to and from the Forum. The Forum also directly pushes information into the formation of the policy. Below, we expand on these processes and try to tease out to what extent research evidence is included in this flow of information.
FIGURE 3.
How evidence flows around the whole licensing system. Arrows push evidence in and bean men pull evidence in.
Findings: what evidence is used by whom and for what purpose?
Research evidence formally enters the licensing process in three main ways: via the Board policy statement, via Board hearings and via the Forum. Each of these represents formal evidence entry points, specific places in the process where evidence can be formally introduced and potentially mobilised in the decision-making process. Each of these three points is described below to illustrate how the formal evidence entry points are actually used in Thistletown and how the action is shaped and modified at every stage by informal processes and other types of knowledge and information.
The Licensing Board policy statement
Licensing Boards’ policy statements must include evidence on the density of provision in the geographical areas under the Boards’ remit and identify areas of overprovision. In this sense the evidence base on reducing the harms created by the wide availability of alcohol is embedded in Board policy and has the capacity to inform all decisions taken during the life of that policy. However, participants acknowledged that while Licensing Board policy statements may be evidence based, they merely guide Board decisions and allow flexibility in decision-making: ‘I mean the Board have a policy, which they don’t have to follow, they can make exceptions from it, when they are looking at a particular application’ (participant 14, emphasis added).
It was acknowledged that many and varied reasons underpin decisions and that decisions are dependent on the specific circumstances and evidence presented to support or counter individual applications. The formal process of a Board hearing was considered useful in providing an arena for evidence from all interested parties to be presented and thus enabling the Board to make an informed decision. However, who presents information and the amount that each party presents varies, suggesting that input may not be balanced. While this process was recognised as highly formalised, it was acknowledged that Board hearings are not legal courts and it was highlighted that evidence and information presented to the Board to aid its decision-making was not stringently tested.
People will provide you with what they believe is either appropriate or they’ll provide you with what’s mostly convenient. Now, you also don’t know whether they’re providing you because it’s convenient or whether they feel its most representative, or whether they feel it’s showing a spike and that’s what they are looking to show, you don’t know if there’s an agenda to the provisional data.
Participant 17
Flexibility was valued, as it allowed Board members to take local knowledge into account and use their discretion to temper this against the evidence base (as embodied in their policy statement). Where the policy statement was over-ruled (e.g. licence granted in an ‘overprovided’ area), some participants felt that economic vibrancy and politics were key elements underpinning such decisions. Participants felt that there was an opportunity for the Board’s overprovision statement, if used in conjunction with the public health objective, to be better applied to refuse or restrict such applications.
Boards are made up of elected members. Some participants felt that this was important as it ensured a good understanding of local knowledge. In Thistletown, Board members are proactive in trying to understand the context of specific premises and the communities within which they will operate and regularly make site visits to inform the decision-making process. This local knowledge and ‘feel’ for the setting was highly valued by Board members. Conversely, participants highlighted that Board members were not likely to be expert in all the issues that would need to be considered in the decision-making process:
I’m not sure any of the elected members that sit on the Board have spent any time with their [specific piece of local research] people or any of the local Alcohol and Drug Partnership (ADP) structures or anything that would give them a particular insight into the alcohol issues in their community [. . .] So there’s quite a lot of local knowledge they will not have.
Participant 16
So, while local knowledge was thought to be important in decision-making, participants felt that Boards could, and should, rely more on input from statutory consultees as topic-specific experts. This represents a second evidence entry point where statutory consultees, along with licensees and any individual or organisation, could present evidence in order to shape decision-making. We start to see the emergence of different knowledge types – situated, embedded, local knowledge and ‘context-free’ research evidence – being brought into play by different groups.
Licensing Board hearings: Health as a statutory consultee
Within Thistletown, at the time of this research, all participants identified that the statutory consultee role was not a role that Health had embraced or actively taken forward; indeed, Health was considered to be, unfortunately, missing.
They’re statutory consultees; they get the information sent to them in an envelope marked ‘for your attention’.
Participant 1
It’s difficult for me to assess, I can’t tell you anything about the evidence they might provide because there hasn’t been any.
Participant 4
I’m not trying to be critical, but it’s just there’s a noticeable absence, if you like, of comment in a very relevant area [health].
Participant 5, legal
However, it is acknowledged that this research took place at a specific time and findings might have been different had the timing of data collection been different, given the history of the Board. This appears to be a conundrum if Health is deliberately missing one of the statutory chances it has to enter public health research evidence, data and information into the decision-making process. In Thistletown, failure to engage within the role of statutory consultee was considered by non-Health participants to be a stance actively chosen by Health, an issue not entirely disputed by Health participants. Participants with an active role in licensing beyond Thistletown stated that disengagement of Health within the statutory consultee role was not unique to Thistletown. These participants felt that the introduction of the statutory consultee role for Health had not improved the provision (quality or quantity) of health evidence, data and information to Boards across Scotland. One interpretation offered (identified as occurring throughout Scotland) is that legislation introduced statutory duties in advance of the readiness of Health’s capacity to respond.
Some of the reasons participants felt that Health did not engage (across Scotland) were identified within interviews as:
-
There is little or no capacity to respond (writing a written response to each application).
-
Inappropriateness of public health data to comment at the individual level needed by each licence and recognition of the difficulties encountered.
-
A deliberate decision to concentrate resources elsewhere based on previous unsuccessful experiences of influencing licensing decisions.
We expand on each of these next.
Capacity to respond
Within Thistletown there is a high volume (‘there are 30-plus a month’: participant 16) of licence applications to respond to. With a rebuttable presumption in favour of granting each licence, the Board requires public health to consider each application or major variation to an existing licence and decide if it wishes to raise an objection. A ‘blanket’ overprovision statement does not refute the rebuttable presumption. Participants identified that, for some time, they had felt that there had been a lack of clarity within Health as to which department should be responsible for responding in the role of the statutory consultee. Some non-Health participants felt there was a disconnect between the strategic sign-up of the Health Board and the Health Board’s capacity to fulfil the statutory consultee role.
The Health Improvement Directorate has now taken on the role to respond, but has little capacity within existing staff workloads to be able to meet this obligation. In addition, the staff tasked with responding within this role have limited knowledge about alcohol licensing and how public health data can best be used to influence individual licensing decisions.
. . . discreet licences that [Health] staff didn’t even know existed, never mind give a view on how relevant or appropriate that might be to give that licence or change that licence . . . staff are not research based staff. They are not staff that actually spend their days going and finding out what is the academic evidence base for this, that and the other. They are more staff that are around implementing what comes from some of this.
Participant 16
Inappropriateness of public health data to be utilised at the individual level needed
Regardless of Health’s capacity or readiness to respond, questions were raised about whether or not the evidence base around alcohol and health was able to answer the questions posed by individual licence applications. While Health has an obligation to respond within its role as a statutory consultee, all participants stated that existing health and public health data (typically population-level data, and often historic) are difficult to apply at the individual level needed for specific licensing decisions. In many instances, the specific licensed premises or variation applied for is not yet operational, and legal challenges can, therefore, question the contribution to harm by this licence, which is not yet in place. Participants recognised that any weaknesses or anomalies in data presented as an objection to a licence could be exploited by licensing lawyers acting for applicants in any appeal situation. It was felt that to be more effective, health data needed to be more specific and localised than those currently used.
[T]o get the data of how many people go into A&E at 3 o’clock in the afternoon drunk, you know that’s the type of thing.
Participant 3
However, those participants who know the data well expressed concern that this level of precision is never likely to be possible.
I don’t think we will ever be able to link health data and alcohol-related harm to individual premises, I just think that’s impossible.
Participant 9
Although it was acknowledged that existing health data themselves were difficult to apply at such a localised level, participants felt that these data would potentially have more value in adding strength to an objection when used alongside other data, such as crime and disorder data. Participants identified the need for increased partnership working between statutory consultees (which is one of the recommended HICs).
. . . there’s real value to what they [Health] can bring to the table, but it must be mapped against what everybody else brings, because we don’t work in, we shouldn’t work in isolation.
Participant 12
The role of ADPs was also raised as being relevant in relation to the mobilisation of locally relevant health data. While it was acknowledged that ADPs do not have a direct role as a statutory consultee, it was felt that they could be consulted by Health to provide data and input into any Health response.
Concentration of resources elsewhere based on previous unsuccessful experiences
Participants with an active role in licensing beyond Thistletown stated that their experiences elsewhere in Scotland of health evidence being presented through the statutory consultee role (and related to the public health objective) often led to unsuccessful outcomes (e.g. licence granted despite Health objections). This seems to suggest that the specific findings from Thistletown were neither atypical nor unusual in this respect.
Participants strongly felt that Health should engage more with the statutory consultee role; however, the difficulties that this raised were recognised. The process is such that Health would be presenting evidence that is difficult to apply at an individual level, within the formal arena of a Board hearing, where it is open to examination, scrutiny and challenge from legal agents. When Health participants were faced with these multiple difficulties of presenting and getting research evidence to make a difference to licensing decisions, they acknowledged that, more recently, health resources had been directed upstream into trying to inform local and national policies instead.
. . . so if they [Public Health] are going along and it’s not making any difference to the application, then it’s possibly a situation they’d rather not become involved in.
Participant 14
. . . they [Board] are looking at individual premises, [Health] are looking at a population basis so [Public Health] have tried to have more influence at a national level, rather than at a local level.
Participant 9
Turning one’s attention (and resources) to policy level, where population data may be better suited to providing answers, is arguably a rational response by health professionals. However, non-engagement by Health with the statutory consultee role still has impact and is not a neutral outcome. Case study site observations by the research team and interview responses identify that where Health does not respond to an application, others, particularly the alcohol industry and their legal agents, infer that, as Health has not responded, there must be no negative health issues related to that application. A public health objection does not necessarily signal a ‘red light’ to applications, but a non-response is taken as a ‘green light’. Although others are able to raise a health objection when (Public) Health does not, this lacks credibility because it is (Public) Health, as the statutory consultee, that is considered to be the topic-specific expert rather than the person raising the issue in its absence. (Public) Health’s non-response is often highlighted by licensing lawyers as a signal that there are no health issues to be raised.
When Health does respond as a statutory consultee, participants recognised that health-related objections can be difficult to refute. Some participants stated that, elsewhere in Scotland, when evidence was presented around the public health objective, licensees and their legal agents were being asked to return to later hearings and describe how granting a specific licence would improve public health, instead of Health being asked to state that the licence could be detrimental.
What they wanted [lawyer] to show was that, to counteract the policy, was that by granting the application, there would be a benefit to public health. Which is impossible. I mean they were wanting to sell more alcohol.
Participant 14
It was acknowledged that within the recent pre-consultation exercises for the current Board policy statement review (2013; full reference withheld), Health had been involved. Engagement, specifically within the verbal evidence sessions, where Health presented data and supported community members to attend and present evidence about the community impact of alcohol use, was highlighted as positive by both Health and non-Health participants.
The presentation to the Licensing Board, and some community reps went along and provided a picture from a very much lived experience of alcohol, [. . .] so in that sense there was capacity build up, where people actually go and tell their stories, and I don’t know what’s, what the Licensing Board have done with all of that, but I certainly understand they found it incredibly powerful. So those telling of stories probably has had a bigger impact than the evidence that we presented. But that’s good, because then you’re getting a holistic picture, not just one set of information to be able to tell a story.
Participant 16
The drug and alcohol forum [ADP] were ones that came in, I think where the most balance between evidence and passion about the harm that drugs and alcohol do in the community because they’re not a dry, the health professionals, they’re people who are actually working with communities and individuals severely affected by it [. . .] and it almost worked better than dry academic health statistics, this was community based experience of the harm that alcohol can do to individuals and to families.
Participant 7
The Local Licensing Forum
As mentioned previously, the role of the Forum is to gather information for the Board, give advice, make representations, provide recommendations and ask crucial questions to challenge the Board’s general licensing activity. The Forum is a well-attended meeting that participants believe provides an excellent opportunity to bring together a diverse range of stakeholders involved in alcohol issues: a good place to network. However, there is much uncertainty among participants around the actual role and function of the Forum and any actions arising from it. Many participants questioned the usefulness of the Forum:
. . . do we actually understand what the Forum’s supposed to deliver, or does it leave you feeling slightly, erm, slightly disappointed at the end that we hadn’t gone with what we thought it would do.
Participant 12
In relation to the Forum’s role in facilitating the entry and flow of evidence and information within the alcohol licensing process, participants raised a number of issues. Concerns were raised regarding membership of the Forum, which it was felt could be more balanced than it is. Current membership was considered to stifle debate and decision-making. Prior to the legislative requirement to have a forum,202 an alcohol trade forum had been operational within Thistletown, out of which the current Forum had developed. Several participants considered it to be as a result of this origin that the current Forum had a high level of trade and council representation and that there was a (perceived) tendency for Forum business to focus largely on non-contentious, practical day-to-day operational issues rather than more complex or strategic issues.
There’s an element where the Forum is almost a grandstand for self-interest.
Participant 7
Participants felt that the number of trade members in the Forum influenced discussion and proved a daunting opposition for less well-represented organisations or individuals to challenge. The convener and the clerk of the Board also attend Forum meetings, and in many ways this was seen as helpful, as they could provide instant answers and feedback from the Board’s perspective. However, their presence was also considered to stifle discussion, debate and any challenging of the Board by the Forum:
The Convenor of the Board attends the General Forum [. . .] and whether that perhaps creates a reluctance, because people are conscious, if they came somewhere and they were critical to someone, 2 weeks later they could be in front of that person, [in] a hearing.
Participant 5
Concerns were raised regarding the extent to which the Forum represents Thistletown and focuses on local issues, as in some roles the member attending represents a national organisation and is not a trader or practitioner in the case study site.
. . . the level of membership you have got is not particularly local, it’s quite high level [. . .] rather than someone who runs a [premises] in the city centre who deals with customers.
Participant 8 Health
Participants stated that communication between the Forum and the Board could be improved and that the Forum was not active enough in questioning the Board or in formally asking for clarification of issues. Participants pointed out that the ADP was missing from the Forum membership but suggested that if representatives from the Partnership were present they could provide useful input, which could then give the Forum a more strategic role. Involvement would also enable the ADP to raise issues within the Forum that, when taken to the Board via the Forum, would, within legislation, require acknowledgement and a written, documented response (although they would not necessarily lead to action).
There’s not a huge amount of [pause] questioning of the Board’s policy or what they do comes through the Forum, erm, less than I anticipated.
Participant 5
There’s been little visibility of that ADP strategy at the Licensing Forum. So they have not got a common, if you like, strategic vision or direction. I think that they would share. Therefore there’s nothing in the commissioning programme that’s been informed by licensing.
Participant 11
Some members of the Forum, however, were also members of the ADP. Information sharing between groups was thought by participants to be lacking, with some participants realising that perhaps they could do more by acting across their varying professional roles. Inputs of evidence into the process that are person dependent literally depend on that person(s) to activate them and to share information, intelligence and research evidence across their multiple professional roles:
Part of the conversation today is about what my role is and I’ve always to date I have gone to react or respond to something, rather than proactively bring something along to the agenda.
Participant 11
Participants felt that the Forum produced, at best, discussion, but no action. Any evidence or information presented consisted of regular long reports or updates and ‘stories’, but little that stimulated debate.
A lot of things get read out that have been circulated prior to the meeting anyway, and then there’s a limited amount of discussion, but it doesn’t seem to go anywhere.
Participant 9; emphasis added
Participants highlighted the diversity of Forum membership as a factor underpinning the lack of debate and actions coming from Forum meetings. Participants felt that there were many differing and competing interests among members, making agreement and action difficult to achieve. The lack of action from the Forum frustrated participants, and most struggled to see the benefit of the Forum in its current format. Many had become disengaged and frustrated. Disengagement often led to non-attendance or deputising attendance, which could stall issues further.
If there was a clash in my diary, it wasn’t a thing that I prioritised for a time.
Participant 9
It has to be the right person at the right meeting to be able to do all of that, but also then make decisions that the Forum can be trusted that are going to be taken back by your industry and move forward. That there’s a level of responsibility on the participants and delegates that they are not just there to keep a seat warm.
Participant 12
What we can see is that although the Forum provides a structural place for entering evidence, it is the action of its members that turns the Forum into an evidence entry point. 208 Some participants felt that a lack of clarity around the role of the Forum, as well as a lack of leadership and direction, also impacted on the ability of the Forum to move issues forward. Others went further:
I would say that the matters discussed, erm, the challenges are about the agendas of the table in the room, in terms of the participants in the room.
Participant 12
Evidence or information for discussion within Forum meetings must be requested or put forward by a member. All issues to be tabled must be submitted in advance and are discussed at a pre-meeting by the convener of the Board, the chairperson of the Forum and the clerk of the Forum. At this meeting, Forum agendas are agreed. Most participants could not remember any research evidence or information being presented at Forum meetings aside from reports from the Licensing Standards Officers and updates from the Local Community Safety Services Initiative. In this way, inclusion on meeting agendas is another possible evidence entry point.
In addition, among participants no one was really sure how well information from the Forum was cascaded among individual members’ own organisations; the Forum appeared to neither pass on information (to the Board) nor pass it back (to the members they represented). Minutes and meeting documents are public documents and are available on the council website; however, participants did not think that awareness of this was high among the public and considered that access to these was sought by only a limited number of people.
Participants involved in alcohol licensing outside the Thistletown site reported that the issues identified within this research around the Forum were not uncommon across Scotland.
What evidence enters and flows: using the evidence entry points
Health research evidence and health-related data
Health participants confirmed that health research evidence and health-related statistics (prevalence, trends and projections in alcohol-related harms) had been presented at Forum meetings. This was not a common event, but some participants recalled the presentation of this evidence to the Board and Forum and provision for inclusion in previous Licensing Board Policy (2010; full reference withheld). The presentation and written evidence had largely not been well received for various reasons. It was considered to be:
-
too complicated to understand (e.g. statistical terminology, key messages not pulled out); as one participant said: ‘it was presented for an expert audience and even then I’m not sure an expert audience would have taken it on’ (participant 17)
-
overly generic (originating in different places and not focusing on how those issues were manifesting locally, in order to suggest honed responses)
-
contradictory of itself
-
out of date
-
lacking actionable messages (e.g. stated the size of the problem, but lacked practical solutions)
-
data, not evidence (e.g. not defining if an issue recorded was one person 100 times or 100 people once, each requiring a different response, a focused service or policy response). Data were also not mapped against local areas in a way that identified options for targeted action.
They did provide health statistics, but the Board considered that it wasn’t erm relevant enough, it was too generalised for them to form any conclusions from that information. And therefore it didn’t really assist in the development of policy, other than to raise general concerns about, erm, alcohol consumption levels throughout the city.
Participant 4
Participants stated that evidence did not ‘speak for itself’, and nor were its messages clear or obvious. There was always more than one interpretation of any data presented. Some participants felt that during Board hearings, licensing lawyers (acting on behalf of their client who was seeking a licence or variation to a licence) actively sought to unpick data presented by Health as the weaker data, undermining its significance.
It wasn’t well received because sometimes it wouldn’t be what people wanted to hear, there was also a comment that because it was quite a complicated formula, that that detracted from the message . . . How much of that was; ‘we actually don’t want to grasp’ it I don’t know.
Participant 15
Overall, participants thought that Health considered its key role in relation to alcohol licensing to be one of strategic involvement and raising public and media awareness rather than engagement with day-to-day operational licensing issues. Where health information does flow, participants highlighted the gathering and presenting of the information in a straightforward way as key to its success. Participants involved within alcohol-licensing decision-making within Thistletown felt that, across Scotland, health information worked better when presented in a more localised way. It was acknowledged, however, that even when health information does become part of debate and policy, it is not necessarily followed through into decision-making.
Health input into the evidence sessions recently conducted as part of the pre-consultation exercise for Thistletown’s Board policy statement review was, however, highlighted by participants as positive, as this input combined statistics and data with real community stories, and impact was easier to understand.
Local data and statistics, and community stories
Within Forum meetings, the only information regularly presented (aside from the Licensing Standards Officers report) was an update report from the Local Community Safety Service initiative. This information was presented at every Forum meeting observed (September 2012 to September 2013). Information was well presented in an easy-to-understand language and graphic format. It included information about the night-time economy, footfall and operation of different elements of the initiative within different localities, and was case study site wide. No participant was able to remember when or why this presentation became a regular agenda item, but there was widespread consensus that the information was of interest (in many different ways) for all parties within the Forum:
. . . that’s very valuable information, from my point of view having seen that, that information that will be relevant to the trade members, there because it’s about footfall and it’s about [Thistletown], it will be relevant to the police and to Health as well.
Participant 6
Some participants, however, felt that the information promoted no debate or controversy (strategic issues) and was very much just an information update (operational), and therefore questioned its added value:
. . . it’s not problematic to anybody . . . it doesn’t create any, it doesn’t create any err, real debate or arguments as a consequence, because it does tick all the boxes.
Participant 11; emphasis added
Evidence ‘ticking all the boxes’ is an interesting concept. It seems to encapsulate ideas about data from here-and-now, related to daily pressing issues, that something can be done about; we return to this notion in the discussion chapter (see Chapter 8).
Competing information and contextual pressures
Participants acknowledged that in relation to alcohol licensing, health was only one of a number of issues that would inform decision-making (and often, for some, not the most important). 37 Below, we explore some of these other contextual factors and how they played out in Thistletown to inform alcohol-licensing decision-making.
Economic well-being
Economic well-being and regeneration opportunities often offer alternative perspectives on ‘what is the best decision’ and pose important (sometimes competing) considerations. Participants felt that one of the key factors was the economic well-being of Thistletown. Economic well-being was related to both on- and off-trade licence applications, and participants considered that the Board would be under pressure from the electorate, businesses and others within the council to ensure that in considering applications they took into account all perspectives.
The economic effects as well as jobs [retailer] come into an area and as a result the actual health if you like, thing, is always competing with you know business, and business demands, [. . .] any arguments that health have got seem to be put on the back burner, ahead of err, votes at elections and ultimately erm, business finance and you know the economy of the local authority area.
Participant 3
Economic well-being (taken together with the rebuttable presumption in favour of granting a licence) was highlighted by some participants as potentially having greater influence than the overprovision policy and statement. Some participants considered the influence of economic well-being to be so great that there could almost be an additional unwritten licensing objective around promoting it.
. . . there isn’t an economic objective but you would think there was, the way people kind-of carry on talking about it.
Participant 8
I can guarantee you now the Licensing Board will never say the entirety of [Thistletown] has overprovision. Sorry, there are all sorts of economic reasons and developmental reasons, why doing that would be suicidal and the rest of the council would be on the Licensing Board like a ton of bricks. Because you would stop any hotel, you would stop every major, err, economic development and shopping centres, etc., etc. It isn’t going to happen.
Participant 13
While all participants recognised and supported the need for Thistletown to be a vibrant, thriving and desirable night-time destination and a good place for businesses to invest in and bring jobs to, there was acknowledgement that it was necessary to find a balance between economic well-being, health and harm.
A backdrop of political influences
Participants felt that power and politics were factors that influenced the flow and use of information and evidence within the alcohol licensing process within Thistletown. This was noted both in observations of the Board and Forum meetings and in interviews.
Local Licensing Forum
As previously highlighted in this chapter, getting information and evidence onto a Forum agenda for discussion is subject to that document being accepted at a pre-agenda meeting. As well as being able to refuse documents, the pre-agenda meeting is also able to defer documents for inclusion at future quarterly meetings, which could impact on the relevance of the issue at a given time.
Participants expressed concerns that, although all Forum members had the same opportunities to contribute within Forum meetings, there was a feeling that not all voices were expressed or listened to. The power and political (small ‘p’) dynamics of Forum membership were identified by participants as an issue underpinning this. As previously mentioned, the Forum has a high membership from the alcohol industry, and participants were concerned that this provided a dominant professional contingent around the table which could be daunting to challenge.
You need to be sure that you could have an impact that was worth fighting that.
Participant 9; emphasis added in bold
Some participants were open about the influence of politics and power within Forum meetings and the potential to use this to their advantage.
. . . sometimes I have to play a political game and I don’t necessarily want to oppose everything that [others] suggest. So there’ll be occasions when I’ll need their support on issues so I gave that support.
Participant 2
The Licensing Board
There was concern among some participants that the power they perceived to be held by larger corporations in both on- and off-sale applications was a factor that influenced the decision-making process. Such organisations were thought by participants to be likely to have excellent legal teams working for them and, in cases where licensing decisions were not favourable, sufficient funds to risk an appeal against the decision.
Like everything in life and everything where there is regulation, people who are part of a big organisation, who have better resources, have the ability to employ lawyers and accountants, and folk that can turn round and put forward the eloquent cases.
Participant 5
They’ll give [supermarkets] a hard time, don’t get me wrong, they’ll give them a, they’ll go through the motions and give them a right hard time about why the area needs another, but [supermarkets] and stuff they just produce all these documents about how, how much they’re investing and, you know, making things better, and reducing their alcohol range and all these investments, we’ve engaged in this and that, erm, to and then they’ll get, they’ll get a hard time. Oh well, we’re not, right OK, granted and they get that.
Participant 3
Negative stereotypes
Difficulties in communications and perceived lack of responsiveness on both sides (Health’s perceived failure to submit public health objections or engage in the process and the Board and Forums’ perceived unwillingness to make licensing decisions in line with the prevailing public health evidence base) led to negative stereotypes on both sides.
The perception of ‘Health’ and ‘the Health lobby’ was of a discipline that is lofty and elitist. Health participants spoke of their own belief that ‘Health’ as a discipline is perceived as lecturing people about what should be done. They believe that others perceive health messages and health thinking around alcohol as having a single negative and gloomy focus – no alcohol is good for you – which ignored alternative recreational and social interpretations of alcohol use. Health participants believed that because people felt that they had heard the negative message so often they now disengaged from it.
. . . as having a single agenda, maybe [Health] do in that respect, but as a consequence of that, err, it is seen as a single err, and being single and repetitive and I think it loses its effect over time because it’s the same message they see.
Participant 11
. . . the community don’t want to hear the health message either, do you know . . . they don’t want to hear it, the Board don’t want to hear it, the Forum don’t particularly want to hear it, erm, no one particularly wants to hear it.
Participant 8
Conversely, there was a tendency to attach negative stereotypes to Board and Forum decisions with concern that they were influenced by the pressures of the night-time economy. Some participants questioned the appropriateness of the high percentage of membership from the alcohol industry, specifically the alcohol retail trade, within the Forum, given the role of the Forum in relation to the Board. There was concern raised that this influence could have an impact on the granting of future licences.
I don’t know whether the right people are influencing the decisions because its people that are already in the market, not people that, you know, want to bring something new to [Thistletown] or that.
Participant 14
A balanced view?
Each party was aware of the negative stereotype the other sometimes held of their activity. Board and Forum members felt that the good work they had done was largely overlooked. While they highlighted that the statistics for alcohol-related health and social harms in Scotland remain extremely high, they stated that locally there had been a downwards trend in recent years (which is supported by the quantitative data; see Appendix 2). There was frustration among participants regarding this and the lack of recognition or praise afforded by the health sector generally to other organisations working to address alcohol issues, and to the general population for, to some degree, taking on board more responsible drinking messages.
. . . even in circumstances when they [Health] ought to pat [businesses and communities] on the back, for want of a better phrase. I’m thinking of some statistics that have come out where the admissions, hospital admissions on alcohol related issues, illnesses where they have actually gone down and other statistics that show that alcohol consumption has gone down [. . .] and its galling at times when you feel there’s some improvement is being made and you don’t even get the credit for it.
Participant 2
Often, Board members expressed the opinion that they would like to be able to do more, but find themselves restricted by both the legal process and other competing demands. In Thistletown, with particular relation to alcohol licensing, it was felt that Health needed to engage more to understand the processes – especially the constraints placed on the licensing process by the legal framework, for example in not being able to request evidence to over-rule the rebuttable presumption in favour of granting a licence. It was felt that this would improve any subsequent input or interaction, as on occasions requests were made that were inappropriate and not in the power of the Board to enact.
Chapter 5 The English case study: evidence use in commissioning to reduce maternal alcohol consumption
Background and context
In the English case study site (Rosetown), excessive alcohol consumption is identified as a growing problem, with mortality from chronic liver disease on an upwards trend and currently almost double the national average for England. The rate of hospital admissions for alcohol-related conditions in Rosetown is one of the highest in England (the latest figures available for Q2 2012–13 show that Rosetown was ranked among the top 10 out of 326 LAs in England) (Figure 4). While alcohol-related hospital admissions in Rosetown remain high, figures indicate that levels have plateaued in recent years, mirroring the pattern seen regionally and nationally. Rosetown has significantly higher death rates than nationally and regionally similar areas, and is currently ranked in the top 10 LAs nationally for its male and female chronic liver disease mortality (2006–8). In addition, although alcohol-related crime rates have fallen by approximately one-third between 2007–8 and 2011–12, they remain significantly higher in Rosetown than the regional and national average for England (full references withheld).
FIGURE 4.
Comparative trends in admission episodes for alcohol-related conditions (Rosetown, regional and national figures). Source: Local Area Profile for England (full reference withheld). Note: alcohol-related definition – alcohol-specific conditions (e.g. alcoholic liver disease or alcohol overdose) plus conditions that are caused by alcohol in some, but not all, cases (e.g. stomach cancer and unintentional injury). For these latter conditions, different attributable fractions are used to determine the proportion related to alcohol for males and females. Figures for 2011–12 were revised using mid-2011 population estimates, which in turn are based on the 2011 census; changes in the rate of admissions between 2011–12 and 2010–11 should therefore be treated with caution, particularly for smaller areas (Local Area Profile for England Publication date 20 January 2013; full reference withheld).
Policy context: the process of commissioning
Legislative changes and new structures
The structures for commissioning health care, including maternity services, in England and Wales changed from 1 April 2013 as part of the reforms introduced under the Health and Social Care Act (2012)173 passed in March 2012. Until then, 151 PCTs were responsible for commissioning health services, overseen by 10 Strategic Health Authorities.
Responsibility for commissioning maternity services in England now rests with 211 Clinical Commissioning Groups (CCGs), overseen and held to account by NHS England, which is an arm’s-length body of the Department of Health but is operationally independent. CCGs commission maternity services from local providers, NHS trusts and NHS foundation trusts, which are regulated by the Care Quality Commission. Individual health-care professionals are regulated by the Nursing and Midwifery Council and the General Medical Council. Public Health England (PHE) is a new national body whose role is to strengthen and support the public health system by providing stakeholders with evidence-based professional, scientific expertise, evidence and intelligence.
NHS commissioning support units (CSUs) have been established to support CCGs in carrying out their commissioning functions, including transactional functions, such as health-care procurement, contract negotiation and monitoring, and information analysis. CCGs retain legal accountability and responsibility for meeting their statutory functions and their commissioning decisions cannot be delegated to other organisations. 208
The Health and Social Care Act (2012)173 required the establishment of a statutory HWB in every LA in England from April 2013. Since then, responsibility for public health improvement has been largely transferred from PCTs to LAs in England. LAs have responsibility for ring-fenced public health funding from the Department of Health until 2015, which includes money to commission alcohol prevention and treatment services. LAs are expected to take the lead in co-ordinating local efforts to improve population health and reduce health inequalities and to provide public health expertise to NHS commissioners. 209
The role of commissioners is complex and involves assessing the needs of the population, planning and buying services, prioritising outcomes, overseeing and managing service providers, monitoring and measuring impact and planning next steps. 210 An essential part of this process involves putting together a Joint Strategic Needs Assessment (full reference withheld), which describes current and future health and social care needs and inequalities impacting on the health and well-being of the population in the case study area. This strategic overview is designed to inform and influence the commissioning process and provide evidence about what works to improve health and well-being outcomes for people across all stages of their lives.
Under a new duty, HWBs are obliged to prepare a Health and Well-being Strategy for endorsement by their CCGs in line with their own strategy (or, if refused by the CCG, then sent for appeal) to respond to needs identified in the Joint Strategic Needs Assessment.
Commissioning for services in pregnancy
Pregnancy is the largest single reason for admission to hospital, and for the average CCG of 250,000 people around 3000 women per year will use maternity services, costing around £8.6M. 162 While responsibility for commissioning maternity and newborn services sits with local CCGs, NHS England sets a national framework for quality and choice, including for antenatal and newborn screening services, and requires clinical services to be developed in accordance with the best available evidence. Commissioning responsibility for all public health services for children, including health visiting and family nursing, are due to transfer from NHS England to LAs in 2015,210 overseen by PHE.
Key to the classification of the quotations
In the data we present below, interviewees have been grouped together in an effort to protect anonymity. Table 7 provides more information on the categories used.
| Category | Participants |
|---|---|
| Public health | Participants working in public health in Rosetown PCT/LA |
| LA | Elected members and participants working in the LA, in posts other than public health |
| Clinicians | Participants employed in NHS trusts, NHS foundation trusts, primary and secondary care |
| Commissioners | Participants with responsibility for commissioning public health services |
The process of commissioning to reduce alcohol-related harm in our English case study site
Rosetown was an early implementer for HWBs, and at the time the fieldwork began in March 2012 in Rosetown transition arrangements were well under way in light of the anticipated transfer of responsibilities for public health to LAs and changes in the commissioning structures, including those to reduce alcohol-related harms and reduce maternal alcohol consumption.
Shadow working arrangements had been established and the Joint Strategic Needs Assessment (full reference withheld) was refreshed as the basis for a shared understanding of community needs and shared strategy during 2011. The HWB operating on a non-statutory basis co-produced joint Health and Well-being Strategies in light of the Joint Strategic Needs Assessment, providing a high-level framework for CCGs, NHS England and LA commissioning plans. Statutory Health and Well-being Strategies were set to become the vehicle by which councils exercised their lead role in integrating the commissioning of health, social care and public health services (full reference withheld). HWB membership in Rosetown largely followed guidance set out by the Department of Health211 and included the posts of:
-
chief executive of LA
-
leader of city council
-
director of adult services and health
-
director of finance and resources
-
director of children’s services
-
director of community services
-
DPH
-
chairperson and representatives of general practitioner (GP) CCG
-
director of regeneration and employment
-
public health business and resources planning manager
-
Healthwatch representative
-
cabinet member for adult health and social care.
Health Commission
The intention of the Health Commission is to find ways to improve the health and well-being of the local population, and, importantly, to find innovative ways to do this, by improving access to high-quality health care and assessing the current health profile of the city, including the impact of factors such as smoking, alcohol and drug misuse on people’s well-being. The commission is focusing on a few key conditions, including obesity, diabetes, rehabilitation after a period of illness, living with one or more long-term conditions, and dementia.
Health Improvement and Better Lifestyles Subcommittee
Reporting for all the separate strands of the Better Lifestyles Programme, as outlined in the Public Health White Paper and the Health and Social Care Act (2012),174 was brought together in the Health Improvement and Better Lifestyles Subcommittee, established in June 2012. It was jointly chaired by the Cabinet Member for Adult Health and Social Care and chairperson of the local NHS Trust. The group was responsible for overseeing an integrated programme of activity; including alcohol, drugs, tobacco control, healthy weight, mental health and sexual health.
Alcohol Strategy Group
Supporting co-ordinated planning and operational delivery of alcohol-related interventions, the Alcohol Strategy Group brought together multidisciplinary stakeholders from partner organisations, including Public Health leads, NHS primary and secondary care providers, the voluntary and community sector, and commissioners. The Alcohol Strategy Group generated ideas and priorities from the partnership before the detailed technical commissioning and procurement processes began.
Activities were informed by a combination of stakeholder consultation, local intelligence (e.g. service monitoring data), local and national policy directives, in which the evidence base is often embedded (e.g. the NHS Outcomes Framework, Public Health Outcomes Framework212,213), and evidence produced by NICE. In this way, the Alcohol Strategy Group formed an important part of the planning and co-ordination of alcohol-related services and interventions by bringing together service providers with strategic leads in public health and commissioning managers, who were responsible for integrated public health interventions, in the light of the available evidence. Monitoring information about commissioned activities, which was available to commissioning managers in public health, could be circulated to members of the Alcohol Strategy Group to inform future plans.
Prioritisation process for the Health and Well-being Strategy (2012–15)
During 2011, the HWB undertook a prioritisation process to identify a small number of priorities to reflect those health-related issues causing greatest unmet need in the population. These were chosen ‘to reflect the responsibilities of the Board, the priorities set out in the Joint Strategic Needs Assessment and focus on the big issues on which it was felt the HWB could reasonably expect to facilitate a measurable outcome’ (reference withheld). The priorities identified were:
-
alcohol
-
cancer
-
mental health
-
child poverty.
In addition, the HWB identified a cross-cutting overarching theme reflecting the life-course to address the impact that these issues have on children and families in Rosetown. During 2013, the Joint Strategic Needs Assessment was undertaken and used to inform a review of the local alcohol strategy and refreshed Health and Well-being Strategy. As part of the process, work was carried out to obtain the views of community groups and stakeholders relating to the health, social and economic impact of alcohol, including on groups most at risk. For each of the issues identified, a summary report was developed as part of the prioritisation process. This examined scale and severity, local impact, possibility of change through local action, effects on reducing health inequalities, and links with other priorities, locally or nationally. Alcohol continues to be prioritised as a risk factor for children, young people, families and adults (Rosetown Joint Strategic Needs Assessment statement of need 2013–14; full reference withheld).
Rosetown Alcohol Strategy (2011–14)
Rosetown has a history of partnership working to reduce alcohol-related harms. The Alcohol Strategy (2011–14) further developed a programme of activity across the key themes of prevention, treatment and control. The overall aim of the strategy was to prevent and reduce alcohol-related problems through partnership working and to use the best available evidence of what works (reference withheld). The strategy aimed to address policy objectives outlined in the Marmot Review15 in recognition of the strong correlation between areas experiencing the highest economic disadvantage and those worst affected by alcohol-related hospital admissions (Rosetown PCT; full reference withheld).
The key outcomes identified in the Alcohol Strategy (2011–14; full reference withheld) included;
-
changing knowledge, skills and attitudes towards alcohol
-
creating safer drinking environments
-
supporting individual needs
-
support for children, young people and parents in need
-
reducing the availability and affordability of alcohol.
It was felt that a focus on outcome 4 would strengthen an area of the strategy which was considered to be less developed. The deliberate focus on the impact of alcohol on children and families fits with the principles of the HWB to address inequality at the outset of life (Rosetown Health and Well-being Strategy Briefing Paper; full reference withheld). This aligns with the recommendations of the Marmot Review,15 which suggests that it is a matter of social justice that every child should be given the best start in life, and that society needs to address the social gradient in children’s access to positive early experiences. This choice, to focus on an underdeveloped cross-cutting area to reduce alcohol-related harms, is significant to the findings we cover later.
A pathway of activity was identified to achieve the following outcomes:
-
improved education and guidance of families and practitioners regarding harmful drinking in pregnancy
-
improved screening, identification and referral of those drinking at harmful levels
-
improved support for those continuing to drink at harmful levels in the antenatal and postnatal period.
Two multiagency workshops held in early 2012 invited local stakeholders to identify programmes and interventions which support a reduction in maternal alcohol consumption in pregnancy. Following the workshops, a summary of main tasks was pulled together to form an action plan to guide implementation of the Health and Well-being Strategy, as shown in Table 8.
| Tasks | Actions | Expected outcomes (evidence and measures) |
|---|---|---|
| Improve guidance and education | Insight work developed to understand women’s motivations, barriers, beliefs, etc., about drinking in pregnancy | Improved understanding of what motivates local population to reduce/abstain from alcohol during pregnancy |
| Insight work developed to understand midwives’ experience, knowledge and beliefs about supporting women regarding alcohol use during pregnancy | Improved understanding of current practice in how women/families are advised and supported regarding alcohol use in pregnancy | |
| Research proposals developed by local university looking at familial and partner influences; benchmarking practice; risk stratification models | Subsequent development of campaign and educational resources based on triggers provided by Insight that are responsive to local population | |
| Social marketing campaign in development; targeted demographic will cover at-risk population | ||
| Improved identification and screening | Pending meeting to discuss current and future practice with Rosetown maternity hospital | Benchmark current practice |
| Literature review completed regarding screening tools | Development of alcohol content in antenatal programme | |
| Alcohol outreach team to focus on families with aligned KPIs | Development of screening tool to identify those women drinking above recommended limits | |
| Development of risk stratification model |
Commissioning of Insight work on maternal alcohol consumption: the commissioning process we followed
In order to meet the objectives outlined above, Rosetown PCT (as it was then) commissioned an independent market research agency to undertake interviews with women and midwives. This is the process our case study partners requested that we follow. Rosetown has an established Insight team based in public health, which takes responsibility for commissioning Insight work. A service evaluation brief (full reference withheld) was devised by a senior executive in the Insight team to be carried out according to the Healthy Foundations segmentation approach. The project was planned to be undertaken in partnership with Rosetown Hospital, as the local provider of maternity services (hereafter referred to as the maternity hospital). Its stated intention was ‘to inform the (then) PCT’s alcohol in pregnancy service improvement strategy which ultimately aims to reduce the number of women who drink alcohol during pregnancy’ (full reference withheld).
The planned Insight work
Details of the Insight fieldwork and recruitment methods are provided in Table 9. The work was to be carried under the Market Research Society code of conduct. 211
| Number of interviews | Who | Focus |
|---|---|---|
| 12 | Midwives | Views on supporting women during pregnancy who drink alcohol, perceived motivations of the women. Identify their own training needs, and service development requirements, views on potential interventions |
| 24 (four focus groups) | Women aged 18–44 years planning to become pregnant | Motivations, barriers and beliefs about drinking in pregnancy |
| 20 | Women who are pregnant and are drinking alcohol | |
| 18 (six triad groups) | Women who recently gave birth and who drank alcohol during pregnancy |
Findings: what evidence is used by whom and for what purpose?
For each commissioning cycle, precisely who is involved will be different. Figure 5 illustrates the systems and structures involved in commissioning services to reduce alcohol-related harms in Rosetown. Parts of this diagram are teased out in the sections that follow. The commissioning process was not immediately apparent or straightforward in the way that this final account makes it appear. No one person knew the ‘whole story’ or had been with the process the whole time, so what follows below is a construction compiled from various accounts of how things happened.
FIGURE 5.
How evidence enters and flows in Rosetown. HWS, Health and Well-being Strategy; JSNA, Joint Strategic Needs Assessment; PH, Public Health; PHO, Public Health Observatory.
The data show that a wide range of information, intelligence, data, statistics, local knowledge, etc. was called ‘evidence’ and was reported as being pulled in the organisational units concerned from diverse local, national and international sources by stakeholders in the case study site. Figure 5 illustrates the potential evidence entry points in Rosetown, including evidence relating to alcohol use in pregnancy.
To simplify this view and to focus on the main sets of stakeholders involved, Figure 6 extracts and simplifies the process by which our case study site reached the point of commissioning the insight work.
FIGURE 6.
Simplified version of the main stakeholders and process involved in deciding to commission Insight work to reduce maternal alcohol consumption. HWS, Health and Well-being Strategy; JSNA, Joint Strategic Needs Assessment.
Our entry to the process starts with the Joint Strategic Needs Assessment, as the HWB declined to be observed owing to the fact that it was in transitionary arrangements (although it took part in the interviews). Discussions about the nature and implications of the evidence took place in the Strategic Alcohol Group, where statistics, monitoring data and anecdotal feedback from NHS service providers were presented and debated alongside epidemiological data from Public Health, research findings commissioned from local universities and national guidelines. Representatives from the CCG and HWB attended these strategic alcohol meetings, although formal feedback from them was rarely observed during the time fieldwork was undertaken. Some key stakeholders, such as maternity providers, were not represented at these meetings during the fieldwork observations. This was significant, as the meetings provide an opportunity for commissioners and providers to come together to inform, plan, co-ordinate and monitor activity to reduce alcohol-related harm. It was also noteworthy that the commissioning process for the Insight research appeared to be separate from, and not well integrated into, the strategic planning structures for alcohol. For example, the draft evaluation briefing document was not circulated for comment.
Evidence use in the Joint Strategic Needs Assessment
Figure 5 illustrates that the process is cyclical. The research team entered in order to study a particular commissioning process; however, this piece of work was situated within a well-established and diverse body of commissioned work to address alcohol-related harms. In writing this report we had to make a choice about where to begin the story. We begin with the way the strategic document (the Joint Strategic Needs Assessment) summarises and embeds the latest evidence base in the process to help commissioners to identify pressing needs.
The Joint Strategic Needs Assessment in Rosetown is recognised as a key document (evidence entry point) which pulls together (inter)national research evidence, local epidemiological data, identifies priorities, trends and areas on which to focus activity to reduce health inequalities and inform strategic planning:
We’ve got the JSNA [Joint Strategic Needs Assessment] and the annual report every year, so that’s the first set of evidence that we use, erm, in terms of health inequalities and all the things that we need to concentrate on, so we set our priorities based on that, and I just think the weight of data that we have, that is important.
LA 38
It was hoped that the Joint Strategic Needs Assessment would be seen as a ‘living document’, which would evolve and change over time as new intelligence became available. To support this, the Rosetown PCT and the Rosetown city council developed a web-based system to enable partner agencies and stakeholders to access information about health and social care needs across the area. The public health team were seen as playing a central role in this process:
Public Health lead on the, you know, the JSNA for the city and its public health, definitely, they are absolutely the most crucial lynchpin for evidence-based information.
Commissioner 3
The Insight team: developing actionable messages
The Department of Health in 2006 developed a Healthy Foundations Lifestage Segmentation Model of the adult population of England to capture a range of attitudinal, psychological and environmental drivers of health-related behaviours to inform local and national health improvement activities. 214 This work is carried out by the Insight team. Alongside colleagues in Public Health who ‘crunch all the raw data and provide an overview’ (commissioner 3), the Insight team present themselves as an integral part of the commissioning process in terms of designing and developing ‘actionable outputs’:
The epidemiology team [. . .] so they do the overview [. . .] we do the more sort of detailed understanding of characteristics, so people profiles, where they are and so on in a particular area [. . .] none of us could do what we do without each other because we have to work collectively, in terms of the raw data and then refining, and then the detail, and then the marketing, in terms of actionable outputs. So it’s quite a streamlined process in terms of the way we work.
Public Health 36
The ‘people profiles’ referred to in this quotation are gathered using Mosaic Public Sector Profiler (a socioeconomic classification tool that uses over 400 data indicators to classify the UK population into 15 groups and 69 types). The local Mosaic profile and assigned characteristics of these groups are used to identify particular cohorts in highly defined geographical areas with specific behaviours, attitudes and lifestyles. Targeted research is undertaken with these groups to help to design and develop local public health interventions, including social marketing campaigns. This localised approach seems to be particularly valued in the light of some of the perceived drawbacks of published research identified by some interviewees:
What comes out gives you some guideline, it [research] doesn’t give you the specifics of how you apply that to a commissioned piece of work or a service specification, so you know, it’s up to us to go away and really think well, what does that actually mean to us when we’re designing this piece of work.
Public Health 37
While it was clear that research evidence informed (and was embedded in) strategic documents (policy, strategy statements and Joint Strategic Needs Assessments) when setting strategic organisational priorities, by identifying the nature, location, prevalence, corelated factors and possible trends within a problem issue, at the level of commissioning (operationalising those priorities) research evidence seemed less useful. Knowledge managers in particular emphasised that different kinds of evidence are needed in the local commissioning process of public health interventions:
Because one of the things that is increasingly apparent around evidence commissioning, is that evidence for commissioning is less likely to appear in your sort of peer review materials. Which are all very good and they’re very nice and, will probably tell you what you need to know if you’re interested in a particular treatment, or a clinical issue, but aren’t very good when it comes to detailing how you might design a service. What sort of outcomes you should reasonably expect, how you’re going to measure the performance of a service, what you might include in the service spec. and that sort of material.
NHS participant 4
It is unclear if research evidence would be able to answer these questions, and at the point where commissioning decisions are made, commissioners find it challenging to identify what to do to address the problems which have been so carefully identified. The shift from knowledge to action is difficult. Participants suggest that when deciding on which interventions to commission, decision-makers tend to request a different type of information:
[. . .] what you almost want to know is, what’s the organisation that is comparable with you elsewhere in the country up to? What have they found out? And commissioners are very, very keen on that idea, they don’t want to be outliers, they don’t want to do anything particularly unusual that’s [not] been done elsewhere. And I think part of that’s because commissioning as a framework, they’re always acutely aware that there may be some legal comeback if they get it wrong.
NHS participant 4
Solutions (actionable messages) that are tried and tested and largely risk averse seem most favoured. However, learning from other areas’ public health activity itself has limitations; the sites that provide the exemplars have to be a ‘good match’. Rosetown members describe their position as unique. Anecdotally, ‘we are different here’ or ‘that wouldn’t work here’ is often heard. There may be reasons for thinking that particular solutions will not work, if local configurations are very different or if the scale of the problem requires a different policy (as opposed to service) intervention. For example, Rosetown has significantly higher alcohol-related hospital admissions and mortality rates than the UK average. The tension between having the ‘same’ problems and being simultaneously ‘different’ is illustrated in the following quote:
Because we’re just different from, we are the same, but we have a lot more issues I think than some places. Like, you know, somewhere down south is maybe doing a piece of work on this, it would probably, might look quite different to what it’s saying in [Rosetown]. Yes, so it wouldn’t work looking at somebody in another area, it’s not the same problem. And the kind of like prevalence about alcohol and stuff, isn’t the same here, it’s a real unique issue.
LA participant 3
Research evidence of proven interventions produced in very different locations are viewed sceptically for these reasons. Commissioners primarily want to know how any intervention will work in their location, with their people, at that time. In this way local Insight is seen as important to inform the development of realistic, acceptable public health messages in light of unclear or ‘cloudy’ evidence. This was important given concerns that alcohol and pregnancy are emotive issues.
So Insight would be really important I think, involving women in pregnancy or pre-pregnancy in that discussion [. . .] because, [. . .], what we need to consider is if you give a message that is completely unachievable, do you then kind of increase the guilt, and you know, anxiety levels of women in pregnancy, and what does that do? So is there a message that is more acceptable? So you know, I think taking a whole range of views might be more important than the evidence because I think the evidence is so cloudy at the moment.
Public Health 37
In this way, women’s involvement in determining ‘appropriate’ messages was seen as important, and Insight appears to provide the mechanism to access these views. While Insight work seems to be embedded in the public health planning processes in Rosetown, it was not without its critics. Some had questions about its scientific rigour:
Some of the weaknesses of the Insight work is that it’s a bit black box at times. It’s a bit unclear how the process is working to do the groupings, to do the segmentation, some of the leaps that are made in developing the hypotheses I think, well I know, that academic social scientists kind of like throw their hands up and say ‘well that’s not very good science.
Public Health 34
Another interviewee recognised that Insight provides a ‘rich source of understanding from local people’, but questioned the way findings from Insight appeared to be seen as more important than other kinds of evidence:
I think we’ve almost gone the opposite way round I think in the PCT [. . .], the Insight work is seen as very, very important, which it is, but it’s not more important than other things [. . .] sometimes I think of the tail wagging the dog [. . .] something which is immediate and erm that’s very sexy at the moment doing the erm Healthy Foundations Segmentation and everybody’s kind of social marketing is flavour of the month and [. . .] I wouldn’t say it wasn’t good, because it is good, and it’s very compelling [. . .] and sometimes it’s quite hard to read academic research and I think some people find it very inaccessible so that needs to be taken into consideration as well.
LA 4
Findings from Insight work are presented in an accessible, user-friendly format, with clear messages about what can be done, which stands in stark contrast to the perception of academic research as somewhat inaccessible and difficult to read. These comments shed light on the importance of presentational style with thought given to targeting messages for particular audiences.
The appeal of the Insight work was that it provided actionable messages, helping commissioners to decide what to do to address the issues identified. However, some doubt remains over what the Insight work has actually achieved and this critique allows us to see a level of local evidence that appears to be missing from consideration. The Insight model values the take-up and recognition by the target audience of the key messages (in this case, that excessive drinking in pregnancy is harmful). As one participant said:
This is one of the figures we’re most sort of proud of. About three-quarters agreed that the adverts were aimed at people like me, and they made me more confident knowing about cancer victims.
Example given by LA 3 from a previous campaign
However, this does not always provide information on the impact of commissioned interventions that relate directly to strategic priorities identified in the earlier stages of the commissioning process; for example, has a social marketing campaign not only made pregnant women more aware of the risk of maternal alcohol consumption but also reduced their drinking in pregnancy and the risks associated with this (e.g. low birthweight, FASD)? There are no feedback or evaluation data to illustrate the impact of the Insight work on health outcomes. As a consequence, some knowledge managers highlighted a need for more specific service evaluations that research longer-term effects and health outcomes, rather than relying on one approach:
Certainly seeing what impacts and what effects is important: has updating different services increased following on from this [intervention] and what are the outcomes longer term, have things improved?
LA 2
Packaging the evidence and appealing to the audience
The need to ‘package’ messages appropriately appears to apply at both national and local levels. There was recognition that having robust scientific arguments is important, but that they may not always be the most persuasive influence:
I think that having a robust scientific argument helps, but it isn’t sort of the whole story, erm er. My early learning on that about 20 years ago I was trying to get [. . .] some money to [. . .] continue a community project and erm er we took our report to the Board of the [. . .] Health Authority Board [. . .], setting out what were kind of like the advantages were and what we’d achieved erm but [. . .] what swung it was bringing in the children’s choir who sang to them for 5 minutes.
Public Health 34
The above example highlights some of the tensions between the potential power of academic, reasoned ‘scientific’ argument and emotional calls to action, which appear to have different effects:
It isn’t just the science that’s influencing decision-making [. . .] It is recognising what different places have got different receptiveness to you know more or less academic approaches [. . .] science and its rigour doesn’t work in lots of places [. . .] It’s a blend of the two, I think there’s a lot of science behind what we do, because we are at base scientists, but there’s plenty of evidence in Rosetown of doing things which connect with people in a different way [. . .] on a much more emotional level.
Public Health 34, emphasis added
Other interviewees referred to the ‘compelling’ nature of different kinds of public health evidence. For example, public health data on hospital admissions or alcohol-related deaths which were described as ‘shocking’ appeared to focus people’s attention in different ways.
Public health data there was very compelling because all of a sudden, we were looking at figures that were really shocking . . . hospital admissions and deaths.
Clinician 14
There was understanding by this interviewee that presenting tables of written figures might not provide most impact to local and national stakeholders, including ministers and policy-makers at national level. During a meeting where he was invited to give evidence to a Health Select Committee on alcohol-related harm, this clinician chose to illustrate potential health outcomes using photographs of babies from the local paper. His experience in clinical practice seemed to provide a powerful impetus:
I remember the death of a 27-year-old woman from liver failure, caused consternation and shock in the unit because we’d never seen it before. We couldn’t understand how this had happened, but that was the first of many.
Clinician 14
These examples highlight the diversity of ‘evidence’ in circulation at any given time, and the importance of interpreting and packaging information for maximum impact on its intended recipients in a way that prompts action. The potential audience for public health evidence is changing, given LAs’ ‘new’ responsibilities for public health. In the light of the need for public accountability, elected members are considered important stakeholders, who have different sets of interests and may require different approaches:
We do know that erm elected members have particular interests and particular constituencies, and er they are they’re very swayed by erm storytelling, so I’m not saying that storytelling isn’t necessarily evidence based, but erm I think it’s an interesting mixture about how much in future we use the hard evidence base and how much we use, that kind of er, more sort of you know qualitative or storytelling type of stuff.
Public Health 33
This suggests that the skills and expertise of public health specialists moving forward will need to include the ability to adapt to their changing environment to ensure that evidence is heard and understood and presented in a way which maximises its potential use.
I think it’s a real skill to be able to present the [research] evidence to the right people, but present it in a way that they understand and motivates them to, er, take it on board and do something with it [. . .] any longer than two pages and people switch off.
Public Health 35
This same interviewee later commented when preparing papers for the HWB on the need to ‘keep it simple, and clear, no jargon’. Another respondent stated that it was important to keep the message ‘short and snappy’. The abilities to be concise and to make a persuasive case using plain language appeared to be valuable skills. Certain members of the knowledge management team had a reputation for being able to do just that, and they often became the people approached to search out and collate evidence. So, while the organisation creates knowledge management roles, it is the agency of the people who fill them that makes the role effective. However, reliance on one or two people may suggest a person-dependent system rather than robust organisational provision to fulfil these tasks:
Somebody will phone, somebody will e-mail, quite often you’ll get a conversation as a preface by, I’ve been told by such and such that you’re the man to talk to.
NHS 4
Blending evidence
A number of interviewees in different roles (Public Health leads, health professionals, managers and commissioners) recognised the importance of blending ‘hard’ evidence with ‘softer’ intelligence to take account of the preferences of some stakeholders, including elected members, who may draw on different sources of localised knowledge from public health professionals:
I use the experience that I’ve got of actual people because I know what’s working and what isn’t working and how people live that I meet every day in my ward and in the surgeries and where I live and the people that live with me.
LA 38
The day-to-day contact that elected members have with their local constituents was seen as an important channel of feedback and communication. A related point was made by a clinical manager when he or she was asked if research always holds the best solutions. The manager recognised that while evidence-based research was important, professional expertise and tacit knowledge gained through clinical experience also have their place:
. . . we will take account of that [research] as well as what we know anecdotally and from experience and knowledge of our own population so obviously research has a part and you know that’s what we’ve been doing over the last few decades isn’t it, using evidence based research and that is seen as the gold standard of the moment but is there a place for other things. I mean you know sometimes you can’t say why you know things can you, [. . .], there’s something about that, whether it be intuition or experientially saying ‘but in my experience’ you know.
Clinician 2
Commissioners with responsibility for public health seemed to value a mix of evidence and recognised the importance of bringing different sources together to inform decision-making:
We have great evidence and we have fabulous evidence, we don’t bring it together in one, so that for me is around some small anecdotal information could feed into something very important. We don’t bring in very small snippets. We seem to have very strategic overarching information, but there’s not a combination between qualitative and quantitative and I think we do need that.
Commissioner 3
This raises questions about how this ‘anecdotal’ evidence, tacit knowledge and intuitive experience is captured and fed in to strategic planning discussions, given its potential value. It also highlights the challenges of inclusive, evidence-informed decision-making processes which take account of the diverse range of available evidence without privileging one set of interests over others. It requires an open, inclusive approach which values difference; however, talking across the purchaser–provider split may be more difficult.
There was this recognition of the value and importance of open, honest dialogue with stakeholders. Participants believed that the tensions and complexities of managing sensitive negotiations across the purchaser–provider split between commissioners and service providers made this difficult:
I think if you’ve got all the sort of high level decision makers round the table, you need to be able to have those open and frank discussions and whether you can do that in the same way when you’ve got potentially people at the table whose jobs will be affected by those decisions would make it untenable at that time.
Public Health 35
A backdrop of political influence
One interviewee was clear that evidence ‘first and foremost’ should form the basis of ‘all that is commissioned’. While she felt that it was important to take account of the changing need and changing demographics in the population, she noted that politics also came into play:
I think also, and perhaps this will happen, you know, it will happen more now we’re in a local authority arena, is there’s a political element, I suppose, to what’s commissioned. And it’s understanding that better, erm, and balancing that against all those other factors, evidence, need and being a bit savvy, I suppose, around, erm, negotiating all those different areas.
Public Health 35, emphasis added
In addition, there was recognition from this interviewee that, ultimately, local political sensitivities can shape services:
I would probably suggest that some things are allowed to continue, maybe with a weaker evidence base, because politically it might be quite sensitive to withdraw services. So it’s a constant, erm, balancing act really about, erm [pause] where the evidence is, but also with the politics in the background.
Public Health 35, emphasis added
Senior local political support
Research participants acknowledged the benefits of having senior-level strategic support and buy-in from politicians and elected members so that alcohol remained a high-profile public health issue for the LA. During fieldwork, a senior local political official was observed making clear public commitments to tackle alcohol-related harms. Investment in the public health post of strategic lead for alcohol had been protected at a time of efficiency savings and squeezed budgets. An existing member of public health staff had been moved to cover these responsibilities when the previous post holder left.
Rosetown has a history of evidence-based commissioning and a national reputation for high-profile research around alcohol. This has allowed the site to make significant contributions to national debates and health guidelines. This leading national role influences local commissioning practices and sets the framework for local health priorities and interventions to address these priorities. However, ironically, this does not always make it easy for local commissioners to explore new avenues for public health interventions when it risks upsetting the national avenues paved by local heroes:
. . . one of the challenges for [Rosetown] has always been, and particularly around alcohol, there are a number of, I describe them as national pillars [. . .] and these are big services, people on NICE groups, people contributing to the national understanding of this, but from a commissioning point of view, you wouldn’t want to be a commissioner because you’d have to sort of really play with the diplomacy around how to get a whole system working, when certain points in the system are irremovable.
NHS participant 1
This suggests that, even when national guidance and evidence is informed by local research and has local senior support, it is still not always easy to ‘get it to work’ to inform local commissioning decisions. Something in the process by which it is turned into guidance or national policy distances it from local need:
. . . and it’s [national guidance] very much developed on the basis of local data [from Rosetown] and knowledge and that sort of stuff. But in a way, it’s got nothing to do with the local area [. . .] the funding has inevitably come from regions and the centre. The projects have been born out of what local areas tell us is relevant and interesting and necessary. And then the product becomes something that goes back into the centre, turns into policy and then gets pushed down to deliver as sort of guidance.
NHS participant 1
The choice of case study process
There was nothing straightforward about the choice of case study process; on one level, the choice appears rational:
We looked at our own strategy, and there’s stuff in there around women and pregnancy and we felt, well actually, that is one area where very little in our strategy is being delivered.
Public Health 37
However, the selection of ‘reducing maternal alcohol consumption’ as the case study process was more complicated than simply stemming from the desire to rationally develop a service or intervention. It simultaneously brought together a number of issues facing practitioners to offer a practical focus. It was a pragmatic decision by senior Public Health managers to try to ‘get a grip on something’ which could be delivered in 1 or 2 years, in recognition of the continuing uncertainty facing public health and the potential to ‘get the Health and Well-being Board up and running’. These decisions emerged amid questions about the role and responsibilities of the newly formed HWB:
Apart from kind of a stamp of approval, imprimatur or whatever, what does the Health and Well-being Board do? [. . .] I suppose the crucial thing is how the Health and Well-being Board develops from now on, and how involved it’s going to be, and how hands-on.
Public Health 33
Other interviewees in public health shared the idea of using the Health and Well-being Strategy as a mechanism to proactively focus on specific actions to reduce maternal alcohol consumption, which could provide ‘added value’ while being seen to ‘do something’ and engage a wide range of stakeholders in the process:
The idea behind the Health and Well-being Strategy is obviously to improve health in the city, but in this first year, our prime goal is, (1) to do something that looks sensible and (2) get some engagement with it from different bits of the system.
Public Health 34
In yet another way the choice to focus on women and pregnancy was interesting. While the topic fits neatly with the Health and Well-being Strategy cross-cutting theme of children, young people and families, it was difficult to show robust evidence of need (i.e. as identified by the Joint Strategic Needs Assessment), as there were identified gaps in local intelligence (no collated routine data capture) – the focus was based on an assumption of need and an extrapolation that if the general levels of alcohol consumption in the population were rising, then it followed that some of those drinking problematically would be pregnant women. There is some research evidence that suggests that the opposite may be true: that pregnancy is a teachable moment, when women change their behaviours and (even if only temporarily) adopt a healthier lifestyle (i.e. in this case consume less alcohol). 127
We felt that given the levels of need we had in the population generally around alcohol misuse, some of that must be transferring over to the pregnant population. But we weren’t able to support that by looking at the local intelligence from our maternity services.
Public Health 35; emphasis added
This observation echoes national research findings which show limited understanding of levels of alcohol-related harm in pregnancy, leading to a call for improved data collection and activities to increase awareness of the risks of drinking in pregnancy among women and with midwives. 13,151 This particular research article (the Morleo et al. 2011 paper151) was mentioned by one commissioner as instrumental in her choice to focus on reducing maternal alcohol consumption. This illustrates that through individual agency research evidence can be directly pulled into commissioning decisions, and that even reasonably strategic decisions can be driven by assumptions and not by hard data and research evidence.
Understanding gaps in routine data capture and missing local evidence
It is worth taking time to explore reasons for the gaps in the data regarding the prevalence and scale of alcohol consumption in pregnancy, as it highlights the dilemmas faced by our commissioners and providers and the need to commission work in this area. One knowledge manager suggests that frontline staff do not value or prioritise the recording of routine data because they misunderstand the data’s purpose and role in informing service development and, therefore, see data recording only as an additional chore:
I think there is a bit of a disconnect really between commissioners and what managers expect from services, and actually what’s delivered on the ground [. . .] It’s kind of getting them to understand the process of why you ask questions and what they would need to consider if they were doing [commissioning]. And I think by the end of it, particularly for the practice nurses, they did kind of say, ‘oh I kind of see why you ask these stupid questions now, you know’. ‘Why do we want to know what the ethnicity is, why do we want to know all those other kind of things we’re doing on the frontline’. When they’re delivering the service or a clinic, they just go, ‘oh it’s just another box to tick’.
LA 2
However, in talking to frontline managers, despite maternal alcohol consumption being identified as a priority for commissioners, maternity service providers noted that they were not asked to collect, collate or return relevant data to commissioners to inform their decision-making, as shown in the following quotation:
They [commissioners] are not asking us for alcohol consumption. So actually, they’re not asking us for it, so that tells its own story, doesn’t it? [. . .] If we were getting some direction, if they asked us for it, we’re collecting that, and we could do more about let’s make it meaningful with the questions of ‘so what’s occasional mean’, but we’re not being asked for that, for us, we just record it, and it sits there, we don’t do anything else with that.
Clinician 2
Interestingly, the service provider felt that these data should belong to the commissioners rather than being data they themselves could proactively act on. So, across the different parts of the system, knowledge managers will value the information but cannot request it, while maternity service providers do collect it, but do not send it to commissioners. There is clearly a level of disconnect.
The notable perceived absence of anyone with power and influence ‘banging the drum’ about alcohol use in pregnancy suggests that there may be differing priorities driving maternity service providers. This raises questions about the public health roles and responsibilities of specialist secondary care providers, and about how public health intelligence from different parts of the system is gathered and used. Given the difficulties in identifying the effects of harmful alcohol use in babies, except in the small number of cases where fetal alcohol syndrome is diagnosed, it highlights the need for shared, co-ordinated data recording and monitoring systems. The importance of using monitoring data, despite their acknowledged limitations, was recognised by both commissioners and service providers, but in practice this seemed difficult to achieve across partners.
Using evidence at the front line
Despite commissioning work to explore why evidence use at the front line was patchy (the Insight work), participants already had views about what was happening. This helps us to understand why local research was undertaken despite a plentiful evidence base outlining barriers to evidence use at the point of care delivery. A senior practitioner reinforced midwives’ willingness to address alcohol issues, but identified conflicting guidelines as problematic:
We already ask the screening question and we already give advice. I think we can step up the information we give around alcohol and the dangers but we need more information about that to be able to give that.
Clinician 2
While it is true that the lack of clear evidence hampers midwives’ efforts to provide clear unequivocal advice to women, it is unlikely that this is the only barrier. Managers, clinicians and commissioners reported a range of explanations for health professionals’ apparent reluctance to address alcohol issues. Some clinical staff believed that data collection was not prioritised by midwives because of the tyranny of small numbers, that is, insufficient numbers of women presenting at maternity services to show significant evidence of the harmful effects of alcohol on babies:
It’s not something they [maternity hospital] perceive as a real issue, I don’t think [. . .] Well if they’re not seeing enough, I think the idea that you’re seeing lots of drunk mothers giving birth to deformed babies, it probably isn’t happening on a scale like that [. . .] I don’t think they’re seeing huge numbers, erm, there’s nobody who is banging the drum.
Clinician 43
The social context, described by one clinician as ‘incredibly powerful’, was considered to have an impact on health-care professionals who are reluctant to be seen to ‘lecture’ people:
It’s something that we’re not, nurses and doctors, haven’t traditionally been very good at, asking about alcohol, and therefore probably not as good at picking up and labelling alcohol related diagnosis [. . .] It’s just typical British awkwardness, and it’s just like, well, where are we going with this? And it’s like the nanny state and I don’t want to be boring and I don’t want to lecture.
Clinician 43
Drinking during pregnancy may conflict with professionals’ own values or, conversely, their drinking behaviours, which may result in them feeling hypocritical; they may question the right they have to ask women about their drinking, let alone to challenge them about it. Drawing on her experience of smoking cessation, one interviewee felt that ‘lecturing’ women was viewed negatively as it was seen to jeopardise the practitioner’s professional relationship with the woman:
A lot of the health visitors and midwives that I did [smoking cessation] training with said it has an effect on the relationship they have with that mum. Because it’s like I’m lecturing them, and you know ‘you shouldn’t be smoking’.
Clinician 44
If the woman did not return for further care, then any influence or support that the midwife might have given would be lost. Midwives felt that asking the difficult question around alcohol consumption was not worth risking this for.
It’s not an easy discussion you’re having anyway, and you certainly want that woman to come back.
Clinician 2
The recognition shown by interviewees of the sensitivities of discussing alcohol, smoking and domestic abuse, and the parallels drawn between them, may indicate that there is scope to address multiple risky behaviours by providing training for health professionals which may result in increased confidence across the board to raise difficult issues. These comments suggest there is some way to go before discussions about alcohol are embedded in routine care:
Midwives are in society just like everybody else [. . .] I don’t think we’re there with it [alcohol] yet. I think we’ve still got work to do . . .
Clinician 2; emphasis added
The following comment by an experienced health professional identifies the challenges that commissioning organisations (and national bodies who synthesise evidence) face in trying to influence the everyday practice of service providers, even where the evidence base is well established:
. . . but we’ve actually had two sets of recommendations, and they’re [midwives are] still not doing it . . .
Clinician 40
This suggests that although training, clear guidelines and fear of damaging the therapeutic relationships are factors that affect the use of evidence at the front line, other factors are also at work, including, perhaps, resistance to imposed change. We explore some of these issues below.
Other implications of a contested evidence base
The evidence about the effects of drinking alcohol in pregnancy is unclear and contradictory, and professional guidance about how much is ‘safe’ to drink varies:
Because of our current evidence, any woman might say ‘well, it’s OK to have a glass or whatever’ and I think that’s confusing for women.
Public Health 37
Both women receiving care and the health professionals providing that care may be unclear about what current guidance recommends about unit measurements and be aware of published research which indicates the beneficial effects of small amounts of alcohol. 139 This makes alcohol more ‘complex’ than tobacco, for example, in terms of public health messages, as there is clear unambiguous evidence that all smoking is bad.
This lack of clear evidence about alcohol consumption in pregnancy led some participants to express concerns about the possibilities of developing consistent messages for women, as ‘getting people to make behaviour changes relies on clear, simple information about what to do’ (Public Health 37). The same interviewee highlighted the wider social implications of alcohol-related harm in families (where there is more robust evidence) providing a rationale for early intervention to reduce drinking:
If there’s not very clear evidence, I would suggest that, you know, erring on the side of caution in pregnancy might be a good thing, but you know the evidence around drinking in pregnancy is not just about the impact on the foetus, although that’s a major impact, it’s also the impact on the family.
Public Health 37
Some participants suggested that the ultimate reason for not asking the awkward questions was that midwives did not want to hear the answers. Hearing that a woman has a problematic relationship with alcohol while pregnant places the onus of professional responsibility on the midwife to do something about this, yet there were only a few onward referral services, as residential places were limited (11 places).
Midwives were very reluctant to talk to women about this because, (A), they don’t know how to talk about it to them and, (B), they don’t know what to offer, in terms of services afterwards.
Academic participant 5
A small number of interviewees (commissioners and providers) did appear willing to challenge national guidance in England on pregnancy and alcohol, as its changing nature was felt to be unclear and unhelpful to women and midwives, and to suggest alcohol abstinence during pregnancy:
I feel very strongly we should be saying zero tolerance and we should be saying that, but it keeps changing about whether it’s a glass a week, two glasses a week, and that muddies the water for women, what’s one glass for me might be a different glass to you.
Clinician 2
The risk of this approach is that precautionary messages about drinking in pregnancy do not stand up to evidence-based scrutiny and they may be seen as protective, value-laden, moral judgements165 which are open to challenge, especially if they are at odds with national or international guidelines. From the perspective of this study, it is curious that local research is commissioned (the Insight work) to replicate a plentiful evidence base on the barriers to evidence use at the front line, especially given that the quotations above illustrate that the commissioners and other key stakeholders anticipated (and largely seemed to already know) the barriers that would be identified.
Imposed and challenged evidence
Hunter215 argues that the imposition of evidence-informed practice may be simply a way of driving imposed change on professional practice, to reduce professional autonomy and the discretion that professional practitioners’ may exercise in their practice. The extent to which health-care/public health activities are centrally driven and how much room there is for local discretion and negotiation is mentioned by participants:
I’m not sure about other organisations and how they approach it, but from an NHS perspective, sometimes a lot of the evidence, or then what happens as a result of the evidence, can be centrally driven. So you get your guidance, you get your directive and you just, you have to do it type of thing [. . .] so it’s top down.
Public Health 35
However, when asked, one NHS clinical manager indicated that even ‘top-down’ messages were not adopted unquestioningly in her trust. ‘We do comply with most NICE guidance but we don’t comply with every bit of it’ (clinician 2). This indicates a willingness to challenge evidence, including national or international guidance, particularly where it appears to not ‘fit’ with the local population or where locally based research is available:
Absolutely we do challenge evidence here and we don’t always go with what it says because we’ll have done our own evidence around our own cohorts of women so often now I’ll say ‘where did we get that from?’ and [. . .] we did our own research around it and actually this is what we’ve shown here for Rosetown maternity hospital, I know this is what national guidance is telling us or international guidance or research but actually when we’ve compared that with what we do that actually doesn’t fit for our population [. . .] so we challenge research in that way.
Clinician 2
The trust in question was a large specialist teaching hospital, which was presented as an organisation willing to question existing practices and change policies and procedures for the benefit of patients. Sometimes, lively debates between senior colleagues were reported to precede decisions about the implementation of changes in policy or procedure, especially where disagreements occurred. One of the senior clinicians took on the dissemination of research evidence as part of his role, effectively paving the way for operational changes ‘on the ground’ – further evidence of a person-dependent system for ensuring research utilisation at the front line:
I think an understanding or having seen it first hand, that evidence base work does make a difference . . . I think you need to have an environment where you can challenge and champion evidence.
Public Health 37
Being part of an environment where using research was part of everyday interaction, and had shown positive benefits, seemed to help it become expected and normalised.
Evidence champions: trusting the person, trusting the evidence
Our findings show that it is easier to secure evidence uptake if a champion or team of champions with sufficient power, credibility or ‘clout’ introduces the evidence, discusses its implications and ‘pushes’ evidence-based practice and decision-making. This may be especially important where significant changes in practice are required or where staffing resources are particularly stretched:
It’s generally you know a conversation or if some evidence is we’ve then changed a guideline because of it, he may be involved in saying, “I’ll tell you why we’ve changed this guideline, that sort of thing.
Clinician 2
Having an understanding of the importance of effective communication and visionary staff in pivotal positions to explain any changes which are being implemented has been shown to help adoption,71 so this bridging role seems critical to oiling the wheels of successful change management. In Rosetown it was not always clinicians who fulfilled this role, nor did it apply only to practice. A ‘research-positive’ approach by a proactive chairperson in a key organisational position (senior decision-maker) was considered able to open doors to research being introduced and used:
He’s [the chairperson] very engaged with new evidence, new research, to support future practice, and sometimes having those ambassadors on board at that high level, really does, you know, push that agenda forward. It’s having some champions, I suppose, isn’t it?
Clinician 2
The ability of powerful individuals to effect change is demonstrated in the following quotation from a senior NHS leader who understands that different drivers and incentives can lead academics and service providers down different paths. He saw the benefits and tensions of bringing them together and using evidence to address public health issues:
So yes, so you’ve got this University Institute, I’ve been very keen to bring that interconnection closely with the public health colleagues particularly, to get the insight from the university, with university people trying to work side by side with the commissioners [. . .] They have to sit next to a commissioner to see how the research might be of benefit and value, even though, as you know, there’s a tension between people wanting to do world-class research, etc., on the one hand, and […] working with the commissioners on this year and next year, rather than, you know, 10 years ahead [. . .] So as a way of trying to encourage the commissioners to use academic literature, background, insight, and get academics to focus on how their work might actually have impact, which they need to do for their Brownie points.
Commissioner 42
Understanding of the research process, the range of evidence available and the different reward systems in operation in different settings, as well as familiarity with the context in which research evidence may be applied and the potential interests involved, was seen as helpful in other ways. Critical appraisal skills in assessing the quality of different kinds of research evidence were recognised as valuable, and as skills that could be acquired:
You need to make a judgement of that research; [. . .] and if you’ve got skills in the organisation to do that then you ought to be getting robust research which actually erm is er going to erm which you can rely on, which you can trust and I suppose it comes down to trust doesn’t it, it’s trusting the research, [. . .] it’s really difficult with individuals unless it’s someone you’ve known for a long time and you trust [them].
LA 4
Trust and reputation featured in a number of ways for commissioners and providers in their assessment of research validity, and the source of evidence was important in helping them determine its credibility. Clinicians appeared to rate as trustworthy evidence produced by other clinicians21 or their representative body, as well as ‘reputable’ national evidence providers such as NICE. In Rosetown, the added benefits of having local consultants involved in the production of NICE guidelines were recognised as placing local providers ‘ahead of the game’ in some ways:
We’re very privileged in Rosetown in that a lot of our consultants actually sit on a lot of the NICE guidelines panels and things like that, so we are often very aware of what’s happening nationally before it comes out, we’re almost one step ahead.
Clinician 2
This ‘ownership’ places an onus on these organisations to identify and disseminate up-to-date, credible evidence, but also suggests that they have considerable power to influence opinion among their followers, especially where they are known locally. NICE was not the only organisation identified in this way, and providers listed others they used, including online resources such as Map of Medicine pathways:
. . . it’s an online resource, there’s a clinical group of people that produce it. Erm, but one of them is one of our partners, so I know that it’s, [. . .] about being reputable, isn’t it? And knowing that, you know, so generally if it comes through NICE, or from the Medical, Medicines Management Team, erm, you know, or through the CCG or the Local Medical Council, people that you know, that it’s reputable, those sort of people I think.
Clinician 41
While supporting the use of Map of Medicine, this interviewee felt that there was no substitute for personal experience when it came to implementing ‘evidence-based medicine’, admitting that ‘I’m always happier when I’ve tried it and I know that it does work’ (clinician 41; emphasis added).
However, even where local clinicians, with substantial expertise, were involved in the development of national guidelines, for instance, there were no guarantees that they would necessarily be in a position to inform local practice, especially where these challenged traditions and routine practices:
You’ve got to have the clinical and research expertise that gives you the credibility to say what you think, and even then sometimes it can be not, not as respected I think, and I think that is a big problem, because if it’s conflicting with what has been planned before they came in the room, then you’re perceived as a bit of a pain . . .
Clinician 1; emphasis added
Taken together, this suggests that there are numerous factors influencing the take-up and use of evidence, including the way it is presented, who presents it, to whom, in what setting and who is present at the time, so to a certain extent the use of evidence is a person-dependent system, reliant on the interpersonal relationships, values, ideas and opportunities available in a context at any given moment. Evidence use depends on the willingness of busy practitioners to seek out the evidence, find time to appraise its value and relevance, discuss and make choices about its implications, and use their power and influence to effect change, as noted by an interviewee working in primary care:
You have to have the time to read the evidence, and then it depends [. . .], if you’ve got the time and the head space to implement the changes. Because it’s often you revert back to type, you know, if you’ve not got the time to [. . .] assimilate and take the bits that you really think are appropriate. And then you have time to set up educational sessions and impart that on to everybody else. And everybody else has time to reflect on it and you have time to reflect to make sure it’s happening.
Clinician 41
This quotation illustrates that evidence-informed changes do not simply happen but that considerable effort is required to implement clinical and organisational changes ‘behind the scenes’, actively supporting their implementation and monitoring their continuation. This, in itself, places increasing pressure on practitioners’ workloads.
Financial pressures
The reality of being part of a LA facing large reductions in budgets, and the uncertainty about the long-term future of public health funding, made forward planning challenging in the face of competing priorities and influenced commissioning choices. There was anticipation that by the end of 2014 the council was potentially facing budget cuts of up to 52% on the previous year, with further reductions likely as a result of changes in the national formulae for allocations, moving away from a focus on tackling socioeconomic disadvantage and towards supporting health and social care for elderly populations.
Instead of holding out for the prospect of large-scale investment, there was recognition of the need for a different approach, influencing and persuading others in the organisation of the benefits of shifting spending upstream into prevention and early intervention. This suggests there is scope to make use of the evidence base about cost-effectiveness to inform public health decisions, although this interviewee was under no illusions about the challenges that this presented as money got ‘tighter’:
But with only 18 months of guaranteed funding, we don’t know what’s going to happen to public health funds after 2015. Having big plans isn’t probably sensible at the moment [. . .] The majority of our money is tied up in clinical contracts, and err, it’s not going to be easy to extricate ourselves from those even if we wanted to. I think the scope for changing how the money’s spent isn’t, [. . .] about wholesale shift in how public health resources are spent. It’s more about how we persuade the rest of the organisation, how we shift, you know, other spend upstream but that’s going to be more and more difficult [. . .] I don’t think anybody’s got the capacity, or has the confidence in the long-term budget to make big plans for investment [. . .] Money’s so tight and likely to get tighter.
Public Health 49
Using evidence in commissioning across a purchaser–provider split
NHS organisations providing antenatal and maternity services fully acknowledged the role that they had to play in promoting public health, acknowledged that there was work to do on ‘both sides’ to build connections and indicated that they would appreciate opportunities to proactively engage with public health commissioners.
There’s something around identifying where maternity and neonates, where we sit in the whole big public health of [Rosetown] so there’s work on both sides, what would be useful is to have some information from public health.
Clinician 50
However, as noted earlier, seating commissioners around a table with potential providers can be seen to create a conflict of interests (see Blending evidence). Yet given that CCGs and LAs will be making commissioning decisions which could have profound implications for the entire health and social care economy in a given geographical area, a degree of co-ordination seems imperative. The challenging logistics of influencing the contracting negotiations of other organisations are clear, as shown in the following quotation:
When the contract’s with another organisation, or when it’s a national contract, it’s much more difficult then, isn’t it, to put in extra requirements [. . .] and then we’re not part of [. . .] a lot of those discussions and negotiations go on when we’re not there.
Public Health 49
This suggests that using the contractual lever offered by commissioning to secure precisely the services commissioners know are needed is not always a possibility. This bears out earlier research which is critical of commissioning as a mechanism for securing services across partners. 158 Contracts may well go ahead with little or no tailoring to meet the needs identified by local evidence. The importance of having conversational spaces, ideally face to face, to influence delivery, then becomes obvious in the light of the limited contractual levers available. 50,53,84,216
You can only influence, that’s why you’ve got to work with them [foundation trusts].
Commissioner 52
Despite commissioning’s appearance as a hard contractual lever, commissioners were clear that its success still rested on soft negotiation and influencing skills. These discussions and relational factors are not a circumvention of the correct commissioning process but a key mechanism to secure co-ordination and practical input. The ability to build trusting and credible interpersonal relationships has been shown as key in facilitating ongoing connections between all stakeholders. 20 Our data cast doubt on the possibility of achieving agreed public health provision across the purchaser–provider split.
Changing structures: a new focus for public health?
Participants expressed concern that, at the moment, what was needed was a sense of long-term stability and a clear indication of funding moving forwards, to allow planning to be realistic. It is debatable whether or not the desired period of stability will ever come about. In the short term, at least, maybe what is needed is the ability to respond continuously to change and learn, and adjust incrementally. In the absence of certainty about planning and pessimism about future funding in the short term, senior public health figures were considering focusing their efforts and attention upstream. Their belief was that more change would be possible by influencing national policy:
I think if I was thinking of radical change in public health, I would probably go back to policy more than commissioning and contracting [. . .] I think what we’re going to do is, just about manage to keep the system going [. . .] it’s about managing, about keeping services going, rather than saying we can invest in a whole range of totally new ideas.
Public Health 49
This was markedly different from the observations made by the same interviewee in her first interview, when she commented:
It’s a bit of a waste of time challenging national policy at the moment you know, it’s not really going to be particularly effective, and actually the thing we can do in Rosetown is do our own thing, do it our own way and do the best we can do [. . .] and you know, take note of national strategy, but you know we’ll try and do it better ourselves, not much point in just haranguing the government, because that isn’t very useful.
Public Health 5
The reasons for this marked change in view is difficult to gauge, but seemed to be part of the pessimism about not having sufficient resources to innovate locally to impact significantly on health outcomes. It was discussed earlier that the scale of public sector reforms and the economic downscaling had created increased workloads and job insecurity for our participants; here we begin to see the impact on commissioning and planning and the very focus of public health activity. The quotation implies that with limited resources, directing these at changing policy (and, therefore, the wider determinants of health) is a better use of those resources than trying to focus on local intervention to effect behaviour change.
A stalled commissioning process
In practice, we did not get to see this commissioning process from start to finish in live time, as was our intention. The commissioned piece of work was delayed for a number of different reasons and we were unable to fully follow the process to its end. The market research company had difficulty recruiting midwives, pregnant women and new mothers who reported drinking in pregnancy to take part. The findings from this commissioned work were due to be presented to local commissioners after our study ended. It was anticipated that this would include a practical set of recommendations which would guide future decision-making and commissioning of interventions in Rosetown to address maternal alcohol consumption. We return to discuss alternative reasons for the delayed commissioning work in Chapter 7, where we reflect on the process of cocreation.
Chapter 6 The Delphi process, the Behaviourally Anchored Rating Scale questionnaire and the national seminar
Introduction
The project design permitted an in-depth look at research utilisation in two case study sites (across two policy contexts). As what counts as evidence, and how it is used, is heavily contextually influenced, it was crucial that attempts were made to explore the transferability and fit of the case study findings in other contexts. In Chapter 2 we discussed the idea of public health issues as ‘wicked problems’, where issues defy precise definition (e.g. poverty) and stakeholders hold varying opinions about the causes of problems (and, therefore, the effective solution). These wicked problems may well take slightly different forms in different contexts, and any solution is only ever partial and temporary as the complex and multifactorial issues morph and rise again (e.g. to use an analogy, squeezing a balloon in one place simply shunts the air to make the balloon bleb up elsewhere). In the analysis and discussion of the findings from the case studies, we identified 10 frequent issues involved in identifying, weighting and using evidence (considering similarities as well as differences across the cases: nomothetic and idiographic data). Interestingly, whereas the issues were common, how they manifest themselves was subtly different and the actions taken to address them varied. For the same ‘problem’ different solutions worked in different places (perhaps even in the same place at different times). With no right or wrong definitions or ‘solutions’, these 10 issues can be considered ‘wicked problems’. They are listed in Table 10 and informed the questions on the Delphi. We identified the ways in which the issues are typically, if temporarily, resolved in the case studies and used these to write behavioural anchors for each issue in a modified Delphi questionnaire. We included an eleventh item to gather the views of respondents on cocreating research.
| Item | Wicked issue | How this manifests itself in the case studies | Behavioural anchors (practical solutions) | |
|---|---|---|---|---|
| Evidence utility | ||||
| 1 | What type of evidence is most useful? | Salience, trustworthiness and utility. Evidence collected here and now vs. evidence created at another time and place | National evidence | Local evidence |
| 2 | Where do I find the most useful evidence? | Range of sources. Few favourite sources vs. wide sweeps of multiple sources | One trusted and tested source | Wide variety of sources |
| 4 | Is the evidence ready for use? | Tailoring and fitting the evidence. Are the messages in the evidence clear? | Evidence speaks for itself | Evidence needs interpretation |
| 5 | What is ‘valid knowledge’? | What ‘counts’. Externally coded vs. situated and tacit knowledge | Practical experience | Academic evidence base |
| Evidence mobilisation | ||||
| 3 | How is the evidence mobilised (and by whom)? | Structure or agency. Organisational systems or person-dependent processes? | People | Organisational systems |
| 6 | How is the evidence mobilised (and by whom?) | Managerial mechanism. Via which mechanism for securing evidence informed activity | Commissioning (purchaser–provider split) | Joint planning (unified organisations) |
| 8 | Where is the evidence mobilised? | Upstream or downstream focus. Where should limited time and resources be focused? | Policy change | Individual behaviour change |
| 9 | What are the contextual influences? What factors compete to shape the decision? | The role of competing forces. What factors compete to inform the decision? | Research evidence | ‘Politics’ |
| 10 | Why is the evidence mobilised? | Aim and purpose of evidence use. Reasons for use/objectives, desired outcomes | Health gains | Economic well-being |
Delphi questionnaire (first round)
The Delphi process: issue of the Delphi questionnaire
(For a detailed description of the issue of the Delphi questionnaire, see Table 9.) The first issue, despite being wide ranging, resulted in only 18 completed questionnaires (and 15 registrations for the national workshop issued at the same time). A second targeted approach issuing questionnaires personally to all alcohol LA leads across England and Scotland (with an entry to a prize draw) and also circulating them more widely through social media generated 16 more responses (n = 34). At the national workshop these 34 responses to the Delphi were used to illustrate the issues and to prompt discussion. Of the 15 workshop registrations, 10 delegates attended on the day. A second vote on the Delphi was taken at the workshop (n = 10). Given that the numbers of the two-stage Delphi were very small (first stage n = 34 and second stage n = 10), we do not include the scores here. For interest the scores are presented in Appendices 3 and 4. Appendix 3 shows the changes in perceived importance of the different items between votes 1 and 2 and Appendix 4 shows the changes in the overall scores on the different items.
In a further attempt to generate data on the transferability of the findings from the two case study sites, the questionnaire (as a standalone instrument) was issued more widely across the UK. A full description of the issue of the questionnaire can be found in Table 11. This table also gives a timeline to show how the wider issue of the questionnaire followed the initial disappointing returns from the two-stage Delphi process. Every respondent who completed the questionnaire was entered into a prize draw and the winner was drawn on 25 November 2013 after all of the entries were verified. The winner of the prize draw was a respondent located in Scotland and received an iPodiD Touch 4th generation (Apple Inc., Cupertino, CA, USA) (identity of winner withheld). Taken together, these extensive efforts still only generated an additional 39 responses (34 + 39 = 73 completed single-stage questionnaires in total). It is these 73 responses we use in our analysis below (unless otherwise stated). Care should be taken when interpreting these results as the numbers are small, but the characteristics of respondents, the importance they attached to each issue and the scores across the 9-point scale are all identified. Findings across respondents from different sectors are also tentatively compared.
| Issue | Recipients | Responses |
|---|---|---|
| Delphi questionnaire and invitation to the national workshop sent by e-mail (and through newsletters, networks, circulation lists, and websites of the national organisations) on 22 August 2013 Reminder sent on 9 October 2013 |
All participants in the research | By 21 October 2013: 18 questionnaires and 15 workshop registrations (not including the research team) |
| 125 other local stakeholders identified through fieldwork | ||
| The Advisory Board members | ||
| In England: all English chairpersons of HWS and CCGs; midwifery officers in the case study sites | ||
| In Scotland: chairpersons and clerks of Licensing Boards in Scotland; councillors with a responsibility for alcohol | ||
| Nationally: national and regional addiction charities; PHE and Health Scotland; NHS Clinical Commissioners press; Association of the Directors of Public Health; Local Government Association; SpR Public Health (BHPH); VONNE; the Institute for local governance; and Balance North East (regional alcohol office) for issue via their networks, newsletter and websites for wider reach | ||
| Telephone calls to all LAs in England and Scotland to identify local alcohol leads (August to October 2013) | ||
| Personal e-mail invitation to the workshop and Delphi questionnaire Social media Prize draw of an iPod or Kindle for return of the Delphi (reissued to all) |
151 named leads for alcohol across the English and Scottish local authorities | By 5 November 2013: 34 responses and 15 workshop registrations (not including research team); 10 external attendees on the day |
| Facebook, Twitter, Linkedin accounts of the research team members and of Fuse (the centre for translational research in public health: www.fuse.ac.uk) | ||
| 5 November 2013: National Dissemination Workshop | ||
| Social media | Workshop attendees issued links to the Delphi questionnaire via their social media sites and circulation lists | An additional 39 questionnaires: 73 in total |
Sample characteristics
Respondents were spread across different sectors as follows: Public Health in LAs (24%); LA employees outside Public Health (18%); and representatives from voluntary and community organisations (18%). Owing to the sampling procedure outlined above, most respondents were from the north-east of England (52%), with 20% from Scotland and 11% from the North West. Small numbers of people responded from other regions, such as Yorkshire and the Humber, East, the East Midlands, the South East, the South West, Greater London and Northern Ireland (Figures 7 and 8).
FIGURE 7.
Sector characteristics of respondents (n = 73). LA-PH, local authority – public health; VCS, voluntary and community sector.
FIGURE 8.
Region characteristics of respondents (n = 73).
Nearly 31% of the respondents came from the case study sites (the north-west 10.8% and Scotland 20%). It is likely that some of these respondents were participants in the study or had attended their local workshop and so they might have known of the findings in advance of filling in the questionnaire. Of the remaining respondents, 52.3% were from the north-east (local to the university hosting the research). The remaining respondents (circa. 17%) were from across the UK. Neither those responding from the North East nor those respondents from other parts of the UK would have had prior knowledge of the research findings or been part of the contexts in which the data were gathered. This suggests that more than two-thirds of the respondents were expressing opinions about how the findings applied in different contexts from those of the case study sites (circa. 50 people).
Findings
Importance of Delphi issues
We offer this tentative descriptive analysis to illustrate the findings of the questionnaire; however, the numbers are small and the sample is not representative of the views of commissioners in all contexts of the UK. We first present the results for the perceived importance of each topic (Table 12), followed by the ratings on the BARS questionnaire (Figure 9).
| Question | Behavioural anchors | % of respondents | |
|---|---|---|---|
| 1 | I find national evidence most useful in my decision-making on Public Health issues | I find local data and information most useful in my decision-making on Public Health issues | Not important: 1.2 |
| Somewhat important: 22.9 | |||
| Very important: 62.7 | |||
| 2 | I typically go to one trusted source of evidence | I typically draw on a wide variety of evidence sources | Not important: 0.0 |
| Somewhat important: 26.5 | |||
| Very important: 61.4 | |||
| 3 | It’s people who make sure evidence is drawn into decision-making in my organisation | It’s the organisational systems and processes that ensure evidence is drawn into decision-making in my organisation | Not important: 2.4 |
| Somewhat important: 47.0 | |||
| Very important: 38.6 | |||
| 4 | Mostly the evidence ‘speaks for itself’ | Mostly the evidence needs active interpretation to pull out key messages | Not important: 3.6 |
| Somewhat important: 30.1 | |||
| Very important: 54.2 | |||
| 5 | The most valuable knowledge comes from practical experience | The most valuable knowledge comes from robust academic research | Not important: 1.2 |
| Somewhat important: 42.2 | |||
| Very important: 44.6 | |||
| 6 | Commissioning across a purchaser–provider split makes it easier to secure evidence-informed interventions | Joint planning across unified organisational arrangements makes it easier to secure evidence-informed interventions | Not important: 10.8% |
| Somewhat important: 31.3 | |||
| Very important: 42.2 | |||
| 7 | Public health should be focused on changing national policy | Public health should be focused on changing individual behaviour | Not important: 0.0 |
| Somewhat important: 24.1 | |||
| Very important: 63.9 | |||
| 8 | Research evidence best informs strategic planning | Research evidence best informs service delivery | Not important: 4.8 |
| Somewhat important: 38.6 | |||
| Very important: 43.4 | |||
| 9 | Research evidence ‘trumps’ politics | Politics ‘trumps’ research evidence | Not important: 6.0 |
| Somewhat important: 31.3 | |||
| Very important: 50.6 | |||
| 10 | The health of the population is most important when making decisions | The economic well-being of the population is most important when making decisions | Not important: 3.6 |
| Somewhat important: 22.9 | |||
| Very important: 59.0 | |||
| 11 | Research evidence should be produced for me | Research evidence should be produced with me | Not important: 14.5 |
| Somewhat important: 38.6 | |||
| Very important: 32.5 | |||
FIGURE 9.
Results of Delphi questionnaire (first round; n = 73); interquartile range for each pair of statements (box plots).
Of the 10 items, seven were rated as ‘very important’ by the majority of respondents. The three items rated as somewhat important were issues (3) whether people or systems and process ensure that evidence is drawn into the decision-making (38.6%); (6) whether a purchaser–provider split or joint planning makes it easier to secure evidence-informed decisions (42.2%); and (11) whether research should be produced for or with decision-makers (32.5%). In addition, items 6 and 11 received the highest ‘not important’ scores, with 10.8% and 14.5%, respectively. This suggests that respondents do not think that who or what draws evidence into the decision-making, or the way knowledge is produced (with or without them), or the system in which this knowledge is used to make decisions (commissioning or planning) makes much difference to securing evidence-informed interventions. This appears to contrast with the findings of the two case studies, which highlighted the importance of person-dependent systems for evidence flows and, in Rosetown, the barriers for sharing evidence imposed by purchaser–provider split. The items considered ‘very important’ are directly related to the type of evidence that is used and what is it used for. The three items considered ‘most important’ are whether Public Health should focus on changing national policy or influence individual behaviour (63.9%), whether or not one trusted source or a variety of sources should be used (61.4%); and what level of evidence is most useful for Public Health decision-making: national or local evidence (62.7%).
Behaviourally Anchored Rating Scale rating for Delphi statements
Figure 9 summarises the ratings on the BARS questionnaire. To illustrate variability in the responses, the range of answers on each item is represented by a box plot which shows the interquartile distances between answers. The thick vertical line in each box is the median, which divides the responses in two, with 50% of the answers displayed on the left side of the line and 50% on the right. The left half of the box represents the 25% of answers below the median, while the right half of the box shows the spread of the 25% of answers above the median. The outer lines on each side of the box highlight the next 25% of answers on each side and therefore show the full range of replies across the scale.
In interpreting Figure 9 we note that no single behavioural anchor attracts a 100% score, indicating that respondents are pulled by both anchors. This illustrates that the anchors, although competing, represent viable solutions, and that overall no one anchor presents a definitive ‘solution’ to the wicked issue. In itself, this suggests that ‘correct solutions’ may only be finally identified in situ, not in abstract.
Some items (e.g. item 1: utility of national vs. local evidence; item 5: value of externally codified vs. situated knowledge; item 7: the focus of Public Health – policy vs. behaviour change; item 8: best use of research utilisation – strategic vs. operational; and item 9: what is most influential – research evidence or politics) provide equal pulls on the respondents, giving an overall score on the median point. This suggests that in differing situations either anchor applies equally; it is literally not possible to decide between them.
Other items do suggest that one anchor may be slightly more appealing. Item 2 suggests an overall preference for drawing on multiple sources of evidence; item 3 that people draw evidence into decision-making; 4 that evidence always need active interpretation; 6 that joint planning makes it easier to achieve evidence informed interventions; 10 that the health of the population is of most importance; and 11 that respondents prefer evidence to be created with them. However, these preferences are not strong and there are differences across stakeholder groups; we explore these differences in the next section.
Behaviourally Anchored Rating Scale ratings by sector
Figure 10 shows that in this small sample opinions vary across sectors, with respondents from some sectors showing a clear preference for one statement over the other, while other sectors are more divided in their opinions. Respondents working in the NHS have a stronger preference for academic research and also believe that research evidence trumps politics, while respondents from LAs put more emphasis on the value of practical experience. Academic respondents (not surprisingly) share NHS participants’ preference for academic research but feel more strongly about the usefulness of national evidence for Public Health decision-making. Contrary to this, LA respondents that work outside Public Health and respondents working in cross-sector partnerships (other category) find that local data and information are more useful for these decisions. If these results were borne out in a larger sample, this might suggest challenges for research evidence use in the new health system, with Public Health moving to LAs, if more emphasis is being placed on local data and practical experiences.
FIGURE 10.
Results of the questionnaire by sectors (voluntary and community organisations, academia and other (first round; n = 73). Note: The ‘other’ category includes many cross-sector partnerships, such as local ADPs and Crime and Safety Partnerships. LA-PH, local authority – public health; VCS, voluntary and community sector.
There are some interesting differences between LA respondents working in and outside Public Health; although both agree that practical experience is more valuable, those working outside Public Health also highlight the usefulness of local data and believe that research evidence (from these sources) will trump local politics. However, members from voluntary and community organisations in our sample disagree with this and argue that politics triumphs over research evidence and highlights the importance of national evidence for decision-making on Public Health interventions. Numbers are very small, but taken together this suggests that different types of evidence may be sought, valued and used across the organisations and sectors involved in public health – a story often anecdotally heard and supported by our case study findings.
On other issues, the findings are very mixed. For instance, respondents working in the NHS and LAs are divided on whether people or organisational systems and processes draw in evidence. However, participants from academia, the voluntary sector and cross-sector partnerships agree that people are more important for drawing evidence into decision-making processes. Additionally, views on the focus of public health (policy or changing individual behaviour) are more divided in the NHS and academia. In contrast, participants working in LAs, both inside and outside public health, and respondents from the voluntary sector prefer an individual focus on changing unhealthy behaviours. These data remind us that ‘sectors’ should not be viewed as unitary bodies (they are pluralistic entities, whose constituents have personal opinions, and where colleagues may only partially share opinions) and that our numbers are too small to draw any firm conclusions from or make finer distinctions.
Overall, our tentative summary is that in the new system for public health, those who seek to use and mobilise research evidence need to take into account the preference of LAs for public health interventions that target individual behaviours and favour local data. The different opinions between sectors are summarised in Box 1, while more details on these differences can be found in Figure 9. We offer these tables as illustrative of the data on the transferability of our findings on the use of evidence in commissioning decisions to other contexts and not as definitive statements on the spread of these views across contexts or sectors.
Stronger preference for academic research, and believe that research evidence trumps politics.
More divided on people vs. systems; and national policy vs. individual behaviour.
Local authority: public healthStronger preference for practical experience.
More divided on people vs. systems; and research evidence vs. politics.
Local authority: non-public healthResearch evidence trumps politics; practical experience and local evidence provide more valuable knowledge.
More divided on people vs. systems.
AcademiaNational evidence is more useful in decision-making; slightly stronger preference for academic research.
More divided on national policy vs. individual behaviour.
Voluntary and community sectorPolitics trumps evidence; national evidence is more useful in decision-making.
OtherPractical experience and local evidence provide more valuable knowledge.
More divided on national policy vs. individual behaviour; and strategic planning vs. service delivery.
We carried out a chi-squared goodness-of-fit test for each Delphi statement (Table 13), which confirmed that the spread of responses and the clustering of responses at certain points on the 9-point scale was unlikely to have happened by chance. The exception is for statement 9: research trumps politics or the other way round [χ2(8) = 14.411; p < 0.072]. These issues were selected for further discussion with participants in the national interactive workshop.
| Delphi item | χ2 | df | Asymptotic significance |
|---|---|---|---|
| National vs. local evidence | 32.411 | 8 | 0.000 |
| One vs. various sources | 37.333 | 7 | 0.000 |
| People vs. systems | 25.778 | 7 | 0.001 |
| Evidence speaks for itself vs. active interpretation | 41.205 | 6 | 0.000 |
| Practical experience vs. academic research | 88.877 | 8 | 0.000 |
| Purchaser–provider vs. joint planning | 41.789 | 7 | 0.000 |
| National policy vs. individual behaviour | 73.096 | 8 | 0.000 |
| Strategic planning vs. service delivery | 175.528 | 6 | 0.000 |
| Research evidence vs. politics | 14.411 | 8 | 0.072 |
| Population health vs. economic well-being | 39.556 | 7 | 0.000 |
| Produced for vs. with me | 26.972 | 6 | 0.000 |
National workshop
The case study findings, the comments from the case study sites’ local feedback workshops and the Delphi findings informed the programme for the national workshop. However, only 10 external participants attended, so we report the resultant discussion only briefly, and with caution. The 10 participants came from across the UK and worked in a range of sectors. Below, we capture the main points of the discussion.
Evidence flows in Thistletown
First, findings from the Scottish case study site were discussed. Participants were particularly concerned with the question of whether or not the introduction of an additional public health objective for licensing in England would be beneficial, given the experiences with this objective in the Scottish case study.
We’ve spent time lobbying for health to be included as part of licensing objectives – it’s clearly not helpful. The people on Licensing Boards are not users of the night-time economy and are therefore influenced by anecdotes and stories, e.g. claims that closing a nightclub earlier would cost 200 jobs. This can have a significant effect on the Board, although not logical.
(Note: the numbers attending the workshop were small and to preserve anonymity we do not identify the quotations in any way.)
It became clear that not everyone was aware that an overprovision statement did not apply generally but that a specific objection had to be made relating to each application. This raised debate about how practitioners use public health evidence in practical ways. Participants commented that it is preferable sometimes to go for conditions on the licence rather than an outright objection based on health data as a middle-ground solution. One participant suggested that the use of cumulative impact policies, which put the onus on the applicant to demonstrate the added value of their application, worked better. Overall, participants agreed that a better fit of public health data would be an important prerequisite and that improved partnership working would be essential to make such an objective effective.
Debate around Rosetown
After the presentation of Rosetown findings, participants explored the role of values among professionals, who could use their contact with the public for a brief intervention: ‘People expect screening questions about drinking and other sensitive topics: is it that the professionals have the taboos not the client?’ A representative from a regional alcohol charity explained that their research among local GPs had highlighted some GPs’ reluctance to raise this issue with patients because of their own uncertainty about the availability of follow-up support and treatment services. This led to a discussion about unclear referral and service pathways for women who were identified as drinking during pregnancy; this lack of clarity was also reported as a problem in Rosetown.
You won’t ask the question if you don’t know what to do with the answer. There is a lesson here from other services.
Overall, the discussion’s conclusion was that midwives might be the wrong people to raise the question about alcohol consumption during pregnancy. Discussing this topic at the first antenatal booking session – at the end of the first trimester of pregnancy – could prove too late, as many women are likely to have continued their ‘normal’ drinking behaviour up to this point. Participants, therefore, discussed using more ‘upstream measures’ to engage women earlier in those conversations about drinking during pregnancy. Educational and advertising campaigns were mentioned as useful in enabling people to make more informed choices. Persuasive communication theory was also mentioned, as well as the value of using a role model (hero figure) to give the message: someone ‘like me’ or ‘this is what happened to me’.
Participants recognised that working steadily on the issue of maternal alcohol consumption for a number of years may eventually have desired effects, similar to the progression of the smoking and 5-a-day fruit and vegetable campaigns. Behavioural changes need to be reinforced by changes in the wider environment alongside the provision of services. The advice was, therefore, to structure the environment at all levels in order to make it easier for people ‘to do the right thing’.
Debate around the findings from the Delphi first round
After the presentations on the case study findings, the results from the first Delphi round (n = 34) were presented and discussed with the audience. Participants expressed surprise at the finding that respondents in LAs believed more strongly that evidence trumps politics.
I’m intrigued that the local authority (outside Public Health) people were saying that evidence trumps politics; I would have expected the opposite.
The unexpected finding was explained in this way: that although the word ‘evidence’ is used, what is meant by that word is different:
Local authorities may stop seeing that politics does trump evidence and assume evidence is being used. Local authorities may think of evidence as what’s happening locally/practical experience.
This triggered a discussion about the interpretation of the statements by respondents and whether they answered from a ‘what is’ or ‘should be’ perspective, reflecting either current experience or future desirability, which is a reasonable critique of the Delphi instrument as used here.
Debate around three ‘curious cases’ from the case studies
Three vignettes from the case studies were presented to highlight the complexities involved in using research evidence for decision-making, and the wicked issues involved as outlined in Table 10. Appendices 13–15 include the three vignettes used. The first case illustrated issues with data interpretation and the difficulties in showing casual relationships between local-level hospital admissions data and premises selling alcohol in the locality due to off-sales in neighbouring localities and ‘preloading’ at home. This issue was one picked up by legal agents for applicants appealing the decision to refuse a licence in an overprovided area and largely undermines the causal link between alcohol-related harm and overprovision (using geographical data), and thus neutralises the power of the research evidence, largely neutering public health objections.
Participants considered times when the nature of evidence may permit it to trump politics, and this sparked a debate about the role of public health. Participants also discussed the relationship between health and economic vibrancy, with some suggesting that costs simply get shunted onto the NHS while others argued that economic vibrancy does not have to run counter to health objectives:
So, what is public health? What is the greater good? Is it [overprovision] only a problem if the facilities are misused? If we all drank safely would the pubs still make money?
It was stated that pubs are often the safest place to drink (as a result of work across agencies, e.g. the police) and, therefore, licensing extensions of other trades and off-sales should be reviewed more carefully.
A comment was made by one alcohol lead that he or she received 8–10 requests per week for comments on licensing applications, of which only 1 in 10 was new. The other applications were extensions of or modifications to existing trading, for example a kebab vendor asking for a licence, in order to secure additional trade and extra profit. This participant stressed that conditions on licences can be used to reduce overconsumption (by reducing availability). He or she also stated that if big supermarkets are challenged about their timings for selling alcohol, they will negotiate. In practice, participants had many practical ‘tips’ for using evidence in quite subtle ways.
The second example highlighted issues around routine data collection and the lack of training for health professionals on data collection and usage. Participants recommended more training.
This should be a mandatory lesson in SRE/PSHE [sex and relationships education/personal, social and health education] curriculum, not science. We need evidence around the efficacy of PSHE, if not done people get their information from anywhere.
However, participants argued that this should not be an excuse for not asking questions, as in some health provider settings the questions are asked without any problems (with references being made to private medical practices):
In private medical systems the questions are asked routinely without carrying a judgemental aura. This needs to be routinised in the public sector, plus made into a consistent message in the first place. If necessary we need to develop a local message in place of the national message, because we have no clear evidence as there was for smoking.
The third example highlighted issues in making national data fit for local purpose; the lack of actionable messages was raised immediately: ‘You go to conferences and are told the scale of the problem but not the solution’. Participants argued that actionable messages from national research would cut down the need for local research. However, other participants agreed that local evidence was necessary for priority setting and exploring the acceptability of potential interventions.
There can be too much research on too many subjects, and too many guidelines, but the local work needs to be done to identify the top priorities.
Discussion
Participants felt that including a fifth licensing objective for public health in England would work only if public health data provided a better fit with the local policy and legal framework, and that improved partnership working would be important to achieve this. More generally, participants agreed that (national) messages need to be tailored to local needs and that identifying local allies was key to this. They recommended a pragmatic approach to data gathering, collation and submission: ‘don’t be a health purist’, but aim to win people over by using the data to tell the story and provide actionable messages. This requires the use of a wider range of methods and sources, such as qualitative messages and tactical knowledge based on practical experience.
Participants’ ‘tips’ for using evidence
Based on the discussion of research utilisation in complex settings, participants concluded the following:
-
Introducing a fifth licensing objective for public health does not provide a panacea solution in securing the use of public health data and intelligence in licensing decision-making.
-
Do not be a health purist but aim to win people over; public health evangelists are counterproductive; be pragmatic.
-
Use the data to tell the story, present qualitative messages as well; this will also make public health people more approachable. If possible, use an influential local person.
-
Identify local allies and tailor messages.
-
Use actionable messages – ask the right questions of the evidence you have.
The above points are discussed more fully in Chapter 8 in order to consider the pragmatic steps that public health commissioners and planners take to manage the competing demands on them and the needs of other stakeholders and the populations they serve.
Chapter 7 The experience of cocreation
Introduction
Part of the overall aim (and objectives) of this study was to work in cocreation with our research participants, and RQ 2 makes learning about cocreation an integral part of this study. In Chapter 2 we outlined the promise of cocreation and the methodology we developed for this research project to work in cocreation with our case study partners in Chapter 3 (the plan). Here, we reflect on what happened. We begin with a brief recap of some of the main points in the literature.
Some authors suggest that when academics and practitioners work together to cocreate research, this enables partners to better understand the complexities each other faces, enhancing ownership and credibility of research and supporting implementation. 34,50 Working collaboratively, however, is not always easy to achieve,86 as researchers and practitioners often approach issues differently and bring different skills and expectations to partnerships. 86 Factors that facilitate collaborative working include trust, communicative relationships and having a shared focus. 51,86,217,218 It is often the people involved and the working relationships developed216 that are considered key to effective collaboration;54 whether or not senior leaders promote collaborative relationships and the large amount of time needed to develop strong partnerships are highlighted as potential barriers.
We will reflect on our experiences and those of our partners in applying the three main components of our cocreation methodology:
-
enabling participants in each case study site to choose their own topic
-
involving participants as coresearchers in the project
-
jointly discussing and interpreting the research findings with participants.
How well did each element work and how useful were they in hindsight? Data were collected from the researchers’ reflective diaries and interviews with case study partners towards the end of fieldwork. After discussing the challenges and opportunities for each element, we will discuss the impact of our cocreation efforts in both case study sites by reflecting on the changes that we and our partners have witnessed as a result of the collaboration process. This leads to a more general reflection at the end of the chapter on the challenges and future opportunities for researching in cocreation.
Cocreation approach 1: choice of topic
In our study, the choice of alcohol as a public health issue was predetermined in the study protocol; beyond this, the precise focus of the study was left loose in the anticipation that this would give our participating sites more scope to mould the topic focus to meet their needs and that, ultimately, once entry had been negotiated, the focus would be more honed. This was only partially realised. In Thistletown the process case study we followed was specific and contained (evidence use in the legal alcohol licensing process); however, in Rosetown the areas of commissioning to address alcohol-related harms were extensive, and honing down a contained commissioning process to follow in live time was a challenge. Most commissioning was already ongoing, and out of our time scales and or recurring spend. However, the research team worked alongside partners to identify the precise focus they wished the research to take to ensure that the process case study topic was timely and relevant to them (and to facilitate buy-in).
In Thistletown a number of meetings and discussions between case study partners and the research team took place between February 2012 and July 2012 (with e-mail correspondence in between) before it was decided that evidence use in alcohol licensing should be the focus. Further negotiations were then necessary to secure ‘buy-in’ and negotiate access to meetings with the LA and the Licensing Board. Fieldwork began in September 2012, and first interviews took place in October 2012. Partners in Rosetown decided at a meeting in March 2012 that their topic choice was evidence use in reducing maternal alcohol consumption before, during and after pregnancy. This included improving guidance and education, identification of and screening for alcohol (mis)use during pregnancy and improving access to alcohol services. We had built in 4 months of research time to negotiate entry, discuss cocreation and agree an alcohol case study topic with our participants; from the above paragraph, it is possible to see that this was more involved than anticipated and needed more time. Although practitioners and senior decision-makers were involved in the writing of the research proposal and protocol and the RQs within it, they were not the same practitioners and decision-makers as our research participants. Our period of negotiated entry was spent ‘selling’ these ideas to a new set of collaborators and discussing how the research would work. Time was needed to let our research participants consider the ideas, consult with others and explore the practicalities of picking different case study topics.
Early meetings provided the researchers with contextual background to some of the relevant local alcohol issues, what agencies and organisations were involved and how these structures fitted together strategically. However, the extended period of negotiated entry in Thistletown (over several meetings across sectors) caused the researcher concerns regarding loss of fieldwork time and falling behind Rosetown. This meant potentially missed opportunities to share fieldwork experiences, to amend topic guides and interview questions, and to give and receive peer support.
Partners in both sites stated that the ability to choose the study topic differentiated this study from previous research studies they had participated in, and was unfamiliar to them. In Thistletown, partners appreciated the opportunity to choose the topic, but found it challenging: ‘maybe one of the hardest bits’ (Thistletown 1).
Despite being given a short one-page summary (see Appendix 5), participants reported initial uncertainty about the expectations of them in relation to working in cocreation and the implications of choosing the topic. Some became frustrated by the time it took for the research to ‘begin’.
Some participants in Thistletown felt that they would have valued more explanation and direction from the research team around topic choice early on and more information regarding the time commitment that involvement with the study would require. These are interesting learning points for the research team. From our perspective, we thought that our case study partners in the Thistletown understood our expectations of them (see Appendix 2) and we were cautious not to rush their decision-making in order to ensure that they reached a choice they were happy with.
[Choosing the topic] was probably the most frustrating bit of it . . . we were brought in, you’re thinking, I don’t know what this is about. What is it we’re doing here? And it was very nebulous and there was I suppose, a concern that, does this cocreation mean we’ve got to set it all up for you? . . . I think our frustrations were probably at the beginning, it seemed to take a long while to get off the ground, and partly that is because you are trying to co-create it, but that seemed like it was awfully vague for an awful long time.
Thistletown 1
In addition to this, involving the case study partners in early discussions (during the compilation of the application, long before funding) had the effect of building frustration at the time it takes for the application to go through review, funding, contracting and ethical approval before fieldwork begins (and, of course, if the application is not successful then these efforts may come to nothing). As fieldwork progressed, we as a research team deliberated about whether or not, in both sites, we had been asked to highlight issues (and relationships) which were particularly difficult for case study partners to address themselves (handed a ‘loaded gun’). Tricky issues (e.g. about relationships between commissioners and the largest and most powerful provider), which everybody knew about but did not feel able to voice, were shared in interviews. By ‘gifting’ this information to us in interviews the strained relationship also become our problem. The data were now ours, and part of the emergent findings to be fed back in the joint interpretation meetings.
One participant hinted at this and claimed that they felt that cocreation process had given them licence to pick tricky issues:
It may have influenced the topic choice because you’re more likely to pick something difficult with a cocreation approach. If we’d chosen something straightforward, it doesn’t test out how we work together with partners. With a tricky issue, it’s better to have a cocreation approach because you can move towards a better understanding through the process.
Rosetown 37
Our research engagement was being used tactically to lever and prompt improved working relationships. It was not simply Rosetown but also Thistletown that picked a troublesome topic, where they currently felt at an impasse. They too used the research team to engage partners where previous relationships (and influencing) had been difficult.
In Rosetown, while the topic fitted the strategic priorities in the Health and Well-being Strategy (2013), decisions about the best approaches to take were complicated by changing commissioning arrangements under way in England, as well as contested, contradictory evidence about ‘safe’ levels of drinking in pregnancy, and perhaps a troubled relationship across the purchaser–provider split.
In both sites, the topics chosen presented challenges to local stakeholders – ‘wicked’ problems, characterised by the involvement of multiple stakeholders, multidisciplinary partnerships, complex interorganisational and interpersonal relationships – and no easy solutions. In both sites, it seemed as if the participants and the research team took substantial risks engaging with each other to explore these thorny issues. For example, would it be possible to raise issues and use the research to influence future working relationships between stakeholders locally (action research) and, if so, was there a risk that it could make relationships worse, and hamper the efforts to collect data, leaving all parties a loser?
Below, we reflect more on this and our ability to engage with wider partners in Thistletown, but not in Rosetown, where the powerful provider declined to engage in the alcohol strategy group and the joint interpretation meetings, and where the difficulties in recruiting midwives to take part in the commissioned research process that we followed (the Insight work) caused the process case study to stall.
Cocreation approach 2: participants as coresearchers
Participants were not only invited to choose the topic for the research but actively encouraged to become coresearchers in the project by shaping or adding RQs, by suggesting respondents for interviews and observations and by conducting interviews and analysis themselves, if they so wished. Unfortunately, our partners in both case study sites did not engage actively with these opportunities. No suggestions were made for additional RQs (to add to and ‘piggy-back’ onto the proposed data collection) and none of our partners took up the opportunity to become actively involved in the fieldwork. The lack of engagement of our partners in both case study sites as coresearchers raises questions about the sharing of roles in cocreation. Although we intended to share our researcher role with participants, in practice a task division occurred between our role and theirs. While our participants were very supportive in negotiating access to relevant stakeholders for the interviews and access to relevant decision-making meetings for us to observe as researchers, they perceived the conducting of fieldwork as the domain of the researchers. Our research participants expected us to carry out the research and discuss the progress and findings with them and were happy to act as commentators on the emergent findings, but not to take a more active role. Where interpretation was shared in the joint interpretation meetings, there were several occasions were new insight was added. This points to potential barriers for working in cocreation based on different role expectations84,220 that might need to be clearly negotiated at the start of the research process to enable role sharing in practice. Participation may also be blocked by other factors (e.g. skills sets, time and other resource barriers). Although the research was considered to be ‘very interesting’, the joint interpretation group in Rosetown was cancelled on three occasions. It is difficult to know why this was the case. It could simply be the inevitable result of a public health system being in transformation or that the topic was genuinely considered a low priority.
In both sites, the involvement of senior public health colleagues was critical to ensuring early buy-in from wider stakeholders. Senior management endorsement helped to facilitate access to potential interviewees, snowball contacts and legitimise access to relevant meetings. It also influenced the nature of our contacts, how we were seen, how the research was presented and by whom and when, and how we presented ourselves. (We say more of this in the following chapter.)
Even with senior ‘buy-in’ and working in cocreation, there were still some places we did not get to see: our access to certain meetings was carefully managed. Some meetings the research team had identified as potential sites to observe (e.g. HWB in England and the convening of the Board/Forum subgroup in Scotland) were rendered out of bounds. The reasons varied: some meetings had legal privilege and were not open to anyone except members; others were meetings being held by new organisational groups and they were keen to focus on core activities in the early days; at others, our access was simply declined. There are limits to the level of scrutiny participants are able and willing to allow themselves to be subject to.
Cocreation approach 3: joint interpretation of findings
Finding time to take part in feedback and joint data interpretation meetings was also an issue for the participants:
. . . it didn’t feel like a spotlight shone on you [. . .] I think there are times when you’ve sort of gone ooh it involves more time than you initially expected [. . .] this clearly has been a far more involved process of bringing people together, talking things through, reporting back etc. etc. and you know I don’t think I quite understood at the start quite the amount of time I don’t regret the amount of time but I don’t think I understood it at the start.
Thistletown 2
Different role expectations also came into play in the third element of our cocreation methodology, not only between researchers and partners but also between partners themselves. For instance, despite Health and LAs in both sites having a strategic commitment to address alcohol-related harm in both sites, there appeared to be a disconnect (or misunderstandings, or power struggles) between some partner organisations involved.
We faced a number of dilemmas as researchers. We had made a commitment to feed back our early findings, and recognised that some of these might be seen as contentious or controversial. We felt that we had developed positive, trusting relationships with case study partners and did not want to disrupt these or appear overcritical. However, not accurately reflecting back what we found jeopardised the integrity of the research process. The research team were being actively used as mediators. However, despite our concerns, when presented with interim findings in both Thistletown and Rosetown, case study partners appeared to have expected them and embraced them, and both sites sought to make changes:
There were comments in terms of our practice; I thought they were fair comments there did need to be more structured working together on it – yeah I am perfectly happy with that.
Thistletown 2
‘That everybody needs to work differently to make this work, I think is a really positive thing to come out of it. And that we, hopefully, are all able to accept that and think, yes well let’s change on the basis of that’
Thistletown 1
The facilitated discussions in the joint interpretation meetings offered opportunities to contribute to a shared understanding of the systems and structures in place locally to respond and the accuracy of the findings was improved through this process:
We were able to give you feedback and say, that bit’s not quite right, and then it changed and even when it doesn’t exactly match, we could see how you’d got to that point.
Rosetown Public Health 37
In Thistletown, there was recognition of the benefits of the considered approach taken by the researcher:
You’ve been very good at that, and, erm, testing things out before you’re going to go wider. And err, these discussions and the organisation of the workshops, I think that’s been really positive. And that’s been part of what we will get out of this apart from the results, is just that process. So I think that’s been really good.
Thistletown 1
Impact of cocreation
Case study participants in both sites reported that involvement in the study had been beneficial (and some believed that the expectations around cocreation were clear):
I have seen people feel very threatened by researchers, but they weren’t with this and you gave a clear explanation about cocreation and what it meant, and because of how the research team were, very hands-on, very approachable, very flexible, it worked well.
Rosetown Public Health 37
In both case study sites, difficult messages were accepted and confirmed (perhaps because these issues were already known to them and this was the reason why the topic was selected in the first place – in order for us to explicate them to other partners). However, the extent in which they were acted on varied between our case studies.
In Rosetown, there was more ambivalence around perceived benefits, indicating that our efforts in cocreation had more limited success in terms of engaging all partners with an interest in reducing maternal alcohol consumption. Our reflection on this is that it is part of the feeling of being handed ‘loaded guns’ – both topics were selected in the hope that we might make headway in breaking down barriers. In Rosetown, as the research progressed, it became apparent that the gaps were too deep seated and not directly about the topics we were working on, but about strategic links between the organisations concerned. We were not able to engage with all stakeholders. In addition, in Rosetown the involvement of these multiple organisations undergoing reorganisation, and staffing changes mid-way through the study, left us without full access to them. Cocreation is limited in what it can achieve if not all partners will ‘play ball’.
If part of the collaborative approach was to get partners to work together, it hasn’t worked, the whole piece of work feels a bit messy if I’m honest, disjointed.
Rosetown Public Health 45
There is perhaps another agenda here: that we were being mobilised as troubleshooters, specifically to ‘sort out’ locally knotty issues. If partner organisations believed this to be true then it perhaps reveals their unwillingness to be ‘sorted out’. This is an important point, that participants who did not join the research process at the beginning may have different views, which influenced buy-in. Being involved in the original planning may be a key event for cocreation. By being identified with ‘one-side’ at the start, we were viewed suspiciously thereafter by the other parties.
There was a suggestion by this participant (Public Health 45) that Rosetown partners might have gained more value if a different topic had been selected. Other participants acknowledged that the timing of fieldwork during the transition of Public Health to LAs had hampered progress.
In Thistletown partners did feel that the process of working with the research team enabled them to designate specific time to address alcohol licensing, suggesting that the process had enabled them to have the time and the space to think, and discuss issues with each other, that otherwise would not have happened (even though they often met at a number of different strategic meetings around other topics).
In Rosetown, participants were more limited in their ability to open up difficult discussions about what evidence was used, by whom and with what effects, because not all stakeholders chose to be present at these meetings, and any opportunities to build alternative ‘organisational bridges’, interpersonal links and forthright discussions were reduced by the constraints imposed by the purchaser–provider split. We may speculate on the reasons for this: these forums did not (yet) seem to exist; Rosetown was too much in transition with roles too new; or, last (but not insignificantly), the purchaser–provider split made such meetings difficult, if not impossible. In Rosetown these conversations appeared to take place outside formalised forums and in smaller groups between representatives of particular organisations, and the ‘gritty’ difficult business was done there. As the research progressed this encouraged us to think about where and when commissioning actually happens. It did not appear to be an event occurring at a set time and place. Often, activity was going on in between the formal organisational spaces, and as we did not get to observe this our accounts remain partial.
Not just in the meetings, but outside meetings, it prompted a lightbulb moment for some.
Rosetown Public Health 37
Participants in follow-up interviews in both sites expressed their appreciation of the joint interpretation and feedback meetings, as they stimulated discussions which would not have taken place otherwise:
I suppose we wouldn’t have had that discussion had you not pulled us all together [laughs], so to that extent, I suppose it helped us kind of articulate our thinking a bit really.
Rosetown Public Health 33
For other participants navigating new structures and job roles, their involvement in the research appears to have prompted them to reflect differently, and make proactive changes as a result of their participation. Reflecting on allocating roles and responsibilities in their own team, one interviewee decided:
I think that’s exactly what I’m going to do now because you’ve made me think. There are areas, like alcohol, that we do need to link in to [name] you know, so this has been very mutually beneficial.
Rosetown Commissioner 40
In Thistletown, the joint interpretation meetings not only provided conversational spaces but also facilitated local changes in relationships and decision-making:
I believe this is an exemplary piece of work that may have real and practical implications for how we do things in [Thistletown].
Thistletown 4, personal correspondence
I think that, erm, dialogue, and particularly with the Board itself has become more constructive. And I think there is a better understanding of each other’s perspective, or we probably, we understood it before but maybe a greater willingness to take account of it in discussions. [. . .] that’s why your research is so important, because if we think, actually if you did it differently, if you stick at it or we can now have a different relationship, it could be worthwhile, then that takes us forward.
Thistletown 1
Did anything change in practice? Instrumental outcomes of research utilisation and working in cocreation
In Rosetown, there are few changes to report, partly because, at the time of writing, decisions were yet to be made following the results of the Insight work commissioned (the case study process we were following). These will be influenced by substantial pressures on LA public health budgets. The NHS trust providing maternity services is undertaking work to improve its data collection systems to enable better understanding of reported levels of drinking in pregnancy. In addition, as mentioned above, despite our best efforts, we were less successful in facilitating collaborative partnerships between purchasers and providers, which may reflect current churn or signify wider challenges of working in these organisational arrangements.
In Thistletown, the researcher observed a number of specific, tangible outcomes, which may have resulted from this study. There are plans to address membership and operation of the Local Licensing Forum to enable ‘more constructive conversations’ to take place (Thistletown 1). Increased partnership working among organisations was observed, which partners felt ‘would not have happened as quickly if it hadn’t been for this process’ (Thistletown 2, emphasis added). There was particular appreciation for ‘how willing and honest some colleagues had been in saying they hadn’t engaged as much as they should’ (Thistletown 3). There are also plans to appoint to a post, jointly funded by the ADP and Community Planning, someone who will support the input of health evidence into the licensing process to directly address the gaps in the flow of evidence we identified. It is always difficult to directly attribute research efforts to specific outcomes; some case study partners felt that the need for this post had previously been identified. Maybe our efforts were simply levers to accelerate change. 38,221
Reflections on cocreation from the research team perspective
Working in cocreation is not straightforward. It presents epistemological, methodological and ethical challenges, and raises questions about truth, (re)presentation and ethical difficulties of protecting participants’ anonymity and confidentiality during and after fieldwork. It also highlights the challenges of maintaining working relationships with stakeholders when difficult or sensitive issues are uncovered, which we had made a commitment to feed back through joint interpretation group meetings.
As observers in meetings, we chose whether or not, when and how we contributed and shared our reflections, ideas and thoughts, formally and informally. Inevitably, our presence did affect proceedings in some ways, whether we adopted the role of ‘constructively clueless’ researcher advocated by Ward et al. 222 or made suggestions drawing on personal/professional skills, knowledge and expertise gathered from our experience as practitioners and academics. There were times during fieldwork when these tensions were difficult to navigate and we were required to make on-the-spot judgements about our role and how the research should be undertaken. There were moments when participants asked questions about our perception of situations or of people. We risked inadvertently disclosing information by the questions we asked, how we framed them and responses we made. We see these issues as integral to the process of all participatory approaches. 80,82,88
We learned a great deal from our research participants. While we had ideas and research skills, and were ‘primed’ by the evidence-base, we were definitely not the experts. It is obvious that local knowledge, organisational memory and the tacit ‘feel’ for the work of commissioning and joint planning are not ‘added extras’ but core to the successful accomplishment of the role. As research agendas move forward, more attention needs to be paid to how research evidence is actually used in practice, what factors influence this, what other approaches are needed to complement this activity and how researchers can be more helpful in all of this.
Cocreation as a long-distance activity
Neither of our case study sites was local. Our work with them was carried out in field visits. While we were able to build good relationships, we remained visitors and outsiders. This may be to our advantage in that we preserved the ability to see issues afresh, but we do feel that it limited our capacity to work in full cocreation. Being on site, ‘rubbing shoulders’ with our participants and working locally in a more concerted way to create collaboration would have honed our approach. It is our ultimate reflection that in order to carry out cocreation to maximise the full benefits of the approach, the researcher needs to be on site for an extended period of time. This suggests that cocreation might fit more easily with ethnographic approaches or via secondments between sectors.
In the final chapter (see Chapter 8), an overarching read across the different findings is given. This analysis is tied back to the findings with examples from both cases. Both the contribution and limitations of the study are discussed and two mid-range (realist) theories are presented. Their capacity to explain the findings is considered.
Chapter 8 Discussion and concluding remarks
Preamble
This chapter is in three parts. The first part identifies the limitations of the study. The second section identifies the contribution made by this study and gives a high-level read across the findings in order to synthesise the various data streams to answer the overarching theoretical question: what works where, for whom, and under what conditions. 171 Included in this section are two mid-range theories as the study’s unique contribution to knowledge. The final section outlines recommendations for practice, researchers and educators, and research funders, and makes suggestions for further research. Care needs to be taken when interpreting the findings, as this is a small study. We start with an explanation of the quantitative analysis conducted as part of this project, the challenges it faced and the format it finally took, along with other limitations of the study.
Limitations of the study
Quantitative analysis: an aspiration too far?
We anticipated that data would be available to examine the overall performance of the case study site organisations (their quality), and that these data would be readily accessible. In fact, this was not the case.
Extensive efforts were made to identify routine data sources that would allow us to address the quantitative component of the study. The quantitative measures of performance would have allowed us to see if there was any reason for considering our sites to be typical (or outliers) in what they were able to achieve, and therefore contextualise the qualitative data (i.e. we would know the standing of the organisations we were working with). Appendices 9 and 10 lay out in some detail the approaches we tried.
However, currently, there is a challenge when drawing out changes in performance over time because high-level performance indicators for England have changed for both the NHS and LAs over the past 5 years, including the organisations that were responsible for their performance assessment. To explain: within England the quality indicators over the three performance frameworks (the NHS outcomes framework, the Public Health framework and the adult social care framework) have not been consistent over time, and it is currently unclear which indicators are relevant to the different commissioning organisations (HWBs, CCGs, LAs, etc.). There are benchmarks for the NHS outcomes frameworks where NHS organisations can compare themselves with other organisations, but access to these is restricted to NHS employees (www.nhscomparators.nhs). This makes it hard for external researchers to obtain any objective (quantitative) measures of quality.
Statutory performance indicators are available for all LA councils, but they are not collated in any one area and nor are they reported consistently by councils. This reporting issue had two main implications for this project. First, it is not straightforward to identify where they are reported. Second, and importantly, measurement between councils and possibly within councils is not standardised. This means that comparative data would be misleading (possibly explaining why collated information is not reported). Taken together, this suggests that little or no national/local performance measures are readily available that would allow meaningful comparison between locations, or indeed over time, to establish good measures of quality.
It may be possible to address these issues (i.e. identify, access, collect, clean and interrogate these data), but this would probably necessitate new primary data collection and be a substantive research project in its own right, which was beyond the scope of this project. Overall, our attempted economic analysis highlights some very real quantitative information gaps.
Where it was possible, descriptive quantitative data have been added to the case study chapters to contextualise the issues in the case study locations and to support the findings. However, in Rosetown, routine data linking alcohol consumption and pregnancy are based on subject categories (e.g. drinks ‘occasionally’), are self-reported or are not recorded at all (making a FOI request of little use). In Thistletown, the number of licences granted was recorded (on a case-by-case basis) but not collated. Requests made to ask for these data to be extracted were declined, as the one practitioner involved had insufficient time to search all records. As part of our negotiated entry to Thistletown, we were required to agree not to issue a FOI request on the LA with regard to its licensing data (again for capacity reasons). Appendices 9 and 10 report on the quantitative scoping work we undertook and the sources consulted, and gives an overall, descriptive quantitative account of the substantive topics in the case study sites (licensing and alcohol use in pregnancy). We choose not to present some of the local and national data, as we balance these against the need to preserve the anonymity of the case sites. Most of the routine data compare the ‘core cities’ within the UK and comparative prevalence and trends over time. It is easy to identify locations from these data.
The challenges of comparative case study design
In-depth case studies permit deep examination of practice in situ and the interplay between multiple factors in that context often in live time to provide a realism (‘messiness’) and richness that controlled designs often do not have. The advantage is that a plethora of data are revealed that could not have been exposed in other ways. We sampled our case study sites according to issues considered to be pivotal in shaping the use of evidence and organisational arrangements (commissioning vs. joint planning) to compare how the issues played out in the different contexts.
‘Preventing alcohol-related harms’ was selected as the overarching cohering topic area, but, by allowing each site to select their own topic area (of most pressing concern, and fitting in with their own strategic objectives), through working in cocreation a gap opened between the two sites. When viewed traditionally, the direct unit of comparison appears lost; in our findings it is impossible to say if any differences between the two sites were due to the organisational arrangements (commissioning vs. joint planning) or the topic (maternal alcohol consumption vs. alcohol licensing). However, in a realist design this direct (controlled) comparison between the sites is of less importance. The prime focus is the interaction between what works, why and under what conditions in looking at the use of evidence. We built an increasing understanding by looking at how evidence is used and what works in one site (and process) and then built on or refuted this understanding by looking at what works across the second site. It was this iterative comparison of the two settings that illuminated factors in the process of using evidence and the opportunities and constraints involved. Our understanding and contribution lies in illuminating these comparisons rather than attempting to pin down the definitive causal pathways determining evidence use.
There were ongoing challenges in the English context. The key stakeholders with whom we were working changed in the middle of the project as original members changed role (displaced). Often stakeholders would not be sure if they were part of the process, as roles were changing and some practitioners were holding portfolios temporarily (caretaking) until the new structures (and roles) were established. Many staff were uncertain regarding their ongoing job security. When the commissioning process in Rosetown stalled (mid-way), we were left in a hiatus until there was movement (behind the scenes, which we did not see). At this point, following the live process, as was our intention, was thwarted, and our data gathering became historical in nature.
Limited wider stakeholder evaluation
In following this design, we acknowledge that it is unlikely that all the findings will apply everywhere (truly generalisable); however, the opposite is also true. It is also unlikely that none of the findings will apply elsewhere. Some will hold important and transferable messages for evidence use in other places (transferability). The key is to identify which findings are more context dependent (apply only locally) and which may be considered more context free (and therefore apply more widely).
Several steps were taken to verify and validate our case study findings, both within the case sites and to explore transferability to other contexts. In each case study site we held a final local workshop to discuss findings, check our interpretation and invite comments and interpretation. This ‘checked’ our findings with other local stakeholders, including other commissioners and senior public health decisions-makers. These stakeholders were forthright in their comments and they helped us to understand the history of, reasons for and significance of the findings. We did not proceed to the national workshop until the local sites were happy that our findings reflected how they understood the process. Through this process, the accuracy of the case study findings was validated by a wider local audience.
Checking the results for transferable messages across contexts was more challenging. These efforts were not entirely successful. Table 9 speaks to our efforts to invite stakeholders to the national workshop and issue the two-stage Delphi process. Despite sending out approximately 1000 questionnaires and telephoning all public health departments/sections in all English LAs and all Licensing Board convenors in Scotland to invite them, and all professional bodies and voluntary and community sector organisations associated with the topic areas, only 10 participants attended the ‘national’ workshop.
It is difficult to say why this was the case. Respondents gave different reasons for non-attendance: they were too busy to leave their desks (dealing with transitionary arrangements); there was no money for training and development events; they were carrying additional portfolios of displaced staff; or they were uncertain about how ‘taking time out from the day job’ would be perceived at a time of job insecurity, etc. Poor attendance may also be due to research overload or perhaps lack of interest in the topic areas (although this did not seem to be the case).
However, the Delphi process was severely limited by this low attendance rate. Only 34 first-stage replies (before the workshop) and 10 second-stage votes (at the national workshop) were received. In an (less than ideal) attempt to gather additional data on the applicability of the results to other contexts, the BARS questionnaire was issued again more widely, giving the 73 replies analysed (albeit cautiously) in Chapter 6. Although every effort was made to approach all those directly interested in alcohol-related harms and commissioning interventions, and so on (as a census sweep), the small number of replies does not constitute a representative sample. Of the responses received approximately one-third of these were from the north-east of England. The results and analysis presented should be viewed with caution.
Researching in a changing context: meeting the study’s objectives
In this section we revisit the study’s original objectives in order to consider whether or not these were met (and to what extent), and the steps taken to ameliorate any ‘drift’. We make reference to where, in the report, these issues are explored in more depth, and the impact on the study is discussed. We consider the objectives in turn and include what we learn from these efforts.
Objective 1: to engage research participants in the research process (cocreation of knowledge) (meets research questions 1, 2 and 3)
Chapter 2 outlined our plan for cocreation (see Researching in new ways: participatory approaches and the promise of cocreation) and Chapter 3 described the plan for making this happen (see Research engagement process, Operationalising the project and A study designed to cocreate knowledge), while Chapter 7 reflected on what actually happened in detail. The reality did not match the promise or the plan. Despite agreeing with our case sites in advance the core conditions that were needed in order for us to be able to select them as a case site (that they pick the topic and set-up a small local group that would work with us to jointly interpret the findings), ultimately these promises were not met. Sometimes participants prioritised other activities and meetings were cancelled or poorly attended (or attended by deputies). This issue almost exclusively related to Rosetown, where the English context was one of major reform, displaced staff and disruption to ‘business as normal’ as transition was managed. There is no controlling this. In both sites, regardless of discussions and written material, the notion of cocreation remained somewhat unclear. It is likely that efforts to work in cocreation need to be reiterated and negotiated at each stage of the research process. Later, we offer recommendations for further research on cocreation.
Objective 2: to track the commissioning process (and joint planning) in public health for research utilisation (meets research questions 1 and 3)
In Thistletown, we were able to engage all stakeholders in the licensing process and to follow the live action as planned. In Rosetown, the process we were following stalled (as the market research company were unable to recruit participants to the local data collection that they were commissioned to undertake). In addition, in Rosetown we were unable to engage the major providers of services fully in the research process (non-attendance at joint interpretation meetings).
In both sites there were places we did not get to see (the Licensing Board in Thistletown and the informal contacts between commissioners and providers in Rosetown), meaning that the full process was not observed. Both of these points appear to suggest that we failed to meet our objectives, and this is one way of understanding what happened; however, an alternative is possible, that these are simply findings. The first identifies a deeper concern between Rosetown commissioners and providers and the second indentifies that there are hidden places in commissioning and joint planning processes. These points are examined more closely later in this chapter and their implications are explored. At the very least, it is fair to say that we did not get to see the full commissioning process in Rosetown and so care must be taken in considering our findings; later we recommend further research in this area (see Recommendations for practice and future research).
Objective 3: to identify research management roles (and the opportunities and challenges posed by this role) (meets research questions 1 and 3)
We interviewed 32 ‘research managers’ across the two sites, carried out six observations and conducted two focus groups (n = 5 and n = 6); Chapter 3 identifies the various roles they held (data analysts, library services, intelligence managers, etc.). The findings from these interviews are incorporated into Chapters 4 and 5 (Thistletown and Rosetown findings). The overall conclusion is that these roles do exist and they represent the organisations’ efforts to identify, collate and mobilise evidence into planning and commissioning processes, but that the role incumbents felt that their skills and capacity (and opportunities to enter evidence) were largely underused. Later, we offer recommendations on how to use the skills of research managers more fully (see Recommendations for practice and future research).
Objective 4: to explore individual and organisational learning around research utilisation (embedding and sustaining research utilisation) (meets research questions 1, 2 and 3)
We were able to investigate this. In both organisations, the first formal organisational efforts to pull in evidence occurred at the time when strategic documents were refreshed, to identify organisational priorities and to provide an evidence-informed backdrop to subsequent activity. The second organisational response to ensuring evidence use was the provision of the ‘knowledge manager’ roles (as above). However, the use of the knowledge manager’s skills, and the decision to pull in evidence beyond that included in strategic documents, came down to the individual commissioner’s or planner’s time, interest and knowledge. Later in this chapter we offer recommendations for practice and future research in this area.
Objective 5: to explore any link between research utilisation and organisational performance (the impact) (meets research questions 1 and 3)
We were not able to meet this objective, as discussed more fully in Quantitative analysis: an aspiration too far? Quantitative data are included in the case study findings chapters (see Chapters 4 and 5) to help the reader to understand the public health issue in the case study sites, the extent of the problem, the trends over time and how the site compares with other parts of the UK (without identifying the site). We identify some very real gaps in routine data and later we offer recommendations in this area (see Quantitative analysis: an aspiration too far?).
Objective 6: to explore and discuss transferability of the findings (meets research questions 1, 2 and 3)
Here, again, our efforts were only partly successful. We were able to feed back and validate emerging findings with our case participants. We were also able to share these findings and explore their applicability more widely in the case studies in two local workshops (n = 23 and n = 15). However, the response rate to the Delphi questionnaire and the attendance at the national workshop (as detailed above) were poor, giving us only a limited understanding of how these issues may play out elsewhere. We have discussed these issues above.
It may be obvious to state (but we will) that in turbulent times, when organisations, processes, roles and personnel are changing, planning and conducting empirical research is challenging. The research team has little control over the activity and can only attempt to work with it and around it. Perhaps, in some ways, this research gives examples of the level of disruption experienced by practice colleagues (at least in Rosetown) during this period (and is a testament to them). The following section will consider the contribution of this study.
The contribution of the study
Reaffirming earlier findings
This study provides two detailed empirical accounts of evidence use in the processes of public health commissioning and planning, which have been advocated as a gap in the evidence base. 61,91 In doing this, it replicates findings from earlier studies that have looked at research utilisation and knowledge mobilisation and reiterates that the process is heavily dependent on several factors. The nature of the evidence itself is patchy; it is often seen as dull, difficult to understand, not timely and historical, not a good fit in local contexts and not always fit for purpose, and therefore it is contested. The uptake of evidence is more likely if it is presented by a trusted and credible source in an active format, if it is relevant to the context and tasks at hand, if key messages are teased out, and if the significance of the evidence is negotiated across stakeholders in trusting and ongoing relationships. In addition, research evidence is only one form of evidence and information used in making public health decisions, and it competes with other local pressures and vested interests; tacit knowledge is always embedded in decision-making, which is seen as an art form rather than a science. 90 Below, we move on to the contribution of this study. It will become clear that there are more commonalities across the sites than there are differences.
Public health and the dominant programme theory
We start the realist analysis with two key observations:
-
Both sites were carrying out activity in line with the EU and with the HICs laid down in national guidelines (see Chapter 2) for reducing alcohol-related harms and, therefore, at the highest level, both sites were carrying out evidence-informed commissioning and joint planning.
-
Both case study sites collected local evidence for themselves. In Rosetown this may be explained by the inconsistencies in national evidence relating to the effects of maternal alcohol consumption in pregnancy. However, in Thistletown, local data and evidence was similarly collected, despite a well-established and robust evidence base. We conclude that the collection of local evidence serves a purpose beyond merely filling in the gaps in the evidence-base.
We return to both of these pivotal observations in what follows in this chapter. We identify and explore the predominant programme theory and the context–mechanism–outcome (CMO) combinations at play in the use of research evidence and other types of evidence in the commissioning and joint planning to reduce alcohol-related harms in the case study sites. We consider structure and individual agency as they facilitate the use of evidence, and what follows takes into account the extent to which the findings resonate across different contexts as illustrated by the national seminar and the BARS questionnaire. To do this, the argument moves away from a simple description of CMO combinations, in order to cope with the complexity of the process. As explained below, what is identified as the best course of action is not clear until who, where, with what resources, when, how, and for what purpose also align. The use of different types of knowledge and information moves iteratively, rising to and falling from prominence, and is not static in the way that a ‘frozen’ CMO combination might suggest. Instead, we give a high-level read-across and present two mid-range theories to account for the findings. One of the unexpected debates our findings open is: ‘what is Public Health?’ We begin by referring to the World Health Organization (WHO) definition of Public Health to initially close off this complexity, but end the chapter with this issue reopened.
The WHO states that public health has three main functions: the223
. . . assessment and monitoring of the health of communities and populations at risk to identify health problems and priorities.
. . . formulation of public policies designed to solve identified local and national health problems and priorities.
. . . assure that all [. . .] have access to appropriate and cost-effective care, including health promotion and disease prevention services.
In other words, Public Health has a defined (if broad) remit and three identified activities through which this remit is to be met.
The programme theory that underpinned and shaped the activity at play in this project (research utilisation and knowledge mobilisation) can be simply expressed as: Research evidence will enable Public Health functions to be met more easily (i.e. the three bullet points above). We chose to focus on the dominant programme theory (largely the one at work in Western societies that value and privilege scientific evidence). In a complex study this focus was helpful to hone the data collection (interviews and observations). Within our focus on commissioning and joint planning, our data address primarily the second two bullet points (the design of Public Health policies and the provision of appropriate Public Health services and interventions).
What works (for whom)
It is clear throughout the project that what counts as evidence (proof) or ‘valid knowledge’53 varies considerably across stakeholder groups. The internal validity of research evidence,52,53 that is the robust and rigorous process through which the evidence is established, gives that evidence a privileged (factual and generalisable) status for academics (and practitioners steeped in evidence-based traditions). However, decision-makers often look to the external validity of different types of evidence. When other forms of data and information appear to tell them more about their world and reflect it more accurately (external validity) than research, evidence (regardless of the precision with which it is produced) is relegated in favour of more salient representations of their immediate world. Participants at each site often referred to things being ‘different here’ or ‘special’, meaning that the conditions under which the research evidence might hold true did not, in fact, apply, rendering its message less applicable.
‘What works’ is about not only the nature of the evidence, but also its mobilisation. How evidence is introduced and by whom and the trust, credibility and likeability of the messenger all impact on the likely uptake of that evidence into the decision-making process. Participants regularly reported using local opinion leaders to introduce material that they themselves would not be able to champion. Messages had to be ‘short and snappy’, relate to the day job, and have clear local implications, with the preferred model (for maximum impact) being face to face (verbal and visual) as opposed to written, because of the immediacy of the former’s impact.
A mid-range theory: saliency and immediacy
Pawson and Tilley171 talk of the ‘mechanisms’ that, within a context, are the sociocultural active ingredients that shape the outcomes that are observed. In both of our sites the sociocultural mechanisms that encouraged the use of evidence (of all kinds) were twofold. One mechanism was the perception of local fit and relevancy to the situation at hand: saliency. The other was the immediacy of the evidence. To use slang, taken together both mechanisms make the message in the evidence ‘up close and real’ and about ‘here-and-now’. Below, these ideas are developed further as the core mechanisms that mediate to encourage (or discourage) the use of evidence. Several factors inter-relate to influence how useful (and usable) any information will be seen to be. We can explain this through a mid-range theory based on these two linked mechanisms: saliency and immediacy. Evidence that is created here-and-now, as opposed to there-and-then, is considered more salient. Evidence that is presented face to face, perhaps visually, and with active interpretation to pull out the key messages has greater immediacy than evidence that is passively presented in written documents. The closer the evidence to the setting of its use (geographically and temporally: context), and the more actively that evidence is introduced (explained, visualised, debated: mechanism) the more likely it is that it will be used (outcome). Researchers cannot assume that the evidence ‘speaks for itself’, or that decision-makers will share their views that the evidence is generalisable and will work anywhere. Figure 11 conceptualises this relationship. The direction of the arrow (overlaid) indicates the increasing likeliness that evidence will be used.
FIGURE 11.
The importance of saliency and immediacy as the sociocultural mechanisms in achieving research utilisation.
Ironically, this suggests that what counts as useful evidence and valid knowledge is intrinsically tied to where it is used and by whom. The data move us away from a simple linear view of research utilisation (i.e. it is more complex than first deciding what to do, and then when, etc.). Clearly, building relationships is important, too, but we consider that to fully understand the use of research evidence, a complex systems view is needed. We develop this argument below. While we continue to make distinctions between what, whom and where, as a heuristic device in order to discuss the findings, we acknowledge that ultimately they may prove to be artificial, as it may be more useful to look at the totality of the decision as it is made and not at the component parts.
Where: evidence entry points
To use research evidence, it has first to be available to be used. Our findings suggest that there are provisions within organisational arrangements in both case study sites to pull in, use and embed research evidence to inform high-level decision-making for both commissioning and joint planning. However, these official evidence entry points are few. The most obvious evidence entry point is when research evidence is both pushed and pulled into each cycle at set times to renew strategic documents (e.g. Joint Strategic Needs Assessments; policy, strategy and planning documents). Most commonly, the research evidence is used to detail the nature and scale of the problem: what it is, where it occurs, and possibly how it is changing over time. This embedded evidence is then used to set organisational priorities. In this important way, all activity that flows from these priorities is evidence-informed. However, research evidence used through this evidence entry point is used rather like floodlights to illuminate a football pitch, showing where to play and take action, but not directly shaping any game that is played there or how it is to be played.
There are implications in relying on these high-level evidence entry points alone. This embedded evidence may raise understanding and awareness of the problem, but not necessarily provide guidance on what to do about it; for example, local actionable messages. The nature of these documents is that they list the priorities in overarching, generic terms (e.g. ‘alcohol’). The precise shape, direction and extent of any activities taken to address these high-level priorities are decisions that are taken downstream in the organisation (i.e. what should be done to meet the priorities). At this ‘lower’ level, when actual commissioning or planning decisions are taken, the use of evidence is largely a person-dependent system, with evidence mainly being mobilised through the efforts of individuals rather than structures or processes, and is, therefore, necessarily patchy and variable.
Some of the formal evidence entry points that did exist were not always used (e.g. the Licensing Forum pushing evidence to and requesting information from the Licensing Board in Thistletown). From this example, the findings suggest that a combination of formal structures is needed (the points where evidence enters), backed up by a requirement to use them and a cultural acceptance that this is the ‘done thing’.
Organisational responses to facilitating evidence use (research and public health data and intelligence) establish various roles (knowledge management, knowledge officers, information scientists, data analysts, etc.) whose remit is, variously, to identify, gather, collate and push research evidence into the cycles. However, these potential evidence entry points are not necessarily used. Whether or not the data analysts’ skills (or those of information scientist, etc.) are drawn on depends on the individual commissioner’s capacity to know that there may be evidence and how to access it, know what it means when they find it, interpret how to apply it locally and have the time to take these actions in collaboration with stakeholders. At these crucial times there are no formal organisational or structural requirements to bring research evidence to the table. In this way the evidence entry points we observed were used to formally pull in research evidence to identify need, but they were used far less so later on in the process as activities to address that need were decided on. Flow of evidence (sometimes cascade of evidence) was not well developed and nor was it seen as an expectation of the process; rather, it was left to the discretion, capacity and capability of commissioners/planners.
What works: asking different questions of the data
One way to understand why it is not seen as crucial to identify the evidence base at the point where decisions on services and interventions are made is that there are few actionable messages to be found there (i.e. it is of little use given the task at hand – ‘not fit for purpose’). 53 When researchers ask ‘what works’ and practitioners ask ‘what will work’, deceptively they appear to be asking the same question of the data. This is not the case. The same words are used but a different question is being asked. Anecdotally, when listening to accounts of the effectiveness of Public Health interventions emerging from research evidence, practice colleagues often say ‘oh that would never work here . . .’ If taken at face value, this seems to run counter to the understanding of research evidence and the research process. Efficacy means that significant health improvements have been measured following an intervention introduced under controlled conditions (across a comparator) to establish proof of concept – that the intervention can improve health outcomes. However, that is not to say that in any one specific context it will. Scientifically, it is perhaps more accurate to claim that under the conditions under which the effect was observed the outcomes (health gains) followed, but not that those conditions (context, backdrop, and fidelity of the interventions) can and will be replicable elsewhere. Field trials are not always available – and even then ‘fields’ differ. It seems to be this difference to which practice colleagues refer.
The case study sites reported ‘It’s different here . . .’, ‘We are special . . .’ and ‘. . . the problems are different . . .’ when asked about the ‘fit’ of national evidence to the local context, meaning that the prevailing backdrop (context) may make it impossible to reproduce the intervention, as it was designed, in order to replicate the health gains. When practice colleagues say ‘that would never work here’, they are not challenging the efficacy (that under the prescribed conditions it will work) but asserting that they could not get it to work – that it would not be implementable locally in that way. The precise conditions under which health gains will follow cannot be replicated. ‘What works’ (although the same words) means different things to Public Health researchers and practitioners.
To understand this dilemma, we may understand the practitioners’ focus as ‘what might work here and now – what can I get to work – how do I do that?’ It is, typically, not that the practitioner is necessarily unaware of the research evidence findings, but more that following the findings is not a recipe. To use an analogy, in practice they may blend ingredients from multiple Public Health recipes according to cooking techniques (tacit and professional knowledge) developed over years, to find a blend that will work in their kitchen to meet their customer’s needs – a kind of local ‘Public Health plat-du-jour’. This departure from the pure science of public health is not necessarily pathological, but fiercely pragmatic. In order to secure what interventions are possible (those that are feasible, acceptable) our practitioners (in the workshops) say that it is better to perhaps, suboptimally, ‘Chip away’ and change things over the longer term rather than fail with unimplementable approaches. Again, taking a complex system view helps us to explain this. 221 Having evidence that is ‘. . . good enough’, that moves things generally in ‘the right direction’ and that is achievable for now, even if is not perfect or likely to last forever, is the best way to deal with wicked issues. 72–74
The interlinked nature of public health issues
Again, we see that what works, for whom, where and under what conditions are parts of the same question; they are not separate, distinct factors. Instead of considering these as sequential considerations when making a decision for commissioning or planning (first decide what, then where, then how, etc.), we suggest that the answer to one part of the question depends on the answer to the others. Public Health issues are not single-stranded and solutions cannot be discrete or compartmentalised. No one part of the question is answered until they all are. The solution is not suddenly apparent, but emerges over time and through discussion as a resultant whole. This slowed-down evolution of solutions opens decisions to influence at different parts of the process by different groups; this allows any final solution to reflect many factors and concerns. The final decision is held in abeyance until the decision-in-progress hits on a solution-in-practice – typical of wicked problems and complex systems, where solutions are only ever partial and temporary.
Under what conditions: context
We move on to consider what these situational factors are that shape ‘what works in situ’. The fieldwork suggests that multiple types of evidence (e.g. research findings, local statistics and local opinions, performance and health data) and information from what has been tried before and how that went (organisational memory) may be drawn in to inform commissioning and planning decisions and weighed against other considerations. These other concerns are largely contextual factors that need to line up to allow any approach to proceed. There needs to be a consideration of ‘local fit’ (i.e. saliency) with regard to the acceptability and feasibility of the approach: how things can be accomplished (with what partners, through what champions and stakeholders and across what managerial mechanism – commissioning or joint planning). Then, all of this is considered against a backdrop of wider forces (political influences, macroeconomic concerns and performance measures). Figure 12 illustrates some of this mix, as identified in our fieldwork.
FIGURE 12.
The dilemmas of using Public Health evidence in situ.
This is a simple and simplistic representation and almost certainly not exhaustive. We use it here to reinforce:
-
that research evidence is only one ‘part of the jigsaw’69
-
that at the centre of the process is a human being subject to human strengths and foibles (peer pressure, forgetfulness, stress, resilience, and capacity to build alliances) and, therefore, commissioning and joint planning are ultimately social activities
-
that if these are the questions and dilemmas that commissioners and joint planners face, they are unlikely to all be answered by traditional Public Health research methods
-
that ‘outcomes’ of commissioning decisions do not follow research findings in any straightforward way.
The curious cases of the collection of local data
Figures 10 and 11 also help us to understand why, regardless of whether or not there are plentiful national data and a well-established evidence base (low saliency and low immediacy), both sites conduct their own local research (high saliency and high immediacy) to provide evidence that they both trust and can use (CMO combination that encourages evidence use). A further explanation is that local commissioners and planners are trying to answer the questions that national evidence does not (and perhaps cannot): what actions to take and how to achieve that change in their setting. However, other factors also seem to be at play. In Rosetown, lack of consistency in national guidelines regarding what public health messages to give to local women regarding drinking in pregnancy made it difficult to identify what interventions to commission. Logically, this prompted the commissioning of Insight work (the process case study we followed) to determine local acceptability and feasibility of any approach that would be proposed. This gives a direct ‘test’ (here and now) of the future likely success of any intervention and allows the public to help shape that approach. However, there is a hidden conundrum: the national evidence base already contains messages about what is typically acceptable to women and to midwives in delivering these messages (from various public health topics: smoking in pregnancy, drugs screening, weight management, etc.), and from our data it is clear that the commissioners knew (or at least largely anticipated) what the local research would find – so why was there a need to collect these primary data? Overtly, it would appear that efforts are simply being duplicated. From our data the reasons for this appear to be twofold: a need to localise national findings in order to be convinced that the approach still holds merit locally (high saliency and high immediacy); and the need to be seen to do something. Both these are explored more fully below.
It is important to commissioners to be seen to do something, not only to address local need but also to address multiple concerns (getting work initiated across new partners, securing early wins across the new public health system, building new relationships and spending allocated monies). Accountability within the role of commissioners and planners is not all about the achievement of key result areas but is in managing informal (and largely covert) relationships and expectations too. This brings face validity to their activities (i.e. they appear to be doing what they should be doing) and helps to maintain credibility across stakeholders.
The need to localise evidence is very strong; high saliency and high immediacy seem to be core ingredients in mobilising evidence into action. A simple way of understanding the high saliency and high immediacy quadrant is that evidence here is ‘up close and real’. In the next section, we continue analysing the implications of this, beginning with a look at the role of local evidence. It is the findings of this local research and data gathering that are more influential in shaping Public Health commissioning and joint planning decisions (i.e. helping Public Health practitioners achieve their aims: improved local outcomes) in both case study sites.
It is in the artful craft of seasoned commissioners and planners in Public Health and the partner groups with which they work that the balance between these competing forces is resolved in order for action to be taken and the system achieves stability by consensual adjustment of competing perspectives. 221
Tacit knowledge and professional identity inform their behaviour in relation to not only what to do, but how best to proceed with that and the wider factors that may still impinge on any decision. In Thistletown, local crime data, footfall and safety statistics that identify problematic areas suggest what factors to take into account when deciding whether or not to grant licences. Relationships with colleagues and the wider council agenda also informed choices (i.e. economic vibrancy and local reputation). We suggest that it is not possible to completely predict, in advance, which one of the factors in Figure 13 will be the most influential in any one setting. It is possible to imagine polar extremes; for example, where a ground-breaking piece of science changes routine practice forever, overnight. It is equally possible to imagine situations where local political pressures dominate what action is taken, and equally it is possible to think that local people can make or break plans. It is to these three groups, science, politics and the public, that we offer a conceptual way of thinking about these competing tensions and a mid-range theory (see Figure 13).
A mid-range theory: blended solutions and a visual schema
Above, we discussed that commissioning and planning decisions are held in abeyance under consideration subjecting them to multiple contextual factors. In Figure 13 we conceptualise this process as a tension across three main forces: politics/influence (power), science and evidence (truth), and public and acceptability (legitimacy).
FIGURE 13.
A visual schema to illustrate competing forces and blended solutions in Public Health decision-making.
To explain the figure we begin by using polar extremes. A fully red decision (ideological dictatorship) would be imposed with force against the will of the people and without scientific basis. A fully blue decision (scientific rationality) would reflect only logic without humanity or consideration of competing world-views. A fully yellow view might be anarchy without political structure or scientific order and discipline, where people would be left to please themselves. What we see in practice is the pull of the competing forces, some weaker and some stronger. Any resultant commissioning or planning decision may reflect the influence of the three forces differently, giving it, literally, a different hue.
In Thistletown, licensing decisions taken in line with local culture (yellow) and with strong local political and trade influence (red) contains ‘less blue’ (weaker reflection of the evidence base), giving a more ‘orange’ decision. In Rosetown, some ‘blue’ exists as midwives collect data in answer to the antenatal booking questions on alcohol consumption and practice in accordance with the evidence as it is embedded in guidance from the RCM. However, in the absence of unified messages from research evidence, this message is dissipated. In absence of the ‘blue’ here, the midwives may draw on personal values and personal experience to give legitimacy to their choices (yellow).
Across different settings and even within the same setting (at different times), the strength of each force will differ (by issue, risk, according to the personalities of those involved, the history, etc.). The schema gives a way of understanding local variation and permits reflection on the somewhat unpredictable ebb and flow of contextual factors in shaping local public health decisions and the nature of interventions (outcomes). The use of evidence in public health decision-making (in both contexts) is fiercely pragmatic, requiring the balancing of scientific evidence with professional judgement in the light of prevailing local conditions. Our findings suggest that this is vital in achieving the sound application of evidence of all types.
Democratic legitimacy
In our case study sites (particularly in Thistletown), the participants referred to the need for any decision taken to have democratic legitimacy. By this they did not mean that the public health decisions taken were subject to a vote or consultation, or that the decisions had to be taken by democratically elected members. To understand what they did mean, it is worth returning to the schema and looking at the yellow force: the ‘people’. In neither of our case studies did ‘the people’ directly get to input to the commissioning or joint planning decisions taken: they simply did not have a direct voice. The schema reflects this, and the ‘pure’ yellow line is thin. The public can directly shape decisions via, for example, pressure groups, social movements and protests; however, most often there is no public ‘voice’ asserting direct influence. Yet the blue and red shapes surrounding the yellow have their colours changed by the yellow influence. Public Health decision-makers and elected members alike can, and often do, shape their decisions on what they think will be acceptable to the public, be feasible in their settings and make the lives of local people better. The public presence is definitely there. In our data there was a strong backdrop (almost a moral dimension to the decision-making: trying to achieve the greatest possible good – utilitarianism). The power base behind the elected members (in Thistletown) arises directly from their position in being elected to ‘speak for’ the people. The power of the foundation trusts in Rosetown arises directly from their capacity to deliver (or not) services to so many people. The people’s voice (the democratic legitimacy) is negotiated among the players active in the commissioning process (commissioners, planners, elected members, etc.) and most often not directly by members of the public, but the assumed public voice is strong. To twist Derrida’s notion, the public is almost like a ‘present absence’. 224 Exactly who is seen to speak most faithfully on behalf of the people (what they want, what is important or good for them) was often at the heart of the differences in views across the stakeholder groups.
The ontological status of the schema
Realist approaches permit the building of mid-range theories. We suggest that the schema above is a mid-range theory presented visually;225 however, there are a number of caveats and limitations to the schema that need to be outlined. First, here, in using the schema to help understand the use of Public Health evidence in commissioning and joint planning we envisage two other competing forces: political forces, and public concerns and aspirations. There could be others; for example, economic forces, risk/safety and legal forces. In the case study sites where economic considerations were part of the action, they clearly fell into the political domain (e.g. in Thistletown as the influence of the ‘night-time economy’). It is possible that if the figure is extended as a heuristic device to explain other situations where competing forces act concurrently to shape planning and practice activity, then the forces identified may change to give different pulls across the schema.
Second, colours are allocated to the competing forces to illustrate the resultant blend between the forces. However, even the colours themselves cannot be taken as constants; given the complexity of the world and its ever-changing nature, what constitutes bright blue today might be different tomorrow (as the evidence base changes). Similarly, what constitutes bright red (power/influence) might also change as elections see the rise and demise of different influential groups, and so the colours themselves may mean different things over time. In other words, although the colours show the blend of factors, we do not know what they represent in the real world. The public health interventions put in place today as the result of a light purple blend may be very different from what that same blend would have looked like 50 years ago or 50 years in the future. The schema and the ideas it illustrates are abstractions, perhaps best used to prompt thinking about how the forces inter-relate, consider the implications of this and devise possibly more useful blends, rather than suggest what direct activity any particular colour represents.
Third, it is necessary to demonstrate the capacity of the schema to illustrate our case study findings. In Thistletown, the full range of dilemmas involved in the consideration of licensing applications can be illustrated in the following example (we have modified the example to preserve anonymity).
There is a (fictitious) piece of local wasteland in an alcohol-dense (overprovided) area that a large supermarket chain wishes to develop. Their planning application includes an application to sell alcohol. Perhaps strict adherence to the local alcohol policy statement might lead to the rejection of the application in an overprovided area (blue). However, there is a rebuttable presumption to grant the licence (red) and legally it is not easy to demonstrate the contribution of this licence to health-related harms, as the premises have not yet sold any alcohol (red). In addition, the supermarket will bring jobs (yellow, red and blue) and sell fresh fruit and vegetables (yellow and blue), bring redevelopment to a crime-ridden area and pay for some upgrading to the roads and footpaths (yellow, blue and red).
It is difficult to be a purist adhering only to the research evidence as it is embedded in the alcohol policy statement (alcohol-related harms correlate to alcohol availability and outlet density) because, as the figure shows, a solution will/needs to meet multiple competing forces. Even if the solution disregards the alcohol research evidence, and the licence is granted and the supermarket is built, it may lead to health gains in other areas. Overall greater health gains may result from granting the licence. Ironically, a blend with less ‘blue’ may give greater health gains, suggesting that not all health gains (or the inter-related complexity between them) are captured by a siloed evidence base.
The schema shows us the inherent complexity in public health issues: what is in the interest of the public’s health? This issue is typically closed down by the controlled designs used to create (single-issue, often siloed) public health evidence, but re-emerges during the commissioning and joint planning processes. In this way, the figure allows us to see that debate, blending evidence types and sources, is not pathological but can result in commissioning and planning decisions that, while they seem to work against the strict research evidence base, actually may result in unrelated wider health gains. This involves taking a wide view of public health. In our democratic society the adversarial role of competing groups maintains checks on the excesses of a single approach and the same may be the case here.
In Rosetown, similar competing forces were observed. Midwives’ reluctance to ask questions regarding alcohol consumption of the pregnant women in their care could be interpreted as ‘resistance’ and a wilful disregard of the evidence base. However, this holds true only if one considers that the research evidence (e.g. on FASD) is the only possible health outcome. Our data suggest that practitioners may, in fact, believe that they are acting for the greater good (and for the wider health) of the women and their unborn children in not jeopardising the relationship and continuity of care. This also raises questions about who is the focus of public health interventions, mother or baby, when there are tensions in the evidence about what works best for whom and at different times. Answering realist questions in public health issues is complex.
The managerial mechanism: commissioning versus joint planning of Public Health interventions
So far we have discussed the similarities in the use of research and other types of evidence in both of our case study sites. Our findings suggest that the similarities far outweigh the differences; however, there are some differences. We have argued two things:
-
First, that there are few evidence entry points and that formal research evidence tends to enter primarily at a strategic level and only at set times in the process (when reviews are undertaken and priorities set). Later in the commissioning (or planning) cycle, where these priorities are operationalised and interventions designed, the use of evidence and other types of information is largely person dependent.
-
Second, we have also argued that the effective use of evidence means taking account of competing pressures in the context to achieve the most feasible and acceptable blend and work towards a wider appreciation of what is in the interest of the public’s health. In line with this, both case study participants emphasised the importance of discussion, co-ordination and collaboration across partners holding shared responsibility for intervention development and delivery.
It was unclear at the start of this project whether it is easier to secure the use of research evidence via a legal contracting process (across the purchaser–provider split) or within unified organisational arrangements with shared responsibilities. It is now much clearer from the data that the different managerial mechanisms permitted this negotiated space to varying extents. Chapter 5 details the difficulties experienced in creating opportunities to discuss options over the purchaser–provider split. Under commissioning (and procurement) restrictions, participants talked about the difficulties of meeting to agree plans and actions and having to use informal (uncondoned) links across the purchaser–provider split to work towards feasible and acceptable solutions. In Thistletown, under joint organisational arrangements the first thing that happened in our work with them (somewhat delaying our entry) was a meeting across partner organisations to agree participation and decide on the topic (no equivalent meeting happened in Rosetown).
Commissioning attempts to give the commissioning organisation increased legal contractual power (red) to help them to lever their preferred choice across the competing forces, but in doing so this drastically reduces the legitimate opportunities for achieving the democratic legitimacy we outline above. Ironically, in not allowing the blend of the competing forces to be resolved through agreement (finding a ‘natural’ balance within a complex system), other parties may resort to red solutions, too (e.g. the power exerted by foundation trusts and other providers in commissioning processes), in order to push back to redress the balance.
Recommendations for practice and future research
Above, we have used the saliency/immediacy mid-range theory to locate our findings and begin to explain the sociocultural mechanisms at play in our contexts. A second mid-range theory, as a visual schema of the competing tensions at play when making complex decisions to address wicked problems in public health, has been presented. Next, we end by making some recommendations for further research.
We fully acknowledge that this has been a small study, and care should be taken not to overinterpret the findings.
Recommendations for practice
Few evidence entry points
In both sites there were few official evidence entry points where there was a requirement to pull research evidence into the decision-making process (and not all of these were used). When research evidence is pulled in, this is mainly to refresh strategic documents (e.g. Joint Strategic Needs Assessments and strategy documents); this identifies public health need, but not necessarily solutions. Beyond this, the research evidence base is rarely directly considered, unless driven by the particular interests of individual commissioners. With a burgeoning evidence base and increased efforts to share research evidence across sector boundaries (e.g. fuelled by the impact agenda within the research excellence framework), it is arguable that a duty lies with organisations to ensure that they manage their use of evidence more effectively.
Professional expertise, tacit knowledge and local and organisational memory are extensively drawn on to decide on commissioning and planning decisions and activities. Research evidence is pushed into the organisations from several providers (universities directly, evidence synthesis organisations, e.g. NICE, and other national bodies, e.g. PHE).
Recommendation: if organisations take seriously their commitment to use the evidence base, then more formal effort could be taken to mobilise the evidence that does exist. There may be many ways of achieving this (beyond simply circulating paper and electronic copies of guidance, etc.): by actively incorporating evidence on interventions (what works, as well as what does not work) into documentation; creating a standing agenda item in commissioning meetings on the research evidence base; co-opting information scientists, analysts and possibly academics on to commissioning groups (and using their skills); identifying the evidence base in commissioning specifications; and senior leaders setting an expectation that consulting the evidence base is culturally the ‘done thing’.
Gaps in quantitative data
This study has identified some real gaps in national and local data capture in the areas of alcohol licensing and maternal alcohol consumption (with data not recorded consistently, or not collated and located in one place, or based on self-reports, or not collected at all). Performance indicators (and other measures of quality) have changed, as have the organisations both that collected them and on which they are based. Some data are available only within the NHS (see the start of this chapter). It is difficult for commissioners to make sound decisions when they are unable to obtain feedback and assess how well current or previous services are working. It is also difficult for academic researchers to draw sound conclusions using valued quantitative metrics. Without sound data capture, which is both consistent and longitudinal, it is difficult to establish if change is an improvement and is impacting positively on health outcomes.
Recommendation: the recent reforms are now operational and overarching bodies such as PHE are becoming established. It could be timely to review routine data capture and whether or not old systems still capture the right data in the right way for the new structures. We hope that this research is helpful in suggesting some areas of focus.
Recommendations for research funders
Cocreation, timescales and urgent research need
It took more time than we expected to explain cocreation to our research participants, identify the topic and generally negotiate entry – longer than the 4 months built into the proposal. Arguably, if the practitioners with whom we had worked on the bid had been our research participants, this delay might not have occurred, as they would already have been familiar with the approach and research aims and wanted to work with us in this way (rather than having to be ‘persuaded’ as our sites were). This suggests that cocreation will work more smoothly if the practitioners who write the bid are to be the research participants. However, as in all complex systems, this ‘solution’ creates additional ‘problems’. Our aim was to work on a topic that was of urgent pressing need for our research participants, and it is not at all clear if the length of time from writing of the proposal through to outline and full stage of application, contracting and recruitment of researchers, all necessary and rigorous processes, would make this possible. It is likely that any issue identified as pressing would be different by the time funding was actually secured (and even more distant once data were collected and results were known). Our research participants already stated surprise and frustration at the time that fieldwork took to get under way (after ethics and R&D approvals had been sought).
Recommendations: there is no one clear way forward here. Several possible parallel steps may ameliorate these issues. For pressing, urgent practice research needs, a rapid and responsive mode of national research application could be created. This would help to keep national research funders aligned with fast-changing practice contexts and needs. A different approach would be to devote national funding solely to longer-term issues (future big issues). A third alternative is that, if national funding is sought for urgent participant-led topics (i.e. following the approach adopted in this study), considerably more time would be allowed for both negotiated entry and consultation during the write-up stages. This entry period would need to be funded in order to allow the research staff who would work at the sites to work alongside and build trust with the research participants.
Recommendations for researchers and educators
Making evidence fit for purpose
Often, the public health research, data and intelligence identified in this study were focused on the nature, prevalence and trends of the problem and much less focused on ‘what to do about it’. This public health research evidence base was often seen as dull, difficult to understand, out of date, largely irrelevant to prevailing local conditions and carrying no actionable messages. Both sites conducted their own local data gathering to create clear actionable messages. Often it was this small-scale local ‘research’ that shaped commissioning and planning decisions. There is often plentiful narrative and qualitative evidence [the sheer scale of which, some argue, can ‘build its own convincingness’ (Foreword)226]. The increasing attention being paid to the impact agenda with the research excellence framework provides recognition to researchers who work in applied research areas and an impetus for researchers more generally to get their research to ‘make a difference’. Research evidence does not ‘speak for itself’.
Recommendations: while many academics already carry out applied and engaged research, there is still scope to develop these approaches further. Academics might consider widening what counts as ‘good evidence’ (not judged solely by its internal rigour) to its external validity and its capacity to be useful (and used) in practice and draw on wider sources of evidence beyond the confines of the hierarchy of evidence (e.g. Cochrane reviews) and to include the contextual pressures that mediate on the effects of ‘what works’ in messy contexts. Evidence does not speak for itself, but needs key messages to be highlighted and presented in clear, active messages to a targeted audience in a way that helps them in their tasks. 53 Academics might also consider using local respected practitioners to pass on key research messages to inform practice and consider what can be learned from the small ‘research companies’ that carry out locally commissioned work for practitioners: why are their ‘research products’ more useful in situ than traditional research approaches? Can any of this learning be used to make robust research evidence more fit-for-purpose, usable and used in practice?
For educators, teaching the substantive health topics is of vital importance, but it is also important to consider teaching students interpersonal skills in persuading and influencing others and in how to have difficult conversations. This may equip them to handle the difficult (sometimes politically charged, sometimes embarrassing) conversations they will need to have in their jobs. This may help to mobilise research evidence in both decision-making and therapeutic settings, without alienating colleagues or clients.
Working in cocreation
Awareness raising was needed to prepare research participants for working in cocreation. The time needed by all stakeholders to engage in cocreation was considerable, and difficult to achieve amid competing pressure (especially in times of significant change). Stakeholders, if not all engaged from the beginning, risk seeing efforts to collaborate as imposing events on them (rather than working with them to find solutions). Not all practitioners (or stakeholder organisations) were willing or able to engage. We reflected in Chapter 7 that because our case study participants were located some distance away from us, we remained ‘outsiders’ and, though we spent short periods of time on site (interviewing and observing formal meetings), this was probably insufficient to build the familiarity and trust needed for full cocreation. We missed the ad hoc conversations, chats in the corridor and impromptu meetings, where some of the work and decision-making associated with planning and commissioning happened (i.e. in the informal spaces between formal meetings).
Recommendations: build in additional, funded research time to facilitate the approach and be prepared to explain what is being done, when and why to raise awareness at all stages of the processes. Cocreation worked better where participants were fully informed (and understood) what was involved, where all stakeholders were willing to engage from the start and where competing tensions were accommodated (e.g. emergency meetings) and collaborations were managed side by side. Simply adopting the orientation of ‘trying to be helpful’, of being ‘guests’, and ‘not being the experts in the room’ was a good starting place. In practical terms, personalities, historical tensions and ongoing churn will be present in any context. A non-judgemental, pragmatic approach, non-partisan and with incremental adjustments to remain feasible, allows the approach to be adapted to the setting rather than sticking to a regimented plan. It is likely that cocreation might be more successful (build greater trust, reach more ‘hidden’ areas) if conducted on site (through secondment and possibly within an ethnographic design).
Recommendations for future research
Researching the new public health system’s use of evidence
It is difficult to identify examples where research evidence is used directly. We found only one example where a piece of research evidence was directly used. Clearly if evidence is used, the overt use of individual studies is not how this is done. Most often an evidence-informed backdrop is assumed (informed by the strategic documents), but this is largely unspoken. More work is needed to understand what using research evidence actually entails if not direct, instrumental application. In addition, we did not get to observe the HWB in Rosetown, as they declined to take part in the research as they were still in the process of formation and grappling with their own key processes. Given that they are the main receivers and synthesisers of research evidence, and responsible for its cascade and flow in their formation of the Health and Well-being Board strategy and the Joint Strategic Needs Assessment (the main evidence entry point), and that these documents shape all the activity that follows, their role in the promotion of research mobilisation is paramount.
Recommendations: it would be useful to research this process further. The new public health system (and wider health and public sector landscape) is so new that it is not clear how, why and what types of information are included (or excluded) from these documents and to what extent practical solutions for local action are (or can be) identified and who does this work. It is also unclear what the roles of other key stakeholders are in this process (e.g. CCGs, PHE and NHS England). It is likely that evidence (of all types) is flowing not only downwards from the HWB, but also across and backwards and forwards between these bodies. It is unclear what evidence entry points exist in each of them and how these bodies manage these processes both internally and across their boundaries to their partners. These processes have the potential to spread research evidence use more widely or alternatively curtail and close off research evidence use locally (reducing variability in practice in a non-evidence informed direction). This merits further investigation.
Researching the elusive nature of commissioning
Commissioning and planning as activities were difficult to pin down [i.e. not a clear-cut thing (one-off decision) happening in an easily identifiable place]. In the research we interviewed key stakeholders and observed the places where we believed the core activity to be taking place, but we did not get to see all of the places (i.e. Licensing Board meetings in Thistletown and informal meetings between commissioners and providers in Rosetown, or e-mail correspondence in either site) where some of the decision-making/action undoubtedly took place. In our study (and our data which is based solely on one commissioning site), research utilisation was made more difficult to achieve across the purchaser–provider split because it reduced the opportunities for interaction on which evidence flow largely depends. 17,19,21,30,50–52,55,69,74,75,77,81–86,90–92 The non-engagement of all stakeholders removed the opportunity to ‘achieve stability by consensual adjustment of competing perspectives’. 221 However, in practice there are different models of commissioning225,227 and the same may not hold true for all of them. It may be that engagement and ownership (or lack of it) are the key sociocultural mechanisms for achieving the sharing of knowledge (and that this is, in turn, facilitated by saliency and immediacy as we have attempted to illustrate) that is driving this finding. Our conclusion here is based on our data which suggest that, within joint planning, engagement is required, and within commissioning engagement is sometimes rejected in favour of the ‘Chinese wall’ that contracting (non-transparent and at a distance) offers. In summary, joint planning provides the context (C) whereby the mechanism (M) (of engagement and ownership) flourishes and helps to shape the outcome (O) of shared knowledge (CMO combination).
Recommendations: it would be helpful to understand how evidence flows in the different models of commissioning. In addition, research that shadowed commissioners and decision-makers to explore the activity that happens in the informal spaces (and the role these places serve in shaping evidence use) to illuminate the full process would be helpful in understanding how to shape evidence for different purposes. Formal research accounts (data and statistics) may fit better with strategic documents, but personal accounts (patient and client stories) may be more impactful in the informal spaces.
Is cocreation worth it?
Our research participants did not fully engage in the research process. Partly, this might have been due to lack of capacity or capability, or the unsettled and changing context in which our fieldwork took place. Our research shows that, where interpretation was shared, new insight was gained and that our research participants believed that involvement in the research had moved their thinking forward. We were able to point not only to such intangible outcomes of working in cocreation but also to some tangible changes (e.g. the creation of a new post in Thistletown to supply public health evidence the gaps in the process that we identified). It would be useful to know more about the barriers and facilitators to active engagement and whether or not the time, effort and resource (meeting time) on the part of the research team and the research participants make this effort worthwhile (tangible and intangible results and possibly cost–benefit analysis).
Recommendations: within the ever-growing emphasis on translational research (impact ‘making a difference’ and ‘what works’) researchers are increasingly working in new ways and in places (mid-way between academe and practice) more than ever before, yet the methodological development for doing this well lags behind. Barriers and facilitators to evidence use are increasingly well known, but what is less well known is how a research evidence base and research efforts can address these. A stream of funding dedicated to the in-between world of knowledge brokerage20 would move theory and research practice beyond viewing translational research as merely a dissemination activity.
Competing tensions in shaping public health interventions
There would be merit in exploring how the competing tensions schema (above) holds up to further testing. As a visual schema, it would have immediacy and, if the competing forces relate to the setting in question, it would also have saliency. It could prove to be a powerful tool in allowing different stakeholders to examine their decision-making and depersonalise sensitive issues that are hard to raise in order to identify the forces that shape the final outcomes. This would enable those making the decisions to ‘stand back’, view their decision-making process more objectively and consider all of the issues and pressures openly, and allow them to assess how, what and if they should privilege different forces. This could empower all stakeholders in the decision-making process. It would also be useful to explore the mid-range theory that identifies immediacy and saliency as core mechanisms in facilitating evidence use and the implications that this may have for the way research findings are created and offered for uptake.
Acknowledgements
We acknowledge the support of the National Institute for Health Research Health Services and Delivery Research programme in funding this project. We also acknowledge the input of Dr Rosie Ilett, formerly of the Glasgow Centre for Population Health, who commented on and helped draft the original research proposal, and Professor Luke Vale, Health Foundation Chair in Health Economics, Newcastle University, who provided additional economic input. This report is dedicated to our research participants who took part in this project and who worked with us in cocreation.
Funding
This project was funded by the National Institute for Health Research Health Services and Delivery Research Programme (funded call 259).
Disclaimers
The views and opinions expressed therein are those of the authors and do not necessarily reflect those of the Health Services and Delivery Research programme, National Institute for Health Research, NHS or the Department of Health.
Contributions of authors
Rosemary K Rushmer Rosemary provided leadership to the project, carried out the overall analysis and wrote the final report.
Mandy Cheetham Mandy was responsible for the fieldwork, analysis and write-up of the English case study site and contributed to the cross-case discussions.
Lynda Cox Lynda attended the early meetings of the project advisory group and commented on the early plans for the project.
Ann Crosland Ann attended the project advisory groups and commented on the early plans and emerging analysis.
Joanne Gray Jo carried out the health economics analysis and contributed to the final report.
Liam Hughes (retired; previously National Adviser for Healthy Communities, Local Government Association) Liam attended the project advisory groups and commented on the early plans and emerging analysis from a local government perspective.
David J Hunter David attended the project advisory groups and commented on early plans, emerging analysis and the draft final report.
Karen McCabe Karen was responsible for the fieldwork, analysis and write-up of the Scottish case study site and contributed to the cross-case discussions.
Pete Seaman Pete attended the later meetings of the project advisory group. He commented on the draft report and added material on the Scottish policy context to the final report.
Carol Tannahill Carol helped to broker access to the Scottish case study site. She commented on the early plans for the project.
Peter Van Der Graaf Peter was responsible for the knowledge manager interviews on each case study site, their analysis and write-up and contributed to the cross-case discussions. He also analysed the Delphi questionnaire and provided day-to-day project management.
Data sharing statement
As a largely qualitative study, data comprised the opinions, views and aspirations of the participants. Information for participants, and the consenting process, made it clear under what conditions participants were agreeing to share their views with us (for this study only and in the publications that arise from it). To use this data for other research purposes, participants would need to be contacted and re-consented. We do not think that this is practical in this case given the level of public sector reform and displacement of personnel since the data was collected. The sources of the small amount of quantitative data are clearly outlined in the report and are freely available.
Disclaimers
This report presents independent research funded by the National Institute for Health Research (NIHR). The views and opinions expressed by authors in this publication are those of the authors and do not necessarily reflect those of the NHS, the NIHR, NETSCC, the HS&DR programme or the Department of Health. If there are verbatim quotations included in this publication the views and opinions expressed by the interviewees are those of the interviewees and do not necessarily reflect those of the authors, those of the NHS, the NIHR, NETSCC, the HS&DR programme or the Department of Health.
References
- Coalition for Evidence-based Policy n.d. http://coalition4evidence.org (accessed January 2014).
- National School of Government n.d. www.nationalschool.gov.uk/policyhub (accessed January 2014).
- House of Commons Science and Technology Committee . Scientific Advice, Risk and Evidence Based Policy Making 2006. www.publications.parliament.uk/pa/cm200506/cmselect/cmsctech/900/900-i.pdf (accessed January 2014).
- Equally Well. Edinburgh: Scottish Government; 2008.
- Closing the Gap in a Generation: Health Equity through Action on the Social Determinants of Health. Geneva: WHO; 2008.
- Commission on Climate Change and Development . Closing the Gaps: Disaster Risk Reduction and Adaptation to Climate Change in Developing Countries 2009. www2.gtz.de/urbanet/opencommunity/focus/docdetail-ccc.asp?number=7906 (accessed June 2014).
- International Law Discussion Group . Business and Human Rights: Closing the Gaps 2008. www.chathamhouse.org/publications/papers/view/108929 (accessed January 2014).
- Crossing the Quality Chasm: A New Health System for the Twenty-first Century. Washington, DC: National Academies Press; 2001.
- Save the Children . No Child Born to Die: Closing the Gaps 2011. www.savethechildren.org.uk/en/54_14725.html (accessed January 2014).
- Morris ZS, Wooding S, Grant J. The answer is 17 years, what is the question: understanding time lags in translational research. J R Soc Med 2011;104:510-20. http://dx.doi.org/10.1258/jrsm.2011.110180.
- Walshe K, Rundall TG. Evidence-based management: from theory to practice in health care. Milbank Q 2001;79:429-57. http://dx.doi.org/10.1111/1468-0009.00214.
- Health Inequalities Report. London: Health Committee Publication; 2009.
- Potvin L, McQueen DV. Health Promotion Evaluation Practices in the Americas. New York, NY: Springer-Verlag; 2009.
- Nutley SM, Walter I, Davies HTO. Using Evidence: How Research Can Inform Public Services. Bristol: Policy Press; 2007.
- Marmot M, Allen J, Goldblatt P, Boyce T, McNeish D, Grady M, et al. Fair Society, Healthy Lives: Strategic Review of Health Inequalities in England Post-2010 2010. www.ucl.ac.uk/gheg/marmotreview (accessed April 2010).
- Hunter DJ. The state of the public health system in England. Public Health 2008;122:1042-6. http://dx.doi.org/10.1016/j.puhe.2008.03.007.
- Ferlie E, Fitzgerald L, Wood M, Hawkins C. The nonspread of innovations: the mediating role of professionals. Acad Manage J 2005;48:117-34. http://dx.doi.org/10.5465/AMJ.2005.15993150.
- Swan J, Bresnen M, Newell S, Robertson M. The object of knowledge: the role of objects in biomedical innovation. Hum Relat 2007;60:1809-37. http://dx.doi.org/10.1177/0018726707084915.
- Lomas J. Using linkage and exchange to move research into policy at a Canadian foundation. Health Aff (Millwood) 2000;19:236-40. http://dx.doi.org/10.1377/hlthaff.19.3.236.
- Lomas J. The in-between world of knowledge brokering. BMJ 2007;334. http://dx.doi.org/10.1136/bmj.39038.593380.AE.
- Gabbay J, le May A. Evidence based guidelines or collectively constructed ‘mindlines?’ Ethnographic study of knowledge management in primary care. BMJ 2004;329. http://dx.doi.org/10.1136/bmj.329.7473.1013.
- Weiss CH. The many meanings of research utilization. Public Admin Rev 1979;39:426-31. http://dx.doi.org/10.2307/3109916.
- Young D. Home birth in the United States: action and reaction (editorial). Birth 2008;35:263-5. http://dx.doi.org/10.1111/j.1523-536X.2008.00261.x.
- Bekelman JE, Li Y, Gross CP. Scope and impact of financial conflicts of interest in biomedical research. JAMA 2003;289:454-65. http://dx.doi.org/10.1001/jama.289.4.454.
- EB Nursing Practice n.d. www.ebnp.co.uk/The%20Hierarchy%20of%20Evidence.htm (accessed January 2014).
- Lawlor DA, Shaw M. Too much too young? Teenage pregnancy is not a public health problem. Int J Epidemiol 2002;31:552-3. http://dx.doi.org/10.1093/ije/31.3.552.
- Hunter DJ. The changing roles of health care personnel in health and health care management. Soc Sci Med 1996;43:799-808. http://dx.doi.org/10.1016/0277-9536(96)00125-6.
- Hunter D. Relationship between evidence and policy: a case of evidence-based policy or policy-based evidence?. Public Health 2009;123:583-6. http://dx.doi.org/10.1016/j.puhe.2009.07.011.
- Hunter DJ, McKee L, Ferlie E, Hyde P. Organizing and Reorganizing: Power and Change in Health Care Organizations. Basingstoke: Palgrave Macmillan; 2009.
- Best A, Hiatt RA, Norman CD. Knowledge integration: conceptualizing communications in cancer control systems. Patient Educ Couns 2008;71:319-27. http://dx.doi.org/10.1016/j.pec.2008.02.013 (accessed May 2008).
- Øvretveit J. What are the Advantages and Limitations of Different Quality and Safety Tools for Health Care?. Copenhagen: WHO Regional Office for Europe; 2005.
- Bartunek J, Trullen J, Bonet E, Sauquet A. Sharing and expanding academic and practitioner knowledge in health care. J Health Serv Res Policy 2003;8:62-8. http://dx.doi.org/10.1258/135581903322405199.
- Blevins D, Farmer MS, Edlund C, Sullivan G, Kirchner JE. Collaborative research between clinicians and researchers: a multiple case study of implementation. Implement Sci 2010;5. http://dx.doi.org/10.1186/1748-5908-5-76.
- Golden-Biddle K, Reay T, Petz S, Witt C, Casebeer A, Pablo A, et al. Toward a communicative perspective of collaborating in research: the case of the researcher–decision-maker partnership. J Health Serv Res Policy 2003;8:20-5. http://dx.doi.org/10.1258/135581903322405135.
- Øvretveit J. Using Evidence to Bring About Quality Improvment n.d.
- Goldenberg MJ. On evidence and evidence-based medicine: lessons from the philosophy of science. Soc Sci Med 2006;62:2621-32. http://dx.doi.org/10.1016/j.socscimed.2005.11.031.
- Dopson S, Locock L, Gabbay J, Ferlie E, Fitzgerald L. Evidence-based medicine and the implementation gap. Health 2003;7:311-30. http://dx.doi.org/10.1177/1363459303007003004.
- Canadian Institute for Health Research . Knowledge Translation Strategy 2004–2009 2004. www.cihr-irsc.gc.ca/e/26574.html (accessed January 2014).
- Davies H, Nutley S, Walter I. Why ‘knowledge transfer’ is misconceived for applied social research. J Health Serv Res Policy 2008;13:188-90. http://dx.doi.org/10.1258/jhsrp.2008.008055.
- Kerner J, Rimer B, Emmons K. Dissemination research and research dissemination: how can we close the gap?. Health Psychol 2005;24:443-6. http://dx.doi.org/10.1037/0278-6133.24.5.443.
- Tetroe JM, Graham ID, Foy R, Robinson N, Eccles MP, Wensing M, et al. Health research funding agencies’ support and promotion of knowledge translation: an international study. Milbank Q 2008;86:125-55. http://dx.doi.org/10.1111/j.1468-0009.2007.00515.x.
- Youngblut JM, Brooten D. Evidence-based nursing practice: why is it important?. AACN Clinical Issues 2001;12:468-76. http://dx.doi.org/10.1097/00044067-200111000-00003.
- Dopson S, FitzGerald L, Ferlie E, Gabbay J, Locock L. No magic targets! Changing clinical practice to become more evidence based. Health Care Manage Rev 2010;35:2-12. http://dx.doi.org/10.1097/HMR.0b013e3181c88e79.
- Lambert H. Accounting for EBM: notions of evidence in medicine. Soc Sci Med 2006;62:2633-45. http://dx.doi.org/10.1016/j.socscimed.2005.11.023.
- Petticrew M, Roberts H. Evidence, hierarchies, and typologies: horses for courses. J Epidemiol Community Health 2003;57:527-9. http://dx.doi.org/10.1136/jech.57.7.527.
- Sheldon TA. Making evidence synthesis more useful for management and policy-making. J Health Serv Res Policy 2005;10:1-5. http://dx.doi.org/10.1258/1355819054308521.
- Dobrow MJ, Goel V, Upshur REG. Evidence-based health policy: context and utilisation. Soc Sci Med 2004;58:207-17. http://dx.doi.org/10.1016/S0277-9536(03)00166-7.
- Tang KC, Ehsani JP, McQueen DV. Evidence based health promotion: recollections, reflections, and reconsiderations. J Epidemiol Community Health 2003;57:841-3. http://dx.doi.org/10.1136/jech.57.11.841.
- Sanderson I. Evidence-based policy or policy-based evidence? Reflections on Scottish experience. Evidence Policy 2011;7:59-76. http://dx.doi.org/10.1332/174426411X553007.
- Armstrong R, Doyle J, Lamb C, Waters E. Multi-sectoral health promotion and public health: the role of evidence. J Public Health 2006;28:168-72. http://dx.doi.org/10.1093/pubmed/fdl013.
- Contandriopoulos D, Lemire M, Denis JL, Tremblay É. Knowledge exchange processes in organizations and policy arenas: a narrative systematic review of the literature. Milbank Q 2010;88:444-83. http://dx.doi.org/10.1111/j.1468-0009.2010.00608.x.
- Kothari A, Rudman D, Dobbins M, Rouse M, Sibbald S, Edwards N. The use of tacit and explicit knowledge in public health: a qualitative study. Implement Sci 2012;7. http://dx.doi.org/10.1186/1748-5908-7-20.
- Denis JL, Lomas J. Convergent evolution: the academic and policy roots of collaborative research. J Health Serv Res Policy 2003;8:1-6. http://dx.doi.org/10.1258/135581903322405108.
- Dwairy M, Dowell AC, Stahl J-C. The application of foraging theory to the information searching behaviour of general practitioners. BMC Fam Pract 2011;12. http://dx.doi.org/10.1186/1471-2296-12-90.
- Rushmer R, Davies HTO. Unlearning in health care. Qual Saf Health Care 2004;13:ii10-15. http://dx.doi.org/10.1136/qshc.2003.009506.
- Rushmer R, Kelly D, Lough M, Wilkinson JE, Davies HT. Introducing the learning practice–I. The characteristics of learning organizations in primary care. J Eval Clin Pract 2004;10:375-86. http://dx.doi.org/10.1111/j.1365-2753.2004.00464.x.
- Boaz A, Ashby D. ESRC UK Centre for Evidence Based Policy and Practice. London: ESRC UK Centre for Evidence Based Policy and Practice, Queen Mary University of London; 2003.
- Grey JM. Knowledge-based commissioning. Otolaryngol Head Neck Surg 2007;137:78-9. http://dx.doi.org/10.1016/j.otohns.2007.06.739.
- Armstrong R, Waters E, Roberts H, Oliver S, Popay J. The role and theoretical evolution of knowledge translation and exchange in public health. J Public Health 2006;28:384-9. http://dx.doi.org/10.1093/pubmed/fdl072.
- Dobbins M, Rosenbaum P, Plews N, Law M, Fysh A. Information transfer: what do decision makers want and need from researchers. Implement Sci 2007;2. http://dx.doi.org/10.1186/1748-5908-2-20.
- Ward V, House A, Hamer S. Developing a framework for transferring knowledge into action: a thematic analysis of the literature. J Health Serv Res Policy 2009;14:156-64. http://dx.doi.org/10.1258/jhsrp.2009.008120.
- Garfinkel H. Studies in Ethnomethodology. Englewood Cliffs, NJ: Prentice Hall; 1967.
- Austin J. How to Do Things with Words. London: Oxford University Press; 1962.
- Wenger E. Communities of Practice: Learning, Meaning, and Identity. Cambridge: Cambridge University Press; 1998.
- Caplan N. The two-communities theory and knowledge utilization. Am Behav Sci 1979;22:459-70. http://dx.doi.org/10.1177/000276427902200308.
- Taylor-Robinson DC, Milton B, Lloyd-Williams F, O’Flaherty M, Capewell S. Planning ahead in public health? A qualitative study of the time horizons used in public health decision-making. BMC Public Health 2008;8. http://dx.doi.org/10.1186/1471-2458-8-415.
- McCafferty S, Williams I, Hunter D, Robinson S, Donaldson C, Bate A. Implementing world class commissioning competencies. J Health Serv Res Policy 2012;17:40-8. http://dx.doi.org/10.1258/jhsrp.2011.011104.
- Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q 2004;82:581-629. http://dx.doi.org/10.1111/j.0887-378X.2004.00325.x.
- Locock L, Boaz A. Research, policy and practice – worlds apart?. Social Policy Society 2004;3:375-84. http://dx.doi.org/10.1017/S1474746404002003.
- Innvær S, Vist G, Trommald M, Oxman A. Health policy-makers’ perceptions of their use of evidence: a systematic review. J Health Serv Res Policy 2002;7:239-44. http://dx.doi.org/10.1258/135581902320432778.
- Rushmer R, Steven A, Hunter D. ‘Hearing What Other People Are Doing Is Always Interesting. . .’ From Research to Reality: A Realist Evaluation of a Knowledge to Action Initiative 2011. http://tees.openrepository.com/tees/bitstream/10149/325967/2/325967.pdf (accessed January 2015).
- Kunz W, Rittel HWJ. Information science: on the structure of its problems. Inform Storage Ret 1970;8:95-8. http://dx.doi.org/10.1016/0020-0271(72)90011-3.
- Rittel HWJ, Webber MM. Dilemmas in a general theory of planning. Policy Sci 1973;4:155-69. http://dx.doi.org/10.1007/BF01405730.
- Weber ER, Khademian AM. Wicked problems, knowledge challenges, and collaborative capacity builders in network settings. Public Admin Rev 2008;68:334-49. http://dx.doi.org/10.1111/j.1540-6210.2007.00866.x.
- Kothari A, MacLean L, Edwards N. Increasing capacity for knowledge translation: understanding how some researchers engage policy makers. Evidence Policy 2009;5:33-51. http://dx.doi.org/10.1332/174426409X395402.
- Gambetta D. Trust: Making and Breaking Cooperative Relations. Oxford: Basil Blackwell; 1988.
- Mitton C, Adair CE, McKenzie E, Patten SB, Perry BW. Knowledge transfer and exchange: review and synthesis of the literature. Milbank Q 2007;85:729-68. http://dx.doi.org/10.1111/j.1468-0009.2007.00506.x.
- Plsek PE, Greenhalgh T. Complexity science: the challenge of complexity in health care. BMJ 2001;323. http://dx.doi.org/10.1136/bmj.323.7313.625.
- Kothari A, Birch S, Charles C. ‘Interaction’ and research utilisation in health policies and programs: does it work?. Health Policy 2005;71:117-25. http://dx.doi.org/10.1016/j.healthpol.2004.03.010.
- McWilliam CL, Kothari A, Ward-Griffin C, Forbes D, Leipert B. South West Community Care Access Centre Home Care Collaboration (SW-CCAC) . Evolving the theory and praxis of knowledge translation through social interaction: a social phenomenological study. Implement Sci 2009;4. http://dx.doi.org/10.1186/1748-5908-4-26.
- Denis J-L, Lehoux P, Hivon M, Champagne F. Creating a new articulation between research and practice through policy? The views and experiences of researchers and practitioners. J Health Serv Res Policy 2003;8:44-50. http://dx.doi.org/10.1258/135581903322405162.
- Van de Ven A. The Innovation Journey. New York, NY: Oxford University Press; 1999.
- Van de Ven AH, Johnson PE. Knowledge for theory and practice. Acad Manage Rev 2006;31:802-21. http://dx.doi.org/10.5465/AMR.2006.22527385.
- Van de Ven AH. Engaged Scholarship: A Guide for Organizational and Social Research. Oxford: Oxford University Press; 2007.
- Ross S, Lavis J, Rodriguez C, Woodside J, Denis J-L. Partnership experiences: involving decision-makers in the research process. J Health Serv Res Policy 2003;8:26-34. http://dx.doi.org/10.1258/135581903322405144.
- Newman L, Biedrzycki K, Patterson J, Baum F. Partnership in knowledge creation: lessons learned from a researcherpolicy actor partnership to co-produce a rapid appraisal case study of South Australia’s Social Inclusion Initiative. Evid Policy 2011;7:77-96. http://dx.doi.org/10.1332/174426411X553025.
- Hunter DJ. Public Health Policy. London: Polity Press; 2003.
- Wimbush E. Strengthening research capacity in health promotion practice settings. Health Educ 1999;99:169-76. http://dx.doi.org/10.1108/09654289910284616.
- Cohen WM, Levinthal DA. Absorptive capacity: a new perspective on learning and innovation. Admin Sci Q 1990;35:128-52.
- Lee A, Head F, Packham C, Robinson M. The challenge of commissioning for populations. Br J Gen Pract 2012;62:e387-9. http://dx.doi.org/10.3399/bjgp12X641618.
- Dobbins M, Jack S, Thomas H, Kothari A. Public health decision-makers’ informational needs and preferences for receiving research evidence. Worldv Evid-Based Nurs 2007;4:156-63. http://dx.doi.org/10.1111/j.1741-6787.2007.00089.x.
- Ginsburg LR, Lewis SJ, Zackheim L, Casebeer A. Revisiting interaction in knowledge translation. Implement Sci 2007;2. http://dx.doi.org/10.1186/1748-5908-2-34.
- Alcohol Attributes Fractions for England, Alcohol Attributes Mortality And Hosptial Admissions. Liverpool: Liverpool John Moores University; 2008.
- Leon DA, McCambridge J. Liver cirrhosis mortality rates in Britain from 1950 to 2002: an analysis of routine data. Lancet 2006;367:52-6. http://dx.doi.org/10.1016/S0140-6736(06)67924-5.
- Calling Time: The Nation’s Drinking as a Major Health Issue. London; 2004.
- Changing Scotland’s Relationship With Alcohol: A Discussion Paper On Our Strategic Approach. Edinburgh: Scottish Government; 2008.
- Shipton D, Whyte B, Walsh D. Alcohol-related mortality in deprived UK cities: worrying trends in young women challenge recent national downward trends. J Epidemiol Community Health 2013;67:805-12. http://dx.doi.org/10.1136/jech-2013-202574.
- Wilson GB, McGovern R, Antony G, Cassidy P, Deverill M, Graybill E, et al. Brief intervention to reduce risky drinking in pregnancy: study protocol for a randomized controlled trial. Trials 2012;13. http://dx.doi.org/10.1186/1745-6215-13-174.
- Government Response to the House of Commons Science and Technology Committee Report of Session 2010–2012: Alcohol Guidelines. London: The Stationery Office; 2012.
- Antenatal Routine Care For The Pregnant Woman. London: NICE; 2008.
- Anderson P, Chisholm D, Fuhr DC. Effectiveness and cost-effectiveness of policies and programmes to reduce the harm caused by alcohol. Lancet 2009;373:2234-46. http://dx.doi.org/10.1016/S0140-6736(09)60744-3.
- Signs for Improvement – Commissioning Interventions to Reduce Alcohol-Related Harm. London: Department of Health; 2009.
- Britton A, McPherson K. Mortality in England and Wales attributable to current alcohol consumption. J Epidemiol Community Health 2008;55:383-8. http://dx.doi.org/10.1136/jech.55.6.383.
- Drummond C, Oyefeso A, Phillips T, Cheeta S, Deluca P, Perryman K, et al. Alcohol Needs Assessment Research Project (ANARP): The 2004 National Alcohol Needs Assessment for England 2004.
- Kreitman N. Alcohol consumption and the preventive paradox. Br J Addict 1986;81:353-63. http://dx.doi.org/10.1111/j.1360-0443.1986.tb00342.x.
- Poikolainen K, Paljärvi T, Mäkelä P. Alcohol and the preventive paradox: serious harms and drinking patterns. Addiction 2007;102:571-8. http://dx.doi.org/10.1111/j.1360-0443.2006.01740.x.
- Chisholm D, Rehm J, Van Ommeren M, Monteiro M. Reducing the global burden of hazardous alcohol use: a comparative cost-effectiveness analysis. J Stud Alcohol Drugs 2004;65. http://dx.doi.org/10.15288/jsa.2004.65.782.
- Warburton A, Shepherd JP. Tackling alcohol related violence in city centres: effect of emergency medicine and police intervention. Emerg Med J 2006;23:12-7. http://dx.doi.org/10.1136/emj.2004.023028.
- Kaner EF, Dickinson HO, Beyer F, Pienaar E, Schlesinger C, Campbell F, et al. The effectiveness of brief alcohol interventions in primary care settings: a systematic review. Drug Alcohol Rev 2009;28:301-23. http://dx.doi.org/10.1111/j.1465-3362.2009.00071.x.
- Antenatal Routine Care for the Pregnant Woman. London: NICE; 2008.
- Anderson P. A safe, sensible and social AHRSE: New Labour and alcohol policy. Addiction 2007;102:1515-21. http://dx.doi.org/10.1111/j.1360-0443.2007.02000.x.
- Licensing Act 2003. London: The Stationery Office; 2003.
- Nicholson Committee Review of Liquor Licensing Law in Scotland. Edinburgh: Scottish Executive; 2003.
- Report of the Departmental Committee on Scottish Licensing Law. Edinburgh: HMSO; 1973.
- Licensing (Scotland) Act 1976. London: HMSO; 1976.
- Ritson B. Alcohol licensing laws: proposals for changes in Scottish law. Alcohol Alcohol 2004;39:2-7. http://dx.doi.org/10.1093/alcalc/agh006.
- The Relationship between Off-Sales and Problem Drinking in Scotland. Edinburgh: Scottish Executive; 2007.
- Graham L, MacKinnon D. Grasping the thistle: the role of alcohol brief interventions in Scottish alcohol policy. Drug Alcohol Rev 2010;29:603-7. http://dx.doi.org/10.1111/j.1465-3362.2010.00245.x.
- Changing Scotland’s Relationship with Alcohol: A Framework for Action. Edinburgh: Scottish Government; 2009.
- Purshouse R, Meng Y, Rafia R, Brennan A, Meier P. Model-Based Appraisal of Alcohol Minimum Pricing and Off-Licensed Trade Discount Bans in Scotland: A Scottish Adaptation of the Sheffield Alcohol Policy Model Version 2. Sheffield: ScHARR, University of Sheffield; 2009.
- Gordon R, Anderson P. Science and alcohol policy: a case study of the EU Strategy on Alcohol. Addiction 2011;106:55-66. http://dx.doi.org/10.1111/j.1360-0443.2010.03324.x.
- Anderson P, Gual A. Reflections on science and the governance of alcohol policy. Addiction 2011;106:67-70. http://dx.doi.org/10.1111/j.1360-0443.2010.03350.x.
- General Lifestyle Survey: A Report on the 2010 General Lifestyle Survey. Newport: Office for National Statistics; 2012.
- Bellis M, Hughes K, Cook P, Morleo M. Off Measure: How We Underestimate the Amount We Drink. London: Alcohol Concern; 2009.
- General Lifestyle Survey Overview: A Report on the 2011 General Lifestyle Survey. Newport: Office for National Statistics; 2013.
- Bollin K, Grant C, Hamlyn B, Thornton A. Infant Feeding Survey 2005. London: Department of Health, Health and Social Care Information Centre; 2007.
- Nilsen P. Brief alcohol intervention to prevent drinking during pregnancy: an overview of research findings. Curr Opin Obstet Gyn 2009;21:496-500. http://dx.doi.org/10.1097/GCO.0b013e328332a74c.
- Popova S, Stade B, Bekmuradov D, Lange S, Rehm J. What do we know about the economic impact of fetal alcohol spectrum disorder? A systematic literature review. Alcohol Alcohol 2011;46:490-7. http://dx.doi.org/10.1093/alcalc/agr029.
- Blackburn C, Whitehurst T. Foetal alcohol spectrum disorders (FASD): raising awareness in early years settings. Br J Special Educ 2010;37:122-9. http://dx.doi.org/10.1111/j.1467-8578.2010.00471.x.
- Fetal Alcohol Spectrum Disorders – A Guide for Healthcare Professionals. London: BMA; 2007.
- Chang G, Wilkins-Haug L, Berman S, Goetz MA. Brief intervention for alcohol use in pregnancy: a randomized trial. Addiction 1999;94:1499-508. http://dx.doi.org/10.1046/j.1360-0443.1999.941014996.x.
- Skagerström J, Alehagen S, Häggström-Nordin E, Årestedt K, Nilsen P. Prevalence of alcohol use before and during pregnancy and predictors of drinking during pregnancy: a cross sectional study in Sweden. BMC Public Health 2013;13:1-10. http://dx.doi.org/10.1186/1471-2458-13-780.
- Shúilleabháin AN, Barry J, Kelly A, O’Kelly F, Darker C, O’Dowd T. Alcohol consumption in pregnancy: results from the general practice setting. Ir J Med Sci 2013:1-10. http://dx.doi.org/10.1007/s11845-013-0996-9.
- Taylor J. Pregnant? It doesn’t Mean You Have to Stop Drinking 2012.
- Grey R, Henderson J. Report to the Department of Health: Review of the Fetal Effects of Prenatal Alcohol Exposure. Oxford: National Perinatal Epidemiology Unit, University of Oxford; 2006.
- Low Dose Alcohol Exposure During Pregnancy – Does it Harm? A Systematic Literature Review. Stockholm: Swedish National Institute of Public Health; 2009.
- Andersen A-MN, Andersen PK, Olsen J, Grønbæk M, Strandberg-Larsen K. Moderate alcohol intake during pregnancy and risk of fetal death. Int J Epidemiol 2012;41:405-13. http://dx.doi.org/10.1093/ije/dyr189.
- Kelly Y, Sacker A, Grey R, Kelly J, Wolke D, Quigley MA. Light drinking in pregnancy, a risk for behavioural problems and cognitive deficits at 3 years of age?. Int J Epidemiol 2009;38:129-40. http://dx.doi.org/10.1093/ije/dyn230.
- Robinson M, Oddy W, McLean N, Jacoby P, Pennell C, De Klerk N, et al. Low–moderate prenatal alcohol exposure and risk to child behavioural development: a prospective cohort study. BJOG 2010;117:1139-52. http://dx.doi.org/10.1111/j.1471-0528.2010.02596.x.
- Kelly YJ, Sacker A, Grey R, Kelly J, Wolke D, Head J, et al. Light drinking during pregnancy: still no increased risk for socioemotional difficulties or cognitive deficits at 5 years of age?. J Epidemiol Community Health 2012;66:41-8. http://dx.doi.org/10.1136/jech.2009.103002.
- Sayal K. Commentary: Light drinking in pregnancy: can a glass or two hurt?. Int J Epidemiol 2009;38:140-2. http://dx.doi.org/10.1093/ije/dyn370.
- Testa M, Quigley BM, Eiden RD. The effects of prenatal alcohol exposure on infant mental development: a meta-analytical review. Alcohol Alcohol 2003;38:295-304. http://dx.doi.org/10.1093/alcalc/agg087.
- Kesmodel US, Bertrand J, Støvring H, Skarpness B, Denny CH, Mortensen EL. The effect of different alcohol drinking patterns in early to mid pregnancy on the child’s intelligence, attention, and executive function. BJOG 2012;119:1180-90. http://dx.doi.org/10.1111/j.1471-0528.2012.03393.x.
- Lewis SJ, Zuccolo L, Smith GD, Macleod J, Rodriguez S, Draper ES, et al. Fetal alcohol exposure and IQ at age 8: evidence from a population-based birth-cohort study. PLOS ONE 2012;7. http://dx.doi.org/10.1371/journal.pone.0049407.
- Hepper PG, Dornan JC, Little JF. Maternal alcohol consumption during pregnancy may delay the development of spontaneous fetal startle behaviour. Physiol Behav 2005;83:711-4. http://dx.doi.org/10.1016/j.physbeh.2004.09.004.
- O’Leary C, Bower C. Guidelines for pregnancy: what’s an acceptable risk, and how is the evidence (finally) shaping up?. Drug Alcohol Rev 2012;31:170-83. http://dx.doi.org/10.1111/j.1465-3362.2011.00331.x.
- Robinson J, Laverty L, Holdsworth C. Talking About Alcohol. Liverpool: Liverpool Primary Care Trust and Liverpool Health Inequalities Research Institute, Liverpool University; 2011.
- Patra J, Bakker R, Irving H, Jaddoe VW, Malini S, Rehm J. Dose–response relationship between alcohol consumption before and during pregnancy and the risks of low birthweight, preterm birth and small for gestational age (SGA) – a systematic review and meta-analyses. BJOG 2011;118:1411-21. http://dx.doi.org/10.1111/j.1471-0528.2011.03050.x.
- Stade BC, Bailey C, Dzendoletas D, Sgro M. Psychological and/or educational interventions for reducing prenatal alcohol consumption in pregnant women and women planning pregnancy. Cochrane Database Syst Rev 2009;2. http://dx.doi.org/10.1002/14651858.CD004228.pub2.
- Foetal Alcohol Spectrum Disorder (FASD): Information for Pregnant Women. London: NOFAS; 2011.
- Morleo M, Woolfall K, Dedman D, Mukherjee R, Bellis M, Cook P. Under-reporting of foetal alcohol spectrum disorders: an analysis of hospital episode statistics. BMC Pediatr 2011;11. http://dx.doi.org/10.1186/1471-2431-11-14.
- Sarkar M, Einarson T, Koren G. Comparing the effectiveness of TWEAK and T-ACE in determining problem drinkers in pregnancy. Alcohol Alcohol 2010;45:356-60. http://dx.doi.org/10.1093/alcalc/agq022.
- Burns E, Grey R, Smith LA. Brief screening questionnaires to identify problem drinking during pregnancy: a systematic review. Addiction 2010;105:601-14. http://dx.doi.org/10.1111/j.1360-0443.2009.02842.x.
- Seib CA, Daglish M, Heath R, Booker C, Reid C, Fraser J. Screening for alcohol and drug use in pregnancy. Midwifery 2012;28:760-4. http://dx.doi.org/10.1016/j.midw.2011.08.003.
- van der Wulp NY, Hoving C, de Vries H. A qualitative investigation of alcohol use advice during pregnancy: experiences of Dutch midwives, pregnant women and their partners. Midwifery 2013;29:e89-98. http://dx.doi.org/10.1016/j.midw.2012.11.014.
- Phillips D, Thomas K, Cox H, Ricciardelli LA, Ogle J, Love V, et al. Factors that influence women’s disclosures of substance use during pregnancy: a qualitative study of ten midwives and ten pregnant women. J Drug Issues 2007;37:357-75. http://dx.doi.org/10.1177/002204260703700207.
- Economidoy E, Klimi A, Vivilaki V. Caring for substance abuse pregnant women: the role of the midwife. Health Sci J 2012;6:161-9.
- Alcohol Strategy. London: Home Office; 2012.
- Skagerstróm J, Chang G, Nilsen P. Predictors of drinking during pregnancy: a systematic review. J Womens Health 2011;20:901-13. http://dx.doi.org/10.1089/jwh.2010.2216.
- Units and You. London: Department of Health; 2008.
- McAuley A. Alcohol Consumption Among Pregnant Women and Brief Interventions in the Antenatal Setting. Edinburgh: NHS Health Scotland; 2009.
- Royal College of Midwives Guidance Paper on Alcohol and Pregnancy. London: RCM; 2010.
- Coveney J. Food, Morals and Meaning: The Pleasure and Anxiety of Eating. Abingdon: Routledge; 2000.
- Rozin P. The meaning of food in our lives: a cross-cultural perspective on eating and well-being. J Nutr Educ Behav 2005;37:107-12. http://dx.doi.org/10.1016/S1499-4046(06)60209-1.
- Lupton D. ‘Precious cargo’: foetal subjects, risk and reproductive citizenship. Crit Public Health 2012;22:329-40. http://dx.doi.org/10.1080/09581596.2012.657612.
- Bell K, McNaughton D, Salmon A. Medicine, morality and mothering: public health discourses on foetal alcohol exposure, smoking around children and childhood overnutrition. Crit Public Health 2009;19:155-70. http://dx.doi.org/10.1080/09581590802385664.
- Cohen S. Folks Devils and Moral Panics: The Creation of the Mods and Rockers. Abingdon: Routledge; 1972.
- Salmon A. Aboriginal mothering, FASD prevention and the contestations of neoliberal citizenship. Crit Public Health 2011;21:165-78. http://dx.doi.org/10.1080/09581596.2010.530643.
- Lowe PK, Lee EJ. Advocating alcohol abstinence to pregnant women: some observations about British policy. Health Risk Soc 2010;12:301-11. http://dx.doi.org/10.1080/13698571003789690.
- Lupton DA. ‘The best thing for the baby’: mothers’ concepts and experiences related to promoting their infants’ health and development. Health Risk Soc 2011;13. http://dx.doi.org/10.1080/13698575.2011.624179.
- Pawson R, Tilley N. Realistic Evaluation. London: Sage; 1997.
- Compact Oxford English Dictionary. Oxford: Oxford University Press; 2005.
- Oxford Dictionaries n.d. www.oxforddictionaries.com/definition/english/evidence.
- Health and Social Care Act 2012. London: The Stationery Office; 2012.
- Potvin L, McQueen D. Health Promotion Evaluation Practices in the Americas: Values and Research. Montreal, QC: Springer; 2009.
- Yin RK. Case Study Research: Design and Methods. London: Sage; 2008.
- Crowe S, Cresswell K, Robertson A, Huby G, Avery A, Sheikh A. The case study approach. BMC Med Res Methodol 2011;11. http://dx.doi.org/10.1186/1471-2288-11-100.
- Schwab DP, Heneman HG, DeCotiis TA. Behaviorally anchored rating scales: a review of the literature. Personnel Psychology 1975;28:549-62. http://dx.doi.org/10.1111/j.1744-6570.1975.tb01392.x.
- Glaser B, Strauss A. The Discovery of Grounded Theory: Strategies for Qualitative Research. Chicago, IL: Aldine; 1967.
- The Scottish Government . Scottish Health Survey 2012 – Volume 1 Main Report 2013. www.gov.scot/Publications/2013/09/3684 (accessed June 2015).
- Dalkey NC. The Delphi Method: An Experimental Study Of Group Opinion. Santa Monica, CA: The Rand Corporation; 1969.
- Campbell JP, Dunnette MD, Arvey RD, Hellervik LV. The development and evaluation of behaviorally based rating scales. J Applied Psychol 1973;57. http://dx.doi.org/10.1037/h0034185.
- Grussing PG, Valuck RJ, Williams RG. Development and validation of behaviorally-anchored rating scales for student evaluation of pharmacy instruction. Am J Pharm Educ 1994;58:25-37.
- Smith PC, Kendall LM. Retranslation of expectations: an approach to the construction of unambiguous anchors for rating scales. J Applied Psychol 1963;47:149-55. http://dx.doi.org/10.1037/h0047060.
- Zedeck S, Imparato N, Krausz M, Oleno T. Development of behaviorally anchored rating scales as a function of organizational level. J Applied Psychol 1974;59:249-52. http://dx.doi.org/10.1037/h0036521.
- Kelly DR, Lough M, Rushmer R, Wilkinson JE, Greig G, Davies HT. Delivering feedback on learning organization characteristics – using a learning practice inventory. J Eval Clin Pract 2007;13:734-40. http://dx.doi.org/10.1111/j.1365-2753.2006.00746.x.
- Rushmer RK, Kelly D, Lough M, Wilkinson JE, Greig GJ, Davies HT. The learning practice inventory: diagnosing and developing learning practices in the UK. J Eval Clin Pract 2007;13:206-11. http://dx.doi.org/10.1111/j.1365-2753.2006.00673.x.
- Rushmer R, Kelly D, Lough M, Wilkinson JE, Davies HT. Introducing the learning practice–II. Becoming a learning practice. J Eval Clin Pract 2004;10:387-98. http://dx.doi.org/10.1111/j.1365-2753.2004.00465.x.
- Guthrie BP, Davies H, Greig G, Rushmer R, Walter I, Duguid A, et al. Delivering Health Care through Managed Clinical Networks (MCNs): Lessons from the North 2010. www.nets.nihr.ac.uk/projects/hsdr/081518103 (accessed November 2014).
- Governance Arrangements for Research Ethics Committees (GAfREC). London: NHS Health Research Authority; 2011.
- Ritchie J, Spencer L, Bryman A, Burgess RG. Analyzing Qualitative Data. Abingdon: Routledge; 1994.
- Geertz C. The Interpretation Of Cultures: Selected Essays. New York, NY: Basic Books; 1973.
- Ellaway A, Macdonald L, Forsyth AJM, Macintyre S. The socio-spatial distribution of alcohol outlets in Glasgow city. Health Place 2010;16:167-72.
- Health First: An Evidence-Based Alcohol Strategy For The UK. Stirling: University of Stirling; 2013.
- MacGregor A, Sharp C, Mabelis J, Corbett J. An Evaluation of the Implementation of, and Compliance with, the Objectives of the Licensing (Scotland) Act 2005: Final Report. Edinburgh: Scottish Centre for Social Research; 2013.
- Alcohol Statistics Scotland 2011. Edinburgh: Scottish Government; 2011.
- McNaughton P, Gillan E. Rethinking Alcohol Licensing. Glasgow: Alcohol Focus Scotland; 2011.
- Winpenny E, Beer G, Manning J. From One Too Many – The Risks of Frequent Excessive Drinking. London: 2020health.org; 2011.
- Popova S, Giesbrecht N, Bekmuradov D, Patra J. Epidemiology and policy. Hours and days of sale and density of alcohol outlets: impacts on alcohol consumption and damage: a systematic review. Alcohol Alcohol 2009;44:500-16. http://dx.doi.org/10.1093/alcalc/agp054.
- Hawkins B, Holden C, McCambridge J. Alcohol industry influence on UK alcohol policy: a new research agenda for public health. Crit Public Health 2012;22:297-305. http://dx.doi.org/10.1080/09581596.2012.658027.
- The Licensing (Scotland) Act (2005). London: The Stationery Office; 2005.
- The Criminal Justice and Licensing Act (2010). London: The Stationery Office; 2010.
- The Alcohol etc. (Scotland Act) (2010). London: The Stationery Office; 2010.
- Policy Background to Community Health Partnerships and Community Health and Care Partnerships. Glasgow Centre for Population Health; 2007.
- The Scottish Goverment . A New Framework for Local Partnerships on Alcohol 2009. www.gov.scot/Publications/2009/04/23084201/4 (accessed June 2015).
- The Road To Recovery: A New Approach To Tackling Scotland’s Drug Problem. Edinburgh: Scottish Government; 2008.
- Alcohol (Minimum Pricing) (Scotland) Act 2012. London: The Stationery Office; 2012.
- NHS England . Commissioning Support: Key Facts 2013. www.england.nhs.uk/wp-content/uploads/2013/03/csu-facts.pdf (accessed January 2014).
- The New Public Health System: Summary. London: Department of Health; 2011.
- Operating Principles for Health and Wellbeing Boards: Laying the Foundations for Healthier Places. London: Department of Health; 2011.
- Code Of Conduct: Evidence Matters. London: Market Research Society; 2010.
- NHS Outcomes Framework 2013 to 2014. London: Department of Health; 2012.
- Improving Outcomes and Supporting Transparency: A Public Health Outcome Framework for England, 2013–2016. London: Department of Health; 2012.
- Healthy Foundations Lifestages Segmentation . Research Report No. 1: Creating the Segmentation Using a Quantitative Survey of the General Population of England n.d. www.thensmc.com/sites/default/files/301846_HFLS%20Report%20No1_ACC.pdf (accessed November 2014).
- Hunter DJ. The changing roles of health care personnel in health and health care management. Soc Sci Med 1996;43:799-808. http://dx.doi.org/10.1016/0277-9536(96)00125-6.
- Rushmer R, Hunter DJ, Steven A. Using interactive workshops to prompt knowledge exchange: a realist evaluation of a knowledge to action initiative. Public Health 2014;128:52-6. http://dx.doi.org/10.1016/j.puhe.2014.03.012.
- Hunter DJ, Perkins N. Partnership Working in Public Health. Bristol: Bristol Policy Press; 2014.
- Walter I, Davies H, Nutley S. Increasing research impact through partnerships: evidence from outside health care. J Health Serv Res Policy 2003;8:58-61. http://dx.doi.org/10.1258/135581903322405180.
- Oliver K, de Vocht F, Money A, Everett M. Who runs public health? A mixed-methods study combining qualitative and network analyses. J Public Health (Oxf) 2013;35:453-9. http://dx.doi.org/10.1093/pubmed/fdt039.
- Goffman E. The Presentation of Self in Everyday Life. Garden City, NY: Double Day; 1959.
- Vickers G. Value Systems and Social Process. London: Tavistock Publications; 1968.
- Ward V, Smith S, House A, Hamer S. Exploring knowledge exchange: a useful framework for practice and policy. Soc Sci Med 2012;74:297-304. http://dx.doi.org/10.1016/j.socscimed.2011.09.021.
- World Health Organisation . Public Health n.d. www.who.int/trade/glossary/story076/en/ (accessed 5 December 2013).
- Derrida J. Of Grammatology. Balitmore, MD: Johns Hopkins University Press; 1997.
- Ham C. World class commissioning: a health policy chimera?. J Health Serv Res Policy 2008;13:116-21. http://dx.doi.org/10.1258/jhsrp.2008.007177.
- Hoggart R, Goffman E. Gender Advertisements. London: Macmillan; 1979.
- Chappel D, Miller P, Parkin D, Thomson R. Models of commissioning health services in the British National Health Service: a literature review. J Public Health 1999;21:221-7. http://dx.doi.org/10.1093/pubmed/21.2.221.
- Scottish Index of Multiple Deprivation . Scottish Index of Multiple Deprivation 2012. A National Statistics Publication for Scotland, 18 December 2012 n.d. http://simd.scotland.gov.uk/publication-2012/ (accessed June 2015).
- Information and Statistics Division (ISD) . Alcohol-Related Hospital Statistics Scotland 2013 14 2014. www.isdscotland.org/Health-Topics/Drugs-and-Alcohol-Misuse/Publications/2014-11-25/2014-11-25-ARHS2013-14-Report.pdf (accessed June 2015).
- Livingston M, Chikritzhs T, Room R. Changing the density of alcohol outlets to reduce alcohol-related problems. Drug Alcohol Rev 2007;26:557-66.
- Stockwell T, Zhao J, Macdonald S, Pakula B, Gruenewald P, Holder H. Changes in per capita alcohol sales during the partial privatization of British Columbia’s retail alcohol monopoly 2003–2008: a multi-level local area analysis. Addiction 2009;104:1827-36.
- Stockwell T, Zhao J, Macdonald S, Vallance K, Gruenewald P, Ponicki W, et al. Impact on alcohol-related mortality of a rapid rise in the density of private liquor outlets in British Columbia: a local area multi-level analysis. Addiction 2011;106:768-76.
- Livingston M. A longitudinal analysis of alcohol outlet density and domestic violence. Addiction 2011;106:919-25.
- Coghill N. One on Every Corner: The Relationships Between Off-Licence Density and Alcohol Harms in Young People. London: Alcohol Concern; 2011.
- Scribner RA, MacKinnon DP, Dwyer JH. Alcohol outlet density and motor vehicle crashes in Los Angeles County cities. J Stud Alcohol 1994;55:447-53.
- Scribner RA, MacKinnon DP, Dwyer JH. The risk of assaultive violence and alcohol availability in Los Angeles County. Am J Public Health 1995;85:335-40.
- Treno AJ, Gruenewald PJ, Johnson FW. Alcohol availability and injury: the role of local outlet densities. Alcoholism: Clin Exper Res 2001;25:1467-71.
- Gruenewald PJ, Johnson FW, Treno AJ. Outlets, drinking and driving: a multilevel analysis of availability. J Stud Alcohol 2002;63:460-8.
- Pollack CE, Cubbin C, Ahn D, Winkleby M. Neighbourhood deprivation and alcohol consumption: does the availability of alcohol play a role?. Int J Epidemiol 2005;34:772-80.
- Hay GC, Whigham PA, Kypri K, Langley JD. Neighbourhood deprivation and access to alcohol outlets: a national study. Health Place 2009;15:1086-93.
- MacGregor S. Barriers to the influence of evidence on policy: are politicians the problem?. Drugs: Educ Prevent Policy 2013;20:225-33.
- Nash K, Sheard E, Rovet J, Koren G. Understanding fetal alcohol spectrum disorders (FASDs): toward identification of a behavioral phenotype. Sci World J 2008;8:873-82.
- GOV.UK . Local Alcohol Profiles for England: 2015 Annual Data Update 2015. www.gov.uk/government/statistics/local-alcohol-profiles-for-england-2015-annual-data-update (accessed June 2015).
- Plant MA, Miller P, Plant ML. Trends in drinking, smoking and illicit drug use among 15- and 16-year-olds in the UK (1995–2003). J Subst Use 2005;10:331-9.
- Morleo M, Cook PA, Bellis MA, Meah S, Threlfall A. Improving Accuracy in Recording Alcohol Consumption: A Survey in Greater Manchester. Liverpool: Centre for Public Health, Liverpool John Moores University; 2011.
Appendix 1 The Word version of the Delphi questionnaire
Appendix 2 Quantitative data for alcohol licensing in Thistletown
Scoping the quantitative data for alcohol licensing in Thistletown
In order to ascertain the extent of alcohol licensing as a public health problem in Thistletown, a scoping exercise was undertaken in order to identify potential/relevant quantitative data sources. This took place between January and August 2013. Table 14 below highlights points of contact and a summary of available data sources.
| Data source | Data domains | Data features |
|---|---|---|
| NHS Scotland | Alcohol-related admissions and deaths by intermediate zone and CHP/sector | Cross-sectional data 2011 and 2012 |
| Health and Well-being Survey | Alcohol consumption levels | |
| The Scottish Public Health Observatory (ScotPho) | Scottish Health Survey (2012)180 | Self-reported answers to annual questionnaire |
| Alcohol Profiles | Epidemiological data comparing local alcohol profiles with Scottish national averages | |
| Information Services Division Scotland | Alcohol licensing statistics by area | Personal and premises licences granted and refused 2010–11 and 2011–12 |
| Literature | Various peer-reviewed academic papers and evaluation reports |
Results of the scoping exercise
The results of the scoping exercise suggest that there is robust and reliable evidence regarding the extent of alcohol licensing and the social, economic and health impact of this at a national and international level. A range of local data exist relating to Thistletown, including alcohol consumption, alcohol-related events and licensing data for Thistletown. However, data regarding outlet density in Thistletown were not available. The research team requested licensing data by area from the Licensing Board but, unfortunately, we were unable to obtain these owing to resource constraints within the council. This could have enabled us to analyse the relationships between licensing behaviour and outlet density for distinct geographical areas of Thistletown and the impact of this on, for example, health-related events. However, in the light of this lack of detailed licensing data, we had to rely on other data sources. These are highlighted in Table 14 and include health-related evidence from the NHS in Thistletown and the licensing statistics provided by Information and Statistics Division of the Scottish Government and the academic evidence base.
Scoping exercise: data analysis
Alcohol consumption in Scotland
Results from the Scottish Health Survey (2012)180 suggest that alcohol consumption levels in Scotland have declined from 2003. Average unit weekly consumption has fallen since 2003. In 2012 men drank an average of 4.6 units less per week than in 2003 (19.8 units in 2003 and 15.2 units in 2012). Average unit consumption for women declined from 9.0 units per week in 2003 to 7.6 units in 2012. Furthermore, one-quarter of men and 18% of women drank at hazardous or harmful levels (more than 21 units per week for men and more than 14 units for women) in 2012. Hazardous or harmful drinking prevalence has also declined since 2003, although levels did not change significantly between 2011 and 2012. One in five (19%) adults in Scotland exhibited signs of a possible alcohol use disorder according to their scores on the AUDIT-C screening tool, with men significantly more likely than women to do so (25% compared with 13% of women). Men in low-income households were more likely than those with higher household incomes to display signs of harmful drinking or possible alcohol dependence based on the AUDIT-C score (11% compared with only 2% in the highest income quintile). When compared with men living in Scotland’s least deprived areas [Scottish Index of Multiple Deprivation (SIMD)228 quintile 5], those living in the 20% most deprived areas (SIMD quintile 1) were significantly more likely to exhibit signs of a potential alcohol use disorder (32% vs. 21%).
Alcohol-related hospital events in Scotland
Alcohol is a significant problem in Scotland in terms of morbidity, mortality and social harm but although alcohol-related events are high, there has been a downward trend in recent years. In 2011–12, there were 38,737 alcohol-related discharges from a general acute hospital in Scotland (a rate of 691 discharges per 100,000 population). Encouragingly, this is a 0.5% decrease in absolute numbers compared with the previous year (2010–11), when there were 38,924 alcohol-related discharges (a rate of 709 discharges per 100,000 population). Furthermore, over the past 5 years, there has been a 14% decrease in rates of alcohol-related discharges from a general acute hospital in Scotland, from 802 discharges per 100,000 population in 2007/8 to 691 discharges per 100,000 population in 2011–12. The largest decrease was in the 45- to 49-year age group and the second-largest decrease in those aged 15–19 years. For people aged 40–44 years the rate increased from 2010–11 to the latest year (also very small increases were seen in these aged 20–24, 25–29 and 30–34 years of age), although since 2007–8 their rates have dropped markedly as well. However, between 2007–8 and 2001–12, the rate of alcohol-related general acute hospital discharges was approximately seven times greater for patients living in the most deprived areas than for those living in the least deprived areas (Information and Statistics Division 2013). 229
Alcohol-related hospital events in Thistletown
The rate of alcohol-related discharges from general acute hospitals in Scotland varied greatly between NHS Boards from 2007–8 to 2011–12. In the latest year (2011–12) the discharge rate in Thistletown was in the top five highest, with 1012 discharges per 100,000 population. Throughout the 5-year time period Thistletown was one of two health boards that recorded the highest alcohol-related discharge rates. However, both recorded an approximately 9% decrease in discharge rates from 2007–8 to 2011–12. In their alcohol profile, the Scottish Public Health Observatory (ScotPho) also suggests that alcohol-related mortality and prevalence of problem drinkers is significantly worse in Thistletown than the national averages for Scotland.
Alcohol consumption in Thistletown
A piece of national research (reference withheld) found, in a survey of residents in 15 relatively deprived communities in Thistletown, that 27% of men and 14% of women respondents reported levels of drinking over the weekly recommended amount. The local NHS Health and Wellbeing Survey in 2011 reported that just under half (45%) of respondents drank alcohol weekly. One in five (20%) respondents had exceeded the recommended weekly limit for alcohol consumption in the previous week. Three in 10 (31%) respondents had been binge drinkers in the previous week.
Relationship between alcohol availability, consumption and harm
Availability of alcohol concerns the ease and convenience with which it can be obtained. The relationship between the physical availability of alcohol in society and levels of alcohol consumption and harm has been explored in a large number of research studies from different countries. MacNaughton & Gillan in their report ‘Re-thinking Alcohol Licensing’3 suggest that there is strong and repeated evidence that shows that increasing access to alcohol, through more outlets and longer trading hours, affects a range of alcohol-related harms and, in certain circumstances, levels and patterns of consumption. They suggest that, overall, the weight of evidence supports a conclusion that restrictions on the availability of alcohol can contribute to a reduction in alcohol-related problems.
Relationship between alcohol outlet density and alcohol-related problems
A substantial number of studies have examined the relationship between outlet density (number and type of outlets) and a variety of alcohol-related problems. (Livingston et al. 2007)230 The clearest evidence of a relationship comes from natural experiments in countries where large-scale policy changes have been implemented over a relatively short time period. Two recent studies from Canada assessed the impact of privatisation of government-run liquor stores on consumption and harm following a 33.4% increase in the number of private liquor stores in British Columbia between 2003 and 2008. The research found increasing density of liquor outlets associated with a rise in alcohol sales per head of population, as well as a 27.5% increase in the alcohol-related death rate per 1000 residents for each extra private liquor store (Stockwell et al. 2009;231 2011232).
Evidence consistently shows a link between areas of high outlet density and increased risky drinking and alcohol-related harm, particularly violence. A recent systematic review found 44 studies showing significant positive relationships between the density of outlets and levels of violence, alcohol-related traffic accidents, self-reported injuries and suicide, sexually transmitted disease and child abuse or neglect (Popova et al. 2009199). An association between outlet density and domestic violence was found in a study from Australia that assessed whether or not changes in outlet density at a postcode level were related to changes in domestic violence rates over a 10-year period. The study showed a 28.6% above-average increase in the domestic violence rate for each additional ‘packaged liquor’ outlet per 1000 residents in a postcode (Livingston 2011233). A recent English study found a positive relationship between density of off-licensed premises and alcohol-related harms among under-18-year-olds. The study demonstrated that nearly 10% of all alcohol-specific hospital admissions in England, excluding London, could be attributed to off-licence density, with every two extra off-licences per 100,000 of population resulting in one alcohol-specific hospital admission of an under-18-year-old per 100,000 (Coghill 2011234). A study by Chiang (2010; full reference withheld to protect anonymity) analysed the relationship between licensed premises and alcohol-related crime across the entire geographical area of the NHS near Thistletown. The findings suggested a strong positive relationship between the number of premises in a geographical area, the proportion of young people aged 16–24 years and the occurrence of crime.
Association between alcohol consumption, alcohol availability and socioeconomic status
Evidence of an association between alcohol consumption, alcohol availability and socioeconomic status remains inconclusive, overall. However, alcohol availability (measured by access to shops and restaurants which sell alcohol) may help to explain the link between neighbourhood deprivation and harmful alcohol consumption. Neighbourhood deprivation has been associated with the number of alcohol outlets, with more outlets in poorer areas; and studies have shown that a higher density of alcohol outlets is associated with increased rates of youth drinking and driving, assault, violence and homicide, rates of injury and rates of traffic collisions and traffic injuries (Scribner et al. 1994,235 1995;236 Treno et al. , 2001;237 Gruenewald et al. , 2002238). Studies at the city level, mainly in North America, have suggested that the density of alcohol outlets may be higher in poorer neighbourhoods (Pollack et al. , 2005239). Studies across nations (New Zealand) have shown a similar pattern, with greater access to alcohol outlets in more deprived urban areas (Hay et al. , 2009240). A recent Scottish-based study by Ellaway et al. (2010)193 investigated the distribution of alcohol outlets by area level deprivation across a large Scottish conurbation. A list was created of alcohol outlets with street addresses obtained from the city council in 2006, including seven categories of outlet ranging from public houses, off-sales, restaurants and bars to private members’ and entertainment clubs (bingo halls, nightclubs and concert halls). All alcohol outlets were mapped and proximity to nearest outlet calculated across quintile of area deprivation. The analysis included 2221 alcohol outlets across the city. The study reported that the sociospatial distribution of alcohol outlets across the conurbation does vary by deprivation but not systematically. Some deprived areas contain the highest concentration while others with a similar deprivation score contain very few. Within Thistletown three areas have the greatest level of alcohol outlets per 1000 population. The most deprived areas in the east end of the inner city have the second greatest number of off-sales. Thistletown’s four main peripheral housing sites did not have a high density of alcohol outlets of any kind. This study suggests that there is a less consistent association between the density of alcohol outlets and deprivation in Thistletown, findings similar to the studies carried out in North America and New Zealand.
Data on alcohol licensing
In 2010–11 there were 16,377 premises licences and 40,266 personal licences in force in Scotland. Of these, 10.5% (1715) of the premises licences and 12% (4832) of the personal licences were in force in Thistletown, which is very similar to the number and type of outlets in Scotland’s other urban centres. Similar figures were also relevant for 2011–12. Table 15 shows premises licences that were granted and refused under section 23 and personal licences that were granted and refused in Scotland as a whole, in another urban centre, in Thistletown and in a more rural setting close to Thistletown in 2010–11 and 2011–12.
| Outcome | Scotland as a whole | Thistletown | Other Scottish urban centre | Rural setting near Thistletown | ||||
|---|---|---|---|---|---|---|---|---|
| 2010–11 | 2011–12 | 2010–11 | 2011–12 | 2010–11 | 2011–12 | 2010–11 | 2011–12 | |
| Premises Licences Section 23 | ||||||||
| Refused | 10 | 21 | 1 | 8 | 1 | 3 | 1 | 1 |
| Granted | 327 | 354 | 66 | 50 | 22 | 69 | 8 | 2 |
| Personal Licences Section 72 | ||||||||
| Refused | 28 | 17 | – | – | 7 | 5 | – | – |
| Granted | 5886 | 5282 | 680 | 683 | 854 | 823 | 85 | 81 |
Of the total premises licences granted in Scotland in 2010–11 and 2011–12, there was a reduction from 20% to 14% granted in Thistletown respectively. In terms of licences refused as a proportion of the total figures in Scotland, there was an increase in refusals in Thistletown from 2010–11 to 2011–12, with a rise from 10% to 38%. These figures are for 2 years only, and therefore it would be difficult to suggest that any improvement in terms of overprovision would be misleading, but the figures for 2011–12 are certainly more encouraging. This picture, however, is not mirrored in the number of personal licences refused, as there were no refusals in both years in Thistletown.
Summary of the evidence: alcohol outlets and drinking behaviours in general and in Thistletown
There is no clear patterning of average alcohol consumption levels by area level deprivation. However, there is a pattern of increased alcohol-related morbidity and mortality by area level deprivation (higher levels of alcohol-related diagnoses in more deprived areas). The density of alcohol outlets is an important factor not just in relation to drinking levels but also in relation to other forms of alcohol-related harms, especially rates of assault and increased violence. There is no clear pattern of alcohol outlet density and area deprivation in Thistletown, but some of the most deprived areas have among the highest number of alcohol outlets, in particular in the east end of the city. Rates of alcohol-related morbidity and mortality remain to be analysed alongside data on alcohol outlet density for small areas.
Evidence regarding the use of evidence by Licensing Boards
MacNaughton and Gillan3 in their report ‘Re-thinking Alcohol Licensing’ suggest that there is limited interpretation of the licensing objectives in Licensing Board policy statements. They suggest that this is perhaps in part a reflection of the newness of policy formulation for Boards and the process of systematically gathering evidence, interpreting it, and then developing a policy position based on the evidence and in relation to the objectives. They reviewed the first batch of published policy statements and found that evidence sources are not routinely cited, nor is there always a clear line of reasoning from the evidence to the policy position adopted. Furthermore, in determining overprovision, they found that 14 Boards gave no information on statistics used or individuals/groups consulted, 14 specifically listed the police as a consultee, attesting to a continued licensing emphasis on public order problems, and four listed the Licensing Forum. Only one Board listed the NHS and just one listed an Alcohol and Drug Partnership. A lack of local data relating to the licensing objectives was sometimes highlighted in this regard. They argue that, while it is arguably the case that data on alcohol-related consumption and harm could be made more easily accessible and usable, a lot of local data exist that are not currently utilised by most Boards or, if referred to, are not translated into policy action. They found that one Licensing Board’s policy statement was a notable exception in demonstrating a systematic approach to gathering and analysing evidence and translating evidence into policy positions that related to the licensing objectives. The authors suggest that the patchy use of evidence of alcohol-related harm in policy statements is perhaps indicative of the degree to which Licensing Boards have traditionally operated in isolation from other Public Health functions and could, therefore, be symptomatic of a lack of integration between licensing policy and other public policies to which licensing matters pertain.
Support and guidance for Licensing Boards
An evaluation of the implementation of the objectives of the Licensing (Scotland) Act 2005201 recently published findings of a study of the implementation of the Licensing (Scotland) Act 2005, undertaken as part of the national monitoring and evaluation of Scotland’s alcohol strategy (MacGregor et al. 2013). 241 The report confirms some of the problems licensing officials have identified in trying to give meaningful effect to the public health objective. Among a number of issues raised by licensing officials is a perceived lack of guidance in how to address the public health objective and a view that it is too wide in scope and, therefore, too difficult to define and measure. The findings identify the need for more support and guidance for licensing authorities in fulfilling their duties under the 2005 Act: specifically, more guidance to be given to Boards on how to address the public health objective in their policy statements and, more generally, the objective to be more clearly defined and advice given as to which data sets to use or indeed develop in the future.
Appendix 3 The results of the second Delphi vote: perceived importance
FIGURE 14.
Importance of Delphi statements compared between first (n = 34) and second (n = 10) Delphi rounds.
Appendix 4 The results of the second Delphi vote: scores
FIGURE 15.
Behaviourally Anchored Rating Scale ratings compared between first (n = 34) and second (n = 10) Delphi rounds.
Appendix 5 Briefing sheet on working together in the cocreation of knowledge
Title: Research Evidence Use in the Commissioning of Alcohol Services and Interventions
Cocreation – working together on the project.
This short document will help explain what is involved in jointly researching how evidence is used in the commissioning of alcohol services and what we can expect from each other.
Before the project begins we would like to:
-
Meet each other (faces to names. . .)
-
Agree which ‘alcohol area’ to research – so any findings are useful and timely for you.
-
Explain what we will do (data collection), when, where & how often, so that is acceptable.
-
Begin to identify the first people to contact.
-
Establish our ‘adoptive family’ – 2–3 people (involved in the area) who will work with us more closely (see below).
As the project progresses:
We will be on site for a few days each month between April 2012–March 2013 with some updates until November 2013 when the project ends.
-
We will want to interview staff involved, observe commissioning meetings and read the documents involved in that alcohol commissioning process; access any routinely collected data you have on the process. (We will look at other routinely collected data on your organisation held externally).
-
Meet our ‘adoptive family’ regularly, to share and discuss early findings, and check our understanding.
-
You may (or may not) want to add 1–2 questions of your own for us to explore or take an active part in data collection. This is entirely your choice.
We will:
-
Explore the organisational processes that help or hinder the use of research evidence.
-
Be available to answer questions.
-
Respect the confidentiality of all staff and the organisation.
-
Seek informed consent from those taking part and be sensitive to practice needs
-
Share findings with our ‘adoptive family’ regularly – so you can act on them if you wish.
-
Offer you opportunities to: share results with our other case study site and an invited national audience; jointly publish and present findings at conferences; disseminate findings across your networks.
-
Offer you input to the final report, and briefing summaries for your use.
More details about the project:
Members of the research team are: Professor Rosemary Rushmer (Principal Investigator Teesside University); Ms Karen Smith (Research Associate Sunderland University). Dr Mandy Cheetham, Teesside University.
Other co-applicants are: Professor Ann Crosland (Sunderland University); Professor David Hunter (Durham University); Dr Joanne Gray (Teesside University); Professor Carol Tannahill, and Dr Rosie Ilett (both Glasgow Centre for population health); Lynda Cox (the NE SHA) and Liam Hughes (formerly the Local Government Group). External members of our advisory group are: Professor Cam Donaldson (Glasgow Caledonian University); Dr Crispin Coombs (Loughborough University); and Dr Claire Sullivan (NHS County Durham and Darlington).
Professor R Rushmer
February 2012
Early tasks:
-
Discussion with senior level people.
-
Initial discussion and scoping of topic.
-
Alcohol-related.
-
Something which will be useful to you.
-
Identification of who should be involved.
-
Contacts for meetings and interviews, including Chief Knowledge Officers or knowledge transfer leaders.
-
-
Identification/establishment of a small team (2–3 people) who have an interest in that area to work as a joint interpretation group (meet/report back every 2–3 months).
-
Assistance with identifying relevant staff.
-
Access to relevant documents.
-
Agreement to observation of meetings (and consent in principle).
-
Interviews with key stakeholders early on in project.
-
Fieldwork.
-
How often?
-
-
Permission required for:
-
Interviews with stakeholders involved in commissioning and/or knowledge exchange and research utilisation.
-
Observation of meetings related to commissioning or joint planning in chosen topic area.
-
Access to documents used in and related to process.
-
-
Interviews and other onsite work arranged around meetings to reduce recurrent disruption for your staff and enable researchers to make efficient use of visits.
-
Joint interpretation group.
-
A small group of 2–3 people who are sensitised to local and organisational issues and are willing to work more closely with the researchers to:
-
Provide advice and guidance.
-
An awareness of context and sensitive issues.
-
Act as a first point of contact on-site.
-
Be involved in regular (every 2–3 months) feedback meetings in order to feedback on findings, help interpret findings and check/clarify our understanding.
-
Take action if needed.
-
What is expected of participants?
-
Two strands: staff involved in commissioning and staff who have the role of providing evidence.
-
-
Commissioning staff.
-
An interview.
-
Observation of some relevant meetings.
-
Possibly 2–3 follow-up interviews (which may be done by phone).
-
-
Evidence-providing staff.
-
One (possibly more) interview.
-
Appendix 6 Interview topic guide for National Institute for Health Research Health Services and Delivery Research research: those involved in commissioning or joint planning process case study
Welcome and introduction
Introduce self and thank participant for taking part.
-
Outline study.
-
Inform participant of the approximate duration of the interview and how the process will be conducted.
-
Ensure participant understands what is involved, has given informed consent and is able to withdraw this at any point during the interview.
-
Ensure participant understands extent of confidentiality.
-
Ensure participant is comfortable with venue/layout/temperature, etc.
-
Introduce self/thank for participation/run through confidentiality and ability to opt out at any moment.
Background/context/setting the scene.
-
How does the commissioning/alcohol licensing process work here?
-
Can you explain your role in relation to the commissioning of services/alcohol licensing?
-
What helps you make decisions about what to commission/alcohol licensing?
-
What influences you?
-
OR
-
What informs commissioning/alcohol licensing decisions?
-
What influences the decision-makers?
-
-
What is evidence and how is it introduced? (Prompt – information can refer to research evidence, figures and trends, professional know-how, collective memory and experience of how things have worked in the past.)
-
When I say ‘evidence’ what comes to mind?
-
How are different types of information used in the commissioning/decision making process? How do you choose what information to use?
-
What kind(s) of evidence do you think is used to inform decision-making here?
-
Where does this evidence come from? Source(s) of evidence?
-
Who uses it/in what ways?
-
Where? When? How?
-
Where does it emerge from? Who brings it in/introduces it/tables it?
-
Why this particular form of evidence, in this situation/context, at this time?
-
-
-
How does this process work/what happens to information/evidence?
-
What factors help/hinder the use of evidence in this process?
-
Barriers/opportunities?
-
Is this the same in relation to other commissioning decisions?
-
-
What is seen/treated as important evidence?
-
Why is this so?
-
Is one person/organisation’s evidence more credible? Why is this so?
-
-
What are the individual/organisational factors that support or hinder the use of evidence?
-
Why/how does evidence spread?
-
-
How is evidence used?
-
How/by whom/to what effect?
-
Do you think different people make different uses of the evidence/prefer different types of evidence?
-
-
Do people challenge the evidence that is presented? Why/not? How?
-
Have things always been like this? Are things changing? If so how and why?
-
This might be explored in various places.
-
-
-
Who decides which evidence should be used and when?
-
What, if any, influence do you have over how evidence is used? (Personal/professional/organisational.) If not you, who does have influence?
-
What kind of influence/with whom/to what effect/what was the impact of this?
-
-
Can you give examples of how evidence is/has been used to guide decision-making?
-
What was this evidence/by whom?
-
Why this particular form of evidence, in this situation/context, at this time?
-
To what effect/what was the impact of this?
-
What factors made a difference? (Professional status, experience, skills, expertise, competence, power, influence, credibility, timing, environment, partnerships, protocols, organisational context, resources, relevance?)
-
-
Can you give me an example of when evidence was used well (to good effect) and/or when you think the outcome was less effective?
-
-
Own experience – if any.
-
Have you ever presented/introduced evidence? If no, why not? If yes:
-
What was this/how did you source it/why did you think it was useful?
-
How did you introduce it – at what forums/meetings/etc./via e-mail?
-
What was the reaction to this/how did you feel?
-
What was the impact of this?
-
Did this evidence inform people’s knowledge, ideas, attitudes? If yes, who and how? Did it inform commissioning/planning decisions? How?
-
-
Does research evidence always hold the best solutions/recommendations? Is it most useful? Why/why not/examples?
-
-
What else influences decisions?
-
What other evidence is there?
-
How is this also used? How is this different to what is used? What value does it have/to whom?
-
-
Is this other evidence valued in the same or different ways to what we have already discussed? How/to whom?
-
Are there sometimes reasons not to act on the evidence? If yes, why? Examples?
-
Do the views of the service users/patients/carers/public feed in to commissioning/decision-making? How?
-
What else should we consider, alongside evidence as important factors which make a difference, not covered so far?
-
Which other stakeholders have important roles to play in whether and how evidence is used here? Who? Why?
-
How do you think you are viewed, in terms of using evidence in commissioning/alcohol licensing, in comparison to other cities/areas?
-
-
Is there anything you would like to add that has not been covered in the interview?
-
Is there anyone else you think we should interview in this study?
-
What is their role?
-
Why do you think it would be important for us to speak to them/what would they bring to this study?
-
-
Recap and closure
-
Thank participant for taking part and for the information they have given.
-
Ask the participant if they are happy for the interview data to be used within the study.
-
Inform the participant that if they would like to see the summary of interview findings/final report how this will be disseminated/made available.
-
Give the participant contact details for the study team and contact details for the project lead.
-
Leave duplicate copy of consent form and PIS.
-
Appendix 7 The interview consent form
Appendix 8 The observation consent form
Appendix 9 Interview topic guide for knowledge managers
-
Welcome and introduction and consent form.
-
Background/context/setting the scene.
-
What is your job (very briefly)?
-
Do you work with the public health commissioners in Rosetown? What do you do as part of this process?
-
How does commissioning work in Rosetown?
-
-
Evidence gathering.
-
Do you gather evidence for decision-making around the commissioning of public health services in Rosetown?
-
What sorts of evidence do you gather for the decision-making?
-
How do you access them? What sources do you use for accessing evidence?
-
How frequently do you access different sources?
-
For whom do you access evidence?
-
For which topics?
-
-
Evidence formatting.
-
How do collate the evidence you source? Is it ready-made and fit for purpose or do you have to rework it to make it useful?
-
Do you reformat or reinterpret evidence?
-
If so, what factors do you take into account when reformatting evidence (local context and knowledge, own experiences and expertise, etc.)?
-
-
Evidence dissemination.
-
How do you pass on the collated evidence?
-
To whom do you pass on the collated evidence?
-
Which media do you use to disseminate evidence?
-
How is this evidence used in your experience? Have you seen any changes occurring in the decision-making process as a result of the evidence you provided?
-
-
Any other questions?
-
Is there anything you would like to add that has not been covered in the interview?
-
Is there anyone else you think we should interview in this study?
-
Recap and closure. Thank you for your time. If we wanted to come back and talk to you again would that be OK (maybe by phone)?
-
Appendix 10 Interview consent form: the participant information sheet
Appendix 11 The lay summary for research participants
Appendix 12 Quantitative data for alcohol consumptions and pregnancy in Rosetown
Fetal alcohol spectrum disorder: difficulty in estimating prevalence
Exact numbers are difficult to define in this spectrum of disorder and there are no accurate figures for prevalence in the UK. This is due to a number of factors, including the differing definitions and conditions along the spectrum, the poor accuracy in self-reporting of alcohol consumption, lack of standardisation of levels of drinking, reluctance to make or accept the diagnosis and paucity of reliable data collection. Most figures come from the USA, where it is estimated that fetal alcohol syndrome occurs in 0.5–2 live births per 1000 and fetal alcohol spectrum conditions occur at least three times as often as this. 242
Scoping the data for drinking in pregnancy in Rosetown
In order to ascertain the extent of the problem regarding drinking in pregnancy in Rosetown, a scoping exercise was undertaken in order to identify potential/relevant data sources. This took place between January and August 2013. Table 16 highlights points of contact and a summary of available data sources.
| Data source | Data domains | Data features | Comments |
|---|---|---|---|
| LAPE – Rosetown Public Health Observatory |
|
Annual trend data 2002–12, comparator cities, comparator local areas comparator PCTs and LAs 2008–13 | Data obtained including data for women of childbearing age. No alcohol consumption in pregnancy data available |
| Rosetown LA/PCT data | HRG inpatient data | 035.4 suspected damage to the fetus related to alcohol P04.3 fetus and newborn affected by maternal alcohol use Q86.0 fetal alcohol syndrome |
Obtained but biased owing to problems with coding |
| Rosetown Integrated Commissioning for Addiction and Offender Health | As above | Obtained as above | |
| Rosetown University Centre for Public Health | Monitoring data for the commissioning group | Same as LAPE data provided by Rosetown Public Health Observatory | |
| Rosetown Hospital | A&E data | No data | Data unreliable owing to data collection and coding within A&E |
| Rosetown Mental Health Trust | Inpatient data Psychiatric Liaison Community Teams (including monitoring data from Rosetown Community Alcohol) Service |
Drugs only/detox figures for alcohol | Not specific to alcohol consumption in pregnancy |
| Rosetown Ambulance Service | Alcohol-related calls | No data | Contacted and they do not record alcohol-related incidents and pregnancy. They record only injury type and would have to go through every individual patient report form to extract data, which would take over the 18 hours allocated for freedom of information requests. No resources to collect existing data. Not specific to alcohol consumption in pregnancy |
| Rosetown Maternity Hospital (foundation trust) | Sociodemographic data on women during pregnancy and a record of reported alcohol consumption from antenatal booking form | Total number of births per annum, local data on fetal alcohol syndrome, numbers of women known to have alcohol/substance misuse issues in contact with midwives. Booking question numbers | Obtained but data biased owing to women being reluctant to share alcohol consumption levels with midwives on booking form |
| National Drug Treatment Monitoring System with Manchester University’s NDEC | No data | No additional data regarding alcohol consumption in pregnancy |
Following the scoping exercise, it can be concluded that there are no robust and reliable data available regarding alcohol consumption in pregnancy either from a local or a national perspective. The most reliable data concerning alcohol misuse and its associated consequences are the local alcohol profiles for England produced by the North West Public Health Observatory (NWPHO), which is now part of PHE. The NWPHO takes a lead role on alcohol intelligence on behalf of PHE.
The NWPHO has produced and published Local Alcohol Profiles for England on an annual basis since 2006. 243 The latest update, Local Alcohol Profiles for England 2012, was released on 16 August 2012. The profiles contain 25 alcohol-related indicators for every LA and 22 for every PCT in England. The indicators measure the impact of alcohol on local communities and include a national indicator generated by the Department of Health – Admission episodes for alcohol-attributable conditions (previously National Indicator 39).
Alcohol misuse has health and social consequences borne by individuals, their families and the wider community. In 2006, the NWPHO and the Alcohol Research Team (Centre for Public Health, Liverpool John Moores University) gathered routine data and intelligence from a range of sources (including the Department of Health and the Home Office) to provide a national indicator set intended to inform and support local, subnational and national alcohol policies. These indicators provided measures to help prioritise and target local areas of concern. In addition, they provided a baseline for monitoring progress in reducing alcohol-related harm at local, subnational and national levels.
During the compilation of the indicators, the NWPHO reviewed methods of deriving population measures of alcohol-related harm, selecting indicators and developing methods consistent with current knowledge and understanding, thus reflecting the wide range of domains that may be affected by alcohol misuse. However, none of these indicators is relevant to alcohol misuse in pregnancy and its associated consequences.
Another data source that has some use to the project is that provided by Rosetown Maternity Hospital. This includes patient-level data on live deliveries by age, postcode, ethnic origin and reported alcohol consumption. However, these data are likely to be an underestimation of the alcohol consumption among young women or pregnant women, because they are self-reported. Such reporting is unreliable because of poor estimation, poor recollection and the social stigma associated with heavy drinking during pregnancy (Plant et al. 2005244). This situation is compounded by variations in the alcoholic concentration of different types of alcoholic drinks.
Data analysis
Alcohol use
Results from Rosetown Lifestyle Survey 2012–13 show that just over half of Rosetown adults drink alcohol (56%). Of those who do, 7 in 10 do so more than once a week, with most saying that they do so 1–3 times per week (54%). Just fewer than 1 in 10 (8%) adults in the city who drink do so every day of the week.
The lifestyle survey also shows that among the adults in Liverpool who do drink, 15% drink above the recommended guidelines (increasing risk), with a further 6% drinking at high-risk levels (greater than 50 units per week for men, and over 35 units per week for women). This equates to an estimated 30,150 and 11,300 residents, respectively. Research shows that surveys can drastically underestimate the level of alcohol consumption. A study245 in Manchester by the Centre for Public Health at Liverpool John Moores University indicated that normal surveys under-reported the amount of alcohol people say they drank by 33%, suggesting the prevalence of high-risk drinking in Rosetown could be much higher than our lifestyle survey results suggest.
Alcohol-related hospital admissions
Approximately 10% of all admissions in the city are estimated to be alcohol related, with approximately 15,000 alcohol-related admissions each year. Alcohol-related conditions include those specific to alcohol, such as alcoholic liver disease or alcohol overdose, and those conditions that are caused by alcohol in some but not all cases (e.g. stomach cancer and unintentional injury). Rosetown’s rate of hospital admission for alcohol-related conditions is one of the highest in England. Latest figures available for Q2 2012–13 show that Rosetown was ranked in the highest 10 out of 326 LAs in England (Q2 2012–13). While alcohol-related hospital admissions in the city remain high, figures indicate levels have plateaued in recent years, mirroring the pattern seen regionally and nationally.
Alcohol-related mortality
It is estimated that alcohol reduces life expectancy in Rosetown substantially, by 14.8 months for men and 7.6 months for women.
Approximately 80 Rosetown residents die each year from an alcohol specific condition, one-third of whom are female. In 2008–10 Rosetown was ranked in the top 10 highest out of 326 local authorities in England for its alcohol-specific mortality among males and in the top 20 highest among females. The rate of alcohol-specific mortality in males was 25.5 per 100,000, a fall of 9% on the previous year, and 12.1 per 100,000 in females, a fall of 13% on the previous year (Local Alcohol Profiler for England 2011–12; full reference withheld). Encouragingly, Rosetown’s alcohol-specific mortality rates for both males and females are at their lowest for 5 years; however, they remain significantly higher than nationally and are in the top five among the core cities.
Deprivation
Harmful drinking of alcohol is closely associated with deprivation, and females of childbearing age in the most deprived wards of the city are more likely to die from an alcohol-specific condition (data withheld to protect confidentiality).
Maternal alcohol use and suspected damage to fetus in Rosetown
From April 2010 to February 2013 in Rosetown there were 11 admissions to hospital using the following Health-related groups:
-
035.4 – suspected damage to the fetus related to alcohol
-
P04.3 – fetus and newborn affected by maternal alcohol use
-
Q86.0 – fetal alcohol syndrome.
As suggested above, these figures cannot be relied on owing to under-reporting. In the UK, there are currently no reliable prevalence figures for fetal alcohol syndrome or FASD as they are not routinely collected or recorded by the British Paediatric Surveillance Unit (BMA, 2007). 130 However, there is consensus arising from separate research studies carried out in different countries that the figure for fetal alcohol syndrome is 1 in 1000, with FASD being 3 to 4 times higher.
Assuming that 1 in 500 babies are born with fetal alcohol syndrome per year, this would equate to 11 babies with fetal alcohol syndrome born in Rosetown per annum.
Numbers of women who report any alcohol consumption at antenatal booking
The crude figures below (Table 17) show data obtained from Rosetown Maternity Hospital who had reported any level of alcohol consumption and had gone on to have a live birth. These suggest that the numbers of women reporting alcohol consumption have actually decreased (see Table 17). However, these figures do not take into account any changes in the numbers of women having their babies at the trust.
| Year | Number of women |
|---|---|
| 2003 | 1606 |
| 2004 | 1960 |
| 2005 | 1742 |
| 2006 | 1444 |
| 2007 | 1121 |
| 2008 | 736 |
| 2009 | 713 |
| 2010 | 498 |
| 2011 | 336 |
| 2012 | 373 |
Appendix 13 The vignette used to illustrate wicked issues in the use of evidence in Thistletown
Appendix 14 The vignette used to illustrate wicked issues in the use of evidence in Rosetown
Appendix 15 The vignettes used to illustrate the wicked issue when local knowledge managers try to use research evidence (both sites)
List of abbreviations
- A&E
- accident and emergency
- ADP
- Alcohol and Drug Partnership
- AUDIT-C
- Alcohol Use Disorders Identification Test
- BARS
- Behaviourally Anchored Rating Scale
- CCG
- Clinical Commissioning Group
- CHP
- Community Health Partnership
- CMO
- context–mechanism–outcome
- DPH
- Director of Public Health
- EBM
- evidence-based medicine
- EU
- European Union
- FASD
- fetal alcohol spectrum disorder
- GP
- general practitioner
- HIC
- high-impact change
- HWB
- Health and Well-being Board
- IRAS
- Integrated Research Application System
- LA
- local authority
- NICE
- National Institute for Health and Care Excellence
- NWPHO
- North West Public Health Observatory
- PCT
- primary care trust
- PHE
- Public Health England
- R&D
- research and development
- RCM
- Royal College of Midwives
- RCT
- randomised controlled trial
- RQ
- research question
- VONNE
- Voluntary Organisations Network North East
- WHO
- World Health Organization
Glossary
A full glossary of licensing terms is available from MacGregor A, Sharp C, Mabelis J, Corbett J. An Evaluation of the Implementation of, and Compliance with, the Objectives of the Licensing (Scotland) Act 2005: Final Report. Edinburgh: Scottish Centre for Social Research; 2013.