Mobilising knowledge to improve UK health care: learning from other countries and other sectors - a multimethod mapping study

Review of research reviews of knowledge mobilisation

The first stage of constructing the conceptual map involved reviewing key research reviews of knowledge mobilisation to identify the main knowledge mobilisation models and strategies in health care, education and social care.

We were already aware of many of the key reviews in this field from our prior work in this area. In order to augment this collection systematically, we contracted with The King’s Fund Information and Library Service (health and social care databases) and with the Evidence for Policy and Practice Information and Co-ordinating Centre (EPPI-Centre) (education databases) to design and carry out systematic searches of key databases in each field. We had detailed discussions with the information scientists, both in advance of and during the searching process, to clarify the objectives and parameters of the search and to refine the search terms. Details of the searches, including the databases searched and the search terms used, are given in Appendix 1.

Excluding duplicates, the search yielded 142 items in health and social care databases and 362 items from the education databases; the larger number of items from the education databases resulted from the more limited indexing facilities in these databases which led to a greater number of less relevant items being retrieved.

The abstracts were then read by the three members of the research team (at least two members for each of the searches) who allocated ‘scores’ to each review on the basis of its apparent relevance to the study: 1 = include (core paper); 2 = of some interest; 3 = marginal; and 4 = not relevant. Those papers which attracted a score of 1 or 2 from either or both of the assessors were then retrieved and read in full by one member of the research team (AP). Papers which met both of the following criteria were retained: substantial research review of the field of knowledge mobilisation; of relevance to health care, social care or education. This included conceptual development papers that contained within them a substantial review section relevant to our concerns. We identified further relevant reviews from reference lists of our key reviews and drew on the expertise of members of the advisory board and our key contacts in social care and education to check whether or not there were any additional key reviews (published or in press) that we had missed. Thus, we were satisfied that we had identified the main research reviews relevant to the development of a conceptual map.

This process resulted in our identifying a total of 71 reviews (listed in Appendix 2), which formed the core set of reviews. At the outset of the study we had anticipated that there would be around 30–40 relevant reviews; however, the process described above uncovered rather more. This somewhat increased the scale and scope of the task, but provided a rich set of conceptual concerns on which we could draw.

Initial development of the conceptual map

The initial development of the conceptual map was carried out as part of the desk research and is described here and in Chapter 3. The conceptual map was then developed further by the addition of knowledge mobilisation ‘archetypes’, which were derived from the interview data; this part of the process is described in a later chapter (see Chapter 6).

The aim of the conceptual map was to provide a ‘map’ of the knowledge mobilisation terrain as it applies to key research funders, producers and intermediaries developing their knowledge mobilisation strategies. The objective was to develop a tool that could be used both for analytic purposes (for the exploration of data gathered later in the study) and for communication purposes (as a practical way into the literature for use by agencies considering the development of their knowledge mobilisation strategies). The first stage of developing the conceptual map was to set out visually the domains identified inductively by our reading of the major reviews of knowledge mobilisation. One member of the research team read all of the reviews and prepared a document for the other researchers summarising the key points from each review as they related to the focus of the study, that is the knowledge mobilisation strategies and approaches of research funders, research producers and research intermediaries; on average, each review took up around one side of A4 in this summary document.

Having read the summary document and drawing on their existing knowledge of the literature, the research team then discussed the key domains that needed to be included in the conceptual map; the aim was to distil the core conceptual areas underpinning knowledge mobilisation and to do so with knowledge of the existing typologies but without seeking to follow or reproduce these. This discussion resulted in six domains being agreed on: knowledge of all kinds; connections and configurations; purpose(s) and goals; people, roles and positions; actions and resources available; and context.

The members of the research team then independently reread the summaries of 10 of the key reviews to check whether or not these initial six domains appeared sufficient to capture the core conceptual areas. One member of the research team then read through the whole of the summary document again to extend this initial ‘checking’ process to the whole group of key reviews (i.e. to check whether or not all of the key issues raised by the reviews could fit well under these domains or if there were additional domains that needed to be added). No further domains were added to the conceptual map as a result of these two processes.

Once the main domains for the conceptual map had been agreed, these domains were used to structure the further exploration of the key reviews (e.g. to consider the key concepts, issues and theories covered under each domain, the evidence underpinning them and the implications for agencies). The findings from this work are set out in Chapter 3.

Mapping of the key agencies

Early in the study we carried out careful mapping of the key research funders, major research producers and key research intermediaries (e.g. research collation agencies, think tanks, charities, professional and membership organisations) in health care in the UK and internationally, and in social care and education in the UK. Our focus was on agencies that had substantial investment in programmes of knowledge mobilisation rather than on delivery units, policy settings or other sites of research application. Throughout the study we used the terms ‘agencies’ and ‘organisations’ interchangeably as useful shorthand terms to cover a variety of organisations.

In order to compile a list of organisations to research, we first drew up from our own previous work and from key reviews an initial list of potential organisations in each sector. In the case of health care, we included organisations in the main English-speaking countries/regions outside the UK known to be active in knowledge mobilisation; in the case of social care and education, we included organisations in the UK alone, in line with the remit of the study. For the UK, we also compiled a cross-sector list in recognition that many UK organisations work across more than one sector. We then consulted two key contacts from each of the social care and education sectors for their comments on the initial lists (social care and cross-sector and education and cross-sector as appropriate) and their suggestions of additional organisations. This was an additional stage carried out in preparing the list of education and social care agencies in recognition that these sectors were less familiar to two members of the research team.

The initial lists were then circulated to contacts in each sector so that they could be ‘validated’ (described below) and expanded further. The social care and education contacts were different from those who had been consulted when the initial lists were being drawn up. All of the contacts at each stage were individuals who were known to have considerable expertise and experience of both knowledge mobilisation and their sector. Individuals were asked to review the lists of organisations in their region/country and sector as follows: UK social care and cross-sector (two contacts); UK education and cross-sector (two contacts); UK health care and cross-sector (three contacts); continental Europe health care (three contacts); health care USA (three contacts); health care Canada (four contacts); and health care Australia and New Zealand (three contacts).

Reviewers were asked to give a ‘broad brush’ rating to each organisation on the list using a scale from 1 (lowest) to 3 (highest) to show which of these organisations were responsible for a significant scale of knowledge mobilisation activity and/or pursuing especially innovative approaches to knowledge mobilisation. We were not looking for absolute consensus but instead were seeking a pragmatic assessment from experts in the field to help us to compile a list that was comprehensive and yet manageable within the constraints of the project.

As an additional check on organisations that we might have missed, participants were also asked to add any key organisations to the list. The process was augmented by e-mail and telephone conversations with some of the participants, which provided additional detail on organisations, particularly those outside the UK with which the research team was less familiar.

We then used the ratings and the additional comments that participants provided to allocate organisations to three groups according to the following criteria:

• Group A: major players in scale or degree of innovation in knowledge mobilisation: selected for interview (preceded by website research); included in sample for web survey (n = 55).

• Group B: agencies with significantly interesting knowledge mobilisation activities: included in website research, with the possibility of an interview if the website research uncovered particularly interesting initiatives that merited further research by the team; included in sample for web survey (n = 131).

• Group C: organisations that carried out knowledge mobilisation activities as part of their role and were, therefore, part of the wider knowledge mobilisation field, but that did not appear to be conducting knowledge mobilisation activities that were significant in terms of scale or degree of innovation – no further action (this broader hinterland of knowledge mobilisation agencies was not fully enumerated and included all of those agencies that came to our attention but which were not considered further for more detailed review).

In order to seek wider confirmation that the list of organisations we had compiled was comprehensive, we posted messages to two key e-mail discussion lists used by the UK research communities (and with some reach beyond the UK): JISCMail Health Services Research list (around 550 subscribers) and JISCMail Evidence Use list (around 200 subscribers). We asked subscribers to review our provisional combined list of organisations and to contact us if they were aware of any prominent research funders, major research producers and key research intermediaries that were particularly active or innovative in knowledge mobilisation but that did not appear on our list. These postings resulted in our being notified of 20 additional organisations, of which we added six organisations to group A (website research, interview and web survey) and eight organisations to group B (website review and web survey); six of the organisations suggested fell outside the remit of the study or were subsumed under other organisational entities already considered.

At the end of this mapping exercise we had a list of 186 organisations for inclusion in the study, of which we had categorised 55 organisations as being in group A (30 in health care, 10 in social care, seven in education and eight cross-sector organisations) and 131 as being in group B. Analysis of the activities of these organisations addresses RQs 1a and 1b, and the organisations themselves are listed in Appendix 3. Interview organisations appear in Appendix 3 in bold font.

In line with our RQs, our aim was to map the knowledge mobilisation strategies that have been developed at the agency level, to understand the models, theories and frameworks underpinning them, and to learn from their success or otherwise (so addressing RQs 1a, 1b, 2a, 3a and 3b). Thus, our primary objective in compiling the list of organisations was to ensure that we had captured the main strategies, approaches and innovations in use. Our intention was not to compile an inventory of all of the organisations carrying out knowledge mobilisation in their respective sectors, and our final sample of agencies should not be interpreted in this way.

This sampling of agencies underpins our interview strategy and web-based survey work described later in this chapter and reported in Chapters 4 and 5.

Review of evaluations of knowledge mobilisation work by the agencies

As part of our review of agencies’ knowledge mobilisation work from their websites, we looked for evidence of evaluations by the agencies of these activities, we considered the evaluation methods used, and we looked for any formative or summative learning reported in the evaluations (addressing RQ 2b). We supplemented these web searches by specifically asking interviewees for information about any informal or formal, internal or external evaluations of their knowledge mobilisation activities, the criteria and methods used for such evaluations and the key findings. Where such evaluations were publicly available, we requested copies of relevant material from the interviewee or downloaded them from the agency’s website as appropriate. We report findings from this work in Chapter 4.

Terminology

The first section, on terminology, explained that the survey was ‘using the term knowledge mobilisation to cover activities aimed at sharing research-based knowledge’ and asked respondents to indicate all other related terms commonly used in that organisation (e.g. knowledge transfer, knowledge translation or research use). The list of terms was drawn from the key reviews.

Knowledge mobilisation activities

This section, consisting of three questions, provided a list of activities that might be carried out as part of a knowledge mobilisation strategy and asked respondents to indicate which of these activities were carried out in that organisation and with what frequency (often; sometimes; planned for the future; never/does not apply). The list of activities was drawn from the reviews and from the interview data. It included (but did not signpost as such) some activities which might be associated with ‘push’ approaches, some that are related to ‘pull’, some which might be associated with ‘linkage and exchange’ and some which specifically involved patients or service users.

Models and frameworks used in knowledge mobilisation

The third section invited respondents to indicate which of a list of models and frameworks were used in developing knowledge mobilisation activities. The models and frameworks listed were those described most often in the key reviews and in the interviews. Respondents were invited to name any other frameworks the organisation used in developing its knowledge mobilisation activities.

Ideas around knowledge mobilisation

The fourth section asked respondents to indicate, on a Likert scale, the degree of agreement with key propositions drawn from the published reviews and from comments made in the interviews.

Developing knowledge mobilisation activities

The fifth section asked respondents to weight in terms of importance (very important; fairly important; not that important; do not know/does not apply) factors that might be considered in developing knowledge mobilisation activities (e.g. acceptability to stakeholders; attention to context; feasibility of evaluating the approach; previous use in that organisation). These factors were drawn from the reviews.

Evaluating the organisation’s knowledge mobilisation activities and measuring impact

A final section concerned the evaluation of the organisation’s knowledge mobilisation activities. Respondents were first asked to indicate whether or not the organisation evaluated their knowledge mobilisation activities (little/no formal evaluation; some evaluation; a comprehensive approach to evaluation) and then to indicate the relative importance of measuring different types of impact (e.g. change in users’ attitudes and intentions; impact on outcomes for service users). The types of impact were drawn from the reviews.

Finally, free-text boxes were provided for respondents to add additional comments or clarifications at appropriate points during the survey. A further free-text box was provided for any further comments at the end of the survey.

Reviewing, piloting and running the survey

International advisory board members (in conjunction with colleagues where possible) reviewed the draft survey and suggested changes which were then incorporated into the survey. The final version was piloted for comprehension and ease of use with a small number of colleagues who were familiar with the field but were not themselves due to receive the survey; these responses were not included in the survey findings.

The survey was sent out by e-mail to a named recipient in each of the 186 agencies. To deal with situations where individuals had more than one institutional affiliation, the covering e-mail specified which organisation was of interest. Recipients were asked to advise the research team if an alternative contact in that organisation would be more appropriate. In addition to careful piloting of the survey to assess comprehension and ease of completion, a range of further strategies was used to ensure a good response rate: sending a personalised invitation to participate in the survey; emphasising to participants the relevance of the study and the importance of participation; sending prompt follow-up reminders, together with an invitation to nominate an alternative contact if appropriate; and changing the wording of reminders rather than sending out successive reminders with identical wording. 44–48

A total of 186 contacts were sent the survey with a request for completion, and after two reminders (one after 2 weeks and one after a further 3 weeks) useable data were obtained from 106 of these giving an overall response rate of 57%. Fifty-one respondents responded to the first invitation, followed by 30 who responded to the first reminder and a further 25 who responded to the second reminder. The level of missing data was generally low (as shown in the results set out by question in Chapter 5), although, as explained in Chapter 5, around one-third of respondents skipped the question on models, theories and frameworks used in knowledge mobilisation. Analysis of the survey data involved compiling descriptive statistics (e.g. frequency counts of use of particular models or frameworks; extent of agreement/disagreement with key propositions) as well as thematic content analysis of free-text comments.

Purpose(s) and goals

In developing knowledge mobilisation strategies, a central concern is to what end is the knowledge being mobilised? Agencies of different kinds will have different views on what they are trying to achieve (and how), and such concerns are central in the literature. While clearly closely related to the second domain (knowledge types and sources) and the fifth domain (actions and resources), a consideration of the purpose and goals of a knowledge mobilising agency can be informed by a wide array of literature.

There are multiple definitions of knowledge use in the literature, with no one definition dominating. 21 Underneath some of these disputes is a concern to tease out the extent to which research-based information is dominant in the aims of knowledge mobilisation efforts. That is, do we wish to see evidence-based actions, evidence-informed actions or just activities that are evidence-congruent or evidence-aware? The centrality or otherwise of research is a key consideration for agencies, then, and this is an issue picked up further when we consider types of knowledge.

Given a central and somewhat privileged position for research-based knowledge, one common and recurring typology55,90,113 suggests that there are three main types of research use: instrumental, conceptual and symbolic.

• Instrumental/direct use: applying research findings in specific and direct ways to influence decision choices.

• Conceptual/indirect use: using research results for changing understanding or attitudes, including introducing new conceptual categories, terminology or theories (sometimes called ‘enlightenment’ uses). 114

• Symbolic/political/persuasive use: using research findings to legitimise and maintain predetermined positions, including the ‘tactical use’ of research, for example justifying inaction while awaiting further study.

These different types of use may be affected by many different factors. For example, direct use is associated with well-defined decision-taking; conceptual use may be longer term and more percolative in style; and symbolic use may be closely associated with political argument and the use of mass media. It is likely, however, that all three will be seen simultaneously in some settings (e.g. in government agencies).

Categorisations such as that above can be further refined by additional analysis within each category. For example, instrumental use may be concerned with decisions that impact on professional processes, individual patient outcomes or aggregate economic outcomes,115 and each of these can become targets or goals of agency activities. An alternative view55 might look for impacts on knowledge-use processes (such as knowledge being seen, discussed and cited), intermediate outcomes (such as key actors’ awareness and attitudes) and decision outcomes (evidence-supported change). Combinations and hybrids of these typologies of use and impact are, of course, possible, and a consideration of these may help to sharpen agencies’ strategic goals.

The categorisations above are related to, and have some overlap with, the influential typology produced earlier by Weiss. 114 Although based on seven ‘meanings’ of research use, these models could also be interpreted as types of use in themselves. While Weiss’ typology emerged from empirical work on policy-makers, it has also influenced thinking on research use in practice contexts. 15

Weiss argued that there were seven different categories in the way that research-based knowledge was used (adapted and extended from the original):

• Knowledge driven: where research produces knowledge that might be relevant to public policy decisions. This is closely aligned with the direct and instrumental use noted above (and is central to strategies of research ‘push’) but can also encompass more conceptual uses of research where the knowledge shared is more theoretical or conceptual in nature.

• Problem-solving: where research is sought out that can provide empirical evidence to help solve contemporary policy problems. This again is most often associated with an instrumental approach (and with notions of research ‘pull’), but can also encompass some conceptual rethinking.

• Interactive: those engaged in policy or practice seek information from a variety of sources to help make sense of their problems and develop solutions; associated with ‘linkage and exchange’ approaches to knowledge mobilisation; may encompass both direct and indirect uses of research.

• Political/symbolic: policy-makers and others search for knowledge to help justify their positions, and so research-based information becomes ammunition for whichever side finds it most useful. Intermediary agencies with well-defined value sets may seek to exploit such opportunities.

• Tactical: where those who could be (and perhaps should be) research users fund or require new research to avoid taking action. Funders and research producers may each see opportunities here for longer-term gains in research-based knowledge.

• Enlightenment: where research has gradual influence over time in shaping conceptualisations of the issues and framings of the policy agenda. Recognition of these slower percolative processes may lead agencies into planning for the longer term by focusing on social processes.

• Societal: where policy interest, public concern and professional interests are meshed and stimulated by new research findings. Such a broader view enlarges the scope of agencies to encompass much broader sets of stakeholders.

Other commentators have sought to expose and critique the limited areas of application for research-based knowledge. Drawing on Habermas’ framework of ‘knowledge-constitutive interests’, Murphy and Fafard97 define three types of knowledge use:

1. instrumental, that is, problem-solving (using research to make decisions)

2. hermeneutic, that is, explanatory (facilitating greater understanding of the social world)

3. emancipatory, that is, equity-seeking (critically analysing institutional forces as a means of advancing social justice).

While having overlaps with other categorisations of use, the authors here suggest that most conventional knowledge mobilisation approaches focus on instrumental, problem-solving use. They suggest that social research has, or should have, other goals that are aimed at hermeneutic and emancipatory objectives. Such a reading brings to the fore political considerations whether an agency seeks to work with the grain of existing policy and practice presumptions, or to challenge these from a values-based position. 15,29,116

Moving into more practice-based models of research use, Walter et al. 60 propose three models. These are described as inductively derived archetypes, and actual practices often combine more than one model:

• The research-based practitioner model of research use: in this model it is largely the responsibility of the individual practitioner to keep abreast of research and ensure that it is used to inform their practice.

• The embedded research model of research use: in this model research use is largely achieved by embedding research in the systems and processes of practice (e.g. via standards, policies, procedures and tools).

• The organisational excellence model of research use: in this model, the key to successful research use largely rests with the leadership, management and organisational arrangements of service delivery organisations and with their promotion of a ‘research minded’ organisational culture.

The models are not seen as prescriptions for action but may be useful at different times and in different service settings. 117 Understanding of these archetypes of use may help agencies direct activities supporting the uptake of research in practice.

So, for agencies developing knowledge mobilisation work, it may be important to consider the types of ‘use’ they are aiming to influence and to what end. Such considerations do not stand alone, but are interconnected with the other domains mapped in Figure 1, most especially that concerned with understanding the nature of knowledge and the role played by research within that.

Knowledge

Although at first sight it might seem obvious what knowledge mobilising agencies are seeking to mobilise – that is, research-based knowledge – in fact the reviews we explored reveal considerable complexity and nuance to the nature of that knowledge. Both descriptive and prescriptive models see unpacking what is meant by knowledge as a central concern. For example, one well-known framework55 sets out five questions for research organisations to consider in relation to knowledge mobilisation. The first question in this framework concerns what should be translated, transferred or exchanged: the knowledge of knowledge mobilisation.

There is no clear dominant definition of knowledge in the literature. 21 The different philosophical paradigms on which the knowledge mobilisation models draw (explicitly or implicitly) make different assumptions about the nature of knowledge. For example, positivism assumes that knowledge can be uncovered and expressed in generalisable laws, constructivism holds that knowledge is socially constructed and that there are multiple truths, and critical theory analyses the relationship between knowledge and power.

A range of types of knowledge are identified in the literature, and these can be grouped in various ways. For example, types of knowledge can be grouped according to the source: does the knowledge arise from structured data gathering (empirical knowledge), from practical experience (experiential knowledge) or from abstract discourse and debate (theoretical knowledge)? Another grouping in the knowledge mobilisation literature contrasts explicit knowledge (such as can be set down in guidelines) and the tacit knowledge held by individuals and groups. 118 Tacit knowledge may be used to inform decisions in the practice setting but may not be susceptible to defining and describing explicitly. Amalgamations of explicit and tacit knowledge in clinical contexts have been referred to as ‘mindlines’. 57

One theory of knowledge creation119 holds that there is a close relationship between tacit and explicit knowledge. It suggests that new knowledge is created most rapidly when conversion between different forms of knowledge occurs continually (e.g. from tacit to explicit and from explicit to tacit). Given this, Oborn et al. 30 suggest that the negotiation and exchange of tacit knowledge in practice settings has been largely overlooked by knowledge mobilisation researchers so far. A related categorisation, drawing on work by Aristotle, distinguishes between episteme (scientific knowledge), techne (craft knowledge) and phronesis (situation-specific practical wisdom and the ability to apply generic knowledge to the current case). 24

In another framing, institutional knowledge is differentiated from individual knowledge, and local knowledge from external knowledge. Such distinctions are made based on whether the knowledge arises from individual or shared experience, and whether the knowledge is created in situ or is imported from elsewhere. A further stream distinguishes between knowledge as data and knowledge as ideas, asserting that data, information and knowledge lie on a continuum and differ in the extent to which human processing and judgement are needed. 120 Such literature also considers the extent to which knowledge has been processed, synthesised, ‘recycled’, reinterpreted or adapted; and if the knowledge is specific to a particular issue and context, or whether or not it is more general. Similar notions underpin the ‘knowledge to action’ framework. 1 Here, knowledge creation is composed of three phases, each involving a greater degree of processing: knowledge enquiry (first-generation knowledge), knowledge synthesis (second-generation knowledge) and the creation of knowledge tools such as practice guidelines and algorithms (third-generation knowledge).

Methodological categorisations of research-based knowledge also abound. 121 This is not just a distinction between quantitative and qualitative findings, but there exist a variety of more-or-less hierarchical distinctions, often with implicit or explicit endorsements as to their validity. 122,123 More prosaically, there is debate about whether or not single studies should be disseminated at all, rather than synthesised accumulations across portfolios of work. 55,124

Connections and configurations

Mobilising knowledge is about making connections. Our agency-based view of activities brings to the fore the need for agencies to connect and communicate in sometimes new and innovative ways. This may mean capitalising on existing networks or building anew. The reviews we uncovered offer insights into what and how such strategies might be developed.

Much of the literature we uncovered discusses the complex institutional, professional and social environment within which knowledge is created, flows (or gets stuck) and is applied. While some of these discussions lay heavy emphasis on ‘context’ as a mediator (which we discuss later in this chapter), there is also more specific consideration of the role of specific networks of interests or the practical configurations of agencies, organisations and relationships. As such connections and configurations are amenable to planned intervention or influence by agencies, it seemed important to tease out literature preoccupations here.

A framework that is increasingly well known (and resonant with other framings) is the ‘three generations’ framework. 20 This proposes that there have been three stages or generations of thinking about knowledge to action processes: linear approaches, relationship approaches and systems approaches. The authors set out the characteristics of each of these approaches and suggest conditions under which such approaches might be more or less appropriate. While these approaches are often linked to historical developments, with ideas of progression of thinking from ‘simple’ linear models to ‘complex’ systems thinking, it may be more helpful to think of these as parallel models of the knowledge mobilisation system with contingent application and different strengths and weaknesses.

People, roles and positions

Agencies interested in mobilising knowledge will act through co-ordinating the actions of their own people and through co-opting the skills and resources of others. In part, this is about the configurations and networks created or utilised as previously discussed, but within these we can discern distinct roles that are performed, by agencies and by individuals.

In this project, we have conceived of a threefold role-based typology of knowledge mobilising agencies: funders; research producers; and research intermediaries (of course, some actual agencies take on multiple roles and many exist in hybrid forms). To this we can add various types of ‘audiences’ for research:58 other researchers; members of the public and service users; practitioners; managers; and policy-makers. Several authors (e.g. Contandriopoulos et al. 21) point out that none of these are discrete categories and that individuals may belong to more than one group; this fact of multiple identities may constrain or facilitate an individual’s actions around knowledge mobilisation. A further categorisation of ‘insiders’ and ‘outsiders’ (e.g. researchers within government departments or external researchers from universities working with government departments) highlights the potential for individuals’ actions to be determined in part by their status in the social context. For example, studies have shown that internal researchers within government departments have greater access to ministers than do external researchers. 135

Moving on from this broad categorisation of audiences or stakeholder groups, Lavis et al. 58 suggest two key questions in defining a narrower group of stakeholders in a given situation: who can act on the basis of the research knowledge and who can influence those who can act? These questions helpfully give prominence to the issue of power, which many authors16,32 suggest has been neglected in relation to understanding of knowledge mobilisation.

There has been little empirical work on the actual or potential roles and responsibilities of different knowledge mobilisation actors. 11 There is, however, a strong focus in the literature on ‘knowledge brokers’ and other ‘mediator’ roles, and a growing number of empirical studies136–140 have investigated these roles. A range of functions has been suggested, including problem definition; research synthesis; facilitating access to research knowledge; developing outputs that are more accessible to users; and developing and brokering networks and other connections. 135,141 Linking and mediator roles have been promoted in many organisational settings and are perceived by health organisations to be an important component of the organisational infrastructure to encourage evidence use. 142 However, one review21 suggests that the structural position of brokers within organisations may mean that they have most scope to intervene in contexts where there is low polarisation of views (i.e. where actors already share similar views on key issues) and significant user investment in knowledge exchange, and that they may have limited ability to have an impact on the many existing networks that exist outside formal communication channels.

Conceptual uncertainty remains around who should perform knowledge broking and what activities should be encompassed by the role. 81,135 For example, it is unclear whether or not and in what ways knowledge broker roles are different from other roles such as opinion leaders, facilitators, champions, change agents or linking agents. 143 There are, nonetheless, some different theoretical assumptions behind the use of such terms: opinion leaders are typically seen as long-term ‘insiders’, whereas change agents are typically seen as ‘outsiders’ who have a short-term role in facilitating action around implementation. Knowledge brokers may come from diverse backgrounds, which then suggests roles that link or span different communities.

Many research funding agencies now make stipulations about what their funded researchers have to do in terms of knowledge mobilisation but these requirements are often limited to more traditional activities (e.g. engagement with potential research users, formal reporting and perhaps some ‘translation’ and dissemination). There is rarely any evaluation as to the effectiveness or impact of these requirements. 11 Indeed, some caution that researchers may not be the most appropriate people to undertake knowledge mobilising roles133 and that it is unrealistic to expect researchers to develop the broad range of skills required for effective knowledge sharing. 126

Leadership (including endorsement of the evidence from expert and peer opinion leaders) is regarded in the literature as important in knowledge mobilisation,93 but the requirements of roles here remain underspecified. Although leadership has been addressed in other literatures, the precise nature of leadership and its defining qualities have not been fully addressed in the knowledge mobilisation literature. 20

While roles matter, some authors argue that there has been disproportionate emphasis on individuals and their roles in relation to knowledge mobilisation. They argue that this focus ignores three key issues: that it is unrealistic to expect researchers to develop the broad range of skills required;126 that sustainable knowledge mobilisation requires multilevel systemic changes80 alongside appropriate technological and organisational infrastructures;78 and that greater attention needs to be paid to the organisational systems in which individuals work and which strongly affect what they are able to do. 77 Such critiques draw attention to other domains of our conceptual map.

One group that has largely been absent from the knowledge mobilisation literature is the public or service users. While patient and public involvement (PPI) has been strongly encouraged in research, the literature has been largely silent on the potential knowledge mobilisation role of these groups. One exception here is more recent work that has considered the evidence base on patient-direct and patient-mediated knowledge mobilisation interventions. 78,144 The gaps, challenges and opportunities of greater involvement of the public, patients, other service users, clients and parents is discussed in a subsequent section, but suffice to say that their role is not widely considered in the knowledge mobilisation literature.

In sum, agencies may find it useful to map their audiences or groups of stakeholders using the broad categories outlined above, differentiating between those who can take action directly (act on the evidence) and those who can influence those who can act, or those who can shape the context within which that action occurs. Moreover, intermediary roles need further elaboration and analysis that takes account of the other domains of the conceptual map (e.g. knowledge types; purpose and goals; existing networks and configurations; and local context). Currently, there is insufficient empirical evidence on the impact of knowledge brokers and other mediating roles, but early findings suggest that they are most likely to have impact when they are working to bring together previously isolated groups and are credible, skilled and well supported.

Actions and resources

To further their goals, agencies need not only to find partners and identify audiences, but also to develop action plans and deploy resources. The actions taken will depend on the underlying model of knowledge mobilisation being used (explicitly or implicitly), and the resource requirements differ for different models of knowledge mobilisation. The wide variety of models uncovered in this review have largely not yet been tested as prescriptions for practice, so it is not clear how suitable they are for planning and evaluating knowledge mobilisation strategies. 19 Many models provide a quite general overview of knowledge mobilisation rather than analysing the key features and intended effects of specific knowledge mobilisation interventions. 96 They thus leave unaddressed the specific actions required and the resources needed. Indeed, many models seem more descriptive of how change occurs rather than directly addressing the planning of change initiatives. 75

Some sets of activities have been identified in the literature that might form the first step in the operationalising of a knowledge mobilisation strategy. For example, Walter et al. 145 highlight the key underlying mechanisms that can be used to build research impact: dissemination; interaction; social influence; facilitation; and incentives and reinforcements. In practice, many strategies will involve a judicious mix of these, and selecting the appropriate mix and emphases remains to be addressed.

Taking a holistic view of encouraging research use, one review in social care60 sets out a collection of imperatives, each of which might suggest collections of (resourced) activities that need to be planned. These include ensuring a relevant research base; ensuring access to research; making research comprehensible; drawing out the practice implications of research; developing best practice models (e.g. pilot or demo projects); requiring research-informed practice (e.g. through regulatory influence); and developing a culture that supports research use (multifaceted). Again, these broad categorisations leave much detail that needs to be fleshed out by agencies in their local context and given their resource constraints.

Agencies wishing specifically to advance the field of knowledge mobilisation may draw on the five functional areas outlined by Holmes et al. 126 These include advancing the science of knowledge mobilisation; building capacity; managing specific projects; funding knowledge mobilisation activities; and advocating for greater knowledge use. Again, such categorisations may provide a starting point for agency strategy development.

Holmes et al. 126 drew on work by Kitson and Bisby to set out nine actions that funding agencies could take to support knowledge mobilisation:

1. require the involvement of research users throughout the research cycle

2. support activities to increase the ability of researchers to communicate with users

3. provide forums for knowledge users and researchers

4. require a knowledge mobilisation action plan for all funded projects

5. provide training and support to granting panels for the assessment of knowledge mobilisation plans

6. include knowledge mobilisation costs as eligible expenditures

7. fund activities that facilitate easier access to research data by knowledge users

8. require open-access publishing

9. fund rapid response programmes to address urgent policy and practice issues.

Agencies could use this list of imperatives to help set priorities for action. 126

The narrative synthesis of conceptual frameworks by Wilson et al. 146 reviewed 33 knowledge translation frameworks, of which 20 were designed to be used by researchers to guide their dissemination approaches. Twenty-eight of the frameworks reviewed were underpinned (at least in part) by one or more of three theoretical approaches: persuasive communication; diffusion of innovations theory; and social marketing. The authors noted that, although 10 UK funders of health services or public health research made reference to dissemination in their research funding application guides, only one [the Economic and Social Research Council (ESRC)] specifically provided a dissemination framework for use by funding applicants and grant holders. The authors suggested that, as a first step, funding agencies could specifically encourage the adoption of a theoretically informed approach to dissemination activities for their grant holders (e.g. by requiring the use of one of the theoretically informed frameworks). Another narrative review147 reviews 61 dissemination and implementation models and discusses key considerations around model selection and adaptation.

Other reviewers have noted79 that many organisations are increasingly developing practices (such as portals, websites and online interactive spaces) despite the limited evidence in support of such initiatives. Thus, many of the practical actions taken by agencies may have more to do with their face validity, stakeholder acceptability and the availability of local expertise than coherent strategy or supportive literature. 11

All of the above suggest that there are actions required by agencies across a number of spheres. This draws attention to the potential for balanced and multifaceted activities. While some reviews suggest that multifaceted approaches are more effective than single interventions,148 some authors suggest that multifaceted approaches may not always be appropriate: there is a risk of a ‘scattergun’ approach, and the effectiveness of multicomponent approaches will depend on the interaction of the different mechanisms within particular contexts. 117 One review of strategies used in public health149 found that simple or single strategies were in some cases as effective as complex multicomponent interventions, and suggested that this was because key messages might be diluted or harder to comprehend in complex multiple interventions. Multifaceted approaches are also likely to be more costly than single interventions and consideration needs to be given to how the different components might interact. 76

Disaggregating some of the above broad categories, a major review in 2012150 collated 68 specific implementation actions, grouped according to six key implementation processes: planning, educating, financing, restructuring, managing quality and attending to the policy context. The authors differentiated between discrete, multifaceted and blended implementation actions (blended was defined as the use of a number of discrete approaches, addressing multiple levels and barriers, interwoven and packaged as an implementation intervention with a brand or protocol). The review challenged the notion that there are only a limited number of strategic actions available, but as it was a narrative review no attempt was made to assess the methodological quality of sources or the empirical evidence for the actions listed. The resourcing of these strategies was also left unaddressed by this review.

An empirical study of research funding agencies in a range of countries, including Canada and the UK,11 found that many had minimal resources for knowledge mobilisation. Similarly, studies in academic institutions suggest that the majority lack the infrastructure resources to support knowledge use in policy and practice. 135 Yet, one of the key lessons from cross-sector review is that knowledge mobilisation requires financial, technical, organisational and emotional resources. 93 A widely shared assumption in the literature is that producers, intermediaries and users will invest and cost-share in knowledge mobilisation to the extent that they perceive such investment to be advantageous. 21 Yet, how such shared plans can be negotiated, or the basis on which they are founded, are rarely properly explored. One review of knowledge mobilisation approaches found that where there was no viable cost-sharing mechanism or where most of the burden fell on producers or intermediaries, significant research use (with the exception of political uses) was unlikely. 21

This account of the actions and resources needed for effective knowledge mobilisation has many implications for agencies. It draws attention to the wide array of actions needed, the breadth and diversity of actions available, the complex and vexed issue of resourcing these, and the need for coherent, interlocking and mutually reinforcing actions within and across agencies. It also draws attention to the significant gap between the articulation of a process of knowledge mobilisation (seen in many of the models, theories and frameworks) and the translation of those accounts into workable, practicable, properly resourced strategies. That is, much of the conceptual background reviewed in this chapter does not readily lend itself to the creation of action plans for agencies. At the heart of these difficulties lies an uncertainty about whose role it is to facilitate knowledge mobilisation. In some senses, effective knowledge mobilisation is a system property, and yet individual actors and agencies have to operate independently and are uncertain in co-ordination. Creating the conditions for shared goals, co-investment and co-ordinated actions remains a major challenge.

Context

Knowledge mobilising agencies are usually alert to the potentially facilitating or (more usually) inhibiting effects of the local environment on their efforts. While some see context as a ‘given’ that simply needs to be taken into account (‘context dictates the realm of the possible’),21 for others it is an active ‘ingredient’ in any successful knowledge mobilisation strategy. As Greenhalgh et al. assert: ‘the multiple (and often unpredictable) interactions that arise in particular contexts and settings are precisely what determine the success or failure of a dissemination initiative’. 31

The ‘context is important’ strand in the knowledge mobilisation literature has a long history in organisational research. 151 The processual-contextual perspective (e.g. the content, context and process framework)152,153 has informed a number of studies in the change literature in recent decades. 154 It is reflected in Pawson and Tilley’s well-known ‘CMO’ (context–mechanism–outcome) configuration in realist evaluation155 and in the PARIHS Framework,52 which was developed in part to address the lack of attention to context in earlier models. 72 The importance of context in quality improvement in health care and the key empirical findings from the literature have recently been explored in a publication for the Health Foundation. 151

In relation to influencing policy, a recent review156 also emphasises the importance of an analysis of context and refers to two frameworks: the ‘three Is’ of political science (institutions, interests and ideas) and to the framework proposed by Contandriopoulos et al. ,21 which considers issues in terms of their polarisation, salience and familiarity. Similar frameworks are available when looking at the uptake of policy interventions. For example, the Reach, Effectiveness, Adoption, Implementation, Maintenance (RE-AIM) framework20 emphasises that it is the broader contextual factors that influence adoption, implementation and maintenance and so decision-makers need to balance evidence on effectiveness against these factors.

Analytic approaches to context typically divide it into inner/internal and outer/external, although many authors emphasise that the interaction between these adds to the challenges of assessing and addressing context. Among the aspects of internal context that authors have suggested are relevant to knowledge mobilisation are organisational structures and processes (e.g. the impact of modes of governance on research use, conditions that affect the facilitation and prioritisation of research activity, including incentives and levers, the degree to which service user preferences are accurately known and prioritised); organisational cultures (e.g. the distribution of power between different groups, perspectives on whose responsibility it is to encourage evidence use, current norms and practices, the climate for innovation); and organisational facilities and resources (e.g. time, equipment). In relation to external context, the knowledge mobilisation literature emphasises three key related aspects: the social and political climate/culture; the degree of environmental stability; and the extent of interorganisational communication and norm-setting. In relation to social and political culture, authors have highlighted a range of issues, including the general climate for research production and use (e.g. how those who fund research, universities, researchers and research users support and value knowledge mobilisation efforts) and the influence of policy directives (e.g. the increasing requirement that schools attend to evidence about effective practice as part of continuing professional development leading to an increase in the number of school leaders taking on knowledge broker roles). 127

There is broad agreement, therefore,33,75,157 that context is an important (if poorly understood) mediator. It is a feature of many models of barriers to knowledge uptake,100 and analysis of context is one of the five common components shared by the majority of models of the knowledge mobilisation process. 19 Indeed, a large proportion of the knowledge mobilisation literature is made up of analyses of ‘barriers and facilitators’. 18 For example, there is strong evidence for a wide range of generic barriers to effective research impact, including, for researchers, a lack of resources, a lack of skills and an absence of professional reward for research impact activities and, for research users, competing organisational pressures, an organisational culture that does not value research, a preference for other sources of evidence and a suspicion that research may displace professional skills and experience. 145 In a similar vein, the Cochrane EPOC group classifies barriers to change into nine categories (information management, clinical uncertainty, sense of competence, perceptions of liability, patient expectations, standards of practice, financial disincentives, administrative constraints and a miscellaneous category). 76

Although identifying and addressing key barriers is recommended in many knowledge mobilisation models as an important consideration when choosing a strategy, many important barriers affecting knowledge use (e.g. difficulties arising from working in multiprofessional teams) are long-standing and complex, and are not actually easily addressed. 76 Moreover, this marked emphasis in the literature on understanding barriers and facilitators has been critiqued for leading to a narrow ‘technicist’ understanding of knowledge mobilisation rather than one that is attentive to knowledge mobilisation as an interactive and deeply situated process. 37

In contrast, the contextual approach taken by Ward et al. 37 proposes that a detailed understanding of local interpersonal interactions, shared experiences and networks may be particularly useful in considering how opportunities for knowledge mobilisation emerge or are constrained within an organisational setting. Thus, they see context as playing a dynamic and interactive role with local actions, not simply existing as a passive and inhibiting backdrop. This perspective emphasises the importance of assessing the existing ‘naturalistic’ knowledge exchange processes that are already occurring (e.g. in relation to other innovations or change programmes), and of building on these when planning formal knowledge mobilisation interventions.

It has also been noted that contexts are multiple rather than singular. Levin’s model of knowledge mobilisation75 refers to three types of contexts for the use of research: the context in which it is produced; the context in which it is used; and all of the mediating processes between these two contexts. Emphasis is thus placed on the multiple dynamics at play within each context. Other authors (e.g. Nicolini et al. 158) emphasise the extent to which any one sector (e.g. health care) will have different subsectors within it (e.g. clinical research, health services research, health policy) that may require different approaches to knowledge mobilisation. In that sense, contexts are not just multiple, parallel and perhaps overlapping, but are also nested. Indeed, the ‘complex adaptive systems’ perspective20,27,78 emphasises how the different levels of the system affect each other: interventions at one level are affected by, and affect, factors at other system levels. 35

There are differing views about the extent to which and how contextual factors can be managed or even influenced. Many authors (e.g. Greenhalgh et al. 31) emphasise that while context is important, it is also unpredictable and not easily controlled. Knowledge mobilisation activities are embedded within a system and changes will be sustained only if attention is paid to the factors that influence that system. 64 This rules out simple prescriptions for approaches which will apply in a range of contexts and points to the need to design, tailor, refine and evaluate any knowledge mobilisation approach with reference to the particular setting and alongside those who will be responsible for implementing the changes. 31,64 Advocates for an integrated knowledge mobilisation research approach (i.e. collaboration between researchers and knowledge users) emphasise that research knowledge has to be integrated with contextual knowledge (e.g. population data, local expertise, knowledge of the characteristics of the local setting) and that this integration is more likely to happen if the potential users are involved in the research process from the outset. 159

For agencies seeking to develop knowledge mobilisation strategies, then, a thoroughgoing and realistic evaluation of context remains central. However, while ‘context’ is a key heading in many models and frameworks of the knowledge mobilisation process, it is variably conceptualised and differentially understood. Moreover, there is divergence of view whether context is a passive (usually inhibitory) backdrop or a potentially modifiable and co-optable ‘resource’ for the knowledge mobilisation effort. What is clear is that it is inadequate to treat context as merely a catch-all term for all that is not modelled: such an approach will disguise vital issues such as goal misalignment, power disparities and political practices. As yet, however, tools to assess and disentangle the role of context in knowledge mobilisation are insufficiently developed.

In developing our conceptual map from the review of reviews we were struck by how little attention was given in the knowledge mobilisation literature to the ultimate recipients of public services: service users, their carers and families. It is to this element that we now turn.

The centrality of knowledge mobilisation

We were interested in the extent to which interviewees saw knowledge mobilisation as a key component of the organisation’s work. Although we had selected agencies for interview on the basis that they were major players in knowledge mobilisation (in terms of either scale or the degree of innovation), there were significant variations between agencies. Some funding agencies had at their core the need to encourage the implementation of the research that they funded: ‘the idea that research should be useful is very, very core to everything that we do’ (19).

One interviewee described how their organisation’s mission was to excel in the creation of new knowledge and its translation in order to bring about a stronger and more effective health-care system and improved health for the population. They emphasised that providing the tools to enable researchers and others to move knowledge into practice was a core part of their work: it was not enough merely to create the new knowledge. This affected every stage of the processes they followed in funding research:

So the philosophy being if you’re really going to do this you need to bring end-users and the whole gamut of stakeholders into the discussion even before the application for something goes in. And so we have a number of kinds of integrated knowledge translation programmes.

8

Another agency had a different explicit goal in mind for which encouraging the use of evidence was one of the key mechanisms:

So, if you like, we’re trying to narrow one gap, the attainment gap between rich and poor, by narrowing another, which is the gap between research and practice.

18

Some producer agencies saw knowledge mobilisation activities as an integral – and funded – part of every research project that they conducted, while for other producer agencies, knowledge mobilisation activities were part of some but not all research projects carried out.

Unsurprisingly, given that the agencies had been selected for interview on the basis that they were regarded as being particularly innovative in knowledge mobilisation or as involved in knowledge mobilisation on a large scale and that this had been flagged up as one of the topics that the interviews would cover, most interviewees were able to describe the organisation’s overall approach to knowledge mobilisation. However, some interviewees struggled to define the activities that stemmed from this role:

. . . we are very clear that our purpose in commissioning the research we do and making awards to support research is to try to influence what happens in practice. However, I think we haven’t really found a way to crack that yet.

41

We asked interviewees to describe how they saw their organisation’s role in relation to knowledge mobilisation. These descriptions could be clustered into three broad categories: roles that emphasised products (describing ‘push’ activities and in some cases activities aimed at encouraging ‘pull’);10,53 roles that emphasised brokering; and roles that emphasised fostering implementation. It is important to emphasise that these role categories did overlap within individual organisations: in many cases organisations were doing a combination of activities.

Target audiences

Inevitably, the target audiences of the agencies we interviewed were closely linked to their overall role in relation to knowledge mobilisation and to the structures of their sector. Most commonly for agencies the main target audiences were policy-makers and/or those working to deliver services (e.g. professionals, managers or commissioners). Many agencies had a broad range of groups as their target audience:

Our main stakeholder groups for all of our network activities include policy-makers and government and other places, system planners, researchers, clinicians and service providers, people with lived experience, and family members.

28

For others it was broader still, ‘from bench to bedside and back’, including groups such as basic scientists and even those in other sectors (e.g. agriculture).

Some interviewees commented on how important it was to keep the target audience in mind:

It is very easy, in my experience, for producers of knowledge products to become very caught up in the methods of production and to forget about, or not pay sufficient attention to, who it’s actually for. So we have tried to get much, much better at understanding our audience.

14

For each initiative I think we try to get clear on who we’re really serving.

30

Target audiences were not fixed but could evolve over time; some agencies found that the work they initially did with one group was increasingly used by other groups. Conversely, one social care agency found that the voluntary sector organisations with which the agency initially worked struggled to identify their knowledge gaps. The agency therefore switched to working additionally with local authorities as partners and found that these organisations had a much clearer sense of their own knowledge gaps and could work more effectively with the agency to improve practice within the sector.

It was not uncommon for agencies to start with a defined audience (e.g. researchers and government) and then to broaden this out to other groups:

Now we’re talking to professionals, we’re talking to people who are actually doing the implementation and also other financers, I mean like insurance companies, that want good-quality care.

27

Another agency described how national legislation resulting in major changes to the structure of the health-care system had opened up a range of new local policy-makers and commissioners and of community bodies as part of their audience. Other changes in the UK health and social care sectors that were aimed at bringing health and social care more closely together had impacted on the work of some agencies, who found that they now had larger and more diverse audiences with which they were less familiar:

The integration of health and social care is going to broaden our audience significantly, which is quite a challenge, because then it’s about building new networks and connections, which is where a lot of the energy goes in terms of both promoting our organisation and its work and sharing knowledge and information.

35

Interviewees commented on the need to tailor approaches and products for the different audiences (e.g. service commissioners or frontline practitioners). However, at times, there was some tension between allocating resources and attention to the various target audiences:

We’re trying to do this work with schools and we think that that’s wonderful, but you know, policy-makers have money and some leverage [laughs] and we would like to influence them very much.

7

There were also constraints arising from lack of resources that prevented agencies from differentiating between different sectors of their target audience (e.g. between different health professional groups):

We have thought a long time about segmenting them, there’s a lovely long set of documents about how we describe them to define the audiences [laughs] but that’s largely for market research, because when it comes to producing the guidance we don’t have the resources to tailor the products for those different professional groups.

40

The extent to which research producers worked alongside potential users varied; some agencies did a lot of partnership work in carrying out the research:

The other thing that’s really core to the way we work is we do a lot of partnership work, so we do a lot of close work with external organisations. So all of our research projects would have non-academic partners of some sort on them, either as partners and researchers, or in an advisory capacity . . . we’re trying to embed relevance in the project from the start, and we’re trying to open up communication channels with potential users from the start.

19

Another agency described how involving potential users from the outset made sense not only in terms of increasing the relevance of the research but also in terms of increasing the likelihood that it would be disseminated and used: ‘You’re building a kind of sea of champions’ (33).

Identifying and understanding audiences was a core component of many agencies’ work. However, tensions were discernible as agencies expanded the view of who were relevant audiences and as they tried to cater for the diverse needs of these audiences. Knowing to whom you were trying to transmit messages, or whom you were attempting to broker between, often brought in train a clearer understanding of the challenges rather than simplifying the actions required.

Involving service users and members of the public

We asked interviewees whether the organisation involved service users (e.g. patients, carers, parents, pupils or clients) or members of the public in their knowledge mobilisation activities. Some organisations had a clearly articulated approach to involving service users and/or the public in their wider activities, but only rarely did this involve specific roles for these groups in knowledge mobilisation. For example, some health research funding agencies had a requirement that the patient perspective be included in all research proposals and they had both patient panels and patient representatives on the commissioning bodies that made decisions about research funding. Another funder described how they held around 200 ‘Cafe Scientifique’ public discussion events every year. Other interviewees stated explicitly that their target audiences were groups other than the public or service users (e.g. policy-makers or practitioners):

I’m not sure we have the resources to access that community [the general public] and I’m not sure that’s the best way of achieving impact either somehow.

3

One interviewee commented that the thinking about including service users and members of the public within knowledge mobilisation activities was at an early stage in the sector:

In terms of knowledge mobilisation I don’t think people are necessarily thinking about it from that perspective, about patients or the public being an agent for that, but I think they probably could be.

35

One interviewee from a health research organisation commented that, although their partner organisations were increasingly familiar with involving patients and members of the public in research projects, it was less common to involve them in service improvement projects and so this had been a challenge, not least in identifying groups from which to draw participants. The interviewee also felt that it was important to add to the currently limited evidence base on involving patients and members of the public in implementation work and so they had conducted a literature review and written up some case studies to help other NHS organisations to learn from their work.

Another interviewee from a health research organisation described how she believed strongly that it was important to involve patients in knowledge mobilisation:

. . . there’s a quote in [Crossing the Quality Chasm] about ‘the transfer of knowledge is care’, and I feel there’s something really profound and important about that, that when a practitioner service is caring for a patient it’s not just the tasks and activities and processes of care, it’s how that knowledge that they both have is actually translated into making things better for the person. So I’ve always wanted very much to make stronger links between the resources of knowledge that are designed for patients and the public, and processes of knowledge that are designed for practitioners and professionals.

38

For some the inclusion of service users was less a matter of the ethos of the service and was more driven by pragmatic concerns:

I think the voice that actually helps bring all of the other voices together is the patient. If a basic scientist can say it clearly to a patient then they can say it clearly to a policy-maker and vice versa, and without, I think, having that voice at times, we tend to be too stuck in our own disciplines.

8

One agency commented that, although the inclusion of service users in determining the organisation’s activities was not new, what was new was that service users were now being asked to consider broader service and system issues:

So it really stretches people, and it can be positive or negative because they’re being asked for advice where they have to interpret their lived experience to provide a response, rather than just explain what their experience was.

28

For one health research organisation, their approach to providing patients with information had changed over time: initially they funnelled published information for patients through health professionals, but more recently they had begun to involve patients and their families directly in all aspects of their work, from protocol development to drafting evidence summaries. Another agency described how this side of their work had recently expanded because of the skills of a particular colleague and that it was increasingly becoming a core component:

. . . for us it’s become essential, because I see that the inputs coming out of that are profoundly complementary to what we’re getting out of the stakeholder dialogue . . . it’s almost one of those cases of I can’t believe we thought we could get away without doing this.

36

Another interviewee also commented on how valuable it was to have infrastructure to support activities with service users and members of the public; this agency had a member of staff who had experience working with this client group and who was able to help to equip staff with the necessary skills and to support the user group.

The need to avoid tokenistic involvement was a strong theme of many comments on this issue. One interviewee commented that engaging the public was a stated objective of the organisation but that in practice this was perceived to take second place to other work:

I think, bearing in mind how some of my colleagues have spoken about it, that they think it’s a poor relation to engagement with particularly the business sector.

43

The balance between producing knowledge products for the public and for professionals was raised by an interviewee from another organisation with a dual focus:

If you talk to different groups there is a differing set of opinions about what’s the balance of our focus on a public citizen audience versus a professional audience. We do in fact have a number of key areas of work where we produce pieces that are for both audiences.

42

Interviewees commented that one of the main benefits of having service users or patients so closely involved in the research was that it gave the research greater credibility and impact. Equally, they felt that it was important that both the researchers and service users benefited from their involvement.

In summary, involving service users or members of the public in knowledge mobilisation activities was not a primary focus for the majority of the organisations we interviewed. Although for some organisations (particularly in social care and in mental health services) this was routine, other organisations, while sympathetic to the principle, struggled to develop such activities in the face of competing priorities and in the absence of clear guidance in the literature or individuals with specific skills and experience in this area. Many interviewees were concerned that any such involvement should be meaningful rather than tokenistic and that it should provide benefit to the service users as well as to the research.

Drawing conclusions on agency roles

We commented at the opening of this section that these three role categories (emphasising products, brokering and implementation) were broad and not exclusive: in many cases, agencies were doing a combination of activities that did not fit neatly into one of these categories. There were also evident tensions in the agencies as they sought to balance their activities. In particular, the pull back towards the creation and promulgation of concrete products was palpable, and contrasted with the open-ended uncertainty of pursuing interactional activities and implementation support. What began to emerge as the data analysis proceeded was that agencies exhibited patterns or ‘bundles’ of practice around knowledge mobilisation, and we develop this analysis further in Chapter 6.

Looking across the agencies, we can see that there was differential engagement with different domains of the conceptual map (see Chapter 3). For some agencies their main emphasis was on knowledge products, suggesting a preoccupation with collating and synthesising particular forms of knowledge, while also being alert to the potential users of that knowledge. Other agencies emphasised brokering, suggesting a greater degree of engagement with the issues raised under the ‘connections and configurations’ domain of the conceptual map: what networks, roles or processes were most conducive to bringing diverse groups together around sharing research knowledge? For other agencies, their main emphasis was on implementation, suggesting engagement with particular areas of the ‘purpose and use’ domain and also close attention to the domains around ‘actions and resources’ and ‘people, roles and positions’. Context was an important consideration for all of the agencies, no matter whether their emphasis was on products, brokering or implementation.

Having discovered the rich range of knowledge mobilisation practices in agencies, we now turn to trying to understand how these portfolios of practices have come about.

Innovation in knowledge mobilisation

We asked interviewees to highlight what they saw as the innovative knowledge mobilisation activities in their organisation. This question prompted reflections from some interviewees as to the very nature of innovation in knowledge mobilisation. One interviewee commented that the question raised issues about the state of the field: as there was a lack of established practice in knowledge mobilisation, almost any activity could be deemed to be innovative. Another interviewee commented that ‘innovation’ had become ‘something of a bandwagon’, and that given the broad spread of activity in the knowledge mobilisation field (with some agencies taking a more proactive stance than others) there was a tendency for some organisations to claim as innovative practices which would be regarded by another organisation in the field as established and mainstream: ‘I’m always a bit wary of claiming to be innovative or at the cutting edge, although we do seek to be’ (13).

Someone gave this whole talk about innovation and then talked about putting on an exhibition in the city centre and I was like, hmm, haven’t we been doing that for quite a long time? Is that innovative?

19

Some commented that they perceived that what they were doing was not particularly innovative for the sector or for the knowledge mobilisation field but that it was new for their organisation. One agency commented that they saw as innovative the fact that they were now seeing research as a whole process and wanting to involve different parties in it. Among the measures funding agencies were taking to make research more useful was working with researchers who submitted unsuccessful funding bids to refine their RQs and designs rather than just rejecting the application. They were also broadening the range of research designs that they funded and were encouraging the publication of a wider range of studies, including more observational studies and so-called ‘failed’ studies:

You’re not going to be able to publish, you know, nice observational studies in top journals. And that’s something that we can influence also, as a financer. Also things like paying for publication of research which has failed [to find things] because people can learn a lot from that.

27

This agency was also focusing on de-implementation as part of the process of implementation: ‘OK, if this is new and cost-effective, what do we need to stop?’ (27).

Sometimes interviewees perceived that although the approach itself was not new, they had been able to go further with it than other organisations. For example, one agency commented that they saw their online network as innovative because it was working well, whereas their earlier experience of online networks was that they often struggled to gain initial momentum and to sustain involvement in the medium term. They considered that one of the success factors in their case was that the online network built on an existing ‘people-driven’ network.

Many interviewees commented that they did not think that their organisation was particularly innovative. They recognised that they were still using traditional approaches and needed to encourage more engagement with research knowledge:

So at the minute we kind of just put stuff on our website and, and hope people see it and use it. I think we’re looking to kind of draw attention to that a bit more.

6

The way we work is pretty much still research outputs modified to suit users, and essentially presented to users with a chance of some interaction, but I wouldn’t describe it as more complicated than that.

48

In discussing innovation and knowledge mobilisation, the use of social media was raised fairly frequently by interviewees. Some interviewees were enthusiastic about its potential to facilitate significantly greater spread of research messages (e.g. by alerting potential audiences to research findings):

Of course it’s limiting in terms of getting sophisticated messages across, but you can refer to the full-blown journal articles in your tweet and just have this incredible scale of potential audience, and very quickly. So yeah, I think it’s not going to be the answer in isolation at all, of course not, but it’s definitely a significant part of the mix.

16

Another interviewee described how they were using social media not only to promote research products but also to encourage interaction and feedback on those products:

So instead of just saying, here’s this great document, you know, send it out and use it, it would be, you know, here’s this fantastic document, come speak about it on our collaborative spaces and let us know how you’re using it so we can try to use it as evaluation as well.

33

Others commented that there were challenges in using social media because it was a relatively new mechanism and one about which some researchers and some organisations had concerns.

Overall, although we had purposefully selected agencies on the basis that they might be innovative, we did not find that many agencies were comfortable with this term and we found much ambivalence about whether or not innovation was in fact being achieved. Many respondents played down their organisation’s achievements in knowledge mobilisation and were reluctant to claim the label ‘innovative’.

The influence of models, theories and frameworks

Interviewees were asked about what lay behind their organisation’s approach to knowledge mobilisation, and specifically whether or not and how any specific models, theories or frameworks from the literature informed the approach. The extent to which the knowledge mobilisation literature had influenced organisations in developing their knowledge mobilisation activities was very variable. Responses lay on a spectrum from organisations whose approach was not underpinned by any obvious literature, through to those who were drawing in explicit and detailed ways on published models (Box 4 lists the main sources cited).

Of those organisations which were drawing on the literature, the degree to which they were using it varied from the explicit use of one or more models from the literature to a more general drawing on theories or approaches set out in the literature (e.g. basing the organisation’s approaches on the concept of ‘linkage and exchange’, or on relational approaches to knowledge mobilisation, or loosely drawing on systems theory). Where interviewees named particular authors (rather than models) as having influenced their thinking, the same few names were mentioned (e.g. Weiss, Lavis, Lomas, Best, Davies and Nutley).

Some organisations positioned themselves very explicitly in relation to the literature and had a clear rationale for this choice:

We’ve looked at what seems to make sense to us, but also importantly what makes sense to our NHS partners . . . this was something that they could understand and identify with. So that’s been a key issue in terms of the different frameworks that we’ve selected.

23

One agency had commissioned its own expert review to inform the development of their knowledge mobilisation strategy; the interviewee described how this theoretical underpinning had been helpful in strengthening the agency’s conviction about the approaches that they were developing and had increased the organisational support for that.

Another agency had, in consultation with experts in knowledge exchange and with policy and practice audiences, developed their own model to guide their work:

We needed to have a model that spoke to a broader range of knowledge and who is engaged in driving the process, not solely from the perspective of developing a research project . . .

42

Another organisation described how they had been strongly influenced by the work of Lomas and by Best’s three generations framework. 63 They wanted to challenge the dominant model of knowledge exchange in their sector (public health) and to bring a more relational approach to sharing knowledge:

The theoretical model that dominated everything was that medical model, that linear view of knowledge implementation, you know, knowledge, you produce it, you publish it, the message speaks for itself, people take it up, and if they don’t they’re stupid or lazy or wilful. That kind of very clear message. And, so there were a number of us . . . (who had) those early discussions about how can we run counter to this tide?

24

They emphasised that they were not following the models slavishly and suggested that a cooking metaphor was a good depiction of their approach:

We’ve got in our head how to cook translational cake or how to bake in this style, and we pull it in but not necessarily as a recipe, it is just an underpinning kind of set of skills and orientation.

24

Several organisations described an eclectic approach, drawing on more than one model, or on a range of ideas and approaches: ‘we were quite explicit that we would draw on several models because we didn’t feel that any one model was adequate.’ (1)

We’re very open-minded. But I’m sure some clever philosopher of science could say, well you’re actually very narrow [laughter], but there isn’t one particular approach I think that we’d say would be dominating.

16

Some agencies did not rely on the knowledge mobilisation literature but drew on related literatures instead or in addition. The quality improvement or implementation science fields were used by some agencies as sources of methods and tools for their implementation work (e.g. small cycle learning, statistical process control). Some interviewees had a disciplinary background other than knowledge mobilisation (e.g. organisational theory, cognitive–behavioural science, political science) that informed their work and whose literature they tended to draw on instead:

I guess maybe because my PhD was in political science and I’ve had so much exposure to that . . . it just kind of pervades all of my thinking. You know, I’m constantly thinking about policy legacies or path dependency and I’m thinking about different types of institutional constraints.

36

Other agencies developed initiatives first and then drew on the literature at a later stage. One interviewee described how the organisation’s work had initially developed organically over time and how he had only later drawn on concepts around communities of practice to inform further development.

Some organisations did not look to the knowledge mobilisation literature for inspiration about the work of the agency but instead they tended to find ways to learn from those who were already working successfully in knowledge mobilisation in a range of fields:

It’s more in the people who have been working really productively and in particularly sort of focused spaces, where the real richness is.

26

Another agency described how they drew less on the literature and more on an ‘experience-based knowledge base’: they brought together people with experience in particular areas (e.g. collaboration, scaling up) to learn from their experience and then produced guides to help share this learning with others:

We ask them, OK, what did you do and what would you do again if you had to do it all over again, and what would you advise someone else who has to start doing that kind of thing now?

27

This interview question prompted many comments on the existing literature and on the adequacy of the current models; several interviewees expressed regret that the existing models and frameworks were overly complex, hard to operationalise or limited in scope:

Some of it is quite difficult, the language that’s used, sometimes it’s quite hard to put all of these disparate groups’ thinking together in a cohesive way, and say well what does that actually mean in practice?

35

I think theory is incredibly important and making conceptual distinctions is very important, but I rather felt that a lot of the complicated models with arrows going in every direction haven’t been particularly helpful. So we’ve tried to kind of always go back to basics and say what are the really crude basic building blocks here and what are we looking at.

7

One interviewee suggested that a further problem with models was that basing knowledge mobilisation activities on any one model was likely to prove restrictive, given the likely changes in contextual circumstances, policies and ideas in the sector. Another interviewee went further and expressed a personal preference for ‘getting things done’:

I have a bias here, I just want to get things done and I have very limited patience for elaborate theoretical frameworks. That’s just my own particular bias and it reflects my background in policy and in administration as opposed to academia perhaps.

15

Others commented that the evidence base was largely around facilitating change at a local level and that there was very little in the literature that could help national agencies to determine their approach. There were also comments that the challenges of knowledge mobilisation in the health, social care and education sectors made it hard to apply simple approaches from other fields:

Now that’s not to say that we can’t learn from how people do their marketing strategy and get people to buy stuff, but it’s just that it’s slightly more complicated than that, I think.

6

Those interviewees who said that their organisations were not using the literature explicitly typically said that knowledge mobilisation theories and frameworks were not in the forefront of individuals’ minds on a day-to-day basis, although specific theories of change might be considered in relation to specific projects. We observed some gaps between ‘espoused theory’ and ‘theory in use’. 174 For example, in some cases models and tools that were fairly prominent on an agency’s website were not referred to by the interviewee, suggesting that these were not at the forefront of the work of that individual or organisation. Conversely, some organisations were implicitly using theories but were not doing so explicitly. For example, one interviewee commented: ‘I think we’re shamelessly atheoretical [laughter]. It’s a very pragmatic environment’ (46); yet another interviewee from the same organisation described how their knowledge mobilisation approaches were designed to use peer pressure between organisations to encourage evidence use: an approach that implicitly drew on existing theories of organisational change.

In summary, some organisations were drawing on models, theories and frameworks from the knowledge mobilisation and/or other literatures (in some cases following an extensive literature review and consultation with experts in the field). Others had developed their own models and tools either from scratch or as adaptations of published work. Use of several models in order to combine the most appropriate features of each was common. No one model or set of models appeared to dominate. We observed a fair degree of frustration with the limitations of the existing models, theories and frameworks, which many interviewees perceived as overly complex and hard to operationalise. It was, therefore, not surprising to find that some organisations had developed their knowledge mobilisation approaches somewhat independently of the published literature, taking a more pragmatic approach.

Strategic and pragmatic considerations

More strategic and pragmatic considerations (rather than theoretical thinking) underpinned much of the knowledge mobilisation activities discussed in the interviews.

We come together and we say, all right, let’s look at the data, what is it going to take, what do we know, and then let’s talk about what we’re hearing in the field, what are people struggling with, and then design a strategy to meet that.

30

Early development was often described as fairly pragmatic, with greater clarity and the development of a more focused strategy emerging over time:

As the projects have developed it’s become easier because we know we’re trying to achieve something that’s . . . become . . . clearly defined as we’ve gone along.

2

Realistic assessment of the potential for change was also a driver. For one agency, being strategic meant choosing to do research only on policy areas that could feasibly be implemented in a reasonable time frame, and not focusing on those that were too far outside the prevailing discourse and policy direction.

However, organisations also commented that a focused and strategic approach did not rule out the need for adaptability:

That’s been the whole narrative of our evolution as a group, that we keep thinking that, oh well once we add this programme, we’ll be sorted out, you know, once we have [name of programme], surely that will help solve the problem, or once we have briefs and dialogues going alongside [name of programme], well that will be enough. And then over time we just realise that there’s so many pieces that need to be in . . . I sometimes call it a healthy eco-system for using evidence in decision-making.

36

Although some organisations described strategic approaches to developing knowledge mobilisation activities, the development of knowledge mobilisation often appeared as a combination of strategy and serendipity (emergent approaches, and ‘muddling through’):

The approaches we take – none of them are as we envisaged, they’ve all morphed in relation to need and contextual changes, and have evolved over time.

24

Thus, for many organisations, the development of their knowledge mobilisation activities arose from a combination of pragmatism and other factors rather than from strategic vision alone.

The influence of funders and other powerful stakeholders

A key driver for many agencies that affected the content or clinical focus of programmes undertaken or the nature of the research that would be shared included political pressure from government or national bodies, changes in external reporting requirements for stakeholders (e.g. around readmissions or medication use) or research evidence ‘horizon scanning’ by leaders within the organisation. The remit that their funder or governing body gave them was also a major driver for many agencies, particularly in the case of agencies funded by governments. They recognised that their work was often driven by high profile policy initiatives and that this gave them less control over their overall direction than some other organisations (e.g. universities):

There is also a very clear agenda from government . . . research funders have got to get into the space of knowledge mobilisation.

43

I would say that there has been a pressure or a push towards prioritising projects that typically go into decisions for money, or to important policy decisions.

45

Key individuals too had often played a strong role in shaping an organisation’s approach to knowledge mobilisation. For example, one interviewee described the influence of the organisation’s founding director:

He had a vision and he was able to communicate that vision to the funders, but also to other people working in the centre, about why these things were important, and that we shouldn’t just be ploughing on, we should be thinking about how we want our research to have an impact.

25

Another agency had been heavily influenced by the prior experience that key individuals had gained working in other national and local organisations involved in practice change.

In sum, funders (e.g. governments) or other powerful stakeholders outside the organisation or senior figures within it played a key part in influencing the content and nature of knowledge mobilisation activities for many of the agencies we interviewed.

Changing conditions in the sector

Recognition of changing conditions in their sector was another key driver for many organisations. For example, one agency described how they had considered that the types of ‘wicked’ problems that the public sector was facing in a difficult economic context meant that a social innovation approach was more likely to succeed than more traditional approaches:

We need to put our limited resources into things that might work, and test and learn and experiment . . . There’s a hunger to do things differently, to learn from the past or from the present, but learning quickly, so if you fail, fail fast and fail low cost . . . You need to just keep having new stabs at it.

16

One agency had been conceived at a time when funding was relatively generous and it was possible to have a vision of ambitious programmes with a substantial evaluation component built in. By the time the agency came to fruition it had become necessary to focus on what could be done internally and more cheaply. Several interviewees referred to responding to the economic downturn as a factor affecting the development of their knowledge mobilisation activities. One agency noted that, although this was not a prime driver for them, they were aware that economic conditions in the sector meant that there were fewer networking opportunities for practitioners. This meant that the agency’s role in encouraging communication through its own network was becoming increasingly important. Another agency responded to emerging pressures on media organisations: they recognised that the media was an important source of information on health policy and that many media organisations were cutting back because of economic constraints and no longer had the expertise to cover health policy stories adequately. The agency, therefore, set up a news service to ensure broad coverage of health policy issues and their implications.

Changes in the national political environment had also prompted changes in knowledge mobilisation activities: as a result of devolution, organisations in Scotland and Wales had gained new policy audiences for research evidence. Changes in the structure of the sector (e.g. new organisations in the field) led to changes in some organisations’ approaches over time. For example, for some health-care organisations in the UK there was a need to consider the role of the new Academic Health Science Networks and how best to work with them in knowledge mobilisation. Organisations in the social care sector in the UK were increasingly considering how best to link their knowledge mobilisation activities up with health organisations (and vice versa) in response to sector-wide developments aimed at aligning health and social care provision more closely.

Agencies had also been affected by changes in the university environment and in university ‘drivers’. Several interviewees, from a range of countries, commented that ‘impact’ had become a stronger focus for the organisation in the past couple of years:

I think the university’s thinking has changed around what matters. When I first started, I went to a seminar where some esteemed academic said ‘Well if you’re going to get money these days you’ve got to say you’re going to disseminate your research’ [laughter] in a very kind of cynical way which you just would not hear now. So the world has changed so much over that 10 years.

19

One agency noted that engagement outside academia was increasingly being accepted as the norm and that academics were increasingly aware that they needed to do that. This had led to growing interest in what the agency could provide to support academics in fulfilling that expectation.

Thus, changing conditions in their sector (e.g. new challenges arising from economic constraints or political developments; a growing discourse about impact) had prompted many of the organisations we interviewed to alter their knowledge mobilisation approaches in several ways: changing the focus or content of their activities, providing new services or programmes to address emerging deficits, or widening their target audiences to embrace new stakeholders.

Doing things differently because of earlier experience or evaluations

Some organisations made changes in their knowledge mobilisation approaches in the light of experience or evaluations. Several agencies described how in the early days of the organisation they had tried one knowledge mobilisation approach, but they were now doing something different, learning in part from experience. One example of such a shift was given by an agency that described how they were now considering implementation issues right from the start of the research process:

It was like: OK, at the end of research we think about implementation. And now we’re thinking about implementation from the beginning, like what are the knowledge needs of these target groups, what kind of questions do they have, what kind of research can you do to answer these questions then?

27

Interviewees gave examples of how early experience had led to new initiatives or to changes in approach. One organisation described how earlier ‘ad hoc’ initiatives around knowledge translation (e.g. involving the user from the outset) led to the development of a long-term user network to ‘avoid having to start from scratch’ each time. Another agency described how they were using knowledge broker roles differently: they had initially used volunteers as knowledge brokers, before realising that the role would be stronger and more sustainable if they had paid employees in that role. Another agency described how they now had people in these roles as part of a wider team rather than as individuals, and that these individuals had a knowledge brokering function alongside a wider project management role. Others spoke of learning the importance of securing both clinical and managerial support to enable brokering teams to work effectively:

You have to show that this is an organisational priority and that the team going through has the support of their leadership at the highest level to get this work done.

5

Change or growth in the agency itself was another driver for change in the knowledge mobilisation approach. One agency that had initially seen its role as one of an independent body assessing research or practice shifted to a more collaborative approach of looking to support practice; in addition, the agency expanded its remit to include related areas (education, health) affecting its target audience. Another, which had originally envisaged being a ‘light touch’ hub with around 500 contacts, found that the interest attracted was considerably greater and that the number of contacts was three times that envisaged. This meant that there were new challenges in ensuring effective co-ordination and avoiding duplication between parts of the network. Another agency had a sense that the earlier activities that they had carried out around effective co-production and knowledge exchange were now becoming more mainstream and, therefore, they were moving to consider evidence use in practice more closely.

Feedback from stakeholders was also a driver for change in knowledge mobilisation approaches in some organisations. One agency received feedback that health professionals saw the organisation as an ‘ivory tower’ and realised that they needed to be clearer both about the organisation’s role and about how research evidence sat alongside other forms of knowledge including practitioner knowledge and patient preferences. Another organisation was alerted to new implementation challenges facing their health professional audience:

We were beginning to get feedback saying how are we supposed to manage this, what should we do with the other? So it was much more in response to feedback from users and the organisations we were sending the guidance to, rather than government or national bodies.

40

Some agencies drew on formal evaluations to guide the development of their approaches. One agency had had their work reviewed by an international panel; this had resulted in recommendations and additional resources aimed at increasing the agency’s knowledge mobilisation work.

Another interviewee described how the focus of the agency’s work had changed from an emphasis on trying to influence national legislative change to an emphasis on working through local policy-makers and practitioners. The driving force for this had been an evaluation of the effect of a successful publication:

Everybody liked it, it was all wonderful. And then when we did the evaluation of the dissemination we found that actually it didn’t really change anybody’s behaviour . . . It was successful because it was speaking to what people were doing before . . . So I wanted to achieve change and we thought that if we could do that, it would be local.

11

Feasibility research was used by one agency to explore demand and to enable broad scoping of the initiative’s parameters before developing a major programme:

We tested how journalists and the press were currently using evidence to inform their work in educational reporting. We found out that it was quite haphazard and it wasn’t particularly systematic in any way. So there was a place for a brokerage type of service that could help connect research and researchers to the press and the media.

2

We observed that, in general, organisations were more likely to draw on their own experience or evaluations rather than on those of other organisations. When asked if there were other organisations that the agency looked to in developing knowledge mobilisation approaches, several agencies commented that they did not look to other organisations as much as they believed they ought to:

I think we’re probably a bit too insular with probably not a very wide knowledge of what other organisations are doing.

41

However, other agencies did describe learning from the cross-organisational work they did with a local research centre or copying some approaches (e.g. the use of social media) because they had become an organisational norm:

Somebody suggested we use Facebook and Twitter. That’s not based on the research on theories either, it’s what all agencies with self-respect do.

45

Several agencies described being influenced by the work of Canadian organisations (e.g. the Canadian Health Services Research Foundation), which were perceived to have been in the vanguard of knowledge mobilisation:

That’s why we have quite a strong emphasis in our work on knowledge brokering, because that was something that they had quite a bit of debate around at that time.

44

Learning from earlier experience then, or in some cases from more formal evaluations, was one of the factors that helped to shape agencies’ knowledge mobilisation activities. In general, this meant learning from their own work and experiences (of which more shortly): few organisations we interviewed looked for learning in any systematic way to other organisations.

Drawing conclusions on how knowledge mobilisation develops

Looking across the organisations we interviewed, we found that a range of factors had contributed to shaping the knowledge mobilisation approaches in use. Agencies were building on local experience and tacit knowledge; internal evaluations; personal inclinations and capabilities; and the interests of key individuals in the organisation. Methodological developments in the field also played a part in some agencies, although often in an indirect or diffuse kind of way. The influence of funders or other powerful stakeholders and the changing conditions in the sector were also significant drivers. We were interested to note that while many organisations drew on earlier experience or on more formal evaluations in designing and carrying out their knowledge mobilisation activities, few had systematic mechanisms for learning from other organisations. The next section explores in more detail what agencies had learnt through their evaluations and from their practical experience.

How agencies talked about evaluation

Interviewee organisations fell into two broad categories: those that described a range of evaluation activities and those organisations that were somewhat apologetic in describing what they saw as only limited activity in this area. Common to both groups were an expressed belief in the importance of evaluating knowledge mobilisation activities, a perception that robust evaluation posed considerable challenges and an aspiration to do more evaluation in future.

Many agencies emphasised that they saw evaluation as an important part of their knowledge mobilisation activities and some had sought additional funding to be able to evaluate the knowledge mobilisation side of their work. Some reflected on the importance of evaluation both to inform future work and also to be seen to ‘practise what we preach’:

We do it for ourselves, so (a) everything we ever do we debrief, and then (b) we debrief lines of activity.

17

We’ve set up an evaluative framework that measures the [programme] itself as a vehicle for taking evidence and getting it into action and creating change, and the extent to which the [programme] was or wasn’t in and of itself an enabler or a barrier to getting things done . . . we’ll reflect on that before we do another one.

5

Despite strong recognition of the need for evaluation, not all agencies felt able to rise to the challenge. Some acknowledged that evaluating their own activities was not something that they had yet been able to do, perhaps because of resource constraints, the relative recency of activities in this area or the rapidity of change. Some organisations evaluated only their knowledge products; others were evaluating broader activities but wanted to make their evaluations more detailed or nuanced.

Many agencies relied on informal measures of how well the approach was being received by the service users and other stakeholders or on broad measures like the renewal of contracts by clients. When asked how they knew how well they were doing, one agency responded: ‘Well – [pause] – we don’t really know, is the proper answer to that. We only know in the sense that most of our work is commissioned and we know whether the funder is happy with what we’ve achieved’ (7).

A degree of hesitancy and apology was, therefore, often expressed around the issue of evaluation in knowledge mobilisation:

I think most of my colleagues would agree that this is an under-appreciated and under-studied area [by us], despite the fact that we have a specific programme whose mission is these very issues . . . There’s relatively little formal, proper, rigorous, independent and objective evaluation of the sort that should be conducted.

37

Of course, some of the agencies interviewed were more thoroughly active in evaluation work, for example having a combination of internal and external evaluation built into each project or programme: ‘you must be constantly thinking, always changing’ (4). Often these evaluations used a mixture of formal approaches (e.g. specific evaluations of individual programmes) and more informal approaches (e.g. collecting impact stories, taking informal soundings from knowledge brokers). These evaluations fed into their future strategy and, they reported, often resulted in significant changes to programmes.

Some organisations sought to track whether or not the information they put out was being used by policy-makers, for example in government hearings on a policy issue or in the research and policy papers published by government agencies. One agency described how they evaluated their work through a range of informal and formal, continuous and one-off mechanisms including regular reporting to the government department about impact on policy, and evaluation with users involved in their knowledge broker services:

We kind of do a structured process about every 2 or 3 years, but every review or other kind of broker experience we interview the person 6 months after the review’s all finished, to explore whether it met their needs and what use they’ve made of the product.

44

Through these evaluations they had learnt of the mismatch in perspectives between researchers and policy-makers about the role that research played in driving policy:

We talked mainly to researchers, and essentially researchers felt that they played a big role in agenda setting, and policy-makers really don’t feel that; they feel that agendas are set by the community and by politicians.

44

For some agencies, although evaluation (e.g. providing impact case studies) was mandated by their funder or was applied when programmes had a large amount of resources invested in them, it was also seen as an important mechanism for learning and for planning changes in direction:

Some of it is mandated by [those] that give us our money, but much more concretely we are doing it to really use our own data to drive programme change.

8

So everything that we set up, we think of as both trying to contribute to better decision-making, but also as a laboratory to try different stuff out. For everything, we try and subject it to some kind of an empirical evaluation.

13

Many agencies collected only basic metrics (e.g. website activity, seminar attendance) but interviewees recognised the limitations of such measures:

It’s very easy for us to produce annual reports saying 80% of people who came to our events said they were good or very good, but that’s not going to be really telling you much more than most people would expect really, given the quality of the speakers and such.

12

We already know how many people access our web pages to look at a particular piece of research, for example. What we surely ought to know, which is the killer question, is what do they then do as a result of doing that and is it actually doing anything to their practice?

6

Other organisations had more sophisticated approaches. For example, one agency was using a contribution analysis approach to tease out what their particular contribution was:

What we’re really interested in is that longer-term impact and the contribution which we may make alongside the morass of other activity that’s going on at any one time.

13

Another we interviewed had an evaluation framework that was developed as part of the externally commissioned review that led to their knowledge mobilisation strategy. This aimed to assess different levels of impact: outputs; level of reach; changes in knowledge, skills, attitudes and behaviours; measurable change in practice or decisions; and measurable change in health and social services outcomes.

Some interviewees recognised that there might be advantages in making their informal internal learning more widely available; however, there were concerns around the resources needed to collate and write up this material, about the ethics of publishing material which had been collected from individuals for other purposes and about the risk of ‘overselling’ learning from a small organisation. Several organisations described aspirations to do more evaluation of their knowledge mobilisation activities in future.

The challenges of evaluation

A common theme expressed both by interviewees from organisations that did seek to evaluate their knowledge mobilisation activities and those who struggled to do so was the challenges of evaluation. A central concern and a reason given by many interviewees for the limited amount of evaluation of knowledge mobilisation activities conducted by their organisation was the methodological difficulties of conducting robust evaluations: the challenges of assessing impact; the difficulties in teasing out the contribution of different strands of activity or mechanisms in a busy sector; and the desire to minimise the reporting burden that they placed on busy stakeholders:

Obviously we want to promote evaluation, but it’s just got to be proportionate and we’re just too small to do anything that meaningful.

16

It’s really hard to measure impact . . . Have we had an impact on [specific outcome]? Well I would suspect we could say, very confidently, no, in a kind of linear way. Have we had an impact on the learning culture of our partner agencies? I would say very much so. But how would you evaluate that and measure that?

12

Others commented on the need for flexibility around the measures used to assess progress in knowledge mobilisation activities:

Sometimes the measures change as well, you know, sometimes we think we want to be looking at numbers signed up, or we’ve realised it’s more important if they’re engaged, and it doesn’t matter how many sign up.

30

It was also seen to be important to consider carefully what programmes were aiming to do and to assess the results against those objectives: ‘. . .we were ending up in the position where we would do an evaluation of a programme and you realise you can’t really evaluate it because you don’t know what it was meant to do’ (34).

The challenges of assessing and discerning impact meant that several interviewees described how, despite their personal beliefs or the beliefs espoused by the organisation about the need for broader or more robust measures, it was often simplest to focus on process measures or on simple metrics (e.g. website use, downloads of publications or attendee satisfaction at events):

We’ve always struggled, from the policy side, to understand the impact that we want to have or measuring whether we do have it or not. So I think, by default, we tend to defer to those more sort of process type things.

39

Overall, agencies usually recognised the importance of evaluation but were really struggling to turn that recognition into practical action. With a few key exceptions, the challenges of delivering wide-ranging and convincing evaluations meant that agencies often focused on short-term process measures and subjective – often anecdotal – reports.

Substantive evaluations

Throughout the desk research and throughout the interviews we specifically sought to identify any substantive formal evaluations of knowledge mobilisation activities (either by the agencies themselves or by external bodies). We looked especially for evaluations that had led to written reports that were publicly available or could be made available to the research team. The combination of website review and specific questioning of interviewees, combined with additional e-mail and telephone contact resulted in the collation of 18 published evaluation studies produced by or on behalf of seven of the agencies in our study. Table 5 sets out the details of these studies: the evaluation methods; the criteria and measures of success; the logic model or theory of change; the knowledge mobilisation mechanisms tested; any contextual evaluation; and the main findings. The studies are set out alphabetically by name of organisation as follows:

• Canadian Institutes of Health Research (CIHR)

• Centre for Effective Services (CES)

• Centre for Research on Families and Relationships (CRFR) (four evaluations)

• Economic and Social Research Council (three evaluations)

• Michael Smith Foundation for Health Research (MSFHR) (three evaluations)

• Social Care Institute for Excellence (three evaluations)

• ZonMw (The Netherlands Organisation for Health Research and Development) (three evaluations).

Formative learning/practical experience

We asked interviewees what they had learned about knowledge mobilisation from practical experience in addition to, or alongside, more formal evaluations. Some of the insights offered related to domains identified from the literature and represented in the conceptual map (see Chapter 3), most often the domains relating to context, roles and connections.

The strongest theme from the interview data on learning from practical experience was observations on context. These included generic observations: the importance of attending to the research users’ context and challenges, and tailoring the knowledge mobilisation approach accordingly; the need to use local contextual knowledge and to take advice on what ‘levers’ to pull; and the importance of ‘riding’ policy waves and capitalising on other political developments:

What we have done therefore is work on some of the NICE quality standards. Some of it’s been linked to CQUINs [Commissioning for Quality and Innovation payments], but some of it has been additional to CQUINs, and with hindsight has enabled our organisations to get ahead of the game. So it’s about understanding what the national drivers are, what the pressures are on an organisation, as an example, and capitalising on those.

23

Interviewees also made specific observations about the health-care, education and social care sectors in which they worked. One interviewee commented on how they had learnt much about how to do implementation in the very turbulent context of the current NHS and how this required persistence and adaptability:

The Trust wanted us to use a particular facilitative model of quality improvement teams who would be ward-based and who would lead on the implementation of national quality standards, to get the evidence into practice. And this had been agreed at various levels of the organisation and written into our proposal, and almost at the point we were ready to start, the person in the Trust who was responsible for facilitating the QIPP [Quality, Innovation, Productivity and Prevention] teams was out of a job, and the pressures on the wards were such that they couldn’t take forward this initiative. So we very quickly changed to a best practice champion model and drew upon the research on that.

23

The challenges of working in the NHS context also included the onerous approval process in some organisations, delays in granting honorary contracts to enable secondments and pressures of workload in the NHS which made it harder to release staff to work with other organisations on research or knowledge mobilisation. Taking account of such challenges was seen as essential in developing effective knowledge mobilisation strategies.

One of the most pervasive challenges across sectors was the stamina needed to make the case for research application and implementation:

I think one’s got to just be brave about that, you know . . . keep working at it, keep struggling with it, keep playing with the language and the different ways of describing what we do, and keep working at persuading people why it’s important. You have to have a bit of a passion for it, I think. The best folks in this field are like that . . . they have a real commitment to passing that on to other people because they feel it is worthwhile and valuable and useful.

21

Part of that process of persuasion meant in the early days ‘going the extra mile’ to build up a reputation and to persuade people of the value of the work. It also required sustained humility on the part of agencies: not claiming to know all of the answers but instead working in an atmosphere of support and co-operation.

Another common challenge was seen as arising from the constant ‘redisorganisation’195 of all three sectors, with increasing fragmentation, shifting relationships between central and local government and the introduction of legislative and policy changes that could have negative or (sometimes) positive impacts on encouraging research use. For example, one interviewee described a recent growth in interest in the use of evidence in the education sector in the UK: ‘There’s been a huge shift I think. Things like when the Cabinet Office got interested, that created more high-level political interest . . . particularly in the last 18 months [there have been] a lot more of these networks and groups and coalitions coming together’ (2). Waiting for, and responding to, propitious circumstances was seen as crucial.

However, in the public health sector in England, many developments were perceived to have been disadvantageous. Significant changes had been made in the public health infrastructures, with public health moving from the NHS to local authorities and with subsequent changes to the role of Directors of Public Health. One interviewee commented on how it was challenging to advocate for evidence-informed policy in this new environment where elected local council members had considerably greater power than senior public health employees:

There have been lots of conversations about how, in the light of that, do you package, present, shape research, to influence those who actually now take the decisions. And the culture of the NHS is basically sort of built on expertise and understands it. It doesn’t always follow it but it understands evidence-based or evidence-informed practice. [Now it’s] ‘Oh where was that done? That would never work here. Not with my constituents. I’ve lived here for years, and I know what would work.’

24

There were also issues for public health initiatives around matched funding for knowledge translation initiatives: it was now local authorities rather than health authorities who would need to provide matched funding for these projects, and some were unable to do that.

It was not just the service context that made knowledge mobilisation challenging but also the university context. One interviewee described how the different histories and missions of ‘old’ and ‘new’ universities made it difficult to form knowledge mobilisation collaborations across different institutions:

Some of the universities are about excellence in research and research outputs and grant and income generation, and others are more about being in and of their community and responsive to their community needs. So they have very different missions and visions.

24

The university infrastructures and reward systems also formed a barrier to translational research: ‘. . . what researchers and academics are rewarded for is not the kind of work you might engage in if you wanted to do knowledge exchange, translational research’ (24).

In contrast, a more optimistic vision was given by another organisation:

I think people are really deeply sort of values-driven and very committed. Some of them I think are beyond redemption in terms of just absolutely wanting to squirrel themselves away and just research for the sake of research and just talk to their peers, and they don’t give a monkey’s about anything else. But I think there’s a much greater sort of proportion, probably working with younger researchers who are just so creative and enthusiastic about how they can be an academic but be really connected to the world as well.

26

In addition to talking about their own sectors and about universities, interviewees made comments about the state of knowledge mobilisation as a field. Some commented on positive developments, observing that there was increasing recognition from policy-makers and from research funders that producing good evidence and facilitating its implementation required time and adequate resources. Others felt that there was still much progress to be made and some suggested that the knowledge mobilisation field itself had stalled:

There’s still a sort of disseminate and hope attitude. It’s surprising how many funders are really unimaginative about this. They’ve picked up that you need an executive summary and you’ll probably want to put it on a website and you might want to give a seminar or two. But thinking more creatively about [it] . . .

21

Some sensed that there was perhaps creeping disillusionment among some policy-makers and practitioners that researchers and universities did not provide ready answers. One interviewee suggested that it was important to learn from other contexts like international development or community development that faced similar challenges. It often seemed that respondents were concerned to keep learning and changing as a way of keeping up momentum for their knowledge mobilisation ‘mission’.

Another strong theme from practical learning concerned issues around people and relationships. Several interviewees emphasised the importance of using the ‘right people in the right role’ for knowledge mobilisation activities. Some went further in suggesting that it was most productive not to ‘waste time’ with researchers or practitioners who had no appetite or aptitude for such work. Rather than forcing people into knowledge mobilisation activities or assuming that practitioners with ‘likely’ job titles would be the most engaged, it was felt to be more realistic and fruitful to assess who had the skills, personality, experience and enthusiasm for this challenging work:

You must have the right people in place: people who can cope with ambiguity. People who can work across professional boundaries . . . If the person requires a set protocol of how to get things done, they aren’t the right person for this work.

4

I think that this kind of work is really tricky, I think it’s the kind of work that needs to be done by people who are already pretty experienced, pretty knowledgeable, pretty confident, about their own professional skills, because you’re always working right on the edge of your comfort zone . . . and you just can’t expect very young and inexperienced people to be able to engage confidently with the kind of challenges practitioners or senior policy-makers have.

21

Respondents had also learned that it was important to value different kinds of roles and expertise that people could bring to engagement work, and to recognise that some of these people might not label themselves as knowledge mobilisers:

That’s my incredibly simplistic conclusion that people who are interested in how other people make meaning and how you negotiate meaning and who have real emotional intelligence, but also have a really strategic brain, they do amazing work, and they’ve never thought of themselves as knowledge mobilisers.

26

It was also seen as important for long-term sustainability of initiatives to have people in paid posts and not to rely solely on goodwill:

You can only get so much for free really, I mean you can only get so much from pro bono and voluntary inputs and the movement, the real momentum, has come when we’ve got some money to be able to employ people to get more money to get these projects off the ground.

2

Interviewees emphasised the importance of engaging potential research users. This required being led by the agenda of those users and recognising that the research evidence that you wanted to share was only ever one part of the knowledge landscape for those users:

You’re always the broker, you’re never the leader. So if you’re a knowledge mobiliser you’re always enabling rather than shoving stuff to people.

48

One interviewee from an organisation that produced evidence-based guidelines endorsed this position: it was necessary to start from the end point of what managers and practitioners were trying to accomplish, and then using guidelines and associated social networks to help them to achieve that. Making the guidelines an end in themselves would not work.

Congruent with both the formal evaluations done by agencies and much of the literature, several interviewees commented on the benefits of co-producing research with policy-makers, practitioners or service users. For example, in one agency, meeting service users and parents regularly on the research consultation panels had proved a valuable way to learn about other issues that they were facing that might not otherwise have come to the researchers’ attention: this process helped researchers to identify possible new areas for future research. However, interviewees also emphasised the considerable challenges that co-production posed. These approaches required commitment, persistence and sufficient resources:

If there’s one thing I’ve learnt over the last couple of years, it’s the vital importance of doing this alongside the people who are using the information, and that is expensive and it’s hard work and it’s time-consuming to do properly rather than tokenistically . . . so there’s some messages in there for funders about the real costs, and that you can’t do this half-heartedly. It’s pointless unless you’re really prepared to do it properly.

21

Interviewees emphasised that it was not sufficient to bring a researcher and policy-maker together and expect that to produce changes in how health research evidence was used; considerable hands-on support was needed to make these connections fruitful and to address the challenges: ‘. . . there is way more at stake, there’s personal relationships, there’s how do you work as a team, there’s the reality of each other’s world and so on’ (34).

Nurturing networks thus also emerged as an important consideration given attention by many agencies. In particular, they saw the importance of having sufficient funds to allow development and evaluation over a realistic time period:

You need to build it and have activity before you open the door, you can’t launch something that’s got tumbleweed drifting [laughs]. And it’s like a restaurant, you’re more likely to go in if there are already diners.

28

A further piece of advice that interviewees gave was the importance, where possible, of building up long-term relationships with policy-makers and practitioners. Although this was difficult to achieve in political and service contexts with rapid staff turnover, it did bring considerable advantages in enabling trust to develop and making broader, more meaningful and frank discussions possible:

I’m still an outsider of course, but because I’ve known them for such a long time now . . . there’s much, much more honesty going on, so there’s a trust situation being established, and so we learn now much more about the background to a particular policy question than they probably would have shared about 6 years ago. And it’s actually quite useful to know that . . . because then it informs how you shape your work essentially.

3

I think building those relationships around, you know, multiyear research projects is probably important, but not because of how research from those partnerships instrumentally informs decision-making, but how, over time, relationships are built up that allow policy-makers and stakeholders to efficiently access researchers who can then open up a whole lot of research to them on other issues.

36

Finally, so much of knowledge mobilisation work involves working with and through others that getting buy-in was seen as essential. This needed to be real commitment, often evidenced by cash investment:

We truly believe that the organisations need to have some skin in the game, so to speak: that they should be contributing. Our philosophy is that everybody has to contribute financially. These are collaborations, both in kind but also in real dollar terms.

5

Taken together, these observations suggest that agencies have clearly benefited from extensive experiential learning and now have much tacit knowledge that can, given prompting, be made explicit. Given this, it seems unfortunate that there is so little capitalising on the potential for interagency learning as agencies develop their options.

Drawing conclusions on agency learning

Agencies demonstrate a keen awareness of the need for learning from diverse sources, focusing on their own experiential learning and (nascently) from more formal evaluation work:

Find the point on the horizon, the North Star or whatever, depending on how much you want to crane your neck, and where do you wanna go . . . And then think about how that translates to building the road to get there and what the bricks need to be, and also what the course might look like, because it’s not gonna be a straight line.

9

While formal evaluation work has often proved difficult to mount and sustain, some clear messages are emerging that chime with the messages from the more conceptual literature. In addition, there is now rich formative experience that agencies have developed through experience that has the capacity to shape new initiatives and to enable the avoidance of pitfalls in the development of new strategies.

‘Push’ activities

Of the nine different types of ‘push’ activities shown in Table 7, agencies in the survey were most likely to produce publications, other written materials and tools, digested research summaries or guidelines, or to provide ‘rapid response’ research synthesis services (at least 80% responded ‘often’ or ‘sometimes’ to these items). Fewer than one-third of respondents often used social media to create debate, although over one-third sometimes did. One-quarter of respondents often used social marketing approaches to communicate research findings, change ideas or promote evidence-based change. Around half of respondents provided research-based commentary in response to issues in the news, although only about 10% did so often. Around two-thirds of respondents provided webinars for practitioners and policy-makers or planned to do so in future. Using the arts (e.g. drama, music, narrative or visual arts) to communicate research findings was rare, although around one-third of respondents sometimes produced videos or animations.

‘Pull’ activities

Facilitating the implementation of research findings in practice or policy settings was the most common ‘pull’ activity identified by agencies in the survey (93% sometimes or often did this), along with developing local collaborations for innovation and improvement (78% sometimes or often) (Table 8). Providing training to build research awareness or critical appraisal skills among practitioners or policy-makers was more common than providing input into pre- or post-registration training for practitioners (77% vs. 44%). Over half of the respondent agencies (58%) used participatory research methods (e.g. action research), although fewer than one-quarter did so often (22%). It was more common to provide local consultancy services on policy or practice issues (64%), although one-third of agencies never did so.

Linkage and exchange

Of the eight activities we have categorised in this analysis as ‘linkage and exchange’, the activities that were performed most often were organising events that bring researchers together with policy-makers and practitioners (97% often or sometimes); facilitating mixed networks of researchers, practitioners and policy-makers (89%); and brokering relationships between these groups (92%) (Table 9). It was less common to broker relationships between researchers and journalists: one-third of organisations never did this and only 15% did so often. It was also less common to arrange secondments from or into the organisation, with 51% and 35% of respondents, respectively, saying that this never happened. Around two-thirds of organisations employed staff in dedicated intermediary roles (e.g. as knowledge brokers) either often (41%) or sometimes (26%). Fostering formal partnerships between university departments and non-university organisations was part of knowledge mobilisation activity for around four-fifths of organisations, with around one-third of organisations often using this approach.

Other activities involving practitioners or policy-makers

It was most common for agencies to involve practitioners and policy-makers at each stage of the research process: almost half of agencies reported that they did so often (49%) and almost as many again reported doing so sometimes (46%). Similarly, high proportions of respondents reported involving practitioners and policy-makers in collaborative research and/or communicating research findings (Table 10). It was less common for agencies to facilitate or fund peer networks or communities of practice among practitioners and policy-makers, and around one-quarter of respondents indicated that their organisation never did this (24%).

Activities involving patients, service users or members of the public

Knowledge mobilisation activities involving patients, service users or members of the public were reportedly much less common than activities involving practitioners or policy-makers (Table 11). Only one of the five types of activities in this category was done ‘often’ by more than one-third of respondents: producing publications, other written materials or tools aimed at lay audiences. The more usual pattern was for agencies to report that they only sometimes did activities with these groups. Indeed, responses to a later question in the survey (see Activities and focus for effective knowledge mobilisation) showed that over 80% of respondents believed that the role of service users/patients in knowledge mobilisation was currently underdeveloped, suggesting that many respondents believed that the current situation could be improved. Hosting ‘Cafe Scientifique’ or similar public debates was particularly rare: nearly two-thirds of respondents stated that their organisation never did so.

Advocating and advancing knowledge mobilisation

Almost all respondent agencies stated that they often (52%) or sometimes (42%) actively made the case for the value of research-based knowledge in policy and practice (Table 12). Over half sometimes or often funded or conducted projects to advance the science of the knowledge mobilisation field (64%). However, the majority of organisations did not provide post-project funding for knowledge mobilisation activities (56%) and only around 10% of agencies often did so; this may in part reflect the relatively small proportion of respondent agencies that identified themselves as research funders (around one-fifth).

The survey findings across all these categories show a wide range of activities are already in place across these agencies, with sometimes an additional large proportion of agencies identifying specific activities as targets for future action (e.g. webinars were under consideration for the future by nearly one-quarter of agencies). Other innovative areas were of less universal interest; for example, using the arts, providing local consultancy, providing post-project funding for knowledge mobilisation, arranging secondments or working with journalists were all ruled out by one-third or more of the agencies. The involvement of patients, service users and lay people in knowledge mobilisation activities was varied, with a greater proportion of agencies reporting their engagement ‘upstream’ (e.g. in prioritising research) rather than ‘downstream’ (e.g. interpreting and communicating findings).

Terminology in use

There was general agreement across respondents (70%) that it is important to secure broad agreement on key terms before starting knowledge mobilisation activities (Table 14). Somewhat paradoxically, however, there was also broad agreement from nearly 60% of respondents that a plethora of terms around knowledge mobilisation is unavoidable, although around one-quarter of respondents neither agreed nor disagreed with this proposition. There was less consensus around the proposition that ‘knowledge mobilisation activities are distinct from quality improvement work’: while around half of respondents agreed with this statement, just over a quarter disagreed and one-fifth did not feel strongly either way. Similarly, although around half of respondents agreed that ‘knowledge mobilisation is distinct from implementation science’, around one-quarter disagreed and a further quarter did not feel strongly either way.

Theories and frameworks and their utility

There was strong consensus that ‘organisations need to use a range of knowledge mobilisation frameworks rather than just one’, with over 80% of respondents either agreeing or strongly agreeing with this proposition (Table 15). However, there was no consensus on whether or not existing frameworks are hard to operationalise: while almost half of respondents (47%) agreed that they were, a larger proportion (52%) either disagreed with this proposition (4%) or had no strong view either way (48%). Opinions were mixed on whether or not the lack of commonly accepted knowledge mobilisation frameworks hindered the development of knowledge mobilisation strategies: the proportion of respondents who agreed with this proposition (37%) was the same size as the proportion who had no strong view on it, while around one-quarter actively disagreed and thought that the lack of commonly accepted frameworks was not a hindrance to the development of knowledge mobilisation strategies.

Activities and focus for effective knowledge mobilisation

There was strong consensus that effective knowledge mobilisation would be enhanced by developing the role of service-users/patients (87%), a stronger focus on more supportive organisational environments (85%) and a stronger emphasis on the active promotion of knowledge products rather than on their production alone (85%) (Table 16). The proposition that ‘Knowledge mobilisation activities need to be carefully targeted at particular bodies of knowledge’ received a mixed response, with over half of respondents in agreement but around one-third indicating that they neither agreed nor disagreed and almost 10% actively disagreeing. There were also mixed views about whether more emphasis should be placed on knowledge mobilisation at the organisation or multiorganisation level rather than at the practitioner level: although half of respondents agreed with this proposition, over 10% disagreed and around one-third did not feel strongly.

The relationship between literature and practice in knowledge mobilisation

There was broad agreement from around two-thirds of respondents that ‘the theory on knowledge mobilisation as set out in the literature is more advanced than the practice in organisations’ (69%), although around 10% of respondents actively disagreed with this statement (Table 17). One respondent added a comment to expand on their answer:

‘The theory is more advanced than the practice’ is a hard one to answer. I much agree that what we need is more practice, not more theory, which would suggest a response of Agree. But I don’t really agree that the theory is ‘advanced’. Far to the contrary, I find much of it to be hot air. What we need is better theory grounded in more and better practice.

Opinions were split over the question of whether or not the lack of evidence on the impact of knowledge mobilisation approaches was hindering development: over half of respondents agreed that it was (59%), but almost one-third (31%) had no strong view and 10% disagreed (see Table 17). The proposition that ‘Organisations are still trying to make “linkage and exchange” work but the literature has moved on to newer approaches’ was met with a largely neutral response: although just over one-third agreed that this was the case, a majority of respondents (64%) neither agreed nor disagreed (57%) or actively disagreed (7%).

Several respondents made comments to the effect that it was difficult to choose one option for some propositions because ‘it depends’:

As always it is difficult to answer ‘in general’ as it depends what knowledge is being mobilised, where, between whom and for what purpose . . . A case-by-case approach is often needed and the willingness to be flexible and respond iteratively.

I think some people get too caught up in the jargon and theory rather than the practicalities and the ‘doing’. KT is about communication, relationship building, change management, political manoeuvring, etc. It is also going to be different depending on who has the knowledge (or the need) and who they are trying to make [an] ‘exchange’ with. There are many potential interactions at multiple levels.

Some respondents had more fundamental objections to this section of the survey and commented that it was privileging a particular and ‘academic’ perspective on knowledge mobilisation:

As one who works in an organisation that constantly seeks to generate research evidence for policy and practice, and with a strong focus on dissemination, debate and exchange, it is hard to focus on this academic analysis of ‘the problem’. There is so much that researchers could do to engage with policy and practice, drawing on experience of the many academic units and foundations who do this every day. Beware intellectualising the issues – this of itself puts off policy makers and practitioners!

Several of the propositions in this section concerned factors that might be hindering development of the knowledge mobilisation field (e.g. the lack of commonly accepted frameworks or the lack of evidence of impact of knowledge mobilisation approaches). One respondent added a comment to qualify their answer, saying that it was not ‘lack of evidence on the impact of knowledge mobilisation approaches’ that was the problem but ‘lack of a richer, more sophisticated understanding of the mechanisms of action and mediators/moderators of the approaches’. Other respondents added suggestions of other factors that were hindering knowledge mobilisation:

What’s missing is marketing the impact that using research evidence can have for practitioners.

It’s hard to get people to buy into knowledge mobilization when the knowledge itself shifts, grows, and is contradicted. As this shift is inescapable, the root challenge is to help laypersons feel comfortable with some degree of ambiguity in the knowledge base, and to give them the tools to make some degree of assessment of new findings and conclusions for themselves, or to provide them with a reliable source of analysis of such new findings and conclusions.

Overall, with some exceptions, there was a good degree of consensus (fewer than 10% disagreeing) over the propositions presented (although large numbers often opted for the neutral stance of ‘neither agree nor disagree’). Areas where there was more marked disagreement were around the use of terminology and the availability and operationalisability of theories and frameworks. This reinforces the point made earlier on the need for conceptual consolidation, translation and communication (with due reference to the need to apply reflexively the lessons learnt from this project).

Factors relating to evidence or experience

An overwhelming majority thought that it was important to have evidence supporting the knowledge mobilisation approach chosen (94%) and to be able to evaluate the approach in use (93%) (Table 18). One respondent commented that it was necessary to take a pluralistic approach to the definition of supporting ‘evidence’. Prior positive experience with a particular approach in that organisation was important to around three-quarters of respondents. However, there was less interest in whether or not similar organisations were using a particular approach: over one-third of respondents did not think that that was an important factor.

Several respondents added comments about the risks of waiting for evidence to support an approach or of waiting for other organisations to adopt an approach first:

I worry that by waiting to see if other organisations are also doing the work . . . or that everything is tailored and well accepted . . . it will be too late! Facilities should start looking for key signals that are appropriate for them so they move more quickly. Part of our problem with adoption of best practices is everyone waiting around for double-blind studies to support every component – we have to move faster. These are all nice to have, but aren’t requirements to have.

The tension between an evidence-backed approach and one which is well-received by end-users can be difficult to manage, and is a major potential obstacle in effective knowledge mobilization. Some degree of compromise is always necessary, but from the perspective of an individual business, perfect evidence-backing must inevitably give way to practicality.

Factors relating to customisation

The four factors relating to customisation were seen as uncontroversial (Table 19), with all but a handful of respondents rating as important that the knowledge mobilisation approach was customised for the target audience(s), that it was tailored to the type of knowledge to be mobilised, that it took full account of the users’ organisational context/s and that it engaged the end-users of the research.

Factors relating to fit with context

Factors relating to fit with context were also seen by an overwhelming majority of respondents to be important (Table 20). In particular, over half of respondents (60%) thought that it was ‘very important’ that the approach was understood and accepted by key people in the organisation that was developing the knowledge mobilisation strategy. Views were slightly more mixed on the issue of whether or not the chosen approach tapped into formal and informal networks, with around 10% of respondents indicating that this factor was not that important.

Most of the factors identified here were confirmed as being either ‘very’ or ‘fairly’ important by a large majority of respondents in choosing knowledge mobilisation activities. Only two stood out as being more controversial: prior experience with previously good results, and whether or not similar organisations were already using the approach. This suggests that the factors identified may form a useful checklist for agencies as they develop knowledge mobilisation innovation.

Archetype A: producing knowledge

As Chapter 4 demonstrated, many of the agencies we engaged with emphasised the production of research-based knowledge ‘products’. These included systematic reviews, research summaries, ‘mythbusters’, web portals and the like. Even when agencies saw the importance of more interactive and socially situated approaches, they frequently experienced a pull back to knowledge production, collation and synthesis activities. Such approaches emphasise the value of explicit and codified knowledge, and naturally lead to distinct sets of activities and investments that enable the production process (see Table 22). Thus, the idea of archetype A (produce and share) emerged naturally from our data and could be seen to underpin – explicitly or implicitly – many agency strategies.

Archetypes B and C: brokering and intermediation (own research; wider research)

The rise of more interactive and relational models for research use has been mirrored by changes in the emphasis of agencies in how they seek to share research. Many have sought to go beyond ‘produce and share’ to create more interactive spaces where different kinds of knowledge and expertise can interact, be exchanged, be integrated and/or be transformed. Such approaches may have some inevitable ‘push’ but also seek to create ‘pull’, and are clearly informed by the ideas of ‘linkage and exchange’ (see Table 22).

Two different kinds of emphases were seen here in our data. First, activities that sought to broker (primarily) new and local research to a wide variety of stakeholders (often with an emphasis on policy-makers). Such actions were thus often focused on the flow of new research and its promulgation. Second, activities that sought to embrace the wider bodies of research available on any given issue; work that focused on the stocks of existing research-based data, whether locally produced or distal in time and place. These observations gave rise to a pair of archetypes, archetypes B and C, which reflect brokering agency activities focused on the new and the local or much more widely cast.

Archetype D: advocating evidence (proselytisers for an evidence-informed world)

As linkage and exchange has become mainstream, and knowledge application and use is seen as heavily contextual, problem-driven and socially situated, so systems thinking has come to permeate some of the discourses underpinning knowledge mobilisation. Evidence advocates proselytise for a greater role for research-based knowledge, and seek the necessary infrastructural, organisational and cultural changes that might facilitate that role. In this way of thinking, interaction is central, use of research may be as much conceptual (‘enlightenment’) as instrumental, and shaping the wider context becomes a central focus (see Table 22). Archetype D, therefore, represents this aspect of agency work around creating the right knowledge context that is properly cognisant of the social and organisational complexity of the arena where influence is sought.

Archetypes E and F: researching practice (research into practice; research in practice)

Many of the agencies with which we engaged emphasised the need to ‘roll up the sleeves’ and provide hands-on support for local implementation, developing networks and building local absorptive capacities. Such work emphasises different aspects of the conceptual map (see Table 22) and ‘resolves’ key tensions in particular ways. Again, here we could discern two rather different emphases that make up archetypes E and F. First, an emphasis on improving practice through the application of research knowledge produced outside the organisation where change is being sought (archetype E). Often, such approaches emphasise more explicit knowledge and ideas of knowledge transfer, but the adaption and adoption of that knowledge may still include consideration of local understandings and contingencies. The second emphasis (archetype F) highlights local learning and absorptive capacity development, as well as locally produced, often co-produced, research knowledge. Both of these archetypes blur the distinction between the roles of a knowledge mobilising agency and the normal functions of a service delivery organisation, such as organisation development, service improvement and organisational learning.

Archetype G: fostering networks (existing or new)

As our review of reviews showed, understanding has grown of the socially situated way in which research contributes to knowledge, is melded with existing tacit knowledge, is moulded by an understanding of local preoccupations and contingencies, and is actioned in complex and ‘political’ environments. In response to these perspectives on situated knowing, agencies have often sought to create, develop or mould collaborations and network that shape and share expertise, and to increase the role that research-based knowledge can play in these networks. Archetype G, therefore, highlights the assumptions and preoccupations that go into this kind of work (see Table 22).

Archetype H: advancing knowledge mobilisation (building knowledge about knowledge and knowing)

Addressing the irony that much of the practice in knowledge mobilisation is not yet underpinned by either a coherent body of theorising or extensive empirical evaluation, some agency work is aimed at addressing these deficiencies and lacunae. Archetype H, therefore, addresses those aspects of knowledge mobilisation practice that are about refining the field, building shared understanding and committing to further empirical study. This archetype reflects a reflexive application of ‘knowledge about knowing’ to the knowledge mobilisation field.

Mapping the knowledge mobilisation landscape

We found a wide range of knowledge mobilisation practices in our agencies, ranging from the standard and commonplace to the highly unusual and innovative. The distinctive patterns of practice within agencies created significant dissonances, tensions and trade-offs.

Understanding the theoretical and other underpinnings of enacted strategies (i.e. the strategies and approaches being used by agencies)

The diversity of terminology and the richness of theoretical development in the academic literature have not always been fully embraced by the agency actors with whom we engaged. While some were very well versed in that literature, others saw it as somewhat disconnected from their day-to-day realities, as ‘jargonistic’ or as ‘too academic’. Data from interviews and the web survey both highlighted the diversity of terminology in use and the hesitancy about its usefulness. In some ways, this diversity and disputation reflects the conclusions from some of the literature: that convergence and agreement on terminology and underlying theory are unlikely to occur. 24,72,202

Even though theoretical considerations often failed to gain much explicit purchase, many agency actors appeared to have been indirectly influenced by some of the ideas in the knowledge mobilisation literature, such as ideas around linkage and exchange. However, in developing their knowledge mobilisation strategies it was clear that agencies were heavily influenced by a wide range of strategic and pragmatic considerations, such as stakeholder needs and funder requirements. In contrast, there were some notable absences in our discussions about key influences: we did not uncover a major role for the public and service users in shaping or delivering knowledge mobilisation; and there was only a limited role played by learning from evaluation work. Despite the diverse influences on agencies’ knowledge mobilisation strategies, we did discover an emerging consensus around a number of propositions about the underpinnings of effective strategies.

Learning from the success or otherwise of agency activities

A key goal of the project was to see how agencies’ knowledge mobilisation work was being evaluated (formally and informally) and to extract the learning from these evaluative efforts.

PubMed

Covering clinical medicine, biomedical sciences, nursing, dentistry, preclinical sciences and health-care systems, PubMed has over 20 million citations from scholarly journals dating back to 1950. Citations are provided from MEDLINE, life science journals and online books, and links to full-text articles are provided where possible.

British Nursing Index

The British Nursing Index is a database of journal articles, most of which come from 250 UK nursing and midwifery titles with only a small number coming from non-UK specialist journals. The articles date from 1994, with abstracts included from 2004 onwards, and cover areas such as accident and emergency nursing, breast cancer, evidence-based practice, learning disabilities, midwifery, nurse practitioners, orthopaedic nursing, perinatal and neonatal mortality, psychiatric nursing, reflective practice, student nurses, theatre nursing and wounds.

Cumulative Index to Nursing and Allied Health Literature

Cumulative Index to Nursing and Allied Health Literature aims to provide information for all allied health professionals by offering complete coverage of English-language nursing journals and publications from the National League for Nursing and the American Nurses’ Association. As well as journal articles, some books, book chapters, dissertations and conference proceedings are offered. The database goes back to 1982 and also offers some technology journals, as well as articles on consumer health, health promotion and legal issues within health care.

EMBASE

Covering journal articles from around 7000 journals in about 70 countries, EMBASE is a major biomedical and pharmaceutical database with a focus on western European sources. Over 40% of the content is drug-related, contained within areas such as drug research, pharmacology, pharmacy, pharmacoeconomics, pharmaceutics, toxicology, human medicine (both clinical and experimental), basic biological research, health policy and management, public occupational and environmental health, drug dependence and abuse, psychiatry, forensic science and biomedical engineering. The database contains resources dating back to 1974.

Health Business™ Elite

Health Business™ Elite contains full-text content from more than 480 journals, and has content ranging back to 1922. The journals included centre around health-care administration and other non-clinical aspects of health-care institution management, such as hospital management, hospital administration, marketing, human resources, computer technology, facilities management and insurance.

Health Management Information Consortium

This database combines resources from the Library and Information Services of both the Department of Health and The King’s Fund.

The Department of Health Library and Information Services database

With a focus on the NHS and health service quality, and including data from 1983 onwards, the Department of Health database covers areas within health service policy, management and administration. It also holds information on the planning, design, construction and maintenance of health service buildings, as well as occupational health, control and regulation of medicines, medical equipment and supplies, and social care and personal social services.

The King’s Fund Information and Library Services database

Also with a UK focus, this database covers health management and services, social care, service development and NHS organisation and administration. Resources include journal articles, books, reports and pamphlets, and cover the years from 1979 onwards.

ScienceDirect

ScienceDirect is a full-text scientific database offering journal articles and book chapters from more than 2500 peer-reviewed journals and more than 11,000 books. There are currently more than 11 million articles/chapters, a content base that is growing at a rate of almost 0.5 million additions per year.

Social Care Online

Provided by the SCIE, Social Care Online makes use of journal articles, websites, research reviews, government documents and legislation, and service user knowledge in order to provide information on all aspects of social care. Its content dates back to 1960 and is widely used by academics, researchers, information professionals, practitioners, service users and carers, social care managers, policy-makers and students.

Search strategy

Search queries

Results

The results of the individual searches were as follows:

• ERIC – 251 items

• AEI – 26 items

• BEI – six items

• ASSIA – seven items

• IBSS – 44 items

• SSCI – 35 items.

With duplicates removed, a total of 362 items were identified across all databases.