Unlocking data: Decision-maker perspectives on cross-sectoral data sharing and linkage as part of a whole-systems approach to public health policy and practice

Article history

The contractual start date for this research was in April 2021. This article began editorial review in August 2023 and was accepted for publication in June 2024. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The Public Health Research editors and publisher have tried to ensure the accuracy of the authors’ article and would like to thank the reviewers for their constructive comments on the draft document. However, they do not accept liability for damages or losses arising from material published in this article.

Copyright statement

Copyright © 2024 Tweed et al. This work was produced by Tweed et al. under the terms of a commissioning contract issued by the Secretary of State for Health and Social Care. This is an Open Access publication distributed under the terms of the Creative Commons Attribution CC BY 4.0 licence, which permits unrestricted use, distribution, reproduction and adaptation in any medium and for any purpose provided that it is properly attributed. See: https://creativecommons.org/licenses/by/4.0/. For attribution the title, original author(s), the publication source – NIHR Journals Library, and the DOI of the publication must be cited.

2024 Tweed et al.

Context and scope

Scotland continues to face persistent challenges with entrenched social and health inequalities, many of which have worsened during the COVID-19 pandemic. 21 Scotland has a multilevel public health system encompassing multiple local authority and health board areas with diverse populations and geographies. 22 The integration of health and social care; the establishment of Community Planning Partnerships; and the development of Public Health Scotland (PHS) as a single agency for public health has created new opportunities for joint working across NHS public health, local authorities and other partners such as police and community organisations. 11 Following the agreement of six national public health priorities in 2018, the Scottish Government, the Convention of Scottish Local Authorities, and PHS committed to a ‘whole-systems approach’ to public health. 11 While Scotland has a strong track record in the use of linked healthcare data, progress in sharing and linkage of data from other sectors, such as social care, education and criminal justice, has been more challenging. 23 It therefore provides an ideal context in which to answer questions about secondary data access, use and value; to contribute to knowledge on translating results into impact on information systems, as well as to knowledge on data infrastructure and decision-making.

We therefore sought to understand the role of cross-sectoral data sharing and linkage in providing evidence for decision-making in this context, from the perspectives of key stakeholders.

Following discussion with our project advisory group (see Appendix 1), we refined our scope to focus only on (1) data sharing, linkage and use within the public and third sector: this reflects the additional nuance and complexity associated with data sharing with commercial entities and (2) data sharing for the purposes of research, intelligence and evaluation, excluding its use in guiding individual-level decisions about service and care provision.

Conceptual framework

In designing the project, we drew on systems science approaches to public health and interdisciplinary perspectives on the use of evidence in healthy public policy and practice. 12,18,24–26

Our rationale for rooting our conceptual approach in systems thinking was based on three factors characteristic of situations where systems perspectives can be helpful:27 (1) we aimed to achieve sustained change on a broad scale rather than a short-term specific goal; (2) efforts to date to address issues around cross-sectoral data sharing and linkage had met with mixed success, with continued uncertainty and diversity of views about the nature of the problem and appropriate solutions; and (3) the wider context in which data sharing was taking place was likely to be influential, including issues such as research funding trends, resource pressures in the public and third sector, and public attitudes to privacy and data protection. Moreover, our study was based within a jurisdiction (Scotland) which had committed to applying systems science perspectives to public health under the rubric of a ‘whole-systems approach’.

To incorporate systems thinking, we explicitly introduced relevant concepts early in participant workshops and revisited these throughout (for instance, emphasising our interest in the ‘big picture’ of messy connections between different parts of the system, and seeking to identify key bottlenecks and levers where effort might have a disproportionate impact). We used Kumu (www.kumu.io) – an online visualisation tool for concept and systems mapping – to capture outputs across the scoping review, case studies and workshops. Study activities and participant materials were tailored to reflect our recognition that varied definitions of ‘evidence’ compete with multiple other factors as part of decision-making for public health across multiple institutions (e.g. government, NHS, third sector) and levels (e.g. local authorities, community planning, health boards, devolved and national government). 7,24,26

Workshops

We held workshops to bring together people working in public health practice; in policy sectors potentially relevant to health; and in information governance, infrastructure and/or support for data and research; as well as a number of public representatives. Our intended outputs from workshops included: a visual representation reflecting diverse stakeholder perspectives of contribution of secondary data to decision-making for public health benefit; qualitative data on stakeholders’ understanding of barriers/facilitators/benefits/risks to cross-sectoral data sharing and linkage; and a consensus set of recommendations underpinned by a smaller number of guiding principles, which identify specific actions to develop optimal use of cross-sectoral secondary data for improving health and reducing inequalities.

To identify potential attendees, we undertook a stakeholder mapping exercise with the project advisory group followed by a review of relevant organisational websites and advice from gatekeeper organisations such as Administrative Data Scotland. Participants were invited to attend via e-mail, with public representatives offered remuneration for time spent preparing and participating in workshops. A total of 39 invitations were issued, of which 20 individuals from 14 organisations (including 2 public representatives) consented to take part. Figure 1 illustrates the sample of stakeholders participating in the workshops: each box represents one individual.

FIGURE 1.

Workshop participants, by sector/role.

Workshops were intended to be sequential and cumulative, with the entire cohort of participants expected to attend all three. Due to the ongoing COVID-19 pandemic, workshops took place online between September and December 2021 using Microsoft Teams^® (Microsoft Corporation, Redmond, WA, USA). 28 Using online meeting tools such as Microsoft Teams has been demonstrated to be an effective means for data collection through focus groups and workshops.

Each workshop lasted 3 hours and consisted of a mix of whole group and breakout room activities, including presentations, facilitated discussions and participatory tools such as live polling. Prior to each workshop, participants were invited to complete asynchronous online activities that provided background information for the workshop and used polls and free-text comments to gather additional data. Table 1 describes the purpose, content and format for each workshop.

Figure 2 illustrates the sequence of project activities, and their inter-relation.

FIGURE 2.

Flow of workshop activities in chronological order.

Workshop 1 introduced the project and invited participants to discuss their experience of decision-making in practice, and the actual and potential contribution of evidence from secondary data in this process, as part of facilitated break-out room discussions supported by the visual mapping tool Kumu. Workshop 2 began with a summary of feedback from the first workshop alongside a description of findings from the scoping reviews (see Scoping reviews), followed by breakout group sessions focusing on one of three case studies (see Case studies). These comprised a presentation on the case study by the lead investigator with an opportunity for questions and answers, then a facilitated discussion of the findings in the context of results from workshop 1 and the scoping reviews.

Workshop 3 started with a presentation of integrated findings from preceding workshops, the scoping reviews, and case studies, before moving on to interactive exercises to brainstorm and refine draft recommendations. Participants were invited to suggest initial recommendations using a framework based on the following themes, identified from the scoping reviews and earlier workshop discussions: features of data; technical; governance; legal and ethical; workforce; relational; political; institutional; and external factors. During this phase of the final workshop, participants worked in breakout rooms with the support of facilitators to first brainstorm initial ideas; then edit and refine them (e.g. clustering those which were similar); and finally discuss the impact and effort associated with each; assign them to relevant organisations; and propose appropriate time scales.

Following the workshops, draft recommendations were reviewed and edited by the project team for consistency and reducing duplication. The updated recommendations were circulated by e-mail to workshop participants for feedback, and discussed with a wider set of stakeholders (see Post-workshop engagement). A final set of recommendations was then agreed by the research team and approved by the project advisory group.

Scoping reviews

We undertook two scoping reviews to support discussions at the workshops: the first aimed to situate our project within the wider context of evidence use in decision-making for healthy public policy and practice, and the second aimed to understand existing evidence about barriers and facilitators affecting the use of secondary data in the UK context. We chose a scoping review approach given our interest in: understanding the broad ‘landscape’ of existing research; exploring concepts; and identifying potential theoretical or thematic frameworks for use in other elements of the project. 29–31

The research questions for the reviews were as follows:

1. What do existing scoping and systematic reviews say about how evidence is used in decision-making for public health practice and healthy public policy?

2. What are the barriers and facilitators at individual, organisational, and societal levels to the use of secondary data in decision-making for public health and healthy public policy in the UK?

Detailed search strategies and inclusion criteria are provided in the review protocol (see Report Supplementary Material 1); an example search strategy is provided in Appendix 2. For each review, we searched three bibliographic databases (MEDLINE, Scopus and the Social Science Citation Index) as well as websites of key organisations and asked our stakeholder advisory group to signpost any other relevant resources. With the support of an information specialist, we developed tailored search strategies for each bibliographic database, comprising terms relating to evidence (e.g. evidence, research, data, knowledge); the settings of interest (e.g. health policy, public health, public policy, health planning); and the article types of interest (i.e. systematic or scoping reviews, qualitative synthesis, or meta-ethnography). For question 2, our search strategy comprised terms relating to secondary data (e.g. secondary data, routine data, administrative data, data sharing); the settings of interest (as for question 1); and factors affecting utilisation (e.g. barriers, facilitators, challenges, support, obstacles).

Studies were eligible if they were published in English in the 10 years preceding the date of searches (June 2021); studies for question 1 additionally had to use systematic or scoping review methodologies, and those for question 2 had to be carried out in the UK (or for reviews, include studies carried out in the UK). Key terms used in inclusion/exclusion criteria were defined in the protocol, based on discussions with the project advisory group.

For each separate review, we screened retrieved citations based initially on the title and abstract and, for those found to be potentially relevant, the full text. An initial 25 titles and abstracts were screened for each review as part of pilot testing for the inclusion and exclusion criteria, and the remaining screening undertaken independently by 2 team members, with opportunities for discussion and feedback from other team members. A 10% sample of full texts was screened by two reviewers for each review, with the remainder of full-text screening undertaken independently by two team members. Data extraction was undertaken by one reviewer and checked by a second, using a structured template in Microsoft Excel. The findings of the two reviews were combined as part of the thematic analysis detailed in a subsequent section.

Case studies

Previous work has highlighted the value of case studies in understanding challenges for data sharing, and in sharing best practice: however, few case studies exist which deal with cross-sectoral data sharing and linkage. 4,17

In this project, we used three case studies of completed public health research projects using cross-sectoral data linkage to provide real-world examples for discussion by participants at the workshops. 32–35 Case study projects were selected on the basis of having attempted to undertake individual-level record linkage between health and non-health data sets for research aiming to inform healthy public policy. They were chosen to cover different policy sectors; data sets; geographies; funding models; and governance processes. The chosen projects are described in Table 2.

Project leads for each case study were included as co-investigators on this project in order to ensure alignment between the case studies and workshop planning. Data on the process and outcomes of each case study project were gathered using a proforma completed by the project lead and subsequent in-depth semistructured interviews (see Report Supplementary Material 2). Each case study lead facilitated a series of breakout room sessions on their project at the second workshop, comprising a presentation; question and answer session; and group discussion.

Post-workshop engagement

After the third workshop, a copy of the draft recommendations was circulated to all workshop participants with an invitation to provide further input, annotated in places to highlight areas which were incomplete or where consensus had not been reached.

We also sought feedback on the draft recommendations from a broader range of stakeholders, particularly those we had struggled to recruit to the workshops. The stakeholders we met with to discuss our emerging findings included the Scottish Government Data and Intelligence Network; the Integrated Joint Board Chairs and Vice-Chairs Network; the Digital Office for Scottish Local Government leadership team; and Research Data Scotland (RDS): further details are provided in Appendix 3, Table 6.

Analysis

We used thematic analysis throughout the study to analyse and interpret findings from the scoping reviews; case studies; and workshops. 36 Analysis of content generated by study activities took place on an ongoing and iterative basis, so that findings could be fed into workshop discussions and could inform the planning of subsequent activities. We used the web-based tool Kumu to facilitate thematic analysis, as well as to visualise findings and produce workshop materials for participants.

Analysis of findings from the scoping reviews identified a set of factors affecting the use of routine data, which were subsequently used to structure the interpretation of material from the case studies and participant discussions.

We sought to use participants’ original wording throughout the project and to be guided by their views and interpretations. This was felt to be especially important in developing the recommendations and accompanying detail (such as impact, effort, timescales and responsibilities): the research team aimed to edit recommendations only lightly to avoid repetition; where there was no input or no consensus on a particular point, we have noted this in the text and have made a clear distinction between participant views and commentary by the researchers, or broader stakeholders.

Public involvement

Our public involvement approach for this project comprised two main strands, reflecting the public’s role as the original source and owners of the data being discussed throughout the project. First, our project advisory group (see Appendix 1) included two public representatives, who contributed to shaping the design, delivery and dissemination of the study. Second, our workshop participants included two public representatives, who contributed to data collection and the development of recommendations. Recruitment of public representatives sought to reflect a range of perspectives, including those with experience of the issues featured in the case study projects (via lived experience networks supported by organisations which the case study leads had existing relationships, such as Homeless Network Scotland) and those with an interest in the use of data (via established public panels run by Administrative Data Research Scotland and the University of Glasgow). Public representatives were remunerated for their time preparing and attending meetings as per National Institute for Health and Care Research (NIHR) guidance and received support to participate in the form of briefings at the project start and informal check-ins and correspondence throughout the project.

Equality, diversity and inclusion

As part of this ‘research about research’, we considered equality, diversity and inclusion both in the substance of the research (i.e. ensuring that our research considered how secondary data could be used as part of efforts to address health inequalities, as well as overall population health) and the way it was conducted.

With regard to the first point, we ensured that ‘reducing inequalities’ was explicitly included as part of the study aim; emphasised health inequalities between diverse population groups as an important motivating factor for the research in our scene-setting with workshop participants; and selected case studies with an explicit focus on health inequalities. With regard to the second point, throughout the research, we opted to use person-first or other inclusive terminology when referring to different population groups and communities, especially in relation to our case studies (such as ‘people affected by homelessness’ or ‘care-experienced children and young people’). We sought to recruit public participants with lived experience of the issues covered by the case studies, although were only able to do this in one case, with the remaining public participants recruited from an existing public panel. We made a commitment in our recruitment materials to meeting participants’ accessibility needs to ensure their full participation in the workshop and asked all participants well in advance of the workshops to advise us of such measures. Measures taken included circulating papers and presentation slides in advance, and reviewing all presentation slides for accessibility. As our participants were primarily professional stakeholders and therefore selected based on existing roles, we were limited in our ability to select on the basis of diversity; however, we did seek to achieve diversity in gender, age and geographical location across Scotland. Our research team was also diverse in terms of gender and career stage, with consideration given to maximising development opportunities for less experienced staff (for instance, all of the co-investigators received facilitation training and opportunities; the project was jointly led between two researchers, one more junior and another more senior; and project staff were supported to develop applications for future research funding).

Scoping reviews

In the first scoping review, which examined previous literature reviews of how evidence is used in healthy public policy and practice, we identified 31 relevant studies from an initial pool of 1490 studies (see Appendix 2, Figures 4 and 5 for review flow charts and Appendix 2, Tables 4 and 5 for details of included studies).

We found that previous reviews of evidence use for healthy public policy generally fell into two groups: those examining evidence use in general terms18,37,38 and those which examined narrower questions about the use of specific types of evidence (e.g. economic evaluations) and/or in specific settings (e.g. for spatial planning and the built environment). We extracted data from both but weighted our synthesis and presentation towards the former.

In the second scoping review, which focused on primary research about barriers and facilitators to the use of secondary data in decision-making for healthy public policy and practice in the UK, we identified 29 relevant studies from an initial pool of 2456 studies, plus 2 studies from grey literature sources (see Appendix 2). Most studies in this review examined data use within health care, including economic evaluation and health technology assessment; post-approval monitoring of medications and devices; monitoring of provider performance; and surveillance systems. Fewer examples were evident of the sharing and use of secondary data beyond the healthcare system. Many of the studies retrieved were descriptive in nature, with limited critical reflection on processes of data access, utilisation, and impact: where challenges were discussed, the focus tended to be on technical aspects. Most existing case studies of secondary data use were descriptive in nature and positive in outlook, with little detail on the challenges encountered. Finally, most studies retrieved represented the perspective of researchers, with some examining public attitudes; very few investigated the views of evidence users from practice or policy.

We synthesised findings from the two reviews under the following key themes, derived inductively from the data: functions of evidence in practice; diverse understandings of what constitutes evidence; and factors influencing the use of evidence in decision-making, with a particular focus on secondary data. Findings from the last of these themes (factors influencing the use of evidence and data) were grouped into a set of key subthemes as follows: characteristics of data and evidence; technical; workforce; legal, ethical and governance; political and institutional; and external factors.

These themes and subheadings were then used to support the synthesis of findings from the case studies and workshops, and to guide participants in brainstorming and identifying recommendations.

Case studies

Case study materials used in the workshops – including linkage diagrams, timelines and presentation slides – are included in Report Supplementary Material 3. Here, we briefly summarise key themes from the case studies which formed the basis for discussions with participants at the workshops.

Common challenges encountered by these projects included: liaison with multiple different organisations, each with their own competing demands and priorities; difficulties identifying who had responsibility and authority for key decisions or clarifying existing permissions or conditions of use for relevant data sets; and sequential and interdependent governance processes which exacerbated delays. Governance processes were perceived as something of a ‘black box’, given uncertainty about the criteria for scrutiny and assessment, and unpredictable timelines. As all three of the studies were undertaking novel linkages, and some were using data sets not previously used for research, few meta-data were available in advance and unanticipated difficulties with data quality or completeness were common, often arising only once the data sets were made available and analysis underway. All studies were impacted by COVID-19 to some extent, most commonly due to additional pressures on data controllers and agencies responsible for data linkage and/or their prioritisation of other research projects focusing on COVID-19. For example, issues identified with one case study data set in March 2020 were not resolved until November/December 2020 due to staff re-assignment at data controller organisations. Most of the case study projects were required to substantially amend their initial plans as a result of these challenges, with negative knock-on impacts on the validity, generalisability and impact of the resulting work: for instance, because the findings were less timely, or for a more limited geographical area, than anticipated.

Facilitators identified by case study leads included existing relationships and networks with relevant organisations (such as data controllers); the availability of precedents as a basis for data sharing and linkage; support from those who had previously used and were familiar with the data sets; flexibility on the part of funding organisations; the ability to answer questions that would be difficult or impossible to address through other research methods; and where available, the skills and expertise of specialist data access co-ordinators. All projects enjoyed substantial goodwill and buy-in from external stakeholders and agencies, but case study leads suggested that the impact of this was limited by the wider systemic problems highlighted above.

Workshops

In our scene-setting discussions about decision-making, participants described significant challenges in trying to use evidence derived from cross-sectoral data sharing and linkage in their day-to-day work: these included organisation and cultural, as well as technical and ethical barriers.

We were looking at prevention in homelessness. But when we started to look at the streams of prevention, we realised that we would need data from so many different sources that it became impossible before we even started. So starting to look at stuff like poverty, and how often people were visiting doctors, and there was a whole cluster of information. But we realised that just trying to get any kind of data consent across such a broad church was almost impossible.

Participant, public/lived experience representative

Yeah, I just know within my own organisation it takes a lot to change the culture. So, you know, those legacy processes and just the way people have always done things. So, coming along and saying, right, okay, from now on all decisions are going to be evidence based, doesn’t really just happen overnight.

Participant, local government

In response to case studies, participants acknowledged that sharing and linkage of routine administrative data could offer unique benefits in terms of evidence for decision-making, especially where combined with other forms of evidence (such as qualitative research), although others highlighted the ways in which decision-making was shaped by what data were (easily) available, and how this might create biases or blind spots. At present, cross-sectoral data sharing and linkage are felt to be happening on an ad hoc basis, contingent on individual relationships.

I think one of the big challenges with secondary data is it’s often used because it’s what there is, rather than because it answers the question … So that’s about what we know we already can get our hands on, or what we know has been asked. There might be much, much better data available, but we don’t necessarily know who has it and who to ask.

Participant, local public health practice

I’ve got some brilliant examples of where we’re sharing secondary data and we are receiving secondary data and we support individual projects. But they’re very specific and they’re based on knowing the right person at the right time who had the same pattern for what you’re trying to achieve. So all the stars were aligned and it just worked.

Participant, local authority

COVID-19 was generally felt to have heightened awareness of the possibilities offered by data sharing and linkage, and to have changed perceptions regarding the balance of risks and benefits, resulting in some processes becoming much quicker. However, this was not universal and some processes remained time-consuming, with knock-on effects on public health decision-making:

I have learned that in times of crisis, public sector organisations seem to become less risk averse. Bureaucratic barriers have been broken down quickly and it would be helpful to maintain this in an appropriate way in respect of how we continue to do business moving forward.

Participant, local government

I’m aware of this project that’s looking at factors that might impact vulnerability to being a positive COVID-19 case, and it went through a fast track process. But by the time we got the decision, some of the restrictions have been eased. So we needed that data within, I’d say, like weeks or a month, but actually, we didn’t get the data until 5–6 months down the line and we didn’t get all the data for the research team. What was the point? That didn’t help with decision making. And if that decision, if that intelligence was available to decision makers within the month, then who knows what might have been the decisions about coming out of the lockdowns? We’ll never know now.

Participant, organisation supporting research

Several participants recognised that sharing and linkage of administrative data were less well-developed outside the healthcare system and that this posed challenges for research on the wider determinants of health:

I think the NHS has kind of spent a lot of time gathering data, and it’s now got, it’s had an infrastructure around ISD [Information Services Division] or whatever it is now for years. And I mean, there’s still areas of the NHS where it, you don’t have it so much like GPs. But like, I think it’s kind of like an example of where a lot of this stuff can be done. Whereas there are other factors, other areas of policy where data has just never been collected in a significant way in like … In [devolved policy sector] we’re starting it from scratch.

Participant, Scottish Government

Participants described duplication of effort as being common, and an appetite for a more co-ordinated and timely approach:

What we see at national level is that’s repeated, so you might have, say, [Health Board A] who’s dealing with getting a data-sharing agreement and they’ve got certain battles. And then we hear it from [Health Board B] as well and they have the same battles …. I think more and more we’re seeing that it should be the same process.

Participant, national public health agency

That, especially in a world which has shifted quite so much over the last couple of years, I’m now very sceptical, if you present me information from 2014, that it necessarily has any direct applicability to how the world is working right now.

Participant, local public health practice

While some technical challenges – such as interoperability of systems – were noted, participants also highlighted the over-riding importance of institutional, ethical, and political factors in determining whether cross-sectoral sharing and linkage took place. Resource and workforce constraints were particularly prominent, especially in the context of other demands and pressures on stakeholders:

It occurs to me that there’s a lot of, there is a division between the technical, which is, you know, just like methodologically can you do this? You know, do you have the legal consent and … But the much more squishier factor around, do you have trust? Do you have the relationships to be able to do this? Because you could have the technical skill and it could be legal, but you might just never get the institution to sign on to whatever it is you want to do.

Participant, Scottish Government

Just thinking about how the rise, elsewhere, of the Black Lives Matter agenda, for example, suddenly meant that finally, we were able to start saying actually, you know, it would be really very useful to be able to talk about ethnicity in the context of COVID vaccinations, use of services more generally … There’s a sudden shift, sometimes, in the political agenda or political environment that means that people are suddenly interested in a topic that they weren’t before. And that can suddenly plug or unplug things.

Participant, local public health practice

And it’s really difficult for IJBs [Integrated Joint Boards] to justify the recruitment or the expansion of data or analytical teams while we’re also cutting six social workers out of the system … So, it’s not that there’s not a desire to do the work or to be helpful or … it just starts to fall into the nice to do pile.

Participant, Health & Social Care Partnership

While there was discussion of potential risks associated with cross-sectoral data sharing and linkage, participants also highlighted that not acting also carried risks:

In my mind I was kind of like, well why bother; you’ve gone through that huge journey, we’re making it [the research] so difficult, why do people bother? And people bother because it matters. And so we have a moral obligation to find that story and act on it because keeping it separate is actually a failing on our part, was kind of my reflection, and the value of … you know, the data comes together to bring its full value I guess … Why would we have this data and this detail and not put it together and then do something about it?

Participant, National Public Health Agency

During workshops 1 and 2, participants began to identify key principles for cross-sectoral data sharing and linkage. We responded to this unexpected output by collating these suggestions into a set of six guiding principles, which were then further refined and validated during dedicated discussion time at the final workshop (Figure 3). The intention was that these serve as overarching principles that should guide cross-sectoral data sharing and linkage, alongside the detailed recommendations for action.

FIGURE 3.

Guiding principles for cross-sectoral data sharing and linkage identified by workshop participants.

During the final workshop, and as part of the post-workshop engagement, a set of recommendations was proposed and organised according to the themes identified in the scoping review (shown in Table 3).

Several recommendations are interlinked: for instance, recommendations A (a strategic rather than piecemeal approach to data sharing and linkage) and S (planning and investment should be guided by long-term considerations as well as contingency planning for future emergencies) both recognised that planning and investment in this area required a mix of proactive and reactive approaches in order to meet key strategic needs while maintaining flexibility to respond to unexpected contingencies.

Many recommendations were based on the scale-up or implementation of approaches which have been successful already, either within specific areas in Scotland or internationally: for instance, the Welsh Secure Anonymised Information Linkage (SAIL) databank was cited as an example of best practice for a ‘one-stop shop’ for key data sets from multiple sectors (recommendation C), while the Local Intelligence Support Teams (LIST) embedded into Health and Social Care Partnerships in Scotland were identified as a potential model for wider integration of research, intelligence, and innovation capacity with community planning structures with jurisdiction over the determinants of health (recommendation O).

Additional detail provided by participants on the likely impact and effort associated with individual recommendations, and the organisations responsible is included in the appendices (see Appendix 4, Table 7). Participants did not always reach consensus on the impact/effort status of the recommendations or on the organisations responsible; where this is the case, it is noted.

Strengths and limitations

Our findings benefit from being informed by wider theoretical and empirical insights about evidence use in policy and practice, via the scoping reviews which underpinned the workshops and recommendation development, and by case studies of real-life challenges and impacts from projects which have undertaken cross-sectoral record linkage. The literature searches for the scoping review were conducted in June 2021, so there may be relevant material published since then that is not reflected in the review. Although uptake of invitations to participate in the workshops was reduced by workload pressures associated with the ongoing COVID-19 pandemic, workshops did include representation across all key sectors of interest and extensive post-workshop engagement with other key stakeholders provided additional opportunities to test and refine draft recommendations. Unsurprisingly, given this broad representation, we did not achieve consensus on every recommendation. Our findings were necessarily heavily informed by the Scottish context for public health and data sharing, but many of the themes and recommendations are of relevance to other jurisdictions with similar legal frameworks and data infrastructures. Our project did not examine in detail the resource implications of improved utilisation of secondary data across sectors, but the adequate investment is likely to be a key determinant of success alongside the proposals made by participants for strengthening governance, staffing, and infrastructure for data sharing.

There is increasing interest in secondary data sharing and linkage across a range of organisations and jurisdictions, partly spurred by evidence demands associated with the COVID-19 pandemic. However, many existing initiatives focus solely on healthcare data – such as the recent Goldacre Review in the UK – which may miss opportunities to understand and intervene on the wider determinants of health, including social, environmental, economic and commercial factors. 42 This is especially important in the context of ongoing and emerging challenges to public health, such as the cost-of-living increases observed in many European countries and worsening impacts of climate change. 43,44

CRediT contribution statement

Emily Tweed (https://orcid.org/0000-0001-6659-812X): Conceptualisation, Data curation, Formal analysis, Funding acquisition, Investigation, Methodology, Project administration, Supervision, Validation, Visualisation, Writing – original draft, Writing – reviewing and editing.

Kristina Cimova (https://orcid.org/0000-0001-7747-9230): Data curation, Formal analysis, Investigation, Methodology, Project administration, Visualisation, Writing – original draft, Writing – reviewing and editing.

Peter Craig (https://orcid.org/0000-0002-7653-5832): Conceptualisation, Data curation, Formal analysis, Funding acquisition, Investigation, Methodology, Project administration, Supervision, Validation, Writing – original draft, Writing – reviewing and editing.

Mirjam Allik (https://orcid.org/0000-0003-1674-3469): Funding acquisition, Investigation, Methodology, Writing – reviewing and editing.

Denise Brown (https://orcid.org/0000-0002-5195-5312): Funding acquisition, Investigation, Methodology, Writing – reviewing and editing.

Mhairi Campbell (https://orcid.org/0000-0002-4416-7270): Investigation, Methodology, Writing – reviewing and editing.

David Henderson (https://orcid.org/0000-0002-4411-8532): Funding acquisition, Investigation, Methodology, Writing – reviewing and editing.

Charlie Mayor (https://orcid.org/0000-0003-4729-3536): Funding acquisition, Investigation, Methodology, Writing – reviewing and editing.

Petra Meier (https://orcid.org/0000-0001-5354-1933): Methodology, Writing – reviewing and editing.

Nick Watson (https://orcid.org/0000-0002-0364-427X): Funding acquisition, Investigation, Methodology, Writing – reviewing and editing.

Valerie Wells: Methodology.

Lorna Dick: Project administration.

Acknowledgements

We would like to thank the following:

• All of our workshop participants for their invaluable contributions;

• Members of our advisory group (listed in Appendix 1), for their guidance throughout the project;

• Valerie Wells, information specialist, for assistance with search strategies for the scoping review;

• Lorna Dick, programme administrative assistant, for administrative support.

Data-sharing statement

De-identified transcripts from the workshops are archived with the UK Data Service’s ReShare repository (https://dx.doi.org/10.5255/UKDA-SN-856285) and are available on application via https://reshare.ukdataservice.ac.uk/. All other queries should be addressed to the corresponding author.

Ethics statement

Ethical approval for the study was granted by the University of Glasgow College of Social Sciences Research Ethics Committee (reference number 400200204; approval date 4 June 2021).

Information governance statement

The University of Glasgow is committed to handling all personal information in line with the UK Data Protection Act (2018) and the General Data Protection Regulation (EU GDPR) 2016/679. Under the Data Protection legislation, the University of Glasgow is the Data Controller, and you can find out more about how we handle personal data, including how to exercise your individual rights and the contact details for our Data Protection Officer here: www.gla.ac.uk/myglasgow/dpfoioffice/

Disclosure of interests

Full disclosure of interests: Completed ICMJE forms for all authors, including all related interests, are available in the toolkit on the NIHR Journals Library report publication page at https://doi.org/10.3310/KYTW2173.

Primary conflicts of interest: All authors except Kristina Cimova and Mhairi Campbell were co-investigators on the project and therefore received support for the present manuscript via the NIHR Public Health Research grant NIHR133585. Other support for the present manuscript included core funding for the Medical Research Council/Chief Scientist Office (MRC/CSO) Social and Public Health Sciences Unit (affecting Emily Tweed, Kristina Cimova, Peter Craig, Mirjam Allik, Mhairi Campbell, Denise Brown, and Petra Meier; MRC grant number MC_UU_00022/2 and CSO grant number SPHSU17). Other NIHR funding to authors included NIHR154243 (Peter Craig and Petra Meier); NIHR134801 (Peter Craig); NIHR131709 (Peter Craig); NIHR153474 (Emily Tweed and Petra Meier) and NIHR12/211/54 (Nicholas Watson). Petra Meier was a member of the NIHR Public Health Research programme’s Funding Board from 2018 to 2022. Emily Tweed was a member of the NIHR Public Health Research programme’s Prioritisation Committee from May 2022 to September 2023.

Department of Health and Social Care disclaimer

This publication presents independent research commissioned by the National Institute for Health and Care Research (NIHR). The views and opinions expressed by authors in this publication are those of the authors and do not necessarily reflect those of the NHS, the NIHR, MRC, NIHR Coordinating Centre, the Public Health Research programme or the Department of Health and Social Care.

This article was published based on current knowledge at the time and date of publication. NIHR is committed to being inclusive and will continually monitor best practice and guidance in relation to terminology and language to ensure that we remain relevant to our stakeholders.

Other outputs from this project

Blogs

• ‘Finding the key to Unlocking Data’

• ‘Unlocking Data – using online workshops to develop recommendations for policy and practice’

Study registration

This study is registered as researchregistry6740.

Funding

This article presents independent research funded by the National Institute for Health and Care Research (NIHR) Public Health Research programme as award number NIHR133585.

About this article

The contractual start date for this research was in April 2021. This article began editorial review in August 2023 and was accepted for publication in June 2024. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The Public Health Research editors and publisher have tried to ensure the accuracy of the authors’ article and would like to thank the reviewers for their constructive comments on the draft document. However, they do not accept liability for damages or losses arising from material published in this article.

Copyright

Copyright © 2024 Tweed et al. This work was produced by Tweed et al. under the terms of a commissioning contract issued by the Secretary of State for Health and Social Care. This is an Open Access publication distributed under the terms of the Creative Commons Attribution CC BY 4.0 licence, which permits unrestricted use, distribution, reproduction and adaptation in any medium and for any purpose provided that it is properly attributed. See: https://creativecommons.org/licenses/by/4.0/. For attribution the title, original author(s), the publication source – NIHR Journals Library, and the DOI of the publication must be cited.

Flow charts

FIGURE 4.

Flow chart for scoping review strand 1: what do existing scoping and systematic reviews say about how evidence is used in decision-making for public health practice and healthy public policy?

FIGURE 5.

Flow chart for scoping review strand 2: what are the barriers and facilitators at individual, organisational and societal level to the use of secondary data in decision-making for public health and healthy public policy in the UK?