Linkage of routinely collected NHS data to evaluate liaison mental health services: challenges and lessons learned

Study design

The study was a retrospective cohort study. For each hospital with a liaison service, patient groups admitted to hospital and seen by the hospital LMHS would be compared with a patient group at the same hospital not seen by the liaison service and another patient group at a separate hospital with no LMHS during the study period. We intended to match all patient groups on characteristics determined from primary and secondary care data and compare outcome for certain marker conditions which are commonly assessed by liaison services (such as physical–mental comorbidity, acute behavioural disturbance, cognitive impairment/dementia) in different LMHS and configurations. Cost of care pathways for patients referred to LMHS and the matched comparison patient groups and the main determinants of those costs over 12 months after an index hospital episode would then be estimated. All analyses of the linked data would be examined by the research team at the University of Leeds. The published protocol describes the rationale for the design and the planned analyses in more detail. 21

Data sources

Routinely collected NHS data from primary and secondary care were required to construct care pathways. Secondary care data were required to determine those patients attending specific hospitals and to determine whether the patient was seen by a LMHS (if applicable). Primary and secondary care data were required to characterise these patients with respect to their physical and mental health, determine their interaction with primary and secondary care services and identify their outcomes beyond the immediate spell in hospital.

Hospital Episode Statistics (HES)22 is a database controlled by NHS Digital23 that contains data that are routinely collected in secondary care and relate to emergency department, inpatient and outpatient episodes for patients at hospitals in England. Data within HES can be used to determine those patients attending specific hospitals, to characterise those patients with respect to certain aspects of their physical and mental health, to determine their interactions with secondary care services and to determine certain outcomes. Episodes represent discrete periods of care under a particular consultant and a spell is composed of one or more episodes – for example, when a patient is transferred from a medical to a surgical team for treatment or from an intensive care unit to a lower-intensity medical ward.

ResearchOne (RO)24 is a database controlled by The Phoenix Partnership (TPP) that contains data that are routinely collected in primary care provided by organisations using the SystmOne clinical information system. 25 SystmOne (34% coverage) and Egton Medical Information Systems (EMIS)26 (56% coverage) are the most prevalent clinical information systems used by organisations in general practice. 27 No specific information was available a priori regarding the intersection of patient populations between HES and RO. Based on the prevalence of SystmOne use by organisations in primary care, we expected that a sufficient number of patients with episodes in the HES data for the specific hospitals to be studied would have data relating to their primary care collected by organisations that use SystmOne and contained within the RO database. We chose not to use the clinical practice research datalink (CPRD)28 which is a database controlled by the Medicines and Healthcare Regulatory Authority29 that contains data that are routinely collected in primary care due to the limited number of organisations whose primary care data were included in CPRD at the time of research design, and the geographical distribution of these organisations.

We divided the study into two phases. In the first phase, we planned to use the data from HES and RO only, to construct care pathways for patients attending hospitals without a LMHS (i.e. to provide data for one of the control groups). This paper focuses on this process, which became so challenging that we were unable to progress the study further.

Population

Patients were included in the study population if they had an accident and emergency (A&E) attendance or inpatient admission to 1 of 11 hospitals without a mental health liaison service between 1 April 2013 and 31 March 2014 (Index Period) for which they were aged 18 or over. At the time we started the study, none of these 11 hospitals had a LMHS, although all now have a service due to the recent expansion in liaison services in England. Patients were excluded from the study population if they had registered a Type 2 objection30 with NHS Digital to prevent their identifiable data from any health and social care settings being released. NHS Digital selected the study population by applying this inclusion/exclusion criteria to emergency department attendances and inpatient admissions recorded as episodes within HES.

Data items

Data items to be included for each patient in the study population were those determined by the research team to be necessary and sufficient for analysis. No patient-identifiable data were included in the data required by the research team.

For patients included in the study population, the following data items were selected for inclusion from HES for the period from 1 April 2012 to 31 March 2015 (Index Period ± 1 year):

• Episodes from HES (A&E). For each episode, data items relating to the following were included: (1) attendance; (2) diagnoses; (3) investigations; (4) treatments; (5) socioeconomic status; (6) provider; and (7) demographics.

• Episodes from HES (Admitted Patient Care). For each episode, data items relating to the following were included: (1) admission; (2) critical care; (3) cause; (4) diagnoses; (5) procedures; (6) specialty; (7) discharge; (8) spell; (9) provider; (10) demographics; (11) legal status of patient; (12) psychiatric care and status; and (13) socioeconomic status.

• Episodes from HES (Outpatient). For each episode, data items relating to the following were included: (1) appointment; (2) referral; (3) diagnoses; (4) procedures; (5) specialty; (6) provider; (7) demographics; and (8) socioeconomic status.

For patients included in the study population whose data relating to primary care are included in RO specific data items relating to the following were selected for inclusion from RO for the period prior to 31 March 2015:

• Demographics and socioeconomic status.

• Selected primary care events including: (1) coded diagnoses/observations that relate to selected comorbidities; (2) prescriptions and repeated prescriptions for selected drugs; (3) non-coded referrals; (4) appointments; and (5) practice registrations.

Linkage

Linkage was required to enable data items relating to the same patient in HES and RO to be determined. No persistent link exists between HES and RO and, to our knowledge, no linkage methodology had been previously enacted between the two databases. We worked with NHS Digital and TPP to design a methodology that would: (1) be able to link data from HES and RO for the study, (2) be acceptable to the data controllers from a technical and organisational perspective and (3) be consistent with legal and ethical frameworks applicable to the use of routinely collected NHS data for research purposes (in the absence of individual patient consent).

Patients are referenced by source-specific identifiers in HES and RO. Different identifiers are used to reference the same patient in HES and RO. No persistent mapping exists or indeed is feasible ex post between these source-specific identifiers. Data from HES and RO are therefore completely disjoint based on these source-specific identifiers alone. Linkage requires a mapping to be established between these source-specific identifiers. NHS Digital and TPP require access to at least one common data item for each patient that can be used to establish this mapping. Moreover, any data item used for this purpose must demonstrate sufficient quality (e.g. completeness) to ensure the integrity of the resultant linkage. To protect the privacy of patients, pseudonyms are generated from NHS numbers by the data controllers and these pseudonyms are used to generate a mapping between source-specific identifiers.

Figure 1 shows a summary of the key data flows and processing activities of the methodology. Linkage is undertaken by NHS Digital, in accordance with recommendations provided in the Caldicott Review: information governance in the health and acre system document published by the National Data Guardian. 31 NHS Digital and TPP generate two unique references for each patient in their databases: (1) a pseudonym, generated by applying a one-way cryptographic hash function (SHA-512) to an input that comprises a cryptographic salt and the NHS number and (2) a source-specific identifier. For a patient with a given NHS number, NHS Digital and TPP will generate the same pseudonym but a different source-specific identifier. Both the pseudonym and source-specific identifier generated for each patient are specific to the study. Pseudonyms are then used by NHS Digital to: (1) communicate to TPP those patients for whom data are required from RO and (2) generate mappings between different source-specific identifiers for each patient. NHS Digital and TPP provide the required data items to the research team at the University of Leeds, including only the source-specific identifier as the unique reference for each patient. Mappings generated by NHS Digital are then provided to the research team and used to enable data items relating to the same patient across databases to be determined.

FIGURE 1.

A summary of the key data flows and processing activities of the methodology.

Infrastructure

Infrastructure provided by the Leeds Institute of Clinical Trials Research (LICTR) at the University of Leeds was used for the study. Information security standards defined in the Data Security and Protection Toolkit32 were met by an infrastructure and the Data-Sharing Framework Contract between the University of Leeds and NHS Digital (REF: CON-315426-K3W7R) including the use of this infrastructure.

Agreed project-specific terms for processing activities involving NHS Digital were defined in a Data-Sharing Agreement established between the University of Leeds and NHS Digital. Agreed project-specific terms for data flows and processing activities involving TPP were defined in the RO Data Request Form provided to TPP by the University of Leeds and approved by the RO Project Committee, and in a Data Processing Agreement established between the University of Leeds and TPP.

Data flows between organisations were performed using one of the following secure data transfer services:

• Secure file transfer (SFT): Provided by LICTR and used for data flows between University of Leeds and TPP.

• Secure electronic file transfer (SEFT) service:33 Provided by NHS Digital and used for data flows between University of Leeds and NHS Digital, and between TPP and NHS Digital.

Approvals

Approval was required from Research Governance at the Sponsor (University of Leeds). A favourable ethical opinion was sought from an NHS Research Ethics Committee (REC) and the Health Research Authority (HRA)34 to ensure consistency with applicable ethical frameworks for medical research, including the Declaration of Helsinki. 35 Additionally, an application was made to the Confidentiality Advisory Group (CAG)36 at the HRA to obtain (if applicable) support under Section 251 of the NHS Act 2006. 37 Figure 2 shows a summary of the envisaged progression of approval processes.

FIGURE 2.

A summary of the envisaged progression of approval processes.

Approvals

The study received: (1) approval from Research Governance at the Sponsor, (2) a favourable ethical opinion from NHS REC (North of Scotland) (REF: 16/NS/0025), (3) management permission from a subset of NHS Trusts which manage the hospitals whose patients were to be included in the study population, (4) a decision from the CAG at the HRA that Section 251 support was not required ‘on the basis that there is no disclosure of patient identifiable data without consent’ (REF: 16/CAG/0037), (5) approval from the Independent Group Advising on the Release of Data (IGARD)38 at NHS Digital, (6) approval from the Information Asset Owner for HES at NHS Digital and (7) approval from RO Project Committee at TPP. Approvals took 907 days in total. Figure 3 summarises the progression of the approval process.

FIGURE 3.

A summary of the progression of the approval processes and the associated time periods.

Favourable opinion from NHS REC (North of Scotland) was contingent on management permission from the NHS Trusts (n = 8) which manage the hospitals (n = 11) whose patients were to be included in the study population. Requests for management permission were submitted to the NHS Trusts following receipt of the outcome from the CAG. We started by trying to obtain linked data from NHS Trusts with no identifiable LMHS during the study period, with the intention of then widening the data collection to NHS Trusts with liaison services. This part of the study, therefore, proceeded based on a subset of the hospitals only (n = 8) – that is, hospitals from which one group of control subjects could potentially be identified.

Following application to TPP, 52 days elapsed before approval by the RO Project Committee. TPP supplied an approval date of April 2017. An indicative date of 15 April 2017 was used to calculate the period of 52 days. Review and approval by the ResearchOne Project Committee were assumed to take place on the same date. Following submission of application to NHS Digital for data from HES [REF: NIC-77953], 454 days elapsed before the application was reviewed by IGARD. Delays were attributable to the resolution of technical and organisational queries raised by NHS Digital in their consideration of the application, including: (1) the process of pseudonym generation, (2) the data flows involved in the linkage methodology, (3) the organisation with legal responsibility for RO, (4) the data controller and data processor responsibilities for organisations involved in the linkage methodology and (5) the data storage locations for University of Leeds and TPP. NHS Digital noted that the linkage methodology was a ‘first of type’ and took the application to IGARD for advice prior to submission for review by IGARD. Following review by IGARD, further information was requested from the project team, which was subsequently supplied and a recommendation to approve the application was provided in a subsequent IGARD meeting. A Data-Sharing Agreement was established between the University of Leeds and NHS Digital, 118 days after approvals – attributable to the University of Leeds awaiting required information from TPP.

Enactment

Figure 4 shows a summary of the progression enactment processes and associated time periods. Following establishment of a Data-Sharing Agreement between the University of Leeds and NHS Digital, the methodology progressed to enactment in which the defined processing activities were undertaken by the University of Leeds, NHS Digital and TPP. NHS Digital generated pseudonyms for patients in the cohort using a Salt file generated and supplied by the University of Leeds. Incorrect pseudonyms were generated by NHS Digital and supplied to TPP on two occasions, which resulted in no matches being found within RO. On the first occasion, NHS Digital applied an incorrect hashing algorithm (SHA-256). On the second occasion, NHS Digital used an incorrect format for the input to the hashing algorithm. NHS Digital determined that the HES data supplied to the University of Leeds had to be resupplied following the first occasion, but not following the second occasion. Following the second occasion, NHS Digital regenerated the pseudonyms using the correct hashing algorithm and values in the correct input format and resupplied to TPP. Receipt of a valid set of pseudonyms from NHS Digital by TPP required 313 days.

FIGURE 4.

A summary of the progression of the enactment processes and the associated time periods.

Following supply of a valid set of pseudonyms, TPP were able to match pseudonyms with those generated for patients in RO. TPP generated and supplied a file to NHS Digital which mapped these pseudonyms to corresponding source-specific identifiers from RO. NHS Digital used the file to generate a file which mapped source-specific identifier in HES to source-specific identifier in RO. This mapping file was supplied by NHS Digital to the University of Leeds. TPP then supplied the data extract from RO to University of Leeds. Enactment took 385 days in total.

Table 1 summarises the data supplied from HES and RO. Data supplied from HES contained 181,063 patients from six hospitals (mean = 30,177, standard deviation = 28,875.86) who matched the required criteria (see Population). Data supplied from RO contained 33,666 (18.6%) of these patients from the 6 hospitals (mean = 5611, standard deviation = 5206.59). For two hospitals (G and H), no patients who matched the required criteria could be determined.

In response to a follow-up query about low rates of case identification, NHS Digital determined the probable cause for this result to be the use of the Provider Code (5 character) (PROCODE5)39,40 to determine the hospital associated with an episode. NHS Digital advised that the use of both the PROCODE5 field and the Site Code of Treatment (SITETRET) field would likely yield a higher number of patients for each hospital, including those for which no patients with an index event were determined. From September 2021 NHS Digital has made changes to the way they process the HES data set, which means the data referenced are no longer available online. 41

Due to the impact on project timescales, we determined that this change to the data specification would not be feasible. After considerable discussion with National Institute for Health and Care Research and a further delay because of the COVID-19 pandemic, it was decided not to proceed with the analysis of the data we held from the six hospital sites. Such an analysis would have been proof that the data linkage we were attempting could be achieved, but its value in terms of the overall outcomes we aimed to achieve was limited. We therefore proceeded to destroy the data we had obtained, and a certificate of destruction was signed by the University of Leeds investigators and data management staff on 15 July 2021.

Lessons learnt

We are not the only group of researchers to encounter significant delays and difficulties with data access and linkage. Taylor and colleagues (2021)20 reported it taking them 2.5 years to be able to link a series of databases to study outcomes for congenital heart disease. To link 5 relevant data sets, they were required to produce 47 documents at the application stage for 11 data controllers or departments which were submitted 162 times in total. Although similar study information was requested for each application, each required different wording, structure and detail.

The biggest barriers to realising the full potential of data linkage as a powerful research tool are gaining and maintaining public trust while simultaneously reducing the cost, delays and inefficiencies in how privacy-preserving data linkage is carried out. 44 The necessities of linking primary and secondary data during the COVID-19 pandemic to facilitate research and rapid transfer of knowledge led to the development of such initiatives as OpenSAFELY45 and the British Heart Foundation’s Cardio Vascular Disease-Coronovirus Disease-united Kingdom (CVD-COVID-UK) consortium. 46 These initiatives show what is possible in principle, and in the distinctively permissive legal context of the pandemic, but they are not easily generalised to enable wider adoption by researchers.

On reflection, we dramatically underestimated the degree of complexity in obtaining permissions to link databases and for the linkage work to be done. At the time of the study, linking primary and secondary care data in the manner we envisioned was a ‘first of type’ linkage and required the various organisations involved to undertake work they had previously not carried out. OpenSAFELY41 was started in response to the need to obtain rapid data at the time of COVID-19 and required a change in law (effective 18 March 2020); so it was not an option for the LP-MAESTRO study. The approach of OpenSAFELY, where data linkage is carried out in-house, would have undoubtedly reduced the complexity of the approval processes for our programme, but the legal basis for accessing and linking data in OpenSAFELY, despite researcher pressure, has not changed, and it is only permitted for COVID-19 purposes [https://digital.nhs.uk/coronavirus/coronavirus-covid-19-response-information-governance-hub/the-nhs-england-opensafely-covid-19-service-privacy-notice#top (20 July 2023)]. Although the advent of OpenSAFELY has illustrated that in the right legal as well as technical environment a basic primary/secondary data linkage is readily achievable, studying more complex care pathways, such as those involving both emergency care and mental health secondary care services, could still have been a problem, as our original plans required access to specific hospital databases which would have required bespoke linkages. At the time of our study, mental health data recorded via HES were also limited in their scope, and much of the work of liaison services were not captured nationally.

Implications for research

Accessible and functional pathways are required to enable projects to adopt research methodologies based on the linkage of routinely collected NHS data that aim to protect the privacy of patients, their personal data and the confidentiality of their information. In the absence of such pathways, there is a risk that such methodologies will not be adopted. This may lead to a detrimental impact on the rights of patients, the quality of research designs and processes, and the ability of researchers to meet policy requirements for research that better reflects populations served. Improved knowledge and skills are required to develop more effective and efficient processes and structures to support research governance across all stakeholder organisations, including academic institutions, data providers and approval/advisory bodies.

Researchers who are planning to apply for funding for a data linkage study need to consider the following when developing their application with reference to: (a) the research team; (b) research governance (includes sponsor, ethics, HRA); (c) the funder; and (d) the data controllers. Several of these issues, including complexity, process delays, bureaucracy, time taken to acquire data post approval, were highlighted by Taylor and colleagues20 but were also experienced by us.

1. Research team

Staff morale – what seem to be insurmountable delays results in low staff morale, as there is inevitable pressure to meet funding deadlines and also a desire to conduct the research which has been planned.

Career progression and institutional reputational damage – failure to complete or even start a planned programme of research due to delays beyond the control of the research team impacts upon the career progression of middle-grade researchers who are employed on the project who are not able to publish as widely or influentially as they would have planned. The Sponsor may also be at risk of reputation damage.

1. Research governance

Complexity – data request forms vary across different data sets, some of which are more suitable for requests for data in relation to randomised controlled trials, rather than large data projects. Streamlining and standardising data request forms across different organisations, while maintaining robust data governance procedures commensurate with the Data Protection Act 2018, would reduce time delays.

Process delays – we encountered numerous complex process delays which have major cost implications for any research study.

Delays lead to further delays – a delay in one part of the acquisition process leads to delays in other parts of the system.

Bureaucracy – each time research staff working on the project are recruited, some data-sharing agreements have to be updated. Minor changes can take weeks to be processed.

Fear of legal misstep – failure to comply with the Data Protection Act 2018 and potentially incur a costly fine mean that some data controllers require excessive safeguards before agreeing to release data when sufficient safeguards are in place.

1. Funder

Budget to access the data sets – it is difficult to anticipate the final costs of access to a database, as limited information is usually available online prior to completion of a full application which cannot be started until funding has been received for the research. Although applications are for the full length of a project, annual data-sharing extensions are required which involve additional costs. Taylor and colleagues20 reported that in the case of NHS Digital, even one additional field request could result in a charge equal to that of the original data extraction.

Most grant applications are for periods of 3 years, during which it is not feasible at present to conduct complete data linkage processes and carry out planned data analyses. Underestimation of the time involved in acquiring linked data also results in costs for the research programme, as researchers employed to conduct data analyses cannot carry out this work until the data are securely acquired.

1. Data controllers

Acquiring data – once all approvals are in place, it can still take at least 6 months to acquire the data.

Difficult-to-spot small mistakes in the data request process that have major implications for the data which are supplied – two out of the eight hospitals for which we requested data were supplied with empty data fields, due to a misunderstanding about the different ways and data fields which HES uses to record hospital site.

Submitted for publication

Romeu D, Guthrie E, House A. Staff experiences of liaison mental health services following implementation of CORE 24 informed by service user preferences.