An evaluation of a referral management and triage system for oral surgery referrals from primary care dentists: a mixed-methods study

Article history

The research reported in this issue of the journal was funded by the HS&DR programme or one of its preceding programmes as project number 11/1022/15. The contractual start date was in December 2012. The final report began editorial review in March 2017 and was accepted for publication in July 2017. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The HS&DR editors and production house have tried to ensure the accuracy of the authors’ report and would like to thank the reviewers for their constructive comments on the final report document. However, they do not accept liability for damages or losses arising from material published in this report.

Declared competing interests of authors

Iain Pretty is involved with the delivery of referral management services to NHS England, for which he receives financial reward. He was involved with the development of early pilot models in referral management at NHS Trafford. He was involved in the evaluation of the Index of Sedation Need (IOSN) tool. Paul Coulthard chaired the Oral Surgery and Oral Medicine Working group that developed the Guide for Commissioning Oral Surgery and Oral Medicine. This guide advocated the use of referral management systems in pathway management. He was involved in the development of the IOSN tool.

Copyright statement

© Queen’s Printer and Controller of HMSO 2018. This work was produced by Goldthorpe et al. under the terms of a commissioning contract issued by the Secretary of State for Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science Park, Southampton SO16 7NS, UK.

2018 Queen’s Printer and Controller of HMSO

Oral surgery

The specialty of oral surgery deals with the diagnosis and management of pathology of the mouth and jaws that requires surgical intervention. Oral surgery involves the treatment of children, adolescents and adults, and the management of dentally anxious and medically complex patients. Oral surgery care is provided by oral surgeons and by oral and maxillofacial surgeons, as the clinical competencies of these two specialties overlap. Oral surgery is a recognised specialty of dentistry, whereas the UK General Medical Council recognises ‘Oral and Maxillofacial Surgery’ as a medical specialty. 1

NHS England’s Commissioning Guide for Oral Surgery and Oral Medicine1 describes the provision of oral surgery within the English NHS. The guidance describes three levels of case complexity, known as Levels 1, 2 and 3. 1 Oral surgery complexity is generally assessed based on both the type of procedure and patient factors. For example, a very simple surgical procedure can be complicated by a patient’s medical history or degree of anxiety. A full list of procedures and conditions that would be treated in each complexity Level can be found in Appendix 1.

In England, oral surgery is typically delivered in one of three settings and by three distinct groups of clinicians:

1. Primary care general dental practice – most surgical procedures are conducted in general practice by general dental practitioners (GDPs). The removal of simple teeth and roots is covered under the mandatory services section of the General Dental Service (GDS) contract. 2 The extraction of one or more teeth or roots in a single course of treatment attracts a band 2 charge for the patient and results in a ‘payment’ of 3 Units of Dental Activity (UDAs) for the dental practitioner (UDA prices vary between practices, but an average value of £25 is usually used). Patients will typically be free of systemic disease and will not require adjunct sedation, and the procedure will not be technically demanding. Such procedures are known as Level 1 procedures. There is an expectation that all Level 1 procedures will be undertaken in practice unless there are patient factors that complicate management. If a GDP does not feel able to undertake the procedure, they should look within the practice to see if another clinician can assist. Ultimately, it is the provider’s (GDS contract holder’s) responsibility to ensure that Level 1 procedures are undertaken in practice.

2. Intermediate services, dentists with a special interest (DwSpIs) – these services provide Level 2 care, and are typically delivered by a clinician with enhanced skills and experience who may or may not be on a specialist register. Indeed, such services could be provided by a consultant-grade clinician operating and remunerated as a Level 2 provider. It is expected that most Level 2 services will be provided in a primary care setting (where additional equipment may be required) under Any Qualified Provider (AQP), GDS or Personal Dental Service (PDS) contracts, in which case patient’s charges will be levied. Level 2 procedures may be delivered as part of continuing care or, as is most usual, by referral. The basis for the development of the DwSpI services was the 2004 framework document produced by the Department of Health (DH) and the Faculty of General Dental Practice, which was followed in 2006 by guidelines for commissioning such services. 3,4

3. Consultant or specialist care – the commissioning guidance describes Levels 3a and 3b but, for the purposes of this report, Level 3 providers are typically consultant-led services delivered in, and by, NHS hospital trusts under NHS standard contracts. Although Level 3 services are led by consultants, they will typically engage a wider workforce, including specialty and associate specialist-grade clinicians and those in formal training positions. Hospitals delivering oral surgery services at Level 3 include district general hospitals, larger training hospitals (foundation trusts) and dental hospitals that have the additional requirement to train dental undergraduates. 5

Health needs assessments

Commissioning of primary care services often takes place with little needs assessment or knowledge of where referrals come from and where populations receiving care are based. Detailed knowledge of the population, their needs and treatment preferences are essential to ensure that primary care services can be delivered effectively. 6

There is no defined methodology for determining the needs of a population for oral surgery services, in contrast to, for example, orthodontic services, where there is a clear and well-defined approach. 7 The Adult Dental Health Survey, of 2009,8 reported that 8% of dentate adults had one or more untreatable teeth (on average 2.2 teeth), but it is difficult to know when or if this need for an extraction is or will be expressed as demand. For example, many decayed roots will be asymptomatic. Data from the NHS Business Services Authority (BSA) may provide indications of activity in relation to band 2 course of treatment provision (which includes tooth extraction, but also includes fillings and root fillings) and the number of extractions provided in primary care, while Hospital Episodes Statistics (HES) can provide similar information for secondary care. None of these methodologies provides information on case complexity; coding and tariff charges in secondary care are not consistently applied and the use of general anaesthesia (GA) or sedation in both settings is poorly understood. 9,10

Referrals from primary care

In 2006, a new dental contract was introduced in England that replaced a fee-for-item service with a banded course of treatment approach. UDAs are awarded based on the type of care delivered, for example 1 UDA for a check-up with preventative care, 3 UDAs for any number of extractions (although most frequently one), any number of fillings or root filings plus any treatment included in band 1, and 12 UDAs for work requiring laboratory input, such as crowns and dentures, plus any treatment included in a band 1 or 2 course of treatment. Co-payments or patient charges are levied according to the three bandings: band 1, £19.70; band 2, £53.90; and band 3, £233.70. These charges apply to services provided under GDS and PDS contracts, but do not apply to services provided under hospital contracts. An anomaly in the contract required dentists to claim the UDA tariff for the procedures that they were referring for and collect the appropriate patient charge revenue. This created a perverse incentive to refer: dentists would be paid the same fee to refer a patient as to undertake the procedure. This resulted in the NHS paying twice for activity: once in primary care, and then again in secondary care. The contract incentive was, in part, responsible for the increase in referrals seen since 2006. Other factors contributing to the seeming relentless increase in referrals,11 which has been mirrored in medicine,12 include a lack of oral surgery experience at the undergraduate level among junior GDPs,13 and the increasing proportion of older patients retaining their teeth but presenting with complex medical histories and polypharmacy. Despite the 2006 contract being causally linked to the increase in referrals, it had been recognised for some time that the capacity in oral surgery services was under pressure. 3 Kendall, in an assessment of English oral surgery services, demonstrated that in a 3-year period from 2004, referrals doubled from a monthly average of 182 to 364. 14

Reasons for referral from primary to secondary care vary, but a questionnaire completed by dentists in Greater Manchester15 found the following:

• anticipated surgical difficulty (69% of cases)

• medical history issues (49% of cases)

• require a second opinion (32% of cases)

• practitioners do not undertake surgical procedures (29% of cases)

• practitioners lack appropriate facilities or staff (28% of cases)

• patients require emergency management of pain, swelling or haemorrhage (11% of cases).

Reasons for referral were not mutually exclusive.

The costs of providing oral surgery in secondary care are substantial. In 2009/10, in the north-west region alone, the total cost amounted to £53,864,857. In addition to the cost element, the increase in referrals has caused issues around workforce insufficiency and capacity, and has negatively impacted on 18-week referral to treatment targets. Although some trusts have welcomed the increased activity, others, especially those departments in district general hospitals (DGHs) staffed by oral maxillofacial surgery (OMFS) services, have found that the oral surgery referrals deflect activity from their core offering and that many are inappropriate.

Primary care (Level 2) services

As with many service developments in the NHS, formal evaluation and published reports on primary care oral surgery services are sparse. Kendall provided one of two such descriptions of how a Level 2 provider might work14 and this is described below, while Bell describes a retrospective audit of a primary care scheme designed to address issues in provision of services to remote areas. 16

Utilising two GDP practices in the Croydon area, combined with a simple referral management system, all non-urgent referrals for oral surgery procedures were captured and then subjected to a two-stage triage process. The first was an administrative check of the paper referral form and the second was a clinical assessment of a patient’s suitability for primary care treatment. The scheme reported no reduction in the total number of referrals received, but the offer of a primary care service did not appear to stimulate demand or increase Level 1 (work that should be performed by a GDP) referrals from GDPs. Of the 3117 non-urgent referrals, 36% were sent to secondary care and 59% to primary care (data were missing in 5% of cases). No referrals were returned to the GDP as being unsuitable – that is, at Level 1.

The removal of a substantial amount of activity from a single trust (nearly 60% of referrals) could cause concern over the stability of the service. Kendall states, however, that, rather than destabilising the unit, the reduction enabled a balance to be obtained, a reduction in waiting times and a re-focus on the core provision of oral maxillofacial procedures rather than oral surgery. 14

The service was revisited in both 201117 and 2012. 18 Kendall reported that, after 2.5 years of service, the background referral numbers continued to rise but the proportion of referrals directed to the primary care (Level 2) service had also increased, from 60% in 2004 to nearly 80% in 2010. With so many referrals being appropriate for Level 2 services, this suggests that much of this work was being undertaken in GDS and, with the provision of a service to deliver this, it is now being referred outwards. The number of Level 1 referrals remained low, at 1.1%. 17 Kendall reports a basic economic analysis with referral management costing around £7 per referral and a £600 saving per case seen in primary care. However, a system-wide economic appraisal was not undertaken and a formal health economic evaluation of costs and effects was not possible from the data available.

Referral management

The recognised increase in referrals from both GDPs and general medical practitioners (GPs) into hospital services has initiated several approaches to management of the problem. A Cochrane systematic review divided these approaches broadly into three main groups: (1) financial, (2) managerial or (3) professional education. 19 Each of the main groups involves varying degrees of active interruption to the referral process:

1. Professional education involves the production, dissemination and support of clear referral guidance, often using harmonised referrals forms to encourage collection of appropriate data, and often supported by targeted continuing professional development initiatives.

2. Managerial systems include the use of referral management services, clinical assessment services, clinical assessment and treatment services and as ‘in-house’ second opinions or peer review.

3. Financial approaches (at least those of relevance to the NHS) are based on fundholding by referrers and, hence, they incentivise care provided within the practice or referred to lower-priced primary care facilities.

The Cochrane review concluded that research into the management of referrals was limited. 19 Preliminary findings suggested that passive systems, such as the introduction of referral guidelines, were unlikely to change referral behaviours. The use of structured referral forms has some potential, but informatics support would be needed to make such forms useable in a practice environment (i.e. to force adherence to completion of mandatory fields). Financial methods risked the application of unselective reductions in referrals and negative impacts on patient care. 19 None of the studies examined a formal referral triage and capture service, such as that employed by Kendall. 14,17

The King’s Fund reviewed referral management systems,20 recognising that such systems can be as simple as a referral guideline through to active interventions in the referral pathway. It summarised that not all referrals were needed, but some patients who needed a referral did not receive one. The review found that the quality of referral letters was often poor, and appropriate primary care treatment or investigations had often not been undertaken prior to referral.

Focusing on capture and assessment referral solutions, the report found a range of strengths and weaknesses (including the filtering of inappropriate referrals, improving quality of referrals and providing commissioning intelligence), but also potentially increasing costs, delay to a patient’s journey and the creation of barriers between primary care and secondary care colleagues. 20 Of interest was the reported belief by primary care trusts (PCTs) that their referral management systems were reducing activity despite the fact that the data from acute trusts did not support this supposition.

A study by Cox et al. 21 highlights this anomaly. Using data from NHS Norfolk and using an active referral management centre (RMC) approach, Cox et al. 21 found that, in all cases, the use of the RMC approach increased referrals rather than decreased them. The authors concluded that the RMC approach, as the most expensive management option, was the least effective. 21 The authors’ retrospective time series design looked only at decreasing attendance, that is, reducing overall referrals, and there were no primary care redirection services. Their approach, therefore, would reduce only those referrals that evidence clearly showed were inappropriate. Referrals that were incomplete or poor quality, although initially returned, would be corrected and resent, thus increasing the number of referrals. These findings are consistent with those of Kendall’s work in dentistry, which revealed very low rates of Level 1 referrals – that is, those that might be considered inappropriate. 14

Two recent comprehensive reviews have examined demand management. 22,23 The key findings of these reviews provide the current context for our study, and, rather than duplicate the reviews in a formal literature review chapter, the key findings of these two large reviews are summarised here. The first, by Blank et al. ,22 sought to examine interventions related to referrals from primary care to specialist services. The work focused exclusively on referrals from GPs and excluded dentistry. The systems of referral were described as complex because of the interplay of local factors, such as waiting times, the directory of services and access to specialists. The review found stronger evidence to support interventions that involved peer review, improved the quality of referral information, offered specialist contact prior to referral, electronic referrals and the provision of community specialist services. 22 It found weaker and conflicting evidence over the use of gatekeeping systems and alterations in remuneration. The current work reported here addresses the issues raised in the review, apart from specialist contact prior to referral. By incorporating a standard referral form with mandatory fields, referral quality is improved, an electronic-only submission route can be implemented and a primary care service for the delivery of appropriate oral surgery procedures is introduced. The referral management and triage process can be considered a ‘gatekeeper’ with the potential to divert referrals and reject those considered ‘inappropriate’.

The second review, by Winpenny et al. ,23 focused on the effectiveness and efficiency of moving hospital services (outpatients) into primary care, and examined 184 studies, some of which included dental settings. They found that minor surgical procedures could be carried out in primary care safely and effectively, but that provision of such services could stimulate demand by addressing previously unmet need. The cost-effectiveness of these services was likely to depend on local contracting, and this also applied to general practitioner with a special interest (GPwSI) services, which also demonstrated evidence of supply-induced demand. 23 The review found that direct access to specialist services in some cases (such as audiology for hearing tests) offered obvious benefits, but that in other cases (such as musculoskeletal services) it risked generating a substantial increase in demand. The review considered referral management services as a substudy group and included a qualitative study element with individuals working, commissioning or implementing such services. The group identified four emerging themes from their interviews:

1. the lack of clarity relating the aims of functions of referral management services

2. the challenge of stakeholder adoption and buy-in

3. practical and administrative difficulties

4. the impact of perceived effectiveness of the aims and priorities of such services.

The group recommended that future schemes should have clarity of aims and defined indicators of success. In addition, the group identified a research need in the evaluation of clinical effectiveness and cost-effectiveness of RMCs.

Electronic referral management systems

The NHS ‘Choose and Book’ system, now known as the NHS e-Referral Service (eRS),24 is an example of a large, national electronic referral system. Choose and Book has reduced the administrative burden associated with appointment booking and may have reduced the non-attendance rate at secondary care clinics. 24,25 However, this system may not be appropriate for all specialties and in all contexts. Prior to and following the introduction of Choose and Book, independent electronic referral management systems were developed. For example, Maddison et al. 26 evaluated electronic referral management with central triage and an adjunct specialist primary care service for uncomplicated musculoskeletal conditions. They found that, although the number of referrals greatly increased following the introduction of referral management, waiting times fell. In addition, duplicate referrals disappeared and a high degree of patient satisfaction was reported. However, Kim et al. 27 found that, although electronic referral management improved access to care, there were some barriers to implementation. Some referring clinics reported that multistep login procedures and a lack of computer access and reliable internet connection contributed to electronic referrals taking longer to complete, which was associated with lower satisfaction with overall clinical care. Again, this finding highlights the importance of considering the context in which the referral management interventions are implemented.

Consultant triage

When GDPs are aware that their referrals are being scrutinised by a peer with a specialist training, their referral behaviours may alter around who and how they choose to refer to secondary care. Studies of peer-reviewed interventions, in which referral quality has been judged by consultants and fed back to GPs, have resulted in some improvement in the quality of referral information and a reduction in the number of overall referrals into secondary care,28–30 although it may not lead to permanent changes in practice. 31 In addition, electronic referrals directly from GPs to consultant triagers prior to making Choose and Book appointments were found to be associated with shorter waiting times for appointments than paper referrals. 25 There is some evidence to suggest that the consultant triage element may improve the quality and appropriateness of dental referrals; however, GDPs may feel that their clinical autonomy is compromised by examination of their referrals during the triage process.

Despite the apparent lack, or contradictory nature, of evidence to support active referral management systems, by 2009, 91% of PCTs had some form of referral management system in place for GPs. 21 These systems seek to influence either the decision to refer, the destination of the referral or the quality of the information provided in the referral. At the time of writing there are several referral management systems in place across NHS England Area Teams (ATs) for dentistry – largely resulting from the guidance issued in the Dental Commissioning Guides in which RMCs are central to the process of directing referrals into Level 2 services. 1

Quality of referrals from general dental practitioners

In addition to managing waiting lists, reducing costs and improving overall patient satisfaction, referral management has the potential to improve the quality of referrals from primary care, which may improve triage efficiency and the overall diagnostic accuracy of the content of referrals. 20 Qualitative work around quality and appropriateness of referrals from GPs assessed by senior NHS clinical and managerial staff in five PCT areas in England32 found that important attributes of appropriate referrals were:

1. Necessity – should the patient be referred based on clinical examination, National Institute for Health and Care Excellence (NICE) guidelines or their own medical history?

2. Destination – could and should the patient be treated in an intermediate setting rather than in secondary care?

3. Quality – is the information contained in the referral relevant and thorough, and have the necessary investigations taken place?

This is congruent with the NHS Quality, Innovation, Productivity and Prevention (QIPP) agenda around promoting quality while making efficiency savings, and offers evidence to suggest that referral management interventions do have the potential to be cost-effective while supporting improvements in quality. In 2011, a commentary on the implications of QIPP for dentistry identified the development of centralised assessment and triage services and establishment of primary care-based specialised services as innovations that could contribute to the QIPP agenda. 33

Many audits have assessed the quality of referrals from GDPs into all specialties. The results are usually poor, with ‘Dear Sir’ letters still being commonly employed, which contain little in the way of clinical detail, rationale for treatment, results of special investigations or the provision of radiographs. 34,35 Increasing the quality of referrals facilitates the provision of triage, informs commissioning (if data are appropriately captured) and increases the efficiencies of primary and secondary care services by enabling appointment scheduling and clinical staff allocation to be appropriate to complexity.

The King’s Fund describes the financial challenges facing the NHS, stating that difficult ‘trade-offs’ will be required. 36 Cost-saving measures in the NHS are rarely welcomed and often viewed as reducing quality and impacting on patient choice. However, beyond the clear need for dentistry to contribute to savings the NHS must make, there is the possibility that referral management allied to primary care diversion has much to offer in terms of quality enhancement, for example:

• care closer to home

• more convenient appointment times and extended opening hours

• reduced waiting times (Kendall reports 6 weeks in primary care vs. 18 weeks in secondary care14)

• single ‘see and treat’ appointments reducing opportunity costs for patients

• greater productivity leading to increased capacity.

Concerns and consequences

Despite the apparent ‘easy win’ that centralised triage and primary care-based oral surgery services offer for patients and the NHS, concerns have been voiced. The removal of ‘simpler’ cases from secondary care is a potentially destabilising move, and the resultant shift of case mix to more complex patients is a perceived risk to the training of both undergraduates and specialist trainees. 13 Indeed, the use of a referral management system could, by reducing undergraduate training experiences, result in an increase in referrals from a population of graduates with no oral surgery skills. Hospital trusts may argue that the current tariff arrangements are based on the assumption that hospitals treat a wide range of cases, with payment for simpler procedures helping to generating revenue to offset the higher costs incurred in treating more complex cases that cannot be fully recovered from the tariffs. 22

Clinicians in secondary care have argued that, although care can be delivered in primary care, there is not always a compelling reason why it should. The reduced governance in primary care (for example, wrong site surgery reporting) combined with the single-handed nature of oral surgery provision in primary care, compared with a team approach in hospital, threatens, it is argued, the quality of the care provided. If patients experience complications, these will largely need to be managed by secondary care and, hence, savings are lost, patients experience poor outcomes and the system fails. 23

A further concern is that implementing primary care specialist services adds another service to the system without adequately managing the supply side, that is, without ensuring that there is corresponding downsizing of secondary care services. So instead of substitution, supplementation occurs, producing an overall increase in costs, which is a significant risk for a financially strapped NHS. A good example of possible pitfalls is described by Richardson et al. 37 in the context of developing skill mix by introducing nurse practitioners.

Aims and objectives

Considering the identified evidence gaps and the pressing need for the NHS to understand the quality and financial impact of referral management services, there is a need for a high-quality, contemporary evaluation of this change in service organisation and delivery. 22,23,38

This project aimed to evaluate the introduction of an electronic referral management system with consultant-level triage and the introduction of a new primary care service for oral surgery within a defined health-care system containing a diverse set of hospital providers. The study design used a mixed-methods approach with ITS design.

In addition, a diagnostic accuracy study of remote clinical triage was undertaken to assess the efficiency of this important stage in the referral management process. The research programme contains the necessary elements to address the research gaps identified by the previous systematic reviews,20,22,23 primarily the impact on quality of referrals, use of electronic referrals, the provision of community specialist services and the effect of gatekeeping systems.

The main research question to be addressed by this work was the following:

• How does a robust online referral management and triage system, allied to provision of a specialist primary care service, impact on the costs and quality of oral surgery services provided by different providers in different settings in a defined health-care system?

At the highest level, we wanted to know if we can change the behaviour of referring GDPs without destabilising a complex, interdependent acute sector, to ensure that only those who need hospital care are managed in this setting. In order to fully answer this main research question, the following secondary research questions were formulated:

• Chapter 2 , Efficiency of remote clinical triage

  • How do remote clinical triage outcomes conducted by an experienced consultant compare with outcomes of face-to-face examination (reference test) performed by the same consultant?

  • How do remote clinical triage outcomes performed by GDPs and different consultants compare with outcomes of face-to-face examination performed by an experienced consultant (reference test)?

  • What are the views of triagers on the benefits and problems of a remote clinical triage system and how can the system be improved based on their experiences?

• Chapter 3 , Implementation and health needs assessment (phase 1)

  • What are the practical issues for the NHS in introducing an all-electronic referral system from scratch?

  • What is the effect of all-electronic referral system on

    • the total number of referrals

    • the quality of referrals including an assessment of compliance with national referral guidelines

    • the time taken to complete referrals?

  • What are the views of key stakeholders on the benefits and problems of the electronic referral system and how can the system be improved based on their experiences?

• Chapter 4 , Active referral management with consultant and general dental practitioner triage: quantitative findings including economic evaluation (phases 2 and 3)

  • What are the differences in referral numbers, referral quality and the mean cost per referral between virtual management (phase 1) and consultant-led active management (phase 2)?

  • What are the differences in referral numbers, referral quality and mean cost per referral between the year of virtual management (phase 1) and GDP-led active management (phase 3)?

  • How do these findings (differences in referral numbers, referral quality and the mean cost per referral between study phases) differ by the provider of secondary care?

  • Does consultant-led triage offer improved costs over GDP self-determined provider choice (phase 2 vs. phase 3)?

  • How do the views and experiences of patients differ between those using primary and secondary care services?

• Chapter 5 , Active referral management with consultant and general dental practitioner triage: qualitative findings (phases 2 and 3)

  • The use of the ITS methodology with robust adjunct and parallel qualitative components enables these issues to be addressed from both a metric and a narrative perspective.

  • What are the issues encountered when establishing a new primary care oral surgery service?

  • What are the views of stakeholders on the development and implementation of the primary care service?

  • What are the views of service users on the quality of service they received from the referral management and triage system?

Public and patient involvement

Public and patient involvement has been a key element of this work, from the design stage, in which consent and patient information sheets were reviewed and revised, through to the extensive involvement of patients (see Appendix 2) in the qualitative component of the research. Patients’ voices are heard and reflected strongly in the current work, as their views and experiences are key to meeting the aims of the research. Service redesign impacts multiple stakeholders and, although professional views are often heard, we have sought to ensure that those of service users in Sefton are recognised and reflected.

Introduction

To manage demand, focus services on need and ensure that patients are seen in the correct setting, a gatekeeper function has been introduced by services within both oral surgery pathways and the wider NHS, for example in dermatology services. 17,39 Such gatekeeper services vary in their design and implementation in terms of what is assessed, and by whom, and how it is delivered, but all can be considered a form of clinical triage. The concept of primary care services underpinned by effective clinical triage has been advocated by NHS England in its commissioning advice to ATs, although there is little detail on how this might be achieved. 1,6

The provision of Level 2 services within primary care is predicated on the safe and efficient diversion of suitable patients to such services. This can be achieved in several ways, such as by:

• undertaking a face-to-face clinical assessment of the patient

• enabling referrers to select determine the case complexity

• using machine learning or algorithms to classify the case complexity

• clinical assessment of the referral – remote clinical triage by consultant staff.

There are advantages and disadvantages to each of these approaches. The use of face-to-face clinical assessments would be expensive, would require significant expansion of the clinical workforce, estate and support staff and would delay the patient journey by introducing an additional step. These factors seem at odds with the primary drivers for introducing Level 2 services and, hence, such an approach could not be recommended. 40

Referrer-based triage makes broad assumptions that the population of referring dentists can adequately assess case complexity, has a good knowledge of the various complicating factors that can affect patient care and understands the local directory of services available for individual procedures. However, such an approach is relatively inexpensive and ensures rapid referral to a provider without a delay to the patient journey. It preserves and enhances clinical autonomy, and it can be argued that dentists know their patients best, as they have had the benefit of making a full clinical examination. However, GDP decision-making may be biased and influenced by patient demands (e.g. for GA or sedation) or by a wish to continue to send patients to ‘known’ consultants or to support colleagues in primary care. Of course, the patient could be allocated to an inappropriate service, leading to either a failure to reduce the burden in secondary care or the need for an onward referral if inappropriately assigned to a primary care service. However, this approach (peer assessment) is endorsed by The King’s Fund20 and is explored in phase 3 of this study (see Chapters 4 and 5).

The use of artificial intelligence, or algorithmic triage, is in its infancy. A number of systems have been assessed in emergency medicine departments and for trauma management. 41 Such systems generally support clinical decision-making rather than taking full control over the process of triage. In dentistry there have been some studies that have examined machine learning for treatment planning. 42 The development of an algorithm that captured, via the referral process, key elements of the pathway would be a simple matter – that is, identify those ‘red flags’ that would indicate a Level 3 patient. However, GDPs may be become familiar with the process and seek to circumvent it to get their patient to their preferred provider. Automated triage does have several benefits, as it:

• could almost instantly send a referral to the provider, ensuring no delay to a patient’s journey

• would be workforce neutral and, hence, capacity to deliver triage is limitless

• would be more economical as there are no costs associated with the number of referrals

• can be audited and assessed by appropriate clinicians to determine accuracy and appropriateness.

In the current work, it was decided not to deploy an algorithmic solution. The use of primary care Level 2 providers was in its infancy at the start of the study, and it was felt that the evaluation of algorithms would be more appropriate with mature services for which data were available to help support the development and assessment of automated decision-making.

It has been proposed that a hybrid of these two systems can be adopted: a system that provides the governance, clinical leadership and independence of the face-to-face approach while mitigating the costs, time and estate requirements of it. Remote clinical triage involves the assessment of standardised referral forms and appropriate attachments (most frequently dental radiographs) to undertake a case complexity assessment. Triagers are asked not to recommend a treatment plan, direct to a specific service or otherwise assess the referral, but to assign a Level 1, 2 or 3 complexity score to each referral. Once scored, an algorithm is applied to the referral to send it to an appropriate provider, and in most systems this is determined by patient choice for secondary care and geographical proximity for primary care Level 2 services (usually based on home postcode).

Remote clinical triage has been utilised in several settings43 and for various clinical disciplines, but the efficiency of this approach for oral surgery referrals based on the described case complexity assignments has not been assessed. 44,45

The use of consultant-level triage was considered as a ‘reference standard’, and designating an experienced clinician to lead the service was thought, by many, to ensure safe and appropriate triage in the absence of established pathways. The use of consultant-led (if not actually delivered) triage is also recommended in the current NHS oral surgery commissioning guides. 1

Consultants involved in the triage process will be aware of appropriate clinical guidelines (e.g. NICE guidance on the removal of third molars35) and the type of procedure and anaesthetic requested. This guidance, along with further obligatory information, such as medical history, social demographic information and levels of anxiety, will be common factors driving decision-making. The referral trajectory, however, involves GDPs initially carrying out a consultation and examination with the patient face to face, deciding there is a need to refer the patient for specialist treatment, then entering the appropriate referral data. Consultants then interpret the referral data and decide on the appropriate level of specialist care.

There are a number interactions taking place among individuals and organisations that may be subject to other influences and drivers, additional to official guidance and system parameters. For example, ‘intuition’ of clinicians is frequently cited as a factor involved in triage decision-making;46 however, this is a phenomenon that is difficult to define. 47 Considine et al. 48 state that knowledge and experience influence triage nurses’ decision-making; the integration of factual knowledge (a series of facts relating to a patient), procedural knowledge (decision rules, clinical guidance) and conceptual knowledge (assimilation of prior knowledge and new information) result in a unified comprehension that is applicable to a range of situations. Clinician experience is defined as a combination of the passing of time and gaining skills and exposure to an event. Together, the combination of knowledge and experience may form the elusive ‘intuition’ that is observed in experienced consultants, particularly as focusing on a specialty affords more opportunity to develop conceptual knowledge through exposure to events in a specific area.

Clinicians undertaking triage may consider several factors, in addition to clinical information and guidelines, in assessing a referral. Most literature around triage decision-making is based in acute emergency settings when the patient is present and available for examination. 41,46,49 Edwards44 investigated decision-making using telephone triage in accident and emergency departments and concluded that experienced nurses considered several additional factors, such as contextual information and risk minimisation, in addition to purely medical information, when making decisions around treatment. In addition, lifestyle factors were found to be important in orthopaedics when deciding whether or not patients should be recommended for planned total joint replacement. 47,50 There is, however, a dearth of research examining the complexities of remote clinical decision-making in the absence of the patient for elective treatment.

Although many consultants in oral surgery would have ‘triaged’ their own referrals – for example, a desk-based exercise to prioritise referrals or determine staff allocation – there were no data on the efficiency of a formal triage process where decision-making was based on case complexity and the diversion of patients to primary care. This study was therefore undertaken to determine the use of such triage in a diagnostic test accuracy study.

Aims

The aim of this research was to undertake a diagnostic test accuracy study of the accuracy of remote clinical triage performed by both GDPs and consultants, compared with a reference standard of face-to-face clinical consultation performed by an experienced consultant.

Recognising that clinical decision-making is a complex area, and that agreement levels between and within clinicians will vary, a qualitative element of the study sought to understand the reasons for this by examining the impact of variation on the feasibility of such services, and suggesting how remote clinical triage may be improved or enhanced.

More specifically, we sought to answer the following research questions:

• How do remote clinical triage outcomes conducted by an experienced consultant compare with outcomes of face-to-face examination (reference test) performed by the same consultant?

• How do remote clinical triage outcomes performed by GDPs and different consultants compare with outcomes of face-to-face examination performed by an experienced consultant (reference test)?

• What are the views of triagers on the benefits and problems of a remote clinical triage system and how can the system be improved based on their experiences?

Methods

Ethics approval was sought and gained from NHS National Research Ethics Service, London Fulham Committee, approval number 12/LO/1912. The research was divided into two main stages:

• The first stage was an assessment of remote triage versus face-to-face clinical assessment.

• The second stage examined the use of different examiners in the assessment and triage of referrals.

A qualitative component featured in both stages. Patients recruited to the study were > 18 years of age, able to consent and had been referred to a secondary care facility by their GDP for an oral surgery procedure. The setting where the study took place was Greater Manchester, as NHS Manchester had recently implemented remote clinical triage as part of a centralised referral management system.

Stage 2: examining the use of different examiners in the assessment and triage of referrals

In the second stage of the diagnostic accuracy and ‘workability’ test, the paper referrals of the cases that had been assessed face to face were provided to four other clinicians (one further consultant in oral surgery, one consultant in OMFS and two GDPs with experience in oral surgery). Each was asked to undertake a triage with the same options as described above. The decisions from these groups were compared with the reference decision (face to face) performed by a single consultant in oral surgery (PC). Although it was intended that this study would utilise consultant-led triage, we felt that it was important to explore if there are any major discrepancies between consultants and other dental professionals in where they believe certain cases should be referred to.

The sensitivity and specificity of the clinicians’ decisions were tested against the original consultant’s face-to-face examination decision. Additionally, tests for paired sensitivities were undertaken to determine the equality between the additional examiners using the median sensitivity score. The participant flow diagram is shown in Figure 1.

FIGURE 1.

Participant flow in the diagnostic test accuracy study.

For the qualitative element, where discrepancies in the decision-making between examiners were identified, referrals were selected for case presentations to be discussed in two focus groups. For the composition of the focus groups, see Table 2.

TABLE 2 - Focus group participants

Focus group		Anonymised code for transcript
1	2
Facilitator 1	Facilitator 2	F1/F2
Consultant oral surgeon (reference standard clinician)	Consultant oral surgeon (reference standard clinician)	C1
Consultant oral surgeon	Consultant oral surgeon	C2
GDP (> 25 years’ experience)	GDP (> 25 years’ experience)	GDP
Dentist with specialist oral surgery contract	Dentist with specialist oral surgery contract	SPD
Research team project manager	Newly qualified dentist (3 years)	NQD1
	Newly qualified dentist (2 years)	NQD2

1. Cases were selected if there was a discrepancies existed between decision-making following the reference standard face-to-face examinations and the reference consultant’s (PC) decision-making following examination of the corresponding paper referral.

2. Cases were selected based on referrals resulting in differences in decision-making between different clinicians (GDPs/consultants).

Results

A total of 551 eligible referrals were considered, of which 460 were booked into study clinics and, following consent, accounting for non-attendance and clinic cancellations, a total of 282 participants were recruited to the study. All participants required an oral surgery procedure. The mean age of participants was 42 years (± 10.2 years) and 53% were female. No adverse events arising from the face-to-face assessments were reported.

Discussion

Triage is described as:

The medical screening of patients to determine their priority for treatment; the separation of a large number of casualties, in military or civilian disaster medical care, into three groups: those who cannot be expected to survive even with treatment, those who will recover without treatment, and the priority group of those who need treatment in order to survive.

Stedman’s Medical Dictionary 56

This definition of triage still permeates the medical literature today (with findings of variation not dissimilar to those reported here), with examples from emergency medicine and military field operations. 57 However, the picture is changing, with the implementation of telephone triage on helplines such as 11144 and remote dermatology services. 39,58 Triage is not merely the selection of those who need care to survive but is becoming a tool by which the complex map of medical services can be navigated to ensure that patients receive the right type of care, in the right setting, delivered by the most appropriate clinical team with the most efficient use of resources.

This mixed-methods element of the project assessed the efficiency of remote clinical triage compared with a reference standard of face-to-face clinical assessment by an experienced oral surgery consultant. The qualitative results from the reference examiner (stage 1) suggest that consultants are willing to triage referrals to primary care, and the quantitative findings suggest the proportion of Level 2 cases is similar to that seen in the work of Kendall14 – around 70% for both remote (76%) and face-to-face (71%) assessment. The specificity of remote triage was high, and those cases that are suitable for primary care were identified as such. The rates of efficient and inefficient triage error were broadly similar, at 11.7% and 12.8%, respectively. The lead consultant (reference triager) had superior triage outcomes to the other examiners (with respect to sensitivity). However, as they had responsibility for face-to-face assessments this afforded a clear continuation of clinical decision-making processes, which was not available to other examiners. The results showed that there were no appreciable differences between the decisions of the other examiners.

The rate of inefficient triage calls is higher than that reported from the Croydon service, where the proportion of patients initially triaged to primary care (Level 2) who were subesquently referred to secondary care (Level 3) was reported to 0.5%, much lower than the 11% rate we report. However, it is important to note that the reference assessment of these patients took place in a secondary care rather than a primary care setting. It may be that an environmental impact facilitated an increase in the assessed complexity of these patients. For example, during an assessment in a primary care, treatment under GA would rarely be discussed, as it cannot be provided. Without the option of GA, the patient may elect for sedation or even treatment under local anaesthetic. 9 However, when offered the possibility of GA in a hospital they may request this option and, hence, create a demand-led uplift to their case complexity.

The rate of Level 1 cases is consistent with that in Croydon: 1% in both settings. This suggests that the majority of referrals from GDPs are appropriate. 17 This suggests that anecdotal explanations for the increase in referrals, for example that it is the result of financial incentives, are incorrect; rather, it may suggest that the increasing age and complexity of dentate patients could be a more important driver. Despite this, it should also be recognised that many GDPs are capable of delivering, and frequently deliver, Level 2 procedures, and the availability of a service may encourage such clinicians to refer rather than treat in practice. Another possibility is the presence of a structure-centralised referral system has changed GDP behaviour and resulted in a reduction in inappropriate (Level 1) referrals. This issue will be discussed in later chapters.

It is appropriate to consider the triage decisions in the context of the likely commissioning framework. It would be unusual to remove 70% of activity from an acute trust provider without causing significant destabilisation. In Greater Manchester it was the stated ambition of commissioners to reduce oral surgery flows into secondary care by 30–40%. A reduction of this size not only creates a stable system in the hospitals but also permits a range of Level 2 referrals to be seen in Level 3 environments, where they can be used for training purposes, and provides the possibility of reducing costs for the local commissioners. The costs of Level 2 care are borne by the primary care budget and not by the local commissioners’ hospital budget. If the triage service is directed to reduce inefficient triage errors (i.e. the incorrect assignment of Level 3 cases to Level 2) at the expense of efficient triage errors (the assignment of Level 2 cases to Level 3), then patient safety, service stability and training opportunities are maximised. The thresholds for such a triage service can be adjusted over time to ensure that there is a balance between capacity in each service area and that all stakeholders’ interests are recognised. The adage of caution first is appropriate, and if in doubt referrals should be ‘moved up’ to the next tier of care.

Given the complexities of patients, their presentation and the fact that a referral form is merely a summary and snapshot of a complex picture, it is not surprising that remote triage is associated with some inefficiencies. However, it should be recognised59 that treatment planning variation is seen across medical disciplines even when clinicians can physically examine patients, and it is not a phenomenon purely of remote assessment. Bader and Shugars60 said of dentists’ clinical decision-making:

Even when differences in patients are controlled, variation in dentists’ clinical decisions is ubiquitous. While its consequences remain undetermined, the variation in basic clinical decisions such as caries diagnosis signals the need to consider the extent to which the appropriateness of care is affected.

Bader and Shugars60

The qualitative assessment sheds useful light on the process of identifying elements that are either intrinsic to clinical behaviour or can be changed and improved to enhance the triage system’s performance. There is a clear recognition that triage can only be as good as the information provided by the referrer, and the clinicians involved in the study were often frustrated by the lack of information or, when information was provided, its quality. Ambiguous terms such as ‘bleeding problems’ or poor social histories caused the triagers to send more cases to secondary care based on lack of confidence rather than a firm decision. Although not optimising the system for identification of all Level 2 cases, the results of the study support the safe operation of a triage service and places patients’ interests first. Given the potential high percentage of cases that are available for diversion, it seems reasonable that some marginal Level 2/3 cases are sent to Level 3 – if only to support training.

The qualitative data make it clear that radiographs, accurate clinical data and medical history are vital to accurate decision-making. The findings highlight potential issues with patients not disclosing sensitive information to GDPs, for example around alcohol and mental health, or GDPs not asking for fear of alienating patients. Patients attending the face-to-face assessment in a hospital environment may have disclosed these issues more readily – or it could simply be they are seeing a new clinician, and one they do not have an enduring and continuing relationship with. Evidence suggests that clinicians vary in their ability to get full disclosure from patients. 61 Consideration should be given to GDPs having access (with appropriate permission to view) to GP summary care records for the purposes of medical and drug history confirmation. Triagers found the IOSN to be helpful in their decision-making process. The IOSN combines medical, treatment and patient factors into a single, consistent one-page referral form. When completed correctly, the IOSN has been shown to be predictive of patients’ needs for sedation. 62

Individual differences in triagers’ experience, training and setting inevitably impact on triage decision-making. Although the variances are not surprising, the underlying reasons behind them seem easier to address for triage than those for full treatment-planning variances. The triager’s role is to assess case complexity, not to design a treatment plan for the patient. The desire for a holistic evaluation of the patient is to be commended, but may also undermine the triage process as clinicians try to second guess the patient’s likely presentation following triage. It is also clear that, although many hospital staff will undertake internal triage, this form of case assessment is different. Consideration should be given to calibration of triagers and the formation of triage groups that can be internally benchmarked and externally audited. The use of triage guidelines and case examples would be helpful.

It should also be noted that these referrals were all seen and accepted by secondary care (to ensure a consistent face-to-face reference assessment was undertaken by PC). There was, therefore, no opportunity for the triage assessors to reject the referral requesting further information or clarification. The use of such a system, where additional information can easily be requested and added, would seem to address many of the concerns expressed by the triagers who felt that their accuracy in decision-making might have benefited from opportunities to ask further questions of the referrer.

Summary of main findings

In this section of the project, we posed three research questions:

1. How do remote clinical triage outcomes conducted by an experienced consultant compare with outcomes of face-to-face examination (reference test) performed by the same consultant?

There was substantial variance between triage decisions and face-to-face examination decisions. The default position was to refer to secondary care if there was a query or uncertainty over a referral.

1. How do remote clinical triage outcomes performed by GDPs and different consultants compare with outcomes of face-to-face examination performed by an experienced consultant (reference test)?

The performance of the various clinicians was very similar, and differed from the reference triager. Experienced GDPs would be more likely to refer to primary care (Level 2) than consultants.

1. What are the views of triagers on the benefits and problems of a remote clinical triage system and how can the system be improved based on their experiences?

The qualitative work highlighted the complexity of the decision-making process important of providing complete information in the referral forms and high-quality radiographs to aid decision-making. The availability of specialised equipment in primary care was an impediment to referral to Level 2 services. Experienced consultants and newly qualified GDPs were more likely than an experienced GDP to err on the side of caution and select secondary care as an appropriate treatment destination. Clinicians seemed to find it difficult to make objective decisions solely on the information provided without attempting to create a holistic picture of the patient. Suggested improvements to the system based on the findings are set out below.

Conclusions and implications

The clinical triage system is imperfect but fit for purpose, recognising that a pragmatic clinical decision-making process is utilised. Although consultants and experienced GDPs performed at a similar level, the use of consultants to triage ensures that there is an acceptable level of governance in the approach, especially in embryonic schemes if there has not been sufficient time or experience to develop cohesive guidelines and to refine and reflect on processes to ensure that they are delivering the objectives of the system. There is a clear emphasis on patient safety embedded within the approach.

This part of the project identified several elements that were incorporated into the electronic referral system. The learning from this part of the project may help to inform the development and design of any oral surgery referral service. Key elements are described below.

The use of an electronic referral form enables a number of issues to be addressed. For example, forms cannot be submitted until mandatory elements are completed. Although it would be impossible to assess the value or accuracy of the submitted information, it does prevent the ‘blank box’ issue described by the triagers. Electronic referrals can also be quickly rejected by triagers and returned to the referring dentist for correction, amendment or the addition of further information. Such two-way, rapid communication between triagers and referrers could also promote the provision of clinical advice, which may be of value when interpreting the impact of medical and drug history on a patient’s outcomes. The increase in patients on polypharmacy makes such a system important for the future. Electronic referral forms enable conditional input to be facilitated, ensuring that referrers enter only information relevant to their patient. Electronic referral systems also lend themselves to aggregation of the information in electronic referral forms into databases to provide a health needs assessment tool for commissioners to understand the needs of the population referred for care.

The provision of high-quality and diagnostic radiographs should be a mandatory part of the referral process. Such images not only facilitate triage, but should also be deployed in the treatment-planning process. The accurate assessment of treatment complexity increases efficiencies of services by enabling ‘see and treat’ appointments. High-quality radiographs provided at referral stage reduce the need for repeat imaging in referral treatment centres, sustainably decreasing appointment times and complying with radiology guidelines. 63

The incorporation of the IOSN tool into referrals seems to add benefit. Although the IOSN tool is not designed to either restrict or promote access to sedation, the data provided were viewed as useful not only by triagers but also by treating clinicians. 62 The IOSN enables assessors to determine if the drive for sedation is based purely on patient anxiety or if it is a combination of medical complexities or treatment burden. 55 This promotes the concept that sedation is not simply for those who are anxious, but may add quality to a service if a procedure is time-consuming or unpleasant. 9

Referral management systems typically end at the disposal of the referral to the indicated provider. Perhaps the inclusion of whole-journey monitoring would assist in refining the triage process – that is, the provider could provide outcomes on the quality of the referral, the accuracy of the triage and whether or not additional information (and of what type) may have helped achieve better care. Such data could inform the development of referral forms and guidance, monitor the performance of individual triagers and identify individual practices or practitioners from which or from whom poor-quality referrals are frequently sent.

Chapter 3 describes how these recommendations have been implemented in an all-electronic referral management system for oral surgery referrals for a defined population.

Introduction

This first stage of the study took place in the calendar year 2014 (1 January through to 31 December 2014). It involved the establishment and provision of an electronic referral management system with no active triage – that is, referrals were simply captured by the system and then passed straight on to the provider of choice of the referring dentist. This phase of the study is identified in the study overview diagram in Table 1.

This phase of the research was conducted in the geographical region previously covered by Sefton PCT. This site was selected for the research as it offered the following:

1. no current referral management system of any kind for GDPs

2. no current primary care oral surgery service

3. patient access to three hospital types –

   1. hospital A – a busy DGH with oral surgery and maxillofacial services

   2. hospital B – a large regional foundation trust offering extensive dental services including tertiary referrals for oral surgery and maxillofacial services

   3. hospital C – a dental school hosted by a foundation trust providing both service and educational elements

4. support from local commissioners to adopt and host the study as well as commitment to undertaking a procurement of a primary care service in years 2 and 3.

Sefton is in the north-west of England within the Merseyside region and was formed by the merger of Bootle and Southport in the mid-1970s. Sefton has a population of approximately 270,000 with a mean deprivation score of 25.7 (England’s best score = 5.7 and worst score = 42.0). 64 The health and lifestyles of adults in Sefton are generally better than the regional (north-west) average and similar to England’s average. Adult oral health for England is described in the 2009 Adult Dental Health Survey;8 however, data were not provided at the PCT/local authority level. The picture of oral health for the north-west region shows that the number of people with no teeth is decreasing but around one-third of all adults have decayed, untreated teeth. 8

Dental services in Sefton reflect the national picture of primary, community and secondary care services. A total of 34 dental practices in Sefton hold a NHS contract for either GDS or PDS provision. Those with PDS provision are typically orthodontic practices. A few practices have small NHS contracts and offer most of their adult services privately but may see children under the NHS (so called child-only contracts). Community dental services provide special care dentistry to adults and specialised paediatric services to children.

Aim

In this phase of research, the aim was to establish a passive electronic referral system (without deflection or presence of a Level 2 provider) and evaluate the impact on oral surgery services and key stakeholder groups in Sefton. We sought to answer the following research questions:

• What are the practical issues for the NHS in introducing an all-electronic referral system from scratch?

• What is the effect of an all-electronic referral system on –

  • the total number of referrals?

  • the quality of referrals including an assessment of compliance with national referral guidelines?

  • the time taken to complete referrals?

• What are the views of key stakeholders on the benefits and problems of the electronic referral system and how can the system be improved based on their experiences?

Data from this first phase of implementation were used to undertake a health needs assessment of referrals from the Sefton area to inform commissioning for the introduction of a primary care (Level 2) service in phase 2 of implementation (reported in Chapter 4).

Description of the electronic referral system

The intervention in phase 1 of a complete centralised referral management system for Sefton was the provision of an electronic referral management system to capture oral surgery referrals from primary care dentists. Early in the design phase it was decided to include oral medicine and oral maxillofacial referrals in the referral system to make the system more attractive to users (a one-stop shop), but these referrals did not form part of the study data set.

The referral system was a zero footprint (no software needed to be downloaded) solution that was developed at NHS Trafford during the pilot phase of this work. The application runs in the Microsoft.NET Framework and was accessed via a secure URL (uniform resource locator). To ensure that patient data were protected, and to meet the governance guidelines for such services, the application was hosted by an approved N3 hosting service and access required the users to enter a username, strong password and then a second one-time password (OTP). The OTP was delivered by text message or voice call. The OTP system provides an additional level of security, as passwords cannot be lost, disclosed to others or hacked.

The application could be accessed by any modern browser and the end users needed only a computer and broadband connection to use the system. Dental radiographs and other files could simply be uploaded and attached to the referral information. Once completed, referrals were transformed into a PDF (portable document format) package with all the relevant information. These could then be viewed and assessed by designated triagers (who accessed the same system) and then sent onwards to providers using NHS NET e-mail.

The system provided a range of administrative functions for both referrers and triagers, and each referral could be tracked using the unique reference number (URN) produced when a referral was submitted. This tracking system was available to both patients and referrers.

The system provided a referral view page that enabled a referrer to view a range of elements related to their referral, including the status and any comments made by the triagers (case notes system). Such comments could be replied to, referral information could be quickly edited and the referral could be resubmitted for further assessment.

Once a referral was assessed as completed, the provider was sent a NHS NET e-mail that contained the PDF wrapper, individual files (for example original images) as well as, within the body of the e-mail, the referrer’s details and the nature of the treatment requested. The software, database and installation instructions are also available for NHS users under an open-source licence.

The referral form reflected the learning from the diagnostic test study described in Chapter 2. We ensured that an IOSN form accompanied any referrals for which sedation was indicated, enabled the uploading of high-quality radiographs (we supplied radiograph scanners to those practices that did not have digital radiographs) and ensured that fields with mandatory information had to be completed prior to submission. The medical history form was comprehensive and sought to identify those elements that would be important for both case complexity assessment and treatment provision. A copy of the referral form is shown in Appendix 3.

All dentists were asked to use the referral management system from 1 January 2014 for all oral surgery, oral medicine and oral maxillofacial referrals.

Methods

The study used a mixed-methods approach using quantitative data collected by the system and contemporary HES data. The qualitative element collected data using multiple methods from representatives of key stakeholder groups.

All NHS dental practices operating in the study area, plus the community dental service, were approached (n = 34) and invited to provide feedback on implementation of the system. All practices were offered the system and provided with login details, but the following were excluded from the analysis:

• one emergency dental service that did not refer patients onwards

• five private practices that used a private oral surgeon and did not refer to NHS services

• one orthodontic practice that did not refer directly to oral surgery.

Therefore, the total number of practices available to contribute to the study was 27.

Results

Results are provided for 2014 (phase 1) only. Chapter 4 provides comparative results across all three phases of the implementation of the centralised referral system following the ITS design.

Discussion

This first stage of the referral project saw the implementation of a passive referral management system that sought to capture all oral surgery referrals from dental practices in the Sefton area. The data suggest that the implementation was a success, with only three practices within the included cohort failing to refer in that period, two of which were practices owned by corporate bodies. It is not clear how oral surgery cases were handled by these practices. They could have been referred internally, referred privately or referred by letter to circumvent the system, or a referral could have been delayed.

Summary of main findings

Implementation of triage without deflection enabled us to evaluate the effects of a mandatory electronic data capture of referrals alone. It also enabled us to explore some of the barriers to, and facilitators of, successful early implementation and adoption of the new system. In this section of the project, we posed a number of research questions:

• What are the practical issues for the NHS in introducing an all-electronic referral system from scratch?

This phase of the project showed that, with the full backing of commissioners and senior consultants in acute trusts, an electronic referral system can be successfully established within a short time period. Clear communication is the most important factor, along with a clear message that there is a universal, mandatory requirement to use the new system. Timely, pragmatic support for dental practices is also important; for example the provision of, and support in using, radiograph scanners.

• What is the effect of an all-electronic referral system on the total number of referrals?

The introduction of the new system seemed to be associated with a fall in the total number of referrals, probably because of the greater inconvenience of providing more detailed clinical information. This fall could have resulted from a reduction in Level 1 referrals (although without comparable baseline data it is not possible to verify this). The introduction of an electronic referral system alone (without deflection) does not produce the efficiencies wanted by commissioners. The majority of referrals were for Level 2 cases and ≈80% in hospitals A and C, but for some reason only ≈50% in hospital B.

• What is the quality of referrals including an assessment of compliance with national referral guidelines?

The completeness of clinical information and the quality of the information content of referrals were markedly improved by the introduction of the referral system, particularly as it was a mandatory requirement to complete all fields of the standardised referral form. A very small proportion of referrals were Level 1 (inappropriate) referrals.

• How much time was taken to complete referrals?

Clinicians reported that, once they were familiar with the system, each referral took about 5 minutes.

• What are the views of key stakeholders on the benefits and problems of the electronic referral system, and how can the system be improved based on their experiences?

Lack of computer literacy, financial implications of computerisation and corporate priorities were significant barriers to implementation. The most important factor that supported uptake and sustained use was having a designated person(s) in each practice who had a clear understanding of the reasons for introducing the new system with the authority to implement the change. Clear communication was also critical in explaining the processes involved, the reasons for introducing the system and the benefits it would bring.

Conclusion

This phase of the study showed that rapid implementation of an electronic referral system for a defined population is possible with the full commitment of commissioners. Most practices quickly adopted the system, often employing their own pragmatic solutions to support implementation within the specific context of their workplace. The system produced an immediate improvement in quality of referrals, provided valuable information to support health needs assessment and may have suppressed inappropriate referrals to some degree, but a casual relationship between implementation and a fall in the number of referrals was difficult to demonstrate.

This qualitative analysis has identified several barriers to, and facilitators of, early implementation of an electronic referral management system. The findings offer pragmatic guidance in the early implementation of similar systems in general dental practices. Although this analysis focuses on one specific area of implementation, the findings from several discrete qualitative studies at each phase of the study can be used to carry out a normalisation process evaluation that encompasses a more holistic theory of implementation.

In Chapter 4, we explore the impact of the introduction of the primary care service, the active direction of referrals into primary care and the resultant impact on services and costs.

Introduction

The second and third phases of the study took place in calendar years 2015 and 2016. Following an appraisal of the health needs assessment, the local commissioning team proceeded with an AQP procurement for a primary care (Level 2) oral surgery service to cover the Sefton geographical footprint. An AQP procurement is a threshold process that enables providers to demonstrate that they meet the requirements of a service specification produced by the service commissioner, and, hence, are awarded a contract, usually at a set price per procedure and without any guarantee of minimum activity. The service specification for the procurement was informed by the health needs assessment in terms of likely clinical activity, likely numbers of patients and the geographical location of patients. Following this process, a single provider met the criteria and two physical locations were identified in Sefton to cover the geographical spread of referrals.

In phase 2 (2015), referrals were actively triaged by consultants in oral surgery and OMFS, and in phase 3 (2016) GDPs could select their own preferred providers (i.e. self-triage). In phase 2, referrals that were deemed to be Level 1 were sent to the new (AQP) Level 2 provider. There was no change in the services offered by the three secondary care services. The primary care (AQP) provider offered the full range of Level 2 services, except for apicectomies (surgical endodontics) and soft-tissue surgery requiring pathology services.

In phases 2 and 3, we undertook an economic evaluation, looking at case-by-case costs for patient management in appropriate settings compared with phase 1 (no deflection). The purpose of the evaluation was to examine if there are cost savings from ‘active’ triage of referrals for minor oral surgery. The NHS has developed tier 2 services in many specialties with the aim of improving care and reducing costs, usually without an evaluation of the costs and effects of the new service. The danger is that a new Level 2 service may improve the service but may also increase costs substantially by introducing a new service and failing to recover resources from secondary care. In this study we had the opportunity to undertake a robust health economic analysis to assess the impact of both consultant-led (phase 2) and GDP-led active triage.

Changes to intervention in phases 2 and 3

Using the same electronic referral system as phase 1, the final two phases of the study used ‘active’ triage to divert non-Level 3 activity to a single primary care (Level 2) service in the Sefton area. The new service started taking referrals from the electronic referral system at the start of phase 2. The same consultant-led triage team was utilised in phase 2. During phase 3, the referral form was modified to enable the referring GDPs in Sefton to select either a primary or secondary care destination for their referral. The decision to add GDP assessment of referral direction was based on The King’s Fund recommendation for peer assessment and the need to reflect local capacity for consultant triaging. 20 Each referral was also assessed by the consultant triage team as an audit of the GDPs’ decisions but the GDPs’ referral decisions were respected, except for those referrals deemed to be suitable for a 2-week wait appointment.

Aims

The overarching aim of this part of the study was to measure the number of referrals, referral quality and costs of an electronic referral management and triage system, allied to provision of a specialist primary care service. More specifically, the following research questions were addressed:

• What are the differences in the number of referrals, referral quality and the mean cost per referral between virtual management (phase 1) and consultant-led active management (phase 2)?

• What are the differences in the number of referrals, referral quality and mean cost per referral between the year of virtual management (phase 1) and GDP-led active management (phase 3)?

• How do these findings (differences in the number of referrals, referral quality and the mean cost per referral between study phases) differ by the provider of secondary care?

• Does consultant-led triage offer reduced costs over GDP self-determined provider choice (phase 2 vs. phase 3)?

• How do the views, and experiences, of patients differ between those using primary and secondary care services?

Methods

In phase 2, each referral entered into the system was triaged by a member of the consultant team to either Level 1, 2 or 3, using the same decisions as those described in Chapter 3. In phase 3, GDPs used the electronic system to decide if their patient should be referred to either to either a Level 2 or 3 service. Further assessment of referrals to determine ASA status, OCPS code and the number of teeth to be extracted was also undertaken.

Consultant triager decisions were dichotomised as primary care (Level 1 or 2) or secondary care (Level 3). Descriptive statistics were generated to describe measures of central tendency and location (mean and SD or median and interquartile range). For each phase of diversion, comparative analyses of referrals to primary and secondary care were undertaken. Categorical data were analysed using the chi-squared test and risk difference, as appropriate; continuous data were analysed using t-tests.

The ITS for phases 2 and 3 were assessed using referral data aggregated monthly for 24 months, with an outcome measure of referrals to primary care as a proportion of all referrals. We were unable to include referrals for phase 1, as all Level 1 and 2 referrals went to primary care, with perfect prediction of the outcome. The ITS model therefore reflects referrals for phase 2 (2015) and phase 3 (2016).

A segmented regression analysis was undertaken, using a heteroscedasticity- and autocorrelation-consistent variance estimate (Newey–West) with a lag of 5. The initial 12 months (phase 2) reported referrals from the consultant-led diversion, and the following 12 months (phase 3) referrals from the GDP-led diversion. Accordingly, the ‘intervention’ indicated the change from consultant- to GDP-led referrals at 12 months. Covariates in the model were the main effects of study month, intervention and an interaction term of study month and intervention.

Patients attending for procedures in each setting were consented to receive a paper questionnaire following their surgery to collect data on patient outcomes, experiences and costs. Patients not returning their questionnaires after 6 weeks were sent up to two reminders at approximately 4-week intervals and were given the option to complete the questionnaire over the telephone with the assistance of a researcher. A £10 gift voucher was offered to those completing and returning the questionnaire. Additional consent was obtained to interview patients for the qualitative component of the study, which is reported in Chapter 5, and again to view their dental records for an assessment of complications following treatment.

Results

It should be remembered that this was a mixed-methods study with qualitative data collected in parallel with (and informed by) the quantitative data. The quantitative results are presented and discussed in Number of referrals, Referral quality metrics and Health economics, and the qualitative element of the study is described in Chapter 5.

Number of referrals

Table 13 provides an overview of trends in number of referrals for each year from 2013 to 2016 using HES data and data from the electronic referral system. HES data are presented for Sefton and two north-west PCTs for reference. The Sefton HES data show an 11% decrease in referrals following the introduction of the electronic referral system.

TABLE 13 - Oral surgery referral patterns from 2013 through 2016

RM, referral management.

Data from referral management service.

Data from routinely collected NHS Data – HES.

Year (including phase)	Location
	Sefton – RMa		Sefton – HESb		PCT 1 – HESb		PCT 1 – HESb
	Number of referrals	± %	Number of referrals	± %	Number of referrals	± %	Number of referrals	± %
2013	–	–	810	–	1480	–	1154	–
Phase 1 – 2014	670	–17	723	–11	1467	–1	1322	+15
Phase 2 – 2015	643	–20	702	–14	1665	+13	1497	+30
Phase 3 – 2016	861	+6	874	+8	1862	+26	2089	+81

Looking at the data produced by the electronic referral system, there was a decrease in the total number of referrals between phase 1 (passive triage) and phase 2, with a corresponding reduction in the HES data. The total number of referrals increased greatly between phase 2 (consultant led) and phase 3 (GDP led), and, again, this rise was mirrored in the HES data.

Comparative HES data from the two other former PCT areas over the same time show year-on-year increases in referrals in one PCT by 81% between 2015 and 2016. In contrast, there seems to have been a modest increase in Sefton referrals, which is much smaller than the trends exhibited in the other PCTs.

It appears from these data that, although there has been a modest increase in Sefton referrals, this reflects a general pattern of increasing number of referrals generally and the referral management system appears to have a suppressive effect.

The number of referrals and their final provider destination are shown in Table 14, which includes phase 1 (no active deflection) data for comparison purposes. In phase 1, although there were no actual referrals to primary care, the same consultant-led team used in phase 2 was making triage decisions. When comparing phase 2 with phase 3, there was an overall 25% increase in the number of referrals (i.e. 643 vs. 861), and the GDP referred more cases to secondary care than the consultants. Figure 2 presents the weighted proportion of referrals to primary care over the 24-month period.

TABLE 14 - Referrals to primary and secondary care

Type of referral	Phase (year), triage type
	2 (2015)	3 (2016)
	Consultant, n (%)	GDP, n (%)
Level 1	15 (2.3)
Level 2	287 (44.7)	369 (42.9)
Level 3	341 (53.0)	492 (57.1)
Total	643 (100)	861 (100)

FIGURE 2.

The proportion of referrals to primary care over phases 2 and 3.

Figure 2 shows the proportion of referrals to primary care over phases 2 and 3. Referrals in months 0–12 are the result of consultant-led triage, and thereafter of GDP-led triage. The y-axis represents the proportion of referrals to primary care. The size of the circles is proportional to the total number of referrals. The dashed line at 12 months indicates the change from consultant-led referral to GDP-led referral at the end of the month.

Before the intervention (the switch from consultant to GDP), there was a non-significant increase [odds ratio (OR) 1.02 per month, 95% confidence interval (CI) 0.98 to 1.06, secular trend, per month]. After the intervention, there was an increase in consultant-led to GDP-led referrals (OR 0.98 per month, 95% CI 0.93 to 1.03, gradual effect, per month), although this finding was not statistically significant. There was, however, an immediate effect of the intervention (OR 0.62, 95% CI 0.42 to 0.93). The odds of referral to primary care following the intervention are about 38% lower than the before the intervention.

Discussion

The final two phases of the programme saw the introduction of the AQP primary care service in Sefton with active deflection, procured based on the health needs assessment data provided by phase 1 of the study. In phase 2, referrals were triaged by the same consultants, in either oral surgery or oral maxillofacial surgery, who triaged in phase 1 (no deflection), and in phase 3 the referring GDP could select either primary or secondary care at the end of the electronic referral form.

Summary of main findings

In phase 2, we could evaluate the effect of a fully working centralised referral management system coupled to a newly commissioned primary care (Level 2) service. In phase 3, we could compare the costs and effects of triage performed remotely by consultants or by the GDPs completing the referral form for their patients. A patient questionnaire enabled us to collect data on patients’ views and experiences. This quantitative data were designed to complement a deeper qualitative assessment of patients’ views, which will be reported in Chapter 5.

In this chapter, we posed a number of research questions:

• What are the differences in the number of referrals, referral mix, referral quality and the mean cost per referral between virtual management (phase 1) and consultant-led active management (phase 2)?

The number of referrals fell slightly between phases 1 and 2, perhaps because of a delay in referral as GDPs familiarised themselves with the new service. The mean cost saving, per referral, from consultant-led triage (phase 1 vs. phase 2) was £108.23 (NHS cost perspective) or £103.92 (societal cost perspective), which in both cases is statistically significant.

• What are the differences in the number of referrals, referral mix, referral quality and mean cost per referral between the year of virtual management (phase 1) and GDP-led active management (phase 3)?

The total number of referrals increased significantly when GDPs performed the triage, and a significantly larger proportion of referrals were sent to Level 3 providers. The mean cost saving per referral from GDP-led triage (phase 1 vs. phase 3) was smaller than that from the consultant-led service [£84.13 (NHS cost perspective) or £80.28 (societal cost perspective)] and in both cases the difference is statistically significant.

• How do these findings (differences in the number of referrals, referral quality and the mean cost per referral between study phases) differ by the provider of secondary care?

There were large differences in cost savings between the hospitals, primarily because of the different approaches to activity coding in each trust. This key finding demonstrated the importance of a comprehensive understanding of the local context before commissioning a referral management service.

• Does consultant-led triage offer improved costs over GDP self-determined provider choice (phase 2 vs. phase 3)?

Yes, but these differences are small in comparison with the effect that trust activity coding has over costs.

• How do the views and experiences of patients differ between those using primary and secondary care services?

Patients demonstrated high satisfaction levels with the service and the metrics used, such as the NHS Friends and Family Test, and the level of complications were the same for each setting.

Conclusion

Primary care oral surgery services, when underpinned by a referral management service, offer suitable alternatives to secondary care for Level 2 procedures. The use of consultant-level triagers produces significantly higher levels of diversions to primary care than GDP-level triagers. Patients appear satisfied with the provision of surgery in primary care settings, and there was no difference in reported complication rates.

There are potential cost savings to be realised using primary care services, but these need to be balanced and understood against the tariff and coding behaviour of the area’s acute trusts. A thorough understanding of the patient flows, charges and procedures can be provided by a referral management service but, without commissioner input and challenge to secondary care, savings will not be realistic and could be inflated by overcharging in the acute sector.

Introduction

This chapter describes three discrete elements assessed as part of the mixed-methods approach:

1. a further exploration of implementation in the primary care service as the system incorporates live diversion to specialist primary care, including a case study of the primary care oral surgery provider

2. the views of secondary care providers and commissioners on the development and implementation of the intervention, and implications for further roll-out across the local area

3. views of service users.

Each contributes to the wider understanding of implementing the intervention and its impact. Findings from each of the stakeholders (primary care, specialist service, secondary care and commissioners) are incorporated into a whole-systems NPT analysis, which can be found in Appendix 4. This synthesises data from each source to produce an applied model of implementation.

Aims

The aim of this study was to explore the implementation of this specialist oral surgery service in Sefton and to examine issues around acceptability and workability of the new service. This part of the study used qualitative methods conducted in parallel and designed to be complementary to the quantitative methods described in Chapter 4. Three broad research questions were posed:

1. What are the issues encountered when establishing a new primary care oral surgery service?

2. What are the views of stakeholders on the development and implementation of the primary care service?

3. What are the views of service users on the quality of service they received from the referral management and triage system?

Methods

Summary of main findings

The three elements of this qualitative part of the mixed-methods design sought to address the three broad research questions that we were posed:

1. What are the issues encountered when establishing a new primary care oral surgery service?

The establishment of the service appeared to go very smoothly. After some initial minor problems because of communication deficits and a lack of understanding of the objectives of the service, the referrals received were appropriate. The need for ongoing communication about the aims of the service were highlighted. Similarly, a good relationship with secondary care and local consultants was essential to provide support and smooth operation of the system. The new service was well accepted by patients, who appreciated the convenience in terms of a local service and short waits for appointments.

1. What are the views of stakeholders on the development and implementation of the primary care service?

The foundation trust and dental hospital secondary care services felt that the new service has little impact on their workload because their catchment area was much larger than Sefton. The service was felt to have a more pronounced effect on the DGH, although this was reduced when consultant triage was replaced with GDP triage. There seemed to be a view among consultants and hospital managers that, because of the financial pressures on the NHS and the demand pressures on its services, the introduction of Level 2 services was inevitable and it was, in general, supported.

1. What are the views of service users on the quality of service they received from the referral management and triage system?

Patients accepted the services and appreciated the timeliness of appointments and the local accessibility of the service. The main requirement of patients was that their problem was dealt with quickly and effectively; the setting and who did it were secondary considerations. Patients expected clear communication about the process and viewed patient-centred care as a key indicator of the quality of the service.

Conclusions

Primary care service changes can be implemented, but timely and effective communication is important. This communication needs to continue and involve all levels of service provision for the system to work effectively. There is a need to recognise that staff turnover may require a regular updating of the directory of services available to primary care referrers.

For a referral management system to have a significant effect on large trusts with large catchment areas, the referral management needs often operate on a footprint that mirrors that of the catchment areas. Provision of timely and rapid alleviation of a patient’s problem was more highly valued than the location of the service and, hence, when commissioning primary care services, an emphasis should be placed on the provider’s ability to deliver care promptly.

Concerns regarding the acceptability of Level 2 services seem unfounded, with patients reporting high levels of satisfaction, with patient-centred care and good communication being cited as important and desirable features.

Summary of main findings

Overall strengths and limitations

The study has a number of strengths. The use of a real-world setting, with a ‘virgin’ territory in which no formal referral management or Level 2 provision had been employed, enables stronger causation assessments to be made. The availability of three hospitals serving Sefton helps generalisability to other areas. The partnership working with commissioners to enable not only the referral management system but also the incremental development of the Level 2 service ensured that the ITS approach was meaningful. The mixed-methods approach facilitates a deeper understanding of the underlying issues and impacts of service implementation and change and, when combined with the high levels of general practice engagement, provides a whole-system view. The use of hospital-specific tariff applications in the health economic evaluation permitted a more sophisticated understanding of the impacts of referral diversions than could have been achieved with the use of average costs or tariff guidance costs.

The small geographical footprint is a limitation of the study – the referral flows were largely into hospital A and, hence, the impact on the two other trusts was minor and prevented a full exploration of the potential issues for these organisations. Further research should take place on footprints that represent the capture area of those trusts studied. The use of commonly collected NHS data was problematic, and this lack of a robust counterfactual limits the casual inferences that can be made regarding systemic changes in referral behaviour. Our follow-up period could also have been extended to view the impact of staff turnover in all clinical care settings.

As with all research projects that extend over a number of years, there have been temporal changes within the dental landscape. The establishment of local professional networks (LPNs) in dentistry, aligned with managed clinical networks (MCNs), has sought to drive improvement in care pathways. There was no MCN in the Sefton area during the study, although, in the future, such networks and groups will probably be the driving force for implementing any referral system, and for supporting those GDPs requiring additional training.

What have we added to the evidence base?

This work addresses evidence gaps highlighted by recent reviews22,23 and supports findings from other dental14 and non-dental21 studies. There is a clear need for a diversion service to be introduced alongside referral management if costs savings are to be achieved – referral management alone is likely to be costly and ineffective – as found by Cox et al. 21 However, a period of non-diversion should be considered as part of a staged implementation to ensure that any additional services are commissioned using locally derived data of demand and needs. The current work provides evidence that health needs assessments can be informed by the capture and assessment of referrals in such a model.

Referral management is not a panacea – its introduction requires careful planning, communication and engagement. 38 Consideration must be made of the local secondary care environment, the tariff costs, the workforce and the needs for training.

Implications

The continuing financial pressures facing the NHS require new ways of working, and demand management plays an important role. Commissioning policy in dentistry advocates for strengthening the Level 2 provision across all dental specialties, with referral management systems ensuring that diversion to such services is safe and efficient. 1 Yet, the policy documents say little about how such services should be designed and implemented. The lack of NHS IT infrastructure in dentistry prohibits the use of the national eRS and, hence, NHS ATs must consider alternatives. When doing so, they need to have sight of the local context, the clinical leadership required to make change and the workforce to deliver it.

Further research

Grant holders and steering group

The following individuals were grant holders on the project and provided invaluable assistance to the delivery of this project.

Dr Colette Bridgman – provided strategic leadership and NHS liaison throughout the project. Dr Bridgman was instrumental the design and implementation of the Greater Manchester dental referrals system that informed much of the thinking behind the Sefton project.

Stephanie Neil (lay member) – Ms Neil provided a valued contribution to the steering group by providing information on the user and patient perspective, including the design and analysis of the patient qualitative component.

Gina Lawrence – Ms Lawrence was responsible for the commissioning of the NHS Trafford electronic referral system that was used in the current work.

Sam Illingworth – provided strategic leadership from NHS England national perspective during the study period.

External commissioners

Ben Squires and Lindsey Bowes – Mr Squires and Ms Bowes were both key to the development of the Greater Manchester referral system that provided the referrals for the diagnostic test accuracy study and were instrumental in the implementation of the system across a large geographical area. Their commissioning knowledge and support was key in the development of the service specification for the Sefton Minor Oral Surgery service. Further to the completion of this work, both were responsible for securing funding for a cloud-based PACS system, a feature recommended within the current work to optimise radiograph quality.

Diagnostic test accuracy

Each of the following individuals spent considerable time examining, assessing and commenting on referrals. Their contribution to many hours of individual and focus groups discussions is gratefully recognised.

• Julian Yates.

• Stuart Clarke.

• Ravi Singh.

• Tina Liu.

• Oral surgery nurses at Manchester Dental Hospital.

Triage team

The following individuals were responsible for triaging all the Sefton referrals that were submitted electronically. In addition to this, each provided invaluable clinical direction and suggestion during the study, including the incorporation of NHS numbers to the referral forms, escalation to 2-week wait and the effective use of supplemental triage for audit purposes.

• Anne Begley.

• Patrick Magennis.

• Rob Laycock.

• Mark Boyle.

• Tim Lord.

Research nurses

• Zena Haslam.

• Moira Morrison.

Commissioning team

Ms Roger and Dr Gough were responsible for the implementation of the system in Sefton and the commissioning of the primary care service, and attended numerous engagement events and research meetings. Their willingness to host the study, commit to a long-term programme of work and facilitate the linkages between primary and secondary care was crucial to the success of the work.

• Jean Rogers – NHS Cheshire & Merseyside.

• Lesley Gough – Public Health England.

Oversight group

Professor Brunton chaired the Oversight Group and he, and the group, provided important strategic direction and advice to the steering group to ensure the success of the project.

• Matt Sutton.

• Margaret Ogden.

• James Darcey.

• Helen Worthington.

• Anne-Marie Glenny.

Contributions of authors

Joanna Goldthorpe (Research Fellow, Qualitative Research) undertook the qualitative analysis and reporting.

Tanya Walsh (Reader, Biostatistics) undertook the quantitative analysis for both the diagnostic test accuracy and ITS components.

Martin Tickle (Professor, Dental Public Health) wrote the collated report and undertook the provision of the referral management solution and NHS interface.

Stephen Birch (Professor, Health Economics) conducted the economic evaluation and prepared the relevant sections of the report.

Harry Hill (Research Fellow, Health Economics) conducted the economic evaluation and prepared the relevant sections of the report.

Caroline Sanders (Senior Lecturer) undertook the qualitative analysis and reporting.

Paul Coulthard (Professor, Oral Surgery) undertook the triage activities for the diagnostic test accuracy and provided clinical leadership for the surgical components of the work.

Iain A Pretty (Professor, Dental Public Health) wrote the collated report and undertook the provision of the referral management solution and NHS interface.

Data sharing statement

We shall make data available to the scientific community with as few restrictions as feasible, while retaining exclusive use until the publication of major outputs. Anonymised data will be available from the corresponding author on request.

Patient data

This work uses data provided by patients and collected by the NHS as part of their care and support. Using patient data is vital to improve health and care for everyone. There is huge potential to make better use of information from people’s patient records, to understand more about disease, develop new treatments, monitor safety, and plan NHS services. Patient data should be kept safe and secure, to protect everyone’s privacy, and it’s important that there are safeguards to make sure that it is stored and used responsibly. Everyone should be able to find out about how patient data is used. #datasaveslives You can find out more about the background to this citation here: https://understandingpatientdata.org.uk/data-citation.

Disclaimers

This report presents independent research funded by the National Institute for Health Research (NIHR). The views and opinions expressed by authors in this publication are those of the authors and do not necessarily reflect those of the NHS, the NIHR, NETSCC, the HS&DR programme or the Department of Health and Social Care. If there are verbatim quotations included in this publication the views and opinions expressed by the interviewees are those of the interviewees and do not necessarily reflect those of the authors, those of the NHS, the NIHR, NETSCC, the HS&DR programme or the Department of Health and Social Care.

Level 1 procedures/conditions

• Extraction of erupted tooth/teeth including erupted uncomplicated third molars.

• Effective management, including assessment for referral unerupted, impacted, ectopic and supernumerary teeth.

• Extraction, as appropriate, of buried roots (whether fractured during extraction or retained root fragments).

• Understanding and assistance in the investigation, diagnosis and effective management of oral mucosal disease.

• Early referral of patients (using 2-week pathway) with possible premalignant or malignant lesions.

• Management of dental trauma, including reimplantation of avulsed tooth/teeth.

• Management of haemorrhage following tooth/teeth extraction.

• Diagnosis and treatment of localised odontogenic infections and post-operative surgical complications with appropriate therapeutic agents.

• Diagnosis and referral patients of with major odontogenic infections with the appropriate degree of urgency.

• Recognition of disorders in patients with craniofacial pain including initial management of temporomandibular disorders and identification of those patients who require specialised management.

Level 2 procedures/conditions

• Surgical removal of uncomplicated third molars involving bone removal.

• Surgical removal of buried roots and fractured or residual root fragments.

• Management and surgical removal of uncomplicated ectopic teeth (including supernumerary teeth).

• Management and surgical exposure of teeth to include bonding of orthodontic bracket or chain.

• Surgical endodontics.

• Minor soft-tissue surgery to remove apparent non-suspicious lesions with appropriate histopathological assessment and diagnosis.

Level 3 procedures/conditions

• Procedures involving soft/hard tissues where there is an increased risk of complications (such as nerve damage, displacement of fragments into the maxillary antrum and fracture of the mandible).

• Management and/or treatment of salivary gland disease.

• Surgical removal of tooth/teeth/root(s) that may involve access into the maxillary antrum.

• Management of temporomandibular disorders and craniofacial pain that have not responded to initial therapy.

• Treatment of cysts.

• Management of suspicious/non-suspicious oral lesions.

• The placement of dental implants requiring complicated additional procedures such as bone grafting, sinus lifts, etc.

• Treatment of complex dentoalveolar injuries.

• Management of spreading infections and incision of abscesses (or abscess) requiring an extra-oral approach to drain.

Referral form

(PDF download)

Medical history form

(PDF download)

Index of Sedation Need form

(PDF download)

Prior assumptions and experience:

Having worked as a researcher in dentistry, but not as a practitioner, I felt my experience was more aligned to that of the patients, rather than the NHS staff and commissioners. I had some prior experience of working with maxillofacial surgeons in the context of pain research, but had little prior knowledge of any professional allegiances or dental care pathways from primary to secondary care. In addition, having never been referred to secondary care dentistry and little experience beyond childhood of secondary care NHS services in general, most of my assumptions and experience was based on being an adult patient in primary care dental surgeries, and anecdotes received from friends relating to dental care and dental anxiety. I am not an anxious patient myself and have never had an adverse experience at the dentist (I find it neutral experience), however I was expecting fear of the dentist to be mentioned more often in interviews than was realised.

Awareness of social setting and the social ‘distance’ between the researcher and the researched

Awareness of wider social and political context

The researchers conducting interviews did not experience attempts to ‘hijack’ the research to elaborate on professional or personal agendas. However, when issues arose that were outside the parameters of the interview, perhaps raising clinical issues or those around service delivery (such as those around clinical guidelines for bisphosphonates), researchers fed back information to the study PI [principal investigator] who then arranged stakeholder meetings to address issues raised. This process appeared to be effective and allowed for iterative development of the intervention and service being researched.