Impact of changing provider remuneration on NHS general dental practitioner services in Northern Ireland: a mixed-methods study

The evolution of a new NHS dental contract in England

By the mid-1990s, difficulties in accessing dental services for NHS patients in England were becoming a political issue and it was increasingly recognised that reform of the NHS contract was necessary. As highlighted by the Bloomfield Report,26 the ‘system of remuneration for GDPs seems to have an inherent leaning towards instability which threatens to undermine the commitment of dental practitioners to the NHS’. In 1997, the NHS (Primary Care) Act enabled the voluntary establishment of personal dental services (PDS) pilot schemes to explore alternative ways of delivering NHS dental services. 27 A key feature of these new contracts was how they were tied to local issues around the need for and access to care through contracting arrangements with NHS primary care trusts (PCTs) in England. For NHS GDPs, the new PDS contracts offered greater flexibility for the provision of services and there was less emphasis on the volume of activity provided. Instead, NHS GDPs were paid on a per-capita basis and rewarded for improving access to care and maintaining oral health for those patients who were already registered. The net effect of paying the PDS schemes was a dramatic reduction in NHS service activity, a fall in the provision of complex treatments and a loss of PCR as a consequence of the fall in activity. 28 The latter development was particularly worrying for the government as patient charges made up a substantial proportion of the budget for NHS dentistry. However, many viewed the PDS pilots as a success, given the focus on addressing the needs of the local population. 28,29

At the turn of the new millennium, ‘Modernising NHS Dentistry’ was introduced in England. 30 Improving access was considered to be the most important policy objective for the government and the new proposals gave PCTs powerful new commissioning tools to improve access to NHS dentistry, to increase the provision of preventative services and to monitor the performance of GDPs. This was further emphasised in ‘Options for Change’ in 2002. 29 However, a tension remained for the government: how could they concomitantly improve access to care and increase the provision of prevention while maintaining PCR, which was reliant on ensuring that activity levels did not fall?

In 2006, a new NHS dental contract was introduced in England and Wales, organised around a local commissioning model. 31 This contract paid GDPs according to activity categorised into three broad bands, rather than the 400 individual treatment items in the Statement of Dental Remuneration, and a new contract currency, units of dental activity (UDAs), was introduced to set activity targets and pay GDPs according to the activity they provided. A band 1 course of treatment attracted 1 UDA and included an examination, radiographs and a simple scale and polish. Band 2 courses of treatment attracted 3 UDAs and included restorations, extractions and root canal treatments, whereas more complex crowns, bridges and dentures attracted 12 UDAs as a band 3 course of treatment. Patient charges were also simplified so that there were only three levels of payment tied to each band of treatment.

For the first time, contracts were raised with practices rather than with individual GDPs. Contracts were held by practice principals (PPs) (equity owing), now referred to in the NHS as providers, and non-equity-holding associate dentists (ADs) were labelled performers. Practice owners raised internal contracts with each of their associates based on a price per UDA delivered. To reflect variation in earnings and activity among NHS practices, the value of a UDA was calculated for each practice individually based on the practice earnings during a reference period, although, for most, this meant that 1 UDA was worth between £20 and £25. Annual activity targets were also set in the form of UDAs required per year, based on historical activity produced during the reference period. One feature of the 2006 NHS contract was that GDPs would receive 3 UDAs for a band 2 course of treatment (i.e. paid the same) if they provided one restoration in a single visit or multiple restorations over a number of visits. In a similar manner, NHS GDPs were also allocated 12 UDAs for one band 3 course of treatment irrespective of whether they provided one crown or multiple crowns within one course of treatment.

A key feature of the 2006 contract was cost containment [i.e. GDPs’ NHS annual activity and revenue were capped at an agreed number of UDAs per year for an agreed price, known as an annual contract value (ACV)]. The NHS GDPs were then paid one-twelfth of their ACV on a monthly basis. As a result, NHS GDPs’ outputs under the new contract in England were constrained and they were penalised if they underperformed (< 96% of their ACV) or overperformed (> 102% of their ACV). Patient registration in the 2006 contract also ceased to exist, along with the contractual responsibility for NHS GDPs to provide continuing care and emergency care for their patients. Instead, this latter responsibility was devolved to PCTs along with the planning and securing of NHS dental services for their locality.

The effect of the 2006 NHS contract change in England was investigated by an earlier National Institute for Health Research Health Services and Delivery Research programme funded study (08/1618/158). Large and abrupt changes in the provision of a number of treatments coincided with the introduction of the 2006 contract. 6 The number of complex treatments provided (root canal treatments, crowns and bridges) fell dramatically whereas the number of extractions rose just as dramatically. This appeared to reflect an increase in those activities that were easier and less costly to perform at the expense of those treatments that were more time intensive or that incurred laboratory costs for making crowns and bridges. The authors concluded that ‘the change in treatment patterns suggests that significant numbers of GDPs [were] attempting to hit their UDA contract targets in the most efficient way possible, i.e. shifting towards treatments where rewards are high relative to costs, as opposed to selecting on the basis of clinical factors’. 6 In addition, McDonald et al. 5 remarked that ‘it is the interests of GDPs, as opposed to patients, which [were] being prioritized’.

In England, the 2006 contract was unpopular with GDPs from the start; a major underlying reason was a loss of autonomy and their accountability to local commissioners. Only 2 years after its introduction, the contract was criticised by the House of Commons Health Select Committee,32 which based its criticism on the four areas that the contract had sought to improve, namely:

1. Patient experience – the contract was criticised for not helping to address the access problem (GDPs could hit their UDA activity targets largely by confining care provision to their long-standing, regularly attending patients) and the simplified patient charging system was also criticised as being insensitive.

2. Clinical quality – the contract did not incentivise preventative care and there were concerns about a fall in the volume of complex treatments such as root canal treatments, crowns and bridges.

3. Local commissioning – there was concern that many PCTs lacked the capacity and expertise to effectively commission dental services.

4. Improving GDPs’ working lives – the UDA activity targets were particularly unpopular with GDPs.

As a result of this criticism, an independent review (the Steele report) was published in 2009. 33 The report emphasised the need for prevention to produce better health outcomes and a standardised approach to patient assessment and disease risk categorisation leading to evidence-based care pathways. To support this approach, a move to a blended contract was envisaged based on capitation, with incentives to provide appropriate levels of activity and to improve quality.

Following the Steele report, in December 2010, the Department of Health and Social Care published NHS Dental Contract: Proposals for Pilots. 34 Three types of pilot were identified to test different remuneration models:

1. Practices were paid on a simple 100% capitation basis.

2. Practices were remunerated based on the number of weighted capitated patients they had; the capitation weighting was calculated according to age, gender and the deprivation status.

3. As per type 2, but with a separately identified budget for higher-cost treatments.

Elements of a reformed contract were piloted in 70 NHS dental practices in England, recruited between July and September 2011, and the findings of the pilots were published in 2014. 34 This report was a service evaluation and a control population was not identified. The evaluation of the oral health assessment, which formed the cornerstone of the new approach to care, was limited to professional and social acceptability. The pilots sought to introduce a new care pathway based on managing disease risk [using a high-, medium- and low-risk scale, the so-called RAG (red, amber, green) rating of risk] through the provision of preventative care. The ability of the associated risk algorithms to correctly classify patients and predict future disease has not been tested. With the methodological limitations, the findings were mainly descriptive in nature. The report acknowledged its methodological limitations, saying that the findings were:

. . . difficult to demonstrate with absolute confidence away from a clinical trial (which the pilots explicitly are not) and we have little or no comparable data from GDS outside the pilot.

Department of Health and Social Care. 34 Contains public sector information licensed under the Open Government Licence v3.0

Initially, quality was to be measured and incentivised by including quality payments as a small percentage (≈10%) of the practices’ remuneration package. The newly derived, but unvalidated, Dental Quality Outcomes Framework (DQOF) was intended to be used to measure quality and form a ‘pay-for-performance’ element of the remuneration system. However, in the first 2 years of piloting, the DQOF remuneration adjustments were not applied:

. . . due to issues with the robustness of the clinical data on which the DQOF clinical indicators are based.

Department of Health and Social Care. 34 Contains public sector information licensed under the Open Government Licence v3.0

As there were no control practices, the evaluation was restricted to a simple before-and-after design using a mixed-methods approach. There were significant problems encountered in collecting clinical data from information technology (IT) systems to support the standardised clinical assessment used in the pilot. The evaluation was also hampered by the fact that the collection of data on individual items of treatment ceased with the 2006 contract. Data entry for the new IT systems in the pilot was slow and onerous for practices, resulting in missing data and concerns about data quality. Dental charting data for many patients were incomplete or absent, making it impossible to apply the DQOF financial adjustments. Quantitative data presented in the report were confined to assessing the impact on access and patient experience. 34 Across all of the practices, there was a fall in the total number of unique patients seen for the duration of the pilots, albeit with significant variation across the practices. There was little association between a patient’s recall interval and the assessment of their risk, the ‘6-month check-up’ seemed to be well established and moving to a recall interval dictated by the risk assessment proved difficult. The pilots were generally popular with GDPs and patient satisfaction rates were higher (95.8%) than the national mean for all NHS dental practices of 92.2%, but these universally high scores demonstrate the unresponsive nature of this metric. In addition, baseline satisfaction rates for the pilot practices were not collected.

In 2015, the Department of Health and Social Care introduced a ‘prototype’ remuneration model but without a formally described, a priori evaluation plan. 35 The prototype process was introduced with the stated intention to test whole versions of a possible new contract rather than key elements required to inform the design of a new system (as was the intention in the pilots). Two contract models were tested, both aiming to continue to test the patient pathway based on the standardised assessment and risk stratification model. The UDAs remained as the contract currency to measure and remunerate activity delivery. Two types of prototype were implemented:

1. blend A prototypes included –

   • capitation (band 1)

   • activity (bands 2 and 3).

2. blend B prototypes included –

   • capitation (bands 1 and 2)

   • activity (band 3).

The intention was for both prototype models to also include remuneration adjustments made according to performance against the DQOF. 36 The prototypes involved greater financial risk for practices than the pilots (in which practices were guaranteed an income matched to a historical reference period); all prototypes were to be able to overdeliver by 2%, but would also have 10% of their contract value at risk if there was underdelivery of patient numbers and UDA activity. The protocol agreement scheme commenced on 1 December 2017. The Department of Health and Social Care and NHS England aimed to recruit a further 20 practices with an anticipated start date of October 2018.

The Department of Health and Social Care published an evaluation of the first year of prototyping in May 2018. 37 The prototype contract models started in spring 2016 and in the evaluation report the prototype practices were described as wave 3 of a larger evaluation programme with pilot practices described as wave 1 (recruited in 2011) and wave 2 (recruited in 2013). The evaluation included practices from all three waves but was problematic in that 40% of the original pilot practices did not continue into the prototype phase. Practices were split into two groups (blends A and B). Although it was originally intended that 10% of the contract value would be based on DQOF performance, it was agreed that for 2016/17 there would be no financial incentives used to try to improve quality. A post hoc matched set of practices working under the 2006 remuneration system was identified to enable comparisons between the prototypes with behaviour under the 2006 system. No methodological details were provided about how the matching was undertaken, nor were any analyses provided that compared the prototype practices with controls. Comparisons were restricted by the limited data available from comparator practices (primarily UDA claims and payments).

Data for the evaluation were provided by the NHS Business Services Authority using the same activity data for the comparator 2006 contract practices. Pilot/prototype practices were also asked to complete a monthly survey; these data were supplemented by two specific surveys undertaken in December 2016–January 2017 and in July–August 2017 to support the evaluation. The report restated the objective of the English Dental Contract Reform Programme:

. . . to maintain or improve access, quality and appropriateness of care and improve oral health, within the current cost envelope, in a way that is financially sustainable for dental practices, patients and commissioners.

Department of Health and Social Care. 37 Contains public sector information licensed under the Open Government Licence v3.0

However, a hierarchy for these (often competing) requirements was not identified.

The evaluation concluded:

. . . progress has been made in the first year of prototyping on the key issues of improving oral health, providing appropriate care and quality, and maintaining or increasing access to merit continuation of the programme.

Department of Health and Social Care. 37 Contains public sector information licensed under the Open Government Licence v3.0

The indicators to measure oral health improvement included the number of teeth with dental decay (children and adults) and the basic periodontal examination scores and the number of sextant bleeding sites in adults. Trends in GDP-assessed caries and periodontal risk were also reported but these oral health indicators were reported for the pilot/prototype practices alone and were not compared with the 2006 contract practices.

Appropriateness was assessed by adherence to the care pathway advocated in the Steele review,33 more specifically by the proportions of adults and children in the pilot/prototype practices who had received an oral health assessment. In all prototypes, 87% of adults and 86% of children received an oral health assessment and 99% of prototypes met the DQOF standard for recording an up-to-date medical history. Appropriateness was also assessed by process indicators identifying adherence to the prevention guidelines Delivering Better Oral Health: An Evidence-based Toolkit for Prevention. 38 Prototypes were compared with 2006 contract practices using FP17 (the form used by NHS GDPs to claim for activity provided) data reporting practice-reported adherence to national guidance. 38 For adults, 62% of prototype practices reported delivery of prevention according to the guidelines compared with 56% in 2006 contract practices; for children, these percentages are 60% and 58%, respectively. However, in prototype practices, only 28% of courses of treatment in children reported that fluoride varnish was applied compared with 41% in the 2006 contract practices. This latter finding gave cause for concern and the report recommended that a patient survey should be carried out to determine the impact of prevention advice on patients’ oral health behaviour. It is difficult to see how a survey would add much to the evidence base for the relationship between providing prevention advice and eliciting a sustained change in behaviour. The issue of recall attendance intervals was also considered under appropriateness, and the report recommend that:

. . . a detailed piece of work is undertaken to understand the reasons and rationale from patient and professional perspectives for the approach to implementing the longest recall periods recommended by NICE.

Department of Health and Social Care. 37 Contains public sector information licensed under the Open Government Licence v3.0

However, there was no mention of the soon-to-be published National Institute for Health Research (NIHR)-funded INTERVAL trial (ISRCTN95933794), the results of which are likely to influence the National Institute for Health and Care Excellence (NICE) guidance on recall intervals. 39

Assessment of the impact of quality in dentistry is limited by the underdeveloped conceptual understanding of quality in dentistry and how to measure it. 40,41 The report identified high percentages of prototype practices meeting all of the DQOF outcome indicator thresholds. The evaluation reported that 97% of patients treated in the prototype system were ‘quite’ or ‘very satisfied’ with NHS dentistry, compared with 96% of patients treated in the 2006 contract system. These very high satisfaction levels may not mean that all NHS services provide a highly satisfactory service; it may reflect that this indicator lacks responsiveness or that patients do not have sufficient knowledge and information to make an informed judgement of the quality of care provided. The report recommended that the development of the DQOF and its application should be continued but without commenting on how research could contribute to this goal.

In the reporting the impact on access and accessibility, these concepts were not defined. Access was measured by the practice capitated patient lists (number of patients attending in a 24-month period), which increased in waves 1 and 2 practices (initial pilot practices), but remained below the expected capitated numbers set in the contracts. During the first year of prototyping in wave 1 practices, attendance in a 24-month period increased from 85% to 88% of the baseline and for wave 2 practices it increased from 89% to 95% of the baseline. During the same period in matched 2006 contracts, access increased from 101% to 103% for wave 1 and stayed constant at 103% for wave 2. The fall in access in the prototypes was attributed to the concentration on embedding and delivering the clinical pathway in the early stages of piloting. Accessibility for patients was measured through a standard patient survey question and 90% of patients who responded reported that ‘the time it took to get an appointment was as soon as necessary’ compared with 91% in the 2006 contract practices. However, this is a limited view of accessibility, applying to only current users of a service and not to individuals living within a practice catchment area who may wish to access the service. This points to the need for clear and agreed definitions of access and accessibility, which are necessary to develop appropriate and valid measures of these important concepts for policy. Value for money was not assessed and the report acknowledged the difficulty of measuring this complex concept and stated that at this early stage there are insufficient ‘steady state’ data to undertake meaningful analysis. There was no elaboration on the measures or possible study designs that could be employed to undertake a rigorous health economic evaluation of different remuneration systems. A significant omission in the report was a lack of analysis of the impact of the pilots on PCR, especially as the objectives of the English Dental Contract Reform Programme include maintaining or improving service and health within the current cost envelope:

. . . in a way that is financially sustainable for dental practices, patients and commissioners.

Department of Health and Social Care. 37 Contains public sector information licensed under the Open Government Licence v3.0

Sustainability of prototype practices was compared with 2006 system practices using percentage contract achievement and, using this blunt measure, practices in all three waves performed to a similar level as the control 2006 system practices. However, a much broader and deeper means of assessing practice sustainability is required, given the multiple levers practices have at their disposal to improve their productivity and profitability, such as adjusting NHS/private mix and employment of skill mix. 4

The report made two further conclusions. The first was that the clinical model advocated by Steele33 is well accepted by the profession (standardised assessment of risk signposting the following of defined care pathways) but the business model requires further development to improve the sustainability for practices. The second conclusion was that additional practices should be recruited to the programme and randomly allocated to the blends to improve the robustness of evaluation. However, there was no suggestion that evaluation could be commissioned from an external party or that academic expertise could be harnessed to improve ‘the robustness of evaluation’ (contains public sector information licensed under the Open Government Licence v3.0). 37

It has been 10 years since the publication of the Steele report33 and a new English NHS dental contract is still to be implemented. This cautious, incremental approach seems to be reflected across all four of the home countries, which is not surprising given the complexity of the issues involved and the clinical, financial and political risk involved.

The evolution of a new NHS dental contract in Northern Ireland

The development of a new NHS contract in Northern Ireland can be plotted from a review of dental policy over the last 10 years traced through the content of a number of key documents.

In 2004, the DHSSPS produced a ‘Public Attitudes Survey on Oral Health’, which was undertaken to inform the development of an oral health strategy and primary dental care strategy for Northern Ireland (Donncha O’Carolan, NIHSCB, personal communication, 2019). Respondents were asked for their views on dental services, which could potentially form part of the Department’s planned Oral Health Strategy for Northern Ireland. 42 Respondents identified access to basic treatment, better access to emergency treatment and treatment for those with learning disabilities to be the most important aspects of the service that they wanted to see protected and enhanced.

The consultation document An Oral Health Strategy for Northern Ireland was subsequently launched in September 2004 and the final report, The Oral Health Strategy for Northern Ireland, was published in June 2007. 42 Although the main focus of the strategy was public health and community-based preventative interventions, it contained a section on dental services. The strategy reiterated one of the main themes of the Northern Ireland Primary Dental Care Strategy, launched in November 2006, which proposed a shift in the focus of a new GDS contract away from treatment of disease towards prevention. 43 The Oral Health Strategy stressed the need to maintain and improve access to dental services, especially for disadvantaged groups and for individuals with special care needs and implicitly acknowledged the policy aim of modernising dental services but warned about the need to maintain high levels of access and satisfaction in any reform of dental services.

The DHSSPS’s Primary Dental Care Strategy is a key document to help understand policy-makers’ thinking on the need for a new dental contract. 43 In the foreword, the Minister for Health said:

. . . there is widespread agreement that the way that dental services are currently delivered needs to change in order to better meet the needs of patients and GDPs and other oral healthcare professionals.

Department of Health. 43 Contains public sector information licensed under the Open Government Licence v3.0

This sentiment was expanded on in the recommendations section of the strategy, in particular recommendation 10 (paragraph 6.19) stated:

. . . a new Northern Ireland wide GDS primary dental care contract framework will be developed and will provide the basis to commission services to meet local need.

Department of Health. 43 Contains public sector information licensed under the Open Government Licence v3.0

Recommendation 17 (paragraph 8.4) stated that:

. . . any new arrangements should be piloted before rolling out across Northern Ireland.

Department of Health. 43 Contains public sector information licensed under the Open Government Licence v3.0

Many of the recommendations echoed developments in England (see The evolution of a new NHS dental contract in England), which were implemented in the PDS pilots of 2004–6, prior to the introduction of the 2006 English contract. Some of the ideas developed in the early English PDS pilots were translated into the NHS dental contract implemented in England in 2006, in particular an emphasis on local commissioning to make services more responsive to population health needs, a need and desire to secure access to dental services for the whole population, a shift in emphasis from repairing the effects of dental disease to disease prevention and a greater role for skill mix. It was telling that recommendation 16 (paragraph 7.3) of the DHSSPS’s Primary Dental Care Strategy stated:

. . . the new patient charging arrangements in England should be monitored with a view to introducing a similar system in Northern Ireland.

Department of Health. 43 Contains public sector information licensed under the Open Government Licence v3.0

This points to a history of health policy in Northern Ireland tracking developments in England and also an implicit acknowledgement that one of the main difficulties faced by policy-makers designing a new dental contract involves protecting that part of the budget derived from PCR. The 2006 English dental contract gave policy-makers certainty on annual spend for the service while buying a minimum level of access to NHS services (but not necessarily guaranteeing accessibility of NHS dental services particularly for those with greatest need).

Following publication of the DHSSPS’s Primary Dental Care Strategy, the evolution of policy thinking can be traced from the content of the DHSSPS’s Dental Branch Annual Reports from 2006/7 to 2010/11 (no further reports were produced after 2011). 44–48 The 2006/7 report contained a ministerial announcement for a new dental contract and commented on how new contractual arrangements in Northern Ireland would differ significantly from the 2006 contract introduced in England and Wales. 44 More specifically, at that time there was a view that providing a defined range of treatments and rewarding quality would be desirable in a new Northern Ireland contract and that the English UDAs contract currency was not favoured by the profession in Northern Ireland. The report stated:

. . . a commitment to piloting the new contract before rolling out across the region in order that any problems may be addressed and allow GDPs to be familiar with the operational aspects of the contract.

Department of Health. 44 Contains public sector information licensed under the Open Government Licence v3.0

The 2007/8 report commented on a growing concern about access to dental services in Northern Ireland. 45 In response to these concerns, a package of financial support for health service GDPs to increase the registration of NHS patients and a tender for additional dental services were announced in the Northern Ireland Assembly. The report also references a joint communication issued from the DHSSPS and the British Dental Association in September 2007. The key points highlighted in the statement were:

• The DHSSPS did not intend to introduce the (2006) English contract into Northern Ireland. The DHSSPS was aware of the problems experienced with the new contract in England and intended to learn from these.

• The contract was to be a block contract but would not be measured in UDAs; alternative remuneration arrangements were to be made.

• There was to be a strong emphasis on prevention.

• The new arrangements were to be piloted before rolling out across Northern Ireland.

The report also referenced a commissioned report produced by Professor O’Neill from Queen’s University Belfast to advise on a remuneration model for a new contract. Professor O’Neill’s report advised a blended system of remuneration comprising payment through a block component along with a limited FFS component.

The 2008/9 Dental Branch Annual Report identified access to dental services as ‘very problematic’ and stated that the Minister for Health was prioritising resolution of this problem through ‘a successful outcome to the dental tender’ (contains public sector information licensed under the Open Government Licence v3.0). 46 In May 2009, the Minister for Health awarded a contract to Oasis Dental Care [Oasis Dental Care (Central) Limited, Bupa Dental Care, Bristol, UK], which resulted in the opening of 14 new NHS dental practices, providing care for 57,000 patients, across Northern Ireland operating under pilot PDS contracts (analogous to the pre-2006, wholly capitation-based, English PDS pilots).

Although the access problem dominated policy, work on a new dental contract continued and more details were given about the anticipated remuneration system. A blended payment system was planned, consisting of regular care (capitation) payments along with supplementary item-of-service payments for a clearly defined range of essential services. The Northern Ireland Dental Branch Annual Report of 2008/9 acknowledged publication of the Steele review and noted that the recommendations were consistent with the framework of the proposed Northern Ireland contract. 45

The Northern Ireland Dental Branch Annual Report of 2009/10 provided an update on the initiative to tender for PDS contracts to improve access supported by the £17M injection of funding. 47 More information was again provided on the planned new contract, in particular details about a proposed weighted capitation formula for the proposed capitation element of the blended contract. The weighted capitation formula used to calculate the capitation fee was based on patient age, patient gender, patient socioeconomic position (based on small area measure of deprivation derived from the patient’s postcode) and practice list turnover (as measured by new registrations). It was also anticipated that block payments would be made to reward quality using indicators such as postgraduate qualifications, charter marks, outputs from practice inspections and clinical audit activities.

An update on the proposed pilots of the new contract was provided in the final Dental Branch Annual Report in 2010/11. 48 The NIHSCB began a consultation on the piloting of a new dental contract on 11 October 2010. The consultation ended on 31 January 2011 and was considered by the board of the NIHSCB on 31 March 2011, which supported the outcome of the consultation process, namely that pilot PDS legislation should be used to pilot the elements of the new dental contract.

In 2011, the Patient and Client Council (PCC) published Talking Teeth: Patient Views of General Dental Services in Northern Ireland. 49 The PCC commissioned the report to explore the experiences of patients as they access GDS services and the care that they receive in the NHS. The Chief Dental Officer provided a foreword to the report, which referenced the planned reforms of dental services including proposals to introduce a new dental contract. The report stressed that its findings should be considered within the context of that time and highlighted three key issues:

1. a £17M investment to ease problems of access

2. a planned new general dental contract

3. that economic recession had the potential to change (reduce) public demand for private dental care thereby freeing up capacity in the general dental system for NHS work and potentially helping to alleviate concerns about NHS dental access.

The final point was an interesting comment, as the acute access problem experienced in Northern Ireland was believed to have been caused primarily by GDPs expanding their care in the private sector prior to the financial crisis of 2007/8. The PCC report concluded that patients’ level of satisfaction with dental services was high but that there was confusion about whether or not treatment is provided via an NHS contract or privately. Dental treatment was viewed as expensive and people were worried about the lack of information regarding costs. Access and inequalities in access were concerns and there was support for the suggestion that NHS funding and services should concentrate on a smaller number of basic treatments. A key recommendation was that:

. . . access to basic dental treatments should be prioritised by Health and Social Care.

Patient and Clinic Council. 49 Contains public sector information licensed under the Open Government Licence v3.0

In June 2016, the NIHSCB produced the report Evaluation of the Oasis Pilot Personal Dental Services (pPDS) Scheme 2010–2015 (Michael Donaldson, NIHSCB personal communication, 2019). The contract started in April 2010 for an initial 3 years then extended for two additional 1-year periods, ending on 31 March 2015. The remuneration system used for these pilots was based on a simple capitation arrangement and co-payments were collected based on the existing item-of-service schedule operated by GDS practices. Capitation payments were based on the 38 GDPs in the programme, each managing a patient list of 1500 patients, resulting in a maximum of 57,000 patients being registered with Oasis Dental Care. Each registered patient attracted an annual fee of £97.74, which resulted in the contract value still being capped at £5.4M per year. A mid-term review of the pPDS was carried out in 2012 by the NIHSCB, approved by the DHSSPS and published in 2013. The mid-term review concluded that access to NHS dentistry had rapidly and significantly improved and that the service appeared to be at least equivalent to the service provided by other (GDS) dental providers in Northern Ireland in terms of the delivery of appropriate, efficient and cost-effective services.

The final report of 2016 concluded that access was not a problem for NHS dentistry in Northern Ireland and that the quality of care provided was comparable to that provided under the GDS. Following completion of the pilot, the 14 Oasis practices moved from pPDS into the GDS. The report commented on the learning, which had been acquired from the pPDS programme to inform the development of a new contract for Northern Ireland, in particular that consideration should be given to how new patients register with dental practices in a capitation-based contract.

The report expressed concerns that it may not be possible to monitor each individual GDP across the whole of Northern Ireland in the same way as the pPDS and that consideration should be given to how monitoring can be scaled up to ensure both quality of care and the level of service provision. There were also concerns that large number of patients who registered with Oasis during the pilot scheme received no treatment. In the pPDS and the GDS, patients aged > 18 years can register with a GDP without ever being examined. The report suggested that by enforcing an examination as part of the registration process more patients might become aware of their treatment needs and receive some preventative advice.

Our research group used data from the pPDS to independently evaluate the effect the newly introduced capitation payment system had on the delivery of primary oral health care and access to services in Northern Ireland. 4,50 This independent study was published in 2017 and treated the pPDS as a natural experiment comparing the NHS treatment claim forms submitted between April 2011 and October 2014 from the 14 Oasis practices with those of a group of matched, control GDS practices, remunerated using the GDS retrospective FFS system. This study could find no evidence of patient selection (excluding high-need patients) in the Oasis practices. However, patients were less likely to visit the GDP and received less treatment when they did attend, compared with those belonging to the control GDS practices. The extent of preventative activity offered did not differ significantly between the two practice groups. The most surprising finding was that there was no difference in PCR between the capitation-remunerated pilot practices and the FFS GDS controls.

In June 2014, the NIHSCB issued a call to all existing GDS practices across Northern Ireland for expressions of interest in participating in wave 1 of the new dental contract pilots and two practices were recruited. This was followed by a similar call in February 2015 for participation in wave 2 of the pilots and a further 11 practices were recruited. The pilot contracts were practice based, not with individual GDPs. The system of remuneration used was based on a form of capitation that was time limited: wave 1 practices would operate under the pilot contract for 18 months and wave 2 practices would operate for 12 months before reverting back to the FFS contract.

In terms of remuneration, the underpinning intention was that practitioners would receive the equivalent gross income during the pilot period that they would have received under the GDS had they maintained their activity and list as per the baseline period (Michael Donaldson and Donncha O’Carolan, NIHSCB, personal communication, 2019). The level and method of payment for patients’ contributions during the pilot would remain the same as for existing GDS patients. The PCR would continue to be collected by the practices and the monthly instalment payments from the BSO to pilot practices would be adjusted to account for the patient charges collected. The risk of any shortfall in patient contributions from the GDPs’ baseline gross income would be carried by the NIHSCB.

In the lead up to the pilots, the Chief Dental Officer posted a policy paper, Current Direction of the New Primary Care (General) Dental Contract, on the DHSSPS website on 6 April 2015, which provided a summary of policy development since 2006 and an update on the thinking of the DHSSPS on the design of the pilots. 51 This paper announced a shift in policy away from the earlier proposed model using a weighted capitation formula supplemented by FFS payments for defined treatments. Instead, the DHSSPS proposed that the latest model for the new dental contract would be calculated through a global sum formula with no routine item-of-service payments. Therefore, the model would be a predominantly simple capitation arrangement with additional payments making up a small proportion of the total remuneration system. It was envisaged that these payments would be determined by developing quality indicators; however, ‘further work on the development of the relevant indicators is necessary’. 51 This change in policy direction was attributed to feedback from GDPs, and was seen as a means of securing a stable budget by avoiding fluctuating costs of FFS claims and producing less complex data returns.

The evolution of a new NHS dental contract in Wales

In Wales, there had been similar dissatisfaction with the 2006 contract, which closely mirrors the current English contract. Dissatisfaction was largely for the same reasons as in England: the 2006 contract did not overtly incentivise prevention or quality and GDPs felt that they were still on a ‘treadmill’, not dissimilar to the treadmill of a FFS system, but with a cap on earnings. In Wales, the pilots for new remuneration systems involved eight dental practices starting in April 2011 and ending in March 2016. 52

Piloting activities could be divided into two phases. In the first phase, two pilots were involved:

1. In the Quality and Outcome Pilot, practices switched their entire GDS contract from the UDA system to a new contract based on a target number of weighted key performance indicators (KPIs), such as number of patients offered a Dental Care Assessment and a capitation target based on the number of unique patients treated in the practice within a defined time period.

2. The Preventive Dental Care for Children and Young People Pilot applied to only patients aged < 18 years. A proportion of each practice’s existing contract value, based on the proportion of children in the practice’s patient profile, was allocated to the pilot. The practices, again, were required to meet a certain number of weighted KPIs, including a capitation target for patients aged < 18 years.

In November 2012, the decision was taken to extend the Quality and Outcome Pilot for an extra year but to discontinue the Preventive Dental Care for Children and Young People Pilot at the end of March 2013, as originally intended. 53 There were some changes to the Quality and Outcome Pilot: a revised Dental Care Assessment was introduced with a disease risk assessment (high, medium or low – similar to England) and the requirement to produce a risk-based preventative care plan for each patient. From April 2015, four of the six pilot practices reverted to their former UDA contract. The remaining two practices were permitted by their health board to continue working under the pilot model into the next financial year, before returning to a UDA contract.

A mixed-methods design was used for the evaluation of the Welsh pilots in a similar approach to that used in England. 49 The main focus was on the qualitative component of the evaluation. Similar findings were reported to those of the evaluation report for the English pilots. The change in remuneration system was universally popular with the GDPs, and there was subjective reporting (by GDPs) of reductions in disease risk and improved oral health of their patients. Like the English pilots, there was a decrease in access: the number of patients seen across all pilot practices ranged between 7% and 10% less than in the year immediately preceding the pilot programme.

Consequently, there was a reduction in PCR: in the first year of the pilot, there was a drop in PCR of over one-third on the previous year under the 2006 arrangements. However, over the 4 years of the pilot, the level of PCR across the practices steadily increased and it was estimated that if the upwards trend in PCR continued at the same rate it would eventually have reached levels similar to the pre-pilot year. However, no time frame was given for when PCR would reach baseline levels. At the end of the pilot programme, the majority of the pilot practices reverted to the 2006 contract and most reported that this has been detrimental to teams and patients, but no specific information was provided on the nature of these detrimental effects. Two of the pilot practices continued as ‘prototype practices’; neither practice is monitored by UDAs but both continue to evolve the model developed in the pilot as part of ‘a learning network’ rather than a formal evaluation.

In August 2017, there was a change of tack in policy. The Welsh Government published Taking Oral Health Improvement and Dental Services Forward in Wales; this policy document outlined the key priorities for NHS dentistry, and contract reform was identified as one of the priorities. 54 The document criticised the current dental contract as being focused on treatment activity and not incentivising needs-led care and prevention. It also criticised the 2006 contract for not encouraging the use of skill mix. The document cited the learning from previous dental pilots in Wales and the ongoing experiences of the dental prototype practices to inform the design of the new programme. Part of this process included the development of a needs and risk assessment tool.

The document did not present a defined ‘new’ dental contract; instead, a more pragmatic approach was advocated to use the flexibility within the existing 2006 contract to produce reform without the need for wholesale change in contract regulations. The document was clear that this approach must be underpinned by robust need and outcome measurement. Examples were given of possible scenarios, such as substituting a proportion of UDAs within a given total contract value by other measures to encourage, for example, a change in care delivery to increase access for high-need groups. This pragmatic approach did not require additional investment and the flexible proportion of the contract could be changed (by negotiation with practices) according to changing population needs.

In February 2018, the Welsh Government announced that from January 2018 a further 23 prototype dental practices would be recruited to expand the flexible adaptation of existing contracts, in particular testing the use of the needs and risk assessment tool. 55 The new Contract Reform programme aims to increase value and quality, based on a needs-based approach to care. Key elements include adopting a needs-led preventive approach to care; extending the use of skill mix as part of prudent dental health care; the acceptance of and provision of care to new patients with higher needs; encouraging ‘well patients’ to attend at longer intervals; and delivering high-quality evidence-based care according to need. As of December 2019, over one-quarter of the dental practices in Wales were participating in the Contract Reform programme.

The evolution of a new NHS dental contract in Scotland

In Scotland, the NHS dental contract has not changed substantially since 1990 and the situation is similar to that in Northern Ireland. The GDPs providing care to children (aged < 18 years) are paid primarily on a capitation basis, receiving a basic monthly fee for the care and treatment of children covering examinations, X-rays, scale and polish, and preventative care. A smaller proportion of the remuneration package for children is made up from additional item-of-service fees for fillings and extractions. The majority of the care provided to adults is still paid on a fee-for-item basis with a small monthly fee payable for the provision of continuing care to registered patients.

In January 2018, the Scottish Government published its Oral Health Improvement Plan, following publication of a consultation exercise in September 2016. 56 The plan is ambitious, setting out 41 actions or commitments to be delivered within a non-specified time scale. Like the other three home countries, there is an aspiration to focus on prevention, highlighting the need for a shift from restorative to preventative dentistry, which was well supported during the consultation exercise.

An explicit action in the plan was to:

. . . change payments to GDPs and introduce a system of monitoring to ensure that all dental practices provide preventive treatment for children.

Scottish Government. 56 Contains public sector information licensed under the Open Government Licence v3.0

There was also an aspiration to introduce a preventative system of care for adults by phasing in an oral health risk assessment. This assessment would be more comprehensive than a risk assessment of dental disease alone and would include a comprehensive clinical examination and a discussion about lifestyle choices, including diet, alcohol use and smoking. The patient would receive a personalised care plan based on the outcomes of the assessment.

Interestingly, and in contrast to the other three home nations, there were no plans for scrapping the item-of-service system of remuneration; instead there was an intention to ‘streamline’ item-of-service payments, which would be progressively introduced. NHS health boards would in future take a more proactive approach to the listing of practices, exerting tighter controls on GDPs wishing to set up new practices in their locality. There were also plans to develop a new system to oversee external clinical quality monitoring under the direction of newly appointed directors of dentistry in each health board. The plan echoes the evolutionary approach taken in the prototypes in England and Wales in that:

. . . our intention is to proceed in a progressive manner in order for practices that provide GDS to have the opportunity to plan accordingly whilst maintaining their financial stability.

Scottish Government. 56 Contains public sector information licensed under the Open Government Licence v3.0

In terms of changes to remuneration, the plan recognised the current, mixed economy of item of service, capitation and continuing care payments as a strength and stated that the Scottish Government would:

. . . continue to retain a mix of payments going forward, but the balance of payments will change accordingly.

Scottish Government. 56 Contains public sector information licensed under the Open Government Licence v3.0

There were no explicit plans described in the document to evaluate the impact of implementing the ≈40 actions set out in the plan or for how the envisaged changes in remuneration would influence and incentivise the behaviours of GDPs.

Study design

A DiD design was employed to quantitatively measure any change in activity levels across the intervention and control groups in each of the three phases of the study:

• phase 1 – 12-month baseline period prior to introduction of the new capitation contract in the intervention group of practices (August 2014 to August 2015)

• phase 2 – 12-month capitation period for the intervention group of practices (August 2015 to August 2016)

• phase 3 – 12-month period following reversion of the intervention group of practices back to FFS (August 2016 to August 2017).

Figure 1 provides a graphical depiction of the DiD design, which compared the difference in outcomes before and after the change in the contract model in the pilot practices with outcomes in a group of matched controls. 59

FIGURE 1.

The DiD design used in the study.

The DiD estimator examined the impact of the contractual change in the intervention practices compared with the control practices at the following points of change:

• baseline FFS to capitation

• capitation to reversion FFS

• baseline FFS to reversion FFS.

Analyses were performed at the practice level. A single outcome measure was used in each DiD model. Clustered standard errors (SEs) were determined to adjust for estimates of the correlation over time. This is because outcomes measured at the patient level (e.g. number of patients, number of treatments delivered to patients) could be influenced over time by a higher-level structure (the practice).

We also used the same DiD approach and outcome measures to analyse activity data at the individual GDP level. The analyses assessed changes in behaviour for the average number of ‘designated GDPs’ in each practice and the average number of ‘other’ GDPs with an NHS contract. A designated GDP is a GDP who is paid practice allowances by the NHS in Northern Ireland. These allowances are to help cover some of the running costs of NHS dental practices. A ‘designated GDP’ is likely to be an equity-owning PP and is labelled as such in the text and results tables. A second analysis was undertaken for all (other) GDPs with an NHS contract who were not ‘designated GDPs’. These were likely to be ADs (non-equity owning) working at the practice and are labelled as such in the text and results tables.

Table 1 describes the outcome measures used in this study. Three broad groups of outcome measures were chosen to assess the impact of the change in remuneration on:

1. access

2. treatment (service mix)

3. finance.

The impact on access is important to assess, as one would expect capitation to increase access, with GDPs wishing to ensure that they hit their capitation targets. The outcome measures used to measure patient access to services were different types of registration as a proportion of the total number of patients on the practice list (NHS dental registration lasts for a period of 24 months):

• mean (total) number of registrations

• mean number of reregistrations (rollover of existing registrations)

• mean number of new patients

• proportion of lapsed patients who returned to the practice list

• mean number of patients lost to the practice.

The service mix measures were chosen to identify any changes in care provision, which would be important to policy and can be broadly categorised as:

• complex treatment involving extensive clinical time or work involving a dental laboratory [endodontics, indirect restorations (crowns) and dentures]

• treatment of established disease – direct restorations (fillings) and extractions

• preventative care, for example examinations, fissure sealants, radiographs, two-visit periodontal treatments

• other composite measures of activity (number of treatment plans and treatment items).

The third group of outcome measures were financial in nature and are important from a policy perspective and to dental practices as small businesses. Financial measures included total health service income and revenue from patients’ charges. Data on income from private practice was not available.

The contents of the HS45 form provided the data for access, service activity and financial information; this form is replicated in Appendix 1. This form is the Northern Ireland equivalent of the FP17 form used by the NHS in England. It is a means of claiming payment for any care or treatment detailed in the Statement of Remuneration. 60 Most practices submit their claims for treatment completed electronically and all the information included in the paper HS45 form is replicated in the electronic version. A HS45 or electronic equivalent is submitted every time a health service course of treatment (including examination only) is claimed. The GDPs also usually tick the part 3 ‘reregistration’ element in order to roll over the patient’s registration for a further 24 months.

The BSO of Northern Ireland is responsible for collating and paying claims for activity. The BSO extracted data from the submitted HS45 forms for both intervention and control practices for the analyses. Prior to analysis, the BSO cleaned the data set to:

• prevent the block transfer of patients between GDPs working in the same practice appearing as new patients

• identify and separate treatment item ‘tail data’ (treatments that took place towards the end of one study period that were claimed in the following study period)

• identify large fluctuations over time in treatment activity caused by maternity leave, major business change (newly started practices hiring staff or closing practices transferring patients to other practices) or takeover activity.

The outcome measures for access and service mix were calculated relative to the practice list size (per 1000 patient registrations). This was important because analyses were based on a comparison of group averages (e.g. changes between study periods within and between control and intervention practices). Using proportionate measures gave each practice an equal weight in determining the group average. Measures of activity alone (not relative to practice size) would have given greater weight in the calculations in our analysis to a small number of larger pilot practices.

Population

The population under study was practices with an NHS contract in Northern Ireland. A selection process overseen by the DHSSPS and NIHSCB was undertaken to select the practices in the intervention group (i.e. those whose process of payment would move across to one based on a capitation-type payment). Prior to the evaluation described here, the DHSSPS and NIHSCB had recruited two practices to a pre-pilot stage that started on 13 November 2014 and had run for 6 months. The purpose of this pre-pilot stage was to determine the financial risk (modelling the drop in PCR). This helped the DHSSPS and NIHSCB ensure that the budget for the full pilot was underwritten.

To select the intervention practices, the DHSSPS and NIHSCB arranged evening meetings in November and December 2014 for GDPs to raise awareness about the pilots. Practices expressing an interest in participating attended information evenings held on 2 and 3 March 2015. Offers were made to practices submitting a formal application to participate on 8 May 2015. The inclusion criteria were determined by the NIHSCB and detailed in an expression of interest document. Practices had to:

• be a provider in one of the following categories –

  • a health and social care trust

  • a GDP whose name is included in a dental list

  • an NIHSCB employee or pilot scheme employee

  • a qualifying body

  • an individual who is providing PDS

• have a commitment to the NHS maintained for the duration of the pilot

• have full registration with the General Dental Council

• have appropriate indemnity

• have completed a vocational training/dental foundation training programme (or have an appropriate exemption).

The final selection of the intervention group was undertaken by an internal NIHSCB panel using the criteria above. Additional criteria that were also considered included practice size (classified as small or large), geography (classified as urban or non-urban) and extent of health service commitment (classified as above or below average). A total of 21 practices submitted expressions of interest to be involved in the pilots: two did not meet the essential criteria – one did not meet the minimum 50% commitment to the NHS and one was unable to provide a service for 37.5 hours per week. A total of 12 practices were selected by the panel and financial templates were produced for all of the selected practices. The selected practices had 2 weeks to consider the expression of interest document and to reply by 22 May 2015; nine practices subsequently signed the service-level agreement to take part in the pilot. These nine practices were added to the two practices that participated in the first-wave practices, making a total of 11 practices who participated in this study. In our original protocol we stated that we would ‘conduct 2 focus groups with dentists involved in the pilot practices to discuss any observed changes in activity and identify areas that dentists feel the simple capitation contract can be improved’. These focus groups did not take place as the DHSSPS/NIHSCB decided that the two pilot practices would be included in the full evaluation. In addition, the DHSSPS/NIHSCB decided to proceed with a simple capitation contract with no changes to that used in the first-wave pilot. These 11 practices and their patients, who were registered during the study period, made up the population who received the intervention. The characteristics of the intervention practices are set out in Tables 3 and 4.

The intervention period lasted 12 months, during which time each practice was required to maintain a health service registered patient list consistent in size and profile with the list registered under the practice on 31 December 2014. The contract stipulated that pilot practices would be closely monitored for any variance outside the tolerance of a decrease in registration of 5%, in which case practices would be asked to address any variance outside the tolerance with the possibility of renegotiation of contract terms and conditions. The level of remuneration under capitation was based on each practitioner’s gross NHS earnings during a reference period, which ran from 1 January 2014 to 31 December 2014. This included appropriate prospective adjustments to reflect and subsequent changes to the Statement of Dental Remuneration made in the period leading up to the start of the pilot. During the pilot, practitioners received their agreed gross income paid in 12 equal monthly instalments. In August 2015, the intervention practices switched to the new capitation contract and returned to FFS 12 months later.

Selection of the control practices

It is not possible to test for the presence of observable characteristics that influence the recorded observations in natural experiments (i.e. there is no certainty that the DiD estimates are free from selection bias). To assess how violations of unconfounded treatment assignment (i.e. selection bias) could affect our study conclusions, we tested the sensitivity of the DiD estimates by using a matched control group of practices with characteristics (which might influence participation in the pilot) that overlapped with the intervention group as closely as possible.

The control group of practices was selected using a two-stage process. The first stage was by stratified random sampling of all NIHSCB practices in Northern Ireland using the following strata: practice list size, proportion of patients who are children, proportion of adults exempt from NHS fees and geographic location of the practice (region size defined by the Northern Ireland Small Areas Code in the 2011 Census geography61). Out of a total of 45 practices initially identified, 15 could not be used because of data inconsistencies, leaving 30 potential control practices. The data inconsistencies were large gaps in the data in some months followed by spikes in the data recorded in later months as the backlog of data records was processed. This occurred because of a change in a computer system that processed dental data sent by practices to the Health and Social Care Board in Northern Ireland. The second stage of selecting control practices involved matching the 11 intervention practices to control practices using a propensity score (PS) approach, which identified 18 matched control practices.

In a DiD model, selection of the intervention group causes the control group to be a poor approximation of the ‘unobservable outcome’ used to estimate a causal effect of treatment in the model, when there are unobserved influences that affect participation in the pilot as well as the outcomes of practices in the intervention group and control group, or both. For example, the standard DiD estimator will have bias in our study if, regardless of the stratified sampling method that was used to (initially) select the control group, the practices that were motivated to join the pilot (intervention practices) have a higher or lower probability of responding to remuneration incentives by altering their health-care provision than a situation in which intervention practices were instead chosen at random and hence were not particularly motivated to join the pilot.

A probit model was used to determine whether or not there were any practice characteristics that influenced the decision to participate in the pilot: 41 observations (baseline) from 30 control practices and 11 intervention practices were tested (Table 2). No statistically significant differences were found between the two groups.

Regardless of these findings, selection bias cannot be ruled out in this study if there were unobserved characteristics, which influenced the motivation of practices to participate in the pilot. We examined the plausibility of this possibility by considering whether or not there were fixed differences between the control and intervention practices in the number of different kinds of treatments delivered in the baseline year. The results are presented in Tables 3 and 4.

Fissure sealants was the only outcome measure for which the monthly number of treatments delivered during the baseline period differed statistically significantly between intervention and control practices. This suggests that unobservable characteristics that could influence treatment outcomes were likely to be absent or balanced between the two groups. Taken with the evidence of no statistical differences between the groups in the practice characteristics presented in Tables 2–4, it appears that the assignment of practices to the control group was as good as it could be without randomisation. However, the findings in Tables 2–4 are best understood as exploratory work and it is possible that statistically insignificant differences are explained by a low sample size of practices. Further detail about the selection process for the controls can be found in Appendix 2.

In the analysis, we undertook DiD analyses on the initial 30 control practices (henceforward referred to as ‘unmatched controls’) and the 18 matched control practices identified by propensity scoring (henceforward referred to as ‘matched controls’) and compared the difference in outcomes. To avoid unnecessary clutter in the results section, we present the results of the matched controls. The full analysis, which includes the unmatched controls, is in Appendix 3.

Multiple testing

There was a large number of outcome measures (n = 22); this initially included two additional variables to measure different types of fissure sealants, which are not included in this report. These outcome measures used data provided by the BSO, and each outcome measure was a dependent variable in three types of DiD models [i.e. practice level (main analysis), PPs and ADs] that were estimated with and without the use of a PS matching process. Consequently, a large number (n = 132) of DiD estimates (variables) were examined for statistical significance, and the more variables that were tested, the more likely a variable would be statistically significant by chance. Specifically, for the 132 DiD models used, the chance of finding one or more significant differences in the DiD estimator was 1 – (0.95) to the power of 132 or 99.89%, which is the chance that one of the estimators in the models will appear to be significant purely by chance. Šidák and Bonferroni corrections are two methods to counteract the problem of multiple comparisons. For each model, the Šidák and Bonferroni adjustments were calculated (based on the correlation of outcome measure to all other outcomes and the number of observations in that model) and reported in the results tables to identify whether or not either adjustment lowered the critical value of 0.05 to below the p-value of the DiD estimate. If the Šidák and Bonferroni adjustments were above the p-value of the estimate, this indicated that even though the chance of a rare event (incorrectly rejecting a null hypothesis, i.e. making a type I error) had increased because multiple hypotheses were tested, the likelihood of this rare event occurring did not increase to such an extent that the null hypothesis (that the DiD estimate is zero) could no longer be reasonably (i.e. a decision based on the sample distribution of estimate) rejected.

Results: access outcomes

The difference between intervention and control practices in the number of registered patients significantly increased during the capitation period (p < 0.01) by 1.5 registrations per month per 1000 registered patients (Tables 5 and 6; Figure 2) when compared with baseline and decreased after the capitation period. This was caused by an increase in registrations in the intervention practices. No statistically significant difference was found between FFS at baseline and at reversion.

FIGURE 2.

Mean number of registered patients.

There was no statistically significant difference in the number of patients who were ‘rolled over’ (i.e. when patients already registered with the practice have a new course of treatment and reregister) (see Tables 5 and 6; Figure 3).

FIGURE 3.

Mean number of reregistrations.

The difference between intervention and control practices, in terms of the number of new patients (i.e. the difference in activity between intervention and control practices per month per 1000 registered patients), fell significantly (p < 0.01) between the reversion period and the capitation period, by 6.8 new registrations, and between the reversion period and baseline, by 5.7 new registrations (see Tables 5 and 6; Figure 4). These statistically significant changes were as a result of changes over time in the control group. There was a statistically significant decrease (p < 0.05) in newly registered patients per month in control practices in the reversion period compared with the capitation period, whereas the only statistically significant change in intervention practices was a decrease during the capitation period compared with FFS at baseline (p < 0.05). The non-significant (p > 0.05) difference between intervention and control practices in the number of new patients joining the practice list per month when capitation was compared with the FFS baseline period suggests that the pilot contract did not cause an immediate change in new registrations in intervention practices compared with control practices.

FIGURE 4.

Number of new patients.

The difference between intervention and control practices in the number of lapsed patients was significant and was caused by a reduction of 27.1 registrations per month per 1000 registered patients. This was caused by a reduction in the number of lapsed patients in the intervention group. The difference increased further by 7.8 registrations per month per 1000 registered patients in the reversion FFS period (see Tables 5 and 6; Figure 5). This was because of a comparatively large reduction (p < 0.05) in returning patients, of 22.8 registrations per month per 1000 registered patients, in intervention practices in the capitation period, and there was an increase (p < 0.05) in returning patients in control practices of 2.61 registrations per month per 1000 registered patients. The number of returning patients in control practices increased (p < 0.05) after the capitation period by 7.19 registrations per month per 1000 registered patients, and the increase in intervention practices was much smaller, with 0.62 returning patients per month per 1000 registered patients (p < 0.05). There was a large change of 33.7 returning patients per month per 1000 registered patients (p < 0.05) in the difference between intervention and control practices in the number of registered patients on the practice list per month when the FFS reversion period was compared with the FFS baseline period (see Tables 5 and 6). This suggests that the changes that occurred in the capitation period in intervention practices compared with control practices did persist after the pilot had ended.

FIGURE 5.

Proportion of lapsed patients who returned to the practice list.

There was no statistically significant change in the difference between intervention and control practices when the capitation period was compared with FFS at baseline, but there was when compared with the FFS reversion period (p = 0.01) and when FFS in the reversion period was compared with FFS during the baseline period (p = 0.02). For an average practice, the difference decreased in the reversion period by 13.4 patients lost to the practice per month per 1000 registered patients when compared with the capitation period, and 15.6 patients lost to the practice per month per 1000 registered patients when compared with FFS at baseline (see Tables 5 and 6; Figure 6). This was as a result of a statistically significant decrease (p < 0.05) in patients lost to the practice in control practices during the reversion period compared with the capitation period, whereas there was a statistically significant increase in patients lost to the practice in intervention practices (p < 0.05). There was not a statistically significant change (p > 0.05) in the per-month number of patients lost to the practice in the control and intervention practices in the capitation period compared with the baseline period.

FIGURE 6.

Number of patients lost to the practice.

Results: treatment outcomes

The difference between intervention and control practices in the mean number of treatments with a gross cost of ≥ £280 per month per 1000 registrations did not significantly change (p > 0.05) between study periods (Tables 7 and 8; Figure 7). There was also no significant difference (p > 0.05) in the number of treatments with a gross cost of ≥ £280 per month per 1000 registrations delivered by PPs in intervention and control practices and no significant difference (p > 0.05) for ADs (see Appendix 4).

FIGURE 7.

Monthly number of treatments with a gross cost of ≥ £280.

The difference between intervention and control practices in the mean number of direct restoration treatments per month per 1000 registrations changed between the study periods. The difference significantly increased (p < 0.05), with 49.0 fewer direct restoration treatments per month per 1000 registrations. This was caused by a reduction in activity in the intervention practices in the capitation period compared with baseline FFS (see Tables 7 and 8). This difference increased (p < 0.05) by 62.0 direct restoration treatments per month per 1000 registrations in the reversion period compared with capitation (see Tables 7 and 8; Figure 8). There was no evidence of a long-term effect from the pilot because the difference between intervention and control practices was not significant (p > 0.05) when the FFS reversion period was compared with FFS at baseline.

FIGURE 8.

Number of direct restoration treatments.

The difference between intervention and control practices in the number of indirect restorations per month per 1000 registrations changed significantly between study periods (see Tables 7 and 8; Figure 9). The difference increased (p < 0.05) and was caused by 3.7 fewer indirect restorations per month per 1000 registrations in the intervention practices in the capitation period compared with baseline FFS; it decreased (p < 0.05) with 2.3 fewer indirect restorations per month per 1000 registrations in the reversion period compared with the capitation period (relative to the control practices). There was no evidence of a long-term effect from the pilot because the difference was not significant (p > 0.05) between intervention and control practices in the FFS reversion period compared with FFS at baseline.

FIGURE 9.

Number of indirect restoration treatments.

A similar picture was seen for ADs, with the difference increasing (p < 0.05) with 2.8 fewer indirect restorations per month per 1000 registrations being undertaken in the intervention practices in the capitation period compared with baseline FFS, and decreasing (p < 0.05) with 2.8 fewer indirect restorations per month per 1000 registrations in the reversion period compared with the capitation period (see Appendix 4). This was not the case for PPs. There was a statistically significant increase (p < 0.05) of 1.6 in the difference in indirect restorations per month per 1000 registrations delivered by the mean number of PPs in intervention and control practices in the FFS reversion period compared with FFS at baseline.

The DiD analyses identified a significant difference between intervention and control practices in the number of extractions per month per 1000 registrations between study periods. The difference increased (p < 0.05) and was caused by 6.8 fewer extractions per month per 1000 registrations being undertaken. This was caused by a reduction of activity in the intervention practices in the capitation period compared with baseline FFS (see Tables 7 and 8; Figure 10). The difference decreased (p < 0.05) with 5.6 more extractions in the reversion period compared with capitation (see Table 8). Any change in treatment provision did not persist after capitation ceased and there was no significant difference (p > 0.05) between intervention and control practices in the number of extractions per month per 1000 registrations in the FFS reversion period compared with FFS at baseline.

FIGURE 10.

Plot of the monthly number of extraction treatments.

A similar picture was seen in extractions, where the difference compared with baseline FFS increased (p < 0.05) and was caused by five fewer extractions for ADs and increased (p < 0.05) by 3.12 for PPs. During the FFS reversion period, compared with capitation, the situation reversed with the difference decreasing (p < 0.05) by 6.3 for ADs but did not significantly change (p > 0.05) for PPs. There seemed to be a persistent change in practice for PPs with a statistically significant increase (p < 0.05) of 2.42 in the number of extractions per month per 1000 registrations delivered in the FFS reversion period compared with FFS at baseline, which was not found for ADs.

In the capitation period compared with baseline FFS, the difference between intervention and control practices in the number of radiographs significantly increased (p < 0.05), caused by a reduction of 40.9 radiographs per month per 1000 registrations. This was caused by a reduction of activity in the intervention practices in the capitation period compared with baseline FFS (see Tables 7 and 8; Figure 11). This decreased (p < 0.05) by 34.1 in the reversion period compared with capitation (see Tables 7 and 8). There was no evidence of a long-term change in provision as the difference was not significant (p > 0.05) between intervention and control practices in the number of radiographs taken in the FFS reversion period compared with FFS at baseline.

FIGURE 11.

Number of radiograph treatments.

For ADs, the difference in radiographs taken in the capitation period compared with baseline FFS increased (p < 0.05) by 38.0; it also increased significantly (p < 0.05) by 16.3 for PPs (see Appendix 4). When comparing the FFS reversion period with capitation, the difference significantly (p > 0.05) decreased by 38.1 for ADs but did not significantly change (p > 0.05) for PPs.

The difference between the intervention and control group in fissure sealants provided increased significantly (p < 0.05) by 9.3 fissure sealant treatments per month per 1000 registrations in the capitation period compared with baseline FFS and decreased (p < 0.05) by 10.4 in the reversion period compared with the capitation period (Table 10). This was caused by a reduction of activity in the intervention practices in the capitation period compared with baseline FFS (Tables 9 and 10; Figure 12). There was no significant difference (p > 0.05) between intervention and control practices in the number of fissure sealant treatments provided per month in the FFS reversion period compared with FFS at baseline.

FIGURE 12.

Number of fissure sealant treatments.

Similar changes were seen for both ADs and PPs (see Appendix 4). The difference in direct restoration treatments in the capitation period compared with baseline FFS increased (p < 0.05), with 56.2 fewer restorations for ADs and 14.3 fewer restorations for PPs. In the FFS reversion period compared with the capitation period, the difference decreased (p < 0.05) with 62.1 fewer restorations for ADs and with 16.2 fewer restorations for PPs.

For ADs, the difference in fissure sealants provided during the capitation period compared with baseline FFS increased (p < 0.05) by 11.3 but this did not significantly change for PPs (see Appendix 4). The difference in fissure sealant treatments provided per month in the FFS reversion period compared with the capitation period decreased (p < 0.05) by 11.1 for ADs but did not significantly change (p > 0.05) for PPs.

The difference between intervention and control practices in the number of two or more visits periodontal treatments provided increased (p < 0.05) by 3.8 during the capitation period compared with baseline FFS and decreased (p < 0.05) by 3.5 in the reversion period compared with the capitation period (see Tables 9 and 10; Figure 13). This was caused by a reduction of activity in the intervention practices in the capitation period compared with baseline FFS (see Tables 9 and 10). There was no evidence of a long-term effect in provision as there was no significant difference (p > 0.05) between intervention and control practices when comparing the FFS reversion period to FFS at baseline.

FIGURE 13.

Number of two or more periodontal treatments.

There was a difference in the pattern of care between ADs and PPs. For ADs, the difference during the capitation period compared with baseline FFS increased significantly (p < 0.05) by 3.8 but it did not significantly change for PPs (see Appendix 4). During the FFS reversion period compared with the capitation period provision, the difference between intervention and control practices significantly decreased (p < 0.05) by 4.1 but it did not significantly change (p > 0.05) for PPs.

The difference in provision of root canal treatments increased (p < 0.05) by 2.7 root canal treatments per month per 1000 registrations during the capitation period compared with baseline FFS and decreased (p < 0.05) by 2.4 in the reversion period compared with the capitation period (see Tables 9 and 10; Figure 14). This was caused by a reduction in activity in the intervention practices in the capitation period compared with baseline FFS (see Table 10). There was no evidence of a long-term effect in provision as a result of the pilot, as the difference was not significant (p > 0.05) between intervention and control practices in the number of root canal treatments provided per month in the FFS reversion period compared with FFS at baseline.

FIGURE 14.

Number of root canal treatments.

Significant increases in the difference in provision of root canal treatments between intervention and control practices between FFS baseline and the capitation period were observed [2.4 for ADs and 0.9 (p = 0.05) for PPs (see Appendix 4)]. When comparing the FFS reversion period with the capitation period, provision of root canal treatments decreased (p < 0.05) by 2.4 for ADs but did not significantly change for PPs.

The difference increased significantly (p < 0.05) by 34.3 treatment plans per month per 1000 registrations in the capitation period compared with baseline FFS, but decreased significantly (p < 0.05) by 28.7 in the reversion period compared with the capitation period (see Tables 9 and 10). This was caused by a reduction of activity in the intervention practices in the capitation period compared with baseline FFS (see Tables 9 and 10; Figure 15). There was no evidence of a long-term effect as a result of the pilot because the difference was not significant (p > 0.05) between intervention and control practices in the number of treatment plans provided per month in the FFS reversion period compared with the FFS baseline period.

FIGURE 15.

Number of treatment plans.

The difference in treatment plans provided per month in the capitation period compared with baseline FFS increased (p < 0.05) by 39.1 for ADs but did not significantly change (p > 0.05) for PPs (see Appendix 4). Likewise, there was a significant (p > 0.05) difference in the number of treatment plans provided per month by ADs in the FFS reversion period compared with the capitation period, with a decrease of 32.3 treatment plans but no statistically significant change for PPs.

The difference in number of treatment items provided increased significantly (p < 0.05) by 174.8 treatment items during the capitation period compared with baseline FFS and significantly decreased (p < 0.05) by 173.9 in the reversion period compared with the capitation period (see Tables 9 and 10; Figure 16). This was caused by a reduction in activity in the intervention practices in the capitation period compared with baseline FFS (see Table 10). There was no evidence of a long-term effect on the number of treatment items provided from the pilot because there was no significant difference between intervention and control practices in the FFS reversion period compared with FFS at baseline.

FIGURE 16.

Number of treatment items.

For ADs, the difference between those working in intervention and control practices in terms of the number of treatment items provided in the capitation period compared with baseline FFS increased (p < 0.05) by 174.5; it also increased (p < 0.05) by 70.4 for PPs (see Appendix 4). Corresponding significant decreases in the difference were seen for both ADs (179.0) and PPs (47.0) in the FFS reversion period compared with the capitation period.

Results: financial outcomes

The difference between intervention and control practices in the mean percentage of patient fee contribution to NHS dental practice income per month did not significantly change (p > 0.05) between study periods (Tables 11 and 12; Figure 17).

FIGURE 17.

Mean percentage of patient fee contribution.

The difference between intervention and control practices in NHS dental practice income per month changed between study periods. The difference in NHS dental practice income per month increased significantly (p < 0.05), by £5920, in the capitation period compared with baseline FFS, and decreased significantly (p < 0.05), by £5248, in the reversion period compared with capitation. This was caused by a reduction of activity in the intervention practices in the capitation period compared with baseline FFS (see Tables 11 and 12; Figure 18). There was no evidence of a long-term effect from the pilot because the difference in NHS dental practice income per month between intervention and control practices was not significantly different (p > 0.05) between the FFS reversion period and the FFS at baseline.

FIGURE 18.

Mean NHS dental practice income.

The difference between intervention and control practices in patient fee contributions per month changed between study periods. The difference in patient fee contributions per month increased significantly (p < 0.05), by £2403, in the capitation period compared with baseline FFS, and decreased significantly (p < 0.05), by £2028, in the reversion period compared with capitation. This was caused by a reduction in activity in the intervention practices in the capitation period compared with baseline FFS (see Tables 11 and 12; Figure 19). There was no evidence of a long-term effect from the pilot because the difference between intervention and control practices in patient fee contributions per month was not significantly different (p > 0.05) between the FFS reversion period and the FFS at baseline.

FIGURE 19.

Mean patient fee contributions.

For all of the analyses presented above, no difference is seen between the analyses conducted with matched practices (with PSs) and unmatched practices (see Appendix 3).

Recruitment

Purposive sampling was used because of the unique population being studied (GDPs and patients who had participated in the NIHSCB pilot contract). The relevant GDPs were identified by the NIHSCB. The PP and one AD from each participating practice were recruited. Patients were identified by PPs and ADs from participating practices and were given information packs about the study with instructions to contact researchers at the University of Manchester if they were interested in taking part in an interview.

Interview structure and context

The interview questions were designed to explore GDPs’ and patients’ experiences of the shift to capitation and were developed iteratively by the research team. We also sought input from patient and GDP representatives to ensure that the questions asked were meaningful. Interviews were conducted face to face with 11 PPs and were held in private rooms within the dental practice (a staff room, office or empty surgery). Interviews with seven ADs and 14 patients were conducted over the telephone. The interviews varied in length, with PP interviews ranging from 30 to 50 minutes and interviews with the ADs ranging from 20 to 30 minutes. Interviews with patients were much shorter, lasting around 8–15 minutes each. All interviews were recorded and then transcribed verbatim.

Data analysis

Data were collected continuously, starting in March 2016 and ending in July 2017, covering the end periods of phases 2 and 3. We undertook a thematic analysis of individual transcripts, which was informed by realist principles in order to determine ‘what works for whom and under what circumstances’. By using an open-coding technique, in which interview data were examined line by line, we systematically coded and catalogued the transcripts. We recorded the results using NVivo software version 10 (QSR International, Warrington, UK) and sought to identify the mechanisms through which specific outcomes occur.

Overview of sample

The practices in Northern Ireland that were visited by the research team were heterogeneous and were based in a range of urban and rural settings. Some PPs were sole practitioners whereas others had up to 14 ADs working with them. There was a broad mixture of male and females, and ages ranged from ≥ 24 to 60 years.

The patients interviewed ranged in age from ≥ 20 to 80 years and again were broadly an equal mix in terms of gender. All patients had recently attended their NHS dental practice for a routine check-up.

Results: practice principals and associate dentists

Results: key stakeholders in the Northern Ireland Health and Social Care Board

By conducting a set of brief interviews with project leads and senior management at the NIHSCB, we gained further insight into the way in which GDPs responded to the contract change.

Some staff mentioned the tendency for some GDPs to exploit loopholes and opportunities within contracts. There was also the recognition that GDPs’ motivations do vary and that they have to juggle the tension between adherence to their professional ethics on one hand versus business acumen and entrepreneurship on the other:

I think, well the first one obviously is you need . . . you’ve got to be careful if you’re completely, you know, changing it. You’ve got to think, are there loopholes, or are there, you know, ways around this, because these are our businesses at the end of the day, and I think we found that out very early on . . . GDPs generally, I think, just want to play by the rules. But, you know, I think if you make the rules too complicated, like our current SDR [Statement of Dental Remuneration], it, kind of, encourages that, maybe, behaviour . . . [I]t was such a diverse group of practices, and characters, and you did expect everybody to respond differently, I think. So, I did expect that, sort of, varied response that we’ve got.

NIHSCB1

There was a lot of rumour that [pilot] practices were advertising on the radio, extra private sessions and, but whenever, I suppose, it’s very difficult from the board’s perspective to monitor that because we have no insight into what their private work is.

NIHSCB4

I mean, the GDPs are shrewd, by and large. I mean, they’re business people, ultimately. And, I’m sure they’ve, some practices more than others, I mean some have probably just slept walked straight into this, and taken it completely at face value, and made the most of it, and thought, oh, this is great, you know, and just reacted to it. Whereas, others, I think, have been proactively basically reading the rules. OK, now just before we get into this, now this is what they’re saying, before we get into this game, let’s understand the rules of this game, how we can make this work for us. And, they are maybe the ones who more quickly started to do things differently. Maybe started to stretch out recalls, thinking, you know, that well, I mean there’s no point spending our time doing a lot of treatment here on patients that were not going to get paid for it, you know, whilst we could get paid for it down the far end deferring the treatments. You know, make it, allowing a little bit more space for their private. All those sorts of angles, you know.

NIHSCB5

I suppose inevitably we all thought that GDPs would reduce their activity. But the other side of that is, you know, the other human bit of it is that there are GDPs out there who want to do the best for their patients . . . I think you’ve a spectrum as well. You’ve one or two who will have looked at the system and thought, how can I maximise it? And that’s good . . . I think a lot of them, it is quality of care as well. I wouldn’t say they don’t care about the quality level. They care very deeply about the quality, but it’s how do you get the quality? And where is the line between the NHS and private? So if I’m not making enough on the NHS, the NHS is not giving me enough, I go private.

NIHSCB6

The idea that GDPs were situated in a particular local community and that this may place constraints on opportunistic behaviour was also mentioned:

There is such a range of practices within Northern Ireland, and different locations. You know, a small single-handed practice in a country area is so different to a large, you know, practice in a, sort of, thriving area in a city.

NIHSCB1

It’s still quite small, Northern Ireland, and people feel a dedication to their patients. So, their patients are their patients for the long term. Their reputation is really important. That’s word of mouth. So, you’re not going to short change, you know, patients, just because you can in a pilot, you know, you’re just not going to do that.

NIHSCB2

Staff commented on what they saw as the future of dentistry in Northern Ireland. The need for a core service was mentioned by some, with cost containment being achieved by a reduction in the range of NHS treatments to be offered. The need to incentivise quality and prevention was also mentioned, as was support for some form of capitation and in one instance a local innovation fund:

I still think the SDR [Statement of Dental Remuneration] is not a bad thing, but I think it needs simplified. I think there needs to be a core . . . well paid, but core things that people should be provided. So, you know, somebody should be always be got out of pain, an extraction, a simple filling, but I think . . . I do think that GDPs should be paid more . . . So, pay them better for less things, and then people can still get all those other things like endodontics and, you know, bridge work, but they pay a private fee for it. And, I think that’s very similar to most other aspects of life.

NIHSCB1

I think I probably would have a central payment, but I would have incentivising additional payments for equality, and for areas like prevention. I’d also have, kind of, an innovation, and almost a grant-based aspect to it, where people have flexibility to apply things differently in their own context on their own site that might work well for them.

NIHSCB2

I think I’m a convert now to a capitation-type model. There may well be extra quality payments involved. There may be some, say, laboratory items that may well have to be fee per item, but the vast majority . . . I think, would be capitation based. You know, I’m a fan now of that system, especially because if you wish to carry out some private dentistry you still can.

NIHSCB3

Going by the feedback, even in the old surgery pilot when we visited a lot of practices and speaking to people face to face, the feeling that I get that what the profession would like is a core service and then other things set outside that . . . So, I suppose, like, your general drilling and filling and anything that you need in terms of maintaining your oral health. But anything . . . crowns and things like that, that could be seen to be just purely from a cosmetic point of view, you want a white filling, you know, you’d be paying a separate fee for that.

NIHSCB4

I think something like capitation has to be in there, because it’s cost containment and no matter what way you look at it, it has to be cost contained. I think that it’s also looking at the administration side of it. I would be much more keen for the capitation to have some sort of quality indicators within that as well, you know, to be paid on quality indicators so that some sort of . . .

NIHSCB6

Finally, the observation that some GDPs might have been telling us what we want to hear regarding their experience of the pilot was mentioned by one staff member:

You see, and that’s, I mean, it’s interesting all of this, because, on the other of it there is a bit of a game of chess going on here, you know, and I would say in the evaluation that there’s always this, sort of, level in their head, that there’s the right answer, that there’s the truth, and a good answer to tell you. And, you know, exactly what happens is hard to know, and you can never really confirm what has gone on. But, you know, we know the hard data that’s, you know, it’s hard to argue against, does seem to suggest that there’s less prevention. And, I sometimes wonder, you know, is there an element of this where the GDPs genuinely believe that they are providing more prevention, and that this is all effective. And, you know, it’s almost a deep-seated, almost like a faith, issue, which they are able to separate out from their rational brain, when maybe they do think that they’ve looked at numbers, and went, gee, you know what, I’m probably not doing as much preventive interventions as I thought I was. I don’t know.

NIHSCB5

Results: views of participating patients

Sampling strategy

The sampling strategy, decided prior to recruitment, was to avoid the sampling error (possible error that stems solely from the fact that data are collected from a sample rather than from every single member of the population) by obtaining a large number of responses collected in each study phase. To reduce bias (potentially caused by a systematic difference between responders and the target population), the PPI group and policy stakeholders (senior staff at the BSO and the NIHSCB) were consulted on a sampling frame that could include all potential representatives of a heterogeneous population. The aim of the sampling frame was to balance, before the analysis began, the demographic and socioeconomic characteristics of patients attending intervention and control practices.

Questionnaires were distributed to patients registered with practices in the intervention and control practices at each study phase (pre pilot, during the pilot and post pilot). The first part of the sampling frame was designed to ensure the adequacy of the analysis by reducing the influence of selection bias on questionnaire answers and on differences in the response rate between intervention and control practices and between study phases. The approach taken was to select for each pilot practice five matched control practices from all dental practices with an NHS contract in Northern Ireland. The five control practices were those most similar to each pilot practice based on matching criteria of practice size, proportion of registered patients exempt from fees, proportion of registered patients who were adults and rural/urban status. Matching took place during the baseline study phase for all three study phases, hence there was no change in the intervention and control practices between study periods. The benefit of this matching approach was that it decreased the likelihood that an analysis of the questionnaire was influenced by the effect different practice characteristics (e.g. small practices vs. large practices) may have had on patients’ opinions of the health care they received. Consequently, findings of a difference in the answers given by respondents in each practice group between study phases can be attributed to changes in the remuneration conditions of intervention practices between study phases rather than differences in practice characteristics.

The next step of the sampling frame was to ensure that the patients who responded to the questionnaire were representative of the registered population of Northern Ireland in domains correlated with the oral health of the population. All eligible (registered) patients in each study phase were subdivided by dental practice group (control or intervention practice group) and then further subdivided into three domains used to stratify the sample: exemption from NHS charges (i.e. a proxy for socioeconomic status), gender and age. This meant that there were 12 strata in each practice group: (1) exempt (from NHS charges), male, aged 18–39 years, (2) exempt, male, aged 40–59 years, (3) exempt, male, aged ≥ 60 years, (4) exempt, female, aged 18–39 years, (5) exempt, female, aged 40–59 years, (6) exempt, female, aged ≥ 60 years, (7) non-exempt, male, aged 18–39 years, (8) non-exempt, male, aged 40–59 years, (9) non-exempt, male, aged ≥ 60 years, (10) non-exempt, female, aged 18–39 years, (11) non-exempt, female, aged 40–59 years, and (12) non-exempt, female, aged ≥ 60 years.

Questionnaire packs (letter of invitation, patient information sheet, questionnaire and Freepost return envelopes to the workplace of the research team) were sent out in strata for the practice groups at each phase. These were sent to eligible patients (registered at each practice) selected at random from each stratum of patients in their practice group using contact information recorded by the practice (and available to the BSO).

Questionnaire distribution

There were 9000 questionnaire packs (4500 for patients in intervention practices and 4500 for patients in control practices), which were sent out in waves of 250. The first wave of questionnaires consisted of an equal number of patients from each stratum in each practice group. In subsequent waves, invitations were not distributed equally across all strata. They were targeted differently, with a greater proportion of the questionnaire packs distributed to strata with low response rates. This approach reduced the overall response rate but had the benefit of producing similar numbers of responders in each stratum to ensure that certain strata were not under-represented.

Data analysis

Difference in difference models were estimated using individual patient responses to the questionnaire to evaluate whether or not there was a change over time in the difference between registered patients in control practices and pilot practices in their responses to the questionnaire. The disadvantage of using each question as a dependent variable in a DiD model was that the answers to a proportion of questions were on a categorical scale. An interval-dependent variable (binary or categorical) requires an assumption that the interval space between each ordered response category is an equal distance apart. This was clearly not the case for a number of the questions on the questionnaire; for example, ‘How often do you use dental floss?’ has ordered responses that are not equally spaced periods of time, namely every day, every other day, two/three times a week, less frequently or never. Athey and Imbens81 provide an overview of the problem.

One way of avoiding the problem of non-interval variables is to collapse the response categories into a dichotomous variable. Regression with a proportions measure (e.g. proportion of patient responses that answered ‘everyday’ to a question on the frequency of brushing teeth) avoids the need for assumptions about equal intervals because it involves creating a dichotomous variable from a continuous measure, and hence the coefficient has a meaningful interpretation. The sensitivity of our results to the equal interval assumption was tested by estimating DiD models with dependent variables that were the proportion of respondents in each sample strata who answered in a ‘relevant’ response category. This approach was used only as a sensitivity check because the number of observations available for analysis was reduced from the number of patient responses in the two study periods being compared (approximately 1200 observations when capitation is compared with baseline) to 48 observations: 12 (the number of strata) pilot practice observations and 12 (the number of strata) control practice observations in each study phase. ‘Relevant’ response categories were decided by examining the distribution of responses to a question and, where necessary, combining categories to create a proportions measure that has the least skew in its distribution. These models were estimated without matching methods and control variables because the proportions measure was already created from a similar population of patients (all from the same sampling strata: similar in terms of age, exemption status and gender). For a given survey question, the statistical significance of the DiD estimator did not change from being non-significant to statistically significant (at typical threshold levels of 1%, 5% and 10%) when the dependent variable was changed from an interval to a proportions measure, and a statistically significant result did not become non-significant. This suggests that the statistical significance of the DiD results was not affected by the assumption of equal proportions.

Incentivising prevention

A change from FFS to capitation does generally incentivise prevention. In this study, items of preventative care, such as fissure sealants, periodontal treatment and the provision of radiographs, reduced along with all other treatments during the capitation period. However, this may have been as a result of the limited duration of the intervention. A more fundamental question is ‘should promoting prevention in primary dental care remain the sole priority?’. This was one of the reasons cited for the change from FFS to capitation in children in the 1990 NHS contract. However, the recent evidence shows that caries prevention provided solely in dental practice can have limited effectiveness, when compared with community-based programmes. 52,53

Incentivising quality

Incentives to improve quality were not included in the intervention. Any investigation into the impact of changes to the organisation and delivery of dental health services on quality will be severely hampered by a lack of understanding of what quality means in primary dental care and how to measure it. The questionnaire used in the study was designed by our PPI group, but may have been limited by this lack of understanding and the indicators used may have lacked responsiveness. This seems to be the case with current patient satisfaction indicators. The results from the GDP patient survey showed that in January to March 2017, 50% of respondents had a very good experience and 35% had a fairly good experience at their GDP’s practice and only 4% had fairly poor and 3% had very poor experiences. At face value, it would seem that the majority of dental services are generally providing a very high standard of care. Alternatively, this could be a result of a measure with poor responsiveness or patients having little knowledge or understanding to enable them to judge what constitutes high-quality care. If the dental profession and NHS policy-makers are not certain of what constitutes good-quality dental care and how to measure it, it is unlikely that the majority of patients are equipped to provide an informed opinion.

Concerns have been voiced that a simple capitation arrangement could lead to ‘supervised neglect’, whereby GDPs receive their capitation fees but neglect to provide essential treatment for patients. The results of this study and studies in the literature1 show that activity is depressed under capitation but the qualitative element of this study shows that many GDPs see themselves as part of the local community with a responsibility to look after the dental needs of that community, which brings with it a sense of community accountability that would check unprofessional behaviour. Concerns about ‘supervised neglect’ would also be tempered by previous studies that suggest that treatment is delayed rather than withheld. 7 However, the Referral Dental Service in Northern Ireland did inspect completed courses of treatment on patients and reviewed records in each practice as part of the ongoing monitoring. No supervised neglect was observed. From a public health point of view, especially with an ageing population for whom self-care of heavily restored dentitions is a future concern, incentivising less activity is probably better than artificially stimulating activity through arbitrary activity targets (in blended contracts) aimed in part to ensure that PCR levels are maintained.

Maintaining and improving access

Registrations increased prior to the introduction of the capitation period to ensure that practices met capitation targets. The impact of the change in remuneration on registration was negligible. We estimated the impact of the introduction of capitation on registration across the whole of Northern Ireland, by applying our results to all NHS practices in Northern Ireland. Using the DiD result for practice list size, the scaled-up effect of the introduction of capitation across Northern Ireland would result in an increase in the number of registered patients in the population by 365. In April 2017, there were 1,198,286 dental-registered patients across Northern Ireland.

One of the attractive elements of a capitation-based system for commissioners is that it secures a minimum amount of access, as measured by patient registration. However, the research findings question how we should measure access. Registration is one way to measure access to NHS care, but for patients this does not seem to be a meaningful measure, as patients reported that they could not get timely access to their GDP when they needed it. One possible downside of a capitation-only contract would be potentially blocking access to care for unregistered patients who may have high needs, and would, therefore, represent a financial risk to practices operating under a capitation system if they were to register them. In this study, expansion of the list size of practices to ensure that capitation targets were met was done largely by encouraging patients with lapsed registrations to reregister. This tactic reduced the risk for practices, as previously registered patients would be known to the practice and would be likely to have low treatment needs, as opposed to new patients who may require a large amount of treatment, the costs of which would have to be met solely from the capitation fees that they generate. If a new contract is to be primarily based on capitation, careful thought needs to be given to how those with greatest need can access care. Providing care for a geographically defined population rather than a historically registered population and making time (and resource) provision through dedicated sessions for unregistered patients may need to be considered as options.

Maintaining income from co-payments

One of the key concerns about moving from a FFS to a capitation-based system is a significant fall in PCR, leaving a gap in the NHS dental budget. This study reported a fall in PCR that was immediate and substantial. If capitation was introduced in Northern Ireland, the impact on the annual NHS dental budget can be estimated by extrapolating our findings to all NHS dental practices. The DiD result for the patient fee contributions showed a reduction by £2403 per practice per month when capitation was compared with baseline FFS. If this figure were multiplied by the number of months in a year and the number of NHS dental practices operating in Northern Ireland (387 in April 2017), capitation would reduce the gross annual co-payment income received by the NHS by £11,159,532, which equates to approximately 8% of the dental budget.

The fall in PCR is worrying if policy-makers want to realise the benefits of capitation (a capped and predicable budget, attractive to the dental profession, removal of incentives for early intervention and possible overtreatment). Although there may be a degree of ‘bounce-back’, PCR rising after an initial abrupt fall because of GDPs delaying treatment, there will be a net fall in co-payment revenue when compared with a FFS system. A key question for policy-makers is how to mitigate this fall in the overall budget. A number of options are available. They could simply take the hit, making good the drop in income by an expansion in the NHS dental budget; this option is unlikely to be attractive in the current and foreseeable economic environment. One simple way of correcting this shortfall would be to increase patient charges. In effect, this happened overnight in England with the introduction of the 2006 contract: a check-up under FFS was £9, whereas the charge for a band 1 course of treatment was £19, an 111% increase, with little public outcry. Our data estimate that, based on the average patient contribution revenue received by control practices per month scaled up to all practices in Northern Ireland, a 27.62% increase in patient charges would be needed to prevent a fall in PCR affecting the overall budget. This assumes that an increase in price would not produce a downturn in demand for NHS treatments; this scenario did not happen in England.

Another option would be to implement a blended contract, in which the bulk of remuneration is based on capitation with a proportion retained as FFS to stimulate activity, as in the English prototypes. The problem with a blended contract is how to calculate the size of the FFS component; this calculation runs the risk of being too focused on artificially generating activity and, hence, income at the expense of encouraging a needs-led approach to care. A more radical option would be to remove co-payments entirely and provide a capitation-based system with an attenuated ‘core’ service funded entirely within the current state-funded component of the dental budget. This could be politically difficult, but if access to a defined list of treatments essential to secure dental health were freely available, and more complex and cosmetic treatments were offered privately, such a system might be attractive. The PCC document Talking Teeth: Patient Views of General Dental Services in Northern Ireland49 suggests that this option merits further investigation.

Ensuring that the profession is content with a capitation-based system

The study demonstrated that capitation is popular with PPs and, therefore, senior members of the profession, but was seen as a threat to non-equity-holding associates. Capitation provided greater opportunities for GDPs to upsell private treatment, which may not be detrimental to NHS care if the focus of privately funded care is largely on cosmetic treatments (e.g. crowns or treatments that patients demand but that have a poor evidence base, such as scale and polish). However, in any new system, patients and the profession need clarity about what treatments are available via the NHS and what treatments are not. One of the possible downsides of a poorly monitored and managed capitation-based system is that NHS care could be marginalised by the upselling of care provided on a private basis that should be available on the NHS. Capitation was less attractive for associates as under a capitation system their contracts would be with practices, not with the NIHSCB. This could reduce their bargaining power, putting downward pressure on wages and possibly reducing job security.

Impact of service redesign on workforce planning

Changing a dental contract will influence the volume and service mix of dental treatments provided and consideration needs to be given to how to provide the resulting service mix in the most efficient way. A capitation-based system incentivises service broadening, providing fewer and a simpler range of treatments to more people. This would suggest a larger role for skill mix and role substitution. Service planning and workforce planning should be conducted simultaneously, and simply replacing like with like in terms of current skill mix is unlikely to meet the evolving needs of the population using the service in the future. A more sophisticated, needs-based approach to workforce planning is one way of addressing this issue.

In conclusion, it is unlikely that a single (complex) NHS dental contract can meet all of the required policy objectives: secure a minimum level of access, provide a predictable capped budget, encourage prevention, ensure that good-quality care is provided, secure sufficient income from PCR and ensure that patients and GDPs are happy with a new system. Policy-makers need a range of tools to both monitor and influence behaviour prospectively in any new system; any contractual change also needs to have built-in flexibility to change as population needs change.

Flexibility to include performance management within a capitation contract may provide GDPs with the autonomy they desire while also enabling policy-makers to ensure that population needs are addressed. The need for performance management was demonstrated in this study by the large between-practice variation in outcome measures. Performance management does not need to be labour intensive; it could be informatics driven by benchmarking practices against agreed metrics using existing activity monitoring and payment systems in Northern Ireland. Simplifying the new NHS dental contract, rather than relying on a complex blended contract, could avoid the need for new complex IT systems that England has been struggling with. The dental reference officer function could also be used to support a peer-reviewed system of quality improvement.

What should be the aims and limitations of NHS dental services?

This would require a consensus view on what NHS services should aim to provide and what they will not provide in the context of changing population needs, developing health technologies and pressure on public sector budgets. Issues such as reducing the levels of disease in children, the needs of a growing, predominantly dentate elderly population and concentration of the dental disease burden in disadvantaged communities remain public health concerns.

Is less more?

Does a more restrained approach to provision of dental restorative treatment, encouraged by capitation, provide greater long-term population health gains and savings for the NHS than an incentivised approach to intervention? In the context of a growing elderly, dentate population this question is growing in importance.

Does paying general dental practitioners for prevention provide an attractive return on investment for the NHS?

The idea of paying GDPs to provide prevention has been accepted since the 1990 NHS contract and is supported by policy in each of the home countries. However, with emerging evidence, perhaps this concept now needs to be challenged. Should GDPs make the main contribution to prevention, on a patient-by-patient basis, within the NHS? This has been a long-standing assumption, but HTA trials (e.g. Northern Ireland Caries Prevention In Practice and the Improving the Quality of Dentistry trials) are starting to challenge this notion. 57,58 Greater use of skill mix in practice and community-orientated approaches to care could help augment existing population-based interventions and provide more continuity, rather than focusing entirely on practice-based interventions delivered by dentists.

What should the patient pay for?

Given the increasing pressure on NHS budgets, is there a case for restricting dental care to a list of essential treatments necessary to secure health? This question raises a number of additional questions. How could or should such a list be decided? What impact would the introduction of a ‘core service’ have on the population’s oral health? What effect would the introduction of a core service have on health inequalities?

What additional interventions can supplement changes in remuneration systems to achieve policy goals?

Research is required to develop and test supplementary interventions that can work in a complementary way alongside a remuneration system to achieve desired goals, for example the quality improvement interventions discussed above, but also efficient performance monitoring and management methods, perhaps using informatics approaches to automatically identify outliers deviating from predefined expected patterns of service delivery.

Research into workforce planning

Changing the payment system changes service provision, which changes the workforce needs of the service. Currently the NHS has an unstructured approach to dental workforce planning that is not aligned to the needs of the system or, more importantly, the needs of the population. Research is required into the development of different workforce planning models and their impact on productivity and overall costs, accounting for role substitution, shifting work patterns and expectations.

How do you measure and improve quality in dentistry?

Wanting to incentivise quality is a common thread for policy-makers, but research into quality in dentistry is 20 years behind primary medical care. Dentistry has some fundamental differences to services provided by general practitioners: patients are encouraged to attend regularly and asymptomatically, it is still a largely surgical discipline and there is a large cosmetic element to the care provided. In England, the quality indicators included in the DQOF have been chosen arbitrarily and have not been validated. Research is needed to understand quality (including patient safety) in the dental context, how to measure it with validity and reliability, and how to improve it. Learning health systems are being developed to support continuous service improvement; it would seem timely to investigate an informatics-supported approach to dental service improvement in the NHS.