Behaviour change intervention (education and text) to prevent dental caries in secondary school pupils: BRIGHT RCT, process and economic evaluation

Introduction

The high prevalence and severity of dental caries adversely affect children and young people in the UK, bringing with it an economic burden. Reducing the disease, and therefore its negative impact, is a public health priority. 1–3 Dental caries is largely preventable and there are a number of successful interventions across the UK for young children, including those based on toothbrushing with fluoride toothpaste. 4,5 However, there is a lack of evidence for community-based oral health improvement programmes in older children and young people that target toothbrushing practices. A behaviour change intervention incorporating a school-based lesson together with mobile health (mHealth) technology (through mobile phones) has the potential to have a positive effect on this oral health behaviour and ultimately reduce dental disease and its sequelae.

Prevalence of dental caries in young people

Dental caries is the most prevalent non-communicable condition worldwide, with untreated caries affecting 2.4 billion people. 6 In the UK, there have been reductions in caries experience for young people reflected in the 2003 and, most recent, 2013 Child Dental Health Surveys (CDHS) which cover England, Wales and Northern Ireland. The proportion of children with teeth showing active carious lesions, dental restorations or which had been extracted due to dental caries dropped from 43% to 34% in 12-year-olds and from 56% to 46% in 15-year-olds over the 10-year period. 7 In Scotland, over a similar period, the annual National Dental Inspection Programme (NDIP) reported reductions in obvious caries experience for 12-year-olds, from 47% in 2005 to 27% in 2015, and in the most recent survey carried out in 2019, there has been a further reduction with 20% affected. 8–10

Despite these improvements across the UK, the prevalence of dental caries remains high. The overall reduction in the prevalence is positive, but potentially masks three underlying issues. First, there is a persistence in underlying oral health inequalities. The positive association between living in a deprived area and prevalence of dental caries is seen across the UK11,12 and globally. 6 A variety of indicators are used for measuring deprivation. Using eligibility for free school meals (FSM) as a measure, the 2013 CDHS found that the proportion of children with obvious caries experience was 46% among 12-year-old children eligible for FSM, compared with 30% among those not eligible for FSM. 7 Similarly, for 15-year-olds, this was 59% for those eligible for FSM compared with 43% for those not eligible. 7 In Scotland, the difference in obvious decay experience levels between those living in the most and least deprived areas, was 26.3% in 2009 reducing to 18.6% in 2019. 10

Second, although the overall prevalence has reduced, the disease is severe in those affected. The burden of disease in children can be measured using the mean number of decayed, missing and filled teeth (DMFT) per child. In 2013, for England, Wales and Northern Ireland, for children with obvious decay experience, this was 2.5 teeth per child7 and in Scotland, 2.1 teeth per child. 10

Finally, although there has been a reduction in the prevalence of dental caries in children, there is a high burden of untreated disease. The 2013 CDHS found obvious untreated decay into dentine in 19% of 12-year-olds and in 21% of 15-year-olds. 7 Similarly, in Scotland, in 2019, untreated caries into dentine was found in 8% of 12-year-olds. However, this rose to 40% who had untreated disease, when only the children with carious lesions were included. 10 COVID-19-related challenges with access to dental care will potentially increase the burden from disease for all and widen inequalities. 13,14

Impact of dental caries on young people and their families

The impact of dental caries on the lives of children and young people has been well documented, including pain15 and difficulties with eating and sleeping, with disrupted social activities and absences from school. 12,16 Families report guilt, lack of sleep and taking time off work related to their child’s carious teeth in 5- to 16-year-old children. 17 Following treatment of dental caries, there is a reduction in these impacts for both children up to the age of 16 years and their families. 16–19

In a secondary analysis of data from the 2013 CDHS, 3859 children were included to investigate the relationship between the presence/absence of severe dental caries (e.g. pulp involvement, ulceration, fistula or an abscess in at least one tooth) and seven items from the Family Impact Scale. 20 Three in 10 parents (29.5%) reported that their child’s oral health impacted on their family life. The areas most frequently affected were parents having to take time off work (15.5%), feeling stressed (14.2%), feeling guilty (10.4%) and their child needing more attention (13.4%); the least frequent impacts related to financial difficulties (2.4%), disruption in normal activities (6.1%) and disturbed sleep (7.4%).

For those children waiting for tooth extractions in hospital in the North of England, 67% of parents reported that their child had pain and 38% reported parental sleepless nights. 21 In a national health survey conducted in the USA, dental pain was reported in 32% (n = 7.5 million) of children whose parents stated that they had caries. 22 Around 50% of 12- and 15-year-olds reported toothache and 6% of 12-year-olds and 3% of 15-year-olds reported difficulty with schoolwork because of their teeth and mouth condition over the previous 3 months in the 2013 CDHS. 7

Economic impact of dental caries

Treatment of oral diseases is expensive, with NHS England spending £3.4 billion in 201423 and £3.6 billion in 2021 for adults and children in treatment costs. 24 Dental expenditure in the UK has been estimated to be around £196 per capita. 25 In 2017, hospital admissions (usually for general anaesthetic) in NHS England for tooth extractions due to dental caries, for children aged 0–19 years cost £33.0 million. 1 However, the economic costs when taking into consideration time off work and school are much greater. Worldwide, these indirect costs are estimated to be US$144 billion per year. 26

Aetiology and prevention of dental caries

Dental caries is a biofilm-mediated, diet-modulated, multifactorial, non-communicable, dynamic disease resulting in mineral loss from dental hard tissues. 27 It is determined by biological, behavioural, psychosocial and environmental factors. The main modifying factors in the development of caries are sugar consumption and the use of fluoride. 28 Therefore, prevention of dental caries can be achieved by optimising exposure to fluoride and reducing sugar consumption by improving diet. These changes can be encouraged at an individual level by professional intervention or through community- or population-based public health programmes. 29

Brushing with fluoride toothpaste is one of the most effective measures to prevent caries. 5 The Scottish multicomponent oral health programme targeted at young children has toothbrushing in schools as a major component. It was introduced as a pilot in 2006 and then delivered across all Health Boards in Scotland from 2011. Around 80% of 12- and 15-year-olds stated that they brushed their teeth twice per day in the 2013 CDHS, although this was 82% of those not eligible for FSM and 72% of those who were eligible for FSM. 30 The same survey found that only 25% of 12-year-olds and 32% of 15-year-olds were considered to have good periodontal health (plaque in no more than one sextant, no gingival inflammation, no calculus), which may demonstrate that there is less adherence to oral hygiene practices than reported. 31 This is in line with observational studies that have shown current levels of efficacy, frequency and duration of toothbrushing to be inadequate,32–34 thereby increasing the risk of caries. 35

School-based oral health promotion programmes

Community-based oral health interventions in the UK have been aimed mainly at younger children of pre-school age or in primary education. Few interventions are aimed at reducing dental caries in young people, despite adolescence being a critical transition stage where independence develops, diet begins to become self-managed and oral health behaviours change,36 with irregular toothbrushing associated with reduced fluoride exposure and resultant increased caries risk reported. 37 While national oral health promotion programmes such as Childsmile have been implemented in Scotland, and Designed to Smile has been developed in Wales, these programmes focus mainly on children under 12 years of age. Examples of current community-based interventions to improve the oral health of young people have been categorised into oral health education interventions and more complex interventions involving additional activities such as clinical prevention measures alongside the education component. 38 However, limitations of these existing programmes include the lack of use of behaviour change theory and that these interventions are not embedded as statutory content within the school curriculum, as recommended by the World Health Organization’s (WHO) Health Promoting Schools framework. 38–42

Behaviour change interventions

To reduce caries prevalence and its associated burden, young people need to adopt and/or maintain oral health behaviours, including regular toothbrushing with a fluoride toothpaste and sugar reduction. Successfully intervening to induce behaviour change, whether at an individual or community-based level, requires an understanding of these behaviours in the context of people’s lives and developing interventions to change behaviour. For example, an intervention to improve toothbrushing behaviour may be considered to require education (improving knowledge and understanding) plus persuasion (that the behaviour will produce a positive effect) and training in the necessary skills. The type of intervention required then influences the choice of the delivery method; both the setting where it is delivered, the vehicle(s) used and the timing and frequency. For example, digital interventions are recommended as appropriate vehicles for delivering behaviour change interventions directed at young people. 43

Use of mHealth for delivering health interventions

There has been increasing interest in the development and use of health interventions (including behaviour change interventions) through the vehicle of mobile phones and wearable technologies. 43,44 mHealth has been defined by the WHO45 as ‘a medical and public health practice supported by mobile devices such as mobile phones, patient monitoring devices, personal digital assistants and other devices’. mHealth is a particular category of a wider eHealth area,46 which provides innovative ways of improving the overall health. 47

Short messaging service (SMS), also known as ‘text messages’, are the most widely investigated mHealth interventions and have been used to deliver health education, promote treatment adherence in a range of conditions and in the prevention of communicable diseases. A systematic review of preventive health behaviour change text message interventions found a small but statistically significant positive effect for the impact of text messages both shortly after cessation of the intervention and also after a period of ‘no intervention’ to demonstrate whether the effects can be maintained longer term. 48 Potential moderators of effect size were considered including the duration, tailoring and targeting of the content and how text messages were used, along with other activities such as educational content. It appeared that interventions lasting 6–12 months were associated with greater effects than shorter interventions. The limitations of the component studies included insufficiently powered studies lacking the ability to detect change, short-term follow-up, failure to blind those assessing outcomes and lack of a theoretical framework to inform the behaviour change intervention. 48,49 It has been recommended that future studies ensure the intervention is developed rigorously, the text messages are appropriate for the target population in terms of their age and wording and the messages use the participant’s name.

Most studies in this field have involved adults, although more recently there have been studies involving young people. In the UK, in a 2022 Ofcom report on children’s media use, it was estimated that 9 in 10 children owned their own mobile phone by the time they reached the age of 11, and 97% of 12- to 15-year-olds used a smartphone for texting and making calls. 50 It is perhaps the ubiquity of mobile phones that explains why smartphone ownership among young people does not seem to vary by socioeconomic status,51 suggesting that mHealth interventions may suit this age group.

mHealth interventions to improve oral health

mHealth has gained popularity and has been investigated as a possible vehicle to deliver dental behaviour change interventions. Three recent systematic reviews have evaluated the effectiveness of mHealth. They cover all teledentistry (including mHealth) on oral health promotion and prevention compared with other strategies and in all ages;52 mobile applications and text messages compared with conventional oral hygiene instructions53 in adolescents, adults and mothers of young children; and text messages only in dental patients. 54 Many of the primary studies were included in more than one review.

In the review of all teledentistry, apps and text messages were found to be used most frequently. 52 This review included 18 studies, with 12 involving young people undergoing orthodontic treatment and 4 focusing on prevention. The nine studies that focused on text messaging reminders often had other parts to the intervention (education element before or during, phone calls, etc.). The meta-analysis was carried out on the basis of outcomes for those receiving all mHealth interventions. There was a demonstration of an overall reduction in plaque scores and improvement in gingival health. However, because of the combined delivery formats (apps, texts, etc.), multicomponent interventions and variable intervention delivery times as well as follow-ups, it was not clear what the exact contribution of each of the components, including text message reminders. The second53 included 15 studies with 12 involving text messaging, of which 11 studies included patients undergoing orthodontic treatment and one study of adult patients. Again, an improvement in plaque control and gingival health was found and an improvement in knowledge. This review assessed the interventions on the basis of behaviour change techniques, which were evaluated according to the Michie and colleagues’ taxonomy. 55 Most of the studies that used text messaging combined two behaviour change techniques, prompts and cues, and information about health consequences. Twelve studies involved text messages and were included in the meta-analysis, which showed improvements for the groups that received the mHealth intervention compared to the control groups. However, follow-up times were short, ranging from 4 weeks to 12 months, although the majority (13/15) were 6 months or less. The GRADE evaluation found the quality of evidence to be very low. The third review subgroup analyses again found consistently in favour of the text intervention improving oral hygiene clinical outcomes (plaque index and gingival index). The overall summary statistics were incorrectly calculated with double counting.

Although there are studies investigating text messages, the actual effect of these as individual components within multicomponent interventions is unclear. They are of variable duration and frequency, follow-up times to investigate effect have been short (usually weeks or months), none seem to have investigated dental caries as an outcome, with dental plaque levels and gingival bleeding most commonly used as indicators of behaviour change, and, less frequently, level of knowledge. In addition, few have been delivered as community-based interventions via settings such as schools when compared to interventions delivered in clinical settings to patients. All three systematic reviews found a high degree of heterogeneity, especially in plaque and gingival health outcome measures, as well as variable, short follow-up times and poor reporting quality. Recommendations were made that longer follow-up periods were needed with standardised methodologies and outcome measures.

Community-based behaviour change text message interventions

The first, and so far only, community-based text message intervention aimed at improving toothbrushing was conducted in New Zealand with unemployed people aged 18–24 years. This study investigated the Keep on Brushing (KOB) programme of weekly text messages and provision of free toothbrushes/toothpaste. 56 The intervention was underpinned by the Health Belief Model. 57 One hundred and seventy-one participants were recruited and completed a baseline survey. No important differences were noted between sex, ethnicity or age. Participants then received a series of motivational text messages over 10 weeks. To increase recruitment, participants were also given a pack containing toothpaste and a toothbrush. Self-reported toothbrushing of twice or more per day increased from 51% at baseline to 70% at week 3, 74% at week 6 and 73% at week 9; however, by week 9, only 26% of the original participants were still taking part. The authors concluded that motivational text messages improved the self-reported oral health of this hard-to-reach group and suggested that a randomised controlled trial (RCT) including a longer intervention with tailoring of the messages was needed.

Aim

The aim of the Brushing RemInder 4 Good oral HealTh (BRIGHT) trial is to establish the clinical and cost effectiveness of an intervention for young people from deprived areas, delivered through a short classroom-based session (CBS) embedded in the curriculum and a series of text messages, compared to usual education and no text messages, on dental caries.

Trial objectives

Objectives of the BRIGHT trial:

1. conduct an internal pilot phase with feasibility components to:

   1. tailor the intervention to young people

   2. test trial processes in schools

   3. assess the feasibility of within-school cluster randomisation (by year group)

2. investigate the effect of the intervention on caries prevalence

3. investigate the effect of the intervention on twice-daily toothbrushing, oral health-related quality of life (HRQoL) and oral health behaviours

4. investigate the cost effectiveness of the intervention

5. explore implementation, mechanisms of impact and context through a process evaluation

Pupils

Pupils from Years 7 and 8 (England and Wales) and S1 and S2 (Scotland) were recruited from participating schools. These year groups were chosen purposefully to minimise disruption to English and Welsh General Certificate of Secondary Education (GCSE) and Scottish Qualifications Authority National 5 exam years; and also to confine final follow-up to within the school setting to avoid the need to follow participants to further education settings.

The LRTs delivered information sessions, typically within assemblies, in each school to pupils in Years 7/S1 and 8/S2. BRIGHT trial information packs were then distributed to the parents/carers of all pupils in participating classes in these year groups (in most cases this was all classes in the year) via post, or by sending them home with pupils. Information packs contained a cover letter signed by the school head teacher, a parent/carer information sheet, parent/carer opt-out form and a copy of the pupil information sheet and consent form. Parents/carers could decline their child’s participation by completing and returning the opt-out form to their child’s school within a 2-week opt-out window. Schools were requested to record which pupils had been opted out on a spreadsheet. If parents/carers did not return an opt-out form within the 2-week window, it was assumed they were happy for their child to decide themselves if they would like to participate. Parents/carers could withdraw their child at any point over the trial.

Eligible pupils, whose parents/carers had not opted them out of the trial, were subsequently invited to consent to take part in the trial. These pupils were provided with a pupil information sheet and were asked to complete a consent form if they were happy to take part. Schools were requested to do this within class or form time in a dedicated consent session and LRTs offered to deliver or facilitate these sessions. As the information session would have taken place at least 2 weeks prior to the consent session (i.e. before the parent/carer opt-out window), pupils were able to consent to take part within the consent session. Schools were requested to make additional pupil information sheets and consent forms available for any pupils who were absent on the day of the consent session or who wanted more time to consider participation.

The school, supported by the LRT, collected and checked completed consent forms and updated their spreadsheet to record which pupils had consented to take part, ensuring that a parent/carer opt-out form had not been received for each consenting pupil. Completed consent forms for pupils whose parents/carers had not opted them out were collected via courier and returned to the YTU where the trial team checked whether all consent forms had been completed correctly.

Pupils were also asked to complete a contact form in order to provide their mobile telephone number and to indicate their text message preference times and preferred name to be used in the text messages should they be in the intervention group. If they did not own their own mobile telephone or could not provide their own mobile telephone number, they were considered ineligible for participation in the trial. For the internal pilot phase, we requested for the contact form to be completed at the consent session and returned with the consent forms; in the main phase of the trial, the contact form was completed at the time of baseline data collection to allow LRTs to provide greater assistance to pupils in the completion of these forms and reduce errors.

Parents/carers of participating pupils

All parents/carers of participating pupils were invited to complete resource use questionnaires59 to provide data for the health economic evaluation. These were either posted to parents’ home addresses or sent home with the pupils from school.

Schools

Each school was asked to nominate a lead contact and member of administrative staff with whom the LRTs liaised closely throughout the trial to try to pre-empt and troubleshoot any problems and maximise retention. In addition, newsletters were issued on a yearly basis over the trial period. Within the final newsletter, delivered prior to the 2.5-year follow-up, schools were reminded that they could claim £500 at the end of the data collection period to cover any administrative costs associated with being involved in the trial.

Participants

The BRIGHT Youth Forum, run by a charity called Children and Young People’s Empowerment Project (Chilypep) whose lead (Lesley Pollard) was one of the co-applicants for BRIGHT, proposed a variety of methods to optimise retention, response rates and completion rates. These included prize draws for shopping vouchers, trial-branded merchandise or ‘freebies’ (such as pens, stickers and pencils), thank you vouchers, using the school’s house-point system to encourage engagement and having more senior school pupils as Research Champions to provide peer support.

All pupils who completed the baseline questionnaire and dental assessment were given a £10 voucher as a thank you. All pupils who completed the final follow-up questionnaire and dental assessment were given a £5 voucher as a thank you. Pupils received trial-branded merchandise such as pens during data collection activities in the trial.

All parents/carers who completed and returned parent/carer questionnaires were entered into a prize draw with the chance of winning £300 in vouchers (with one annual prize draw each for the internal pilot phase and the main phase).

Internal pilot phase

For the internal pilot, the sample size was calculated in order to address the following progression criterion:

• an indication of a positive effect of the intervention on self-reported frequency of toothbrushing at 12 weeks using an 80% one-sided CI approach.

As randomisation for this trial occurred within schools at the year-group level, year groups within schools acted as the ‘clusters’. At least four clusters per arm are recommended for cluster pilot RCTs. 67 It was determined that 1200 pupils from 10 schools [equivalent to approximately 284 young people in an individually randomised trial, assuming 60 recruited pupils per year group, 20% attrition at follow-up and an intracluster correlation coefficient (ICC) of 0.02 (see justification in next section)] would be sufficient to produce an 80% one-sided CI that excluded a 5% difference in the event of a zero or negative effect of the BRIGHT intervention on self-reported toothbrushing at the 12-week follow-up, assuming 66% reported brushing twice daily in each of the two groups. 68,69 A trial of this size would also allow a participation rate of 50% and a completion rate of 80% to be estimated within a 95% CI of ± 6% and ± 5%, respectively. 70 We therefore aimed to recruit 1200 pupils across 10 schools, an average of 60 pupils per year group (Year 7/S1 and Year 8/S2) per school, into the internal pilot.

Full trial (internal pilot and main phase)

In 2013, the estimated proportion of UK 12-year-olds with caries was 34%. 7 The definition of caries here is described as ‘obvious decay experience’, which incorporates untreated decay into dentine and decay that has previously been subject to restorative treatment (fillings) or tooth extraction. Based on a systematic review of interventions to increase the frequency of toothbrushing for caries prevention, a reduction of caries prevalence of 8% might be expected. 71 An individually randomised trial powered at 90% (5% two-sided α) to detect an 8% absolute reduction, from 34% to 26%, in caries would require 1376 pupils. Few estimates of school-level ICC are available for dental data. In a previous study evaluating a behaviour change programme for preventing dental caries in primary schools, an ICC of 0.01 was used, which was estimated using their own unpublished data. 72 It was agreed that the present trial would use a more conservative ICC of 0.02.

Assuming partial contamination effects (i.e. those contaminated gain half the treatment benefits) for 27% of the control group (based on findings from the internal pilot), this trial required 42 schools in total across the internal pilot and main phase of the trial, assuming within-school (year-group level) randomisation, an average of 60 pupils per year group per school, an ICC of 0.02 and 20% attrition at follow-up. This would give 90% power (5% two-sided α) to detect an 8% absolute difference, from 34% to 26%, in the proportion of pupils with ‘obvious decay experience’. Therefore, this trial aimed to recruit a total of 42 schools and 5040 pupils across the internal pilot and main phase of the trial.

Primary outcome

Secondary outcomes

Other data collected

Dental assessments

Dental assessments were carried out in participating schools, under standard dental epidemiological data collection conditions, at baseline, 2 (internal pilot only) and 2.5 years following CBS delivery. LRTs each put together dental teams, consisting of at least a dentist/dental therapist and dental nurse, for each dental assessment time point. Each child’s dental assessment consisted of a caries assessment, taking around 10 minutes, and a plaque and gingivitis assessment, taking around 5 minutes. All dental assessment data were recorded on paper case report forms. 59

Training and calibration

Prior to the start of baseline data collection for the internal pilot, a hands-on training and calibration event was run in a school with an experienced dental epidemiologist for all dental team members. Prior to baseline data collection for the main phase, another training and calibration event was held for new dental team members and update training/recalibration was provided for those who attended the initial training and calibration event. Update training/recalibration was also provided for all dental team members prior to the 2-year follow-up (for the internal pilot) and prior to the 2.5-year follow-up. Details of the training and calibration can be found as an appendix to the trial protocol, and in Appendix 4. 59

Reproducibility

At each dental assessment time point, where time constraints allowed, dental teams completed a second ICDAS assessment for 5% of participating pupils at each school to assess intra- and interexaminer reproducibility. For plaque and gingivitis assessments, the initial examination disturbs plaque and probing and can increase susceptibility to gingival bleeding so, as is standard practice, there was no reproducibility measured for these outcomes.

Pupil questionnaires

The time points for pupil questionnaire completion are given in Table 1. At dental assessment time points [baseline, 2 years (internal pilot only) and 2.5 years post CBS delivery], LRTs requested pupils to complete the pupil questionnaire either before or just after completing their dental assessment and were on hand to encourage questionnaire completion and answer any queries. At other follow-up time points, pupil questionnaires were distributed and collected by school staff or LRTs, and schools were requested to allow pupils approximately 10 minutes of class time for questionnaires to be completed. School staff and LRT members recorded reasons for non-completion of questionnaires. All pupil questionnaires were completed on paper case report forms59 and were returned to YTU via courier.

The measures included within the pupil questionnaire at each time point are summarised in Table 1. More details about the measures are provided in the Outcomes section and copies of each questionnaire can be found on the NIHR project web page. 59

Parent/carer questionnaires

For the health economic evaluation, parents/carers of participating pupils were asked to complete a resource use questionnaire at baseline, 1, 2 (internal pilot only) and 2.5 years post CBS delivery. Parent/carer questionnaires59 were handed to pupils at the time they completed their pupil questionnaire (apart from the baseline time point for the internal pilot) and they were asked to take these home to their parents/carers to complete. The questionnaire was enclosed in an envelope along with a cover letter and a freepost envelope, which parents/carers were requested to use to return the completed questionnaire via post to YTU. Approximately 2 weeks after parent/carer questionnaires were distributed (excluding the baseline time point for the internal pilot), schools were provided with a second copy of the questionnaire to distribute in order to try to maximise response rates. For the internal pilot, baseline parent/carer questionnaires,59 along with a cover letter and freepost envelope, were posted to parents/carers who had returned a consent form and shared their name and address so questionnaires could be posted directly to them.

Sociodemographic characteristics and school attendance data of participants

At baseline, schools were also asked to complete and return an encrypted spreadsheet to YTU, via the University of York DropOff secure data transfer service, in order to provide the following for each consenting pupil: sex, year group, form group, current eligibility for FSM, past and current year’s school attendance, and home postcode [to facilitate data linkage for the economic evaluation and for calculation of Index of Multiple Deprivation (IMD) scores]. Participating schools were also asked to provide the past or current year’s attendance record for each participating pupil at 1, 2 (internal pilot only) and 2.5 years post CBS. Average year group attendance rates were also requested after final data collection for all previous academic years within which the BRIGHT trial had been active.

School-level data were also captured at recruitment, from publicly available sources, on the proportion of children eligible for FSM. 64,97–99 Publicly available school inspection ratings (i.e. Ofsted for schools in England, Estyn for schools in Wales and Education Scotland for schools in Scotland) were also obtained. 64,100,101

Adverse events, suspected pathology and child safeguarding

Due to the nature of participant involvement, no serious adverse events or adverse events were anticipated that would be unexpected and related to being in the trial. However, a procedure was described in the protocol58 to capture and report any complications associated with the trial raised by participants, parents/carers or dental assessors. Any suspected serious pathologies were also recorded and a procedure put in place to enable a referral to a relevant dental service. A procedure was also established to deal with child safeguarding issues to ensure the young person’s school and parent/carer could be informed and any further action taken.

Data summaries

All data collected at follow-up (both dental and from the pupil questionnaires) are summarised by randomised group and overall. This includes the primary and secondary outcomes that were formally analysed, and also data not included in hypothesis tests such as toothbrush/paste availability, influences on toothbrushing and contamination. The following pupil-level dental outcomes are summarised descriptively for each time point:

• presence of obvious decay experience in at least one permanent tooth as measured by D_ICDAS4–6MFT

• total number of permanent teeth assessed

• total number of D_ICDAS4–6MFT

• number of teeth extracted due to caries

• number of decayed permanent teeth based on ICDAS 4–6 definition, that is, number of teeth whose highest surface caries severity code is 4–6 or restoration code is 8

• number of filled permanent teeth based on ICDAS 4–6 definition, that is, number of teeth whose lowest surface caries severity code is 0–3 and the restoration code is 3–7

• presence of decay experience in at least one permanent tooth as measured by D_ICDAS1–6MFT

• total number of D_ICDAS1–6MFT

• number of decayed permanent teeth based on ICDAS 1–6 definition, that is, number of teeth whose highest surface caries severity code is 1–6 or restoration code is 8

• number of filled permanent teeth based on ICDAS 1–6 definition, that is, number of teeth whose lowest surface caries severity code is 0 and the restoration code is 3–7

• dental plaque score

• gingival bleeding score

• number of teeth where there were sites with gingival bleeding

• presence of an orthodontic appliance, by type, and for pupils with a removable orthodontic appliance, whether this was removed during the assessment

• whether blinding of the dental assessor to the child’s group allocation was maintained.

We also summarise the following, based on comparing baseline and 2.5-year dental assessments:

• the number (%) of pupils who moved from ‘caries negative’ at baseline to ‘caries positive’ at 2.5 years (for both D_ICDAS1–6MFT and D_ICDAS4–6MFT definitions)

• the number (%) of pupils who developed new carious lesions over this time, defined by an increase in total number of D_ICDAS1–6MFT/D_ICDAS4–6MFT

• mean caries increment using the individual surface scores to do the calculations (dmft_surface_1@2.5 years – dmft_surface_1@baseline), where the individual surface scores are 1 (positive for caries, based on D_ICDAS1–6MFT/D_ICDAS4–6MFT definitions) or 0 otherwise. Therefore, increments for each surface can take values of −1, 0 and 1, where −1 indicates reversal from carious to sound, 0 no change in surface status and 1 change from sound to carious surface. A maximum of 120 surfaces were assessed per pupil. The total caries increment was calculated as the sum of these individual increments, treating reversals in the following three different ways:

  • a net caries increment on a surface level – leave the reversals as is when calculating the caries increment

  • a crude caries increment on a surface level – summing only the positive increments and ignoring any reversals

  • consider that the negative reversals have been erroneously coded and replace instances where the total caries increment (all surfaces combined) was negative with a zero increment.

Changes from the protocol

In the published trial protocol, the proposed method of analysis of dental outcomes involved regression models that accounted for repeated measures, that is, the dental outcomes at 2 and 2.5 years. Similarly, for self-reported twice-daily toothbrushing, a repeated measures binary logistic model incorporating 6 months, 1, 2 and 2.5 years was planned (not including earlier time points since these were for pilot schools only). However, a subsequent trial amendment removed the 2-year dental assessment for main trial schools, and, at this point, it was clear that very few main trial schools had been able to provide data at 1 year due to school closures caused by the COVID-19 pandemic. Therefore, a repeated measures approach was deemed no longer appropriate nor necessary, and it was detailed in the statistical analysis plan (SAP) (reviewed and approved by the TMG, TSC and DMEC prior to the 2.5-year data collection) that analyses would largely take the form of separate (cross-sectional) regression models for the outcomes at relevant time points, as detailed below. The principle and output of the two approaches are the same, to provide a comparison of the outcome at a specific time point between groups, so this does not reflect a fundamental change in the primary end-point analysis.

Primary analysis

The primary analysis compared the proportion of pupils with any presence of obvious decay experience measured by D_ICDAS4–6MFT at 2.5 years in any permanent tooth, between the intervention and control groups using a binary logistic mixed-effect model. The model adjusted for school year (Year 7/S1 or Year 8/S2) and number of D_ICDAS4–6MFT (excluding primary teeth) at baseline as fixed effects and school as a random effect. The adjusted odds ratio (OR) from the model was estimated with a 95% CI and p-value. The predicted probabilities in each group and the adjusted risk difference and 95% CI are also presented, so these can be compared with the probabilities assumed in the sample size calculation. This analysis was based on the available case population, including participants who completed both baseline and 2.5-year dental assessments, and had at least one permanent tooth (as opposed to all primary teeth, which was very unlikely). Analysis of the primary outcome was checked and verified by a second statistician.

Sensitivity analyses

Subgroup analyses

Analysis of secondary outcomes

Exploratory analysis of pilot trial schools only

Since some of the follow-ups only took place in schools recruited during the internal pilot phase (CBS, 12 weeks, 2 years) or mainly in these schools (at 1 year when few main trial schools responded), analyses, as described above, were replicated but using repeated measures models incorporating all post-randomisation time points and restricted to pupils in pilot schools only. Mixed-effect models were used including allocation, time point, allocation by time point interaction and other baseline covariates as described in the previous sections. School and participant were included as random effects to account for the clustering by school and repeated measures per participant, respectively. Estimates of effect at each time point are provided with a 95% CI and p-value.

Reproducibility

Cohen’s kappa coefficient and percentage agreement were used to measure the intrarater (when the same dentist assessed the child) and inter-rater (when the child was assessed by two different dentists) reliability for the presence of carious lesions for the pupils who were re-examined at each dental assessment time point.

Adverse events, suspected pathology and child safeguarding

The number and type of adverse events, suspected serious pathology and safeguarding issues are summarised by trial arm.

Overview

A cost–utility analysis was conducted in line with current recommendations from NICE. 102 In particular, an NHS and Personal Social Services perspective was adopted for costs, and health benefits were quantified using QALYs. The value of undertaking longer-term cost-effectiveness modelling was assessed, with an appropriate model structure having been identified through a literature review and consultation with experts.

Quality-adjusted life-years were estimated via the trapezium rule using the CHU9D that was reported at baseline and annually thereafter. The CHU9D was valued using the UK tariff values. 94 NHS resource use was measured for each participant at baseline and annually up to 2.5 years. This included all medication costs (e.g. antibiotics) and visits to dental practices for treatment and health services (e.g. dental admission for a general anaesthetic) using the parent/carer resource use questionnaire.

The detailed methods were set out in a health economics analysis plan (HEAP). 59 The HEAP was developed in advance of the health economics analysis being undertaken. The main components of the HEAP relating to the measurement of resource use, unit costs and analysis are summarised below. The items of resource associated with the intervention, together with their source(s) used for costing, are shown in Table 2.

There were multiple sources for data relating to dental treatments/visits, and after reviewing the quality of the data sources, it was decided that the trial dental assessment data should be used to estimate resource use in the primary analysis. Sensitivity analyses were to be undertaken using the parent/carer questionnaire data if imputation was deemed appropriate. A subgroup analysis was planned for Scotland using administrative data from Public Health Scotland to take advantage of the availability of those data and recognise the potential impacts of a different dental fee schedule. As the administrative data were not obtained in time for this report, this subgroup analysis will be reported separately.

In order to generate costs from the dental assessment data for the primary analysis, two steps were required. First, the translation of clinical findings to associated procedures (e.g. a missing permanent tooth implied an extraction has taken place); this process is reasonably uncontentious. Second, the translation of procedures into numbers of visits generated a lot more uncertainty. However, after discussion with dentists who have worked in general practice, it was felt that a simple approach could be adopted. By recognising that children do not normally have more than two extractions or four restorations at any visit, we calculated the number of visits as such:

• In the case of a child having had x extractions, the number of visits for treatment was x/2, rounded up to the next whole number.

• In the case of a child having had y restorations, the number of visits for treatment was y/4, rounded up to the next whole number.

An alternative approach that used parent/carer questionnaire data was undertaken in a sensitivity analysis.

Unit costs

The unit costs to be used are shown in Table 3 and are at 2020–1 price levels, which represents the most recent year for which earlier costs can be adjusted to using the NHS Cost Inflation Index (NHSCII). Costs derived from previous years were inflated to this level using the NHSCII taken from Unit Costs and Health and Social Care. The individual unit costs for different items of treatment in Scotland were obtained from weighted costs by activities presented in the Statement of Dental Remuneration (SDR) items of service claims for 2019–20 uplifted to 2020–1 prices above. Patient charges used in the sensitivity analyses (Table 4) were obtained from SDR Amendment No. 155 at 2020–1 prices. The full list of treatments for Scotland is presented in Appendix 5, Table 23.

The derivation of English Unit of Dental Activity (UDA) costs used NHS data pertaining to payments to dental practices in England 2018–9. While more recent data were available, the impact of the COVID-19 pandemic was such that dental activity for 2019–20 and 2020–1 were not thought to be appropriate. We started by identifying data on 8581 practices, for which UDAs and Net Payment to Dental Contract were available (www.nhsbsa.nhs.uk/dental-data/nhs-payments-dentists). Practices that also had non-zero units of orthodontic activity were excluded, to produce 6899 practices. The Net Payment includes several adjustments relating to performance and other activities, and so we have excluded these. Cost per UDA, net of patient charges were calculated for the NHS perspective, and a cost including patient charges for the societal perspective was also calculated. This produced costs of £23.63 per UDA and £44.65 per UDA, respectively. These were then uplifted 5.36% to 2020–1 price levels using the NHSCII for pay and prices.

Analytical approach

Both a within-trial analysis and a longer-term model-based analysis were planned. However, if there were lasting clinical effects, it was deemed that the trial data would be sufficient to assess the cost and health impacts of the intervention. This assessment was based on the presence of two clinical outcomes at 2.5 years that are associated with long-term patient benefits: the primary outcome measure (D_ICDAS4–6MFT) and the secondary outcome of frequency of self-reported toothbrushing. However, so that we were not too restrictive on the assessment of an observed effect being due to random variation, we used a p-value of 0.1 for our assessment, rather than 0.05.

The within-trial analysis followed published recommendations103 and was undertaken within STATA v17. All costs and benefits were discounted at the rate of 3.5% in line with the NICE reference case. Incremental costs and QALYs were estimated using multilevel modelling to take into account the hierarchical structure of the data. Models for costs and QALYs were estimated separately using maximum likelihood estimation. For costs, bias-corrected and accelerated bootstrapping with 1000 replications was used. 104 For QALYs, normal-based 95% CIs were estimated via regression models with a dummy variable describing the randomised group, a random effect associated with schools adjusting for relevant baseline characteristics. We ran logistic regressions to assess whether any baseline variables (allocation, age, sex, FSM status, number of D_ICDAS4–6MFT and cariogenic score) were statistically significant predictors of missingness of costs and QALYs. Missing data were imputed using multiple imputation, following published recommendations for economic evaluations. 105

Sensitivity analysis

Deterministic sensitivity analyses were carried out to test the impact of data, assumptions and analysis methods on results. The following deterministic sensitivity analyses were undertaken:

• A societal perspective was taken with respect to costs that included costs of intervention delivery to schools, private expenditure and production losses related to dental treatment/problems. Time taken away from school was also measured but was not valued.

• Utilities generated by the Caries Impacts and Experiences Questionnaire for Children – Utility Measure (CARIES-QC-U), which is a condition-specific utility measure generated from the CARIES-QC. 96

• Alternative time horizons of 5 and 20 years (for model-based analyses that include a long-term effect).

• Use of parent/carer questionnaire responses for an ITT analysis (with or without imputation, as appropriate).

Probabilistic sensitivity analysis was planned for both the within-trial analysis and the model-based analysis.

Subgroup analysis

Two sets of subgroup analyses were undertaken. The first set provides consistency with the SAP; subgroup analyses were undertaken in relation to eligibility for FSM at baseline and pilot versus main trial schools. The second provides information based on consistent costs of dental treatment, by splitting the full sample into Scotland and ‘Other’. Analyses were undertaken by separately estimating cost effectiveness in each of the subgroups (e.g. Scottish schools vs. all other schools) using multilevel modelling as above. The subgroup analysis based on eligibility of FSM was also undertaken at the school level, with two groups of schools being defined by having a proportion of children eligible for FSM either above or below the median value across the 41 schools (which was 22%).

Incremental analysis

Incremental analysis was undertaken using the results of the within-trial regression analyses and/or the model-based probabilistic sensitivity analysis. Results were plotted on the cost-effectiveness planes with associated cost-effectiveness acceptability curves.

Intervention implementation (objective 1)

For the BRIGHT trial, fidelity was operationalised as the degree to which the two components of the intervention (CBS and follow-up text messages) were carried out according to the protocol. For the CBS, it included aspects such as date of delivery, duration and timing, any adaptations of the lesson to fit with PSHE education and school curriculum and whether the CBS was structured as one session or broken down over more than one session. For text messages, it involved assessing whether twice-daily text messages were delivered to pupils as intended.

The component dose includes ‘dose delivered’ and ‘dose received’. ‘Dose delivered’ can be defined as the number or amount of intended units of each intervention or each component delivered or provided, whereas ‘dose received’ refers to the extent to which participants actively engage with, interact with, are receptive to and/or use materials or recommended resources. 107 In the BRIGHT trial, the dose delivered was defined as the number of CBS and text messages sent. Dose received was defined as the number of pupils who attended the CBS and the number of text messages received. Reach was assessed in terms of the extent to which the intervention was reaching the target population according to the eligibility criteria, particularly in terms of FSM status. Data collection on intervention implementation involved asking schools to confirm they had delivered the CBS and to whom by providing a delivery date and pupil attendance details.

Text message dosage (the number of texts sent to participants) was collected from recording information via the TextApp software. Start date of text messages, all messages sent and any replies received via the TextApp software were logged and audited in the underlying database with date and time stamps. Similarly, delivery receipts were recorded with date and time when the phone network provider acknowledged successful delivery of the message. Unfortunately, there was no facility to confirm if messages were read. If the network did not receive a successful delivery receipt within 24 hours, the message was considered to be undelivered and would not be resent. Undelivered messages could occur if the mobile phone was switched off, out of signal or the number was no longer in use. These function logs were used to determine the following metrics:

• number of sent text messages per participant

• number of text messages undelivered per participant

• the number of pupils texting back STOP and when this occurred

• total number of replies to text messages

• the number of replies per participant/per message sent

• timings between message delivered and reply

• number of participants who reported a change of telephone number.

Intervention acceptability (objective 2)

Intervention acceptability was explored through interviews with pupils, school staff and key stakeholders involved in health and education policy.

Recruitment and consent

The sample for the qualitative component was drawn from the three BRIGHT trial sites: England, Scotland and Wales, and included six schools.

Data collection

Data analysis

The framework approach,110 a matrix-based method for the analysis of qualitative data, was employed for data analysis. This involved the following stages: familiarisation, identifying initial themes, labelling the data, sorting the data by theme and synthesising the data. The data were primarily analysed by two experienced doctoral researchers. This involved reading and re-reading the transcripts, independently identifying initial themes and independently and systematically coding transcripts line by line using an a priori thematic coding framework. This framework was developed from several sources: familiarisation with the interview transcripts; the theoretical framework of acceptability;109 and research team discussion. A pragmatic approach was adopted that drew on both deductive and inductive processes, enabling the exploration of a priori themes identified from the literature search and allowing new themes to be identified. Any discrepancies were resolved through discussion. In addition, any relevant field notes taken were used to help interpret the data.

Qualitative data were handled through the software NVivo Version 12 QSR International, providing data management and retrieval facilities to support analysis and write-up. The NVivo retrieval facilities allowed the researchers to remain connected to the original raw data facilitating the verification of conclusions throughout the refinement stages. Further refinement was undertaken by one of the postdoctoral researchers and any discrepancies were discussed and resolved through discussion. This process strengthened inter-rater reliability and credibility and thus ensured the trustworthiness of the data analysis.

Sample size

The sample size was revised prior to the main phase of the trial, following the progression criteria review point. This is because we planned for the worst-case scenario in the original sample size calculation by assuming the less efficient design of randomisation at the school level (between-schools), rather than at the year-group level (within-schools). Assuming between-school randomisation, an ICC of 0.02, an average of 60 participating pupils per year group (120 per school), and allowing for 20% attrition, we originally estimated that 48 schools would be required in total (5760 pupils). Within-school randomisation was piloted in the internal pilot with 10 schools in order to address the following relevant progression criteria:

• assessment of contamination in the control group and whether feasible to undertake randomisation within schools (by year group) or whether randomisation at the school level will be required, and calculation therefore of the required school sample size.

Following the progression criteria review point, within-school randomisation (by year group) was considered feasible and the sample size was revised to assume within-school randomisation and partial contamination effects (i.e. those contaminated gain half the intervention benefits) for 27% of the control sample (based on findings from the internal pilot, see Appendix 6 for more detail). The other assumptions for the sample size remained the same (i.e. an average of 60 pupils per year group, an ICC of 0.02 and 20% attrition at follow-up). With these assumptions, we calculated that 42 schools were required in total across the internal pilot and main phase of the trial. This would provide 90% power (5% two-sided α) to detect an 8% absolute reduction, from 34% to 26%, in the proportion of pupils with ‘obvious decay experience’ (the primary outcome).

While the revised sample size still assumed an average of 60 pupils per year group, the average number of pupils randomised per year group in the internal pilot was lower than this (49 pupils). Within the internal pilot, we were able to calculate an estimate of the participation rate for the number of pupils approached: an average of 121 pupils per year group were invited to partake in the trial, and 49 (40%) were randomised. Based on this participation rate of 40% and considering the size of the schools that had expressed an interest in taking part in the main phase, we were satisfied that we could achieve an average of 60 recruited pupils per year group in the main phase of the trial by approaching a larger pool of pupils in each year group (i.e. by inviting, on average, at least 150 pupils per year group).

Parent/carer consent for questionnaires

For the internal pilot, following the return of pupil consent forms, schools were asked to provide parents/carers of children who had consented with an additional information sheet and consent form regarding the completion of parent/carer questionnaires. 59 Parents/carers were requested to complete and return the consent form to the YTU trial team, using the provided freepost envelope, if they were happy to complete parent/carer questionnaires and to share their name and address so questionnaires could be posted directly to them. However, an amendment was made prior to the main phase of the trial to no longer complete this process. This was because the return rate of parent/carer consent forms for questionnaire completion was low and the process of distributing information sheets and consent forms was a burden on schools. Instead, for the main phase of the trial, and for all subsequent time points for the internal pilot, schools distributed parent/carer questionnaires via pupils (rather than the trial team posting these to parents/carers). 59 Parents/carers were provided with a cover letter explaining what the data would be used for and a freepost envelope to return the questionnaire if they were happy to complete it.

Final follow-up time point

The original planned final follow-up was at 3 years following CBS delivery, but it was amended to 2.5 years with funder approval. This was to avoid the final clinical examinations coinciding with school exams during the spring/summer when school staff and space are at a premium.

Changes to the wording of the primary and secondary outcomes

The primary outcome was initially described as the incidence of carious lesions in permanent teeth, measured using DMFT where decay is measured as caries into dentine (ICDAS 4–6). The word ‘incidence’ was felt, by independent members of the TSC and DMEC, to be ambiguous in its meaning, so language was clarified throughout the protocol to make the primary outcome and relevant secondary outcomes clear, using the words ‘prevalence’ and ‘presence’ instead. This also served to make clear that the outcomes were looking at the prevalence of caries at each time point, rather than the development of new caries from baseline, which was how some members of the TSC and DMEC thought ‘incidence’ could be interpreted. More detailed definitions of DMFT were also added to the protocol in relation to the primary and secondary outcomes.

Previously, the incidence of enamel carious lesions in permanent teeth, measured using D_ICDAS1–3MFT, was included within the protocol as a secondary outcome. This secondary outcome was incorrectly specified and was subsequently corrected to clarify that caries prevalence for all carious lesions will be reported, defined as the presence of at least one treated or untreated carious lesion of any severity (D_ICDAS1–6MFT, as opposed to just D_ICDAS1–3MFT) in any permanent tooth at 2 (internal pilot only) and 2.5 years.

The number of permanent teeth with any treated or untreated carious lesions (using the D_ICDAS1–6MFT) and caries into dentine (D_ICDAS4–6MFT) at 2 (internal pilot only) and 2.5 years was added as a secondary outcome. Although previously referred to in the data analysis section of the protocol (and listed as a secondary outcome in the grant proposal), this secondary outcome was omitted in error in the lists of secondary outcomes.

Finally, oral health behaviours, toothbrush/paste availability, contamination and intervention compliance were previously listed as secondary outcomes, but were reclassified as ‘other collected measures’ instead, as they were collected for the purposes of describing the sample, confounding factors and compliance rather than strictly as secondary outcomes.

The impact of the COVID-19 pandemic on data collection

Final follow-up data collection for the internal pilot and main phase of the trial (planned for collection at approximately 2.5 years post CBS) was delayed as a consequence of COVID-19-related suspension of data collection activities during the pandemic. It actually ended up being an average of about 3 years, as originally planned, due to these unforeseen delays. The protocol was amended to include the following measures to mitigate any potential risk during the dental examinations and questionnaire completion:

• All participating institutions conducted necessary risk assessments.

• Pupils and their parents/carers were sent an updated information leaflet to remind them of upcoming data collection, and that they could withdraw from the dental assessments and/or questionnaire completion if they so wished.

• The LRTs asked schools to identify any pupils who would be classed as clinically extremely vulnerable (high risk) or clinically vulnerable (moderate risk), so they would not be invited to participate in the dental assessments (however, they were asked via schools to complete the pupil questionnaires).

• All school visits followed current government, research and school guidance on measures to reduce risk (e.g. social distancing, use of appropriate personal protective equipment and use of COVID-19 screening questionnaire, as required).

Where it was infeasible to conduct dental examinations at participating schools due to the restrictions of the COVID-19 pandemic (e.g. dental teams not allowed into the school), schools were asked to just distribute the pupil and parent/carer questionnaires. Where this happened, but the schools later allowed dental teams into schools for the dental assessments, non-responders were asked to complete their pupil questionnaire during the dental assessments.

School recruitment

Approximate numbers of schools assessed for eligibility in each region, via use of publicly available data, are provided in Figure 2 [i.e. all state-funded schools in Scotland, England (South and West Yorkshire) and Wales (Cardiff local authority, Vale of Glamorgan local authority, Rhondda Cynon Taf local authority and Merthyr Tydfil local authority)].

Figure 2 also presents the approximate number of schools excluded due to having a lower proportion of pupils eligible for FSM than the average for that devolved nation. Due to the availability of school-level data and the expanse of the regions eligible to participate, it was not feasible to calculate the number of schools excluded due to not meeting other inclusion criteria (e.g. having at least 60 pupils per year group) or due to meeting the exclusion criteria. LRTs checked school eligibility where they could before directly approaching individual schools and checked the eligibility of each school that expressed interest in participating regions against the school inclusion and exclusion criteria. Due to the recruitment methods used in some regions, which reached an unknown number of schools (e.g. advertising through local authority networks and through local or national organisations for schools, such as School Leaders Scotland), it also was not possible to calculate the number of schools approached across all regions.

In total, 47 schools were recruited into the trial (i.e. a school representative signed an Agreement to Participate Form to confirm their involvement). For the internal pilot, 11 schools were recruited (Scotland, n = 3; England, n = 4; Wales, n = 4); however, two of the schools recruited in Wales were due to merge in the 2018–9 academic year (1 year post CBS delivery; two single sex schools merged into two mixed sex schools under one executive head) and therefore were considered as one school for the purposes of the trial (i.e. they were randomised as one school) and hereafter in this report. For the main phase, 36 schools were recruited (Scotland, n = 9; England, n = 19; Wales, n = 8). As shown in Figure 2, four schools withdrew pre randomisation (one school in England withdrew after signing the Agreement to Participate Form, but before starting participant recruitment; three schools in Wales withdrew during participant recruitment). In total, and in line with the sample size calculation, 42 schools were randomised; 10 were randomised during the internal pilot phase (Scotland, n = 3; England, n = 4; Wales, n = 3) and 32 during the main phase of the trial (Scotland, n = 9; England, n = 18; Wales, n = 5).

Participant recruitment

As shown in Figure 2, pupil recruitment was completed in 42 schools. Of the 14,083 pupils approached, 4699 (33.4%) consented and were eligible (i.e. they provided a valid mobile phone number) and were asked to complete baseline data collection. However, 19 participants withdrew pre randomisation (e.g. due to changing their mind) and so a total of 4680 pupils were included in the randomised sample (intervention, n = 2262; control, n = 2418). This was 92.9% of our target of 5040. The average number of pupils recruited per school was 111.4 (SD 35.9, median 107, range 46–189) and per year group was 55.7 (SD 21.6, median 53, range 13–119).

Randomisation for the BRIGHT trial occurred shortly after baseline data collection at the year-group level (i.e. at each school, 1 year group was randomised to the intervention group and the other to the control group), rather than at the pupil level. All consenting and eligible pupils (who provided a valid mobile number; excluding those that withdrew before randomisation) were considered as part of the randomised sample of pupils, regardless of whether baseline data were collected for them. A small number of randomised pupils did not complete the baseline dental assessment and did not provide any pupil questionnaire data (n = 47, 1.0%). In addition, seven participants completed the dental assessment but not the pupil questionnaire (0.15%) and eight completed the pupil questionnaire but not the dental assessment (0.17%).

Follow-up

At the 2.5-year follow-up, 2400 randomised pupils (51.3%; intervention 1160, 51.3%; control 1240, 51.3%) had a valid dental assessment. This was completed an average of 35.9 months (SD 3.2, range 26.7–45.9) after the baseline dental assessment. Reasons for missing data are (percentages out of 2280 missing): dental assessment not conducted in the school [n = 1060 pupils (46.5%) from seven schools], absent from school on day of data collection (n = 408, 17.9%); declined to complete questionnaire (n = 273, 12.0%); withdrew from trial prior to data collection time point (n = 247, 10.8%), no longer at the school (n = 195, 8.6%); and unknown (n = 97, 4.3%).

Year group as a random effect

Year group was included in the primary analysis model as a random effect nested within school, which produced virtually identical results to the primary analysis (OR 1.04, 95% CI 0.85 to 1.26, p = 0.72).

Missing data

Several statistically significant associations between baseline variables and missingness (in terms of inclusion in the primary analysis model) were observed. The following were all associated with an increased likelihood of having missing data:

• females [missing 1331/2537 (52.5%) vs. males 966/2143 (45.1%); OR 1.47, 95% CI 1.24 to 1.73, p < 0.001]

• pupils eligible for FSM [missing 565/1025 (55.1%) vs. non-FSM 1659/3483 (47.6%); OR 1.59, 1.32 to 1.91, p < 0.001]

• pupils in Year 8 at recruitment/Year 11 during the dental assessment [missing 1054/2057 (51.2%) vs. Year 7 1243/2623 (47.4%); OR 1.70, 95% CI 1.26 to 2.30, p < 0.001]

• more D_ICDAS4–6MFT at baseline (OR associated with a one unit increase in score 1.13, 95% CI 1.07 to 1.20, p < 0.001)

• higher cariogenic score (OR associated with a one unit increase in score 1.006, 95% CI 1.002 to 1.011, p = 0.01)

• higher plaque index score (OR associated with a one unit increase in score 1.31, 95% CI 1.10 to 1.55, p = 0.002).

When these variables were included in the primary analysis model (school year and number of D_ICDAS4–6MFT at baseline are already covariates in the model), the OR for the effect of the intervention was 1.01 (95% CI 0.83 to 1.24, p = 0.91), which is similar to the primary result, providing additional evidence that the data were missing at random.

Timing of follow-up

The average time interval between the baseline and 2.5-year dental assessment was 36 months. Among the pupils with a valid dental assessment at both time points whose final assessment was completed between 33 and 39 months after their baseline, the treatment effect was very similar to the primary analysis (OR 1.02, 95% CI 0.78 to 1.33, p = 0.87).

Classroom-based session attendance

In total, 39 out of the 42 participating schools confirmed that they had held the CBS. One school confirmed that they did not deliver the CBS and no response was received from the remaining two; therefore, we assumed these schools did not deliver the session. All but one school delivered the CBS in a single session; one school decided to deliver the CBS over three lessons, rather than one, over a week for timetabling reasons. Among the 39 schools, the only/initial CBS took place a median of 26 days (range 7–87) after the letter was sent to the school informing them of the allocations for their year groups.

Registers of attendance at the CBS were received from 30 schools. For the remaining schools that confirmed they held a CBS, we assumed that all randomised pupils in the intervention year group attended the session. For the school that delivered three CBS lessons, we only classed the pupil as attending if they attended all three sessions. Therefore, in total, 2016 (89.1%) of randomised pupils in the intervention group attended the CBS. However, this is likely to be a slight overestimation as not all pupils in the intervention year groups for which a register was not received would have attended the session. Indeed, we know that two out of the eight classes in the intervention year group for one school did not receive the CBS. We failed to obtain the CBS registers from that school despite multiple reminders.

One school started to deliver the session to the wrong year group in error (i.e. delivered to the year group allocated to control). This was due to a communication error at the school. This led to some partial contamination in the control group, but it is not clear how much of the CBS was delivered (as this was the school that delivered the CBS over three sessions eventually to the correct year group) nor to whom, as registers were not requested for the control group. Therefore, we conservatively assumed that all control pupils in this school received the full CBS (n = 69).

Text messages

Text messages were triggered to commence for pupils in the intervention year group of a school as soon as the school provided confirmation that they had delivered the CBS. For schools that did not confirm they had delivered the CBS, a date was chosen for participants to start receiving the text messages. Text messages were sent to 2258 (99.8%) of the 2262 pupils randomised to the intervention group. The other four withdrew shortly after the date of the CBS at their schools and so their messages were not commenced. The first message was sent a median of 5 days (range −19 to 168) after the CBS (or imputed) date at the school. Participants were sent text messages until they requested them to stop or until 12 July 2020. The text messages should have been available for participants to continue to receive or restart after July 2020. However, on 12 July 2020, HIC moved to a new text messenger provider platform and in December 2020, they informed the BRIGHT trial team that < 2% of the text messages from 12 July 2020 had been delivered successfully. At that stage, 1368 (60.5%) participants had not withdrawn or texted STOP and would have been eligible to receive the messages. A decision was made to cease text messages. For these summaries, we have assumed that no messages were sent after 12 July 2020.

A total of 962 intervention participants (42.5%) withdrew from receiving the text messages, a median of 2.8 months after they commenced (range 1 day to 30 months). Time to intervention withdrawal is depicted in the Kaplan–Meier curve below (Figure 3). Participants were sent messages for between 0 and 127 weeks (approximately 30 months, mean 53.4 weeks, SD 35.4, median 62). This equated to between 1 and 1708 text messages (mean 694.5, SD 468.9, median 789). On average, 71.4% of the text messages sent to a participant were successfully delivered.

FIGURE 3.

Kaplan–Meier survivor curve of time to intervention withdrawal.

Complier-average causal effect analysis

The CACE estimate of the treatment effect based on attending the CBS session was similar to the ITT estimate (OR 1.05, 95% CI 0.85 to 1.31, p = 0.64). The CACE estimate for attending the CBS session and having at least 50% of their messages delivered per week for the first 12 weeks was also similar (OR 1.07, 95% CI 0.72 to 1.59, p = 0.74). For the continuous compliance variable of number of text messages sent, the CACE estimate was OR 1.00 (95% CI 0.999 to 1.001, p = 0.93), which indicates that for every additional text message, there was no evidence of a decrease in likelihood of having a carious lesion.

Eligible for free school meals

Of the pupils included in the primary analysis, 460 (19.3%) were eligible for FSM, 1824 (76.5%) were not, and data were missing for 99 (4.2%). Among those who were and were not eligible for FSM, the proportion who reported brushing their teeth at least twice a day at baseline was 71.6% and 80.6%, respectively, and 73.6% and 81.5% at 2.5 years. At 2.5 years, the proportion of FSM pupils with at least one D_ICDAS4–6MFT was 54.8% and 41.3% among non-FSM pupils.

There was a statistically significant interaction between randomised group and FSM status (p = 0.04) for the primary outcome, providing evidence that the treatment effect was different between pupils who were and were not eligible for FSM; this was a qualitative interaction. The OR for the intervention among pupils with FSM status was 0.69 (95% CI 0.44 to 1.08, p = 0.10; predicted proportions 46.8% and 53.7% in intervention and control groups, respectively), demonstrating a benefit, whereas there was a negative effect among non-FSM pupils (OR 1.17, 95% CI 0.93 to 1.46, p = 0.18; predicted proportions 44.0% and 41.4% in intervention and control groups, respectively).

Baseline caries

There was no evidence of an interaction between randomised group and total number of D_ICDAS4–6MFT in permanent teeth at baseline for the primary outcome (interaction effect p = 0.87).

Pilot or main trial school

There was no evidence of an interaction between randomised group and whether the pupil attended a school that was recruited in the pilot or main trial phases of the trial for the primary outcome (interaction effect p = 0.81).

Caries prevalence for all carious lesions (D_ICDAS1–6MFT) at 2.5 years

Of the 2400 randomised pupils with a valid dental assessment at the final follow-up, 1474 (61.4%; intervention 723, 62.3%; control 751, 60.6%) had at least one treated or untreated carious lesion in any permanent tooth as indicated by D_ICDAS1–6MFT.

Of the 2383 randomised pupils with a valid dental assessment at both time points, 1463 (61.4%; intervention 717, 62.2%; control 746, 60.7%) had at least one D_ICDAS1–6MFT at the final follow-up (Table 10). There was no evidence of a difference in this outcome between the intervention and control groups (OR 1.05, 95% CI 0.86 to 1.28, p = 0.64). The predicted probabilities from the model were 61.3% (95% CI 57.1 to 65.5) in the intervention group and 60.4% (95% CI 56.3 to 64.6) in the control group (adjusted risk difference 0.8, 95% CI −2.6 to 4.3).

For those pupils who had at least one D_ICDAS1–6MFT recorded at the final dental assessment, the mean number of D_ICDAS4–6MFT was 1.9 (SD 2.1, median 1) and the mean number of D_ICDAS1–6MFT was 3.9 (SD 3.2, median 3).

Number of carious teeth at 2.5 years

At the 2.5-year follow-up, the mean number of D_ICDAS4–6MFT per pupil was 1.08 (SD 1.72, median 0) in the intervention group and 1.20 (SD 2.07, median 0) in the control group (Table 10). There was no evidence of a difference in this outcome between the groups (IRR 0.96, 95% CI 0.85 to 1.07, p = 0.45), nor in the number of D_ICDAS1–6MFT (intervention mean 2.37, SD 3.02, median 1; control mean 2.47, SD 3.27, median 1; IRR 0.98, 95% CI 0.89 to 1.08, p = 0.65).

Frequency of self-reported toothbrushing

At baseline, 4616 (98.6%; intervention 2230, 98.6%; control 2386, 98.7%) provided a response to the question asking how frequently they brushed their teeth. Of these, 3631 (78.7%; intervention 1757, 78.8%; control 1874, 78.5%) responded that they brushed their teeth at least twice a day.

At 6 months, 3177 (67.9%; intervention 1480, 65.4%; control 1697, 70.2%) provided a response to the question asking how frequently they brushed their teeth. Of these, 2696 (84.9%; intervention 1287, 87.0%; control 1409, 83.0%) responded that they brushed their teeth at least twice a day.

Of the 3154 randomised pupils with a valid response at both time points, 2674 (84.8%; intervention 1275, 86.9%; control 1399, 83.0%) responded that they brushed their teeth at least twice a day. There was evidence that pupils in the intervention group were more likely to report brushing their teeth at least twice a day compared to the control group, at the 6-month time point (OR 1.30, 95% CI 1.03 to 1.63, p = 0.03).

At the 2.5-year follow-up, 2638 (56.4%; intervention 1285, 56.8%; control 1353, 56.0%) provided a response to the question asking how frequently they brushed their teeth. Of these, 2121 (80.4%; intervention 1039, 80.9%; control 1082, 80.0%) responded that they brushed their teeth at least twice a day.

Of the 2616 randomised pupils with a valid response at both time points, 2105 (80.5%; intervention 1033, 81.0%; control 1072, 79.9%) responded that they brushed their teeth at least twice a day. There was no evidence that the intervention group were more likely to report brushing their teeth at least twice a day compared to the control group, at the final follow-up (OR 1.05, 95% CI 0.84 to 1.30, p = 0.69).

Dental plaque and gingival bleeding scores, and Caries Impacts and Experiences Questionnaire for Children at 2.5 years

At the 2.5-year follow-up, the mean plaque score was 0.90 (SD 0.69, n = 1159) in the intervention group and 0.87 (SD 0.70, n = 1240) in the control group (Table 10). There was no evidence of a difference between the two groups in plaque score (adjusted mean difference −0.02, 95% CI −0.07 to 0.02, p = 0.31).

At the 2.5-year follow-up, the mean gingival bleeding score was 0.13 (SD 0.18, n = 1157) in the intervention group and 0.14 (SD 0.20, n = 1239) in the control group (Table 10). Visual inspection of the Q–Q plot of the residuals from the initial model demonstrated deviations from normality. Therefore, the outcome was log transformed and the model rerun, which resulted in a small improvement in the normality of the residuals. In this model, there was borderline evidence of a difference in gingival bleeding score between the two groups (0.92, 95% CI 0.85 to 1.00, p = 0.05). These estimates are interpreted such that the intervention group is predicted to have a mean gingival bleeding score 0.92 times that of the control group.

At the 2.5-year follow-up, the mean CARIES-QC score was 2.79 (SD 2.97, n = 1279) in the intervention group and 2.95 (SD 3.22, n = 1347) in the control group. The residuals from the untransformed model showed evidence of deviation from normality; therefore, the outcome was log transformed. Responses to the global CARIES-QC item at 2.5 years are reported in Appendix 8, Table 41 and are very similar between the two groups. Two-thirds of pupils reported that their teeth were ‘not at all’ a problem (intervention 67.5%, control 68.4%), just under a third that their teeth were ‘a bit’ of a problem (intervention 29.9%, control 29.3%), and only a small number reported that their teeth were ‘a lot’ of a problem (intervention 2.6%, control 2.2%).

There was no evidence of a difference between the two groups in CARIES-QC score (adjusted mean difference 1.00, 95% CI 0.94 to 1.06, p = 0.89). As above, the coefficients from the model were exponentiated such that the interpretation is that the intervention group has a predicted mean CARIES-QC score 1 time that of the control group.

Child health utility 9D

At baseline, the mean CHU9D scores were 0.91 (SD 0.09, n = 2221) and 0.91 (SD 0.09, n = 2366) in the intervention and control groups, respectively. At the 2.5-year follow-up, the mean CHU9D score was 0.89 (SD 0.10) in both the intervention group (n = 1277) and in the control group (n = 1341).

Affective attitude and perceived effectiveness

The dimensions of affective attitude and perceived effectiveness have been presented jointly due to some overlap in the themes presented. Overall, both staff and pupils found the lesson acceptable. Staff reported that it was appropriate and had gone well.

It was good. It was very thorough. I would say that it was a success … Yes, for year sevens and eights, I would say it was appropriate definitely.

School staff: 33:1

Pupils described the lesson as informative and reported being more interested in oral health as a result, particularly regarding the consequences of poor oral hygiene and the importance of brushing their teeth.

I didn’t know you had to brush in two minutes so I used to do it a minute but now I do it for two minutes.

Pupil: 37:Y7:PS7

Like what would happen if I didn’t brush my teeth every day and I didn’t used to be like interested in that before.

Pupil: 75:Y8:PS8

Pupils provided mixed responses regarding the delivery of the lesson. Some reported the lesson ‘was covered well’ (Pupil: 37:Y7:PS10) and they found it interesting and fun. Others, however, found the lesson a bit long, and had become bored and disengaged by the end.

They’ve done it in a way that it was interesting like fun. But then, like it wouldn’t be too boring, but like you’re learning about something that you don’t know you want to learn about. But then when you’re doing it, you actually enjoy doing it.

Pupil: 16:S1:PS3

At the end it got very boring … because it’d been like really long.

Pupil: 57:Y7:PS6

In terms of engagement, the content of the lesson was deemed appropriate by some staff participants and consequently pupils ‘were very engaged with it’ (School staff: 33:1), while others found that for their pupil cohort, it was not engaging enough.

… it’ll be different levels of abilities in different areas … I mean, the area that we’re based in is more of a deprived area. So … it was probably better to be at that level for the students.

School staff: 33:1

… well we have boys … that play on quite high-level computer games and things like that. It wasn’t gripping enough for them.

School staff: 39:1

The setting of the delivery also appeared to impact engagement. Pupils who attended the lesson as part of a whole year assembly reported not having the opportunity to ask questions.

I didn’t really get the chance to ask why we should brush our teeth twice a day so yeah.

Pupil: 75:Y8:PS6

And I didn’t feel like I could ask questions because there was so many people.

Pupil: 75:Y8:PS7

As mentioned previously, this was an adaptation to the intervention and appears to have had a negative impact on pupil engagement.

Generally, staff found the materials and lesson activities to be suitable and appropriate. There were mixed views, however, regarding their fit with the duration of the lesson. Some felt the lesson plan was suitable, whereas others felt there was either too much or insufficient content for the time allocated.

… our lessons are 50 minutes long. Now that was a stretch to keep that going for 50 minutes … so there wasn’t enough content.

School staff: 39:1

I think the resources that were given were enough … I think with the activities that the students had to do with the time that you spent discussing … it was a nice mixture … it lasted in total about an hour. So it was a good length, I think, to engage the students.

School staff: 33:1

One element of the lesson plan (an educational animated video on toothbrushing) was described by some staff participants as ‘babyish for the year group that it was targeted at’ (School staff: 75:1). Similarly, several pupils reported finding it too childish. Some pupils felt they already knew how to brush and would have preferred if the video elaborated more and explained why toothbrushing is important.

Like it was like telling me like how to brush your teeth and stuff like that. It was like…it just felt like it wasn’t meant for our age.

Pupil: 33:Y8:PS7

The photos that were provided as part of the lesson to illustrate the consequences of inadequate toothbrushing were received positively by both pupils and staff.

Moderator: What were your favourite parts?

I enjoyed it … yeah and when they showed us the pictures of really disgusting teeth.

Pupil: 75:Y8:PS6

And they talked about facts and stuff that I found interesting.

Pupil: 75:Y8:PS11

I liked the pictures that you sent of the before and after sort of pictures that you sent, and they liked those as well.

School staff: 39:1

Pupils expressed wanting more oral information. As part of the lesson, they wanted to know more about:

Why brush your teeth twice a day.

Pupil: 75:Y8:PS6

How different toothpastes like affect your teeth and how they help them.

Pupil: 75:Y8:PS7

And what mouthwash to use as well.

Pupil: 75:Y8:PS8

They also suggested the delivery of several lessons over the pupil’s time at secondary school, rather than a one-off lesson.

… more lessons would be really helpful to tell us more about like teeth.

Pupil: 75:Y8:PS5

To improve the lesson, more visual resources such as photos and videos were recommended, particularly on the consequences of inadequate toothbrushing, with pupils regarding graphic pictures as potentially influential in behaviour change.

And like show more like stuff what could happen, you know, like more graphic pictures. It’s like what they do with the car crash isn’t it like showing like everywhere was like learning, like great graphic images so then it’s stopping dangerous driving.

Pupil: 33:Y8:PS13

There could have been more videos.

Pupil: 75:Y8:PS7

More videos.

Pupil: 75:Y8:PS4

Yeah, and less like talking about it.

Pupil: 75:Y8:PS7

Recent government guidance in England now requires oral health to be included as part of the curriculum. 114 However, at the time of intervention delivery, this was not the case, though many school staff were aware of its upcoming inclusion in the curriculum. Introducing new content that does not relate to national qualifications can be challenging. Incorporating the content covered by the CBS into the PSHE education curriculum had a positive impact on the acceptability of the oral health lesson.

… but now obviously, with the government agenda which is to prepare students to … they’ve introduced PSHE now … so, it’s gone into the curriculum, so that will allow far more flexibility to put that in because it ticks a lot of boxes for PSHE.

School staff 38:1

Indeed, embedding oral health into the curriculum was viewed, by stakeholders, as a key factor for sustainability of the intervention. Moreover, stakeholders attributed the acceptability of the intervention to its sustainability.

… it seems to be a really good programme in terms of sustainability, because if we could overcome the barrier of the phone number, and then if we can, bringing something as important as dental health into the curriculum in secondary schools.

STP03

Self-efficacy

Belief in ability to deliver the lesson was reported as an important acceptability factor. Staff members in leadership teams felt that it was important for those delivering the CBS to feel confident by having sufficient oral health knowledge and support.

Probably that lack of knowledge themselves maybe, lack of confidence in delivering it if they didn’t have that knowledge.

School staff 62:1

It’s something that we’d maybe have to look at freeing up so that maybe she could have a conference call with somebody beforehand to go through it all … maybe just be given a little bit more at our end as well … to give her the confidence in delivering it.

School staff 75:2

Ethicality

The personal beliefs and values of staff and perceiving it as ‘worthwhile’ (School staff 39:2) was a key factor for acceptability of the intervention.

… it was a good idea because we’ve got a lot of kids losing their teeth so I felt it was definitely worthwhile … one of the girls who was in the class told me … she was 12, she’d already had 8 teeth out, so that made it feel like this feels important.

School staff 37:1

Moreover, recognising oral health as an element of general health underpinned the positive interpretation, by staff members, of curriculum targets such as promoting personal hygiene.

… if we put it straight into that personal hygiene sort of framework, how you keep yourself healthy in all aspects … So, it’s about your body health, your mental health, your physical health … So, I think it all ties straight into that.

School staff 38:1

Similarly, stakeholders spoke of the significance of the personal beliefs and values of those involved in delivering the intervention. The recognition of the importance of oral health in particular was perceived as a critical acceptability factor.

I think it depends on school … if they see that as a priority and they see the benefits of such a programme they would be more than happy to do that.

STP03

Interviewer: So, possibly something that the council would commission as opposed to individual schools running it for their own children?

I can’t – schools don’t tend to see it as a priority.

STP02

Burden

A fundamental acceptability factor for staff members was the preparation time and effort required to deliver the CBS. Those delivering the lesson appreciated the resources provided and not having to prepare material for the lesson.

The resources were good … they were grateful they didn’t have to prepare anything … happy to get on board … I may have had some initial comments about whether or not they have to prepare the resources once I assured them … they just have to … review the material before the lesson and deliver it, they were happy.

School staff 62:1

However, some staff members who delivered the lesson found themselves having to print off the materials and arrange a dedicated time for the lesson which was viewed as burdensome.

… that someone just has it dropped on them as an additional extra like what I was given … obviously the printing as well, obviously that took money out of my budget that wasn’t necessarily signposted for this.

School staff 75:1

Moreover, the lack of dedicated PSHE education lessons in this particular school added logistical challenges of finding the time to incorporate something new.

We don’t have PSHE lessons. No, we teach that throughout the curriculum. So … like the logistics of getting them all together … it did work well but like I said, it did take a lot of teacher time for me to prepare it and make sure it was ready.

School staff 75:1

Stakeholders echoed the views of staff and were conscious of the intervention being perceived as burdensome in a busy school environment and reported that minimising the time and effort required by those involved in delivering the intervention was a critical factor for acceptability.

That might be issues around right attitudes, about taking on what could be deemed as extra work. It would really have to be very low in administration, schools are very driven by a lot of pressure on schools about attainment.

STP03

Affective attitude and perceived effectiveness

Pupils generally found the text messages to be acceptable and useful reminders. The language used within the texts was described as appropriate; nonetheless, some pupils described them as ‘cringey’ (Pupil: 37:Y7:PS10) and ‘annoying’ (Pupil: 75:Y8:PS6) as time went on.

The language is just perfectly fine for our age group.

Pupil: 62:Y8:PS6

I think the most helpful thing … was like the reminding that like help me do that.

Pupil: 16:S1:PS5

I find it nice and good because it’s just saying about like teeth ‘Have you brushed your teeth?’… makes you think ‘Oh yeah’ and then I’ll go through my teeth. It’s definitely better than just, someone in your house still telling you to do your teeth it’s better than them … it’s more … pleasant.

Pupil: 57:Y7:PS6

Some pupils reported the texts improved their toothbrushing habits, for instance brushing twice a day and brushing at night.

It was a nice, like reminder to remind me about brushing my teeth every day anyway.

Pupil: 75:Y8:PS11

I’m not used to brushing twice a day, but it helped me to brush them twice a day.

Pupil: 75:Y8:PS7

They helped me remember in the night because I didn’t used to do it in the night but I do now.

Pupil: 75:Y8:PS6

Despite some pupils finding the text messages annoying, the perceived effectiveness of the text messages appears to have played a role in choosing to continue receiving them.

It was good but I found that the text messages were really annoying … I do like to use it as a reminder to brush your teeth.

Pupil: 33:Y8:PS15

Feeling annoyed with the text messages was attributed to their frequency and repetitiveness. Some pupils felt the messages were repetitive and came too often and ‘drains your energy after a while’ (Pupil: 33:Y8:PS8). This may have led to the pupils disengaging with the oral health messages in the texts. To improve the texts, they recommended more varied and creative messages.

Got them every single day.

Pupil: 75:Y8:PS6

Like how often they come, like not so much.

Pupil: 75:Y8:PS10

Just like they’re helpful, but they’re a bit like repetitive and like they don’t vary, is like a set five that just keeps repeating.

Pupil: 33:Y8:PS8

Maybe it should be creative, more creative with texts.

Pupil: 16:S1:PS6

In addition, some pupils found the text messages annoying as they interrupted mobile phone activities such as playing games.

When I’m playing something or like watching something it just pops up and the game just freezes when I’m like about to win so it gets me annoyed a bit.

Pupil: 57:Y7:PS4

It just keeps popping up when you’re like playing stuff.

Pupil: 57:Y7:PS5

Moreover, upon receiving a text message and realising it was not from a friend but rather from the BRIGHT trial, some pupils described feeling disappointed.

Makes you feel important like somebody’s trying to talk to you and then it’s like, no.

Pupil: 16:S1:PS7

Nonetheless, pupils still reported the texts were helpful reminders that prompted them to ‘get up and brush your teeth’ (Pupil: 57:Y7:PS5). For some pupils, however, alert fatigue of receiving texts eventually outweighed their usefulness and they reported blocking the messages as they had ‘done my head in’ (Pupil: 62:Y8:PS2), while others reported blocking them as they felt confident regarding their toothbrushing routine.

I blocked it because I know what times to do them, I just blocked it because I knew my routine.

Pupil: 37:Y7:PS10

Regarding the delivery timing of the text reminders, pupils generally found them suitable; however, for some pupils, the timings did not suit their personal circumstances. This was despite pupils being offered a choice of two set times for receiving the morning texts and two set times for the evening texts with later delivery times offered on weekends.

I think the timing was suitable for my timetable. Like I wake up a couple of minutes before got the message and then the message came and then that would usually be the time I’d go down and brush my teeth.

Pupil: 75:Y8:PS11

No, because I play football. I don’t get home sometimes till half nine.

Pupil: 16:S1:PS2

Pupils recommended having more choice of delivery times particularly on weekends and holidays where they were more likely to stay up late and sleep in.

I think on weekends and holidays, they could have been a bit later … Like I wake up around 11:00.

Pupil: 75:Y8:PS11

Furthermore, pupils suggested providing more information regarding oral health as a way of improving the text messages. In addition, they spoke of the importance of being able to interact and engage with the content delivered, and that they would appreciate the opportunity to be active receivers of information. They proposed an interactive app where they could control and set the delivery times of the reminders in addition to serving as an oral health resource with more detailed content and videos that they could access.

Like an interactive app … Like you can do stuff.

Pupil: 75:Y8:PS4

So like you can have an app with how many times you should brush your teeth regularly, and then facts about it, and then like your reminder, something like that.

Pupil: 75:Y8:PS3

Yeah, so maybe some of the things you had in there … in the lesson be in the apps that you could go back to it.

Pupil: 75:Y8:PS4

Moreover, engagement was also an important factor reported by stakeholders who spoke of the importance of engaging young people in oral health. Furthermore, they valued understanding novel ways to engage with this particular age group suggesting that the results could potentially be applicable to other interventions targeting this cohort.

… once you find a way of communicating with the teenagers, in particular, it’s like everybody wants to get on board. If you find it works, you can find an avenue for loads of stuff.

STP02

As mentioned previously, stakeholders voiced concerns regarding the sustainability of the intervention and incorporating oral health into the curriculum was seen to ensure the sustainability of the CBS. They also highlighted the importance of addressing barriers regarding the text message component such as those related to accessing and maintaining up-to-date phone numbers.

Not necessarily … in your more affluent areas, but definitely in your deprived communities, they lose the phone, they change the phone, they sell the phone.

STP02

Ethicality

Similarly to the views expressed by staff and stakeholders, personal beliefs and values regarding oral health played a key role in the acceptability of the intervention with pupils perceiving it as ‘good’ (Pupil: 57:Y7:PS5).

Well it’s good that like they want to encourage young people to like brush their teeth. So yeah … I mean like it influences them not to have like teeth like smoker’s teeth or teeth that people don’t brush properly and they’re all just like holey and yellow and black.

Pupil: 57:Y7:PS4

The personal significance of oral health led some pupils, despite being initially frustrated and stopping them, to restart receiving the text messages.

Moderator: Does anyone block the messages?

At one point I was because I got really angry with it.

Pupil: 57:Y7:PS5

Moderator: And then what made you start up again?

But it’s like something good because it reminds you to brush your teeth.

Pupil: 57:Y7:PS5

The results of this chapter report participant’s views and experiences of the intervention. This includes the acceptability of the intervention and implementation. Together with the quantitative results, the qualitative results provide a better understanding of how and what was implemented and how participants interacted with the intervention. These results will be brought together and discussed in more detail in the following chapter.

Measuring the clinical impact of toothbrushing behaviour change

Self-reported toothbrushing is considered to be a reliable measure of toothbrushing behaviour85 in epidemiological studies. The primary outcome in the BRIGHT trial was caries prevalence, with the ICDAS system used as the measure for the presence of any type of restoration or carious lesions at the tooth-surface level. In contrast to earlier scoring systems that simply scored the presence or absence of carious lesions, ICDAS allows the assessor to score lesions at different stages and has been widely used in contemporary clinical trials. This gives more sensitivity to detect change in severity of carious lesions. In addition, ICDAS allows the measurement of carious lesions confined to enamel, whereas the DMFT system is almost exclusively used at the threshold of recording only carious lesions that have reached dentine and it cannot detect the extent of carious lesions. ICDAS as an outcome measure is more sensitive. However, because carious lesions are slow to progress, often showing periods of inactivity and even reversal (when confined to enamel), detecting any change in the prevalence of the disease in a population is dependent on time for the disease to progress. If a behaviour is only changed over a short period of time, it will be difficult to detect any reduction in carious lesion progression, even at the level of enamel (where the disease begins and would be initially detected). Where enamel lesions are prevented in a group or reversed for a short time (through increased toothbrushing) if the behaviour is not sustained, the disease will continue again.

If there had been any caries reduction effect from even a relatively short-term behaviour change, this might have been seen in the secondary outcome measuring ICDAS 1–6, if enamel lesions, rather than lesions into dentine, had occurred. However, this was not seen, indicating that there was no evidence of an effect on the caries levels, even at the enamel stage by any increased toothbrushing.

Radiographs could not be used to improve the sensitivity of carious lesion detection because of the unjustifiable risk of using X-rays for research purposes only. ICDAS has been shown to be as sensitive as a dentinal carious lesion diagnostic tool as radiographs, and other supplementary measures. However, there were some constraints imposed by the clinical examinations being carried out in schools rather than in a dental clinical setting. For example, it was not possible to dry the teeth by using a triple syringe or have access to a spittoon/sink, so food debris from meals was removed by the examiner using a piece of gauze or cotton roll to wipe the teeth clean. We had initially attempted to get the young person to brush their teeth (with toothbrushes which were provided) immediately prior to the examination but there was nowhere for them to spit out, which they did not like, so this was changed to simply wiping with gauze/cotton rolls. There was no access to compressed air to dry the teeth and again, gauze and cotton rolls were used to remove the saliva film and allow the examiner to see the surface of the teeth clearly. As a full dental lamp was not available, a mobile dental lamp matching the specifications for epidemiological trials was used.

Plaque levels and gingival bleeding (indicating gingivitis as a result of plaque accumulation) are commonly used as surrogate markers for plaque removal by toothbrushing. These were measured only at 2 (for pilot school participants only) and 2.5 years. There was some evidence of a benefit in gingival bleeding scores in the intervention group compared to the control group at 2.5 years; however, this was a very modest effect. The mean bleeding scores differed by 0.01 points, and the median number of bleeding sites was 1 in both groups. There is little literature on the likely minimum clinically important differences in plaque and bleeding scores with which we can compare our results; it seems that the observed differences would be of borderline, if any, clinical significance, but more research is required to investigate clinically meaningful effects in these outcomes.

Classroom-based session

The findings suggest that staff valued the lesson and reported that the content was generally appropriate for their pupils. Factors that improved the acceptability of the CBS included its inclusion in the curriculum, having available support and having minimal preparation required. In addition, school staff appreciated having the freedom to adapt the lesson to better suit the school context such as varying capabilities for different pupil cohorts and lesson duration. However, to allow pupils to ask questions, it should be recommended that the lesson plan is delivered as a CBS as it was intended rather than an assembly.

In addition, there were some challenges with assessing the actual dose received of the CBS. Three participating schools did not confirm delivery of the CBS and from the 39 schools that confirmed delivery, only 30 provided registers of CBS attendance. It was assumed that all pupils in the intervention year group attended where registers were not provided; however, we know anecdotally that the CBS was delivered to only six of the eight intervention year classes in one of these schools, highlighting the challenges in accurately assessing the dose received. An adaptation was made by one school to deliver the lesson over three sessions rather than one, and for this school, attendance was only recorded if the pupil attended all three sessions. We therefore estimated that 2016 (89.1%) of randomised pupils in the intervention group attended the CBS; however, for the reasons outlined above, this is likely to be an overestimation. There was some contamination as one school started to deliver the CBS to the control year group (n = 69 pupils) in error, before correctly delivering the CBS to the intervention year group.

To further improve the CBS, consideration should be given to the participants’ feedback and appropriate changes made. Accordingly, the CBS lesson plan and activities should include more information on specific aspects of oral health such as explaining the reason why brushing twice a day is important and the difference between toothpastes. In addition, it was suggested that a lesson about dental health should be offered more than just once over a pupil’s time at secondary school to consolidate their learning. The addition of an optional activity should be added to the CBS plan for lessons of longer duration. However, once these changes have been made, the level of testing to date makes the lesson plan appropriate for use in schools across the UK. If a lesson was delivered to pupils who were older than the age group for which the BRIGHT lesson was developed, the content would need to be appropriate to that age group and build on the original lesson plan and activities.

Recently, specific oral health content in the curriculum has become a statutory requirement in England114 and will now be embedded within the curriculum. It is important to note that at the time of the development and delivery of the intervention, this was not the case. Being aware of the upcoming changes in policy appears to have had a positive impact on the acceptability of the lesson for school staff members in England. However, in Scotland and Wales, although general health is part of the curricula, there is less emphasis placed specifically on oral health. The lesson plan was designed based on the literature and tailored by education professionals and young people but may need to be amended if used internationally.

The addition of oral health in the curriculum also addresses the concerns raised by professional stakeholders involved in health or education policy regarding sustainability. Indeed, a lesson on oral health will no longer be considered an add-on but a mandatory part of the formal curriculum thus ensuring it is delivered as part of pupils’ education. This is critical in terms of sustainability and improving young people’s oral health, reinforced by the high levels of caries prevalence we found in the participating schools. Nevertheless, the formal curriculum is only one of the three interacting spheres of a health-promoting school as described by the WHO’s Health Promoting Schools framework. 118 The other interacting elements include promoting health through the ethos and social and/or physical environment and engaging with communities to reinforce health messages. Consequently, for behaviour change to happen, the necessary structures must be in place, such as conducive healthy environments, and enabling pupils to apply their knowledge as education alone is necessary but insufficient to support behaviour change.

Text messages

The findings of this study suggest that, overall, the text messages were acceptable to pupils and were perceived as useful brushing reminders. Other studies reporting on the acceptability of text message behaviour change interventions for young people have reported similar findings. The interventions included those aimed at improving clinic attendance, oral hygiene, physical activity and weight management, contraception use, sun-protective measures or reducing smoking and alcohol misuse. 119,120

Some pupils, however, found the text messages ‘annoying’ and described the wording as ‘cringey’, other pupils described feeling frustrated by their frequency and repetitiveness and, consequently, reported blocking or muting the messages. This is despite the messages being designed with young people during the intervention development stage. Moreover, in addition to the text messages being co-designed, a short pilot was conducted for 2 weeks involving the BRIGHT youth forum. This finding highlights the importance of exploring the perspectives of participants receiving the intervention over time. This is critical to intervention effectiveness as participant engagement needs to be maintained. Indeed, the findings of the process evaluation suggest that dose delivered did not equate to dose received, which will be discussed in more detail below. Previous studies have similarly reported boredom, annoyance, habituation (ignoring messages) and alert fatigue as challenges of mHealth interventions that potentially impact long-term engagement. 121–123 This finding has implications for the development and evaluation of future text message interventions, particularly those aimed at young people.

To further improve the acceptability of text message interventions, the following should be considered: choice of delivery times, interactivity to reduce tedium, varied and creative messages and messages that are informative rather than purely reminders. Some pupils suggested using an app instead of text messages. Indeed, text messages lack the interactivity an app can provide. Furthermore, an app could serve as an oral health resource and be used for signposting. However, using an app in isolation also has limitations. Reminders would still need to be sent twice daily, in accordance with brushing guidelines, potentially resulting in habituation and alert fatigue. Relying solely on an app may exclude those young people without smartphones, those with insufficient space/storage/data on their phone to install the app or those with older models that may be incompatible with the app or unable to make the necessary software updates,124,125 potentially increasing oral health inequalities. There is evidence suggesting oral health interventions that use a mixed approach, adopting text messages and an app, are more effective than employing either approach in isolation. 119

As part of assessing implementation fidelity of the intervention, it was important to record the dose delivered (number of CBS and text messages delivered) and as part of assessing mechanisms of impact, to record the dose received (number of pupils who attended the sessions and the number of text messages received). As mentioned in Chapter 2, a system was in place to collect this information. It is estimated that messages were sent to pupils for a median of about 14 months, and just over 70% of messages sent were recorded as being delivered. However, it was not feasible to document how many messages were blocked or muted and thus the actual dose of text messages received is unclear.

Pupils were given the option to request that messages no longer be sent to their number by texting back STOP and 42.5% of pupils did this, at a median of 2.8 months after they started. However, the focus group interviews revealed that pupils who no longer wanted to receive the messages mainly spoke of blocking or muting them instead of following the formal mechanism in place for stopping delivery of the messages. This meant that the texts were still recorded as delivered but in effect were not received or read. There was some evidence of engagement with the texts with 7124 free-text responses received (excluding the STOP and START messages). This was despite participants being told and reminded that the number was not monitored. Over 60% of these responses were positive or affirmative of compliance with the reminder to brush.

In terms of the technical feasibility of the text messages, the HIC provided the text message service for the BRIGHT trial. In early December 2020, the HIC made the BRIGHT team aware of an issue with the provision of text messages originating from July 2020 when the service provider moved to a new cloud platform; it was discovered that very few messages had been successfully sent or delivered after this time. The BRIGHT trial team were under the impression that the issue had been resolved and that messages had restarted in December 2020; however, in early February 2021, the HIC provided a further update explaining that issues with text messages were ongoing. Following discussions with the TMG, TSC and DMEC, it was decided not to restart the text messages as the hiatus had already been greater than 6 months, most participants had already received text messages for greater than 10 months (unless they had requested the messages be stopped) and that this duration was longer than in other health studies of the impact of text messages. It was agreed that restarting messages after such a long delay would be confusing for the participants.

Strengths

This is one of the largest dental trials in the UK to investigate a novel approach to improving oral health of young people based on behaviour change theory. It used the gold-standard experimental design of a RCT, and successfully recruited from schools with above average proportions of pupils who are eligible for FSM. The methodology included an embedded process evaluation, rigorous health economic analysis and meaningful PPIE activities at the design stage and throughout the trial. The multidisciplinary nature of the team brought methodological experience of conducting school-based trials, with dental, public health, education and psychological expertise. The findings are likely to have implications for (1) secondary-school-based oral health promotion activities across the UK and potentially internationally and (2) the more general development and evaluation of mHealth interventions, particularly text message interventions aimed at young people. There is little information on the content, optimal frequency of or the length of time that text interventions should be sent to support health behaviour change and so this intervention was designed to be open-ended for the participants to decide on when they no longer wanted to receive them.

Limitations

The trial investigated a two-component behaviour change intervention: the CBS and the text messages. It was designed to fulfil the requirements of the commissioning brief (HTA number 15/166) which called for an evaluation of ‘a programme which initiates good oral health practice followed by a series of text or other media messages’ to improve toothbrushing in ‘deprived young people’ and, as a result, improve oral health. As required, the intervention was designed based on behaviour change theory with the two components acting together and so it was not possible to determine the relative contribution of each to the outcome.

The trial was based in schools and recruited pupils across two school year groups. During conception of the trial, consideration was given to the trial design including the unit of randomisation (school or year-group). Randomising at the school level would have minimised the potential for contamination between the intervention and control. Contamination in trials occurs when members of the control group receive the intervention. In this trial, this would have involved (but is not limited to) the school delivering the CBS to control pupils, and/or pupils in the control group independently seeking out technology that would remind them to brush their teeth twice a day. In the design where the unit of randomisation is the year group, as both the intervention and control conditions are present within each school, there is increased risk of contamination. However, a design whereby we randomised at the school level would have required a larger sample size (estimated to be at least 48 schools) relative to within-school randomisation. We decided to commence with the more efficient design of randomising year groups within schools and measure contamination during the pilot phase. This was achieved by asking schools to record who they delivered the CBS to, and asking pupils if, and from where, they had received useful messages about their teeth. We then inflated the sample size to allow for a level of assumed contamination, and it was determined that retaining this design was still more efficient than switching to randomising at the school level (which would have resulted in a larger, more expensive and time-consuming trial). In addition, the trial design was discussed with school representatives who expressed a preference for within-school randomisation, as this meant that at least one of their year groups received the intervention. Discussions regarding the trial design were held with the independent oversight committee (TSC and DMEC) including consideration of potential contamination and they supported a within-school randomisation design. However, it is possible that we did not capture all possible routes of contamination or that this was under-reported by schools and pupils. Increased contamination can dilute the observed treatment effect and so, if not properly accounted for, the trial can ultimately be underpowered to detect this smaller effect.

While the trial recruited well and achieved the target number of schools (n = 42), pupil recruitment was slightly lower, at 4680, than the target of 5040. The observed ICC for the primary outcome associated with school was 0.05, which was higher than the 0.02 assumed in the sample size calculation. Therefore, even if attrition had only reached 20%, as was allowed for in the sample size calculation, the trial would have been underpowered (~74%) to detect a difference from 34% to 26% in the primary outcome, even assuming some partial contamination.

Furthermore, follow-up data collection in secondary schools proved challenging, particularly where there were changes to key trial contacts, changes to the school leadership teams, school mergers and, for schools in England, changes to academy trust status. This was exacerbated by the length of the trial follow-up, particularly as the key trial contacts in the schools had to be changed several times. This was not only because of natural staff turnover but also because the trial contacts were often linked to the year group when the pupils were recruited and changed as the pupils progressed through the school. It is also worth noting the potential implications of these challenges on the implementation of the CBS as well as on data collection.

The biggest impact on the follow-up rate resulted from the COVID-19 pandemic, which led to a national shutdown of schools and prevented or disrupted follow-up data collection. Once schools reopened, several schools did not allow collection of any trial data, or only allowed collection of questionnaire data and not clinical assessments. Schools quoted particular pressures due to: participants in the final year of school were due to complete qualification certificates so research activities could not coincide with these periods of assessment; high levels of COVID-related staff and pupil absence; periods of local school shutdowns or remote learning; and prioritisation of flu and COVID-19 vaccine programmes. The timing of the data collection periods for the schools in the pilot and main trial phase coincided with peak infection rates of the virus particularly in the winter of 2021–2. In addition, during this time, in Scotland, many schools had policies in place which prevented visitors entering schools.

Ultimately, follow-up data collection rates in the trial were low. While the levels of missing data resulted in an underpowered trial to detect the difference assumed in the sample size calculation, ultimately the difference observed was smaller than this and not clinically significant anyway, so statistical significance was of less relevance. Due to the method of randomisation being at the year-group level, levels of missing data were similar in both groups, which lessens the chance for differential attrition and selection bias; however, differences in schools that completed follow-up and those that did not may limit the generalisability of the trial results.

The huge disruption to the lives of young people in terms of routine, schooling and home life caused by the pandemic will also have had an unquantifiable impact on the effect of the intervention. The aim of the intervention was to increase twice-daily toothbrushing within the young people’s normal day-to-day life. With the change in schooling for most pupils from in-person to online, instability of life and lack of routine during the pandemic, the intervention’s ability to establish toothbrushing within their daily routines and linked to specific activities at particular times of the day: waking and getting up (usually for school) and going to bed was likely to have been adversely affected.

The COVID-19 pandemic rendered in-person training and calibration impossible as of March 2020. To mitigate this, online training with one-to-one calibration in-person by a previously trained and experienced assessor was developed. In addition, pandemic restrictions meant examining dentists joining the trial team after March 2020 were trained in completion of the paper data collection forms online. There was significant turnover of examining dentists throughout data collection for a variety of reasons, and instances when dentists could not attend school visits due to COVID-19 infection or were reallocated to other duties because of the pandemic. This required significant allocation of staff, time and financial resources, ad hoc, to ensure school visits to facilitate data collection could proceed as planned in light of unforeseen examiner absence.

The limitations of the process evaluation include the potential for response bias from some participants as a result of the vouchers given to thank them for their time in participating in the focus group. Nonetheless, the data predominantly captured participants’ likes and dislikes of the intervention. It was not possible to gain more information from the participants who after receiving text messages for several weeks requested for them to stop as they could not be purposively sampled in order to maintain anonymity of participants. However, the reasons could be similar to those reported for blocking or muting the texts as a result of the frequency and repetitiveness, and consequent alert fatigue. Another limitation acknowledged is the challenges in assessing implementation fidelity as a result of insufficient data regarding the dose of the intervention.