A school intervention for 13- to 15-year-olds to prevent dating and relationship violence: the Project Respect pilot cluster RCT

Aims

• With stakeholders, to elaborate and optimise Project Respect, informed by existing research.

• To conduct a pilot RCT (four intervention schools and two control schools) in southern England.

Objectives

• To elaborate and optimise Project Respect and produce intervention materials in collaboration with the NSPCC, four secondary schools, youth and policy stakeholders, and the originators of effective US programmes informing our intervention.

• To adapt and cognitively test the Safe Dates and short Conflicts in Adolescent Dating Relationships Inventory (CADRI-s) scales prior to piloting.

• To recruit six schools, undertake a baseline computer-assisted self-interviewing (CASI) survey of two cohorts of students at the ends of year 8 and 9, respectively, plus an online staff survey, and to randomise four schools to receive the intervention and two schools to act as usual treatment controls.

• To ensure that Project Respect is implemented for students in years 9 and 10 and conduct process evaluation, plus follow-up student CASI and staff online surveys 16 months post baseline.

• To address the above research questions to inform progression to a Phase III RCT.

Review of existing literature and materials

The research team reviewed existing systematic reviews, Safe Dates and Shifting Boundaries evaluation reports, and literature on school-based interventions, to identify best practices and inform the intervention design and materials. The NSPCC reviewed Safe Dates and Shifting Boundaries programme materials, as well as materials from other interventions and resource packages, to inform drafting of intervention materials with research team input.

Production of draft materials

The NSPCC led the drafting of intervention materials. These were reviewed by the research team and then redrafted. The redrafted versions were used during optimisation sessions with schools and in a session with the ALPHA young researchers group.

Optimisation sessions with schools

Refinement of materials

The research team and the NSPCC intervention lead reviewed findings from each optimisation session to inform the initial draft of intervention materials (wave 1) and the refine the planned intervention (wave 2).

Recruitment

The participating school selected one year 9 form group to participate in the survey. All students in the participating form group were eligible to participate. We provided information sheets to the school to send home with students for their parents or carers to review (see Report Supplementary Material 4). The information sheet included the study manager’s contact information for parents and carers wishing to opt their child out of participation, and indicated that parents and carers could contact the school directly for this purpose. On the day of the survey, the fieldworker described the CASI pre pilot, answered student questions and administered informed assent.

Data collection

Two fieldworkers administered surveys for the CASI pre pilot using electronic tablets during one class period. Each tablet was preloaded with a survey linked to a unique enrolment identification number. At the time of the CASI pre pilot, the pilot RCT was intended to involve a longitudinal cohort of students and so fieldworkers piloted the process for assigning unique student identifiers to link baseline and follow-up data at the level of the individual student. Fieldworkers followed the CASI pre-pilot standard operating procedure (see Report Supplementary Material 5) to administer the survey and then returned to the university where survey data were uploaded via Wi-Fi to the servers of the partner company that developed the survey software. Once it was confirmed to be complete, the uploaded data set was transferred to the clinical trials unit (CTU) of the London School of Hygiene & Tropical Medicine (LSHTM) for data management and analysis. Fieldworkers took notes on student behaviour, students’ questions, and logistical and technical issues encountered.

Data analysis

We reviewed fieldnotes from the CASI pre pilot to identify any problems with the survey or with the CASI survey methods.

Refinement of survey methods

The CASI survey methods and survey items were refined based on findings from the CASI pre pilot.

Recruitment

We planned to conduct cognitive interviews in one of the optimisation schools. As none could accommodate this, we recruited a replacement school not yet involved in the study (this school later also became a replacement for a school that withdrew from the pilot RCT).

We asked the school to select 16 students (eight girls, eight boys) of varying academic ability from years 8–10, including at least two girls and two boys from each year group. We recommended that young people who had experienced DRV should not participate because of the sensitive nature of the DRV items, but explained that we would not exclude any students on this basis if they wished to participate.

Data collection

Cognitive interviews tested the following:

• Routing items about relationship history (one CADRI-s routing question for girls and one for boys; one Safe Dates routing question).

• Adapted Safe Dates (62 items) and CADRI-s (28 items) DRV measures. For each scale, half of the items measure victimisation and half measure perpetration.

• Survey instructions explaining when a report of DRV would require a safeguarding notification to the school.

• Stereotypical gender-related attitudes (one item).

• Stereotypical gender-related norms (two items).

• DRV descriptive norms (two items).

• Injunctive norms supportive of DRV (one item).

Following a brief self-complete paper questionnaire on sociodemographic characteristics, students were given a warm-up exercise to practice ‘thinking aloud’,56 a cognitive interviewing method in which participants describe their thought processes as they respond to survey items. 57 We then used a combined think-aloud and probing58 approach to test items on relationship history, personal attitudes and social norms, to assess comprehension (understandability) and recall, judgement and response (answerability) for each item. 59 We used show cards to display the items as they would appear on a survey.

Students self-completed the adapted Safe Dates and CADRI-s scales on paper, noting any words or items they found confusing or unclear. The interviewer then verbally probed to explore any issues with the scales overall, before probing in more depth about five specific items. Items explored using probing were drawn from both adapted scales, selected to assess terms we expected might not have the intended meaning for adolescents in England (and to represent five DRV categories: online perpetration,6 physical/sexual perpetration, psychological perpetration, physical/sexual victimisation and psychological victimisation). Probes also explored participants’ comfort responding to these items and their views on the likelihood that girls (for girls) and boys (for boys) would answer DRV items truthfully.

Each interview was scheduled for one class period and the interviewer took detailed notes on participants’ responses to each interview question.

Data analysis

Responses to each cognitive interview question were first summarised by year group and then for the full sample. We highlighted issues identified with understandability or answerability of the tested items and noted when the items appeared to work as intended.

Refinement of survey measures

Survey measures were refined based on findings from the cognitive interviews prior to conducting baseline surveys.

Study population

State secondary schools (including free schools and academies) in southern England were eligible to participate. Private schools, pupil referral units or schools for those with learning disabilities were excluded. Students nearing the end of years 8 and 9 at baseline survey were eligible to participate. No students in participating schools were excluded from our study. Those with mild learning difficulties or poor English were supported to complete the questionnaire by fieldworkers.

Sample size

The pilot RCT focused on feasibility and no power calculation for this was performed. Four schools implementing the intervention in the pilot trial balances the need to assess implementation in a diversity of schools, while ensuring that the pilot is small enough to be appropriate as a preliminary to a larger Phase III RCT. The analytic sample for outcome assessment in the pilot was to be a minimum of 1800 students in years 8 and 9 (aged 12/13 years and 13/14 years) at baseline, with follow-up at 16 months. Data on fidelity and acceptability were intended to provide site-specific descriptive estimates, rather than to be generalisable to a broader group of schools.

Inclusion/exclusion criteria

All students in years 8 and 9 during baseline surveys, in years 9 and 10 during the school year when the intervention was piloted and the process evaluation occurred, and in years 10 and 11 during follow-up surveys, were eligible to participate in research within the pilot RCT. All students in the appropriate year groups were eligible to participate in student surveys, with the exception of students with severe cognitive limitations that would prevent them from understanding what they were being asked to do and assenting to participate, and students who would be unable to participate in the survey in English without the support of a language interpreter.

Recruitment

Three schools in the south-east of England and three in the south-west of England were recruited for the pilot RCT phase. These were purposively sampled to ensure variation by deprivation and school-level value-added academic attainment as indicators of school capacity to deliver Project Respect. School recruitment proceeded via e-mails and telephone calls to schools. Response rates were recorded, as were any stated reasons for non-participation.

Intervention

Although the intervention was designed to be delivered in a Phase III RCT over 2 academic years (targeting students progressing from year 9 to year 10), in this pilot RCT the intervention was implemented only for the 2017/18 school year to students in years 9 and 10 (aged 13–15 years). One year of implementation was judged sufficient to assess feasibility and acceptability to address our research questions.

Control condition

The comparator consisted of schools allocated to the control group, not implementing Project Respect but continuing with existing gender, violence or sexual health-related provision. At the request of the NSPCC Ethics Committee, we undertook a number of additional activities across intervention and control schools, described in Confidentiality and safeguarding. Although these activities meant that provision in control schools differed slightly from treatment as usual, this was deemed to be essential to fulfilling our duty of care to trial participants. The nature of the comparator was assessed by examining provision in and around comparator schools. Retention of control schools was maximised via £500 payment and feedback of survey data.

Outcome and mediator measures

For survey items constituting all outcome and mediator measures and scoring for each measure, see Appendix 4.

Data collection

Data analysis

Informed assent and consent

Prior to all data collection, students eligible to participate were given an information sheet at least 1 week in advance. Immediately prior to data collection, researchers also orally described the study and gave students the opportunity to ask questions. Students were provided with information about school and, when relevant, other local safeguarding support, and a national helpline and other agencies for those experiencing DRV and other forms of abuse or neglect. We then sought student assent (see Report Supplementary Material 4). Students participating in pre piloting of CASI surveys, optimisation sessions, cognitive interviews and qualitative research were informed that our safeguarding policy would require the researchers to report to the school if any students disclosed that they were at risk of serious harm.

We advised students participating in cognitive testing that, in cases when a student reported any of the following, we would need to speak with the student and then notify the school safeguarding officer, per the safeguarding policy in place at the time:

• sexual activity before age of 13 years

• prespecified forms of severe abuse from a current partner

• any other abuse for which the participants themselves asked us to breach confidentiality.

We reviewed each cognitive testing participant’s survey responses for such experiences to determine the need for a safeguarding response.

This policy was not applied to the student surveys in the pilot RCT phase, however, as these were completely anonymous with no linkage to individual names or other identifiers. Results from cognitive interviews (see Chapter 3, Cognitive testing) and consultation with the ALPHA young researchers group (see Report Supplementary Material 2) had highlighted the importance to students of ensuring that survey responses could not trigger disclosures to school safeguarding officials. Students were, however, given the opportunity to speak privately with a researcher if they wanted to disclose any safeguarding concerns.

As is conventional with UK trials in secondary schools,75,76,95 parents and carers were sent a detailed information sheet at least 1 week prior to data collection, via the means of communication preferred by each school, and asked to contact the school or research team should they have questions or not wish their child to participate (see Report Supplementary Material 4). We also offered to hold an information session for parents and carers, if the school wished. Schools were asked to make alternative arrangements, if possible, for students who were known ahead of time not to be participating, so that they would not be present during surveys.

Staff members invited to participate in staff surveys or interviews were sent an information sheet (at least 1 week before in the case of the staff survey) and invited to provide consent online before beginning the survey. The first page of the staff survey also provided information on the study and required participants to give their informed consent electronically before opening the survey.

Confidentiality and safeguarding

Quantitative and qualitative data were managed by project staff using secure data management systems and stored anonymously. Quantitative data were managed by LSHTM’s accredited CTU. All data were stored in password-protected folders. The names used in qualitative data were replaced with pseudonyms in interview transcripts. In reporting the results of the qualitative research, care was taken to use quotations not revealing respondents’ identities. In line with Medical Research Council guidance,99 we will retain all research data for 20 years after the end of the study. This is to allow secondary analyses and further research to take place, and to allow any queries or concerns about the conduct of the study to be addressed. To maintain the accessibility of data, files will be refreshed annually and upgraded if required.

The NSPCC Ethics Committee required that additional safeguarding support be provided to all schools participating in intervention and control arms of the pilot RCT. The NSPCC offered a support session to the safeguarding officers of all schools prior to baseline surveys in case more students sought support. The research team also provided a short report to all trial schools on the prevalence of DRV in their school. NSPCC briefed its ‘Childline’ telephone helpline staff so that they were aware of the project, in case the trial caused students to contact them.

School safeguarding leads were also advised of the reporting procedure regarding any parent, student or staff complaints received about the research. Any member of the research and fieldwork team visiting a school to conduct unsupervised research with a student was required to have a full disclosure and barring services check.

Consent materials for qualitative research indicated that anonymity would be broken if serious abuse was reported. Although this research did not aim to explore students’ personal experiences of sex, relationships or DRV, it was possible that disclosures of abuse might have still occurred. In focus groups, we instructed all participants not to disclose experiences of abuse, as we could not guarantee that all participants would keep this information confidential. All focus groups were conducted by researchers trained to steer discussion away from potential disclosures. Had any disclosures of sexual intercourse before the age of 13 years or other abuse occurred in qualitative research, the researcher would have established that the reported abuse met our criteria for referral and then informed the student that the researcher must report this to the school safeguarding officer. We defined a priori categories of harm warranting such responses with the advice of a social worker specialising in child protection. We consulted with school safeguarding officers in advance to ensure that this process was in line with school policies (see Report Supplementary Material 7). We gave all participants information on school and national sources of support. We also gave young people the contact details of the research team to report any concerns relating to the research.

Any events that met the criteria for a serious adverse event (SAE) or suspected unexpected serious adverse reaction (SUSAR) (defined as an unexpected SAE) were reported to the SSC (which, because this was a pilot not a Phase III RCT, undertook data monitoring and ethics duties), LSHTM and the NSPCC Ethics Committee in anonymised form and in real time, if it was judged to be plausibly caused by the intervention or research. Other SAE or SUSARs were reported to these committees annually in anonymised form (see Report Supplementary Material 8).

Ethics review and conduct

Ethics approval for the study was obtained from the LSHTM and the NSPCC Ethics Committee. All work was carried out in accordance with guidelines laid down by the Economic and Social Research Council, the Data Protection Act 1998,100 and the latest Directive on Good Clinical Practice (2005/28/EC) and General Data Protection Regulation 2018. 101

Recruitment

Four schools consented to participate in optimisation of Project Respect. The schools selected students and staff to participate in optimisation sessions.

School characteristics

Four mixed-sex secondary schools (two in the south-east of England and two in the south-west of England) participated. The first wave (April 2017) comprised two sessions in the south-east of England and one in the south-west of England; one school in the south-west of England was unable to arrange an in-school session and so consultation with one member of school staff occurred by telephone. A total of 31 students and nine members of staff participated in this wave (Table 2). The second wave (July 2017) comprised two focus groups in the south-east of England and one in the south-west of England, with a total of 35 students and six members of staff taking part (see Table 2). The school that had participated via telephone consultation in wave 1 was unable to arrange an optimisation session for wave 2.

All participating students were in years 9 and 10. Staff held various roles. Each session lasted 45–90 minutes. One wave 2 session in the south-east of England did not finish by the end of the time allotted and the year 9 boys who planned to participate were unable to join. A researcher returned to the school 2 days later to complete the session with year 9 students. Three year 9 girls who had taken part the first day were joined by another year 9 girl and three year 9 boys at this follow-up.

Findings

Optimisation sessions with ALPHA group

Participation

One mixed-sex secondary school in the south-east of England that was also involved in our optimisation sessions agreed to participate in the initial pre piloting of the CASI student survey. The school selected one year 9 form group. Twenty-one of the 25 students on the register were in class on the day of the pilot and all did the survey.

Findings

The piloting occurred with a school staff member in the room. Students were given approximately 45 minutes to complete the survey. Most students filled in the survey without questions or comments. Several asked for clarification about terms used in the survey or for help with technical issues. Several reached the end of the survey during the session.

All 21 surveys uploaded successfully and were transferred to the CTU. Through the CASI pre pilot, we identified a few recommendations to refine survey wording for this age group, several recommendations to improve survey logistics and some technical issues with the electronic tablets.

Participation

We initially planned to conduct cognitive testing in one of the Project Respect optimisation schools in the south-east of England, but neither was able to accommodate this. We instead recruited a mixed-sex secondary school academy in London that had expressed interest in the pilot earlier, but consented too late to participate. Fifteen students participated (eight girls and seven boys, aged 13–15 years and in years 8–10) (Table 4). Most (n = 12) reported white British ethnicity and 11 reported their religious group as ‘none’. One parent or carer opted out a boy, who was replaced by another boy in the same year group. No students declined to participate, although one did not turn up for his interview.

Findings

Interviews occurred in April 2017 and lasted an average of approximately 40 minutes, including the informed consent process. When a late arrival or a slower interview pace prevented our testing all survey items, we prioritised testing DRV items and their filter questions then rotated through the other items to ensure that all were tested with at least one girl and one boy in each year group. Report Supplementary Material 9 shows the sample with which each item was tested. Four girls and four boys completed the full cognitive interview and all but one participant responded to the paper-based Safe Dates and CADRI-s survey in full (n = 13) or in part (n = 1), and the dating and relationship history filter questions were tested with all eligible participants. Each of the other items was tested with ≥ 10 participants, as were our draft survey instructions, explaining when a report of DRV would trigger a safeguarding disclosure to the school (reflecting our initial plans for safeguarding).

Cognitive interviews elicited valuable information on how students in years 8–10 interpreted and responded to the tested items. The findings generated a number of recommendations for refinements to item wording and structure, as well as insights into concerns that students might have about privacy and confidentiality. The cognitive interview guide (see Appendix 2) included all tested items. For the items modified after cognitive testing, see Appendix 4.

School characteristics

All schools participating in the pilot RCT were mixed-sex secondary schools (Table 5). Five were academies and one was a community school. School size ranged from 690 to 1654 pupils and the proportion of pupils eligible for free school meals ranged from 4.8% to 41.9%. Value-added attainment scores are school-level measures of students’ progress from school entry to GCSE exams. One such measure, the progress 8 score, compares students’ GCSE performance against predicted performance based on prior attainment. The progress 8 score for most schools falls between –1 and 1. 103 A negative score indicates that, on average, students in the school do less well than those nationally with similar prior attainment and a positive score indicates they do better than this group. Progress 8 scores for participating schools ranged from –1.13 to 0.17. Most items had low rates of missing data, but this was higher for the family affluence scale because of poor response rates for questions (e.g. family holidays and ownership of computers).

Participant flow

Figure 2 shows the flow of schools taking part in Project Respect and the number and proportion of eligible students from each school participating in baseline and follow-up surveys. At each survey wave, four surveys failed to upload because of technical issues. These are not included in the final data set or in the number of students completing each survey reported above.

FIGURE 2.

Participant flow.

Student surveys

Staff surveys

Baseline school and student characteristics

Table 5 shows pilot school characteristics and baseline student characteristics. For the items constituting each measure and information on scoring, see Appendix 4. Of the five schools that participated in baseline student surveys, three were in the south-east of England (two of which would be allocated to the intervention group and one which would be allocated to the control group) and two in the south-west of England (one intervention and one control school). All six pilot RCT schools were mixed sex. One intervention school was a community school and three were academies, as were both control schools. At the time of the baseline surveys, all four intervention schools had a ‘good’ Office for Standards in Education, Children’s Services and Skills (Ofsted) rating, one control school had a ‘requires improvement’ rating and one control school had not yet been rated. The mean value-added attainment score (measured as the progress 8 score) was –0.29 for intervention schools and –0.16 for control schools. The mean percentage of students entitled to free school meals was higher among intervention than control schools (21.8% vs. 11.5%). Intervention schools were larger, with a mean school size of 1189 students compared with a mean of 723 students for control schools. The mean IDACI score was 0.29 for intervention schools and 0.11 for control schools. Schools in the intervention and control groups differed in school characteristics, such as Ofsted rating, mean value-added score, free school meal entitlement, size and IDACI score.

In terms of the characteristics of students participating in baseline surveys, both intervention and control schools were similar in terms of age, gender, living situation and SES. Respondent mean age was slightly over 13 years. Approximately half of respondents were female, and between 11% and 14% reported their gender as non-binary, other or unsure. Around half reported that they lived with their mother and father, around 70% reported that at least one adult in their household was in paid work, 41% reported living in a house owned by the family, and the family affluence scale mean was 6.1 [standard deviation (SD) 1.7] among intervention students and 6.2 (SD 1.4) among control students (possible scores range from 0 to 9, with 9 representing the most affluent). A higher proportion of students in intervention schools than in the control schools reported their ethnicity as other than white British (45.1% vs. 31.0%), and in both arms around 40% reported no religion and slightly more than 20% reported that they were Christian.

Among students in intervention schools, 41.9% reported a boyfriend or girlfriend in the previous 12 months and 68.9% reported ever dating. Among control students, these proportions were higher at 50.1% and 79.7%, respectively. Between 81% and 86% of students in intervention schools reported their sexual identity as straight. Mean scores were very similar among intervention group and control students for attitudes accepting of DRV, injunctive norms supportive of DRV, DRV descriptive norms (among those with friends who reported girlfriends or boyfriends), stereotypical gender-related attitudes and stereotypical gender-related norms. Between 52% and 60% of students self-reported awareness of services to access if experiencing violence in a relationship, and the proportion who had talked to an adult about violence experienced in a relationship was between 39% and 45% in each arm (based on subsamples of 13.6% of intervention students and 9.2% of controls who indicated within this measure that they had experienced relationship violence). The median percentage of correct answers to dating violence knowledge questions was higher among control than intervention students (85.7% vs. 71.4%). At baseline, reported downloading of an app to use to get help when feeling threatened was < 10% in each arm.

Primary and secondary outcomes at baseline

Table 8 shows primary and secondary outcome values at baseline, excluding sexual DRV, sexual health measures and school attendance, which were reported only at follow-up. For the items constituting each measure and information on scoring, see Appendix 4. Outcomes are expressed as a proportion of the overall sample to reflect population risk.

Respondents reporting ever dating someone were routed to the Safe Dates measure (71.7% of the overall baseline student sample). A smaller proportion of students in intervention schools than control schools reported ever dating someone (68.9% and 79.7%, respectively). Ever-occurring DRV victimisation (Safe Dates scale) was reported by more than half of students in intervention and control schools (54.4% and 60.3%, respectively). Regarding subscales, in each arm students reported higher rates of psychological abuse than physical violence victimisation.

Respondents reporting a girlfriend and/or boyfriend currently or in the previous 12 months were routed to the CADRI-s measure (44.0% of the overall baseline student sample). Similar to findings on dating experience, a smaller proportion of students in intervention than control schools reported a girlfriend or boyfriend now or in the past 12 months (41.9% and 50.1%, respectively). DRV victimisation in the past 12 months (CADRI-s measure) was reported by around one-third of students in intervention and control schools (30.3% and 37.7%, respectively) and was slightly higher in the latter.

Around half of students from intervention and control schools reported ever perpetrating DRV, as measured by the Safe Dates scale (46.2% and 52.0%, respectively), slightly lower than rates of reported victimisation according to this scale. As with victimisation, reported rates of ever-occurring perpetration were higher for psychological abuse than for physical violence in both arms. As with the Safe Dates scale, reported DRV perpetration in the past 12 months measured by the CADRI-s was slightly lower than reported victimisation among both intervention and control students (26.2% and 31.2%, respectively).

Regarding secondary DRV outcomes at baseline, reported mean (SD) frequencies of overall DRV victimisation ever (Safe Dates measure) and in the past 12 months (CADRI-s) were similarly low, regardless of measure, ranging from 0.15 (SD 0.22) to 0.18 (SD 0.36), depending on the measure and arm (possible scores range from 0 to 3, with higher scores indicating more DRV). The mean reported frequency scores of ever-occurring DRV perpetration (Safe Dates measure) and in the past 12 months (CADRI-s) were also low, at ≤ 0.10 across measures and arms. Regarding subscales of the Safe Dates measure, mean frequencies of both victimisation and perpetration were higher for psychological DRV than physical DRV.

Regarding other secondary outcomes at baseline, mean scores were similar in intervention and control schools for well-being (SWEMWBS), quality of life (PedsQL), psychological functioning (SDQ) and health-related quality of life (CHU9D). Students in intervention and control schools reported similar rates of sexual harassment (9.8% and 8.4%, respectively), past-year use of NHS services (65.8% and 69.1%, respectively) and being stopped or told off by the police in the past year (17.7% and 18.7%, respectively).

Baseline staff characteristics

Table 9 shows characteristics of staff respondents. Of the 54 members of staff completing the survey at baseline, a large majority described themselves as female (81%). Respondents reported a variety of roles within their schools. Only a minority were in SLT roles, such as deputy or assistant head teacher (n = 5, 9%), but a further 12 (23%) described their role as head of year or of department. The largest category of respondents was subject teacher (n = 19, 35%) followed by ‘other’ (n = 10, 19%). These roles included cover supervisors, examination officers, support staff and subject leads. Three respondents (6%) described themselves as school safeguarding leads or deputy leads and two (4%) were PSHE co-ordinators.

Intervention and control school context at baseline

Student surveys

Student response rates

The overall response rate at follow-up was 78.2% and response rates were > 80% in four schools (Table 12).

One intervention school identified seven students who were not eligible to participate in the survey. Among eligible students, 16.4% from intervention and 8.8% from control schools were absent on the survey day. Only one survey, from an intervention school, was returned via an absence pack.

On survey days, 31 students from intervention schools and 11 students from control schools were deemed ineligible owing to cognitive abilities, English-language fluency, long-term absence or having left the school. Among eligible intervention school students who did not participate in the survey, 16.4% of the eligible sample were absent and 6.7% opted out or were opted out by parents or carers. Among eligible control school students not participating, these figures were 8.8% and 5.2%, respectively. Two surveys from intervention school students and two from control school students failed to upload because of technical issues and are therefore not included in the data set and do not contribute to the response rates.

Staff surveys

Completion rates

Interitem reliability: Cronbach’s and ordinal alphas

Regarding interitem reliability, Cronbach’s and ordinal alphas for the Safe Dates measure of victimisation were 0.94 and 0.97, respectively (with alphas for subscales ranging from 0.90 to 0.95) and for the CADRI-s measure of victimisation these were 0.82 and 0.89, respectively (see Table 14). Cronbach’s and ordinal alphas for the Safe Dates measure of perpetration were 0.91 and 0.96, respectively (with alphas for subscales ranging from 0.84 to 0.94) and for the CADRI-s measure of victimisation these were 0.77 and 0.88, respectively. These alphas suggested very good to excellent reliability of each scale.

Fit: confirmatory factor analysis

Fit indices from confirmatory factor analyses suggested adequate fit. RMSEA estimates generally suggested adequate fit against a benchmark of 0.08,104 except for CADRI-s victimisation measure (RMSEA = 0.107). However, both Comparative Fit Index and TLI estimates suggested that scales had mediocre goodness of fit compared with null models, for which items were allowed to vary but with no correlation.

Completion rates

Across all students at baseline, completion rates for the questions relating to mental well-being, quality of life, sexual harassment, psychological functioning, use of health services, contact with the police and health-related quality of life were very high, ranging from 97.6% to 99.5% (Table 15). Completion rates for DRV frequency as a secondary outcome are the same as those reported for the primary outcomes (see Table 14). School attendance and sexual behaviour were reported only at follow-up; the proportion of missing responses for these measures are reported in Chapter 6.

Interitem reliability: Cronbach’s and ordinal alphas

Cronbach’s and ordinal alphas for interitem reliability of secondary outcome measures were high, ranging from 0.75 to 0.93 for mental well-being (SWEMWBS), quality of life (PedsQL), psychological functioning (SDQ) and health-related quality of life (CHU9D) (see Table 15). Cronbach’s and ordinal alphas of potential mediator measures (attitudes and norms supportive of DRV, DRV descriptive norms, stereotypical gender-related attitudes and norms, and dating violence knowledge) were < 0.7 (ranging from 0.43 to 0.68) (Table 16).

Completion rates and utility scores

The completion rate for the CHU9D health-related quality-of-life measure was 97.8%. The completion rate for the SF-12 was 92.5%. Utility scores using the CHU9D health-related quality-of-life measure were computed for 1397 of 1426 (98%) students who responded to the survey at baseline, or 1397 of 1664 (84%) of all baseline student participants (Table 17).

At follow-up, utility scores using the CHU9D health-related quality-of-life measure were computed for 1512 of 1529 (98.9%) students who responded to the survey or 1512 of 1955 (77.3%) of all follow-up student participants (Table 18).

Mean (SD) and median [interquartile range (IQR)] utility scores at baseline were 0.834 (SD 0.121) and 0.861 (IQR 0.770–0.921) (Table 19). At follow-up, these scores were 0.825 (SD 0.129) and 0.851 (IQR 0.754–0.921), respectively. The distribution of utility scores at baseline and follow-up were broadly similar (Figures 3 and 4, respectively).

FIGURE 3.

Distribution of baseline CHU9D utility scores among students, all respondents (n = 1397).

FIGURE 4.

Distribution of follow-up CHU9D utility scores among students, all respondents (n = 1512).

Utility scores using the SF-6D were computed for 50 of 54 (92.5%) staff who responded to the survey at baseline, or 55 of 723 (7.6%) of all baseline staff participants. At follow-up the figures were 55 of 58 (95%) staff and 55 of 905 (6.1%) staff, respectively. Noting the relatively low numbers, the distribution of values was similar at baseline and follow-up (Table 20, Figures 5 and 6, respectively).

FIGURE 5.

Distribution of baseline SF-6D utility scores among school staff, all respondents (n = 50).

FIGURE 6.

Distribution of follow-up SF-6D utility scores among school staff, all respondents (n = 55).

Reliability

Reliability of the health-related quality-of-life measures was assessed at baseline and was > 0.7: for the CHU9D it was 0.84 (Cronbach’s alpha) and 0.88 (ordinal alpha), and for the SF-12 it was 0.85 (Cronbach’s alpha) and 0.87 (ordinal alpha) (see Table 15).

Fieldwork

Most students were interviewed alone or in pairs. In one intervention school, all six year 9 students participating in interviews did so as a group. All interviews with parents and with members of school staff occurred on a one-to-one basis.

Response rates

The NSPCC-delivered training was audio-recorded in all four intervention schools and school-delivered training was audio-recorded in three intervention schools (Table 23). One intervention school did not complete the school-delivered training. Staff from all four intervention schools returned logbooks, with the number per school ranging from 4 to 13. Some members of school staff delivering Project Respect lessons did not submit logbooks. Informal feedback suggests that, for some staff, logbooks could be seen as an extra administrative task beyond their normal work duties. One lesson was observed in three intervention schools (75% coverage); one teacher whose class was randomly selected declined to participate in the observation, requiring a replacement class to be randomly selected. No lessons were observed in the fourth intervention school because the school had finished delivering lessons before observation could be arranged. Results of classroom observations and a comparison of lesson fidelity, as reported via classroom observation and teacher logbooks, where a logbook was submitted, are presented in Report Supplementary Material 10.

Two interviews were conducted with the NSPCC trainer, one mid-way through and one after the implementation period. Interviews were completed with four staff-members in each intervention school, as planned, and with one additional staff member in one school, resulting in 17 staff interviews (106% coverage). Interviews were conducted with two parents in two intervention schools, as planned, one in one school and none in one school, for a total of five parent interviews (62.5% coverage). As planned, eight students were interviewed in each intervention school, giving a total of 32 across the four intervention schools. Two staff members were interviewed in each control school, giving a total of four. Four students were interviewed in each control school, giving a total of eight.

Quantitative findings on intervention

Qualitative findings on intervention

Sources of unit cost data

Unit costs for NSPCC trainer time were based on market prices obtained from the employer. Costs for school staff time were taken from data provided by the Department for Education,91 which should account for the hourly costs of different grades of staff. Unit costs for use of health services can be obtained from several sources, including the Unit Costs of Health and Social Care,105 NHS Reference Costs,106 the British National Formulary107 and the New Economy Manchester Unit Cost Database. 108 Unit costs for police costs and other criminal justice system costs are available from the Home Office. 109

For the NSPCC-delivered training, the mean total trainer time per school, including preparation, travel and delivery time, was 19 hours and 13 minutes (Table 31). Assuming an hourly cost of £31.07 (Craig Keady, personal communication), the mean cost per school was £597.06. On average, eight members of staff in each school attended this training for a mean duration of 3 hours and 38 minutes. We did not collect data on the grade of each staff member attending training; assuming each attendee was paid at the maximum value of the upper pay range for classroom teachers (£39,406/year)91 and worked 1265 hours per year of directed time,91 the hourly rate was £31.15. The mean cost per school for staff attendance was therefore £905.46. The mean cost per school for NSPCC training was therefore estimated to be £1502.52.

Intervention costs

For the all-staff training, the mean duration of the training session was 47 minutes, and was attended by, on average, 76 staff. Assuming that each trainee was paid at level 3 of the main pay range for classroom teachers (£27,653/year)91 and worked 1265 hours of directed time,91 the hourly rate was £21.86. The mean cost per school for staff attendance was therefore £1301.40. Trainer preparation time was on average 6 hours and 30 minutes, giving a total trainer time, including delivery, of 7 hours and 17 minutes. Assuming a trainer cost of £31.15 per hour, the mean cost was £226.88. The cost of all staff training per school was therefore £1528.29.

Combining these figures, the estimated training cost per school was £3030.80, although this is likely to be an underestimate.

An interview conducted with the NSPCC trainer as part of the process evaluation, identified the following additional costs that ought be considered, which were not included in the calculations above:

• travel costs for the NSPCC trainer (e.g. taxi and train fares)

• time spent liaising with schools to arrange the training session

• time spent corresponding with schools to discuss queries raised about the intervention after the NSPCC training

• time spent supporting schools with all-staff training.

Interviews conducted with school staff identified that for 8 of 28 (29%) staff, the school needed to pay for cover so the staff member could attend the NSPCC training (Table 32). Among staff, 14 of 27 (52%) staff identified that, as a result of attending the training, some work was not done, and in every case the school was required to temporarily allocate a room for the NSPCC training.

Participation

In June 2018, we held two consultations with clients and staff of RCSL, an organisation that works with survivors of rape and sexual assault. All participants were female and the groups were facilitated by two female researchers. These consultations were designed to seek views on Project Respect, with an emphasis on how the programme might impact on those who had experience of abuse.

Findings

Clients welcomed the overall approach but thought that the programme should be made available to younger students aged 11 or 12 years, so that they were prepared for when they started dating. They thought that some parents and carers would like to work through the activity booklet with their child, but that individuals from some cultural backgrounds might not use it because they might not approve of dating during secondary school.

Clients liked the student-led campaigns, which they felt would equip students with knowledge of what was abusive behaviour and empower them to report it. One survivor of sexual abuse said such a campaign would have made her feel that she had had a right to speak up, sharing that she did not disclose the abuse she experienced when younger because she did not feel it was her place to.

Clients thought that the programme would be helpful to a young person with experience of violence at home, as such a programme would help young people to reject a perpetrator’s view that abusive behaviours were normal. Clients also noted that it was important, if someone had experienced abuse, for family relationships to be addressed as part of the programme, and the staff interviewed concurred with this point. A client thought that it was valuable to have a focus on DRV that was separate from RSE, because at her school sex education had been ‘quite triggering, they threw everything in at once, dildos and all and that’s when I was realising, I was being abused’. However, this client also felt that the video and accompanying lessons included within Project Respect might be triggering for someone yet to disclose abuse. She thought that a school counsellor should be available to students when lessons were being delivered.

In their discussion, RCSL staff added that it might take some time after lessons for students to feel ready to disclose, and that a good approach, when delivering lessons on abuse, is to offer an easy ‘out’ if students need to leave, saying up front that students can leave at any time for any reason and that a staff member will come out to check in with them.

Rape Crisis South London staff were supportive of the overall aim of Project Respect, favoured the whole-school approach and student-led campaigns, and thought that delivering the intervention at an earlier age might be desirable. Their view was that young people should be introduced to the concepts of respect and consent from a very early age, and if the programme targets young people aged 13–15 years it will miss preventing family abuse that can begin at a younger age. Staff supported addressing gender at the start of the curriculum. In terms of gendered patterns of abuse, they felt that it was important to teach that people of any gender can perpetrate DRV and that it can happen in relationships regardless of sexuality, but that most DRV is perpetrated by males against females. In terms of gender stereotypes, they said that the curriculum should put DRV into the context of gender stereotypes and discuss ways in which these stereotypes hurt boys as well as girls, making sure to bring boys into the conversation.

The staff were, however, critical of some programme materials. Staff were especially concerned about use of one Childline video about being in a potentially abusive situation because they felt that it was too explicit and would make people who had experienced a similar abusive situation feel very uncomfortable. Staff also thought its portrayal of a very young woman being plied with alcohol and pressurised into sex, which she is resisting, sent the wrong message that if the young woman went along with what was happening to her then she had made the wrong choice and it was her fault. Instead, staff felt that there should have been a greater focus on the perpetrator of the abuse and what he was doing. Staff also felt that the handout provided in lesson 4 on ‘warning signs’ could inadvertently reinforce this implicit message by emphasising the need to ‘be on alert’ and to verbally say no, emphasising the responsibility of the survivor rather than that of the perpetrator. Staff also felt that as young people perpetrating abuse might not know what appropriate behaviour was, the programme content needed to be much clearer about exactly what behaviour was sexually inappropriate and abusive. They critiqued use of the word ‘unhealthy,’ which they felt was too indirect and implied a mutuality, rather than focusing on perpetration of abuse. Staff emphasised that, overall, the programme should be framed to target potential perpetrators, while also ensuring that messaging speaks to survivors in the room. The latter needed to bear in mind how a student with experience of abuse will receive what is being said and ensure that no aspects of the programme imply that abuse is the survivor’s fault.

Disclosure was a major theme in the focus group with staff. They thought that the policy audit within Project Respect was particularly important in this regard. They thought that all schools should work to develop a policy that is responsive to disclosures of abuse and ensures that young people are clear about what school staff are required to disclose; aware of resources outside the school, in case they prefer this route; and understand that after disclosing abuse, they will remain in control over what happens next. Staff suggested that Project Respect could ask schools to produce a flow chart on safeguarding and how they respond to disclosures that could go up in each classroom. The staff liked the booklet provided for parents and the way that it dealt with disclosure. They were, however, sceptical about whether or not teenagers would be willing to undertake the suggested activities with their parents. Clients also thought that these activities would be feasible for some families but not others.

The conclusion from both discussions was that Project Respect was a good idea and had some commendable features, notably the student-led campaigns and policy audit. RCSL clients and staff agreed that it should be undertaken with younger school students before they started dating. Both clients and staff expressed concerns about the potentially negative impact of the film on those who had experience of abuse, and thought that some of the lesson materials needed to be modified to be more sensitive to survivors of abuse, ensure that they do not blame survivors and provide clearer information on exactly what behaviours were abusive.

Participation

We held a consultation with 16 stakeholders from non-governmental organisations and local and national government in March 2018 (see Report Supplementary Material 13). The meeting aimed to build support for the research, identify potential implementation facilitators or barriers to explore in the process evaluation and ensure that the research would be policy relevant.

Findings

Study refinement

To improve response rates for the staff survey, staff recommended engaging school leadership to promote the survey and arranging a scheduled time for respondents to fill it in. Staff suggested that the parental opt-out approach was acceptable to schools; holding a parents’ evening about the study would not add value; student information sheets should be much shorter; and the extent of the requirements for distributing information before and after data collection was burdensome on schools.

Intervention refinement

Teachers suggested that Project Respect could be improved by taking better account of schools’ engagement with DRV issues and school staff members’ knowledge about DRV, pitching training at a more advanced level, and initiating implementation with a better understanding and acknowledgement of the school’s past work. One person we spoke with suggested holding a meeting early on in the project to learn about the school’s past efforts and current needs.

Knowledge transfer

Teachers suggested that schools would be particularly interested in findings that directly inform work the school is doing, school-specific survey results and findings that allow them to compare their school with other similar schools.

Progression criteria

The first question that our study focused on was whether or not pilot trial results suggest the appropriateness of progression to a Phase III RCT in terms of prespecified criteria. The first criterion was that randomisation occurs, and four or more schools (out of six) accept randomisation and continue in the study. Randomisation did occur and all six schools accepted the results of the randomisation and continued in the study. However, one school was randomised despite not being involved in baseline surveys. The differences between intervention and control schools in school characteristics, such as Ofsted rating, mean value-added score, free school meal entitlement, size and IDACI score, are unsurprising given the small number of schools randomised, and did not bias the pilot RCT given its focus was not on estimating intervention effects.

The second criterion was that the intervention is implemented with fidelity in at least three of the four intervention schools. The target for delivery of NSPCC training was 100% coverage of essential topics. The target for delivery of school-delivered elements was 75% coverage of essential topics. NSPCC delivered training in all four schools to key staff leading the intervention, but with fidelity below 100%. Schools aimed to deliver training to all staff in their school. In two schools, this occurred with fidelity > 75% and in one school with fidelity < 75%, whereas in one school this training was not delivered. Therefore, the target of fidelity in three schools was not achieved. Review of school policy occurred in two schools, therefore not achieving the target of this review in three schools. Hotspot mapping was undertaken by staff in all four schools, therefore achieving the target. Hotspot mapping was undertaken by students in three schools, therefore achieving the target. No schools modified how staff patrolled the school site informed by the hotspot mapping, therefore not achieving the target. Overall, the student curriculum was delivered with fidelity > 75% in three schools, therefore achieving the target. Parent information was sent out in all four schools, therefore achieving the target. However, qualitative data from parents and staff suggest that some schools did not send the parent booklet to all parents (e.g. either producing only a limited number of hard copies distributed to some parents and carers or putting the booklet on the school website). Details of the Circle of 6 app were provided to students in all four schools, therefore achieving the target. However, qualitative research suggested that although some schools provided students with time in lessons to download the app, other schools merely mentioned the app as part of Project Respect lessons. Considering our targets for fidelity as 100% for NSPCC-delivered elements and 75% for school-delivered elements, overall, one school delivered the school components with fidelity and no schools achieved overall fidelity for both NSPCC- and school-delivered components.

The third criterion was that our process evaluation indicates that the intervention is acceptable to ≥ 70% of year 9/10 students and the staff involved in implementation. Surveys with year 9/10 students in intervention schools found that among those who reported that their school had been taking steps to reduce DRV, almost 90% supported this work, which meets the target of 70% acceptability for this measure. However, students in intervention schools were less likely than students in control schools to report that the school had been taking such steps. In addition, of those reporting that their school had been taking these steps, students in intervention schools were slightly less likely to report that they supported this work than controls. Of the approximately 37% of students in intervention schools reporting that in the past year they had been learning about respectful relationships, just under 60% reported that these lessons were good. These figures in respect of students do not meet the target of 70% acceptability. We intended to examine acceptability of the intervention to staff via staff questionnaires, but response rates were very low, particularly among senior staff with most overview of the intervention. Therefore, instead we assessed acceptability to staff involved in intervention using the qualitative interviews. According to these, the intervention was acceptable to 10 (59%) staff and unacceptable to two (12%) staff, with three (17%) staff having mixed feelings and two (12%) staff being insufficiently aware of the intervention to have an opinion. These figures in respect of staff, also do not meet our target of 70% acceptability.

The fourth criterion was that CASI surveys of students are acceptable and achieve response rates of at least 80% in four or more schools. Overall response rates at baseline were 82.5% (with one school not doing the baseline survey and not contributing to this total response rate) and response rates were > 80% in three schools. Overall response rates at follow-up were 78.2% and response rates were > 80% in four schools. Therefore, the target of achieving response rates of at least 80% in four or more schools was achieved at follow-up, but not at baseline. CASI surveys enabled good data collection and were acceptable to students. Benefits of the CASI approach include automated routing, which meant only students reporting sexual experience in an initial routing question were asked further questions with sexual content; and data being upload via Wi-Fi, eliminating the need for data entry and the risk of potential data entry errors for most surveys. However, the use of tablets introduced logistical challenges for the fieldwork teams. Tablets were bulky to transport, and set-up could be complicated and reduce the time available for surveys. The devices also required considerable time and a specially equipped room to upload data, clear and recharge tablets, and prepare them for the next survey day, which meant that fieldworkers worked extremely long days. Reliance on a limited number of CASI tablets made it harder to schedule survey dates with schools. Our surveys were not associated with increases in students seeking support from school safeguarding leads.

The fifth criterion for progression was that methods for economic evaluation in a Phase III RCT are feasible. We examined whether or not an economic evaluation (within-trial cost–utility analysis from public and voluntary sector perspectives) would be feasible. The evaluation would include the mean cost of the intervention per school, and its impact on use of services and health-related quality of life. We determined that it would be feasible to calculate the costs of each intervention component listed in Table 1; we identified some costs not quantified in our analysis (e.g. post-training contacts between the NSPCC and schools) that should be included in any future Phase III RCT. Usable survey data on use of health services and contacts with police were available for 98% of respondents at both baseline and follow-up. The resource use data collected in the present study, for both the intervention and the follow-up, seem appropriate and plausible, and response rates were high. We note that the data collection tools that would need to be used in a full RCT would need to be disaggregated by different types of health service use and use of the criminal justice system, and allow for a full range of values to be recorded. It was possible to compute utility scores using the CHU9D health-related quality of life measure for 98% of respondents at baseline and 99% at follow-up. However, we report very low response rates among staff, indicating that it would not be feasible to use these to collect data on SF-6D as a measure of staff health-related quality of life, even with the use of multiple strategies to promote staff responses. Our research highlighted that cost-effectiveness analyses based on the primary and secondary outcomes of the trial (e.g. using measures of DRV victimisation and perpetration) could be conducted alongside a cost–utility analysis. With regard to economic outcomes, we recommend that data collection tools in any future trial disaggregate different types of health service use (e.g. emergency department visits, general practice visits, practice nurse visits) and involvement with the criminal justice system, and allow for a full range of values to be recorded. It also highlighted that long-term modelling of costs and outcomes beyond the end of the trial would be challenging because of the lack of data, producing uncertain estimates; our recommendation was therefore to conduct a within-trial analysis only alongside a Phase III RCT. Aside from not achieving the baseline target student survey response rate and also achieving very poor response rates in the staff survey, it was determined that an economic evaluation focused on student health-related quality of life would be feasible.

We identified the following cost components that ought to be included in any future economic evaluation of the intervention:

• time spent by the NSPCC trainer preparing for the training, travelling to schools and delivering the training

• travel costs for the NSPCC trainer

• time spent by the NSPCC trainer liaising with schools to arrange the training session

• time spent by the NSPCC trainer corresponding with schools to discuss queries raised about the intervention after the NSPCC training

• time spent by the NSPCC trainer supporting schools with all staff training

• time spent by school staff attending NSPCC and all staff training (this should also account for the grade of staff)

• use of school space for training activities

• impact on use of health services and use of the criminal justice system.

In our calculations we did not include the costs of providing information to parents; staff time spent preparing and delivering the classroom curriculum to students; making the Circle of 6 app available to students; time spent reviewing school policies and rules; time spent hotspot mapping; or time spent modifying school patrols in response to identified hotspots.

These activities may impose additional costs to the school, but these are likely to be small. This could be tested in a future evaluation.

A future economic evaluation should include a cost–utility analysis, with outcomes measured in terms of QALYs. We have demonstrated in this study that it is feasible to collect data on health-related quality of life for students using the CHU9D and therefore to estimate QALYs. Given the poor response rate to the staff survey, and that school staff are not the primary audience for the intervention, we recommend that a future economic evaluation does not require utility measurements for school staff.

Given the range of potential outcome measures for the intervention demonstrated by this study, we recommend that cost-effectiveness analyses based on the primary and secondary outcomes of the trial should also be conducted alongside a cost–utility analysis.

Research questions not pertaining to progression criteria

In addition to the question of progression, our study examined other research questions. The second research question was ‘which of two existing scales, the Safe Dates and the CADRI-s, is optimal for assessing DRV victimisation and perpetration as primary outcomes in a Phase III RCT?’. Reliability of our outcome measures was assessed at baseline across the overall sample. Completion rates for both the Safe Dates and the CADRI-s measure of DRV were very high, at around 99%. Cronbach’s and ordinal alphas were also very high, with those for the Safe Dates measures of both victimisation and perpetration being marginally higher, at > 0.9. Fit indices from confirmatory factor analyses suggested adequate fit. Findings suggest that both measures are reliable, so, as per our protocol, our recommendation would be to use CADRI-s, as this is a more established measure involving fewer items. This recommendation is supported by consultation with the ALPHA young researchers group (see Report Supplementary Material 2).

The third question that our study aimed to assess was ‘what are likely response rates in a Phase III RCT?’. As reported above, overall response rates at baseline were 82.5% and at 16-month follow-up were 78.2%. We would expect follow-up rates in a 28-month Phase III trial to be slightly lower than this. However, response rates were much lower for the staff survey.

Our fourth research question was whether or not estimates of prevalence and ICC of DRV derived from the literature look similar to those found in the UK so that they may inform a sample size calculation for a Phase III RCT. At baseline, the overall prevalence of DRV victimisation, as measured by the Safe Dates scale (ever occurring), was just over 50%, with just under 40% reporting physical DRV, and the prevalence, as measured by CADRI-s (past year), was around 30%. At baseline, the overall prevalence of DRV perpetration ever, as measured by the Safe Dates scale (ever occurring), was also around 50%, with around 30% reporting perpetration of physical DRV, and prevalence, as measured by CADRI-s (past year), was just under 30%. At follow-up, when both scales measured past-year DRV, reported prevalences were more similar. The overall prevalence of past-year DRV victimisation, as measured by the Safe Dates scale, was around 35% in both trial arms, with around 20% in both arms reporting physical DRV. The prevalence of past-year DRV victimisation, measured by the CADRI-s, was just over 30%. At follow-up, the overall prevalence of past-year DRV perpetration, as measured by the Safe Dates scale, was about 25% in both arms, with just over 10% reporting perpetration of physical DRV in both arms. The prevalence of past-year DRV perpetration at follow-up, as measured by the CADRI-s, was just under 30% in both arms. Our DRV prevalence estimates cannot be directly compared with other estimates from UK samples, which report prevalence only among daters and use different measures of DRV. 6,25,37 However, our estimates of different forms of DRV victimisation reflect patterns from the broader literature, in which psychological DRV is reported most frequently, followed by physical and then sexual DRV. Given our small sample, we recommend that power calculations draw on the estimate of 50% based on existing studies. 6,25 It was not possible to estimate ICCs for any measures of DRV because of the small number of clusters. Therefore, we recommend that power calculations draw on the conservative estimate of 0.07 based on existing studies. 111–113

The fifth research question was whether or not our secondary outcome and covariate measures are reliable and ‘what refinements are suggested?’. As with primary outcomes, the reliability of our secondary outcome and mediator measures was assessed at baseline across the whole sample. Cronbach’s and ordinal alphas for all multi-item measures were > 0.7. The reliability of our multi-item mediator measures was, however, much lower, with all scoring < 0.7. This suggests that these measures need to be refined before further use.

Our sixth question was ‘what refinements to the intervention are suggested?’. Our process evaluation suggested that the universal approach targeting male and female students together was acceptable to staff and students, with some evidence that this facilitated the challenging of sexist gender norms. Our evaluation identified a number of challenges and areas for refinement. Interviews with school staff suggested the need for changes to the training for school leaders and other key staff. The goals of the training need to be clearer so that schools can field the most appropriate staff. Different staff might attend different elements of the training, such as that for senior leaders on policy review and changes to patrols, and that for curriculum leads on lesson planning. School staff thought that there should be less attention in the training to background information and attitudes to DRV, and more on how to deliver the classroom curriculum. Staff also recommend that the all-staff training led by key staff who have been trained by NSPCC should have clearer goals (e.g. relating to intervening in DRV or sexual harassment on the school site).

Project Respect’s intervention materials did identify the key staff in each school that should be involved with its implementation. These included a lead, responsible for day-to-day management, and a SLT member to champion the intervention, as well as the school’s leads for safeguarding and PSHE. However, the manual was not clear about the goals of the training, whether or not the goals included training those staff who would actually deliver the curriculum and whether or not all such staff should attend the NSPCC training or just the staff member responsible for co-ordinating delivery of the curriculum. The training manual suggested that the teachers who would deliver the curriculum should meet with the curriculum co-ordinator to go through lesson plans and slides. The NSPCC training materials did not aim to review each lesson, but did aim to provide an overview of the curriculum, distribute the curriculum materials to participants, and spend some time looking at the lesson plans and slides for a sample of lessons. Just under 1 hour of the training was timetabled to be spent on the curriculum.

Staff, students and parents were often only hazily aware of written information for parents, including a booklet for parents to work through with their children. There needs to be more attention given to ensuring that schools send these materials to all parents and carers. Staff and students liked the Circle of 6 app, but schools varied in whether or not they provided time for students to download the app. This should be a required element of the curriculum. The classroom curriculum, more generally, attracted mixed views. The intervention should provide schools with a system for identifying suitable staff to deliver the curriculum. There were suggestions that lessons be designed so that they can be taught in a variety of lesson formats; slides have more images and fewer words; lesson plans be easier to read; lesson plans include suggestions for how lessons could be adapted for different contexts; discussion activities be better directed, for example through suggested group activities; and curriculum materials have greater attention to student diversity. Consultation indicated that student-focused components should be clear about perpetrators’ sole responsibility for abuse and be sensitive to the experiences of those who have survived abuse. Staff also suggested that the role of the NSPCC in ongoing support for intervention delivery should be better defined. Given the challenges with fidelity, there is probably a greater role for the intervention provider in working proactively to promote delivery and ensuring that fidelity is continually assessed and promoted.

The seventh research question asked what qualitative data suggest about how contextual factors might influence implementation, receipt or mechanisms of action. In terms of May’s theory of implementation,87 schools varied in their collective commitment to the intervention. There was generally a high and shared commitment to address DRV, which supported implementation. In some schools, single individuals took responsibility, leading to lack of broader buy-in and discontinuities if these staff members left the school. Schools’ commitment could be undermined and therefore not sustained over time by insufficient buy-in from school leaders, negative perceptions of the training, new priorities arising (e.g. responding to worsened Ofsted ratings or examination results) and reduced staff morale (e.g. because of staff restructuring programmes). Intervention components had variable workability. Schools were generally able to implement the curriculum, despite the limited lead-in period making timetabling and staffing of lessons not easily workable. Staff views on whether or not curriculum materials were sufficiently inclusive and supported delivery varied and were partly influenced by views on the training. Other elements proved less workable: policy review and changes to patrols, because intervention systems and timescales did not match with those of schools; and student campaigns, because classroom teachers lacked the time and skills to facilitate such work. Workability would be facilitated by notifying schools of their allocation to receive the intervention much earlier in the school year before delivery was to start, so that this could be factored into the planning of teacher training sessions, policy review, staff patrol rotas and the student timetable. Some staff also saw the size of the intervention as detracting from its workability. These staff recommended that the intervention involve fewer lessons. Staff relationships influenced implementation. In schools in which management–staff relationships were damaged (e.g. by staff restructuring), this could hamper collective action to implement (e.g. the curriculum). Schools successfully integrated the curriculum into existing timetables and reported that they would sustain some of these lessons, as well as some elements of all-staff training, in future years. In terms of reviewing our theory of change, drawing on our qualitative data viewed through a realist lens,89,90 our findings suggest that activities associated with the intervention could not only challenge student attitudes and norms uncritical of sexual harassment and DRV, but could also increase student awareness of DRV (particularly in non-urban settings in which there may be lower baseline awareness), and encourage students and staff to challenge sexually abusive and sexist language at school (particularly in contexts in which such language is currently normative, but where there is management support to challenge this).

Our eighth research question was whether or not qualitative data suggest any potential harms and how these might be reduced. We found some evidence that aspects of the intervention might be harmful. In one school a staff member reported that the lack of clarity of messages about seeking consent for sexual activity left some students confused about how to seek this. For some staff, the need to drive forward delivery of the programme in a short space of time led them to feel that their well-being had been threatened.

Our final research question was what sexual health- and violence-related activities occur in and around control schools. According to staff, control schools had written policies addressing bullying and sexual harassment, which did not refer explicitly to DRV. Staff reported that their schools responded to incidents of sexual harassment, DRV or homophobic abuse via the safeguarding officer. Schools punished perpetrators and involved the police when necessary. Staff in control schools reported that violence prevention was covered in lessons, assemblies and events scheduled as part of antibullying weeks. This provision generally did not focus specifically on DRV, but did sometimes refer to domestic violence. Staff reported that some RSE lessons encompassed topics relating to the prevention of DRV, but could not quantify this. Staff in control schools also referred to various forms of student-led action on antibullying, as well as challenging sexism.

Serious adverse events and suspected unexpected adverse reactions

Participating schools reported SAEs and SUSARs annually to the research team. The mean number of SAEs and SUSARs reported per school was six among intervention schools and three among control schools (excluding data missing from one intervention school not reporting on this in the second year of the pilot), and no reported SAEs and SUSARs were deemed to be plausibly linked to Project Respect.

Study objectives

In addition to addressing our research questions, other objectives of the research included the collaborative optimisation of the intervention; the cognitive testing of DRV measures, and measures of norms and attitudes to gender and DRV; the piloting of intention-to-treat analyses for primary and secondary outcomes; and assessment of intervention reach and whether or not this varied by student or school characteristics. The intervention was successfully optimised in collaboration with the NSPCC and four secondary schools, as well as youth and professional stakeholders. Cognitive testing of measures suggested that items were generally well understood but informed some rewording. Despite the study being underpowered to assess intervention effectiveness, we piloted intention-to-treat analyses of primary and secondary outcomes, finding no trends indicative of intervention impacts. Intervention reach in terms of student awareness of activities was poor, with only slightly higher rates of awareness in intervention compared with control schools. We found evidence that affluent students were more aware of the intervention overall, but awareness of this did not vary by student ethnicity and gender, and awareness of the curriculum did not vary by student affluence, ethnicity or gender. Our small sample precluded quantitative assessment of variation in reach by school characteristics. There was likely to be lower reach in those schools delivering the intervention with lower fidelity, especially in schools in which all curriculum lessons were not delivered to all year 9 and 10 students.

Amendments to protocol

The protocol was amended at several time points, with changes logged and approved by the SSC (see Appendix 1). Most changes were procedural or made for the purpose of clarification. Randomisation was stratified by region rather than by school attainment, as the former was judged to be more important to explore as a potential influence on implementation. Cognitive interviewing was broadened to encompass measures of attitudes and norms about gender and DRV.

Student surveys were rendered fully anonymous so that students could report experiences of DRV without fear that this might lead to identifying them to school safeguarding officers without their consent. The protocol was also updated to indicate that schools could expect approximately weekly ongoing support in the form of a 1-hour conversation with the NSPCC. The protocol was also updated to define specific fidelity targets for each intervention component. Non-volitional sex was removed as a secondary outcome, as this overlapped with the sexual abuse element of DRV, as measured in primary outcomes.

Study ethics and safeguarding procedures were amended as required by the NSPCC Ethics Committee. This included the requirement that schools send out study information to students and parents and carers 1 week before any data collection, as well as further information after data collection; the study team provide a short report to all participating schools on the baseline prevalence of DRV in their schools; and school safeguarding officers be provided with training in responding to DRV. These additional requirements proved very onerous for schools and undermined their commitment to the study. They also meant that schools in both arms of the trial were engaging in additional activities focused on DRV, potentially leading to contamination and the undermining of trial generalisability. The offer of training for school safeguarding officers was widely regarded as patronising among school staff. The full anonymisation of student survey data led to the trial shifting from one based on a longitudinal design to a repeat cross-sectional design.

In addition, at the request of the NSPCC Ethics Committee, student surveys did not feature detailed questions on sexual experiences at baseline, but these questions were retained at follow-up.

In terms of mediators, the Rape Myth Acceptance Scale was removed at the request of the NSPCC Ethics Committee as well as our youth stakeholders, as items involved a barrage of negative and upsetting statements.

Deviations from protocol

There were also a number of deviations from protocol not included within the above amendments (Table 37). Several of these were also procedural but others were more substantive.

Other limitations

In general, response rates for student surveys were good, although one school, which joined the study late to replace another school that had dropped out, was not able to undertake baseline surveys. Response rates for the staff survey, including the SF-12 questionnaire, were extremely low (< 10%), despite our use of multiple strategies to encourage participation. This meant that our assessment of staff awareness and acceptability of the intervention had to be based on interviews, thus drawing on a smaller sample. However, this still provided an adequate means of determining that acceptability of the intervention to staff was suboptimal. Most elements of the process evaluation had very good response rates, but completion of logbooks by staff delivering the curriculum was patchy. This meant that our assessment of the fidelity of delivery of this intervention component is somewhat uncertain. In retrospect, our use of simple ‘stop’ or ’proceed’ progression criteria was too crude. It would be more appropriate for external pilot studies, such as this one, to use criteria which allow for ‘stop’, ‘proceed immediately’ or ’proceed with refinements’, when the thresholds of success for ‘proceed with refinements’ might, for example, be lower than for ‘proceed immediately’. This would not have changed our decision in the case of this study, but in other studies would not prevent minor, surmountable challenges from preventing progression to a Phase III RCT. Our consultations were participative meetings rather than research, and so results from them could not be quantified. Those participating had not experienced the intervention and some comments, such as the best age to target for DRV prevention, went beyond the remit of the present study. Our pilot RCT was focused on questions of feasibility and acceptability; lessons were delivered to two student cohorts and not to one cohort over 2 years; and the pilot could not estimate intervention effects, which must, instead, be examined in larger, Phase III studies. Although such Phase III RCTs can assess school-based interventions, including those with whole-school elements,114 realistically they can focus only on interventions delivered over relatively short periods of 1–3 years. Therefore, RCTs cannot practically be used to evaluate provision which stretches across longer periods of students’ schooling.

Implications for schools and education policy

High rates of reported DRV victimisation and perpetration highlight an ongoing need for effective approaches to reducing DRV. Although this is a societal problem, which schools alone are unlikely to fully address, there is evidence from existing trials that school-based interventions can make an important contribution. 20,29,30 Our pilot RCT suggested that schools saw addressing DRV as part of their responsibility to address safeguarding and were supportive of the multicomponent approach that Project Respect took.

Implications for research

Our study suggests, however, that there should not be an immediate proposal for a Phase III trial of this intervention. This is informed by our finding that interviews with staff and students suggest mixed views, particularly among staff, on certain elements of the intervention in its current form, including the training and the curriculum materials. Interviews also suggest that the number of lessons dedicated to DRV might be too large for busy schools for a topic which is best considered one element within broader RSE, particularly in the context of RSE becoming statutory in all state schools. 115 This suggests that if any future Phase III trial is warranted, then this should focus on a broader intervention focused on RSE and include DRV in this. This aligns with the increasing recognition that a separate curriculum for each health topic is unrealistic and integrated health education is a more feasible strategy. 116

Our findings from staff interviews suggested that a refined intervention should have a longer preparatory phase so that schools have time to plan the intervention; ensure stronger SLT buy-in so that teachers delivering the intervention have senior support; ensure training components have clearly defined audiences and objectives so that the right staff attend and can be confident what they will learn; have a longer timetable for policy review to accommodate differences between schools in how policies are reviewed; provide schools with a system for identifying suitable staff to lead the intervention, including the curriculum; ensure curriculum materials encompass planned adaptability and more support for discussion elements, including more material to present beforehand to help stimulate discussions; ensure student-focused components are clear about perpetrators’ sole responsibility for abuse and are sensitive to the experiences of those who have survived abuse, as well as being inclusive and accessible; allow time for students to download the Circle of 6 app; ensure schools have comprehensive systems to send materials to parents and carers; and include a defined package of external delivery support so that the intervention might be implemented more consistently.

Although staff suggested the need for fewer lessons on DRV, existing effective interventions have 6 to 10 lessons dedicated to DRV. Within a broader RSE intervention, DRV could be integrated in multiple lessons addressing diverse topics relating to DRV, as well as gender, relationships and communication, ensuring both feasibility of delivery and an adequate ‘dose’ addressing DRV and contributing factors.

Any future RCT requires schools to be randomised some time before the intervention is to be delivered, to enable preparation time. It should focus on the CADRI-s measure as its primary DRV-focused outcome measure. To enable analyses of student-level confounders, mediators and moderators, any future trial should examine innovative ways to ensure that student surveys are anonymous, while allowing individuals’ baseline and follow-up surveys to be linkable. This might occur, for example, via use of student-created identifiers based on facts known to students. 117 If surveys are administered using electronic tablets, they should be undertaken with careful planning in regard to facilities, fieldworker staffing and the supply of tablets to mitigate logistical challenges introduced by this approach. Surveys should use paper questionnaires, as these are logistically less challenging for fieldworkers. Ethics procedures should remain focused on protecting students from harm, but should aim not to excessively burden schools. Staff surveys appear unfeasible, despite use of multiple strategies to increase response rates, therefore other methods are required to assess staff experiences and views, and assessment of staff health-related quality of life is challenging, if not impossible.