A video-feedback parenting intervention to prevent enduring behaviour problems in at-risk children aged 12-36 months: the Healthy Start, Happy Start RCT

Article history

The research reported in this issue of the journal was funded by the HTA programme as project number 13/04/33. The contractual start date was in October 2014. The draft report began editorial review in January 2020 and was accepted for publication in October 2020. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The HTA editors and publisher have tried to ensure the accuracy of the authors’ report and would like to thank the reviewers for their constructive comments on the draft document. However, they do not accept liability for damages or losses arising from material published in this report.

Copyright statement

© 2021 O’Farrelly et al. This work was produced by O’Farrelly et al. under the terms of a commissioning contract issued by the Secretary of State for Health and Social Care. This is an Open Access publication distributed under the terms of the Creative Commons Attribution CC BY 4.0 licence, which permits unrestricted use, distribution, reproduction and adaption in any medium and for any purpose provided that it is properly attributed. See: https://creativecommons.org/licenses/by/4.0/. For attribution the title, original author(s), the publication source – NIHR Journals Library, and the DOI of the publication must be cited.

2021 O’Farrelly et al.

Behaviour problems and their impact on health and development

Behaviour problems are among the most common mental health problems in young children, affecting an estimated 5–10% of children. 1,2 These disorders typically include attention deficit hyperactivity disorder (ADHD), conduct disorder and oppositional defiant disorder,3 and are a key concern for the NHS. According to the National Institute for Health and Care Excellence (NICE),4 ≈ 30% of a typical general practitioner’s (GP’s) child consultations are for behaviour problems and 45% of community child health referrals are for behaviour disturbances. Behaviour problems are also one of the most common reasons for children to be referred to mental health services. 4,5

Behaviour problems are distinct in having one of the earliest onsets of mental health problems, with increasing research indicating that early symptoms can be identified in children aged 1 and 2 years. 6 Where problems endure over time, they can give rise to poorer educational attainment and physical health, as well as elevated risk of psychiatric disorders, substance misuse, antisocial behaviour and criminality. 2,5,7,8 In addition to the distress caused to individuals and families, there is a considerable cost to society through the health-care, social care and criminal justice systems. 9 Recent estimates put the lifetime costs of support for a child with conduct disorder at £280,000. 10

Interventions for behaviour problems

The quality of the early parental care that children experience has been causally linked to the development of behaviour problems. Where children experience low levels of sensitive parenting behaviour and greater use of harsh discipline, they are at a high risk of developing behaviour problems. 11 Consequently, these parenting behaviours, sensitivity and discipline, represent the key targets of most early intervention programmes in the UK, based on attachment theory and social learning theory, respectively. 12,13

There is an established evidence base demonstrating positive effects of parenting interventions for preschool- and school-aged children’s behaviour. An umbrella meta-analysis14 of 26 meta-analyses identified 411 studies of parenting interventions for children with externalising behaviours. The authors found that those studies that reported on externalising behaviour showed evidence of moderate positive effects. A recent meta-analysis15 of 154 randomised controlled trials (RCTs) also found a moderate positive effect of parenting interventions on children’s behaviour. The authors also reported an individual participant data meta-analysis of 13 RCTs of a specific parenting intervention, Incredible Years, which showed similar positive effects on conduct problems and ADHD symptoms. 15 Estimates of the cost savings associated with such programmes suggest that parenting interventions could provide savings of approximately £16,425 per family over 25 years. 16 As most programmes target preschool- (aged 3 or 4 years) and school-aged children, we know less about the effectiveness of programmes delivered in infancy and toddlerhood. Intervening earlier in childhood could be more effective both clinically and economically, provided that there is reliable identification of children at risk, as there is increased opportunity to intercept psychopathology before it becomes embedded and to reduce the burden of suffering experienced by families. 17

An evidence-based programme that can be delivered from aged 12 months is the Video-feedback Intervention to promote Positive Parenting and Sensitive Discipline (VIPP-SD) programme. 18,19 VIPP-SD represents a powerful combination of social learning theory and attachment theory, and targets the two key parenting behaviours: sensitivity and discipline. The goal of the intervention is to promote parents’ sensitivity (i.e. their capacity to identify their child’s attachment cues and exploratory behaviour, and respond to them appropriately) and sensitive discipline, which involves a consistent but non-harsh response to challenging behaviour. The VIPP-SD programme has been developed systematically and has been tested in 12 RCTs20–31 in a range of clinical and non-clinical populations. A recent meta-analysis32 of these trials demonstrated combined positive effects on caregivers’ sensitivity and children’s behaviour problems. However, VIPP-SD has yet to be tested in a routine health service context in the UK.

Rationale for the research

Early intervention has the potential to improve behaviour problems before they become established, which could yield cascading benefits for children’s outcomes across their life course. Indeed, early intervention has become a key priority of global and domestic policy. 33–35 However, there are few effective early psychological interventions that target behaviour problems in very young children. 36 The literature also demonstrates that no single programme or service delivery method has been shown to be a panacea for the varied challenges that families face. 37 Rather, effective early intervention for children at risk of behaviour problems is likely to require the identification of a range of efficient and repeated interventions at different developmental stages. 38 This is in keeping with global policy recommendations on the importance of staged and developmentally informed intervention across early childhood that is grounded in nurturing caregiving. 39 Successful early intervention, therefore, first requires effective programmes that can target behaviour problems at their earliest onset in children aged 1 and 2 years. The Healthy Start, Happy Start study was designed to address this gap by testing whether or not a brief parenting intervention (VIPP-SD) could be effective in preventing enduring behaviour problems in children aged 1 and 2 years in a pragmatic health service context.

Aims and objectives

The aim of the Healthy Start, Happy Start study was to evaluate the clinical effectiveness and cost-effectiveness of a brief early parenting intervention to prevent enduring behaviour problems in young children aged 12–36 months.

The objectives were to:

• undertake a RCT to evaluate whether or not a brief parenting intervention (VIPP-SD) leads to lower levels of behaviour problems in young children who are at high risk of developing these problems, compared with usual care in the NHS

• undertake an economic evaluation to assess the cost-effectiveness of the intervention compared with usual care.

Design

The Healthy Start, Happy Start study was a pragmatic, assessor-blinded, multisite, two-arm, parallel-group RCT, to test the clinical effectiveness and cost-effectiveness of a brief parenting intervention (VIPP-SD) for parents of young children (aged 12–36 months) at risk of behaviour difficulties. VIPP-SD was predominantly delivered by trained therapists through NHS health visiting teams and was compared with receiving usual care alone.

Ethics and governance

Ethics approval for the study was given by Riverside Research Ethics Committee (14/LO/2071), which is part of the NHS Research Ethics Service. The trial was registered with the Integrated Research Approval System (IRAS) under the reference number 160786 and the National Institute for Health Research (NIHR) Clinical Research Network (CRN) Portfolio under the reference number 18423. The trial was registered with the International Standard Randomised Controlled Trial Number (ISRCTN) registry as number ISRCTN58327365. Approval for the study to be conducted within participating NHS sites was obtained from the NHS Health Research Authority (HRA). Local NHS permissions were also given for each recruitment site. The trial was conducted in accordance with the Declaration of Helsinki,40 UK Policy Framework for Health and Social Care Research41 and Imperial Clinical Trials Unit (ICTU) standard operating procedures.

Participants

Participants were children aged 12–36 months who demonstrated behaviour problems and their caregiver(s).

Setting

Recruitment for the study was conducted in six UK NHS trusts: Central and North West London NHS Foundation Trust, Whittington Health NHS Trust, Oxford Health NHS Foundation Trust, North East London NHS Foundation Trust, Cambridgeshire and Peterborough NHS Foundation Trust, and Hertfordshire Community NHS Trust. Recruitment efforts were predominantly targeted across seven areas within these trusts: the London Boroughs of Camden, Hillingdon, Islington, and Barking and Dagenham, as well as Oxfordshire, Peterborough and Hertfordshire. Research assessments were carried out in participants’ homes or in another setting if the participant preferred (e.g. a private room in a children’s centre).

Recruitment procedure

There were two stages of recruitment in the study. In stage 1, potential participants in participating NHS sites were screened for behaviour difficulties using the parent-reported SDQ. 42 The principal pathway for recruitment was through health visiting services, with health visitors screening families using a study screening pack during routine 12- and 24-month child health reviews. This was supplemented by direct screening undertaken by members of the research team and CRN support staff in the waiting room of these clinics and children’s centres. Health visiting services also posted screening information to families with children within the target age range, and advertisements for the study were placed in health visiting centres and GP surgeries. Participants were also recruited through wider community services including nurseries, libraries, and through social media advertisements in local and community forums. Recruitment advertisements included the questions ‘[I]s your child’s behaviour sometimes a challenge?’ and ‘[A]re you interested in finding out more about your child’s behaviour?’.

In stage 2 of recruitment, the research team contacted caregivers by telephone if their children met the eligibility criteria based on their scores on the SDQ questionnaire (i.e. scoring in the top 20% of population norms). During this telephone call, the family’s full eligibility and interest in participating in stage 2 of the study were assessed. Researchers also enquired as to whether or not there were two parents/caregivers in the family and, if so, if both would be willing and available to participate in the study. Interested participants were sent an information pack for stage 2 of the study by e-mail or post, and a date to meet with them at their home for the baseline assessment visit was scheduled.

Informed consent

Participating parents/caregivers provided written informed consent for each stage of the study. All participant resources were ethics approved [see www.journalslibrary.nihr.ac.uk/programmes/hta/130433/#/documentation (accessed January 2021) for participant information sheets (PISs) and consent forms used through both stages of the trial, as well as template letters used to inform GPs and health visitors of families’ study participation]. In stage 1, potential participants were given screening packs containing an information leaflet and the stage 1 PIS and consent form, alongside the screening questionnaire. Parents/caregivers were able to provide written or electronic consent to complete the screening stage of the study. Consent for stage 2 of the study was not provided at this point.

Participants who were eligible for and interested in the second stage of the study, the full RCT, were sent the stage 2 PIS either by e-mail or by post following a telephone call from the research team and ahead of their scheduled baseline assessment. The PIS and the clauses of the consent form were then fully explained in person to participants by a trained researcher at the beginning of the research assessment. Participants’ right to withdraw from the trial at any point was explained. Participants were given the opportunity to ask any questions that they had.

Baseline assessment

Baseline assessments were carried out in participants’ homes (or, in a very small number of cases, in private rooms of children’s centres close to participants’ homes). At baseline, trained researchers obtained written informed consent and collected family demographic information. Parent demographic information included parents’ sex, age, ethnicity, education level, employment status and relationship status. Child demographic data included children’s sex, age and ethnicity. Measures administered during the baseline assessment can be seen in Table 1.

The baseline visits took between 2 and 2.5 hours to complete (depending on the number of caregivers participating). All baseline assessments were conducted between July 2015 and July 2017.

Follow-up

Follow-up visits took place at two time points. The first was a post-treatment assessment (i.e. after the VIPP-SD intervention for those in the treatment group), approximately 5 months post randomisation, whereas the second was scheduled for 24 months post randomisation, when the children were aged 3–5 years. These assessment points were timed to allow for an initial post-intervention assessment when treatment would have usually been completed (5-month assessment) and to assess for longer-term outcomes (24 months). To assess whether or not the intervention had clinically significant effects on child behaviour problems, the measures of child behaviour collected at baseline were also administered at both follow-up visits. Similarly, measures of self-reported parenting practices, as well as parent mental health measures, and couple functioning, where appropriate, were also repeated at follow-up time points. Families were also asked to complete a structured interview of service use at both follow-up time points to assess cost-effectiveness. Administration of all trial measures is outlined in Table 1.

The follow-up visits took between 2 and 2.5 hours to complete. All follow-up data were collected between December 2015 and July 2019. Figure 1 shows the Consolidated Standards of Reporting Trials (CONSORT) flow diagram outlining participant recruitment and retention during the study.

FIGURE 1.

The CONSORT flow diagram for the Healthy Start, Happy Start RCT.

Intervention

Video-feedback Intervention to promote Positive Parenting and Sensitive Discipline is a manualised, home-based intervention developed by Juffer, Bakermans-Kranenburg and van IJzendoorn and based on attachment theory and social learning theory. 18 The goal of the intervention is to promote parents’ sensitivity (i.e. their capacity to identify their child’s attachment cues and exploratory behaviour, and respond to them appropriately), as well as sensitive discipline, which involves a consistent but non-harsh response to challenging behaviour. VIPP-SD was delivered by trained therapists over six sessions, lasting 1–2 hours, at approximately fortnightly intervals. The role of the therapist was to develop a trusting and empathetic relationship with the parent(s) and to deliver the six sessions (four core sessions and two booster sessions) in line with the manual.

The first half of each session involved the therapist filming the parent with their child during interactions that aimed to reflect everyday situations, such as reading a book together, playing with toys and a mealtime. Following the session, the therapist used the video-recorded interactions to write a feedback script based on an appraisal of the parent’s interaction profile needs, the guidelines of the protocol, and the sensitivity and discipline themes for that session. The parent and therapist spent the second half of each session reviewing the interactions filmed in the previous visit, with the therapist frequently pausing the video to deliver feedback based on the script they had prepared following the last session. The content and focus of each session were manualised, but the feedback was delivered in an individualised way, as feedback is unique to each parent and child in that moment. A modified version of VIPP-SD, Video-feedback Intervention to promote Positive Parenting for Co-parents (VIPP-Co),43 was delivered when two caregivers in the family were participating. The content and themes of the VIPP-Co intervention broadly mirrored the VIPP-SD manual, with additional emphasis on interactions involving both caregivers together with the child and on positive co-parenting. In the first three sessions, feedback was delivered separately to each caregiver (as in VIPP-SD) and, from session 4 onwards, feedback was delivered to both caregivers together. The intervention was delivered by 40 trained therapists, the majority of whom were health visitors, community nursery nurses and clinical psychologists, as well as a small number of professionals from other backgrounds, such as family and child therapy, psychology and psychiatry.

Fidelity

Each therapist delivering the intervention undertook 4 days of VIPP-SD training, as well as an additional day of training on delivering the intervention to two caregivers and on the study protocol. Therapists undertook supervised clinical practice with a practice case before becoming a therapist on the trial and received regular clinical supervision throughout the trial. Therapists audio-recorded the feedback of sessions and reported on fidelity in terms of the delivery of key components of the treatment, as well as reporting on global adherence to the manual. We randomly selected 10% of audio-recordings to allow two assessors trained in the intervention to rate the quality of sessions and adherence to the VIPP-SD manual.

Usual care

Participants in both groups continued to receive their usual care, which was minimal in most cases (there are no standard care pathways in the NHS for early-onset behaviour problems). Some participants received support and advice from a health visitor or GP, referral to early intervention mental health services linked to a children’s centre, or parenting advice and support sessions. Data were collected on concurrent use of health and social care services and are presented in Chapter 4.

Outcome measures

Sample size

A sample size of 300 children and their caregiver/parent(s) was selected to provide between 80% and 90% power to detect standardised effect sizes of 0.36 and 0.42, respectively, on the primary outcome, at a 5% level of statistical significance, and assuming a 20% attrition rate. The analysis was adjusted for baseline behaviour score, time from randomisation to follow-up, recruitment site, age of child at recruitment, and caregiver involvement (one vs. two participating caregivers) as fixed effects. This is likely to have increased power to > 90% as such adjustment, especially for baseline scores of the same variable, will have reduced the residual error variance in our model.

Randomisation and blinding

The randomisation list was prepared by an independent statistician using block sizes of two, four and six (varying at random) and 1 : 1 allocation to either VIPP-SD or usual care. The randomisation list was uploaded to the study electronic data capture system before the trial commenced. Randomisation was performed using a web-based randomisation system linked to the trial database following enrolment by the researcher who conducted the baseline assessment and who was blind to all treatment allocations. Eligible participants were allocated online to the next available treatment code in the appropriate stratum. Randomisation was stratified by recruitment site and the number of caregivers participating (one vs. two). Access to the allocation sequence was restricted to an independent statistician and appropriate members of the InForm™ (version 4.6) technical support team (Imperial College London, London, UK) to maintain allocation concealment.

Following randomisation, the trial manager informed participants of their allocated groups and matched participants to therapists for treatment according to the availability of both the therapists and the parents. All assessors remained blind to treatment allocation for the duration of the study. Participants were reminded of the importance of researchers remaining blinded to group allocation by the trial manager or administrative assistant at each contact when scheduling their follow-up assessments. Researchers also reminded parents of the importance of blinding in person at the start of the follow-up visits to ensure that researchers stayed blinded to group allocation during the assessment. In cases where unblinding did occur, the primary outcome (PPACS) was re-scored by a blinded trained assessor.

Data collection and data management

The trial’s source data included paper forms, questionnaires, written interview notes, scoring from researchers’ research assessments, and feedback notes and log books from trial therapists’ intervention sessions. Source data were collected and stored securely in accordance with the International Council for Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use (ICH) good clinical practice (GCP),61 the Data Protection Act62 and the UK Policy Framework for Health and Social Care Research. 41 Following the assessments/intervention visits, data were entered on an electronic Case Report Form (eCRF) developed in InForm™, an electronic data capture system built around an Oracle database. The InForm system includes automated range checks and validation rules for data entry to help ensure data accuracy. A computer-generated audit trail is in place that records the date, time, operator, operation and previous value of all manipulation of clinical data.

InForm storage and management were undertaken by the Imperial College Information and Communication Technologies (ICT) team. InForm sits on a server behind a firewall connected to the College Storage Area Network (SAN). The data are backed up regularly to removable media, which allows for disaster recovery. In addition to the College backup facility, every 20 minutes, the activity logs for the trial are moved to another server in a different location to facilitate rapid recovery of data should it become necessary (e.g. in a disaster recovery scenario). Access to the system was restricted to trained staff with unique password-protected accounts. Identifiable data were not recorded in the eCRF and participants were identified by a unique trial identifier only.

All data monitoring and cleaning were also completed on InForm. Predefined data ranges were included in the eCRF, which raised automated queries if data outside the expected range were entered. In addition to the automated queries, the trial data were reviewed on a regular basis by the trial manager and trial statistician for discrepancies and errors. Data cleaning was performed prior to each Data Monitoring and Ethics Committee (DMEC) meeting and prior to database lock. Final data checks were performed by the statistician once the database had been soft locked and before hard lock was complete. All outstanding queries were resolved prior to the database hard lock.

Quality assurance and monitoring

Quality assurance (QA) and monitoring were performed in accordance with ICTU standard operating procedures (SOPs). A risk assessment was conducted by the ICTU QA manager prior to the start of the study, which assigned a risk category of ‘low risk’ to the trial. A monitoring plan was developed in accordance with the risk assessment to define how monitoring procedures would be carried out and the level of source data verification required for the study.

In accordance with the monitoring plan, 100% of participant consent forms and SAEs were source-verified. Further to this, 20% of key trial data were also source-verified against the original paper records, which included the primary outcome (PPACS); two of the secondary outcomes (i.e. CBCL and SDQ); and 10% of demographics, therapist logbooks and all other outcomes (Parenting Scale, PHQ-9, GAD-7, RDAS and CA-SUS).

Discrepancies noted between the paper records and trial database were queried and corrected in the trial database. Monitoring was carried out centrally by the research team, ensuring that the member of staff carrying out source data verification was not the same person who had performed data entry in each case. Participant data to be monitored were identified by the trial statistician through random selection.

Data analysis

A comprehensive statistical analysis plan was produced before undertaking the analysis and agreed with the Project Management Group (PMG), Trial Steering Committee (TSC) and DMEC. Primary analyses were conducted on an intention to treat (ITT) basis.

Multiple linear regression was used to assess the primary outcome measure, the difference in PPACS score between the trial groups at 5 and 24 months post randomisation. The PPACS score at the 5-month follow-up was the dependent variable, whereas trial group, PPACS score at baseline, time since randomisation, recruitment centre, age of the child at recruitment and number of parents/caregivers participating (one or two) were included as independent variables. Separate models were fitted at 24 months (adjusting for the baseline measurements, but not the 5-month measurements). These were distinct from the models fitted at 5 months (which, again, were adjusted for baseline measurements). Similar models, with adjustment for the appropriate baseline score, were used to assess the difference in CBCL and SDQ scores between the trial groups at the 5-month follow-up.

Regression residuals were plotted to assess their fit to the normal distribution using probability plots. PPACS scores (total and subscales) were normally distributed at 5 months and close to normal at 24 months. Inspection of residuals plots suggested assumption of homoscedasticity and of linearity were also reasonably well met. Visual inspection of the residuals from the regression analyses of secondary outcomes at 5 and 24 months suggested that they were close to normally distributed. Repeating the regression models using bootstrap methods (that allow for departure from normality) produced very similar results. For consistency, the results based on linear regression only are presented.

Secondary outcomes that were completed by each participating caregiver were analysed separately by sex. Standardised effect sizes (also known as d) were calculated by dividing the differences in means by the standard deviation in the usual-care group at follow-up.

Missing answers to individual questions in the PPACS were imputed. Unrateable or missing items in the attention/hyperactivity scale occurred most often when children did not regularly undertake an activity required for scoring (e.g. watching television or playing with a sibling/friend on the same activity). This assumption of ‘missing at random’ inherent in the multiple imputation analysis seems reasonable here. This implies assessing attention based on the activities that the child regularly undertakes (e.g. playing alone and eating a meal), by imputing values for activities that the child does not regularly undertake (e.g. watching television and/or playing with a friend or sibling). There were negligible missing data in the conduct disorder scale. The level, pattern and likely causes of any missingness in the baseline variables and primary outcome were investigated. Multiple imputation was of individual missing items in the PPACS interview when it was partially completed (because this always implied that at least 50% of items in the scale had been completed). Multiple imputation of the whole score was performed if the entire scale was missing. At the 5- and 24-month follow-ups, imputation was based on randomised group, sex of child, age of child at the 5- and 24-month assessments and baseline score, utilising information from both the baseline and 5- and 24-month PPACS interviews. Imputation at baseline was based on randomised group, sex of child, age of child at baseline and information from the baseline PPACS questionnaire.

Missing items in the other secondary outcome scales (child behaviour as assessed by the CBCL and SDQ; and self-reported parental discipline behaviour, mood, anxiety and couple functioning) were dealt with by scaling up. Thus, the sum of the maximum possible scores for non-missing items in each scale was divided by the maximum possible total score for a fully completed scale to give the proportion of the scale that had been completed (inflation factor). Scaling up was achieved by dividing the observed total score for each scale by the inflation factor. The proportion of missing items was generally negligible. See Appendix 2 for additional information on multiple imputation.

Analysis of the primary outcome was repeated without adjustment for time since randomisation and using complete cases after multiple imputation of missing items only. Analysis was also repeated assuming that those who did not have PPACS follow-up data available had better than anticipated results and then worse than anticipated results (anticipated by multiple imputation, better and worse by one standard deviation of the difference between PPACS scores at follow-up and at baseline). In further analyses, unrateable items within PPACS were filled in as the best possible scores and then as the worst possible scores. Analysis of primary and secondary child behaviour outcomes was repeated for each domain separately. Complier-average causal effects (CACE) analysis was performed, using two-stage least squares regression analysis, to determine the effect of receiving the intervention, rather than just being randomised to receive it. Compliance was defined as receipt of the four core VIPP-SD visits.

Planned subgroup analyses of the primary outcome were of the effects of child age at baseline (12–23 months compared with 24–36 months) and of the number of parents participating (one vs. two). We also undertook an ad hoc subgroup analysis to assess the effect of quartiles of severity of child behaviour problems at baseline and of fidelity of the intervention in those randomised to VIPP-SD. The magnitude of the treatment effect was estimated by repeating analysis of the primary outcome for each subgroup separately (and for each fidelity group compared with all patients allocated to the usual-care group).

All analyses were conducted using Stata^® (StataCorp LP, College Station, TX, USA) software (versions 13.0 and 15.0).

Trial organisation and management

Patient and public involvement group

Our PPI group was composed of seven representatives and included members of the pilot study, as well as those identified through community and health-care services. Group members were mothers and fathers of young children and/or educational/child care professionals. The PPI group contributed to the study’s progress throughout the trial and provided significant and important input on the design and management of the study. The group met annually from the start of the trial in 2015 until the end of the trial in 2019, with an average of four members attending each meeting.

Public involvement was a key element in the study’s success, with the PPI group providing ideas, advice and feedback that were integral to the study’s recruitment processes, participant retention, data collection procedures and dissemination of study findings. Specifically, the group was able to advise and offer feedback on the optimal settings and services needed to be targeted for trial recruitment, especially for populations, such as fathers, that previous studies have often struggled to engage and accommodate. The PPI group also helped to produce and develop a brief, informative video for use in recruitment purposes. The video outlined what participation in the trial would involve and was provided to families alongside the PIS. A study within a trial (SWAT) aimed to explore whether or not the addition of this video aided the consent process.

A key study success was sustaining high levels of participant retention across the 2-year follow-up period (95% and 94% retention at the 5- and 24-month follow-ups, respectively). The PPI group was, again, integral to this process and advised on the key engagement strategies implemented by the study team. These included the creation of a study website for caregivers to visit, a welcome pack for families, the use of certificates and visit trackers for the children following each assessment, the sending of birthday cards to the children, a thank-you toy to acknowledge children’s participation, and the sharing of sample clips from the visits as a keepsake. Regular newsletters sent to participating families by e-mail were also a key initiative of the PPI process. The newsletters provided an update on the study, informed participants about what was coming next and constituted a pathway for families to easily update the study team with any changes in contact details. This meant that the study team was able to communicate more efficiently and effectively with families when it came to arranging follow-up assessments. The PPI group was also involved in decisions around how to communicate and explain the study findings to participants. The group recommended considering a range of different dissemination materials, including an animation, a brief newsletter, an infographic, and an example story to illustrate the study and its outcomes in a more personal way.

The PPI members were highly engaged and motivated throughout the project. At the start of each meeting, a study summary was provided, as well as a recap on how the group’s suggestions and feedback (from the previous meeting) had been implemented in the wider project. Two members of the PPI group were also full members of the TSC and participated in the piloting of baseline and follow-up assessments. All PPI members were reimbursed for incurred travel expenses and were offered a voucher following each meeting in keeping with the INVOLVE guidelines.

Amendments to protocol

• We changed the eligibility criteria so that participants who had previously received individualised video-feedback programmes were no longer excluded. We also extended the eligibility criteria to exclude parents who had a sensory impairment or learning disability that precluded their participation in the trial.

• Timing of the primary outcome was extended from 4 to 5 months post randomisation because of scheduling difficulties in completing the treatment in the allocated time for the intervention group of the trial. This was to reduce the likelihood of systematic differences in the timing of the follow-up assessments between the trial groups.

• Reporting of outcomes in the current report is confined to those that were funded as part of this project and were included in the prespecified statistical analysis plan. Following an ethics amendment, we amended the protocol to include one secondary and five exploratory outcomes. As a secondary outcome, parent–child interaction data were collected across all three time points to allow for the measurement of parental sensitivity, which was a target of the intervention. This measure may be instructive in better understanding the mechanisms of this intervention. Exploratory outcomes were also added at the 24-month time point. These measures included direct assessment of children’s executive functions, emotion regulation, delay of gratification and narrative representations during dolls house play, as well as genetic variance (through buccal swabs) and parental involvement. Analyses of these variables is yet to be undertaken and findings will be published once available.

Participant flow and retention

Between 30 July 2015 and 26 July 2017, a total of 2248 families were assessed for eligibility. Of these families, 1430 were ineligible, 518 did not progress to the trial (declined/could not be contacted) and 300 were randomised to stage 2 of the trial. In total, 151 participants (50%) were randomly allocated to the VIPP-SD group and 149 participants (50%) to the usual-care group. We recruited to target (n = 300; see Appendix 3 for the study’s recruitment graph), with the majority of participants recruited from health visiting services (30% face to face and 25% through posting screening information to families with children in the target age range) and children’s centres (30%). The remaining participants were recruited largely from other community venues and online adverts (Table 2 shows a full outline of recruitment pathways). Participants were recruited across seven NHS sites (Table 3). Follow-up data were collected between December 2015 and July 2019. Follow-up data were analysed for 286 (VIPP-SD, n = 140; usual care, n = 146) participants and 282 (VIPP-SD, n = 140; usual care, n = 142) participants at 5 and 24 months, respectively. See Figure 1 for the CONSORT flow diagram summarising the eligibility, trial group allocation and subsequent progress of randomised participants. Approximately one-third of children who were screened were eligible to take part in the study, which was higher than the anticipated 20%. It is possible that families with children with elevated problems self-selected into the study and/or that health visitors were more likely to complete screening with families who they thought may be more likely to be eligible based on the child’s behaviour.

Baseline participant characteristics

Participant characteristics at baseline were well balanced between groups. Table 4 shows the baseline characteristics of participating children and primary caregivers. The participating children had a mean age of 23 months [standard deviation (SD) 6.7 months]; 54% were male and 65% were recorded as being of white ethnicity. Participating primary caregivers were predominantly female (96%), with a mean age of 34.2 years (SD 5.8 years). Most primary caregivers were married or cohabiting (85%), were of white ethnicity (72%), had a graduate-level qualification (64%) and were in some form of employment or on paid parental leave (59%). There were more male children in the usual-care group (58%) than in the VIPP-SD group (50%), and the usual-care group was also more diverse in terms of primary caregiver and children’s ethnicity. Appendix 4, Table 24, shows the baseline characteristics of participating secondary caregivers.

Delivery of intervention

Primary outcome: Preschool Parental Account of Children’s Symptoms score at the 5- and 24-month follow-ups

We present the primary outcome (total PPACS score) and its subscale scores at the 5-month (primary time point) and 24-month follow-ups in Table 6. The group mean difference in PPACS score was modelled using linear regression adjusted for baseline scores, treatment centre, randomised group, length of follow-up, age of the child and the number of participating caregivers. At the post-treatment 5-month follow-up, the mean scores of child behaviour problems on the PPACS were found to have decreased from 33.5 (SD 9.0) to 28.8 (SD 9.2) in the VIPP-SD group and from 32.4 (SD 10.6) to 30.3 (SD 9.9) in the usual-care group. This represents a significant difference in reduction of problems favouring the VIPP-SD group [adjusted mean difference, VIPP-SD vs. usual care 2.03, 95% confidence interval (CI) 0.06 to 4.01; p = 0.04]. The adjusted standardised effect size was Cohen’s d = 0.20 (95% CI 0.01 to 0.40). VIPP-SD was also found to be superior to usual care, in particular, on the conduct problems subscale of the primary outcome [adjusted mean difference 1.61, 95% CI 0.44 to 2.78; p = 0.007 (d = 0.30, 95% CI 0.08 to 0.51)], but not the hyperactivity subscale [adjusted mean difference 0.29, 95% CI –1.06 to 1.65; p = 0.67 (d = 0.05, 95% CI –0.17 to 0.27)].

At the 24-month follow-up, the adjusted mean difference on the total PPACS score remained about the same, with a decrease in effect size of d = 0.03 [VIPP-SD vs. usual care 1.73, 95% CI –0.24 to 3.71; p = 0.08 (d = 0.17, 95% CI –0.02 to 0.37)]. The adjusted mean difference on the conduct subscale continued to favour the VIPP-SD group [VIPP-SD vs. usual care 1.07, 95% CI –0.06 to 2.2; p = 0.06 (d = 0.20, 95% CI –0.01 to 0.42)]. There was no evidence of the superiority of VIPP-SD over usual care on the hyperactivity subscale [difference 0.62, 95% CI –0.60 to 1.84; p = 0.32 (d = 0.10, 95% CI –0.10 to 0.30)].

The positive effect of VIPP-SD on the PPACS total score at the 5-month follow-up was robust to sensitivity analyses, which estimated the impact of varying the assumptions made in respect of missing data and adjustment for length of follow-up (see Appendix 5, Table 28). Figures 2 and 3 present the forest plots for the sensitivity analyses of the 5-month and 24-month follow-up PPACS scores, respectively. Appendix 5 details the profiles of families with and without missing PPACS data (see Appendix 5, Table 29) and reports descriptive outcome data for those with and without PPACS scores at the 5- and 24-month follow-ups (see Appendix 5, Table 30).

FIGURE 2.

Forest plot for sensitivity analysis at the 5-month follow-up. adj, adjusted; f/u, follow-up.

FIGURE 3.

Forest plot for sensitivity analysis at the 24-month follow-up. adj, adjusted; f/u, follow-up.

The primary analyses included multiple imputation (MI) for unrateable items on the PPACS scales, MI for PPACS total scores in those who did not complete their 5-month follow-up assessment and adjustment for length of follow-up. Sensitivity analyses included lack of adjustment for length of follow-up (combined with other assumptions). Complete-case analyses are then reported (using MI solely for unrateable items on the PPACS scale and excluding those who did not complete their 5-month follow-up). Further analyses assume better PPACS scores and then worse PPACS scores (than predicted by MI by one standard deviation of the mean change in PPACS scores between the baseline and 5-month follow-up assessment) in those who did not complete their 5-month follow-up. The final analyses assume that unrateable PPACS items were replaced with the highest possible score and then the lowest possible score.

Sensitivity analyses were completed at the 24-month follow-up in the same way as in the 5-month follow-up sensitivity analyses. This included lack of adjustment for length of follow-up (combined with other assumptions). Complete-case analyses are then reported (using MI solely for unrateable items on the PPACS scale and excluding those who did not complete their 24-month follow-up). Further analyses assume better PPACS scores and then worse PPACS scores (than predicted by MI by one SD of the mean change in PPACS scores between the baseline and 24-month follow-up assessment) in those who did not complete their 24-month follow-up. The final analyses assume that unrateable PPACS items were replaced with the highest possible score and then the lowest possible score.

Secondary outcomes: parent-reported measures of child behaviour at the 5- and 24-month follow-ups

On the key secondary outcomes, parent-reported questionnaire measures of children’s behaviour (total scores of the CBCL and SDQ), the results indicate a positive direction of effect favouring the VIPP-SD group at the 5-month follow-up, yet indicate little evidence of a sustained effect at the 24-month follow-up (Table 7). At the 5-month follow-up, the VIPP-SD group had lower CBCL scores [adjusted mean difference 3.24, 95% CI –0.06 to 6.54; p = 0.05 (d = 0.15, 95% CI 0.00 to 0.31)] and SDQ scores [adjusted mean difference 0.93, 95% CI –0.03 to 1.90; p = 0.06 (d = 0.18, 95% CI –0.01 to 0.36)] than the usual-care group. At the 24-month follow-up, although the VIPP-SD group continued to have lower behaviour scores than the usual-care group, these differences had diminished somewhat and there was little evidence of sustained effects of VIPP-SD on the CBCL (adjusted mean difference 2.82, 95% CI –1.82 to 7.45; p = 0.23) and SDQ (adjusted mean difference 0.35, 95% CI –0.78 to 1.47; p = 0.54). There was no evidence of superiority of VIPP-SD over usual care on the teacher-reported total score for the SDQ at the 24-month follow-up (difference 0.54, 95% CI –1.00 to 2.08; p = 0.49).

Table 8 shows the adjusted group differences on the subscale scores for the secondary behaviour outcomes (CBCL and SDQ) at the 5- and 24-month follow-ups. At the 5-month follow-up, there was evidence of an effect favouring the VIPP-SD group over the usual-care group on the SDQ externalising subscale [adjusted mean difference 0.68, 95% CI 0.00 to 1.37; p = 0.05 (d = 0.18, 95% CI 0.00 to 0.37)] as well as weaker evidence favouring the VIPP-SD group over the usual-care group on the CBCL internalising scale [adjusted mean difference 1.03, 95% CI –0.10 to 2.16; p = 0.07 (d = 0.15, 95% CI –0.01 to 0.32)]. There was no strong evidence of group differences on the remaining subscales. In general, differences on the subscales had attenuated at the 24-month follow-up and there was little evidence of sustained effects on these measures. On the teacher-reported SDQ, although teachers tend to report lower behaviour problems for the VIPP-SD group than for the usual-care group on key subscales, there was only weak evidence of group differences on these subscales: conduct [adjusted mean difference 0.38, 95% CI –0.16 to 0.92; p = 0.17 (d = 0.18, 95% CI –0.08 to 0.45)], hyperactivity [adjusted mean difference 0.46, 95% CI –0.25 to 1.16; p = 0.20 (d = 0.17, 95% CI –0.09 to 0.43)] and externalising [adjusted mean difference 0.81, 95% CI –0.29 to 1.92; p = 0.15 (d = 0.19, 95% CI –0.07 to 0.45)] subscales.

Table 9 shows the results of the secondary CACE analysis estimating the effects of receiving the intervention (defined as at least four VIPP-SD sessions), rather than merely being randomised to it. The CACE analysis was completed on the primary and secondary behaviour outcome measures (PPACS, CBCL and SDQ). For the PPACS scores, the results showed a larger adjusted mean difference favouring the VIPP-SD group at the 5-month follow-up when accounting for treatment adherence [difference 2.59, 95% CI 0.24 to 4.94; p = 0.03 (d = 0.26, 95% CI 0.02 to 0.50)] than in the ITT analysis [difference 2.03, 95% CI 0.06 to 4.01; p = 0.04 (d = 0.20, 95% CI 0.01 to 0.40)]. At the 24-month follow-up, the difference remains in the same range, similar to the outcome in the ITT analysis, although the adjusted group difference and effect size tends to be larger in the CACE analysis [difference 1.96, 95% CI –0.30 to 4.23; p = 0.09 (d = 0.20, 95% CI –0.03 to 0.43)]. A similar pattern is observed on the CBCL [adjusted mean difference 3.56, 95% CI 0.04 to 7.09; p = 0.05 (d = 0.17, 95% CI 0.00 to 0.34)] and SDQ [adjusted mean difference 1.03, 95% CI –0.01 to 2.06; p = 0.05 (d = 0.20, 95% CI 0.00 to 0.39)] at the 5-month follow-up, such that both outcomes show slightly larger adjusted mean differences and effect sizes, and stronger evidence of group differences in the CACE analysis than in the ITT analysis. As with the ITT analysis, there is little evidence of a sustained group difference on the secondary behaviour outcomes (CBCL and SDQ) at the 24-month follow-up in the CACE analysis. Tables 25–27 in Appendix 4 show the analysis of the secondary behaviour outcomes as reported by secondary participating caregivers.

Table 10 shows the ITT analysis for secondary outcomes relating to participating caregivers’ reported parenting practices, mood, anxiety and couple functioning, disaggregated by sex of the reporting caregiver. There was no evidence of group differences on these secondary outcomes (p-values = 0.15–0.95).

Figures 4 and 5 show exploratory subgroup analyses that disaggregate the adjusted mean difference between VIPP-SD and usual care on the basis of the child’s age at baseline, the number of participating caregivers, the severity of baseline SDQ score (by quartile) and the fidelity of the therapist. At the 5-month follow-up, the analysis suggested a greater positive response to treatment in children aged 12–23 months (adjusted mean difference 2.91, 95% CI 0.06 to 5.76), in families with one participating caregiver (adjusted mean difference 2.79, 95% CI 0.63 to 4.94), those with higher baseline behaviour scores (as indicated by total scores in the third quartile: adjusted mean difference 6.42, 95% CI 1.61 to 11.23; and fourth quartile: adjusted mean difference 3.48, 95% CI 0.03 to 6.93; of the baseline SDQ), and those who had received the programme from a therapist rated high (adjusted mean difference 3.47, 95% CI 0.83 to 6.12), but not the highest, on fidelity. At the 24-month follow-up, larger effects remained in the families with one participating caregiver (adjusted mean difference 2.19, 95% CI 0.00 to 4.39) and those who had received the programme from a therapist rated with high (adjusted mean difference 3.51, 95% CI 0.88 to 6.14), but not the highest, fidelity scores. Appendix 6 shows the subgroup analysis for baseline behaviour based on quartiles of externalising scores on the SDQ at the 5- and 24-month follow-ups (see Appendix 6, Figures 13 and 14).

FIGURE 4.

Forest plot for exploratory subgroup analyses at the 5-month follow-up.

FIGURE 5.

Forest plot for exploratory subgroup analyses at the 24-month follow-up.

The age of a child at baseline and the number of participating caregivers were predefined subgroup analyses. Analyses by quartile of SDQ (at baseline) and (in those randomised to VIPP-SD) by fidelity of the intervention (compared with those randomised to usual care) are post hoc. Fidelity is independently assessed, based on the fidelity of the therapist; ‘fidelity zero’ represents a family who completed fewer than four VIPP-SD visits, despite being randomised to VIPP-SD.

Missing data

Eleven participants (7%) allocated to the VIPP-SD group and three participants (2%) allocated to the usual-care group were not assessed for the primary outcome measure (PPACS) at the 5-month follow-up. Eleven participants (7%) allocated to the VIPP-SD group and seven participants (5%) allocated to the usual-care group were not assessed for the primary outcome measure (PPACS) at the 24-month follow-up. Baseline characteristics of participants with and without missing primary outcome data are given in Appendix 5, Table 29.

Blinding

Outcome assessors reported having been unblinded for 11 participants (4%) [seven (5%) in the VIPP-SD group and four (3%) in the usual-care group] as a result of participants informing assessors of their treatment allocation at their 5-month follow-up visit. At the 24-month follow-up visit, five participants (2%) [three in the VIPP-SD group and two in the usual-care group] informed assessors of their allocation. In instances where unblinding occurred before the primary outcome had been collected, the audio-recording of the interview was double-scored by a second assessor, who remained blinded to allocation, to minimise the risk of bias. Assessors who were unblinded at the 5-month follow-up assessment did not collect primary outcome data from the same family at the 24-month follow-up visit.

Acceptability

Participants allocated to the VIPP-SD group were invited to complete a feedback questionnaire at the end of treatment. Of the 139 participants who received at least one therapy session, 39 (28%) returned the feedback questionnaire. Seven families included two participating caregivers who each completed a questionnaire, giving a total of 46 respondents. Table 11 shows that approximately two-thirds to three-quarters of respondents endorsed the programme’s positive impact on their relationship with their child; their understanding of their child’s thoughts, feelings, behaviours, and likes and dislikes; as well as their reactions to their child’s behaviour and communication. All respondents reported that their preferred setting for the programme was in the home. The main reasons endorsed by participants were that the home setting removed the need for travel and reflected everyday interactions (Table 12).

Serious adverse events

During the study, there were three children (2%) admitted to hospital in the VIPP-SD group by the 5-month follow-up and 10 children (7%) were admitted by the 24-month follow-up. In the usual-care group, there were four children (2%) admitted to hospital by the 5-month follow-up and eight children (5%) by the 24-month follow-up. All admissions reasons were unrelated to treatment and study procedures (e.g. accidents and respiratory infections). There was no evidence of group differences and no other adverse events were reported. Protocol deviations and violations are detailed in Appendix 7.

Aim

The aim of the economic evaluation was to assess the cost-effectiveness of VIPP-SD compared with usual care at 24 months post randomisation.

Methods

Results

Outcomes

Table 20 summarises the PPACS score outcomes at baseline, and at the 5- and 24-month follow-ups used in the cost-effectiveness analysis. The PPACS scores were poorer in the VIPP-SD group than in the usual-care group at baseline (mean 33.61 for VIPP-SD vs. mean 32.27 for usual care), but improved and were better in the VIPP-SD group at 5 months (mean 28.79 for VIPP-SD vs. mean 30.28 for usual care) and 24 months (mean 23.12 for VIPP-SD vs. mean 24.68 for usual care). As outlined in the clinical results, these differences were statistically significant at 5 months (adjusted mean difference –2.55, 95% CI –4.70 to –0.39) and similar but not significant at 24 months (adjusted mean difference –1.87, 95% CI –4.10 to 0.37). Results from the clinical analysis of PPACS scores (see Table 6) are very similar to the economic analysis, which used a separate imputation protocol combining both outcomes and costs.

TABLE 20 - Mean PPACS scores per participant applied to the cost-effectiveness analysis

SE, standard error.

Lower scores indicate better outcomes.

Comparison based on multiply imputed data.

Adjusted for recruitment centre, age of child at recruitment, number of caregivers participating, time to follow-up and baseline score.

PPACSa time point	Trial group				VIPP-SD minus usual careb
	VIPP-SD		Usual care
	n	Mean (SE)	n	Mean (SE)	Unadjusted difference	Adjusted differencec	95% CI	p-value
Baseline	151	33.61 (1.15)	149	32.27 (0.81)	1.25	0.98	–1.58 to 3.55	0.450
5 months	140	28.79 (1.14)	146	30.28 (0.80)	–1.62	–2.55	–4.70 to –0.39	0.021
24 months	140	23.12 (1.08)	142	24.68 (0.76)	–1.48	–1.87	–4.10 to 0.37	0.102

Cost-effectiveness analysis using Preschool Parental Account of Children’s Symptoms at 24-month follow-up

The results of the primary cost-effectiveness analysis using PPACS as the outcome of interest suggest that VIPP-SD is more costly and more effective than usual care. Figure 6 shows the scatterplot of bootstrapped mean differences in costs and PPACS score. The majority of scatter points (98%) lie in the north-east quadrant, where VIPP-SD is more costly and more effective, whereas the remaining scatter points (2%) lie in the north-west quadrant, where VIPP-SD is more costly and less effective.

FIGURE 6.

Bootstrapped mean differences in costs and PPACS scores at the 24-month follow-up. NE, north-east (more costly, more effective); NW, north-west (more costly, less effective); SE, south-east (less costly, more effective); SW, south-west (less costly, less effective).

The CEAC for the primary cost-effectiveness analysis (Figure 7) indicates that VIPP-SD has a higher probability of being cost-effective than usual care at WTP thresholds > £800 for a 1-point improvement in PPACS score on a scale of 0–70 points (equivalent to 0.10 SD based on usual-care group SD post treatment). However, it is not possible to make firm conclusions about the cost-effectiveness of VIPP-SD without agreed thresholds for society’s WTP for improvements in PPACS score.

FIGURE 7.

Cost-effectiveness acceptability curve showing the probability that VIPP-SD is cost-effective compared with usual care at different values of WTP thresholds for a 1-point improvement in PPACS score at the 24-month follow-up. The dashed line represents the point at which a probability of 0.5 (50%) is reached.

Sensitivity analyses

The results of the two sensitivity analyses – (1) examining the effect of multiple imputation and (2) examining the results at the 5-month follow-up – are presented in Table 21. The distribution of bootstrap iterations across the cost-effectiveness plane changes for both analyses compared with base case, but, in both cases, the majority of scatter points fall in the north-east quadrant, indicating that the VIPP-SD group has better outcomes but also higher costs than the usual-care group.

TABLE 21 - Mean total cost (£) and outcomes per participant for sensitivity analyses

NE, north-east (more costly, more effective); NW, north-west (more costly, less effective); SE, south-east (less costly, more effective); SW, south-west (less costly, less effective).

Adjusted for recruitment centre, age of child at recruitment, number of caregivers participating, time to follow-up and baseline score.

Comparison based on multiply imputed data.

Lower scores indicate better outcomes.

Excluding influential outliers, i.e. participants with total costs in the 99th percentile.

Analysis	Trial group, mean (SE)		VIPP-SD minus usual care		Distribution of bootstrap iterations (%)
	VIPP-SD	Usual care	Adjusted differencea	95% CI	NW	NE	SW	SE
Base case	n = 140	n = 142
Total cost (£)	3131.93 (416.54)	1525.38 (293.49)	1449.99b	619.45 to 2280.52b	2	98	0	0
PPACS scorec	23.12 (1.08)	24.68 (0.76)	–1.87b	–4.10 to 0.37b
Complete cases	n = 137d	n = 142
Total cost (£)	2638.89 (231.84)	1525.38 (162.46)	1175.17	730.42 to 1619.92	7	93	0	0
PPACS scorec	23.14 (1.10)	24.68 (0.78)	–1.85	–3.85 to 0.15
5-month follow-up	n = 128	n = 124
Total cost (£)	1827.55 (120.71)	510.18 (85.35)	1281.34b	1022.40 to 1540.29b	0	100	0	0
PPACS scorec	28.68 (1.22)	30.34 (0.87)	–2.55b	–4.70 to –0.39b

In the sensitivity analysis considering complete cases and excluding influential outliers, the adjusted difference in mean costs is lower (£1175.17) than in the base-case analysis (£1449.99); meanwhile, the PPACS scores remain relatively unchanged. In the sensitivity analysis examining results at the 5-month follow-up, the adjusted mean difference is greater for both the costs (£1281.34) and PPACS score (–2.55) than in the base-case analysis (£1449.99 and –1.87, respectively).

The cost-effectiveness results of the sensitivity analysis considering complete cases and excluding influential outliers were very similar to those of the base-case analysis. The scatterplot of bootstrapped mean differences and PPACS scores for this analysis (Figure 8) shows that the majority of scatter points (93%) lie in the north-east quadrant (i.e. VIPP-SD is more costly and more effective than usual care) whereas the remaining scatter points (7%) lie in the north-west quadrant, suggesting that VIPP-SD is more costly and less effective than usual care. The CEAC for this analysis (Figure 9) suggests that VIPP-SD has a higher probability of being cost-effective than usual care for WTP thresholds > £800 for a 1-point improvement in PPACS score on a scale of 0–70 points (equivalent to 0.10 SD based on usual-care group SD post treatment).

FIGURE 8.

Bootstrapped mean differences in costs and PPACS scores at the 24-month follow-up: complete cases and excluding influential outliers. NE, north-east (more costly, more effective); NW, north-west (more costly, less effective); SE, south-east (less costly, more effective); SW, south-west (less costly, less effective).

FIGURE 9.

Cost-effectiveness acceptability curve showing the probability that VIPP-SD is cost-effective compared with usual care at different values of WTP thresholds for a 1-point improvement in PPACS score at the 24-month follow-up: complete cases and excluding influential outliers. The dashed line represents the point at which a probability of 0.5 (50%) is reached.

The cost-effectiveness results at the 5-month follow-up were also similar to those in the base-case analysis. The scatterplot of bootstrapped mean differences and PPACS scores for this analysis (Figure 10) shows that almost 100% of the scatter points lie in the north-east quadrant, suggesting that VIPP-SD is more costly and more effective than usual care. The CEAC for this analysis (Figure 11) suggests that VIPP-SD has a higher probability of being cost-effective than usual care for WTP thresholds > £625 for a 1-point improvement in PPACS score on a scale of 0–70 points (equivalent to 0.10 SD based on usual-care group SD at post treatment).

FIGURE 10.

Bootstrapped mean differences in costs and PPACS scores at the 5-month follow-up. NE, north-east (more costly, more effective); NW, north-west (more costly, less effective); SE, south-east (less costly, more effective); SW, south-west (less costly, less effective).

FIGURE 11.

Cost-effectiveness acceptability curve showing the probability that VIPP-SD is cost-effective compared with usual care at different values of WTP thresholds for a 1-point improvement in PPACS score at the 5-month follow-up. The dashed line represents the point at which a probability of 0.5 (50%) is reached.

Discussion

Summary of findings

We found clear evidence that a brief, home-based intervention, VIPP-SD, was more effective than usual care in reducing behaviour problems in this group of children aged 1 and 2 years. The effect was strongest for children’s conduct problems, with evidence suggesting that most of the effect was sustained at a 24-month follow-up. This intervention uses video feedback from predominantly play-based sessions with a caregiver and their child, where the therapist shares feedback to promote sensitive responding and consistent discipline strategies with parents/caregivers of young children.

For the primary outcome (the interview-based PPACS assessment) at the 5-month post-treatment assessment, evidence of superiority was found for VIPP-SD. The secondary analysis, which tested the benefits of the intervention for those participants who received the core number of sessions (i.e. four sessions or more), showed a larger reduction in behaviour problems. This evidence, showing that increased exposure to the intervention further intensified its benefits, endorses the positive effect of the VIPP-SD programme. The VIPP-SD treatment effect was also strongest on the conduct problems scale of the PPACS assessment rather than the ADHD/hyperkinesis scale. This is particularly of note as social learning theory and the sensitive discipline component of the VIPP-SD programme specifically target conduct problems. Thus, these differential effects are in keeping with expectations and it is instructive to identify that the effects on conduct problems do not appear to extend to ADHD symptoms. Furthermore, evidence, including new evidence since this trial was designed, strongly suggests that it is conduct problems in particular that predict worse long-term outcomes for children. 94 Relatedly, early childhood conduct problems in particular, rather than hyperkinetic/ADHD problems only, are associated with greater health-care and criminal justice costs in adulthood. 9 Findings on the two secondary measures of child problems (CBCL and SDQ questionnaires) were consistent in suggesting an early benefit of VIPP-SD for children’s behaviour problems.

The findings at the longer-term follow-up (24 months post randomisation) indicated a sustained effect on children’s behaviour problems. The main outcome measure (PPACS) showed a small diminution of the treatment effect (approximately 15%) over the time between the 5- and 24-month follow-ups. This meant that the 95% CI did overlap with zero (with a p-value of 0.08), but the consistency of effect across the main outcome measures suggests that an ongoing treatment benefit is likely. The standardised effect size for the treatment difference on the total PPACS measure was 0.20 at 5 months post randomisation and 0.17 at 24 months post randomisation. Although both effect sizes are small, they are of the magnitude that may be expected to make a real difference for a brief early intervention rolled out across a large population. Similarly, the effect on the conduct disorder scale at 24 months, although somewhat reduced (0.30 standardised effect size at 5 months post randomisation and 0.20 at 24 months post randomisation), is likely to represent a meaningful effect of the intervention.

The main findings are consistent with previous, smaller studies of VIPP-SD. A recent meta-analysis32 demonstrated a similar effect size to that seen in the present study for children’s behaviour problems (0.26) and larger effect sizes for parental sensitivity (0.47). Previous VIPP-SD research also indicates the intriguing possibility that certain parents and children may be especially sensitive to the benefits of the intervention and show greater improvements in outcomes – this is known as differential susceptibility. 95–97 Data on which to base our original power calculation were limited, and in the original protocol we had anticipated a larger possible effect size than that seen in the present study. Some caution is therefore warranted in interpreting these findings. The effect size found for the VIPP-SD intervention in this study (0.20) is somewhat difficult to interpret for an individual child or family; however, it represents a mean difference of 2 points on the PPACS measure. As an example, for tantrums, this 2-point difference would equate to a change from severe (breaking things) to mild (shouting) or a change in frequency from daily to once or twice per week. Although tantrums are a normative feature of early childhood, daily tantrums are not normative and occur in < 10% of preschool-aged children. 98

As the first pragmatic trial of VIPP-SD conducted in a routine NHS health-care setting, it is striking that the effectiveness of the intervention is robust in this setting. The intervention is also highly acceptable for a psychological treatment as indicated by the high levels of adherence. There was also positive feedback from families, although this was limited to a small proportion of participating families. A substantial body of evidence for the effectiveness of several interventions already exists for childhood behaviour problems; however, this is the first study, to our knowledge, that demonstrates a beneficial treatment effect in such young children and in a pragmatic trial in routine practice. This is important as there are many examples of promising interventions for child mental health problems that do not show a clear benefit over routine care in NHS practice settings, despite some initial evidence of effectiveness in other, sometimes more tightly controlled, trial conditions. 99–101 Consequently, the findings of this study can be more reliably extrapolated to routine clinical care delivered by NHS staff, including health visitors and community nursery nurses.

The findings of this study are also able to partly address a potential concern about brief early interventions, such as VIPP-SD, for childhood behaviour problems, namely whether or not brief interventions are sufficient to be effective for children with more severe behaviour problems. The findings from this study suggest that the effect of the intervention was greater in children with more severe behaviour problems. Children with more severe behaviour problems at the beginning of the study tended to show more improvement in their behaviour, which inspired for confidence that this predominantly health visitor-led intervention can benefit families and children who may need it most. The findings suggest that interventions might be most beneficially targeted at those children with higher levels of behaviour problems. Similarly, the findings show that the intervention is effective even in children aged 1 year. The findings also give some confidence that those who may benefit from treatment can be identified using a simple screening questionnaire (SDQ). 42 The high levels of adherence and fidelity of treatment delivery seen in this pragmatic trial also suggest that this intervention can be reliably and successfully delivered in routine NHS practice.

There was no benefit of the intervention on other secondary outcome measures of parental mood, anxiety or relationship adjustment. Parent-mediated interventions often anticipate effects on parental well-being; however, this evidence suggests that these programmes are unlikely to be sufficiently powerful in and of themselves to confer benefits on global assessments of parental distress. 102

Economic evaluation

The primary economic analysis suggests that VIPP-SD is associated with significantly higher costs than usual care as a result of the additional cost of the VIPP-SD intervention and greater improvements in behaviour. When these costs are combined with the differences in PPACS scores that favour the intervention over usual care, the probability of VIPP-SD being cost-effective compared with usual care increases as the WTP for improvements in PPACS scores increases, with VIPP-SD having a higher probability of being cost-effective at WTP values of approximately £800 and above per 1-point improvement in PPACS score (equivalent to 0.10 SD). In theory, this would equate to a cost of approximately £7920 for 1 SD improvement. As the PPACS is not associated with a WTP threshold to support decision-making, it is not possible to come to any firm conclusions about the relative cost-effectiveness of VIPP-SD in the short term. Any judgement of costs needs to be weighed against the lifetime costs of behaviour problems, estimates for which range from £90,000 per case for those with subthreshold behaviour problems to £280,000 for those with early conduct disorder. 10 Indeed, the long-term costs for conduct disorder in particular are so high that early interventions that generate even modest improvements could provide a positive return on investment in the long term. 10

The cost of the VIPP-SD intervention was the key cost driver in the cost-effectiveness analyses. It is difficult to contextualise the cost of the intervention (mean cost £1466) because there is a lack of publicly available information on the costs of early intervention programmes,103 differences in costing methods and few cost-effectiveness evaluations of similar early intervention programmes. However, the costs identified in the current study are lower than the unit costs (£1612–2418) reported from a group-based parenting intervention (Incredible Years) for preschool aged children delivered in the UK. 104 That study reported that a 1-point improvement on the SDQ would cost £1295 on a 40-point scale (where 1 point represents 0.14 of the SD of the control group at follow-up). 104 Overall, effective early interventions for preschool-aged children at risk of conduct disorder that cost in the region of £1500 are seen as relatively low cost. 10

It is also noteworthy that intervention costs commonly reduce in the longer term as therapists become more experienced, and service providers and commissioners benefit from both experience and economies of scale. In particular, the length of sessions (1.5 hours) in the current trial may reflect the fact that all of the VIPP-SD trial therapists were new to the intervention and had been trained only recently. Similarly, the length of sessions in the current trial includes the longer sessions needed to deliver VIPP to two caregivers in some cases; thus, we would expect the length of sessions to be shorter if VIPP-SD was delivered to only one caregiver.

Strengths and limitations

A strength of this study is that it is a multisite, pragmatic, RCT that recruited families in routine health-care settings in urban, suburban and rural settings in the UK. The VIPP-SD intervention was acceptable to participants in this setting, with very high levels of retention (95%) achieved and very low levels of missing data. The intervention was largely delivered as intended, with the majority of assessments indicating that therapy met the necessary threshold for adherence to the manual and that the vast majority of participants received the full dosage of the programme. The intervention was delivered by NHS staff in a pragmatic context, and this is the first time, to our knowledge, that VIPP-SD has been investigated in this context. This is also the largest study of VIPP-SD to date, the first cost-effectiveness study of VIPP-SD to be attempted and one of the earliest intervention studies of children demonstrating risk for developing enduring mental health problems in the UK. The study also benefited from the use of a structured interview to assess the primary outcome, which is considered the gold standard outcome measure for mental health problems. 46 Efforts to maintain blinding were largely effective and, in the very small number of cases where research assessors were unblinded, a second researcher rescored the assessment to minimise any risk of bias.

The study had limitations. First, as with many RCTs, the participants in the final sample had a higher level of education than the general population, such that 64% of participants had a graduate-level qualification, compared with ≈ 40% of people aged 25–34 years in England (based on the 2011 Census). 105 This is notable as low parental education is a risk factor for persistent behaviour problems in young children. 10 As a large proportion of families were recruited from London (where educational attainment tends to be higher than national levels), caution should be exercised when considering the generalisability of the study findings to the rest of the UK. In most other respects, the sample was generally representative of the broad range of communities that participated in the study.

Second, approximately half of those eligible for the trial proceeded to take part. Although this is similar to the rate of progression seen in many RCTs of psychological intervention96,97 and may relate to taking part in a clinical trial rather than to taking part in routine intervention, it does suggest that not all families will engage in this intervention in routine practice, and so alternative forms of help and provision are will also be necessary.

Third, we found that not all therapists were able to deliver the intervention, largely because of service changes that meant that the time planned for delivery of the therapy became restricted. This was evident in the variation in the number of VIPP-SD cases delivered by individual therapists (range 1–12 cases, median three cases). Thus, it is possible that the treatment effect observed here is an underestimation of the effect that may be seen over a longer implementation period as therapists become more experienced.

A limitation of the health economic analysis is the lack of valid methods for measuring health-related quality of life, and thus QALYs, in young children. As a result, we used a published algorithm to map health state utilities from clinical outcomes using the SDQ, as mapping is a widely used approach in economic evaluation where direct measurement of health utilities is not possible and mapping is recommended by NICE and ISPOR. 65,86 However, the validity of the methods used is debated in the literature,87–89 loss of information at each stage of transformation leads to uncertain estimates and thus potentially erroneous conclusions, and proxy reporting by parents is also associated with a number of limitations. 90,91 The mapping algorithm applied in the current study was identified in a recent systematic review as the only algorithm available that converts any of the outcomes included in the current study to child health utilities. 92 However, the algorithm was developed in Australia with data collected from adolescents and, therefore, its relevance to much younger children in the UK is uncertain. The short-term nature of the within-trial analysis is also a limitation as participants are less likely to incur costs for resource use relating to behaviour problems during this preschool period. Service use, and thus further costs, are more likely in later years.

It is worth noting that there is a lack of evidence for effective preventative interventions for behaviour problems in very young children and a lack of services delivering interventions in the UK. The findings of this study do, therefore, offer important evidence to suggest that positive outcomes can be achieved with the VIPP-SD intervention.

Conclusions

Contributions of authors

Christine O’Farrelly (https://orcid.org/0000-0002-9269-6564) (Trial Manager and Research Associate, Developmental Psychology) was responsible for study management and data collection, and contributed to data interpretation, drafting and revising the report.

Beth Barker (https://orcid.org/0000-0001-7811-3417) (Research Assistant) recruited participants; collected data; and contributed to study procedures and aspects of measurement selection and implementation, and the drafting and revising of the report.

Hilary Watt (https://orcid.org/0000-0001-9576-3753) (Statistician, Public Health) conducted the clinical data analysis and contributed to the writing and editing of the report.

Daphne Babalis (https://orcid.org/0000-0001-9654-4130) (Operations Manager, Clinical Trials) contributed to the oversight and conduct of the trial, and the writing and editing of the report.

Marian Bakermans-Kranenburg (https://orcid.org/0000-0001-7763-0711) (Professor in Clinical Child and Family Studies) contributed to the design of the study, provided expert advice on the parenting intervention and study methodology, and contributed to the writing and editing of the report.

Sarah Byford (https://orcid.org/0000-0001-7084-1495) (Professor of Health Economics) contributed to the design of the study, supervised and undertook data analysis, and contributed to the writing and editing of the report.

Poushali Ganguli (https://orcid.org/0000-0002-1764-5273) (Research Associate, Health Services and Population Research) conducted the health economics analysis, and contributed to the writing and editing of the report.

Ellen Grimås (https://orcid.org/0000-0002-5952-8622) (Research Assistant) recruited participants, contributed to study procedures and aspects of measurement selection and implementation, collected data, co-ordinated the study’s PPI, and approved the final version of the report.

Jane Iles (https://orcid.org/0000-0002-3602-9898) (Clinical Psychologist) co-ordinated the delivery of the parenting intervention; contributed to clinical training, supervision and the development of the fidelity assessment; and approved the final version of the report.

Holly Mattock (https://orcid.org/0000-0003-4237-955X) (Research Assistant) collected data, contributed to study procedures and aspects of measurement implementation, co-ordinated the study’s PPI and approved the final version of the report.

Julia McGinley (https://orcid.org/0000-0002-4468-7516) (Head of Parent Support) provided expert advice on engaging families in the study and intervention, and approved the final version of the report.

Charlotte Phillips (https://orcid.org/0000-0001-5960-1087) (Research Assistant) recruited participants, contributed to study procedures and aspects of measurement selection and implementation, collected data and approved the final version of the report.

Rachael Ryan (https://orcid.org/0000-0002-2648-8142) (Research Assistant), recruited participants, contributed to study procedures and aspects of measurement selection and implementation, collected data, co-ordinated the study’s PPI and approved the final version of the report.

Stephen Scott (https://orcid.org/0000-0003-4680-6213) (Professor of Child Health and Behaviour) contributed to the design of the study, provided expert advice on the evaluation of parenting interventions and behaviour problems, and contributed to the writing and editing of the report.

Jessica Smith (https://orcid.org/0000-0001-5334-6222) (Trial Manager) was responsible for study management and data collection, and contributed to the writing and editing of the report.

Alan Stein (https://orcid.org/0000-0001-8207-2822) (Professor of Child and Adolescent Psychiatry) contributed to the design of the study, provided expert advice on early childhood intervention and psychopathology, and contributed to the writing and editing of the report.

Eloise Stevens (https://orcid.org/0000-0001-7671-3670) (Clinical Therapist) co-ordinated the delivery of the parenting intervention, contributed to clinical supervision and the development of the fidelity assessment, and approved the final version of the report.

Marinus van IJzendoorn (https://orcid.org/0000-0003-1144-454X) (Professor of Human Development) contributed to the design of the study, provided expert advice on the parenting intervention and study methodology, and contributed to the writing and editing of the report.

Jane Warwick (https://orcid.org/0000-0002-7320-6603) (Professor in Clinical Trial Statistics) contributed to the design of the study, supervised and undertook data analysis, and contributed to the writing and editing of the report.

Paul Ramchandani (https://orcid.org/0000-0003-3646-2410) (Professor of Child and Adolescent Mental Health) was chief investigator, conceived and designed the study, was responsible for its conduct, contributed to data interpretation and the drafting and revising of the report.

Publications

Ramchandani PG, O’Farrelly C, Babalis D, Bakermans-Kranenburg MJ, Byford S, Grimas ES, et al. Preventing enduring behavioural problems in young children through early psychological intervention (Healthy Start, Happy Start): study protocol for a randomized controlled trial. Trials 2017;18:543.

Mattock HC, Ryan R, O’Farrelly C, Babalis D, Ramchandani PG. Does a video clip enhance recruitment into a parenting trial? Learnings from a study within a trial. Trials 2020;21:856.

O’Farrelly C, Watt H, Babalis D, Bakermans-Kranenburg MJ, Barker B, Byford S, et al. A brief home-based parenting intervention (VIPP-SD) to prevent enduring behaviour problems in young children: a pragmatic randomised controlled trial [published online ahead of print March 15 2021]. JAMA Pediatrics 2021.

Data-sharing statement

All of the de-identified data included in the report will be available 12 months following publication and for 5 years after the date of publication. Data will be made available to researchers who provide a methodologically sound and hypothesis-driven proposal, and have the required institutional approvals in place to achieve the aims in the approved proposal. To gain access to the data, proposals should be directed to Professor Paul Ramchandani for approval by the investigator group. Requestors will be asked to sign a data access agreement. The study protocol and consent form will also be made available.

Disclaimers

This report presents independent research funded by the National Institute for Health Research (NIHR). The views and opinions expressed by authors in this publication are those of the authors and do not necessarily reflect those of the NHS, the NIHR, NETSCC, the HTA programme or the Department of Health and Social Care. If there are verbatim quotations included in this publication the views and opinions expressed by the interviewees are those of the interviewees and do not necessarily reflect those of the authors, those of the NHS, the NIHR, NETSCC, the HTA programme or the Department of Health and Social Care.